AsciiDataTable Example

The AsciiDataTable class is meant to deal with data in the form of a table with a header and footer. It has the ability to change formating, load a file, and do simple manipulations.

Some key elements of the AsciiDataTable class:

  1. It can be created by opening a file with the right options. To preserve the options of a given file use the save_schema method and the read_schema function.
  2. It can save, or print.
  3. It has a large amount of formatting flexability

To make a new data table:

new_table=AsciiDataTable()
#or
new_table=AsciiDataTable(header=["My Header","Line2" ],column_names=["a","b","c"],data=[[1,2,3]])

To open an existing table:

#in default format (csv with no #)
existing_table=AsciiDataTable("My_Table.txt")
# for an existing format with a saved schema
schema=read_schema("My_Schema")
existing_table=AsciiDataTable("My_Table.txt",**schema)
In [1]:
# As an example
import pyMez.Code.DataHandlers.GeneralModels as GeneralModels

Importing pyMez, this should take roughly 30 seconds
Importing Code.DataHandlers.GeneralModels
It took 0.521029 s to import Code.DataHandlers.GeneralModels
Importing Code.DataHandlers.HTMLModels
It took 0.099006 s to import Code.DataHandlers.HTMLModels
Importing Code.DataHandlers.NISTModels
The module smithplot was not found,please put it on the python path
It took 0.723041 s to import Code.DataHandlers.NISTModels
Importing Code.DataHandlers.TouchstoneModels
It took 0.0 s to import Code.DataHandlers.TouchstoneModels
Importing Code.DataHandlers.XMLModels
It took 0.128008 s to import Code.DataHandlers.XMLModels
Importing Code.DataHandlers.ZipModels
It took 0.014 s to import Code.DataHandlers.ZipModels
Importing Code.InstrumentControl.Experiments
It took 3.360193 s to import Code.InstrumentControl.Experiments
Importing Code.InstrumentControl.Instruments
It took 0.001 s to import Code.InstrumentControl.Instruments
Importing Code.Utils.Names
It took 0.0 s to import Code.Utils.Names
It took 4.846277 s to import all of the active modules
In [2]:
#Lets make a table and put it into the AsciiDataTable 
new_table=GeneralModels.AsciiDataTable()
In [4]:
# The data is a list of lists with the first dimension # of rows and the second dimension # of columns
table_data=[[i+j*3 for i in range(3)]for j in range(3)]
print(table_data)

[[0, 1, 2], [3, 4, 5], [6, 7, 8]]
In [5]:
#input the data directly
new_table.data=table_data
In [6]:
# assign the column names
new_table.column_names=['x1','x2','x3']
In [7]:
# when a string related command is called it uses build string to output the string print(new_table) also works
print(new_table.build_string())

x1,x2,x3
0,1,2
3,4,5
6,7,8
In [8]:
# the attribute options stores the schema for the data
new_table.options['column_names_begin_token']='#'
In [9]:
print(new_table.build_string())

#x1,x2,x3
0,1,2
3,4,5
6,7,8
In [10]:
# changing a formating option doesn't change the data
new_table.options['column_names_begin_token']='['
new_table.options['column_names_end_token']=']'
In [11]:
print(new_table.build_string())

[x1,x2,x3]
0,1,2
3,4,5
6,7,8
In [12]:
# There is the ability to control the output format in many ways
new_table.options['data_begin_token']='*'*80+'\n'
new_table.options['data_end_token']="\n"+'*'*80
In [13]:
print(new_table.build_string())

[x1,x2,x3]
********************************************************************************
0,1,2
3,4,5
6,7,8
********************************************************************************
In [14]:
# the header is where we would store metadata
new_table.header=['My First Header Line','My Second Header Line']
In [15]:
print(new_table)

My First Header Line
My Second Header Line
[x1,x2,x3]
********************************************************************************
0,1,2
3,4,5
6,7,8
********************************************************************************
In [16]:
new_table.options['comment_begin']='#'
new_table.options['comment_end']='\n'
In [17]:
print(new_table)

#My First Header Line
#My Second Header Line

[x1,x2,x3]
********************************************************************************
0,1,2
3,4,5
6,7,8
********************************************************************************
In [18]:
# get_options returns all options printed in a nice form 
new_table.get_options()

data_delimiter = ,
column_names_delimiter = ,
specific_descriptor = Data
general_descriptor = Table
directory = None
extension = txt
comment_begin = #
comment_end = 

inline_comment_begin = None
inline_comment_end = None
block_comment_begin = None
block_comment_end = None
footer_begin_line = None
footer_end_line = None
header_begin_line = 0
header_end_line = 2
column_names_begin_line = 3
column_names_end_line = 4
data_begin_line = 4
data_end_line = None
footer_begin_token = None
footer_end_token = None
header_begin_token = None
header_end_token = None
column_names_begin_token = [
column_names_end_token = ]
data_begin_token = ********************************************************************************

data_end_token = 
********************************************************************************
metadata_delimiter = None
metadata_key_value_delimiter = None
header_line_types = None
column_types = None
column_descriptions = None
column_units = None
footer_line_types = None
header = None
column_names = None
data = None
footer = None
inline_comments = None
row_begin_token = None
row_end_token = None
row_formatter_string = None
empty_value = None
escape_character = None
data_table_element_separator = 

treat_header_as_comment = None
treat_footer_as_comment = None
metadata = None
data_list_dictionary = None
save_schema = True
open_with_schema = True
use_alternative_parser = True
validate = False
In [19]:
# when we created the file it autonamed itself. This is the place the file will save to using new_table.save()
new_table.path
Out[19]:
'Data_Table_20181113_001.txt'
In [20]:
# Each formatting option can be turned off
new_table.options['data_begin_token']=None
new_table.options['data_end_token']=None
In [21]:
print(new_table)

#My First Header Line
#My Second Header Line

[x1,x2,x3]
0,1,2
3,4,5
6,7,8
In [22]:
# We can build the table this way or using 
# AsciiDataTable(header=new_table.header,column_names=new_table.column_names)
new_table2=GeneralModels.AsciiDataTable()
new_table2.header=new_table.header
new_table2.column_names=new_table.column_names
new_table2.options=new_table.options
In [23]:
print(new_table2)

#My First Header Line
#My Second Header Line

[x1,x2,x3]
In [24]:
new_table2.data=[[2*(i+j*3) for i in range(3)]for j in range(3)]
In [25]:
print(new_table2)

#My First Header Line
#My Second Header Line

[x1,x2,x3]
0,2,4
6,8,10
12,14,16
In [26]:
# Adding the tables, if column names are the same
new_table+new_table2
Out[26]:
<pyMez.Code.DataHandlers.GeneralModels.AsciiDataTable at 0x9149080>
In [28]:
print(new_table)
print("\n"+" --End of Table 1 --"*5+"\n")
print(new_table2)

#My First Header Line
#My Second Header Line

[x1,x2,x3]
0,1,2
3,4,5
6,7,8
0,2,4
6,8,10
12,14,16

 --End of Table 1 -- --End of Table 1 -- --End of Table 1 -- --End of Table 1 -- --End of Table 1 --

#My First Header Line
#My Second Header Line

[x1,x2,x3]
0,2,4
6,8,10
12,14,16
In [29]:
print(new_table)

#My First Header Line
#My Second Header Line

[x1,x2,x3]
0,1,2
3,4,5
6,7,8
0,2,4
6,8,10
12,14,16
In [30]:
print(new_table2.build_string())

#My First Header Line
#My Second Header Line

[x1,x2,x3]
0,2,4
6,8,10
12,14,16
In [31]:
str(new_table2).count("\n")
Out[31]:
6
In [32]:
test_string=str(new_table2)
In [33]:
test_string
Out[33]:
'#My First Header Line\n#My Second Header Line\n\n[x1,x2,x3]\n0,2,4\n6,8,10\n12,14,16'
In [34]:
test_list=test_string.split('\n')
In [35]:
test_list
Out[35]:
['#My First Header Line',
 '#My Second Header Line',
 '',
 '[x1,x2,x3]',
 '0,2,4',
 '6,8,10',
 '12,14,16']
In [36]:
print(new_table)

#My First Header Line
#My Second Header Line

[x1,x2,x3]
0,1,2
3,4,5
6,7,8
0,2,4
6,8,10
12,14,16
In [37]:
new_table.options['inline_comment_begin']='(*'
new_table.options['inline_comment_end']='*)'
new_table.inline_comments=[["Hello, This  is an inline comment",0,3]]
print(new_table)

#My(*Hello, This  is an inline comment*) First Header Line
#My Second Header Line

[x1,x2,x3]
0,1,2
3,4,5
6,7,8
0,2,4
6,8,10
12,14,16
In [38]:
import random
In [39]:
data_delimiters=[",",".",":",";","-"," ","\t","::","~"," | "]
begin_end_pairs=[["<",">"],["(*","*)"],["[","]"],["/*","*/"],["(",")"],["<","/>"],["#",""]]
random_int=random.randint(0,len(data_delimiters)-1)
random_list=[random.randint(0,len(data_delimiters)-1) for i in range(20)]
print(random_list)

[6, 7, 2, 8, 6, 2, 0, 2, 4, 4, 7, 9, 8, 8, 1, 3, 9, 4, 3, 1]
In [40]:
random_delimiter_list_1=[data_delimiters[random.randint(0,len(data_delimiters)-1)] for i in range(20)]
random_delimiter_list_2=[data_delimiters[random.randint(0,len(data_delimiters)-1)] for i in range(20)]
random_pairs_1=[begin_end_pairs[random.randint(0,6)] for i in range(20)]
random_pairs_2=[begin_end_pairs[random.randint(0,6)] for i in range(20)]
In [41]:
print(begin_end_pairs)
new_table.inline_comments=[["Hello, This  is an inline comment",0,-1]]

[['<', '>'], ['(*', '*)'], ['[', ']'], ['/*', '*/'], ['(', ')'], ['<', '/>'], ['#', '']]
In [42]:
# An example of all the formats randomized
for index in range(len(random_delimiter_list_1)):
    new_table.options["data_delimiter"]=random_delimiter_list_1[index]
    new_table.options["column_names_delimiter"]=random_delimiter_list_2[index]
    new_table.options["comment_begin"]=random_pairs_1[index][0]
    new_table.options["comment_end"]=random_pairs_1[index][1]+"\n"
    new_table.options["column_names_begin_token"]=random_pairs_2[index][0]
    new_table.options["column_names_end_token"]=random_pairs_2[index][1]    
    print( "\n{0}\n".format(new_table))

(*My First Header Line*)(*Hello, This  is an inline comment*)
(*My Second Header Line*)

(x1,x2,x3)
0 1 2
3 4 5
6 7 8
0 2 4
6 8 10
12 14 16


<My First Header Line>(*Hello, This  is an inline comment*)
<My Second Header Line>

#x1 | x2 | x3
0~1~2
3~4~5
6~7~8
0~2~4
6~8~10
12~14~16


(*My First Header Line*)(*Hello, This  is an inline comment*)
(*My Second Header Line*)

#x1;x2;x3
0 1 2
3 4 5
6 7 8
0 2 4
6 8 10
12 14 16


<My First Header Line>(*Hello, This  is an inline comment*)
<My Second Header Line>

<x1-x2-x3>
0 | 1 | 2
3 | 4 | 5
6 | 7 | 8
0 | 2 | 4
6 | 8 | 10
12 | 14 | 16


[My First Header Line](*Hello, This  is an inline comment*)
[My Second Header Line]

[x1	x2	x3]
0.1.2
3.4.5
6.7.8
0.2.4
6.8.10
12.14.16


[My First Header Line](*Hello, This  is an inline comment*)
[My Second Header Line]

/*x1~x2~x3*/
0:1:2
3:4:5
6:7:8
0:2:4
6:8:10
12:14:16


<My First Header Line>(*Hello, This  is an inline comment*)
<My Second Header Line>

(*x1:x2:x3*)
0:1:2
3:4:5
6:7:8
0:2:4
6:8:10
12:14:16


/*My First Header Line*/(*Hello, This  is an inline comment*)
/*My Second Header Line*/

(*x1	x2	x3*)
0::1::2
3::4::5
6::7::8
0::2::4
6::8::10
12::14::16


<My First Header Line>(*Hello, This  is an inline comment*)
<My Second Header Line>

(*x1;x2;x3*)
0 | 1 | 2
3 | 4 | 5
6 | 7 | 8
0 | 2 | 4
6 | 8 | 10
12 | 14 | 16


#My First Header Line(*Hello, This  is an inline comment*)
#My Second Header Line

#x1-x2-x3
0:1:2
3:4:5
6:7:8
0:2:4
6:8:10
12:14:16


(My First Header Line)(*Hello, This  is an inline comment*)
(My Second Header Line)

[x1::x2::x3]
0,1,2
3,4,5
6,7,8
0,2,4
6,8,10
12,14,16


(*My First Header Line*)(*Hello, This  is an inline comment*)
(*My Second Header Line*)

<x1~x2~x3/>
0,1,2
3,4,5
6,7,8
0,2,4
6,8,10
12,14,16


(*My First Header Line*)(*Hello, This  is an inline comment*)
(*My Second Header Line*)

(*x1~x2~x3*)
0,1,2
3,4,5
6,7,8
0,2,4
6,8,10
12,14,16


<My First Header Line>(*Hello, This  is an inline comment*)
<My Second Header Line>

(x1::x2::x3)
0 1 2
3 4 5
6 7 8
0 2 4
6 8 10
12 14 16


[My First Header Line](*Hello, This  is an inline comment*)
[My Second Header Line]

(x1-x2-x3)
0 | 1 | 2
3 | 4 | 5
6 | 7 | 8
0 | 2 | 4
6 | 8 | 10
12 | 14 | 16


<My First Header Line>(*Hello, This  is an inline comment*)
<My Second Header Line>

[x1::x2::x3]
0 1 2
3 4 5
6 7 8
0 2 4
6 8 10
12 14 16


(My First Header Line)(*Hello, This  is an inline comment*)
(My Second Header Line)

<x1,x2,x3>
0 | 1 | 2
3 | 4 | 5
6 | 7 | 8
0 | 2 | 4
6 | 8 | 10
12 | 14 | 16


<My First Header Line/>(*Hello, This  is an inline comment*)
<My Second Header Line/>

[x1;x2;x3]
0 1 2
3 4 5
6 7 8
0 2 4
6 8 10
12 14 16


[My First Header Line](*Hello, This  is an inline comment*)
[My Second Header Line]

(x1;x2;x3)
0	1	2
3	4	5
6	7	8
0	2	4
6	8	10
12	14	16


<My First Header Line>(*Hello, This  is an inline comment*)
<My Second Header Line>

(*x1;x2;x3*)
0 | 1 | 2
3 | 4 | 5
6 | 7 | 8
0 | 2 | 4
6 | 8 | 10
12 | 14 | 16
In [43]:
GeneralModels.show_structure_script()

Printing the string representation of the table
--------------------------------------------------------------------------------
{header_begin_token}
self.header[0]
self.header[1]{inline_comment_begin}inline_comments[0][0]{inline_comment_end}
{comment_begin}self.header[2]{comment_end}

{block_comment_begin}
self.header[3]
self.header{4]
{block_comment_end}
{header_end_token}
{data_table_element_separator}
{column_names_begin_token}column_names[0]{column_names_delimiter}column_names[1]{column_names_delimiter}column_names[2]{column_names_end_token}
{data_table_element_separator}
{data_begin_token}
{row_begin_token}data[0][0]{data_delimiter}data[1][0]{data_delimiter}data[2][0]{row_end_token}
{row_begin_token}data[0][1]{data_delimiter}data[1][1]{data_delimiter}data[2][1]{row_end_token}
{data_end_token}
{data_table_element_separator}
{footer_begin_token}
{block_comment_begin}
self.footer[0]
self.footer[1]
{block_comment_end}
{footer_end_token}
Printing the lines representation of the table with line numbers
--------------------------------------------------------------------------------
0 {header_begin_token}
1 self.header[0]
2 self.header[1]{inline_comment_begin}inline_comments[0][0]{inline_comment_end}
3 {comment_begin}self.header[2]{comment_end}
4 
5 {block_comment_begin}
6 self.header[3]
7 self.header{4]
8 {block_comment_end}
9 {header_end_token}
10 {data_table_element_separator}
11 {column_names_begin_token}column_names[0]{column_names_delimiter}column_names[1]{column_names_delimiter}column_names[2]{column_names_end_token}
12 {data_table_element_separator}
13 {data_begin_token}
14 {row_begin_token}data[0][0]{data_delimiter}data[1][0]{data_delimiter}data[2][0]{row_end_token}
15 {row_begin_token}data[0][1]{data_delimiter}data[1][1]{data_delimiter}data[2][1]{row_end_token}
16 {data_end_token}
17 {data_table_element_separator}
18 {footer_begin_token}
19 {block_comment_begin}
20 self.footer[0]
21 self.footer[1]
22 {block_comment_end}
23 {footer_end_token}
--------------------------------------------------------------------------------
The result of self.lines[0:10] is :
{header_begin_token}
self.header[0]
self.header[1]{inline_comment_begin}inline_comments[0][0]{inline_comment_end}
{comment_begin}self.header[2]{comment_end}

{block_comment_begin}
self.header[3]
self.header{4]
{block_comment_end}
{header_end_token}
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
The result of self.lines[11:12] is :
{column_names_begin_token}column_names[0]{column_names_delimiter}column_names[1]{column_names_delimiter}column_names[2]{column_names_end_token}
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
The result of self.lines[13:17] is :
{data_begin_token}
{row_begin_token}data[0][0]{data_delimiter}data[1][0]{data_delimiter}data[2][0]{row_end_token}
{row_begin_token}data[0][1]{data_delimiter}data[1][1]{data_delimiter}data[2][1]{row_end_token}
{data_end_token}
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
The result of self.lines[18:None] is :
{footer_begin_token}
{block_comment_begin}
self.footer[0]
self.footer[1]
{block_comment_end}
{footer_end_token}
--------------------------------------------------------------------------------
In [ ]: