Graph Transformation Example

Data in a table form can be transformed into numerous different formats using the class pyMez.Code.DataHandlers.GraphModels.TableGraph. The abstract data model is a table with column names.

Import needed modules

In [1]:
from pyMez.Code.DataHandlers.GraphModels import *
Importing pyMez, this should take roughly 30 seconds
Importing Code.DataHandlers.GeneralModels
It took 1.013 s to import Code.DataHandlers.GeneralModels
Importing Code.DataHandlers.HTMLModels
It took 0.161 s to import Code.DataHandlers.HTMLModels
Importing Code.DataHandlers.NISTModels
It took 7.008 s to import Code.DataHandlers.NISTModels
Importing Code.DataHandlers.TouchstoneModels
It took 0.009 s to import Code.DataHandlers.TouchstoneModels
Importing Code.DataHandlers.XMLModels
It took 0.255 s to import Code.DataHandlers.XMLModels
Importing Code.DataHandlers.ZipModels
It took 0.039 s to import Code.DataHandlers.ZipModels
Importing Code.InstrumentControl.Experiments
It took 1.284 s to import Code.InstrumentControl.Experiments
Importing Code.InstrumentControl.Instruments
It took 0.014 s to import Code.InstrumentControl.Instruments
Importing Code.Utils.Names
It took 0.01 s to import Code.Utils.Names
It took 9.794 s to import all of the active modules
In [2]:
# create the graph
graph=TableGraph()
In [3]:
# now we have access to the folowing data formats
graph.node_names
Out[3]:
['DataFrame',
 'AsciiDataTable',
 'HdfFile',
 'XmlDataTable',
 'ExcelFile',
 'OdsFile',
 'HtmlString',
 'JsonFile',
 'JsonString',
 'CsvFile',
 'MatFile',
 'XmlFile',
 'HtmlFile']
In [4]:
# to visualize the graph use the show method
graph.show()
Out[4]:
<matplotlib.figure.Figure at 0xf40f710>
In [5]:
# if we want to open an ascii data table or subclass we open the schema and then open the file
schema=read_schema(os.path.join(os.getcwd(),"./TableGraph_Files/Line_4909_WR15_Wave_Parameters_20180313_003.schema"))
ascii_table=AsciiDataTable(os.path.join(os.getcwd(),"./TableGraph_Files/Line_4909_WR15_Wave_Parameters_20180313_003.w2p"),
                           **schema)
In [6]:
graph.set_state(node_name="AsciiDataTable",node_data=ascii_table)
In [7]:
# now if you want a pandas data frame just move to that node
graph.move_to_node("DataFrame")
In [8]:
#now dataframe is just
pandas_dataframe=graph.data
In [9]:
pandas_dataframe[0:10]
Out[9]:
Frequency reA1_D1 imA1_D1 reB1_D1 imB1_D1 reA2_D1 imA2_D1 reB2_D1 imB2_D1 reA1_D2 imA1_D2 reB1_D2 imB1_D2 reA2_D2 imA2_D2 reB2_D2 imB2_D2
0 50.0 -0.067716 -0.018169 -0.002698 0.000736 0.002505 0.006582 0.004642 -0.024663 -0.002776 0.000737 -0.009040 -0.013720 -0.030438 0.039048 -0.000666 -0.001240
1 50.1 0.022439 0.066658 -0.002310 -0.001366 -0.007025 0.003140 0.003018 0.025235 0.002766 0.000807 -0.012986 0.009946 -0.006615 0.049959 0.000561 0.000726
2 50.2 0.045270 -0.052561 -0.001734 -0.001928 -0.002514 -0.007838 -0.007694 -0.024228 -0.000439 0.002662 0.002923 -0.015854 0.027449 -0.042675 0.000953 0.000268
3 50.3 -0.068064 -0.006680 -0.000584 -0.002533 0.008351 -0.001583 0.013007 0.021694 -0.002353 -0.000478 0.006114 0.014497 -0.046726 0.018792 0.000219 0.001005
4 50.4 0.019932 0.065229 0.001364 -0.002947 -0.000787 0.008783 -0.014357 -0.021132 0.001078 -0.001850 -0.010481 -0.011547 0.046113 0.021435 0.000240 0.000686
5 50.5 0.061203 -0.029331 0.002601 0.000242 -0.007656 -0.004735 0.011427 0.023315 0.000980 0.001591 0.013222 0.008356 -0.015426 -0.049494 -0.001176 0.000562
6 50.6 -0.045496 -0.050363 0.002401 0.001195 0.007171 -0.005577 -0.011494 -0.023691 -0.001749 -0.000151 -0.014867 -0.004185 -0.025840 0.044895 -0.000025 -0.000391
7 50.7 -0.036221 0.058172 -0.000036 0.003033 0.002033 0.008626 0.009857 0.024808 0.001344 -0.001364 0.015103 0.002441 0.051798 0.000304 -0.000797 -0.000757
8 50.8 0.063027 0.029029 -0.001881 0.001356 -0.007953 -0.003059 -0.003294 -0.027181 0.000194 0.002238 -0.015260 -0.001379 -0.022791 -0.047313 0.001282 -0.000510
9 50.9 0.020003 -0.066620 -0.002146 -0.000444 0.006057 -0.004937 -0.001895 0.027502 -0.002241 -0.001488 0.015140 -0.000094 -0.029189 0.043710 0.000713 0.000487
In [10]:
# if we want to transform to json string
graph.move_to_node("JsonString")
In [11]:
json=graph.data
In [12]:
# if we would like to transform it to all possible data types -the xslx and ods require a confirmation
# first change the working directory and now all files will be put in the folder
os.chdir("./TableGraph_Files")
visit_all_nodes(graph)
C:\ProgramData\Anaconda2\lib\site-packages\pandas\io\pytables.py:270: PerformanceWarning: 
your performance may suffer as PyTables will pickle object types that it cannot
map directly to c-types [inferred_type->unicode,key->axis0] [items->None]

  f(store)
C:\ProgramData\Anaconda2\lib\site-packages\pandas\io\pytables.py:270: PerformanceWarning: 
your performance may suffer as PyTables will pickle object types that it cannot
map directly to c-types [inferred_type->unicode,key->block0_items] [items->None]

  f(store)