Most of the time, when working in Pipeline Pilot, the data we work with is flat. For example, when reading data from an Excel or a csv file, we end up with a set of data records, each containing the properties from one row in the input file as a set of key/value pairs attached to a root node. This data can then be easily visualized in a tabular format using one of the viewers, such as the HTML Table Viewer. An example of a flat, one-level data record is shown below, with the green ball at the top representing the root node, and the purple diamond shapes representing data properties attached to that node. The output shown below is from the Data Record Tree Viewer component, which is a good tool for visualizing hierarchical data when working in the Pipeline Pilot client.
However, there are cases when the data is more complex than that, and a simple one-level flat data model is not sufficient to hold all the information in the input. In these cases, Pipeline Pilot uses a hierarchy, with a root node at the top with properties associated with it just like in the flat example described above, and additional child nodes attached to that root node in a multi-level data model. This representation is critical when working with more complex data, and Pipeline Pilot provides multiple ways in creating, manipulating and visualizing this data. An example of hierarchical data record is shown below (as above, green balls represent nodes, purple diamonds are properties):
Creating hierarchies is easy. Many built-in components (sequence analysis and imaging components are just two examples) use deep hierarchies to hold the data. Hierarchies are also often created when converting JSON or XML strings into Pipeline Pilot data records (provided the JSON or XML string represents a multi-level hierarchy, of course). This is particularly useful when executing Pipeline Pilot protocols from a web page. Finally, hierarchies can be created manually using PilotScript or the special components available in Pipeline Pilot.
Manipulating hierarchies is also easy using the specialized PilotScript functions or the provided components. Nodes can be detached from their parent and output as their own data record, moved elsewhere in the hierarchy or deleted. They can also be detached, manipulated in some way, and then easily reattached in the same place in the parent hierarchy. Multiple data records with different hierarchies can be merged together. Hierarchies can be flattened into a one-level data record to help with visualization. And finally, hierarchies can be converted to JSON or XML to be used elsewhere.
All this is possible using the tools that Pipeline Pilot provides for working with hierarchies, and you shouldn't be afraid to try it.
Components that are available for manipulating hierarchies are shown below:
PilotScript functions for manipulation hierarchies:
There are lots of example protocols that cover many different use cases under Protocols/Examples/Generic/Data Tree Manipulators:
