com.nicta.scoobi.impl.plan.mscr
A layer contains group by keys and floating nodes defining mscrs so that none of them have dependencies relationship
collect all the sinks in the computation graph
collect all the sinks in the computation graph
compute the children of a node.
compute the children of a node.
This is similar to calling the initTree method on the node but this stores the information as an attribute instead of storing it as a tree. This is a way to avoid conflicts if we have 2 processes trying to initialise the same graph
compute all the descendents of a node They are all the recursive children reachable from this node
compute all the descendents of a node They are all the recursive children reachable from this node
this attribute stores the fact that a Sink has received data
this attribute stores the fact that a Sink has received data
collect all input nodes to the gbks of this layer
Mscrs for mscrs built around gbk nodes
\ * @return a gbk output channel based on the nodes which are following the gbk
\ * @return a gbk output channel based on the nodes which are following the gbk
create a gbk output channel for each gbk in the layer
true if a given Sink has already received data
initialise the parent/child relationship recursively from node s
initialise the parent/child relationship recursively from node s
compute the inputs of a given node For a ParallelDo node this does not consider its environment
compute the inputs of a given node For a ParallelDo node this does not consider its environment
return true if a CompNode is a Combine
return true if a CompNode is a Combine
return true if a CompNode is a GroupByKey
return true if a CompNode is a GroupByKey
return true if a CompNode is a Load
return true if a CompNode is a Load
return the node if a CompNode is a Materialise
return the node if a CompNode is a Materialise
return true if a CompNode is a ParallelDo
return true if a CompNode is a ParallelDo
return true if a CompNode is a ProcessNode
return true if a CompNode is a ProcessNode
return true if a CompNode is a Combine
return true if a CompNode is a Combine
return true if a CompNode needs to be computed
return true if a CompNode needs to be computed
return true if a CompNode has a cycle in its graph, this will be detected by Kiama throwing an exception when fetching the descendents of a node
return true if a CompNode has a cycle in its graph, this will be detected by Kiama throwing an exception when fetching the descendents of a node
node at the end of the graph
return true if a CompNode is a GroupByKey
return true if a CompNode is a GroupByKey
return true if a CompNode is a Load
return true if a CompNode is a Load
return true if a CompNode is a Materialise
return true if a CompNode is a Materialise
return true if a CompNode is an Op
return true if a CompNode is an Op
return true if a CompNode is a ParallelDo
return true if a CompNode is a ParallelDo
a node is said to be reducing if it is in a "reducing chain of nodes" after a gbk
a node is said to be reducing if it is in a "reducing chain of nodes" after a gbk
return true if a CompNode is a Return
return true if a CompNode is a Return
return true if a CompNode is a Root
return true if a CompNode is a Root
return true if a CompNode needs to be persisted
return true if a CompNode needs to be persisted
true if a node is used at most once
true if a node is used at most once
return true if a CompNode needs to be loaded
return true if a CompNode needs to be loaded
the layer that a selected node is in. None if this is not a selected node
mark a sink as filled so it doesn't have to be recomputed
mark a sink as filled so it doesn't have to be recomputed
all the mscrs for a given layer
true if a process node has been filled
compute the parent of a node.
compute the parent of a node. This relationship is actually maintained while getting the children of a node if the parent node has not been set while recursing for children, then it is None
compute the parents of a node, that is all the chain of parents from this node up to the root of the graph
compute the parents of a node, that is all the chain of parents from this node up to the root of the graph
Mscrs for parallel do nodes which are not part of a Gbk mscr
a nested text representation of the nodes graph
a nested text representation of the nodes graph + graph if it's not too big
reinitialise all the attributes related to a node, starting from all the parent/children relationships
reinitialise all the attributes related to a node, starting from all the parent/children relationships
reset the attributes, then recreate the parent/children relationships recursively
reinit usages
reinit usages
the root of the graph, computed from a given node
the root of the graph, computed from a given node
The nodes which are selected to define a layer are the nodes which delimitate outputs.
The nodes which are selected to define a layer are the nodes which delimitate outputs.
What we want is to layer the computation graph so that output dependencies are respected:
- if output1 depends on output2 then output2 must be computed first
Show instance for a CompNode
Show instance for a CompNode
Show is not covariant so it is necessary to add this implicit to prove that we can show subclasses of CompNode
Show is not covariant so it is necessary to add this implicit to prove that we can show subclasses of CompNode
an ASCII representation of the nodes graph
show the structure without the ids or type annotations
show the structure without the ids or type annotations
compute all the indirect uses of a given node, that is all the nodes which have this node as a descendent
compute all the indirect uses of a given node, that is all the nodes which have this node as a descendent
the nodes which have this node as a direct child
the nodes which have this node as a direct child
for efficiency, this uses a table of all the nodes using a given one, computed from the root
compute all the nodes which use a given node as an environment
compute all the nodes which use a given node as an environment