mscr

Type Members

1. case class BypassOutputChannel(input: ParallelDo, nodes: Layering = ...) extends MscrOutputChannel with Product with Serializable

   This output channel simply copy values coming from a ParallelDo input (a mapper in an Input channel) to this node sinks and bridgeStore

2. trait Channel extends AnyRef

   Abstract trait for both input and output channels

3. class FloatingInputChannel extends MscrInputChannel

   This input channel is a tree of Mappers which are not connected to Gbk nodes

4. class GbkInputChannel extends MscrInputChannel

   This input channel is a tree of Mappers which are all connected to Gbk nodes

5. case class GbkOutputChannel(groupByKey: GroupByKey, combiner: Option[Combine] = scala.None, reducer: Option[ParallelDo] = scala.None, nodes: Layering = ...) extends MscrOutputChannel with Product with Serializable

   Output channel for a GroupByKey.

6. trait InputChannel extends Channel

   An input channel groups mapping operations from a single DataSource, attached to a source node (a Load node, or a GroupByKey node from a previous Mscr for example).

7. case class InputChannels(channels: Seq[InputChannel]) extends Product with Serializable

8. case class KeyTypes(types: Map[Int, (WireReaderWriter, KeyGrouping)] = ...) extends Product with Serializable

   encapsulation of expected key types for each tag

9. trait Layering extends ShowNode

   Simple layering algorithm using the Longest path method to assign nodes to layers.

10. case class Mscr extends Attributable with Product with Serializable

    This class represents an MSCR job with a Seq of input channels and a Seq of output channels

11. trait MscrInputChannel extends InputChannel

    Common implementation of InputChannel for GbkInputChannel and FloatingInputChannel

12. trait MscrOutputChannel extends OutputChannel

    Implementation of an OutputChannel for a Mscr

13. trait MscrsDefinition extends Layering with Optimiser

    This trait processes the computation graph created out of DLists and creates map-reduce jobs from it.

14. trait OutputChannel extends Channel

    An OutputChannel is responsible for emitting key/values grouped by one Gbk or passed through from an InputChannel with no grouping

15. case class OutputChannels(channels: Seq[OutputChannel]) extends Product with Serializable

16. case class ValueTypes(types: Map[Int, (WireReaderWriter)] = ...) extends Product with Serializable

    encapsulation of expected value types for each tag