com.nicta.scoobi.impl.plan.comp

ParallelDo

case class ParallelDo(ins: Seq[CompNode], env: ValueNode, dofn: DoFunction, wfa: WireReaderWriter, wfb: WireReaderWriter, nodeSinks: Seq[Sink] = Seq(), bridgeStoreId: String = randomUUID.toString) extends ProcessNodeImpl with Product with Serializable

The ParallelDo node type specifies the building of a CompNode as a result of applying a function to all elements of an existing CompNode and concatenating the results

Linear Supertypes
Serializable, Serializable, ProcessNodeImpl, ProcessNode, CompNode, Attributable, Cloneable, Cloneable, Product, Equals, AnyRef, Any
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. ParallelDo
  2. Serializable
  3. Serializable
  4. ProcessNodeImpl
  5. ProcessNode
  6. CompNode
  7. Attributable
  8. Cloneable
  9. Cloneable
  10. Product
  11. Equals
  12. AnyRef
  13. Any
  1. Hide All
  2. Show all
Learn more about member selection
Visibility
  1. Public
  2. All

Instance Constructors

  1. new ParallelDo(ins: Seq[CompNode], env: ValueNode, dofn: DoFunction, wfa: WireReaderWriter, wfb: WireReaderWriter, nodeSinks: Seq[Sink] = Seq(), bridgeStoreId: String = randomUUID.toString)

Type Members

  1. type C = ProcessNode

    Definition Classes
    ProcessNode → CompNode

Value Members

  1. final def !=(arg0: Any): Boolean

    Definition Classes
    AnyRef → Any

  2. final def ##(): Int

    Definition Classes
    AnyRef → Any

  3. final def ->[T, U](a: (T) ⇒ U)(implicit b: (ParallelDo.this.type) ⇒ T): U

    Definition Classes
    Attributable
    Annotations
    @inline()

  4. final def ->[U](a: (ParallelDo.this.type) ⇒ U): U

    Definition Classes
    Attributable
    Annotations
    @inline()

  5. final def ==(arg0: Any): Boolean

    Definition Classes
    AnyRef → Any

  6. def addSink(sink: Sink): C

    Definition Classes
    CompNode

  7. final def asInstanceOf[T0]: T0

    Definition Classes
    Any

  8. lazy val bridgeStore: Bridge

    ParallelDo, Combine, GroupByKey have a Bridge = sink for previous computations + source for other computations

    ParallelDo, Combine, GroupByKey have a Bridge = sink for previous computations + source for other computations

    Definition Classes
    ProcessNodeImpl → ProcessNode

  9. val bridgeStoreId: String

    unique identifier for the bridgeStore storing data for this node

    unique identifier for the bridgeStore storing data for this node

    Definition Classes
    ParallelDo → ProcessNodeImpl → ProcessNode

  10. def bridgeToString: String

    display the bridge id

    display the bridge id

    Definition Classes
    ProcessNodeImpl

  11. def children: Iterator[Attributable]

    Definition Classes
    Attributable

  12. def cleanup(environment: Any, emitter: EmitterWriter)(implicit sc: ScoobiConfiguration): Unit

    cleanup

  13. def clone(): Attributable

    Definition Classes
    Attributable → AnyRef

  14. val dofn: DoFunction

  15. val env: ValueNode

  16. def environment(implicit sc: ScoobiConfiguration): Any

    returns

    the environment object stored within the env node

  17. final def eq(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  18. def equals(a: Any): Boolean

    Definition Classes
    CompNode → Equals → AnyRef → Any

  19. def finalize(): Unit

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )

  20. def firstChild[T]: T

    Definition Classes
    Attributable

  21. final def getClass(): Class[_]

    Definition Classes
    AnyRef → Any

  22. def hasCheckpoint: Boolean

    Definition Classes
    CompNode

  23. def hasChildren: Boolean

    Definition Classes
    Attributable

  24. def hashCode(): Int

    Definition Classes
    CompNode → AnyRef → Any

  25. val id: Int

    unique identifier for this computation node

    unique identifier for this computation node

    Definition Classes
    ProcessNodeImpl → CompNode

  26. var index: Int

    Definition Classes
    Attributable

  27. def initTreeProperties(): Unit

    Definition Classes
    Attributable

  28. val ins: Seq[CompNode]

  29. def isFirst: Boolean

    Definition Classes
    Attributable

  30. final def isInstanceOf[T0]: Boolean

    Definition Classes
    Any

  31. def isLast: Boolean

    Definition Classes
    Attributable

  32. def isRoot: Boolean

    Definition Classes
    Attributable

  33. def lastChild[T]: T

    Definition Classes
    Attributable

  34. def map(environment: Any, value: Any, emitter: EmitterWriter): Unit

    Use this ParallelDo as a Mapper

  35. final def ne(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  36. def next[T]: T

    Definition Classes
    Attributable

  37. var next: Attributable

    Definition Classes
    Attributable

  38. val nodeSinks: Seq[Sink]

    list of additional sinks for this node

    list of additional sinks for this node

    Definition Classes
    ParallelDo → ProcessNodeImpl → ProcessNode

  39. def nodeSinksString: String

    display the sinks if any

    display the sinks if any

    Definition Classes
    ProcessNodeImpl

  40. final def notify(): Unit

    Definition Classes
    AnyRef

  41. final def notifyAll(): Unit

    Definition Classes
    AnyRef

  42. def parent[T]: T

    Definition Classes
    Attributable

  43. var parent: Attributable

    Definition Classes
    Attributable

  44. def prev[T]: T

    Definition Classes
    Attributable

  45. var prev: Attributable

    Definition Classes
    Attributable

  46. def pushEnv(result: Any)(implicit sc: ScoobiConfiguration): Unit

    push a computed result to the distributed cache for the parallelDo environment

  47. def reduce(environment: Any, key: Any, values: Any, emitter: EmitterWriter)(implicit sc: ScoobiConfiguration): Unit

    reduce key and values

  48. def setup(environment: Any)(implicit sc: ScoobiConfiguration): Unit

    setup this parallel do computation

  49. lazy val sinks: Seq[Sink]

    returns

    all the additional sinks + the bridgeStore

    Definition Classes
    ProcessNodeImpl → CompNode

  50. lazy val source: Option[Load]

  51. final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes
    AnyRef

  52. val toString: String

    Definition Classes
    ParallelDo → AnyRef → Any

  53. def updateSinks(f: (Seq[Sink]) ⇒ Seq[Sink]): ParallelDo

    Definition Classes
    ParallelDo → CompNode

  54. final def wait(): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  55. final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  56. final def wait(arg0: Long): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  57. def wf: WireReaderWriter

    object defining how to serialise / deserialise data for that node

    object defining how to serialise / deserialise data for that node

    Definition Classes
    ParallelDo → CompNode

  58. val wfa: WireReaderWriter

  59. val wfb: WireReaderWriter

  60. def wfe: WireReaderWriter

Inherited from Serializable

Inherited from Serializable

Inherited from ProcessNodeImpl

Inherited from ProcessNode

Inherited from CompNode

Inherited from Attributable

Inherited from Cloneable

Inherited from Cloneable

Inherited from Product

Inherited from Equals

Inherited from AnyRef

Inherited from Any

Ungrouped