core

Type Members

1. sealed trait Association1[+K, +V] extends AnyRef

   An association of a key to 1 or many values.

2. case class Checkpoint(path: Path, expiryPolicy: ExpiryPolicy = ExpiryPolicy.default) extends Product with Serializable

   store the output path of a Sink as a checkpoint

3. trait CompNode extends Attributable

   Base trait for "computation nodes" with no generic type information for easier rewriting

4. case class Compression(codec: CompressionCodec, compressionType: CompressionType = ...) extends Product with Serializable

   store the compression parameters for sinks

5. trait Counters extends AnyRef

6. trait DList[A] extends DataSinks with Persistent[Seq[A]]

   A list that is distributed across multiple machines.

7. trait DObject[A] extends Persistent[A]

   A wrapper around an object that is part of the graph of a distributed computation

8. trait DataSink[K, V, B] extends Sink

   An output store from a MapReduce job

9. trait DataSinks extends AnyRef

   specify an object on which it is possible to add sinks and to compress them

10. trait DataSource[K, V, A] extends Source

    DataSource for a computation graph.

11. trait DelegatedScoobiJobContext extends ScoobiJobContext

12. trait DoFn[A, B] extends EnvDoFn[A, B, Unit]

    Interface for specifying parallel operation over DLists in the absence of an environment

13. trait Emitter[A] extends EmitterWriter

    Interface for writing outputs from a DoFn

14. trait EnvDoFn[A, B, E] extends DoFunction

    Interface for specifying parallel operation over DLists.

15. trait Environment extends AnyRef

    An object holder which can hold a distributed value

16. case class ExpiryPolicy(expiryTime: FiniteDuration = ..., archive: (Path, ScoobiConfiguration) ⇒ Unit = ExpiryPolicy.deleteOldFile) extends Product with Serializable

    Define the expiry policy for checkpoint files

17. sealed trait Grouped[K, V] extends AnyRef

    A distributed list of associations.

18. trait Grouping[K] extends KeyGrouping

    Specify the way in which key-values are "shuffled".

19. trait GroupingFunctions extends AnyRef

20. trait GroupingImplicits extends GroupingImplicits0

    Implicit definitions of Grouping instances for common types.

21. trait GroupingImplicits0 extends AnyRef

22. trait Heartbeat extends AnyRef

23. trait InputConverter[K, V, A] extends FromKeyValueConverter

    Convert an InputFormat's key-value types to the type produced by a source

24. class InputOutputContext extends AnyRef

25. trait InputOutputContextScoobiJobContext extends ScoobiJobContext

26. trait InputOutputConverter[K, V, B] extends OutputConverter[K, V, B] with InputConverter[K, V, B]

    fusion of both trait when bi-directional conversion is possible

27. trait Iterable1[+A] extends AnyRef

    A non-empty iterable contains at least one element.

28. trait Iterator1[+A] extends TraversableOnce[A]

    A non-empty iterator contains at least one element.

29. sealed trait Mode extends AnyRef

30. trait NoCounters extends Counters

31. trait NoHeartbeat extends Heartbeat

32. trait NoScoobiJobContext extends NoCounters with NoHeartbeat

33. trait OutputConverter[K, V, B] extends ToKeyValueConverter

    Convert the type consumed by a DataSink into an OutputFormat's key-value types.

34. trait Persistent[T] extends DataSinks

35. trait ProcessNode extends CompNode

36. trait Reduction[A] extends AnyRef

37. trait Reductions extends AnyRef

38. trait ScoobiConfiguration extends AnyRef

    This class wraps the Hadoop (mutable) configuration with additional configuration information such as the jars which should be added to the classpath.

39. trait ScoobiJobContext extends Counters with Heartbeat

40. trait SinkSource extends Sink

    This is a Sink which can also be used as a Source

41. trait UniqueInt extends AnyRef

    Trait that is sub-classed by objects to provide sets of unique identifiers.

42. trait ValueNode extends CompNode

43. trait WireFormat[A] extends WireReaderWriter

    Typeclass for sending types across the Hadoop wire

44. trait WireFormatImplicits extends GeneratedWireFormats

    Implicit definitions of WireFormat instances for common types.

45. trait WireReaderWriter extends AnyRef