com.nicta.scoobi.core

SinkSource

trait SinkSource extends Sink

This is a Sink which can also be used as a Source

Linear Supertypes
Sink, AnyRef, Any
Known Subclasses
AvroSink, SeqSink
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. SinkSource
  2. Sink
  3. AnyRef
  4. Any
  1. Hide All
  2. Show all
Learn more about member selection
Visibility
  1. Public
  2. All

Abstract Value Members

  1. abstract def checkpoint: Option[Checkpoint]

    returns

    the checkpoint parameters if this sink is a Checkpoint

  2. abstract def compressWith(codec: CompressionCodec, compressionType: CompressionType = CompressionType.BLOCK): Sink

    returns

    a new sink with compression enabled

    Definition Classes
    Sink

  3. abstract def configureCompression(configuration: Configuration): Sink

    configure the compression for a given job

    configure the compression for a given job

    Definition Classes
    Sink

  4. abstract def id: Int

    unique id for this Sink

    unique id for this Sink

    Definition Classes
    Sink

  5. abstract def outputCheck(implicit sc: ScoobiConfiguration): Unit

    Check the validity of the DataSink specification.

    Check the validity of the DataSink specification.

    Definition Classes
    Sink

  6. abstract def outputConfigure(job: Job)(implicit sc: ScoobiConfiguration): Unit

    Configure the DataSink.

    Configure the DataSink.

    Definition Classes
    Sink

  7. abstract def outputConverter: OutputConverter[_, _, _]

    Maps the type consumed by this DataSink to the key-values of its OutputFormat.

    Maps the type consumed by this DataSink to the key-values of its OutputFormat.

    Definition Classes
    Sink

  8. abstract def outputFormat(implicit sc: ScoobiConfiguration): Class[_ <: OutputFormat[_, _]]

    The OutputFormat specifying the type of output for this DataSink.

    The OutputFormat specifying the type of output for this DataSink.

    Definition Classes
    Sink

  9. abstract def outputKeyClass(implicit sc: ScoobiConfiguration): Class[_]

    The Class of the OutputFormat's key.

    The Class of the OutputFormat's key.

    Definition Classes
    Sink

  10. abstract def outputPath(implicit sc: ScoobiConfiguration): Option[Path]

    returns

    the path for this Sink.

    Definition Classes
    Sink

  11. abstract def outputTeardown(implicit sc: ScoobiConfiguration): Unit

    This method is called just after writing data to the sink

    This method is called just after writing data to the sink

    Definition Classes
    Sink

  12. abstract def outputValueClass(implicit sc: ScoobiConfiguration): Class[_]

    The Class of the OutputFormat's value.

    The Class of the OutputFormat's value.

    Definition Classes
    Sink

  13. abstract def stringId: String

    unique id for this Sink, as a string.

    unique id for this Sink, as a string. Can be used to create a file path

    Definition Classes
    Sink

  14. abstract def toSource: Source

Concrete Value Members

  1. final def !=(arg0: Any): Boolean

    Definition Classes
    AnyRef → Any

  2. final def ##(): Int

    Definition Classes
    AnyRef → Any

  3. final def ==(arg0: Any): Boolean

    Definition Classes
    AnyRef → Any

  4. final def asInstanceOf[T0]: T0

    Definition Classes
    Any

  5. def checkpointExists(implicit sc: ScoobiConfiguration): Boolean

    returns

    true if this Sink is a checkpoint and has been filled with data

  6. def checkpointPath: Option[String]

    returns

    the path of the checkpoint

  7. def clone(): AnyRef

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  8. def compress: Sink

    returns

    a new sink with Gzip compression enabled

    Definition Classes
    Sink

  9. final def eq(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  10. def equals(arg0: Any): Boolean

    Definition Classes
    AnyRef → Any

  11. def finalize(): Unit

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )

  12. final def getClass(): Class[_]

    Definition Classes
    AnyRef → Any

  13. def hashCode(): Int

    Definition Classes
    AnyRef → Any

  14. def isCheckpoint: Boolean

    returns

    true if this sink is a checkpoint

  15. final def isInstanceOf[T0]: Boolean

    Definition Classes
    Any

  16. def isSinkResult(tag: Int): (Path) ⇒ Boolean

    returns

    true if the file path has the name of an output channel with the proper tag and index or if it is a _SUCCESS file

    Definition Classes
    Sink

  17. final def ne(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  18. final def notify(): Unit

    Definition Classes
    AnyRef

  19. final def notifyAll(): Unit

    Definition Classes
    AnyRef

  20. def outputSetup(implicit sc: ScoobiConfiguration): Unit

    This method is called just before writing data to the sink

    This method is called just before writing data to the sink

    Definition Classes
    SinkSource → Sink

  21. final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes
    AnyRef

  22. def toString(): String

    Definition Classes
    AnyRef → Any

  23. final def wait(): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  24. final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  25. final def wait(arg0: Long): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Inherited from Sink

Inherited from AnyRef

Inherited from Any

Ungrouped