com.nicta.scoobi.io

sequence

package sequence

Visibility
  1. Public
  2. All

Type Members

  1. class CheckedSeqSource[K, V, A] extends SeqSource[K, V, A]

    This class can check if the source types are ok, based on the Manifest of the input types

  2. class PartitionedSequenceOutputFormat[P, K, V] extends PartitionedOutputFormat[P, K, V]

    This format creates a new sequence record writer for each different path that's generated by the partition function Each record writer defines a specific OutputCommitter that will define a different work directory for a given key.

    This format creates a new sequence record writer for each different path that's generated by the partition function Each record writer defines a specific OutputCommitter that will define a different work directory for a given key.

    All the generated paths will be created under temporary dir/sink id in order to collect them more rapidly with just a rename of directories (see OutputChannel)

  3. trait SeqSchema[A] extends AnyRef

    Type class for conversions between basic Scala types and Hadoop Writable types.

  4. case class SeqSink[K, V, B](path: String, keyClass: Class[K], valueClass: Class[V], outputConverter: InputOutputConverter[K, V, B], overwrite: Boolean, check: OutputCheck = Sink.defaultOutputCheck, checkpoint: Option[Checkpoint] = None, compression: Option[Compression] = None) extends DataSink[K, V, B] with SinkSource with Product with Serializable

    class that abstracts all the common functionality of persisting to sequence files.

  5. class SeqSource[K, V, A] extends DataSource[K, V, A]

  6. trait SequenceInput extends AnyRef

    Smart functions for materialising distributed lists by loading Sequence files.

  7. trait SequenceOutput extends AnyRef

    Smart functions for persisting distributed lists by storing them as Sequence files.

Value Members

  1. object SeqSchema

  2. object SequenceInput extends SequenceInput

  3. object SequenceOutput extends SequenceOutput

Ungrouped