avro

Type Members

1. case class AvroDataSource[K, A](paths: Seq[Path], inputConverter: InputConverter[AvroKey[K], NullWritable, A], schema: AvroSchema[A], checkSchemas: Boolean = true, check: InputCheck = Source.defaultInputCheck) extends DataSource[AvroKey[K], NullWritable, A] with Product with Serializable

   

2. trait AvroFixed[T] extends AnyRef

   

   Provide an implicit of this, if your type is a fixed.

   Provide an implicit of this, if your type is a fixed. Then a FixedSchema can kick-in

3. trait AvroInput extends AnyRef

   

   Functions for materialising distributed lists by loading Avro files

4. trait AvroOutput extends AnyRef

   

   Functions for persisting distributed lists by storing them as Avro files.

5. trait AvroParsingImplicits extends AnyRef

   

6. trait AvroSchema[A] extends AnyRef

   

   Defines the Avro schema for a given Scala type.

7. case class AvroSink[K, B](schema: AvroSchema[B], path: String, outputConverter: OutputConverter[AvroKey[K], NullWritable, B], overwrite: Boolean = false, check: OutputCheck = Sink.defaultOutputCheck, checkpoint: Option[Checkpoint] = None, compression: Option[Compression] = None) extends DataSink[AvroKey[K], NullWritable, B] with SinkSource with Product with Serializable

   

8. class GenericAvroKeyInputFormat[T] extends AvroKeyInputFormat[T]

   

9. class GenericAvroKeyOutputFormat[T] extends AvroKeyOutputFormat[T]

   

   This AvroKeyOutputFormat is used when writing out Generic Record.

   This AvroKeyOutputFormat is used when writing out Generic Record.

   It uses the schema that is attached to the first key to be written to file

10. trait LowPriorityAvroSchemaImplicits extends AnyRef