rtt

Type Members

1. trait Configured extends Configurable

   

2. class JarBuilder extends AnyRef

   

   Class to manage the creation of a new JAR file.

3. case class MetadataClassBuilder[T](className: String, metaData: Any, scoobiClassLoader: ClassLoader, configuration: Configuration)(implicit mf: Manifest[T]) extends Product with Serializable

   

   A class for building a class extending T at run-time.

   A class for building a class extending T at run-time.

   The purpose is to create a unique class which, when instantiated, will be able to perform the operations described in T, while using some metadata object serialised to the distributed cache

   It takes:

   • a type T which is going to be the parent of the generated class
   • the className (which must be unique for the class to build)
   • some metadata for the class (like a Map from tags to WireFormats, if it's a TaggedValue)

   The metadata is distributed to the cache and a "metadataTag" method is added to the built class so that it can be retrieved.

   For example, a TaggedValue is a class, defined for several tags, which is able to read/write values of different types (described by their WireFormats), with one type per tag. A concrete instance of that class will have a unique name, TV92, and will provide a method returning the path of the distributed cache file containing all WireFormats per tags.

   The TaggedKey class then retrieves the map with the help of the ScoobiMetadata class and can use the wireformats to serialise/deserialise values

4. trait MetadataGroupings extends TaggedMetadata

   

   Set of Groupings accessible by tag

5. abstract class MetadataScoobiWritable extends ScoobiWritable[Any]

   

6. abstract class MetadataTaggedGroupingComparator extends TaggedGroupingComparator with MetadataWireFormats with MetadataGroupings

   

7. abstract class MetadataTaggedKey extends TaggedKey with MetadataTaggedWritable with MetadataGroupings

   

   Tagged key with some metadata describing each channel.

   Tagged key with some metadata describing each channel. It has WireFormats and Groupings

8. abstract class MetadataTaggedPartitioner extends Partitioner[TaggedKey, TaggedValue] with TaggedPartitioner with MetadataWireFormats with MetadataGroupings

   

   This partitioner uses the grouping of the current key tag and partitions based on the key value

9. abstract class MetadataTaggedValue extends TaggedValue with MetadataTaggedWritable

   

10. trait MetadataTaggedWritable extends MetadataWireFormats

    

    This trait represents a Writable capable of reading/writing different type of data depending on a tag and having WireFormats accessible for each tag

11. trait MetadataWireFormats extends TaggedMetadata

    

    WireFormats accessible by tags

12. final case class RuntimeClass(name: String, clazz: Class[_], bytecode: Array[Byte]) extends Product with Serializable

    

    A class representing a class that has been generated at run-time.

13. abstract class ScoobiWritable[A] extends Writable with Configured

    

    The super-class of all "value" types used in Hadoop jobs.

14. trait Tagged extends Configured

    

    A collection of types indexed by a tag

15. trait TaggedGroupingComparator extends RawComparator[TaggedKey]

    

    Custom GroupingComparator for tagged keys.

16. trait TaggedKey extends Tagged with WritableComparable[TaggedKey]

    

    A tagged value for Hadoop keys.

    A tagged value for Hadoop keys. Specifically this will be a K2 type so must implement the WritableComparable interface

17. trait TaggedMetadata extends Configured

    

    Set of metadata, which can be a different tuple for each tag (and all tuples don't have to have the same size)

18. trait TaggedPartitioner extends Partitioner[TaggedKey, TaggedValue]

    

    Custom partitioner for tagged key-values.

19. trait TaggedValue extends Tagged with Writable

    

    A tagged value for Hadoop values.

    A tagged value for Hadoop values. Specifically this will be a V2 type so must implement the Writable interface.

    Before using a TaggedValue the appropriate tag must be set. By default, it is 0