User Guide

User Guide

Scoobi - Bringing the productivity of Scala to Hadoop

Hadoop MapReduce is awesome, but it seems a little bit crazy when you have to write this to count words. Wouldn't it be nicer if you could simply write what you want to do:

val lines = fromTextFile("hdfs://in/...")

val counts = lines.mapFlatten(_.split(" ")).
                   map(word => (word, 1)).
                   groupByKey.
                   combine(Reduction.Sum.int)

counts.toTextFile("hdfs://out/...").persist

This is what Scoobi is all about. Scoobi is a Scala library that focuses on making you more productive at building Hadoop applications. It stands on the functional programming shoulders of Scala and allows you to just write what you want rather than how to do it.

Scoobi is a library that leverages the Scala programming language to provide a programmer friendly abstraction around Hadoop's MapReduce to facilitate rapid development of analytics and machine-learning algorithms.

In this user guide, you will find: