Installing Storm on Ubuntu

Storm is an open source ETL created by Nathan Marz in late 2011. Unlike Hadoop where data are processed offline in big batches, Storm takes another approach by aggregating streaming data on the fly so that aggregated data are immediately available. It is scalable, fault tolerant (no data loss guarantee) and the benchmarks showed that every node can process over a million tuples per seconds.

We describe below the different steps to install Storm in Ubuntu Linux describing the issues we had during the process.

Requirements

  • g++ (sudo apt-get install g++)
  • uuid-dev (sudo apt-get install uuid-dev)
  • leiningen 1.7. (sudo apt-get install leiningen)
  • Java (JDK from http://www.oracle.com/technetwork/java/javase/downloads/index.html). Make sure to have JAVA_HOME set to your JDK directory
  • Maven (install it from maven.apache.org)
  • Git (sudo apt-get install git)

Storm Installation

We recommend you download the stable version from the download page: http://storm-project.net/downloads.html.

First install ZeroMQ as follows:

$ bin/install_zmq.sh

If you have this error:

 Problem with the SSL CA cert (path? access rights?) while accessing...

You might want to ignore SSL verification as follows:

$ git config --global http.sslVerify false

And then re-run:

$ bin/install_zmq.sh

Now, if you are a Ubuntu user or a Mac user you might have the following error:

make[1]: *** No rule to make target `classdist_noinst.stamp', needed by
`org/zeromq/ZMQ.class'.  Stop.

This is a known bug found by ebroder (https://github.com/zeromq/jzmq/issues/114). To fix it, edit the file jzmq/src/Makefile.am and replace “classdist_noinst.stamp” by “classnoinst.stamp
and then re-run:

$ bin/install_zmq.sh

Add the bin directory to your PATH.

Running the Storm Starter examples

It contains some examples to play with. It can be downloaded as follows:

git clone http://github.com/nathanmarz/storm-starter

Build the project:

$ lein deps
$ lein compile
$ lein jar

Then run the ExclamationCount example:

$ java -cp $STORM_PATH/lib/*:$STORM_PATH/storm-0.8.2.jar:\
storm-starter-0.0.1-SNAPSHOT.jar \
storm.starter.ExclamationTopology

Assuming STORM_PATH is set to your storm directory.

Advertisements

About chimpler
http://www.chimpler.com

One Response to Installing Storm on Ubuntu

  1. Alex says:

    Hey,

    I am new to storm. I did everything thats mentioned here. I can see the UI on localhost:8080 . I can also see the number of active spouts and bolts. But where can i see the actual output.

    Thanks

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: