Hello World from OSS Silicon Valley

• List of Backups
• View the diff.
• View the diff current.
• View the source.
• Go to HowToUse/Hadoop/3.0.
  • 1 (2016-10-01 (Sat) 09:34:38)
  • 2 (2016-10-01 (Sat) 09:36:38)
  • 3 (2016-10-02 (Sun) 10:21:30)

---

_ Prerequisite

• CentOS installation (You can refer HowToUse/CentOS/6.5)
• Java installation (You can refer HowToUse/Java/1.8)

_ Install&Setup

Step.1

Create user account for Hadoop.

$ sudo useradd hadoop
$ sudo passwd hadoop

Step.2

Download source files from here and unarchive the file.

$ wget http://ftp.meisei-u.ac.jp/mirror/apache/dist/hadoop/common/hadoop-3.0.0-alpha1/hadoop-3.0.0-alpha1.tar.gz
$ tar xzvf hadoop-3.0.0-alpha1.tar.gz
$ mv hadoop-3.0.0-alpha1 /usr/local
$ chown -R hadoop:hadoop /usr/local/hadoop-3.0.0-alpha1

Step.3

Setup environmental variables in .bashrc.

$ vi ~/.bashrc

export JAVA_HOME=/usr/lib/jvm/java-1.8.0
export HADOOP_INSTALL=/usr/share/hadoop-3.0.0-alpha1
export PATH=$HADOOP_INSTALL/bin:$JAVA_HOME/bin:$PATH
export HADOOP_CLASSPATH=${JAVA_HOME}/lib/tools.jar

_ HowToUse

Step.1

Create Hadoop code.

$ vi WordCount.java

You can see sample code from here

Step.2

Compile sample code and create jar file.

$ hadoop com.sun.tools.javac.Main WordCount.java
$ jar cf wc.jar WordCount*.class

Step.3

Prepare input files for WordCount.

$ mkdir input
$ vi input/file01
$ vi input/file02

You can see sample code from here

Step.4

Execute hadoop job for WordCount.

$ hadoop jar wc.jar WordCount ../input/ ../output

Then you will see output in output directory as below.

[hadoop@localhost work]$ ls output/
_SUCCESS  part-r-00000

$ vi output/part-r-00000

Bye     1
Goodbye 1
Hadoop  2
Hello   2
World   2

_ Author

S.Yatsuzuka