Сообщения

Сообщения за Январь, 2015

Big Data on your local machine : Installing Hadoop Cluster (Multi Node regime)

Изображение
A few days ago I published an article about Hadoop installation (in Single Node regime). After that  I received so many emails  from guys who wanted to change their single node to cluster (Multi-Node regime).

So, I’ll help you ‘cause I had similar problems a few months ago when my System Administrator took his holidays and left me face to face with a lot of single machines without Cluster Ring of Power.


Thinking about future cluster

Let’s start!
Of course, we should have a Virtual Machine (VM) with Ubuntu 14.10 and installed Hadoop 2.6.0 according instructions from previous paper.
In the first, we need to design future cluster and understand how many nodes are required for our deals. If it is your first cluster and your have few separated primary nodes, it will be easy to join them one by one.




The master node will run the ‘father’ threads for each floor: NameNode for the HDFS storage layer, and JobTracker for the MapReduce processing layer. All machines (including master) will run the ‘chil…

Big Data on your local machine : How to install Hadoop 2.6.0

Many people from time to time google next phrase: “how to install hadoop in my cloud”


Today I try to help all these guys with their problem. I am going to describe step by step all my troubles with Apache Hadoop 2.6.0 installation.

Exordium
Let’s start!


Of course, you should have a Virtual Machine (VM) with Ubuntu 14.10, for example. I don’t give you garantee for another Unix systems, sorry.


Let’s suppose you have already installed the operating system.

If you haven’t ssh and rsync clients, please install it

$ sudo apt-get update && sudo apt-get upgrade $ sudo apt-get install ssh $ sudo apt-get install rsync

Before Hadoop installation you need to have a fresh version of Java on your VM. If you have no Java, you should install it by next commands:

if you want OpenJDK 7
$ sudo apt-get install openjdk-7-jdk
if you want Oracle JDK, please clear all OpenJDK dependencies
$ sudo apt-get purge openjdk*
And after that you can install Oracle JDK 7 by next commands
$ sudo apt-get install python-so…