IT & Software  >  Hadoop Tutorials: Brief Introduction  >  How to Install Hadoop 2.7 on Ubuntu | Hadoop Installation Steps

How to Install Hadoop 2.7 on Ubuntu | Hadoop Installation Steps Notes | Study Hadoop Tutorials: Brief Introduction - IT & Software

Document Description: How to Install Hadoop 2.7 on Ubuntu | Hadoop Installation Steps for IT & Software 2022 is part of Hadoop Tutorials: Brief Introduction preparation. The notes and questions for How to Install Hadoop 2.7 on Ubuntu | Hadoop Installation Steps have been prepared according to the IT & Software exam syllabus. Information about How to Install Hadoop 2.7 on Ubuntu | Hadoop Installation Steps covers topics like and How to Install Hadoop 2.7 on Ubuntu | Hadoop Installation Steps Example, for IT & Software 2022 Exam. Find important definitions, questions, notes, meanings, examples, exercises and tests below for How to Install Hadoop 2.7 on Ubuntu | Hadoop Installation Steps.

Introduction of How to Install Hadoop 2.7 on Ubuntu | Hadoop Installation Steps in English is available as part of our Hadoop Tutorials: Brief Introduction for IT & Software & How to Install Hadoop 2.7 on Ubuntu | Hadoop Installation Steps in Hindi for Hadoop Tutorials: Brief Introduction course. Download more important topics related with notes, lectures and mock test series for IT & Software Exam by signing up for free. IT & Software: How to Install Hadoop 2.7 on Ubuntu | Hadoop Installation Steps Notes | Study Hadoop Tutorials: Brief Introduction - IT & Software
1 Crore+ students have signed up on EduRev. Have you?

1. Install Hadoop 2.7 on Ubuntu Tutorial: Objective

This Install Hadoop 2.7 on Ubuntu tutorial explains about How to install and configure Hadoop 2.7.x on Ubuntu? In this tutorial, we will step by step guide you on how to install Hadoop and deploy Hadoop on the Single server (single node cluster) on Ubuntu OS. This quick start will help you to install Hadoop 2.7 on ubuntu, configure and run it in less than 10 min. While installation we will enable YARN so that apart from MapReduce you can run different types of applications like Spark.

Looking to start career in Big Data and Hadoop – Learn from Experts


2. How to Install Hadoop 2.7 on Ubuntu?

In this section of  Hadoop 2.7 installation tutorial, we will learn step by step to install and configure Hadoop 2.7.x on Ubuntu OS. Follow the steps given below to install Hadoop 2.7 –

2.1. Prerequisites to install Hadoop 2.7 on Ubuntu

If you are using Windows/Mac OS to install Hadoop 2.7 you can create a virtual machine and install Ubuntu using VMWare Player, alternatively, you can create a virtual machine and install Ubuntu using Oracle Virtual Box.

I. Install Oracle Java 8

a. Install Python Software Properties

sudo apt-get install python-software-properties

b. Add Repository

sudo add-apt-repository ppa:webupd8team/java

c. Update the source list

sudo apt-get update

d. Install Java

sudo apt-get install oracle-java8-installer

II. Setup Password-less SSH

a. Install Open SSH Server & Open SSH Client

sudo apt-get install openssh-server openssh-client

b. Generate Public & Private Key Pairs

ssh-keygen -t rsa -P ""

c. Configure password-less SSH

cat $HOME/.ssh/id_rsa.pub >> $HOME/.ssh/authorized_keys

d. Check by SSH to localhost

ssh localhost


3.1. Configure, Setup and Install Hadoop 2.7 on Ubuntu

I. Download Hadoop

https://archive.apache.org/dist/hadoop/common/hadoop-2.7.1/hadoop-2.7.1.tar.gz

II. Untar Tar ball

tar xzf hadoop-2.7.1.tar.gz

Note: All the required jars, scripts, configuration files, etc. are available in HADOOP_HOME directory (hadoop-2.7.1)

III. Setup Configuration

a. Edit .bashrc

Edit .bashrc file located in user’s home directory and add following parameters:


  1. export HADOOP_PREFIX=/home/hdadmin/hadoop-2.7.1
  2. export PATH=$PATH:$HADOOP_PREFIX/bin
  3. export PATH=$PATH:$HADOOP_PREFIX/sbin
  4. export HADOOP_MAPRED_HOME=${HADOOP_PREFIX}
  5. export HADOOP_COMMON_HOME=${HADOOP_PREFIX}
  6. export HADOOP_HDFS_HOME=${HADOOP_PREFIX}
  7. export YARN_HOME=${HADOOP_PREFIX}
  8. export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_PREFIX/lib/native
  9. export HADOOP_OPTS="-Djava.library.path=$HADOOP_PREFIX/lib"

Note: After above step restarts the terminal so that all the environment variables will come into effect

b. Edit hadoop-env.sh

Edit hadoop-env.sh (hadoop-env.sh is located in etc/hadoop inside Hadoop installationdirectory) and set JAVA_HOME:

export JAVA_HOME=<root-of-your-Java-installation> (eg: /usr/lib/jvm/java-8-oracle/)

c. Edit core-site.xml

Edit core-site.xml (core-site.xml is located in etc/hadoop inside Hadoop installation directory) and add following entries:

  1. <configuration>
  2. <property>
  3. <name>fs.defaultFS</name>
  4. <value>hdfs://localhost:9000</value>
  5. </property>
  6. <property>
  7. <name>hadoop.tmp.dir</name>
  8. <value>/home/hdadmin/hdata</value>
  9. </property>
  10. </configuration>

Note: you must have Read Write privileges in /home/hdadmin/hdata else specify a location where you have Read Write privileges.

d. Edit hdfs-site.xml

Edit hdfs-site.xml (hdfs-site.xml is located in etc/hadoop inside Hadoop installation directory) and add following entries:

  1. <configuration>
  2. <property>
  3. <name>dfs.replication</name>
  4. <value>1</value>
  5. </property>
  6. </configuration>


e. Edit mapred-site.xml

Edit mapred-site.xml (mapred-site.xml.template is located in etc/hadoop inside Hadoop installation directory, copy the file with the name mapred-site.xml) and add following entries:

  1. <configuration>
  2. <property>
  3. <name>mapreduce.framework.name</name>
  4. <value>yarn</value>
  5. </property>
  6. </configuration>


f. Edit yarn-site.xml

Edit yarn-site.xml (yarn-site.xml is located in etc/hadoop inside Hadoop installation directory) and add following entries:

  1. <configuration>
  2. <property>
  3. <name>yarn.nodemanager.aux-services</name>
  4. <value>mapreduce_shuffle</value>
  5. </property>
  6. <property>
  7. <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
  8. <value>org.apache.hadoop.mapred.ShuffleHandler</value>
  9. </property>
  10. </configuration>


4.1. Start the Cluster

I. Format the name node:

hdfs namenode -format

NOTE: Namenode should be formatted just once when you install Hadoop.

II. Start HDFS Services:

start-dfs.sh

III. Start YARN Services:

start-yarn.sh

IV. Check whether services have been started

  1. jps
  2. NameNode
  3. DataNode
  4. ResourceManager
  5. NodeManager
  6. SecondaryNameNode

5.1. Run Map-Reduce Jobs

I. Run word count example:

  1. hdfs dfs -mkdir /data
  2. hdfs dfs -put <file> /data
  3. yarn jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.1.jar wordcount /data /data-out
  4. hdfs dfs -cat /data-out/*

To work with HDFS and perform various operations follow this guide


6.1. Stop the Cluster

I. Stop HDFS Services:

stop-dfs.sh

II. Stop YARN Services:

stop-yarn.sh

This was all on the tutorial to install Hadoop 2.7 on Ubuntu in 10 minutes.

The document How to Install Hadoop 2.7 on Ubuntu | Hadoop Installation Steps Notes | Study Hadoop Tutorials: Brief Introduction - IT & Software is a part of the IT & Software Course Hadoop Tutorials: Brief Introduction.
All you need of IT & Software at this link: IT & Software

Related Searches

pdf

,

study material

,

ppt

,

Summary

,

Free

,

How to Install Hadoop 2.7 on Ubuntu | Hadoop Installation Steps Notes | Study Hadoop Tutorials: Brief Introduction - IT & Software

,

Semester Notes

,

practice quizzes

,

Sample Paper

,

Previous Year Questions with Solutions

,

Important questions

,

Viva Questions

,

Objective type Questions

,

mock tests for examination

,

video lectures

,

past year papers

,

Exam

,

shortcuts and tricks

,

How to Install Hadoop 2.7 on Ubuntu | Hadoop Installation Steps Notes | Study Hadoop Tutorials: Brief Introduction - IT & Software

,

How to Install Hadoop 2.7 on Ubuntu | Hadoop Installation Steps Notes | Study Hadoop Tutorials: Brief Introduction - IT & Software

,

Extra Questions

,

MCQs

;