您现在的位置 >> Hadoop教程 >> Hadoop实战 >> hadoop专题  
 

hadoop_install

【作者:Hadoop实战专家】【关键词:hadoop ssh 】 【点击:11883次】【2013-03-2】
http://www.easyhadoop.com //hadoop自动化安装 #!/bin/sh if [ $# -lt 3 ]; then echo "Usage: $0 map.tasks.maximum reduce.tasks.maximum memory child.java.opts" else map=$1 reduce=$2 mem=$3 yum -y install lrzsz gcc gcc-c++ libstdc++-devel /usr/  

相关热门搜索:hadoop集群架构图 hadoop开发者第三期

大数据标签:hadoop hive bigdata

http://www.easyhadoop.com

//hadoop自动化安装

#!/bin/sh

if [ $# -lt 3 ]; then

echo "Usage: $0 map.tasks.maximum reduce.tasks.maximum memory child.java.opts"

else

map=$1

reduce=$2

mem=$3

yum -y install lrzsz gcc gcc-c++ libstdc++-devel

/usr/sbin/groupadd hadoop

/usr/sbin/useradd hadoop -g hadoop

mkdir -p /opt/modules/hadoop/

mkdir -p /opt/data/hadoop1/

chown hadoop:hadoop /opt/data/hadoop1/

echo "-------------config hosts----------------"

wget http://www.easyhadoop.com/hadoop/hosts

cat hosts >> /etc/hosts

echo "----------------env init finish and prepare su hadoop---------------"

HADOOP=/home/hadoop

cd $HADOOP

mkdir .ssh

ssh-keygen -q -t rsa -N "" -f $HADOOP/.ssh/id_rsa

cd $HADOOP/.ssh/ && cat id_rsa.pub > $HADOOP/.ssh/authorized_keys

chmod go-rwx $HADOOP/.ssh/authorized_keys

wget http://www.easyhadoop.com/hadoop/hadoop-0.20.203.0.tar.gz

wget http://www.easyhadoop.com/hadoop/hadoop-gpl-packaging-0.2.8-1.x86_64.rpm

wget http://www.easyhadoop.com/hadoop/jdk-6u21-linux-amd64.rpm

wget http://www.easyhadoop.com/hadoop/lrzsz-0.12.20-19.x86_64.rpm

wget http://www.easyhadoop.com/hadoop/lzo-2.04-1.el5.rf.x86_64.rpm

wget http://www.easyhadoop.com/hadoop/lzo-2.06.tar.gz

wget http://www.easyhadoop.com/hadoop/lzop-1.03.tar.gz

wget http://www.easyhadoop.com/hadoop/hive-0.7.1.tar.gz

wget http://www.easyhadoop.com/hadoop/pig.tar.gz

mkdir $HADOOP/hadoop

mv *.tar.gz $HADOOP/hadoop

mv *.rpm $HADOOP/hadoop

cd $HADOOP/hadoop

rpm -ivh jdk-6u21-linux-amd64.rpm

rpm -ivh lrzsz-0.12.20-19.x86_64.rpm

rpm -ivh lzo-2.04-1.el5.rf.x86_64.rpm

rpm -ivh hadoop-gpl-packaging-0.2.8-1.x86_64.rpm

tar xzvf lzo-2.06.tar.gz

cd lzo-2.06 && ./configure --enable-shared && make && make install

cp /usr/local/lib/liblzo2.* /usr/lib/

cd ..

tar xzvf  lzop-1.03.tar.gz

cd lzop-1.03

./configure && make && make install && cd ..

chown -R hadoop:hadoop  /opt/modules/hadoop/

cp hadoop-0.20.203.0.tar.gz /opt/modules/hadoop/

cd /opt/modules/hadoop/ && tar -xzvf hadoop-0.20.203.0.tar.gz

sed -i "s/^6<\/value>/${map}<\/value>/g" /opt/modules/hadoop/hadoop-0.20.203.0/conf/mapred-site.xml

sed -i "s/^2<\/value>/${reduce}<\/value>/g" /opt/modules/hadoop/hadoop-0.20.203.0/conf/mapred-site.xml

sed -i "s/^-Xmx1536M<\/value>/-Xmx${mem}M<\/value>/g" /opt/modules/hadoop/hadoop-0.20.203.0/conf/mapred-site.xml

mkdir -p /opt/modules/hive/

cp hive-0.7.1.tar.gz /opt/modules/hive/

cd /opt/modules/hive/ && tar -xzvf hive-0.7.1.tar.gz

chown -R hadoop:hadoop  /opt/modules/hive/

chown -R hadoop:hadoop /opt/modules/hadoop/

chown -R hadoop:hadoop /home/hadoop/

mkdir -p /opt/modules/pig/

cp pig.tar.gz /opt/modules/pig/

cd /opt/modules/pig/ && tar -xzvf pig.tar.gz

chown -R hadoop:hadoop  /opt/modules/pig/

#sudo -u hadoop /opt/modules/hadoop/hadoop-0.20.203.0/bin/hadoop namenode -format

#sudo -u hadoop /opt/modules/hadoop/hadoop-0.20.203.0/bin/hadoop-daemon.sh start namenode

#sudo -u hadoop /opt/modules/hadoop/hadoop-0.20.203.0/bin/hadoop-daemon.sh start jobtracker

#sudo -u hadoop /opt/modules/hadoop/hadoop-0.20.203.0/bin/hadoop-daemon.sh start datanode

#sudo -u hadoop /opt/modules/hadoop/hadoop-0.20.203.0/bin/hadoop-daemon.sh start tasktracker

fi

curl -# http://www.easyhadoop.com/setup.html?type=setup

大数据系列hadoop相关文章:

最新评论
开心哒2014-09-09 06:35:36
荒芜了烽火边城 2014/8/20 10:02:10
郑旭进2014-09-09 01:26:41
【关于Hadoop里面“批处理,实时,交互,流,NOSQL”等术语的解释】很多人会对Hadoop里面使用的一些专业术语而感到困惑,如Hadoop的术语,流输入(streaming),实时(real time)等等,这里会对这些术语简单介绍一下: 批处理(Batch) 批处理是指按照预定的方式运行数… http://t.cn/8FsPwMh
幸福~~弦2014-09-08 03:48:16
问个问题 hbase有没有 session失效时间 类似的失效时间 概念?
弩兵2014-09-07 08:46:20
你把两行交换一下
Real谭曾2014-09-07 02:36:01
刚的问题查出来了
fajingt2014-09-06 02:14:59
hadoop 学习笔记:mapreduce框架详解 - 夏天的森林 - 博客园 http://t.cn/zHdCvMC
惠子2014-09-05 04:24:19
主动权在你手里
旋律→放松2014-09-04 05:58:15
hadoop......
Avril2014-09-03 04:49:57
发表了博文 《Hadoop的预测评分引擎:来自Zementis的UPPI》 - Hadoop的预测评分引擎:来自Zementis的UPPI PMML 工具 PMML 由 Data Minin http://t.cn/8F3DMv0
鮟琪菈2014-09-02 11:38:55
发表了一篇转载博文 《[转载]Hadoop0.23.0初探2---HDFS Federation部署》 - http://t.cn/8s19308
 
  • Hadoop生态系统资料推荐