Bigdata

Hadoop eclipse环境

Submitted by Lizhe on Tue, 04/04/2017 - 14:06

https://github.com/winghc/hadoop2x-eclipse-plugin/blob/master/release/hadoop-eclipse-plugin-2.6.0.jar

iptables -t nat -A DOCKER -p tcp --dport 9000 -j DNAT --to-destination 172.17.0.19:9000

iptables -t nat -A DOCKER -p tcp --dport 9001-j DNAT --to-destination 172.17.0.19:9001

iptables -A DOCKER ! -i docker0 -o docker0 -p tcp --dport 9000 -d 172.17.0.2 -j ACCEPT

iptables -A DOCKER ! -i docker0 -o docker0 -p tcp --dport 9001 -d 172.17.0.2 -j ACCEPT

Hadoop环境配置2

Submitted by Lizhe on Mon, 04/03/2017 - 18:56

安装jdk1.8

yum  -y install java-1.8.0-openjdk java-1.8.0-openjdk-devel 

安装wget

yum install wget

下载hadoop

wget http://mirrors.hust.edu.cn/apache/hadoop/common/hadoop-2.6.5/hadoop-2.6.5.tar.gz

重命名然后解压

mv hadoop-2.6.5.tar.gz hadoop.tar.gz

tar -zxvf hadoop.tar.gz

 

http://blog.csdn.net/emptyset110/article/details/49835101

这里我是把hadoop的根路径放在了

/download/hadoop-2.6.5

Hadoop环境配置1

Submitted by Lizhe on Mon, 04/03/2017 - 00:15

打算按集群的模式做,所以先建几个docker容器

docker run --name master -p 8088:8088 -p 50070:50070  -i -t 2785d012ae3e

docker run --name slave1 -i -t 2785d012ae3e

docker run --name slave2 -i -t 2785d012ae3e

然后是让这几台server互相免密码通信

docker 镜像的centos竟然没带ssh, 先装ssh ( 在所有容器上 )

yum -y install openssh-server openssh-clients

去掉/etc/ssh/sshd_config其中2行的注释,每台服务器都要设置,
#RSAAuthentication yes
#PubkeyAuthentication yes

同时打开root远程登录

PermitRootLogin yes #root能使用ssh登录

在3台主机下

ssh-keygen -t rsa

然后直接全部回车,空密码