首页 / HADOOP / Step by step install and run Hadoop 2.9.1 on Windows 10 64 bit (最全步骤整理)

Step by step install and run Hadoop 2.9.1 on Windows 10 64 bit (最全步骤整理)

内容导读

互联网集市收集整理的这篇技术教程文章主要介绍了Step by step install and run Hadoop 2.9.1 on Windows 10 64 bit (最全步骤整理)，小编现在分享给大家，供广大互联网技能从业者学习和参考。文章包含4625字，纯文字阅读大概需要7分钟。

内容图文

1. Java

Download the Java 1.8 64 bit version from https://java.com/en/download/  --------- If not 64 bit java there will be error when start resource node manager

Java 安装自选目录：比如 C:\java64 -----这里重要

Once installed confirm that you’re running the correct version from command line using ‘java -version’ command, output of which you can confirm in command line like this:

C:\Users>java -version
java version "1.8.0_251"
Java(TM) SE Runtime Environment (build 1.8.0_251-b08)
Java HotSpot(TM) 64-Bit Server VM (build 25.251-b08, mixed mode)

2. WinRAR/7-ZIP

Downloaded and installed WinRAR 64 bit release from http://www.rarlab.com/download.htm that will later allow me to decompress Linux type tar.gz packages on Windows. Or use 7-zip https://www.7-zip.org/download.html

3. Hadoop

The next step was to install a Hadoop distribution. Download a binary form:

https://archive.apache.org/dist/hadoop/common/hadoop-2.9.1/hadoop-2.9.1.tar.gz

Unzip it and rename folder name to hadoop and put it under C:\Learning  -------  如果用你自己的目录，名字中间不要有空格(因为Hadoop初始开发基于linux， linux 文件名字不能有空格)。 后续设置配置文件内容时目录都要修改

 

It‘s time to start the Hadoop cluster with a single node.

3. 1 Setup Environmental Variables

list as below:

技术分享图片

 

 

 技术分享图片

技术分享图片

 

Add following dir to path:

 

 技术分享图片

 

3.2 Edit Hadoop Configuration

注意：如果你不想自己配置下面 3.2 过程， 除了3.2.3 其它的步骤可以忽略， 那么下载下面链接（bin data etc）文件夹放到你的hadoop目录下覆盖（https://github.com/yjy24/bigdata_learning ）, 然后从3.2.3 开始继续！

 

3.2.1 新建data 目录和子目录 

c:/Learning/hadoop/data

c:/Learning/hadoop/data/namenode

c:/Learning/hadoop/data/datanode

 

3.2.2 Configure Hadoop 

a) C:\Learning\hadoop\etc\hadoop\core-site.xml

<configuration>
  <property>
    <name>fs.defaultFS</name>
    <value>hdfs://localhost:9000</value>
  </property>
</configuration>
 

 

 

b)  C:\Learning\hadoop\etc\hadoop\mapred-site.xml

<configuration>
  <property>
    <name>mapreduce.framework.name</name>
    <value>yarn</value>
  </property>
</configuration>
 

c) C:\Learning\hadoop\etc\hadoop\hdfs-site.xml

<configuration>
    <property>
    <name>dfs.replication</name>
    <value>1</value>
  </property>
      <property>
    <name>dfs.namenode.name.dir</name>
    <value>file:///Learning/hadoop/data/namenode</value>
  </property>
      <property>
    <name>dfs.datanode.data.dir</name>
    <value>file:///Learning/hadoop/data/datanode</value>
  </property>
</configuration>
 
d) C:\Learning\hadoop\etc\hadoop\yarn-site.xml
</configuration>
<configuration>
    <property>
    <name>dfs.replication</name>
    <value>1</value>
  </property>
      <property>
    <name>dfs.namenode.name.dir</name>
    <value>file:///Learning/hadoop/data/namenode</value>
  </property>
      <property>
    <name>dfs.datanode.data.dir</name>
    <value>file:///Learning/hadoop/data/datanode</value>
  </property>
</configuration>
 
e) C:\Learning\hadoop\etc\hadoop\hadoop-env.cmd
显性设置JAVA_HOME
set JAVA_HOME=C:\java64
 

3.2.3 windows 需要的的 libs

1. install Visual C++ 2010 Redistributable Package (x64)： https://www.microsoft.com/en-us/download/details.aspx?id=14632

2. 把 winutils.exe 等 windows 需要的文件放到 C:\Learning\hadoop\bin 下面
下载 https://github.com/yjy24/bigdata_learning 下面的bin 直接覆盖C:\Learning\hadoop\bin
里面有window需要的所有文件 -- 针对 2.9.1
 
3. 把里面的 hadoop.dll  放置到 C:\Windows\System32 一份

3.3 Start Hadoop all service

3.3.1 格式化 Hadoop namenode：

命令： hdfs namenode -format
 Log on scren:

20/04/19 10:40:13 INFO common.Storage: Storage directory \Learning\hadoop\data\namenode has been successfully formatted.
20/04/19 10:40:13 INFO namenode.FSImageFormatProtobuf: Saving image file \Learning\hadoop\data\namenode\current\fsimage.ckpt_0000000000000000000 using no compression
20/04/19 10:40:13 INFO namenode.FSImageFormatProtobuf: Image file \Learning\hadoop\data\namenode\current\fsimage.ckpt_0000000000000000000 of size 320 bytes saved in 0 seconds .
20/04/19 10:40:13 INFO namenode.NNStorageRetentionManager: Going to retain 1 images with txid >= 0
20/04/19 10:40:13 INFO namenode.NameNode: SHUTDOWN_MSG:

3.3.2 启动 Hadoop all service：

C:\Learning\hadoop\sbin\start-all.cmd      ---- 要用管理员身份运行

四个service窗口会打开运行 ------ 任何一个都不能有错误

恭喜你可以开始运行你的Hadoop程序了！

3.4 Open Hadoop GUI

Once all above steps were completed,opened browser and navigated to: http://localhost:8088/cluster

 这里可以查看job的运行情况 log 信息。

技术分享图片

--------------------------------------------------End---------------------------------------------------

原文：https://www.cnblogs.com/yjyyjy/p/12731968.html

内容总结

以上是互联网集市为您收集整理的Step by step install and run Hadoop 2.9.1 on Windows 10 64 bit (最全步骤整理)全部内容，希望文章能够帮你解决Step by step install and run Hadoop 2.9.1 on Windows 10 64 bit (最全步骤整理)所遇到的程序开发问题。 如果觉得互联网集市技术教程内容还不错，欢迎将互联网集市网站推荐给程序员好友。

内容备注

版权声明：本文内容由互联网用户自发贡献，该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容， 请发送邮件至 gblab@vip.qq.com 举报，一经查实，本站将立刻删除。

内容手机端

扫描二维码推送至手机访问。

本文链接：https://qyyshop.com/info/1210401.html

来源：【匿名】