13432878738 发表于 2017-12-18 10:03:06

Hadoop如何修改HDFS文件存储块大小

  一、 临时修改可以在执行上传文件命令时,显示地指定存储的块大小。
  1. 查看当前 HDFS文件块大小
  我这里查看HDFS上的TEST目录下的jdk-7u25-linux-x64.gz文件存储块大小。
  1.1 终端命令方式查看
$ ./bin/hadoop fs -stat "%o" TEST/jdk-7u25-linux-x64.gz
  51200
  单位为B(字节)
  1.2或页面方式查看
  打开haddop datanode节点的hdfs监控页面:
  http://hadoop01:50070
http://img.07net01.com/upload/images/2015/01/05/1398928051632431.gifhttp://img.07net01.com/upload/images/2015/01/05/1398928051632432.png
  如下所示:
http://img.07net01.com/upload/images/2015/01/05/1398928051632433.png
  二、永久修改
  这个需要修改hdfs-site.conf配置文件,增加全局参数dfs.block.size。
  如下:
  修改后参数
http://img.07net01.com/upload/images/2015/01/05/1398928051632434.png
  把配置同步到其它节点,并重启hdfs。
$ bin/stop-dfs.sh
  no namenode to stop
  hadoop02: stopping datanode
  hadoop01: stopping secondarynamenode
$ bin/start-dfs.sh
  starting namenode, logging to /home/xiaoyu/hadoop-1.1.2/libexec/../logs/hadoop-xiaoyu-namenode-hadoop02.out
  hadoop02: starting datanode, logging to /home/xiaoyu/hadoop-1.1.2/libexec/../logs/hadoop-xiaoyu-datanode-hadoop02.out
  hadoop01: starting secondarynamenode, logging to /home/xiaoyu/hadoop-1.1.2/libexec/../logs/hadoop-xiaoyu-secondarynamenode-hadoop01.out
  重新查看TEST/jdk-7u25-linux-x64.gz 文件存储的块大小:
http://img.07net01.com/upload/images/2015/01/05/1398928051632435.png
  重新上传文件到HDFS,并重命名为 比较用_jdk-7u25-linux-x64.gz
http://img.07net01.com/upload/images/2015/01/05/1398928051632436.png
页: [1]
查看完整版本: Hadoop如何修改HDFS文件存储块大小