二：HDFS 命令指南

Posted 2020-10-04 月饼馅饺子

命令具体选项请参考：

http://hadoop.apache.org/docs/r2.6.3/hadoop-project-dist/hadoop-hdfs/HDFSCommands.html

 

1.HDFS命令通过bin/hdfs执行，语法如下：

 hdfs [--config confdir] [COMMAND] [GENERIC_OPTIONS] [COMMAND_OPTIONS]

        配置文件          用户命令   基本选项           命令选项

COMMAND_OPTION	Description
--config confdir	Overwrites the default Configuration directory. Default is $HADOOP_HOME/conf.
GENERIC_OPTIONS	The common set of options supported by multiple commands. Full list is here.
COMMAND_OPTIONS	Various commands with their options are described in the following sections. The commands have been grouped into and . 来源： http://hadoop.apache.org/docs/r2.6.3/hadoop-project-dist/hadoop-hdfs/HDFSCommands.html

 

2.GENREIC_OPTIONS基本命令如下：

GENERIC_OPTION	Description
-conf <configuration file>	Specify an application configuration file.  指定配置文件
-D <property>=<value>	Use value for given property.                   指定属性
-jt <local> or <resourcemanager:port>	Specify a ResourceManager. Applies only to job.   指定资源管理器
-files <comma separated list of files>	Specify comma separated files to be copied to the map reduce cluster. Applies only to job.
-libjars <comma seperated list of jars>	Specify comma separated jar files to include in the classpath. Applies only to job.
-archives <comma separated list of archives>	Specify comma separated archives to be unarchived on the compute machines. Applies only to job.

 

3.用户命令

文件系统命令：hdfs dfs [GENERIC_OPTIONS] [COMMAND_OPTIONS] 参考： File System Shell Guide.

口令命令   ：hdfs fetchdt [GENERIC_OPTIONS] [--webservice <namenode_http_addr>] <path>

文件检查命令：hdfs fsck [GENERIC_OPTIONS] <path> [-list-corruptfileblocks | [-move | -delete | -openforwrite] [-files [-

            locks   -locations | -racks]]]] [-includeSnapshots]

版本信息   :hdfs version

 

4.管理命令

dfsadmi

平衡        ：hdfs balancer [-threshold <threshold>] [-policy <policy>]

启动回滚撤销datanode ： hdfs datanode [-regular | -rollback | -rollingupgrace rollback]

dfsadmin管理：hdfs dfsadmin [GENERIC_OPTIONS]

mover数据迁移工具   :hdfs mover [-p <files/dirs> | -f <local file name>] 类似于balancer，会检查块是否满足存储要求

namenode管理：

Usage: hdfs namenode [-backup] |
          [-checkpoint] |
          [-format [-clusterid cid ] [-force] [-nonInteractive] ] |
          [-upgrade [-clusterid cid] [-renameReserved<k-v pairs>] ] |
          [-upgradeOnly [-clusterid cid] [-renameReserved<k-v pairs>] ] |
          [-rollback] |
          [-rollingUpgrade <downgrade|rollback> ] |
          [-finalize] |
          [-importCheckpoint] |
          [-initializeSharedEdits] |
          [-bootstrapStandby] |
          [-recover [-force] ] |
          [-metadataVersion ]

 

secondary namenode:

hdfs secondarynamenode [-checkpoint [force]] | [-format] | [-geteditsize]

COMMAND_OPTION	Description
-checkpoint [force]	Checkpoints the SecondaryNameNode if EditLog size >= fs.checkpoint.size. If force is used, checkpoint irrespective of EditLog size.
-format	Format the local storage during startup.
-geteditsize	Prints the number of uncheckpointed transactions on the NameNode.

 

 

注意：

sbin/start-dfs.sh实际上就是启动了namenode datanode secondarynamenode zooker等

  --script "$bin/hdfs" start namenode $nameStartOpt

  --script "$bin/hdfs" start datanode $dataStartOpt

  --script "$bin/hdfs" start secondarynamenode

  --script "$bin/hdfs" start zkfc

通过SSH连接到nanenode上启动

5.存储策略

hdfs storagepolicies 列出策略

hdfs dfsadmin -setStoragePolicy <path> <policyName> 设置策略

hdfs dfsadmin -getStoragePolicy <path> 获取策略

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

<wiz_tmp_tag id="wiz-table-range-border" contenteditable="false" style="display: none;">

 

 

 

 

来自为知笔记(Wiz)