二:HDFS 命令指南

Posted 月饼馅饺子

tags:

篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了二:HDFS 命令指南相关的知识,希望对你有一定的参考价值。

命令具体选项请参考:
 

1.HDFS命令通过bin/hdfs执行,语法如下:

 hdfs [--config confdir] [COMMAND] [GENERIC_OPTIONS] [COMMAND_OPTIONS]
        配置文件          用户命令   基本选项           命令选项
COMMAND_OPTIONDescription
--config confdir Overwrites the default Configuration directory. Default is $HADOOP_HOME/conf.
GENERIC_OPTIONS The common set of options supported by multiple commands. Full list is here.
COMMAND_OPTIONS Various commands with their options are described in the following sections. The commands have been grouped into and .
 
2.GENREIC_OPTIONS基本命令如下:
GENERIC_OPTIONDescription
-conf <configuration file> Specify an application configuration file.  指定配置文件
-D <property>=<value> Use value for given property.                   指定属性
-jt <local> or <resourcemanager:port> Specify a ResourceManager. Applies only to job.   指定资源管理器
-files <comma separated list of files> Specify comma separated files to be copied to the map reduce cluster. Applies only to job.
-libjars <comma seperated list of jars> Specify comma separated jar files to include in the classpath. Applies only to job.
-archives <comma separated list of archives> Specify comma separated archives to be unarchived on the compute machines. Applies only to job.
 
3.用户命令
文件系统命令:hdfs dfs [GENERIC_OPTIONS] [COMMAND_OPTIONS] 参考: File System Shell Guide.
口令命令   :hdfs fetchdt [GENERIC_OPTIONS] [--webservice <namenode_http_addr>] <path>
文件检查命令:hdfs fsck [GENERIC_OPTIONS] <path> [-list-corruptfileblocks | [-move | -delete | -openforwrite] [-files [-
            locks   -locations | -racks]]]] [-includeSnapshots]
版本信息   :hdfs version
 
4.管理命令
dfsadmi
平衡        :hdfs balancer [-threshold <threshold>] [-policy <policy>]
启动回滚撤销datanode : hdfs datanode [-regular | -rollback | -rollingupgrace rollback]
dfsadmin管理:hdfs dfsadmin [GENERIC_OPTIONS]
mover数据迁移工具   :hdfs mover [-p <files/dirs> | -f <local file name>] 类似于balancer,会检查块是否满足存储要求
namenode管理:
Usage: hdfs namenode [-backup] |
          [-checkpoint] |
          [-format [-clusterid cid ] [-force] [-nonInteractive] ] |
          [-upgrade [-clusterid cid] [-renameReserved<k-v pairs>] ] |
          [-upgradeOnly [-clusterid cid] [-renameReserved<k-v pairs>] ] |
          [-rollback] |
          [-rollingUpgrade <downgrade|rollback> ] |
          [-finalize] |
          [-importCheckpoint] |
          [-initializeSharedEdits] |
          [-bootstrapStandby] |
          [-recover [-force] ] |
          [-metadataVersion ]
 
secondary namenode:

hdfs secondarynamenode [-checkpoint [force]] | [-format] | [-geteditsize]

COMMAND_OPTIONDescription
-checkpoint [force] Checkpoints the SecondaryNameNode if EditLog size >= fs.checkpoint.size. If force is used, checkpoint irrespective of EditLog size.
-format Format the local storage during startup.
-geteditsize Prints the number of uncheckpointed transactions on the NameNode.
 
 
注意:
sbin/start-dfs.sh实际上就是启动了namenode datanode secondarynamenode zooker等
  --script "$bin/hdfs" start namenode $nameStartOpt
  --script "$bin/hdfs" start datanode $dataStartOpt
  --script "$bin/hdfs" start secondarynamenode
  --script "$bin/hdfs" start zkfc
通过SSH连接到nanenode上启动
5.存储策略
hdfs storagepolicies 列出策略
hdfs dfsadmin -setStoragePolicy <path> <policyName> 设置策略
hdfs dfsadmin -getStoragePolicy <path> 获取策略
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

<wiz_tmp_tag id="wiz-table-range-border" contenteditable="false" style="display: none;">





以上是关于二:HDFS 命令指南的主要内容,如果未能解决你的问题,请参考以下文章

markdown 打字稿...编码说明,提示,作弊,指南,代码片段和教程文章

参会指南HDFS 2019湖南口腔展最全观展攻略!

玩转HDFS常用命令

实验二 HDFS的Shell命令操作,和HDFS的API操作

HDFS问题集,使用命令报错:com.google.protobuf.ServiceException:java.lang.OutOfMemoryError:java heap space(示例(代码

Vue3官网-高级指南(十七)响应式计算`computed`和侦听`watchEffect`(onTrackonTriggeronInvalidate副作用的刷新时机`watch` pre)(代码片段