如何清理hadoop过期历史数据

Posted

tags:

篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了如何清理hadoop过期历史数据相关的知识,希望对你有一定的参考价值。

  hadoop 添加删除datanode及tasktracker

  首先:

  建议datanode和tasktracker分开写独立的exclude文件,因为一个节点即可以同时是datanode和tasktracker,也可以单独是datanode或tasktracker。

  1、删除datanode

  修改namenode上的hdfs-site.xml

  dfs.hosts

  /usr/local/hadoop/conf/datanode-allow-list

  dfs.hosts.exclude

  /usr/local/hadoop/conf/datanode-deny-list

  其中dfs.host列出了连入namenode的节点,如果为空,则所有的datanode都可以连入namenode。如果不为空,则文件中存在的datanode可以连入。

  dfs.hosts.exclude列出了禁止连入namenode的节点。

  如果一个节点同时存在于dfs.hosts和dfs.hosts.exclude,则禁止连入。

  具体步骤

  (1)将要删除的datanode加入到dfs.hosts.exclude指定的文件中。(最好使用主机名,IP有时可能不生效)

  (2)动态刷新配置,不需要重启namenode

  hadoop dfsadmin -refreshNodes

  (3)通过hadoop dfsadmin -report或webui,可以看到,该datanode的状态为Decommissioning

  (4)等待一段时间,该节点为dead状态。

  (5)删除dfs.hosts文件中该节点 即下架目标机器后,再次编辑dfs.hosts.exclude,把刚才下架的机器的ip或机器名移走

  (6)

  hadoop dfsadmin -refreshNodes

  注:按照上面的操作后,如果你想要重新启用该节点为datanode,从dfs.hosts.exclude中删除该节点,refreshNodes,然后,到该节点上,重启启动该datanode:

  /usr/local/hadoop/bin/hadoop-daemon.sh stop datanode

  /usr/local/hadoop/bin/hadoop-daemon.sh start datanode

  注:正确的删除datanode的方法应该是按照上面的方法,使用exclude文件,而不应该直接去datanode上去sotp
datanode,这样会造出数据丢失,而且stop以后,webui和hadoop dfsadmin
-report都仍然显示该datanode节点。除非重新启动namenode。

  之所以重新启用exclude的节点时可以stop
datanode,因为此时该datanode不包含在cluster中,所以,stop后不会造成数据丢失。

  2、添加datanode

  如果存在dfs.hosts文件,且不为空,则添加新的datanode到该文件中,refreshNodes。

  到新节点上,启动即可

  /usr/local/hadoop/bin/hadoop-daemon.sh start datanode

  如果不存在dfs.hosts文件,或文件为空,直接启动新节点即可。

  3、删除tasktracker

  原理和步骤与删除datanode一样。

  mapred.hosts

  /usr/local/hadoop/conf/tasktracker-allow-list

  mapred.hosts.exclude

  /usr/local/hadoop/conf/tasktracker-deny-list

  动态刷新配置的命令为:

  hadoop mradmin -refreshNodes

  立刻生效,可以在webui中看到,nodes节点数量的变化,以及Excluded Nodes节点的变化。

  具体的步骤参考上面的删除datanode的步骤

  注:按照上面的操作后,如果你想要重新启用该节点为tasktracker,从mapred.hosts.exclude中删除该节点,refreshNodes,然后,到该节点上,重启启动该tasktracker:

  /usr/local/hadoop/bin/hadoop-daemon.sh stop tasktracker

  /usr/local/hadoop/bin/hadoop-daemon.sh start tasktracker

  注:正确的删除tasktracker的方法应该是按照上面的方法,使用exclude文件,而不应该直接去tasktracker上去sotp
tasktracker,这样会造成job失败,而且stop以后,webui上仍然显示该tasktracker节点。除非重新启动jobtracker。

  我遇到的一个问题:

  在exclude文件中,我使用了IP,发现tasktracker仍然参与计算。

  在webui中发现,Excluded Nodes中存在该IP,Nodes中仍然存在该tasktracker的hostname。

  解决的办法就是,在exclude中使用hostname而不使用IP。

  判断是否真正生效:如果exclued文件中正确的填写了要失效的node,则总得nodes数量应该减小。

  4、添加tasktracker

  如果存在mapred.hosts文件,且不为空,则添加新的tasktracker到该文件中,refreshNodes。

  到新节点上,启动即可

  /usr/local/hadoop/bin/hadoop-daemon.sh start tasktracker

  如果不存在mapred.hosts文件,或文件为空,直接启动新节点即可。

  6、添加或删除datanode后,平衡磁盘利用率

  运行bin/start-balancer.sh,这个会很耗时间

  备注:

  如果不balance,那么cluster会把新的数据都存放在新的node上,这样会降低mr的工作效率;

  /usr/local/hadoop/bin/start-balancer.sh -threshold 0.1

  7下架目标机器后,再次编辑mapred.hosts.exclude,把刚才下架的机器的ip或机器名移走

  threshold 是平衡阈值,默认是10%,值越低各节点越平衡,但消耗时间也更长。

  balancer也可以在有mr job的cluster上运行,默认dfs.balance.bandwidthPerSec很低,为1M/s。在没有mr
job时,可以提高该设置加快负载均衡时间。

  在namenode的hdfs-site.xml中增加设置balance的带宽,默认只有1M:

  dfs.balance.bandwidthPerSec

  10485760Specifies themaximum bandwidth that each datanode can utilize for
the balancing purpose interm of the number of bytes per second.

  
参考技术A 日志数据的删除直接删除log就好。
datanode上过期数据块的删除建议在配置文件中配置dfs.blockreport.intervalMsec,意思就是datanode多久向namenode发送heartbeat,默认值是3600000ms,你可以改的小点,我初学做实验就改成了60000(每60s报告一次)。祝好!
参考技术B hadoop过期历史数据?指的是任务日志以及任务记录文件吗?直接删除就好了吧

pinpoint体系中,关于如何清理过期hbase数据

版本:

  pinpoint:1.7.1

  hbase:1.2.6

命令行命令:

  $HBASE_HOME/bin/hbase shell  

备注:保留一天半的数据(秒)

TTL => 129600

 

 1 disable AgentInfo
 2 disable AgentStatV2
 3 
 4 disable AgentLifeCycle
 5 disable AgentEvent
 6 disable ApplicationIndex
 7 
 8 disable StringMetaData
 9 disable ApiMetaData
10 
11 disable SqlMetaData_Ver2
12 
13 disable ApplicationTraceIndex
14 disable TraceV2
15 
16 
17 disable ApplicationMapStatisticsCaller_Ver2
18 disable ApplicationMapStatisticsCallee_Ver2
19 disable ApplicationMapStatisticsSelf_Ver2
20 
21 disable HostApplicationMap_Ver2
22 
23 
24 drop AgentInfo
25 drop AgentStatV2
26 drop ApplicationStatAggre
27 drop AgentLifeCycle
28 drop AgentEvent
29 drop ApplicationIndex
30 
31 drop StringMetaData
32 drop ApiMetaData
33 
34 drop SqlMetaData_Ver2
35 
36 drop ApplicationTraceIndex
37 drop TraceV2
38 
39 drop ApplicationMapStatisticsCaller_Ver2
40 drop ApplicationMapStatisticsCallee_Ver2
41 drop ApplicationMapStatisticsSelf_Ver2
42 
43 drop HostApplicationMap_Ver2
44 
45 
46 
47 
48 
49 
50 create AgentInfo, { NAME => Info, TTL => 129600, DATA_BLOCK_ENCODING => PREFIX }
51 create AgentStatV2, { NAME => S, TTL => 129600, DATA_BLOCK_ENCODING => PREFIX }, {SPLITS=>["x01x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x02x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x03x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x04x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x05x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x06x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x07x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x08x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x09x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x0ax00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x0bx00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x0cx00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x0dx00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x0ex00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x0fx00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x10x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x11x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x12x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x13x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x14x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x15x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x16x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x17x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x18x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x19x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x1ax00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x1bx00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x1cx00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x1dx00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x1ex00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x1fx00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x20x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x21x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x22x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x23x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x24x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x25x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x26x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x27x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x28x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x29x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x2ax00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x2bx00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x2cx00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x2dx00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x2ex00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x2fx00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x30x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x31x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x32x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x33x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x34x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x35x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x36x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x37x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x38x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x39x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x3ax00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x3bx00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x3cx00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x3dx00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x3ex00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x3fx00x00x00x00x00x00x00x00x00x00x00x00x00x00x00"]}
52 create ApplicationStatAggre, { NAME => S, TTL => 129600, DATA_BLOCK_ENCODING => PREFIX }, {SPLITS=>["x01x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x02x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x03x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x04x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x05x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x06x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x07x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x08x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x09x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x0ax00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x0bx00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x0cx00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x0dx00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x0ex00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x0fx00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x10x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x11x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x12x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x13x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x14x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x15x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x16x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x17x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x18x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x19x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x1ax00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x1bx00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x1cx00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x1dx00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x1ex00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x1fx00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x20x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x21x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x22x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x23x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x24x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x25x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x26x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x27x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x28x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x29x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x2ax00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x2bx00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x2cx00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x2dx00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x2ex00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x2fx00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x30x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x31x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x32x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x33x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x34x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x35x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x36x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x37x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x38x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x39x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x3ax00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x3bx00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x3cx00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x3dx00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x3ex00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x3fx00x00x00x00x00x00x00x00x00x00x00x00x00x00x00"]}
53 
54 create ApplicationIndex, { NAME => Agents, TTL => 129600, DATA_BLOCK_ENCODING => PREFIX }
55 create AgentLifeCycle, { NAME => S, TTL => 129600, DATA_BLOCK_ENCODING => PREFIX }
56 create AgentEvent, { NAME => E, TTL => 129600, DATA_BLOCK_ENCODING => PREFIX }
57 
58 create StringMetaData, { NAME => Str, TTL => 129600, DATA_BLOCK_ENCODING => PREFIX }, {SPLITS=>["x01x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x02x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x03x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x04x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x05x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x06x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x07x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00"]}
59 create ApiMetaData, { NAME => Api, TTL => 129600, DATA_BLOCK_ENCODING => PREFIX }, {SPLITS=>["x01x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x02x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x03x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x04x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x05x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x06x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x07x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00"]}
60 
61 create SqlMetaData_Ver2, { NAME => Sql, TTL => 129600, DATA_BLOCK_ENCODING => PREFIX }, {SPLITS=>["x02x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x04x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x06x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x08x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x0ax00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x0cx00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x0ex00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x10x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x12x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x14x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x16x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x18x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x1ax00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x1cx00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x1ex00x00x00x00x00x00x00x00x00x00x00x00x00x00x00"]}
62 
63 create TraceV2, { NAME => S, TTL => 129600, DATA_BLOCK_ENCODING => PREFIX }, {NUMREGIONS => 256, SPLITALGO => UniformSplit}
64 
65 create ApplicationTraceIndex, { NAME => I, TTL => 129600, DATA_BLOCK_ENCODING => PREFIX }, {SPLITS=>["x02x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x04x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x06x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x08x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x0ax00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x0cx00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x0ex00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x10x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x12x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x14x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x16x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x18x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x1ax00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x1cx00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x1ex00x00x00x00x00x00x00x00x00x00x00x00x00x00x00"]}
66 
67 create ApplicationMapStatisticsCaller_Ver2, { NAME => C, TTL => 129600, VERSIONS => 1, DATA_BLOCK_ENCODING => PREFIX }, {SPLITS=>["x02x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x04x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x06x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x08x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x0ax00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x0cx00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x0ex00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x10x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x12x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x14x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x16x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x18x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x1ax00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x1cx00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x1ex00x00x00x00x00x00x00x00x00x00x00x00x00x00x00"]}
68 create ApplicationMapStatisticsCallee_Ver2, { NAME => C, TTL => 129600, VERSIONS => 1, DATA_BLOCK_ENCODING => PREFIX }, {SPLITS=>["x02x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x04x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x06x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x08x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x0ax00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x0cx00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x0ex00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x10x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x12x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x14x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x16x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x18x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x1ax00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x1cx00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x1ex00x00x00x00x00x00x00x00x00x00x00x00x00x00x00"]}
69 create ApplicationMapStatisticsSelf_Ver2, { NAME => C, TTL => 129600, VERSIONS => 1, DATA_BLOCK_ENCODING => PREFIX }, {SPLITS=>["x01x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x02x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x03x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x04x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x05x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x06x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x07x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00"]}
70 
71 create HostApplicationMap_Ver2, { NAME => M, TTL => 129600, VERSIONS => 1, DATA_BLOCK_ENCODING => PREFIX }, {SPLITS=>["x01x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x02x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00","x03x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00"]}
72 
73 enable AgentInfo
74 enable AgentStatV2
75 enable ApplicationStatAggre
76 enable AgentLifeCycle
77 enable AgentEvent
78 enable ApplicationIndex
79 
80 enable StringMetaData
81 enable ApiMetaData
82 
83 enable SqlMetaData_Ver2
84 
85 enable ApplicationTraceIndex
86 enable TraceV2
87 
88 enable ApplicationMapStatisticsCaller_Ver2
89 enable ApplicationMapStatisticsCallee_Ver2
90 enable ApplicationMapStatisticsSelf_Ver2
91 
92 enable HostApplicationMap_Ver2
93 
94 list
95 
96 exit

 

以上是关于如何清理hadoop过期历史数据的主要内容,如果未能解决你的问题,请参考以下文章

pinpoint体系中,关于如何清理过期hbase数据

mysql 数据库清理过期数据的时候失败  提示是删除数据量太大! 怎么办?

MySQL数据篇--存储过程实现定时每天清理过期数据

hadoop清理某个分区的数据

Flink从入门到精通100篇(二十三)-Flink实战应用案例:如何清理过期的 Checkpoint 目录?

mysql定时任务event——清理过期数据 (转)