hadoop生产环境中jps的使用
Posted
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了hadoop生产环境中jps的使用相关的知识，希望对你有一定的参考价值。
参考技术A jps是jdk提供的一个查看当前java进程的小工具， 可以看做是JavaVirtual Machine Process Status Tool的缩写。非常简单实用。hadoop用户启动hadoop的hdfs，然后用root用户执行kill -9 $(pgrep -f hadoop)，然后用jps分别在root和hadoop用户查看Java进程信息：
再次用hadoop用户启动hdfs，然后用jps分别查看信息：
从以上结果能够看出jps得出的状态信息并不能完全准确的判断一个进程的真实状态
当jps发现获取进程信息有异常时，第一步应该用ps -ef|grep pid，确定进程是否真的存在，如果存在则说明进程是活的，不可轻易kill，如果不存在那说明该进程已经死掉了。第二步，确定进程已死之后，我们需要确定jps异常信息产生的用户，执行ll /tmp/hsperfdata*，会显示所有用户（目录命名格式为hsperfdata_xx用户）进程信息文件（这些文件以进程号命名，直接对照进程号即可确定用户）。第三步，我们需要到/tmp目录执行rm -rf hsperfdata_xx用户清除部分内容再重启hdfs。第四部，重启hdfs之后，jps查看进程信息。
Hadoop.2.x_无秘钥设置
1、在实际生产环境中为Hadoop配置无秘钥登录非常有必要
# 在没有配置时：
[[email protected] hadoop-2.5.0]$ jps 26163 Jps [[email protected] hadoop-2.5.0]$ sbin/start-yarn.sh starting yarn daemons starting resourcemanager, logging to /home/liuwl/opt/modules/hadoop-2.5.0/logs/yarn-liuwl-resourcemanager-linux-66-64.liuwl.com.out [email protected]‘s password: # 启动 nodemanager 需要密码 linux-66-64.liuwl.com: starting nodemanager, logging to /home/liuwl/opt/modules/hadoop-2.5.0/logs/yarn-liuwl-nodemanager-linux-66-64.liuwl.com.out [[email protected] hadoop-2.5.0]$ jps 27134 ResourceManager 27475 Jps 27417 NodeManager [[email protected] hadoop-2.5.0]$ sbin/stop-yarn.sh stopping yarn daemons stopping resourcemanager [email protected]‘s password: # 关闭 nodemanager 需要密码 linux-66-64.liuwl.com: stopping nodemanager no proxyserver to stop
# 可想,实际生产中无秘钥登录的重要性
2、生成公钥与私钥
做好无秘钥登录用户选择(在每个用户下都会有个二.ssh文件夹,默认为空)
[[email protected] .ssh]$ ssh-keygen -t rsa Generating public/private rsa key pair. Enter file in which to save the key (/home/liuwl/.ssh/id_rsa): Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /home/liuwl/.ssh/id_rsa. Your public key has been saved in /home/liuwl/.ssh/id_rsa.pub. The key fingerprint is: 12:26:91:a1:da:29:91:a7:ec:3e:1a:f6:92:f5:7d:cc [email protected] The key‘s randomart image is: +--[ RSA 2048]----+ | oo | | . ... | |o o . o | |.* . o . | |+.o . S | |.. . . | |.oo . . o | |o=. . . E | |o.o. . | +-----------------+ [[email protected] .ssh]$ ls id_rsa id_rsa.pub # 私钥与公钥
3、发送到节点(机器)上
[[email protected] .ssh]$ hostname linux-66-64.liuwl.com [[email protected] .ssh]$ ssh-copy-id linux-66-64.liuwl.com The authenticity of host ‘linux-66-64.liuwl.com (192.168.0.111)‘ can‘t be established. # 第一次设置需要提示是否确定 RSA key fingerprint is 93:26:f9:f8:57:d1:0d:cb:5f:69:87:26:f6:77:f8:e0. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added ‘linux-66-64.liuwl.com,192.168.0.111‘ (RSA) to the list of known hosts. [email protected]‘s password: # 输入密码 Now try logging into the machine, with "ssh ‘linux-66-64.liuwl.com‘", and check in: .ssh/authorized_keys to make sure we haven‘t added extra keys that you weren‘t expecting. [[email protected] .ssh]$ ls authorized_keys id_rsa id_rsa.pub known_hosts # authorized_keys 也是公钥,注意秘钥权限不能随便乱改
[[email protected] .ssh]$ ll total 16 -rw------- 1 liuwl liuwl 409 Oct 14 00:43 authorized_keys -rw------- 1 liuwl liuwl 1675 Oct 14 00:40 id_rsa -rw-r--r-- 1 liuwl liuwl 409 Oct 14 00:40 id_rsa.pub -rw-r--r-- 1 liuwl liuwl 417 Oct 14 00:43 known_hosts [[email protected] .ssh]$ cat authorized_keys ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5d/DiNJ8BRdsoTlfZqMMY+RUWeUel9oOfSGg/EdVifgryHHpUGc/PziuDZAZb0R/zV9HebYgIZ8YDsJvUOZleUP6n2p+8kRqJGsKCru2lM8lVPtkICZq+KUHHzMvavgwa2K6r9kITc1LMr8plBzdfPkN4uygYK6otLdlAn9qAGAv47FTGcbMJuUhB4e3MKqVg5z5ZFI3FRE3MmLoazVLQ3kCOHlSzUdmJ9ZyAQRdSP9OTSbtL81KSxFIJbvgZ2qoTDL26mjg4rTntvwyXoxPWQ/9twWbJ9Sov1k1SZVHfZ8XnHFsCTF0YzFrxD/UZQUpWnR5/T+/cR2mtU/n08+rtQ== [email protected] [[email protected] .ssh]$ cat known_hosts linux-66-64.liuwl.com,192.168.0.111 ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAuJR7MP7Bo21+OGDD2GKGYuyJoJKlDOfrIWQwClCsEn2tdgRW0IFAQkIgo4bcaRTUxJ8ctJom3Qi+NykIe1lqMA2ONTcclNKh/a8teXvd/DOhnol+OXJypL/S2vSRviR/QOerq7KWXiixzAjxBSTQtvYvMBY/fGfHu6wXy/DVViaTu0MdsWLAQlpwkfRe+D0rlDJIQqVTemdX8zxqPsEednlDuk7YlntC1e/A3bds2r2hGLX6PV8M9g5ef3dmw3s5LZD89Bh/8LlzEr42zEGApgKICrn+Wm8FCbrG2BrUOOz3srf4Y2aaNicZ4sm0X8FepPDYUWKW7lFxLv4YLRcG1w== [[email protected] .ssh]$ cat id_rsa -----BEGIN RSA PRIVATE KEY----- MIIEogIBAAKCAQEA5d/DiNJ8BRdsoTlfZqMMY+RUWeUel9oOfSGg/EdVifgryHHp UGc/PziuDZAZb0R/zV9HebYgIZ8YDsJvUOZleUP6n2p+8kRqJGsKCru2lM8lVPtk ICZq+KUHHzMvavgwa2K6r9kITc1LMr8plBzdfPkN4uygYK6otLdlAn9qAGAv47FT GcbMJuUhB4e3MKqVg5z5ZFI3FRE3MmLoazVLQ3kCOHlSzUdmJ9ZyAQRdSP9OTSbt L81KSxFIJbvgZ2qoTDL26mjg4rTntvwyXoxPWQ/9twWbJ9Sov1k1SZVHfZ8XnHFs CTF0YzFrxD/UZQUpWnR5/T+/cR2mtU/n08+rtQIBIwKCAQATtBgTCrosUncGgUKx BqlKZAc658DLL/KdAuHpvPi0DfUfzz/iUf4bXKE0XM766J09fTIDHj1Etd19m6Mk MQFiKmXwaDbETwHIm3XjqbCQaYbbZgFElZQj8ONachoB2sJSWOtuKIuu5bX9CRIx RE1/vZN52cSakqC/CGe+YrFQfRxVPmFaK+xd2ao8rUCAY4TIiWtN+n3UOPeJP/C7 GqlvGN6Bi6in69qdz07MRuHylUHnR91UGH3loJom2EjZzYMPNXsWikn+utWemhnd cXpOpuVDzgco7arb2m1TKzGmA3JAg0F/ttP+2iRIbCsmuBlOlUF817gDxIDzKJOZ GFKrAoGBAPkHzjR3aKlHBprlFfHJa6YzFh8ZU/jMfX22FGkwqT7mxgSL/HHDl8RZ yrZs2vaFFDRNzjnH8qaKkcfQkfvzUReZVNMDh804iJ6jjLwCNdl0lbC8kpiUVHeV VErIpKEObgVXrnge6b0cYzq+bn+TYN7oSbGUHwYZ5E/Zb9dNp0z5AoGBAOxOtgQx CF1DncQJNyo4CzeCQ6ESk9ecpcbludwI/6j3O4RvyjFimqtHmSRE7nALHcPKZ509 Z5Ge2HnlTpKHEUJWw4ff6Qm4/eIuaZ3vf+L5ido4ugRgq2fPu0RhTg3eXjKpck0Y SA3+3TRjjggN6yMyssxjTAErRMvHBqvhYe+dAoGBAKOmCynKz7+yVMwwJF0PVV6W mYImlkRpH0P7Tz3PhSlV0pVGDErCXGsWb0St7vnFKojFakM6QF7QFqfg1PYGSztk wraNSpzNYRfKj66/n7rI9KdXWQUuRiKylnpJVj3ziiDElzj+XxXfdGhugyCirV9I MG1hVjc1lgiWM418LByjAoGAPMPIZ3pSm6OsObk6EiurHOb7co+/ndfS21+k0i4z HNHcGr2pBWJ/iyEKH0Ts2vuLT5pyakL9X/WlYSxWCGvf3dvL4RxR3e29oI+XgGIo Mwz3j+KzfXfxj7kS5bKfCuForfEOw14D5ld6tbMzJqR+SuEmqZXgWBJwxq+FX2XX Wt8CgYEAgEVzR3Zt3+AcvwGGKoz33JnyD6+DiFDS8jJFyVg0SDVWn+pfiWJ1Qrgs JbrYrBoTxncNTZgyVzfMqaptLEpOkldWOfcnqn1WaWSfw06x5fCIwpnee3qZtPf6 h05Q81dKC2Wx4BP+gUdPQgINRt9Eln8m3cDVZROZKrQes7IKuvU= -----END RSA PRIVATE KEY----- [[email protected] .ssh]$ cat id_rsa.pub ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5d/DiNJ8BRdsoTlfZqMMY+RUWeUel9oOfSGg/EdVifgryHHpUGc/PziuDZAZb0R/zV9HebYgIZ8YDsJvUOZleUP6n2p+8kRqJGsKCru2lM8lVPtkICZq+KUHHzMvavgwa2K6r9kITc1LMr8plBzdfPkN4uygYK6otLdlAn9qAGAv47FTGcbMJuUhB4e3MKqVg5z5ZFI3FRE3MmLoazVLQ3kCOHlSzUdmJ9ZyAQRdSP9OTSbtL81KSxFIJbvgZ2qoTDL26mjg4rTntvwyXoxPWQ/9twWbJ9Sov1k1SZVHfZ8XnHFsCTF0YzFrxD/UZQUpWnR5/T+/cR2mtU/n08+rtQ== [email protected]
4、测试一下
[[email protected] hadoop-2.5.0]$ sbin/start-yarn.sh starting yarn daemons starting resourcemanager, logging to /home/liuwl/opt/modules/hadoop-2.5.0/logs/yarn-liuwl-resourcemanager-linux-66-64.liuwl.com.out linux-66-64.liuwl.com: starting nodemanager, logging to /home/liuwl/opt/modules/hadoop-2.5.0/logs/yarn-liuwl-nodemanager-linux-66-64.liuwl.com.out [[email protected] hadoop-2.5.0]$ jps 25171 NodeManager 25072 ResourceManager 25331 Jps
OK！
以上是关于hadoop生产环境中jps的使用的主要内容，如果未能解决你的问题，请参考以下文章