2023年7月17日发(作者：)

Hadoop3.0.3HDFS常⽤shell命令1、启动Hadoop./[root@elk-server sbin]# ./ting namenodes on [elk-server]上⼀次登录：⽇ 11⽉ 24 21:57:43 CST 2019从 192.168.88.1pts/0 上elk-server: Warning: Permanently added 'elk-server,192.168.88.135' (ECDSA) to the list of known ng datanodes上⼀次登录：⽇ 11⽉ 24 22:44:47 CST 2019pts/0 上node1: WARNING: /usr/local/hadoop/hadoop-3.0.3/logs does not exist. ng secondary namenodes [elk-server]上⼀次登录：⽇ 11⽉ 24 22:44:50 CST 2019pts/0 上Starting resourcemanager上⼀次登录：⽇ 11⽉ 24 22:44:55 CST 2019pts/0 上Starting nodemanagers上⼀次登录：⽇ 11⽉ 24 22:45:02 CST 2019pts/0 上2、检查Hadoop服务启动情况jps[root@elk-server sbin]# jps7713 NameNode8516 Jps8201 ResourceManager7965 SecondaryNameNode3、检查HDFS 系统根⽬录hadoop fs -ls /[root@elk-server sbin]# hadoop fs -ls /4、HDFS创建⽂件夹hadoop fs -mkdir /test[root@elk-server sbin]# hadoop fs -mkdir /test[root@elk-server sbin]# hadoop fs -ls /Found 1 itemsdrwxr-xr-x - root supergroup 0 2019-11-25 00:15 /test5、HDFS 创建级联⽂件夹hadoop fs -mkdir -p /input/data[root@elk-server sbin]# hadoop fs -mkdir -p /input/data[root@elk-server sbin]# hadoop fs -ls /Found 2 itemsdrwxr-xr-x - root supergroup 0 2019-11-25 00:18 /inputdrwxr-xr-x - root supergroup 0 2019-11-25 00:15 /test6、HDFS查看系统根⽬录下指定⽂件夹包括的⽂件hadoop fs -ls -R /input[root@elk-server sbin]# hadoop fs -ls -R /inputdrwxr-xr-x - root supergroup 0 2019-11-25 00:18 /input/data7、HDFS上传⽂件hadoop fs -put /input/data[root@elk-server data]# hadoop fs -put /input/data[root@elk-server data]# hadoop fs -ls -R /input/data-rw-r--r-- 1 root supergroup 38 2019-11-25 00:24 /input/data/8、HDFS 下载⽂件hadoop fs -get /input/data/ /usr/local/data/[root@elk-server data]# hadoop fs -get /input/data/ /usr/local/data/[root@elk-server data]# ll总⽤量 8-rw-r--r-- 1 root root 38 11⽉ 25 00:27 -rw-r--r-- 1 root root 38 11⽉ 25 00:23 9、HDFS ⽂件复制hadoop fs -cp /input/data/ /test[root@elk-server data]# hadoop fs -cp /input/data/ /test[root@elk-server data]# hadoop fs -ls /testFound 1 items-rw-r--r-- 1 root supergroup 38 2019-11-25 00:30 /test/10、HDFS ⽂件移动hadoop fs -mv /test/ /input[root@elk-server data]# hadoop fs -mv /test/ /input[root@elk-server data]# hadoop fs -ls /inputFound 2 itemsdrwxr-xr-x - root supergroup 0 2019-11-25 00:24 /input/data-rw-r--r-- 1 root supergroup 38 2019-11-25 00:30 /input/11、HDFS 删除⽂件或⽂件夹hadoop fs -rm /input/[root@elk-server data]# hadoop fs -rm /input/eted /input/[root@elk-server data]# hadoop fs -ls /inputFound 1 itemsdrwxr-xr-x - root supergroup 0 2019-11-25 00:24 /input/data删除空⽬录hadoop fs -rmdir /test[root@elk-server data]# hadoop fs -rmdir /test[root@elk-server data]# hadoop fs -ls /Found 1 itemsdrwxr-xr-x - root supergroup 0 2019-11-25 00:37 /input强制删除hadoop fs -rm -r /input[root@elk-server data]# hadoop fs -rm -r /inputDeleted /input[root@elk-server data]# hadoop fs -ls /12、HDFS 查看⽂件内容hadoop fs -cat /input/data/[root@elk-server data]# hadoop fs -mkdir -p /input/data[root@elk-server data]# hadoop fs -put /input/data[root@elk-server data]# hadoop fs -ls /Found 1 itemsdrwxr-xr-x - root supergroup 0 2019-11-25 00:42 /input[root@elk-server data]# hadoop fs -cat /input/data/l zhangsanhello lisihello wangwu[root@elk-server data]#

13、HDFS 修改⽂件权限，参考Linux 指令之chmod14、HDFS 修改⽂件所属组，参考Linux指令之chgrp15、HDFS 修改为⽂件所属⽤户，参考Linux指令之chown16、HDFS 显⽰⽂件相关属性值，参考Linux 指令之du[root@elk-server data]# hadoop fs -du /input/data/38 38 /input/data/17、HDFS 显⽰⽂件⼤学，参考Linux 指令之dus[root@elk-server data]# hadoop fs -dus /input/data/: DEPRECATED: Please use 'du -s' instead.38 38 /input/data/18、HDFS 清空回收站,参考Linux 指令之expunge[root@elk-server data]# hadoop fs -expunge #清空回收站19、setrep使⽤⽅法：hadoop fs -setrep [-R] 改变⼀个⽂件的副本系数。-R选项⽤于递归改变⽬录下所有⽂件的副本系数。⽰例：hadoop fs -setrep -w 3 -R /user/hadoop/dir1返回值：成功返回0，失败返回-1。

20、tail使⽤⽅法：hadoop fs -tail [-f] URI将⽂件尾部1K字节的内容输出到stdout。⽀持-f选项，⾏为和Unix中⼀致。⽰例：hadoop fs -tail pathname返回值：成功返回0，失败返回-1。

21、test使⽤⽅法：hadoop fs -test -[ezd] URI选项：-e 检查⽂件是否存在。如果存在则返回0。-z 检查⽂件是否是0字节。如果是则返回0。

-d 如果路径是个⽬录，则返回1，否则返回0。⽰例：hadoop fs -test -e filename22、查看Hadoop集群状态hdfs dfsadmin -report[root@elk-server data]# hdfs dfsadmin -reportConfigured Capacity: (16.99 GB)Present Capacity: (12.36 GB)DFS Remaining: (12.36 GB)DFS Used: 16384 (16 KB)DFS Used%: 0.00%Replicated Blocks: Under replicated blocks: 0 Blocks with corrupt replicas: 0 Missing blocks: 0 Missing blocks (with replication factor 1): 0 Pending deletion blocks: 0Erasure Coded Block Groups:

Low redundancy block groups: 0 Block groups with corrupt internal blocks: 0 Missing block groups: 0 Pending deletion blocks: 0-------------------------------------------------Live datanodes (1):Name: 192.168.88.136:9866 (node1)Hostname: node1Decommission Status : NormalConfigured Capacity: (16.99 GB)DFS Used: 16384 (16 KB)Non DFS Used: 4969267200 (4.63 GB)DFS Remaining: (12.36 GB)DFS Used%: 0.00%DFS Remaining%: 72.75%Configured Cache Capacity: 0 (0 B)Cache Used: 0 (0 B)Cache Remaining: 0 (0 B)Cache Used%: 100.00%Cache Remaining%: 0.00%Xceivers: 1Last contact: Mon Nov 25 00:53:27 CST 2019Last Block Report: Sun Nov 24 22:44:58 CST 2019