This cheat sheet outlines some of the main Hadoop commands that we’ve found useful while building our Cloudways alternative hosting service.

Generic

• hadoop fs -ls <path> list files in the path of the file system
• hadoop fs -chmod <arg> <file-or-dir> alters the permissions of a file where <arg> is the binary argument e.g. 777
• hadoop fs -chown <owner>:<group> <file-or-dir> change the owner of a file
• hadoop fs -mkdir <path> make a directory on the file system
• hadoop fs -put <local-origin> <destination> copy a file from the local storage onto file system
• hadoop fs -get <origin> <local-destination> copy a file to the local storage from the file system
• hadoop fs -copyFromLocal <local-origin> <destination> similar to the put command but the source is restricted to a local file reference
• hadoop fs -copyToLocal <origin> <local-destination> similar to the get command but the destination is restricted to a local file reference
• hadoop fs -touchz create an empty file on the file system
• hadoop fs -cat <file> copy files to stdout

Yarn commands

• yarn node -list list nodes in the yarn cluster
• yarn node -status <node id> status of a node (memory used, free, number of containers, etc) for <node id> (first column from command above)
• yarn application -list list of Yarn applications and their state
• yarn logs -applicationId <appid> dump the logs for a particular application

Configuration commands

• hdfs getconf return various configuration settings in effect
• hdfs getconf -namenodes namenodes in the cluster
• hdfs getconf -confkey <a.value> return the value of a particular setting (e.g. dfs.replication)

HDFS commands

• hdfs dfsadmin -safemode get find out if you’re in safemode
• hdfs dfsadmin -report find out how much disk space us used, free, under-replicated, etc.