Hadoop Commands Cheat Sheet

Published on March 3, 2022 by Kevin

This cheat sheet outlines some of the main Hadoop commands that we’ve found useful.

Generic

  • hadoop fs -ls <path> list files in the path of the file system
  • hadoop fs -chmod <arg> <file-or-dir> alters the permissions of a file where <arg> is the binary argument e.g. 777
  • hadoop fs -chown <owner>:<group> <file-or-dir> change the owner of a file
  • hadoop fs -mkdir <path> make a directory on the file system
  • hadoop fs -put <local-origin> <destination> copy a file from the local storage onto file system
  • hadoop fs -get <origin> <local-destination> copy a file to the local storage from the file system
  • hadoop fs -copyFromLocal <local-origin> <destination> similar to the put command but the source is restricted to a local file reference
  • hadoop fs -copyToLocal <origin> <local-destination> similar to the get command but the destination is restricted to a local file reference
  • hadoop fs -touchz create an empty file on the file system
  • hadoop fs -cat <file> copy files to stdout

Yarn commands

  • yarn node -list list nodes in the yarn cluster
  • yarn node -status <node id> status of a node (memory used, free, number of containers, etc) for <node id> (first column from command above)
  • yarn application -list list of Yarn applications and their state
  • yarn logs -applicationId <appid> dump the logs for a particular application

Configuration commands

  • hdfs getconf return various configuration settings in effect
  • hdfs getconf -namenodes namenodes in the cluster
  • hdfs getconf -confkey <a.value> return the value of a particular setting (e.g. dfs.replication)

HDFS commands

  • hdfs dfsadmin -safemode get find out if you’re in safemode
  • hdfs dfsadmin -report find out how much disk space us used, free, under-replicated, etc.

Want to speed up your web site?

WP Bolt makes it easy and affordable to host your sites using popular Cloud providers.

View Pricing & Sign Up Now