Hadoop Commands Cheat Sheet

Published on March 3, 2022 by Kevin Graham

This cheat sheet outlines some of the main Hadoop commands that we’ve found useful while building our Cloudways alternative hosting service.

Generic

  • hadoop fs -ls <path> list files in the path of the file system
  • hadoop fs -chmod <arg> <file-or-dir> alters the permissions of a file where <arg> is the binary argument e.g. 777
  • hadoop fs -chown <owner>:<group> <file-or-dir> change the owner of a file
  • hadoop fs -mkdir <path> make a directory on the file system
  • hadoop fs -put <local-origin> <destination> copy a file from the local storage onto file system
  • hadoop fs -get <origin> <local-destination> copy a file to the local storage from the file system
  • hadoop fs -copyFromLocal <local-origin> <destination> similar to the put command but the source is restricted to a local file reference
  • hadoop fs -copyToLocal <origin> <local-destination> similar to the get command but the destination is restricted to a local file reference
  • hadoop fs -touchz create an empty file on the file system
  • hadoop fs -cat <file> copy files to stdout

Yarn commands

  • yarn node -list list nodes in the yarn cluster
  • yarn node -status <node id> status of a node (memory used, free, number of containers, etc) for <node id> (first column from command above)
  • yarn application -list list of Yarn applications and their state
  • yarn logs -applicationId <appid> dump the logs for a particular application

Configuration commands

  • hdfs getconf return various configuration settings in effect
  • hdfs getconf -namenodes namenodes in the cluster
  • hdfs getconf -confkey <a.value> return the value of a particular setting (e.g. dfs.replication)

HDFS commands

  • hdfs dfsadmin -safemode get find out if you’re in safemode
  • hdfs dfsadmin -report find out how much disk space us used, free, under-replicated, etc.

Want to speed up your web site?

WP Bolt makes it easy and affordable to host your sites using popular Cloud providers.

View Pricing & Sign Up Now