Print hadoop properties in CLI Code Answer

Hello Developer, Hope you guys are doing great. Today at Tutorial Guruji Official website, we are sharing the answer of Print hadoop properties in CLI without wasting too much if your time.

The question is published on by Tutorial Guruji team.

Am writing a utility program with Hadoop using java and I need to extract all the hadoop properties in a single command as like set -v in hive through the Jsch.

I tried this hadoop org.apache.hadoop.conf.Configuration but it returns only the core-site properties. Is there any other commands available to print all the properties like set -v in hive

Answer

One possible solution could be:

package config;

import java.util.Map.Entry;
import org.apache.hadoop.conf.*;
import org.apache.hadoop.util.*;


public class HadoopPropDump extends Configured implements Tool {

  static {

    //Hadoop by default specifies two resources, loaded in-order from the classpath:  
    // 1. core-default.xml
    // 2. core-site.xml

    Configuration.addDefaultResource("hdfs-default.xml");
    Configuration.addDefaultResource("hdfs-site.xml");
    Configuration.addDefaultResource("mapred-default.xml");
    Configuration.addDefaultResource("mapred-site.xml");
    Configuration.addDefaultResource("yarn-default.xml");
    Configuration.addDefaultResource("yarn-site.xml");
  }

  @Override
  public int run(String[] args) throws Exception {
    Configuration conf = getConf();
    for (Entry<String, String> entry: conf) {
      System.out.printf("%s=%sn", entry.getKey(), entry.getValue());
    }
    return 0;
  }

  public static void main(String[] args) throws Exception {
    int exitCode = ToolRunner.run(new HadoopPropDump(), args);
    System.exit(exitCode);
  }
}

The static block makes sure that the HDFS, YARN, and MapReduce configurations are picked up, in addition to the core ones (which Configuration knows about already).”

Reference: O’Reilly’s “Hadoop: The Definitive Guide” by Tom White

We are here to answer your question about Print hadoop properties in CLI - If you find the proper solution, please don't forgot to share this with your team members.

Related Posts

Tutorial Guruji