org.apache.hadoop.yarn.conf.ConfigurationProviderFactory分析加载配置文件两种方式

标签：

ConfigurationProviderFactory结构如下：

/**
* Creates an instance of {@link ConfigurationProvider} using given
* configuration.
* @param bootstrapConf
* @return configurationProvider
*/
@SuppressWarnings("unchecked")
public static ConfigurationProvider
    getConfigurationProvider(Configuration bootstrapConf) {
  Class<? extends ConfigurationProvider> defaultProviderClass;
  try {
    defaultProviderClass = (Class<? extends ConfigurationProvider>)
        Class.forName(
           YarnConfiguration.DEFAULT_RM_CONFIGURATION_PROVIDER_CLASS); //org.apache.hadoop.yarn.LocalConfigurationProvider默认使用的类
  } catch (Exception e) {
    throw new YarnRuntimeException(
        "Invalid default configuration provider class"
            + YarnConfiguration.DEFAULT_RM_CONFIGURATION_PROVIDER_CLASS, e);
  }
  ConfigurationProvider configurationProvider =
      ReflectionUtils.newInstance(bootstrapConf.getClass(
          YarnConfiguration.RM_CONFIGURATION_PROVIDER_CLASS,//查看用户配置项yarn.resourcemanager.configuration.provider-class如果没有使用默认Provider
          defaultProviderClass, ConfigurationProvider.class),
          bootstrapConf);
  return configurationProvider;
}

具体查看一下两种方式源码，一目了然

public static final List<String> RM_CONFIGURATION_FILES =
    Collections.unmodifiableList(Arrays.asList(
        CS_CONFIGURATION_FILE,
        HADOOP_POLICY_CONFIGURATION_FILE,
        YARN_SITE_CONFIGURATION_FILE,
        CORE_SITE_CONFIGURATION_FILE));

org.apache.hadoop.yarn.LocalConfigurationProvider

基于linux本地加载配置文件

@Override
public InputStream getConfigurationInputStream(Configuration bootstrapConf,
    String name) throws IOException, YarnException {
  if (name == null || name.isEmpty()) {
    throw new YarnException(
        "Illegal argument! The parameter should not be null or empty");
  } else if (YarnConfiguration.RM_CONFIGURATION_FILES.contains(name)) { // 检查是否属于预定义配置文件
    return bootstrapConf.getConfResourceAsInputStream(name); //Configuration方法
  }
  return new FileInputStream(name);
}

public InputStream getConfResourceAsInputStream(String name) {
  try {
    URL url= getResource(name);

    if (url == null) {
      LOG.info(name + " not found");
      return null;
    } else {
      LOG.info("found resource " + name + " at " + url);
    }

    return url.openStream();
  } catch (Exception e) {
    return null;
  }
}

org.apache.hadoop.yarn.FileSystemBasedConfigurationProvider

不多解析看到这两个变量就知道了。基于非Linux本地系统加载配置文件（可能描述不准确）。

private FileSystem fs;
private Path configDir;

@Override
public synchronized InputStream getConfigurationInputStream(
    Configuration bootstrapConf, String name) throws IOException,
    YarnException {
  if (name == null || name.isEmpty()) {
    throw new YarnException(
        "Illegal argument! The parameter should not be null or empty");
  }
  Path filePath;
  if (YarnConfiguration.RM_CONFIGURATION_FILES.contains(name)) {
    filePath = new Path(this.configDir, name);
    if (!fs.exists(filePath)) {
      LOG.info(filePath + " not found");
      return null;
    }
  } else {
    filePath = new Path(name);
    if (!fs.exists(filePath)) {
      LOG.info(filePath + " not found");
      return null;
    }
  }
  return fs.open(filePath);
}

org.apache.hadoop.yarn.conf.ConfigurationProviderFactory分析加载配置文件两种方式

标签：

原文地址：http://www.cnblogs.com/yanbit/p/4745954.html