Preparing the HDFS Authentication Mechanism

Scenario

Before accessing services in the secure cluster environment, you must be authorized by Kerberos.Codes for security authentication need to be written into the HDFS applications to ensure that the applications can work properly.

Two security authentication methods are described as follows:

Authentication by running command lines:
Before submitting the HDFS application for running, run the following command in the HDFS client to obtain authentication:

kinit component service user

This method applies only to the Linux OS that is installed with the HDFS client.
Authentication by adding codes:
Authenticate by obtaining the principal and keytab files of the client.

Change the value of PRINCIPAL_NAME in the code to the actual value.
```
private static final String PRNCIPAL_NAME = "hdfsDeveloper";
```

Safety Security Code

The safety authentication of the example codes is completed by invoking the LoginUtil class.

In the HDFS sample project code, different sample projects use different authentication codes which are basic safety authentication and the basic safety authentication with the ZooKeeper authentication.

Basic safety authentication:

Sample projects of the HdfsExample class in the com.huawei.bigdata.hdfs.examples package need only the basic safety authentication codes because these sample projects do not need to access the HBase or ZooKeeper. Add the following codes in the program:

...
    private static final String PATH_TO_HDFS_SITE_XML = HdfsExample.class.getClassLoader().getResource("hdfs-site.xml").getPath();
    private static final String PATH_TO_CORE_SITE_XML = HdfsExample.class.getClassLoader().getResource("core-site.xml").getPath();
    private static final String PRNCIPAL_NAME = "hdfsDeveloper";
    private static final String PATH_TO_KEYTAB = HdfsExample.class.getClassLoader().getResource("user.keytab").getPath();
    private static final String PATH_TO_KRB5_CONF = HdfsExample.class.getClassLoader().getResource("krb5.conf").getPath();
    private static Configuration conf = null;
    //private static String PATH_TO_SMALL_SITE_XML = HdfsExample.class.getClassLoader().getResource("smallfs-site.xml").getPath();
...
    private static void confLoad() throws IOException {
        System.setProperty("java.security.krb5.conf", PATH_TO_KRB5_CONF);
        conf = new Configuration();
        // conf file
        conf.addResource(new Path(PATH_TO_HDFS_SITE_XML));
        conf.addResource(new Path(PATH_TO_CORE_SITE_XML));
        // conf.addResource(new Path(PATH_TO_SMALL_SITE_XML));
    }
...
    private static void authentication() throws IOException {
        // security mode
        if ("kerberos".equalsIgnoreCase(conf.get("hadoop.security.authentication"))) {
            System.setProperty("java.security.krb5.conf", PATH_TO_KRB5_CONF);
            LoginUtil.login(PRNCIPAL_NAME, PATH_TO_KEYTAB, PATH_TO_KRB5_CONF, conf);
        }
    }

Basic safety authentication with ZooKeeper Authentication:

Sample projects of the ColocationExample class in the com.huawei.bigdata.hdfs.examples package require not only the basic safety authentication, but also the Principal of the server in ZooKeeper to complete the safety authentication. Add the following codes in the program:

...
    private static final String ZOOKEEPER_SERVER_PRINCIPAL_KEY = "zookeeper.server.principal";
    private static final String PRINCIPAL = "username.client.kerberos.principal";
    private static final String KEYTAB = "username.client.keytab.file";
    private static final String PRNCIPAL_NAME = "hdfsDeveloper";
    private static final String LOGIN_CONTEXT_NAME = "Client";
    private static final String PATH_TO_KEYTAB = System.getProperty("user.dir") + File.separator + "conf" + File.separator + "user.keytab";
    private static final String PATH_TO_KRB5_CONF = ColocationExample.class.getClassLoader().getResource("krb5.conf") .getPath();
    private static String zookeeperDefaultServerPrincipal = null;
    private static Configuration conf = new Configuration();
    private static DFSColocationAdmin dfsAdmin;
    private static DFSColocationClient dfs;
    private static void init() throws IOException {
        LoginUtil.login(PRNCIPAL_NAME, PATH_TO_KEYTAB, PATH_TO_KRB5_CONF, conf);
        LoginUtil.setJaasConf(LOGIN_CONTEXT_NAME, PRNCIPAL_NAME, PATH_TO_KEYTAB);
        zookeeperDefaultServerPrincipal = "zookeeper/hadoop." + KerberosUtil.getKrb5DomainRealm().toLowerCase();
        LoginUtil.setZookeeperServerPrincipal(ZOOKEEPER_SERVER_PRINCIPAL_KEY, zookeeperDefaultServerPrincipal);
    }	
...

The HdfsDeveloper user and the user's user.keytab and krb5.conf in the safety authentication codes are used as an example. In practical operations, contact the administrator to obtain the corresponding account and the keytab and krb5 files related to the account.
You can log in to FusionInsight Manager, choose System > Permission > Domain and Mutual Trust, and check the value of Local Domain, which is the current system domain name.
zookeeper/hadoop.<system domain name> is the user name. All letters in the system domain name contained in the user name of the system are lowercase letters. For example, if Local domain is set to 9427068F-6EFA-4833-B43E-60CB641E5B6C.COM, the user name is zookeeper/hadoop.9427068f-6efa-4833-b43e-60cb641e5b6c.com.