MRS Application Security Authentication Description

Kerberos Authentication Description

For clusters in security mode with Kerberos authentication enabled, security authentication is required during application development.

Kerberos, named after the ferocious three-headed guard dog of Hades from Greek mythology, is now used to a concept in authentication. The Kerberos protocol adopts a client–server model and cryptographic algorithms such as AES (Advanced Encryption Standard). It provides mutual authentication, that is, both the client and the server can verify each other's identity. Kerberos is used to prevent interception and replay attacks and protect data integrity. It is a system that manages keys by using a symmetric key mechanism.

Figure 1 Kerberos Principle Architecture

An application client can be a service in a cluster or an application developed by a customer. An application can submit tasks or jobs to the application service.

1. Before submitting a task or job, the application client needs to apply for a ticket granting ticket (TGT) from the Kerberos service to establish a secure session with the Kerberos server.
2. After receiving the TGT request, the Kerberos service resolves parameters in the request to generate a TGT, and uses the key of the username specified by the client to encrypt the response.
3. After receiving the TGT response, the application client (based on the underlying RPC) resolves the response and obtains the TGT, and then applies for a server ticket (ST) of the application server from the Kerberos service.
4. After receiving the ST request, the Kerberos service verifies the TGT validity in the request and generates an ST of the application service, and then uses the application service key to encrypt the response.
5. After receiving the ST response, the application client packages the ST into a request and sends the request to the application server.
6. After receiving the request, the application server uses its local application service key to resolve the ST. After successful verification, the request becomes valid.

Basic Concepts of Security Authentication

This document uses HDFS component application security authentication as an example to describe the basic concepts related to security authentication. This helps users reduce the time spent on learning the Kerberos framework and better understand Kerberos services.

TGT

A Ticket-Granting Ticket is generated by the Kerberos service and is used by applications to establish authentication sessions with the Kerberos server. The default validity period of the Ticket is 24 hours. After 24 hours, the Ticket automatically expires.

TGT application mode (HDFS is used as an example):

• Obtain the value through the interface provided by the HDFS.

  /**
    * login Kerberos to get TGT, if the cluster is in security mode
    * @throws IOException if login is failed
    */
    private void login() throws IOException {       
    // not security mode, just return
      if (! "kerberos".equalsIgnoreCase(conf.get("hadoop.security.authentication"))) {
          return;
      }
          
      //security mode
      System.setProperty("java.security.krb5.conf", PATH_TO_KRB5_CONF);
          
      UserGroupInformation.setConfiguration(conf);
      UserGroupInformation.loginUserFromKeytab(PRNCIPAL_NAME, PATH_TO_KEYTAB);        
    }

• Obtain the value using kinit on the MRS cluster client.
  1. Log in to the node where the MRS cluster client is located and go to the client installation directory.

     cd {Client installation directory}

  2. Run the following command to configure environment variables:

     source bigdata_env

  3. Run the following command to perform user authentication:

     kinit MRS Trunking service user

ST

A server ticket is generated by the Kerberos service and is used by an application to establish a secure session with the application service. The ticket is valid once.

STs are generated in MRS based on Hadoop-rpc communication. The underlying RPC layer automatically submits requests to the Kerberos server and the Kerberos server generates STs.

Kerberos Authentication Code Example

package com.huawei.bigdata.hdfs.examples;

import java.io.IOException;

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileStatus;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.security.UserGroupInformation;

public class KerberosTest {
    private static String PATH_TO_HDFS_SITE_XML = KerberosTest.class.getClassLoader().getResource("hdfs-site.xml")
            .getPath();
    private static String PATH_TO_CORE_SITE_XML = KerberosTest.class.getClassLoader().getResource("core-site.xml")
            .getPath();
    private static String PATH_TO_KEYTAB = KerberosTest.class.getClassLoader().getResource("user.keytab").getPath();
    private static String PATH_TO_KRB5_CONF = KerberosTest.class.getClassLoader().getResource("krb5.conf").getPath();
    private static String PRNCIPAL_NAME = "develop";
    private FileSystem fs;
    private Configuration conf;
    
    /**
     * initialize Configuration
     */
    private void initConf() {
        conf = new Configuration();
        
        // add configuration files
        conf.addResource(new Path(PATH_TO_HDFS_SITE_XML));
        conf.addResource(new Path(PATH_TO_CORE_SITE_XML));
    }
    
    /**
     * login Kerberos to get TGT, if the cluster is in security mode
     * @throws IOException if login is failed
     */
    private void login() throws IOException {       
        // not security mode, just return
        if (! "kerberos".equalsIgnoreCase(conf.get("hadoop.security.authentication"))) {
            return;
        }
        
        //security mode
        System.setProperty("java.security.krb5.conf", PATH_TO_KRB5_CONF);
        
        UserGroupInformation.setConfiguration(conf);
        UserGroupInformation.loginUserFromKeytab(PRNCIPAL_NAME, PATH_TO_KEYTAB);        
    }
    
    /**
     * initialize FileSystem, and get ST from Kerberos
     * @throws IOException
     */
    private void initFileSystem() throws IOException {
        fs = FileSystem.get(conf);
    }
    
    /**
     * An example to access the HDFS
     * @throws IOException
     */
    private void doSth() throws IOException {
        Path path = new Path("/tmp");
        FileStatus fStatus = fs.getFileStatus(path);
        System.out.println("Status of " + path + " is " + fStatus);
        //other thing
    }


    public static void main(String[] args) throws Exception {
        KerberosTest test = new KerberosTest();
        test.initConf();
        test.login();
        test.initFileSystem();
        test.doSth();       
    }
}

• During Kerberos authentication, you need to configure the file parameters required by Kerberos authentication, including the keytab file path, Kerberos authentication user name, and krb5.conf file required by Kerberos authentication.
• The login() method is used to invoke the Hadoop interface to perform Kerberos authentication and generate a TGT ticket.
• The doSth() method invokes the Hadoop interface to access the file system. In this case, the underlying RPC automatically carries the TGT for Kerberos authentication and generates an ST ticket.
• The preceding code can be used to create KerberosTest.java in the HDFS secondary development sample project in security mode, run the KerberosTest.java, and view the commissioning result.