Configuring Security Authentication for MapReduce Applications

Scenario Description

In cluster environment with Kerberos authentication enabled, the components must be mutually authenticated before communicating with each other to ensure communication security.

When submitting MapReduce applications, users need to communicate with Yarn and HDFS. Code for security authentication needs to be written into the MapReduce application to be submitted to ensure that MapReduce can work properly.

Two security authentication modes are available.

• CLI authentication

  Before submitting and running the MapReduce application, run the following command on the MapReduce client to obtain authentication:

  kinit Component service user

• Code authentication

  Obtain the principal and keytab files of the client for authentication.

Security Authentication Code

Currently, the LoginUtil class is invoked for security authentication in a unified manner.

In the MapReduce sample project code, test@HADOOP.COM, user.keytab, and krb5.conf are examples. In actual operations, contact the administrator to obtain the keytab and krb5.conf files corresponding to the account and the permission. Save the keytab and krb5.conf files to the conf directory of the sample code. The code for security login is as follows:

Modify the authentication information based on the site requirements.

public static final String PRINCIPAL= "test@HADOOP.COM";
public static final String KEYTAB = FemaleInfoCollector.class.getClassLoader().getResource("user.keytab").getPath();
public static final String KRB = FemaleInfoCollector.class.getClassLoader().getResource("krb5.conf").getPath();

// Check whether the security mode is used.
    if("kerberos".equalsIgnoreCase(conf.get("hadoop.security.authentication"))){
       // Security login
       System.setProperty("java.security.krb5.conf", KRB);
       LoginUtil.login(PRINCIPAL, KEYTAB, KRB, conf);
    }