Preparing the Kafka Application Security Authentication

Prerequisites

Kerberos authentication has been enabled for the MRS cluster. Skip this step if Kerberos authentication is not enabled for the cluster.

Preparing the Authentication Mechanism Code

In the environment with Kerberos authentication enabled, the components must be mutually authenticated before communicating with each other, in order to ensure communication security. The Kafka, ZooKeeper, and Kerberos security authentications are required for Kafka application development. However, you only need to generate one JAAS file and set related environment variables accordingly. LoginUtil related interfaces are provided to complete the configuration. In the following sample code, only the account applied by a user and the keytab file name need to be configured. The keytab file of a human-machine account becomes invalid when the user password expires. Therefore, you are advised to use a machine-machine account for configuration.

Code sample:

Configure the keytab authentication file module.

    /**
     * keytab file name of the account that a user applies for
     */
    private static final String USER_KEYTAB_FILE = "keytab file name of the account that a user applies for";
    
    /**
    * Account that a user applies for
    */
    private static final String USER_PRINCIPAL = "Account that a user applies for";

Kerberos authentication module of MRS. If the Kerberos authentication is not enabled for the service, this logic does not need to be executed.

    public static void securityPrepare() throws IOException
    {
        String filePath = System.getProperty("user.dir") + File.separator + "conf" + File.separator;       
        String krbFile = filePath + "krb5.conf";
        String userKeyTableFile = filePath + USER_KEYTAB_FILE;
        
        //Replace separators in the Windows path.
        userKeyTableFile = userKeyTableFile.replace("\\", "\\\\");
        krbFile = krbFile.replace("\\", "\\\\");
        
        LoginUtil.setKrb5Config(krbFile);
        LoginUtil.setZookeeperServerPrincipal("zookeeper/hadoop.hadoop.com");
        LoginUtil.setJaasFile(USER_PRINCIPAL, userKeyTableFile);
    } 

If you change the Kerberos domain name of a cluster, you need to add kerberos.domain.name to the code and configure a correct domain name following the hadoop.expr=toLowerCase(%{default_realm}%{KerberosServer}) rule. For example: If the domain name is changed to HUAWEI.COM, set this parameter to hadoop.huawei.com.

Obtaining the Keytab File

1. Access MRS Manager with Kerberos enabled. For details, see section "Accessing MRS Manager" in MapReduce Service User Guide.
2. Choose System > Manage User. Locate the specified user, click More > Download authentication credential.
3. Decompress the downloaded .zip file to obtain the krb5.conf file and the keytab file of the user.
4. Copy the krb5.conf file and the keytab file of the user to the conf directory of the sample project.