Preparing the Kafka Application Security Authentication

Updated on 2024-08-16 GMT+08:00

View PDF

Prerequisites

Kerberos authentication has been enabled for the MRS cluster. Skip this step if Kerberos authentication is not enabled for the cluster.

Preparing the Authentication Mechanism Code

In the environment with Kerberos authentication enabled, the components must be mutually authenticated before communicating with each other, in order to ensure communication security. The Kafka, ZooKeeper, and Kerberos security authentications are required for Kafka application development. However, you only need to generate one JAAS file and set related environment variables accordingly. LoginUtil related interfaces are provided to complete the configuration. In the following sample code, only the account applied by a user and the keytab file name need to be configured. The keytab file of a human-machine account becomes invalid when the user password expires. Therefore, you are advised to use a machine-machine account for configuration.

Code sample:

Configure the keytab authentication file module.

    /**
     * keytab file name of the account that a user applies for
     */
    private static final String USER_KEYTAB_FILE = "keytab file name of the account that a user applies for";
    
    /**
    * Account that a user applies for
    */
    private static final String USER_PRINCIPAL = "Account that a user applies for";

Kerberos authentication module of MRS. If the Kerberos authentication is not enabled for the service, this logic does not need to be executed.

    public static void securityPrepare() throws IOException
    {
        String filePath = System.getProperty("user.dir") + File.separator + "conf" + File.separator;       
        String krbFile = filePath + "krb5.conf";
        String userKeyTableFile = filePath + USER_KEYTAB_FILE;
        
        //Replace separators in the Windows path.
        userKeyTableFile = userKeyTableFile.replace("\\", "\\\\");
        krbFile = krbFile.replace("\\", "\\\\");
        
        LoginUtil.setKrb5Config(krbFile);
        LoginUtil.setZookeeperServerPrincipal("zookeeper/hadoop.hadoop.com");
        LoginUtil.setJaasFile(USER_PRINCIPAL, userKeyTableFile);
    }

NOTE:

If you change the Kerberos domain name of a cluster, you need to add kerberos.domain.name to the code and configure a correct domain name following the hadoop.expr=toLowerCase(%{default_realm}%{KerberosServer}) rule. For example: If the domain name is changed to HUAWEI.COM, set this parameter to hadoop.huawei.com.

Obtaining the Keytab File

Access MRS Manager with Kerberos enabled. For details, see section "Accessing MRS Manager" in MapReduce Service User Guide.
Choose System > Manage User. Locate the specified user, click More > Download authentication credential.
Decompress the downloaded .zip file to obtain the krb5.conf file and the keytab file of the user.
Copy the krb5.conf file and the keytab file of the user to the conf directory of the sample project.

Parent topic: Preparing a Kafka Application Development Environment

Previous topic: Importing and Configuring Kafka Sample Projects

Next topic: Developing a Kafka Application

Feedback

Was this page helpful?

Helpful Not helpful

Provide feedback

Thank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.

The system is busy. Please try again later.

Which of the following issues have you encountered?

Content is inconsistent with the product UI

Unclear descriptions

Lack of examples or code

Incorrect steps

Can't find what I need

Lack of best practices

Feedback (optional)

0/500

Select at least one type of issue, and enter your comments or suggestions.

Enter a maximum of 500 characters.

Submit Cancel

For any further questions, feel free to contact us through the chatbot.

Chatbot