Connecting Flume to Kafka in Security Mode

Scenario

This section describes how to connect to Kafka using the Flume client in security mode.

This section applies to MRS 3.x or later.

Procedure

1. Create a jaas.conf file and save it to ${Flume client installation directory} /conf. The content of the jaas.conf file is as follows:

   KafkaClient {
   com.sun.security.auth.module.Krb5LoginModule required
   useKeyTab=true
   keyTab="/opt/test/conf/user.keytab"
   principal="flume_hdfs@<System domain name>"
   useTicketCache=false
   storeKey=true
   debug=true;
    };

   Set keyTab and principal based on site requirements. The configured principal must have certain kafka permissions.

2. Configure services. Set the port number of kafka.bootstrap.servers to 21007, and set kafka.security.protocol to SASL_PLAINTEXT.
3. If the domain name of the cluster where Kafka is located is changed, change the value of -Dkerberos.domain.name in the flume-env.sh file in ${Flume client installation directory} /conf/ based on the site requirements.
4. Upload the configured properties.properties file to ${Flume client installation directory} /conf.