Help Center/ MapReduce Service/ Component Operation Guide (LTS)/ Using Kafka/ Enterprise-Class Enhancements of Kafka/ Configuring a Secure Transmission Protocol for Kafka Data
Updated on 2024-10-09 GMT+08:00
View PDF
Share

Configuring a Secure Transmission Protocol for Kafka Data

This section applies to MRS 3.x or later.

Brief Introduction to Kafka APIs

  • Producer API

    Indicates the API defined in org.apache.kafka.clients.producer.KafkaProducer. When kafka-console-producer.sh is used, the API is used by default.

  • Consumer API

    Indicates the API defined in org.apache.kafka.clients.consumer.KafkaConsumer. When kafka-console-consumer.sh is used, the API is used by default.

In MRS 3.x or later, Kafka no longer support old Producer or Consumer APIs.

Protocol Description for Accessing Kafka

The protocols used to access Kafka are as follows: PLAINTEXT, SSL, SASL_PLAINTEXT, and SASL_SSL.

When Kafka service is started, the listeners using the PLAINTEXT and SASL_PLAINTEXT protocols are started. You can set ssl.mode.enable to true in Kafka service configuration to start listeners using SSL and SASL_SSL protocols. The following table describes the four protocols:

For details about how to view or configure parameters, see Modifying Cluster Service Configuration Parameters.

Protocol

Description

Default Port

PLAINTEXT

Supports plaintext access without authentication.

Obtain the value of port. The default value is 9092

SASL_PLAINTEXT

Supports plaintext access with Kerberos authentication.

Obtain the value of sasl.port. The default value is 21007

SSL

Supports SSL-encrypted access without authentication.

Obtain the value of ssl.port. The default value is 9093

SASL_SSL

Supports SSL-encrypted access with Kerberos authentication.

Obtain the value of sasl-ssl.port. The default value is 21009

ACL Settings for a Topic

To view and set topic permission information, run the kafka-acls.sh script on the Linux client. For details, see Kafka User Permissions.

Use of Kafka APIs in Different Scenarios

  • Scenario 1: accessing the topic with an ACL

    Used API

    User Group

    Client Parameter

    Server Parameter

    Accessed Port

    API

    Users need to meet one of the following conditions:

    • Assigned the System_administrator role
    • In the kafkaadmin group
    • In the kafkasuperuser group
    • In the kafka group and be authorized

    security.inter.broker.protocol=SASL_PLAINTEXT sasl.kerberos.service.name = kafka

    -

    sasl.port (The default number is 21007.)

    security.protocol=SASL_SSL sasl.kerberos.service.name = kafka

    Set ssl.mode.enable to true.

    sasl-ssl.port (The default number is 21009.)
  • Scenario 2: accessing the topic without an ACL

    Used API

    User Group

    Client Parameter

    Server Parameter

    Accessed Port

    API

    Users need to meet one of the following conditions:

    • Assigned the System_administrator role
    • In the kafkaadmin group
    • In the kafkasuperuser group

    security.protocol=SASL_PLAINTEXT sasl.kerberos.service.name = kafka

    -

    sasl.port (The default number is 21007.)

    Users are in the kafka group.

    Set allow.everyone.if.no.acl.found to true.

    NOTE:

    In normal mode, the server parameter allow.everyone.if.no.acl.found does not need to be modified.

    sasl.port (The default number is 21007.)

    Users need to meet one of the following conditions:

    • Assigned the System_administrator role
    • In the kafkaadmin group
    • In the kafkasuperuser group

    security.protocol=SASL_SSL sasl.kerberos.service.name = kafka

    Set ssl.mode.enable to true.

    sasl-ssl.port (The default number is 21009.)

    Users are in the kafka group.
    1. Set allow.everyone.if.no.acl.found to true.
    2. Set ssl.mode.enable to true.

    sasl-ssl.port (The default number is 21009.)

    -

    security.protocol=PLAINTEXT

    Set allow.everyone.if.no.acl.found to true.

    port (The default number is 9092.)

    -

    security.protocol=SSL
    1. Set allow.everyone.if.no.acl.found to true.
    2. Set ssl.mode.enable to true.

    ssl.port (The default number is 9063.)