Help Center/ MapReduce Service/ Component Operation Guide (Normal)/ Using Kafka/ Enterprise-Class Enhancements of Kafka/ Configuring a Secure Transmission Protocol for Kafka Data
Updated on 2024-12-11 GMT+08:00
View PDF

Configuring a Secure Transmission Protocol for Kafka Data

This section applies to MRS 3.x or later.

Brief Introduction to Kafka APIs

  • Producer API

    Indicates the API defined in org.apache.kafka.clients.producer.KafkaProducer. When kafka-console-producer.sh is used, the API is used by default.

  • Consumer API

    Indicates the API defined in org.apache.kafka.clients.consumer.KafkaConsumer. When kafka-console-consumer.sh is used, the API is used by default.

In MRS 3.x or later, Kafka no longer support old Producer or Consumer APIs.

Protocol Description for Accessing Kafka

For details about how to view or set parameters, see Modifying Cluster Service Configuration Parameters.

The protocols used to access Kafka are as follows: PLAINTEXT, SSL, SASL_PLAINTEXT, and SASL_SSL.

When Kafka service is started, the security authentications using the PLAINTEXT and SASL_PLAINTEXT protocols are started. You can set ssl.mode.enable to true in Kafka service configuration to start the security authentications using SSL and SASL_SSL protocols. The following table describes the four protocols:

Protocol

Description

Default Port

PLAINTEXT

Supports plaintext access without authentication.

The value of port. The default value is 9092.

SASL_PLAINTEXT

Supports plaintext access with Kerberos authentication.

The value of sasl.port. The default value is 21007.

SSL

Supports SSL-encrypted access without authentication.

The value of ssl.port. The default value is 9093.

SASL_SSL

Supports SSL-encrypted access with Kerberos authentication.

The value of sasl-ssl.port. The default value is 21009.

ACL Settings for a Topic

To view and set topic permission information, run the kafka-acls.sh script on the Linux client. For details, see Kafka User Permissions.

Use of Kafka APIs in Different Scenarios

  • Scenario 1: accessing the topic with an ACL

    Used API

    User Group

    Client Parameter

    Server Parameter

    Accessed Port

    API

    Users need to meet one of the following conditions:

    • Assigned the System_administrator role
    • In the kafkaadmin group
    • In the kafkasuperuser group
    • In the kafka group and be authorized

    security.inter.broker.protocol=SASL_PLAINTEXT sasl.kerberos.service.name = kafka

    -

    sasl.port (The default number is 21007.)

    security.protocol=SASL_SSL sasl.kerberos.service.name = kafka

    Set ssl.mode.enable to true.

    sasl-ssl.port (The default number is 21009.)
  • Scenario 2: accessing the topic without an ACL

    Used API

    User Group

    Client Parameter

    Server Parameter

    Accessed Port

    API

    Users need to meet one of the following conditions:

    • Assigned the System_administrator role
    • In the kafkaadmin group
    • In the kafkasuperuser group

    security.protocol=SASL_PLAINTEXT sasl.kerberos.service.name = kafka

    -

    sasl.port (The default number is 21007.)

    Users are in the kafka group.

    Set allow.everyone.if.no.acl.found to true.

    NOTE:

    In normal mode, the server parameter allow.everyone.if.no.acl.found does not need to be modified.

    sasl.port (The default number is 21007.)

    Users need to meet one of the following conditions:

    • Assigned the System_administrator role
    • In the kafkaadmin group
    • In the kafkasuperuser group

    security.protocol=SASL_SSL sasl.kerberos.service.name = kafka

    Set ssl.mode.enable to true.

    sasl-ssl.port (The default number is 21009.)

    Users are in the kafka group.
    1. Set allow.everyone.if.no.acl.found to true.
    2. Set ssl.mode.enable to true.

    sasl-ssl.port (The default number is 21009.)

    -

    security.protocol=PLAINTEXT

    Set allow.everyone.if.no.acl.found to true.

    port (The default number is 9092.)

    -

    security.protocol=SSL
    1. Set allow.everyone.if.no.acl.found to true.
    2. Set ssl.mode.enable to true.

    ssl.port (The default number is 9063.)