Updated on 2023-11-30 GMT+08:00

Running the Sample Project

The following describes how to access DMS Kafka queues to produce and consume messages in Java.

Procedure

  1. Log in to the ECS.

    You can run the sample project on an ECS with an IP address in the 192 network segment.

  2. Install JDK or Java runtime environment (JRE). Add the following settings of environment variables JAVA_HOME and PATH to the ~/.bash_profile:

    export JAVA_HOME=/opt/java/jdk1.8.0_151
    export PATH=$JAVA_HOME/bin:$PATH

    Run the source .bash_profile command for the modification to take effect.

    Use Oracle JDK instead of ECS's default JDK (for example, OpenJDK), because ECS's default JDK may not be suitable for the sample project. To obtain Oracle JDK, download Java Development Kit 1.8.111 or a later version from https://www.oracle.com/technetwork/java/javase/downloads/index.html.

  3. Run the following command to download the code package of the sample project DmsKafkaDemo.zip.

    $ wget https://dms-demo.obs.cn-north-1.myhuaweicloud.com/DmsKafkaDemo.zip

  4. Run the following command to decompress DmsKafkaDemo.zip.

    $ unzip DmsKafkaDemo.zip

  5. Run the following command to navigate to the DmsKafkaDemo/dist directory, which contains pre-compiled binary files and executable scripts.

    $ cd DmsKafkaDemo/dist

  6. Edit the config/dms_kafka_client_jaas.conf file and configure access_key, secret_key, and project_id.

    $ vim config/dms_kafka_client_jaas.conf

    The values in bold are examples. Replace them with actual values.

    KafkaClient {
      com.huawei.middleware.kafka.sasl.client.KafkaLoginModule required
      access_key="********************"
      secret_key="**********"
      project_id="bd67aaead60940d688b872c31bdc653b"
      target_project_id="bd67aaead60940d688b872c31bdc6539";
    };

    To access the queues authorized by other tenants, set target_project_id to the project ID of the authorizing tenant.

  7. Edit the config/producer.properties file and configure topic and bootstrap.servers.

    $ vim config/producer.properties

    The values in bold are examples. Replace them with actual values.

    topic=k-bd67aaead60940d688b872c31bdc653b-4df89da6-ede4-4072-93e0-28dc6e866299 
    bootstrap.servers=dms-kafka.cn-north-1.myhuaweicloud.com:37000
    ssl.truststore.password=************
    acks=all
    retries=1
    batch.size=16384
    buffer.memory=33554432
    key.serializer=org.apache.kafka.common.serialization.StringSerializer
    value.serializer=org.apache.kafka.common.serialization.StringSerializer
    security.protocol=SASL_SSL
    sasl.mechanism=DMS

    The parameter topic can be set to a queue name or a Kafka topic name. For more information, see Table 2.

  8. Edit the config/consumer.properties file and configure topic, bootstrap.servers, and group.id.

    $ vim config/consumer.properties

    The values in bold are examples. Replace them with actual values.

    topic=k-bd67aaead60940d688b872c31bdc653b-4df89da6-ede4-4072-93e0-28dc6e866299
    bootstrap.servers=dms-kafka.cn-north-1.myhuaweicloud.com:37000
    group.id=g-7ec0caac-01fb-4f91-a4f2-0a9dd48f8af7
    ssl.truststore.password=************
    security.protocol=SASL_SSL
    sasl.mechanism=DMS
    key.deserializer=org.apache.kafka.common.serialization.StringDeserializer
    value.deserializer=org.apache.kafka.common.serialization.StringDeserializer
    auto.offset.reset=earliest
    enable.auto.commit=false

    The parameter topic can be set to a queue name or a Kafka topic name. For more information, see Table 2.

  9. Run the sample project to produce messages:

    $ bash produce.sh

    After the command is run, 10 messages are automatically sent to the Kafka queue.

  10. Run the sample project to consume messages:

    $ bash consume.sh