Connecting to an MRS Kafka Data Source

Overview

ROMA Connect can use the MRS Kafka data source for data integration tasks. Before using the MRS Kafka data source, you need to connect it to ROMA Connect.

Prerequisites

• Each connected data source must belong to an integration application. Ensure that an integration application is available before connecting a data source, or create one first.
• Kerberos authentication has been enabled for the MRS cluster where the MRS Kafka data source is located. The execution permission has been configured for machine-machine interaction users. For details, see Preparing a Development User.

Procedure

1. Log in to the ROMA Connect console. On the Instances page, click View Console of an instance.
2. In the navigation pane on the left, choose Data Sources. In the upper right corner of the page, click Access Data Source.
3. On the Default tab page, select MRS Kafka and click Next.
4. Configure the data source connection information.

   Table 1 Data source connection information

   Parameter	Description
   Name	Enter a data source name. Using naming rules facilitates future search.
   Encoding Format	Default: utf-8
   Integration Application	Select the integration application to which the data source belongs.
   Description	Enter a brief description of the data source.
   Machine-machine Username	Enter the machine-machine username for connecting to MRS Kafka.
   Configuration File	Click Upload to upload the MRS Kafka configuration file. For details about how to obtain the files, see "Obtaining MRS Kafka configuration files".

   Obtaining MRS Kafka configuration files

   1. Obtain the krb5.conf and user.keytab files.

      Download the user authentication file from MRS Manager by following the procedure described in Downloading a User Authentication File, and decompress the file to obtain the krb5.conf and user.keytab files.

   2. Obtain the producer.properties, consumer.properties, and hosts files.

      Download the client configuration file from the MRS console by following the procedure described in Updating a Client Configuration File. After the file is decompressed:

      • Obtain the hosts file from xxx_Services_ClientConfig_ConfigFiles.
      • Obtain the producer.properties and consumer.properties files from xxx_Services_ClientConfig_ConfigFiles > Kafka > config.

        Change producer.properties to fikafka-producer.properties and consumer.properties to fikafka-consumer.properties, and copy bootstrap.servers in fikafka-producer.properties to fikafka-consumer.properties.

   3. Obtain the MRS Kafka configuration files.

      Save the obtained files to a new directory and compress them into a .zip package. All files are stored in the root directory of the .zip package.

      

      • The file name contains a maximum of 255 characters, including only letters and digits.
      • The file size cannot exceed 2 MB.
5. Click Create.