Accessing Kafka Using a VPC Endpoint Across VPCs

Is Plaintext Access or Ciphertext Access Used When a Client Accesses Kafka Across VPCs Using A VPC Endpoint?

It depends on Cross-VPC Access Protocol. The cross-VPC access protocol can be configured when you create a Kafka instance. After an instance is created, the setting cannot be changed.

Options:

• PLAINTEXT: There is no authentication required in such a connection and data is transmitted in plaintext.
• SASL_SSL: Clients can connect to a Kafka instance with SASL and the data will be encrypted using the SSL certificate.
• SASL_PLAINTEXT: Clients can connect to a Kafka instance with SASL and the data will be transmitted in plaintext.

Creating a VPC Endpoint Service

1. Log in to the Kafka console.
2. Click in the upper left corner to select the region where your instance is located.
3. Click the desired instance to go to the instance details page.
4. In the Advanced Settings area on the Overview page, obtain the listeners IP addresses and port IDs of the instance for Cross-VPC Access.
   Figure 2 Cross-VPC access–related listeners IP addresses and corresponding port IDs of the Kafka instance
   

5. In the Network area on the Overview page, view the VPC to which the Kafka instance belongs.
   Figure 3 Viewing the VPC to which the Kafka instance belongs
   

6. Click the VPC to obtain the VPC ID on the VPC console.
   Figure 4 Obtaining the VPC ID
   

7. Call the VPC Endpoint API to create a VPC endpoint service. Set request parameters by referring to Table 1, and other parameters as required. For details, see Creating a VPC Endpoint Service.

   POST https://{endpoint}/v1/{project_id}/vpc-endpoint-services

   Table 1 VPC Endpoint service creation parameters

   Parameter	Description
   port_id	ID of the backend resource. Enter a port ID obtained in 4.
   vpc_id	ID of the VPC of the backend resource. Enter a VPC ID obtained in 6.
   server_type	Resource type. Enter VM.
   client_port	Access port. Enter 9011.
   server_port	Service port. Enter 9011.
   protocol	Port mapping protocol. Enter TCP.
   approval_enabled	Whether approval is required. Entering false indicates that no approval is required and the connected VPC endpoint will be in the accepted state.
   service_type	Service type. Enter interface.
   endpoint	VPCEP endpoint obtained from Regions and Endpoints. The region must be the same as that of the Kafka instance.
   project_id	project ID obtained from Obtaining a Project ID. The region must be the same as that of the Kafka instance.

   Record the value of service_name in the response. This parameter indicates the name of the VPC endpoint service.

8. Repeat 7 to create VPC endpoint services for other port IDs obtained in 4 and record the VPC endpoint service names.

(Optional) Adding a Whitelist

The VPC endpoint service can be used across accounts through a whitelist.

If the Kafka client and Kafka instance belong to different accounts, add the ID of the account to which the Kafka client belongs to the whitelist of the endpoint service. For details, see Add a Whitelist Record.

Buying a VPC Endpoint

1. Click in the upper left corner of the console. Then choose Network > VPC Endpoint.
2. Click Buy VPC Endpoint.
3. Set parameters by referring to Table 2. Retain the default values for other parameters. For details, see Buying a VPC Endpoint.
   Figure 5 VPC endpoint parameters
   

   Table 2 VPC endpoint creation parameters

   Parameter	Description
   Region	Region where the endpoint is located. Select the region that the Kafka instance is in.
   Service Category	Cloud services: Select this option if the VPC endpoint service to be accessed is a cloud service. Find a service by name: Select this option if the VPC endpoint service to be accessed is a private service of your own. Select Find a service by name.
   VPC Endpoint Service Name	Enter the VPC endpoint service name recorded in 7 and click Verify. If Service name found is displayed, proceed with subsequent operations.
   (Optional) Create a Private Domain Name	If you want to access a VPC endpoint using a domain name, select Create a Private Domain Name.
   VPC	VPC where the endpoint is located. Select the VPC that the Kafka client is in.
   Subnet	Subnet where the endpoint is located. Select the subnet that the Kafka client is in.
   IPv4 Address	IPv4 address of the endpoint. Select Automatically assign IP address.

4. Click Next.
5. Confirm the configurations and submit the request.
6. Go back to the VPC endpoint list and check whether the status of the created VPC endpoint has changed to Accepted. The Accepted state means that the VPC endpoint has been connected to the VPC endpoint service.
   Figure 6 Checking the VPC endpoint status
   

7. Click the VPC endpoint ID. On the Summary tab page, obtain the private IP address. You can use the private IP address to access the VPC endpoint service.
   To access a VPC endpoint using a domain name, view and record the private domain name on the Basic Information page.

   

   To copy a private domain name, do not carry the last period (.). Correct domain name format: vpcep-5xxx7-e302-4ab4-bc6c-54xxxe52.xx-xx-xx.xxx.com.

   Figure 7 Viewing the private IP address
   

8. Repeat 1 to 7 to buy a VPC endpoint for each VPC endpoint service created in 8, and view and record the private IP addresses of the VPC endpoint services.

   To access a VPC endpoint using a domain name, record the private domain name.

Modifying Parameter advertised.listeners IP

1. Click and choose Middleware > Distributed Message Service (for Kafka) to open the Kafka overview page.
2. In the navigation pane, choose Kafka Instances.
3. Click the desired Kafka instance to view its details.
4. In the Advanced Settings area on the Overview page, click Modify for Cross-VPC Access to change the value of advertised.listeners IP address/domain name to the private IP addresses recorded in 7 and 8. Each IP address must match the corresponding port ID. Otherwise, the network will be disconnected. After the modification, click Save.
   To access an endpoint using a domain name, in the Advanced Settings area on the Overview page, click Modify. In the advertised.listeners IP Address/Domain Name column, click to switch to the domain name mode. Change it to the private domain name recorded in 7 and 8. Each private domain name must match the corresponding port ID. Otherwise, the network will be disconnected.

   Figure 8 Changing the advertised.listeners IP addresses
   

Verifying Connectivity

Check whether messages can be created and retrieved by referring to Connecting to Kafka Using the Client (Plaintext Access) or Connecting to Kafka Using the Client (Ciphertext Access).

Notes:

• The address for connecting to a Kafka instance is in the format of "advertised.listeners IP:9011". For example, the addresses for connecting to the Kafka instance shown in Figure 8 are 192.168.0.71:9011,192.168.0.11:9011,192.168.0.21:9011.

  To access an endpoint using a domain name, the address of the Kafka instance is domain name:9011. For example, vpcep-5xxx7-e302-4ab4-bc6c-54xxxe52.xx-xx-xx.xxx.com:9011.

• Configure inbound rules for the security group of the Kafka instance to allow access from 198.19.128.0/17 over port 9011.
• If a network access control list (ACL) has been configured for the subnet of this instance, configure inbound rules for the network ACL to allow access from 198.19.128.0/17 and from the subnet used by the VPC endpoint.

198.19.128.0/17 is the network segment allocated to the VPCEP service. To use VPCEP, allow access from this network segment.