Buying a Kafka Instance

Preparing Instance Dependencies

Before creating a Kafka instance, prepare the resources listed in Table 1.

Quick Config of a Cluster Kafka Instance

1. Log in to the console.
2. Click in the upper left corner to select a region.

   Select a region near you to ensure the lowest latency possible.

3. Click in the upper left corner and choose Application > Distributed Message Service for Kafka to open the Kafka instance list.
4. Click Buy Kafka Instance in the upper right corner of the page.
5. Set basic instance configurations on the Quick Config page.

   Table 2 Basic instance configuration parameters

   Parameter	Description
   Billing Mode	Pay-per-use is a postpaid mode. You can pay after using the service, and will be billed for your usage duration. The fees are calculated in seconds and settled by hour.
   Region	DMS for Kafka instances in different regions cannot communicate with each other over an intranet. Select a nearest location for low latency and fast access.
   AZ	An AZ is a physical region where resources use independent power supply and networks. AZs are physically isolated but interconnected through an internal network. Select one AZ or at least three AZs. The AZ setting is fixed once the instance is created.

6. Select a bundle.

   DMS for Kafka provides preconfigured specifications. You can select one as required. Specify the disk type and capacity as required. The disk type cannot be changed once the Kafka instance is created.

   The storage space is consumed by message replicas, logs, and metadata. Specify the storage space based on the expected service message size, the number of replicas, and the reserved disk space. Each Kafka broker reserves 33 GB disk space for storing logs and metadata.

   Disks are formatted when an instance is created. As a result, the actual available disk space is 93% to 95% of the total disk space.

7. Set the network information.

   Table 3 Instance network parameters

   Parameter	Description
   VPC	Select a created or shared VPC. A VPC provides an isolated virtual network for your Kafka instances. You can configure and manage the network as required. You can click Manage VPC on the right to go to the VPC console and view or create a VPC. After the Kafka instance is created, its VPC cannot be changed.
   Subnet	Select a created subnet. After the Kafka instance is created, its subnet cannot be changed.
   IPv6	This parameter is displayed after IPv6 is enabled for the subnet. The instance with IPv6 enabled can be accessed on a client using IPv6 addresses. The IPv6 setting is fixed once the instance is created.
   Private IP Addresses	Select Auto or Manual. Auto: The system automatically assigns an IP address from the subnet. Manual: Select IP addresses from the drop-down list. If the number of selected IP addresses is less than the number of brokers, the remaining IP addresses will be automatically assigned. This parameter is not displayed when IPv6 is enabled.
   Security Group	Select a created security group. A security group is a set of rules for accessing a Kafka instance. You can click Create Security Group on the right to go to the security group page and view or create a security group. Before accessing a Kafka instance on the client, configure security group rules based on the access mode. For details about security group rules, see Table 2.

8. Configure the instance access mode.

   Table 4 Instance access mode parameters

   Parameter	Description
   Access Method	There are two methods: Plaintext access: Clients connect to the Kafka instance without SASL authentication. Ciphertext access: Clients connect to the Kafka instance with SASL authentication. If you enable Ciphertext Access, specify an SSL username and password. Once enabled, private network access cannot be disabled. Enable plaintext or ciphertext access, or both.
   Security Protocol	SASL_SSL: SASL is used for authentication. Data is encrypted with an SSL certificate for high-security transmission. SASL_PLAINTEXT: SASL is used for authentication. Data is transmitted in plaintext for better performance. SCRAM-SHA-512 authentication is recommended for plaintext transmission.
   SASL Mechanism	If PLAIN is disabled, the SCRAM-SHA-512 mechanism is used for username and password authentication. If PLAIN is enabled, both the SCRAM-SHA-512 and PLAIN mechanisms are supported. You can select either of them as required. The SASL/PLAIN setting cannot be changed once ciphertext access is enabled. What are SCRAM-SHA-512 and PLAIN mechanisms? SCRAM-SHA-512: uses the hash algorithm to generate credentials for usernames and passwords to verify identities. SCRAM-SHA-512 is more secure than PLAIN. PLAIN: a simple username and password verification mechanism.
   SSL Username	Username for a client to connect to a Kafka instance. A username should contain 4 to 64 characters, start with a letter, and contain only letters, digits, hyphens (-), and underscores (_). The username cannot be changed once ciphertext access is enabled.
   Password	Password for a client to connect to a Kafka instance. A password must meet the following requirements: Contains 8 to 32 characters. Contains at least three types of the following characters: uppercase letters, lowercase letters, digits, and special characters `~!@#$%^&*()-_=+\|[{}];:'",<.>? and spaces, and cannot start with a hyphen (-). Cannot be the username spelled forward or backward.

9. Configure advanced settings.

   Table 5 Advanced configuration parameters

   Parameter	Description
   Instance Name	You can customize a name that complies with the rules: 4–64 characters; starts with a letter; can contain only letters, digits, hyphens (-), and underscores (_).
   Enterprise Project	This parameter is for enterprise users. Enterprise projects facilitate project-level management and grouping of cloud resources and users. The default project is default.
   Capacity Threshold Policy	Specify how messages are processed when the disk usage threshold (95%) is reached. Automatically delete: Messages can be produced and consumed, but 10% of the earliest messages will be deleted to ensure sufficient disk space. This policy is suitable for scenarios where no service interruption can be tolerated. Data may be lost. Stop production: New messages cannot be produced, but existing messages can still be consumed. This policy is suitable for scenarios where no data loss can be tolerated.
   Automatic Topic Creation	Enable automatic Kafka topic creation if needed. If this option is enabled, a topic will be automatically created when a message is produced in or consumed from a topic that does not exist. The default topic parameters are listed in Table 6. For cluster instances, after you change the value of the log.retention.hours (retention period), default.replication.factor (replica quantity), or num.partitions (partition quantity) parameter, the value will be used in later topics that are automatically created. For example, assume that num.partitions is changed to 5, an automatically created topic has parameters listed in Table 6.
   Tags	Tags are used to identify cloud resources. When you have multiple cloud resources of the same type, you can use tags to classify them based on usage, owner, or environment. If your organization has configured tag policies for DMS for Kafka, add tags to Kafka instances based on the policies. If a tag does not comply with the policies, Kafka instance creation may fail. Contact your organization administrator to learn more about tag policies. If you have predefined tags, select a predefined pair of tag key and value. You can click Create predefined tags to go to the Tag Management Service (TMS) console and view or create tags. You can also create new tags by specifying Tag key and Tag value. Up to 20 tags can be added to each Kafka instance. For details about the requirements on tags, see Configuring Kafka Instance Tags.
   Manager Username	You can customize a name that complies with the rules: 4–64 characters; starts with a letter; can contain only letters, digits, hyphens (-), and underscores (_). Enter "manager-test".
   Password	Set the password of Kafka Manager. A password must meet the following requirements: Contains 8 to 32 characters. Contains at least three types of the following characters: uppercase letters, lowercase letters, digits, and special characters `~!@#$%^&*()-_=+\|[{}];:'",<.>? and spaces, and cannot start with a hyphen (-). Cannot be the username spelled forward or backward.
   Description	Enter a Description of the instance for 0–1024 characters.

   Table 6 Topic parameters

   Parameter	Default Value	Modified To
   Partitions	3	5
   Replicas	3	3
   Aging Time (h)	72	72
   Synchronous Replication	Disabled	Disabled
   Synchronous Flushing	Disabled	Disabled
   Message Timestamp	CreateTime	CreateTime
   Max. Message Size (bytes)	10,485,760	10,485,760

10. Click Confirm.
11. Confirm the instance information, and click Submit.
12. Return to the instance list and check whether the Kafka instance has been created.

    It takes 3 to 15 minutes to create an instance. During this period, the instance status is Creating.

    • If the instance is created successfully, its status changes to Running.
    • If the instance is in the Failed state, delete it by referring to Deleting Kafka Instances and try creating another one. If the instance creation fails again, contact customer service.
      

      Instances that fail to be created do not occupy other resources.

Standard Config of a Cluster Kafka Instance

1. Log in to the console.
2. Click in the upper left corner to select a region.

   DMS for Kafka instances in different regions cannot communicate with each other over an intranet. Select a nearest location for low latency and fast access.

3. Click in the upper left corner and choose Application > Distributed Message Service for Kafka to open the Kafka instance list.
4. Click Buy Kafka Instance in the upper right corner of the page.
5. Set basic instance configurations on the Standard Config page.

   Table 7 Basic instance configuration parameters

   Parameter	Description
   Billing Mode	Pay-per-use is a postpaid mode. You can pay after using the service, and will be billed for your usage duration. The fees are calculated in seconds and settled by hour.
   Region	DMS for Kafka instances in different regions cannot communicate with each other over an intranet. Select a nearest location for low latency and fast access.
   AZ	An AZ is a physical region where resources use independent power supply and networks. AZs are physically isolated but interconnected through an internal network. Select one AZ or at least three AZs. The AZ setting is fixed once the instance is created.

6. Configure the following instance specifications:

   Table 8 Instance specifications parameters

   Parameter	Description
   Version	Kafka version, which can be 1.1.0, 2.3.0, 2.7, or 3.x. The version is fixed once the instance is created.
   Architecture	Select Cluster.
   Broker Flavor	Select a broker flavor as required. Maximum number of partitions per broker × Number of brokers = Maximum number of partitions of an instance. If the total number of partitions of all topics exceeds the upper limit of partitions, topic creation fails.
   Brokers	Specify the broker quantity.
   Type	Select the disk type for Kafka data storage. The disk type cannot be changed once the Kafka instance is created.
   Capacity	Specify the disk size for Kafka data storage. The storage space is consumed by message replicas, logs, and metadata. Specify the storage space based on the expected service message size, the number of replicas, and the reserved disk space. Each Kafka broker reserves 33 GB disk space for storing logs and metadata. Disks are formatted when an instance is created. As a result, the actual available disk space is 93% to 95% of the total disk space.

   Figure 1 Instance flavor
   

7. Set the network information.

   Table 9 Instance network parameters

   Parameter	Description
   VPC	Select a created or shared VPC. A VPC provides an isolated virtual network for your Kafka instances. You can configure and manage the network as required. You can click Manage VPC on the right to go to the VPC console and view or create a VPC. After the Kafka instance is created, its VPC cannot be changed.
   Subnet	Select a created subnet. After the Kafka instance is created, its subnet cannot be changed.
   IPv6	This parameter is displayed after IPv6 is enabled for the subnet. The instance with IPv6 enabled can be accessed on a client using IPv6 addresses. The IPv6 setting is fixed once the instance is created.
   Private IP Addresses	Select Auto or Manual. Auto: The system automatically assigns an IP address from the subnet. Manual: Select IP addresses from the drop-down list. If the number of selected IP addresses is less than the number of brokers, the remaining IP addresses will be automatically assigned. This parameter is not displayed when IPv6 is enabled.
   Security Group	Select a created security group. A security group is a set of rules for accessing a Kafka instance. You can click Create Security Group on the right to go to the security group page and view or create a security group. Before accessing a Kafka instance on the client, configure security group rules based on the access mode. For details about security group rules, see Table 2.

8. Configure the instance access mode.

   Table 10 Instance access mode parameters

   Parameter	Description
   Access Method	There are two methods: Plaintext access: Clients connect to the Kafka instance without SASL authentication. Ciphertext access: Clients connect to the Kafka instance with SASL authentication. If you enable Ciphertext Access, specify an SSL username and password. Once enabled, private network access cannot be disabled. Enable plaintext or ciphertext access, or both.
   Security Protocol	SASL_SSL: SASL is used for authentication. Data is encrypted with an SSL certificate for high-security transmission. SASL_PLAINTEXT: SASL is used for authentication. Data is transmitted in plaintext for better performance. SCRAM-SHA-512 authentication is recommended for plaintext transmission.
   SASL Mechanism	If PLAIN is disabled, the SCRAM-SHA-512 mechanism is used for username and password authentication. If PLAIN is enabled, both the SCRAM-SHA-512 and PLAIN mechanisms are supported. You can select either of them as required. The SASL/PLAIN setting cannot be changed once ciphertext access is enabled. What are SCRAM-SHA-512 and PLAIN mechanisms? SCRAM-SHA-512: uses the hash algorithm to generate credentials for usernames and passwords to verify identities. SCRAM-SHA-512 is more secure than PLAIN. PLAIN: a simple username and password verification mechanism.
   SSL Username	Username for a client to connect to a Kafka instance. A username should contain 4 to 64 characters, start with a letter, and contain only letters, digits, hyphens (-), and underscores (_). The username cannot be changed once ciphertext access is enabled.
   Password	Password for a client to connect to a Kafka instance. A password must meet the following requirements: Contains 8 to 32 characters. Contains at least three types of the following characters: uppercase letters, lowercase letters, digits, and special characters `~!@#$%^&*()-_=+\|[{}];:'",<.>? and spaces, and cannot start with a hyphen (-). Cannot be the username spelled forward or backward.

9. Configure advanced settings.

   Table 11 Advanced configuration parameters

   Parameter	Description
   Instance Name	You can customize a name that complies with the rules: 4–64 characters; starts with a letter; can contain only letters, digits, hyphens (-), and underscores (_).
   Enterprise Project	This parameter is for enterprise users. Enterprise projects facilitate project-level management and grouping of cloud resources and users. The default project is default.
   Capacity Threshold Policy	Specify how messages are processed when the disk usage threshold (95%) is reached. Automatically delete: Messages can be produced and consumed, but 10% of the earliest messages will be deleted to ensure sufficient disk space. This policy is suitable for scenarios where no service interruption can be tolerated. Data may be lost. Stop production: New messages cannot be produced, but existing messages can still be consumed. This policy is suitable for scenarios where no data loss can be tolerated.
   Automatic Topic Creation	Enable automatic Kafka topic creation if needed. If this option is enabled, a topic will be automatically created when a message is produced in or consumed from a topic that does not exist. The default topic parameters are listed in Table 12. For cluster instances, after you change the value of the log.retention.hours (retention period), default.replication.factor (replica quantity), or num.partitions (partition quantity) parameter, the value will be used in later topics that are automatically created. For example, assume that num.partitions is changed to 5, an automatically created topic has parameters listed in Table 12.
   Tags	Tags are used to identify cloud resources. When you have multiple cloud resources of the same type, you can use tags to classify them based on usage, owner, or environment. If your organization has configured tag policies for DMS for Kafka, add tags to Kafka instances based on the policies. If a tag does not comply with the policies, Kafka instance creation may fail. Contact your organization administrator to learn more about tag policies. If you have predefined tags, select a predefined pair of tag key and value. You can click View predefined tags to go to the Tag Management Service (TMS) console and view or create tags. You can also create new tags by specifying Tag key and Tag value. Up to 20 tags can be added to each Kafka instance. For details about the requirements on tags, see Configuring Kafka Instance Tags.
   Manager Username	You can customize a name that complies with the rules: 4–64 characters; starts with a letter; can contain only letters, digits, hyphens (-), and underscores (_). Enter "manager-test".
   Password	Set the password of Kafka Manager. A password must meet the following requirements: Contains 8 to 32 characters. Contains at least three types of the following characters: uppercase letters, lowercase letters, digits, and special characters `~!@#$%^&*()-_=+\|[{}];:'",<.>? and spaces, and cannot start with a hyphen (-). Cannot be the username spelled forward or backward.
   Description	Enter a Description of the instance for 0–1024 characters.

   Table 12 Topic parameters

   Parameter	Default Value	Modified To
   Partitions	3	5
   Replicas	3	3
   Aging Time (h)	72	72
   Synchronous Replication	Disabled	Disabled
   Synchronous Flushing	Disabled	Disabled
   Message Timestamp	CreateTime	CreateTime
   Max. Message Size (bytes)	10,485,760	10,485,760

10. In Summary on the right, view the selected instance configuration.
11. Click Confirm.
12. Confirm the instance information, and click Submit.
13. Return to the instance list and check whether the Kafka instance has been created.

    It takes 3 to 15 minutes to create an instance. During this period, the instance status is Creating.

    • If the instance is created successfully, its status changes to Running.
    • If the instance is in the Failed state, delete it by referring to Deleting Kafka Instances and try creating another one. If the instance creation fails again, contact customer service.
      

      Instances that fail to be created do not occupy other resources.

Purchasing a Kafka Instance with Same Configurations

To purchase another Kafka instance with the same configuration as the current one, reuse the current configuration through the Buy Another function.

1. Log in to the console.
2. Click in the upper left corner and select a region.

   DMS for Kafka instances in different regions cannot communicate with each other over an intranet. Select a nearest location for low latency and fast access.

3. Click in the upper left corner and choose Application > Distributed Message Service for Kafka to open the Kafka instance list.
4. Select a target Kafka instance and choose More > Buy Another in the Operation column.
5. Adjust the automatically replicated parameter settings as required. For details, see Standard Config of a Cluster Kafka Instance.

   For security purposes, parameter settings involved in the following scenarios will not be replicated and a re-configuration is required:

   • SSL username and password of a Kafka instance with ciphertext access enabled.
   • Public IP addresses of a Kafka instance with public access enabled.
   • Manually assigned IP addresses of a Kafka instance. Select Manual for Private IP Addresses again and specify private IP addresses.
   • Name of a target Kafka instance.

6. In Summary on the right, view the selected instance configuration.
7. Click Confirm.
8. Confirm the instance information, and click Submit.
9. Return to the instance list and check whether the Kafka instance has been created.

   It takes 3 to 15 minutes to create an instance. During this period, the instance status is Creating.

   • If the instance is created successfully, its status changes to Running.
   • If the instance is in the Failed state, delete it by referring to Deleting Kafka Instances and try creating another one. If the instance creation fails again, contact customer service.
     

     Instances that fail to be created do not occupy other resources.