Multi-tenant Management and Resource Isolation

Overview

GaussDB(for MySQL) provides multi-tenant management to maximize database resource utilization. Data is isolated among tenants. Different tenants can only access their own data. There are tenant-level resource isolation and user-level resource isolation to avoid resource waster and improve performance. Resources can be dynamically adjusted to process workload peaks and troughs of different tenants or users in a timely manner.

The following figure shows the principle of multi-tenant management.

Figure 1 Multi-tenant management principle

Basic Concepts

Table 1 describes the terms for tenant-level and user-level resource isolation.

Table 1 Term description

Level	Term
Tenant-level	Tenant Tenants are under a DB instance and above databases and users. Tenants are used for data isolation and resource isolation. However, databases can be accessed as a user. There are system tenants and common tenants. System tenant (sys_tenant) System tenants are designed to accommodate users that exist before the implementation of the multi-tenant management. By default, these existing users belong to system tenants and are also referred to as system users. If a user of a system tenant can connect to and access the DB instance, the user can access the databases of all tenants. Common tenant (user_tenant) Common tenants need to be created in the system tenants. The users and databases of each common tenant are completely isolated from those of the other common tenants. In addition, common tenants cannot access databases of the system tenants. During vCPU scheduling, common tenants are classified into dedicated tenants and shared tenants based on whether the value of min_cpu is greater than 0. Dedicates tenants: min_cpu > 0. vCPUs allocated to dedicated tenants at any time are not less than the value of min_cpu. Shared tenants: min_cpu = 0. The system preferentially ensures the resource requests of dedicated tenants and then allocates the remaining resources to shared tenants. In addition, the system reserves some vCPUs (specified by mt_shared_cpu_reserved) for shared tenants. You can change the value of min_cpu to change the roles between dedicated and shared tenants. Resource configuration (resource_config) A resource configuration indicates the resources that can be used by a tenant, enabling tenant-level resource isolation. Currently, only vCPUs can be isolated and scheduled.
User-level	Users are under DB instances and tenants. A tenant can have multiple users. Figure 2 User-level resource configuration relationship Resource consumer group (consumer_group) A set of users under the same resource configuration. Multiple users can belong to the same resource consumer group. Users in the same resource consumer group share resource configurations. Resource plan directive (plan_directive) Specific resource configuration. One resource plan directive is bound to one resource consumer group to describe the resource configuration of the consumer group. Resource plan (plan) Resource configuration set. One resource plan is bound to one or more resource plan directives. Enabling or disabling a resource plan can make the corresponding resource plan directive valid or invalid.

Constraints

• Multi-tenant management and resource isolation can only isolate tenant data, tenant vCPUs, and user vCPUs.
• The kernel version of your GaussDB(for MySQL) instance must be 2.0.54.240600 or later.
• Thread Pool must be enabled.
• Database names and usernames in a DB instance do not contain the at sign (@).
• Serverless instances do not support multi-tenant management and resource isolation.
• Tenant migration across instances:
  After the multi-tenant management and resource isolation is enabled, DRS can migrate data of all tenants. However, DRS does not synchronize multi-tenant metadata, so tenant information is not synchronized to the destination instance. To migrate a tenant from an instance to another instance, perform the following steps:

  1. Select an instance that support multi-tenant management as the destination instance and manually create a tenant for the destination instance.
  2. Use DRS data synchronization to create a database-level synchronization task. (If the tenant names at the source and destination ends are changed, you need to change the destination database name.)

     

  3. Synchronize data.
• Binlog:

  Currently, tenant isolation cannot be implemented using binlogs. If common tenants pull binlogs, data among tenants will not be isolated. Users of common tenants are not allowed to pull binlogs.

  GaussDB(for MySQL) uses APIs related to tenants and resource isolation to record binlogs in row format. If binlog_format is not set to row, tenant and resource configuration information cannot be synchronized using binlogs.

• Proxy:
  • The show processlist command can be used with a proxy. When the command is used with a proxy, the command output displays node information.
  • The HTAP standard and lightweight editions do not support databases containing at signs (@). When a database of a common tenant is migrated to the HTAP engine, the destination database name will be changed. After Auto Assign Requests to Column Store or Row Store Nodes is enabled, the proxy requires that the source and destination database names be the same, so this function must be disabled for migrating databases of common tenants.
• Backup and restoration:

  When you restore data of an instance with multi-tenancy enabled to a new instance with multi-tenancy disabled, users and databases with at signs (@) in their names cannot be created for the new instance.

• Compatibility:
  • If multi-tenancy is enabled and then disabled, the names of databases or users cannot contain at signs (@).
  • For a common tenant, the maximum length of a database name is reduced from 64 characters to 50 characters, and the maximum length of a user name is reduced from 32 characters to 20 characters.
  • Currently, common tenants can only access information_schema. Other system databases cannot be accessed by common tenants.
  • Common tenants do not support tablespace-related syntax.

Enabling the Multi-Tenant Mode

1. Log in to the management console.
2. Click in the upper left corner and select a region and project.
3. Click in the upper left corner of the page and choose Databases > GaussDB(for MySQL).
4. On the Instances page, click the instance name to go to the Basic Information page.
5. In the Instance Information area, click under Multi-tenant. In the displayed dialog box, click OK.

   Before enabling multi-tenancy, ensure that the existing database names and usernames do not contain at signs (@), or the function fails to be enabled.

Resource Management

Resource configurations and tenants are in one-to-many relationship. When a tenant is bound to a resource configuration, the vCPUs used by users of the tenant is restricted.

• Create a resource configuration.

  CREATE resource_config config_name MAX_CPU [=] {max_cpu_value} [MIN_CPU [=] {min_cpu_value}];

• Alter a resource configuration.

  ALTER resource_config config_name MAX_CPU [=] {max_cpu_value} [MIN_CPU [=] {min_cpu_value}];

• Drop a resource configuration.

  DROP resource_config config_name;

• Query a resource configuration.

  SELECT * FROM information_schema.DBA_RSRC_TENANT_RESOURCE_CONFIGS;

• Only the root user can call the APIs for creating, altering, and dropping resource configurations.
• Parameter description:
  • config_name: resource configuration name. The value can contain up to 64 characters. Only uppercase letters, lowercase letters, digits, and underscores (_) are allowed.
  • MAX_CPU: maximum number of vCPUs that can be used by the resource configuration. The value ranges from 0.1 to the value of mt_flavor_cpu. The granularity is 0.1U.
  • MIN_CPU: minimum number of vCPUs that can be used by the resource configuration. This parameter is optional. The default value is the same as the value of MAX_CPU. Value range: 0 to the value of MAX_CPU. The granularity is 0.1U. If the value is set to 0, the tenant is a shared tenant. If the value is greater than 0, the tenant is a dedicated tenant.
• When a resource configuration is updated, if it has been bound to a tenant and the updated MIN_CPU value is greater than the original MIN_CPU value, check whether the new value meets the resource constraints, or the resource constraints are not verified.
• If a tenant is using a resource configuration, the resource configuration cannot be deleted.
• In actual use, the resource allocation of shared tenants does not depend on MAX_CPU. Resources are randomly contested.
• During vCPU contention, resources are allocated to tenants based on the value of MIN_CPU specified for each tenant. However, there is a certain error, which is usually within 1U.
• In give specifications, the performance of an instance with multi-tenancy enabled is lower than that of an instance with multi-tenancy disabled. In actual use, when multiple tenants preempt resources, the instance performance may be even lower.

Tenant Management

When creating a tenant, you need to bind it to a resource configuration to restrict the vCPUs used by users of the tenant.

• Create a tenant.

  CREATE TENANT tenant_name RESOURCE_CONFIG config_name [COMMENT [=] 'comment_string'];

• Alter a tenant.

  ALTER TENANT tenant_name RESOURCE_CONFIG config_name [COMMENT [=] 'comment_string'];

• Drop a tenant.

  DROP TENANT tenant_name;

• Query a tenant.

  SELECT * FROM information_schema.DBA_RSRC_TENANT;

• Only the root user can call the APIs for creating, altering, and dropping tenants.
• Creating a tenant:
  • The value of tenant_name can contain up to 10 characters. Only lowercase letters, digits, and underscores (_) are allowed.
  • When a tenant is created, the system checks resource constraints to ensure that the sum of the MIN_CPU values in the resource configurations of all tenants meets the resource constraints.
• Altering a tenant:
  • If the MIN_CPU value of the newly bound resource configuration is at least that of the original resource configuration, the system checks the resource constraints. Ensure that the sum of the MIN_CPU values in the resource configurations of all tenants meets the resource constraints.
  • If the MIN_CPU value of a resource configuration bound to a dedicated tenant is 0, the tenant becomes a shared tenant and the metadata related to user-level resource isolation associated with the tenant is deleted.
• Dropping a tenant:
  • Before dropping a tenant, you need to ensure that the databases and users of the tenant have been deleted, or the tenant cannot be deleted.
  • After a tenant is deleted, the user-level resource isolation configuration associated with the tenant will be deleted.

User Management

After the multi-tenant mode is enabled, there are users of system tenants and users of common tenants. Existing users belong to system tenants. New users can belong to system tenants or common tenants based on the interface semantics.

• Managing users under a system tenant

  Creating a user

  Create a user for a system tenant.

  CREATE user [IF NOT EXISTS] user_name@host;

  Create a user for a common tenant.

  CREATE user [IF NOT EXISTS] 'user_name@tenant_name'@host; 

  Renaming a user

  Rename a user under a system tenant.

  RENAME USER user_from@host1 TO user_to@host2; 

  Rename a user under a common tenant.

  RENAME USER 'user_from@tenant_name'@host1 TO 'user_to@tenant_name'@host2; 

  Dropping a user

  Drop a user from a system tenant.

  DROP USER [IF EXISTS] user_name@host; 

  Drop a user from a common tenant.

  DROP USER [IF EXISTS] 'user_name@tenant_name'@host;

  Authorizing a user

  Grant the priv_type permissions of tenant_1 to user1@tenant_1.

  GRANT priv_type ON *.* to 'user_1@tenant_1'@'%' with grant option;

  View permissions.

  SHOW grants for 'user_1@tenant_1';

• Managing users under a common tenant

  Creating a user

  Create a user for the current tenant.

  CREATE user [IF NOT EXISTS] user_name@host; 

  Renaming a user

  RENAME USER user_from@host1 TO user_to@host2;

  Dropping a user

  DROP USER [IF EXISTS] user_name@host;

  Authorizing a user

  Grant the priv_type permissions of the current tenant to user1.

  GRANT priv_type ON *.* to 'user_1'@'%' with grant option;

  View permissions.

  SHOW grants for 'user_1';

• When creating or dropping a user of a common tenant under a system tenant, you need to use the user_name@tenant_name format.
• The username of a common tenant can contain up to 20 characters.
• Some special users cannot be created in a tenant, including mysql.sys, mysql.session, mysql.infoschema, and users reserved in the rds_reserved_users parameter.
• When renaming a user under a common tenant, ensure that the values of tenane_name in user_from and user_to are the same, or an error is returned.
• When multi-tenancy is disabled, users under a common tenant cannot be renamed.

Database Management

There are databases of system tenants and databases of common tenants. System tenants can access all databases, and common tenants can only access their own databases.

• Managing databases under a system tenant

  Creating a database

  Create a database for a system tenant.

  CREATE DATABASE [IF NOT EXISTS] `db_name`;

  Create a database for a common tenant.

  CREATE DATABASE [IF NOT EXISTS] `db_name@tanent_name`; 

  Dropping a database

  Drop a database from a system tenant.

  DROP DATABASE [IF EXISTS] `db_name`;

  Drop a database from a common tenant.

  DROP DATABASE [IF EXISTS] `db_name@tanent_name`;

• Managing databases under a common tenant

  Create a database for the current tenant.

  CREATE DATABASE [IF NOT EXISTS] 'db_name'; 

  Drop a database from the current tenant.

  DROP DATABASE [IF EXISTS] 'db_name';

• When creating or dropping a database of a common tenant under a system tenant, you need to perform operations on the database in db_name@tenant_name mode.
• Currently, common tenants can only access the INFORMATION_SCHEMA system database. They cannot access the PERFORMANCE_SCHEMA, SYS, and MySQL system databases.
• Some special databases, such as INFORMATION_SCHEMA, PERFORMANCE_SCHEMA, MYSQL, SYS, and __recyclebin, cannot be created in a tenant.

• Allocating tenants to existing databases

  To ensure compatibility, after the upgrade or migration to a multi-tenant instance, the existing databases are under system tenants by default. You can use the following syntax to allocate the existing databases to specified tenants. After multi-tenancy is enabled, you can use the following syntax to allocate databases that are created by system tenants and are not allocated to common tenants to corresponding tenants.

  Allocating a database

  Allocate a database to common tenant tenant_name.

  ALTER DATABASE db_name TENANT = `tenant_name`;

  Reclaim a database to a system tenant.

  ALTER DATABASE db_name TENANT = ``;

  Querying the mappings

  SELECT * FROM information_schema.DBA_RSRC_TENANT_DB;

  

  • Only the root user can call the API for allocating a database.
  • If a database is created after multi-tenancy is enabled, the database named in the format of db_name@tenant_name cannot be allocated and the API for allocating a database cannot be called. Otherwise, an error is returned.
  • If the tenant is not found and is not empty, the API for allocating a database returns an error.
• Connecting to a database as a user of a tenant

  Under a system tenant, the original connection mode remains unchanged.

  Under a common tenant, the user must be in the format of user_name@tenant_name. The database must be in the format of db_name or db_name@tenant_name.

  mysql --host=**** -u user1@tenant_1 -D db1 -p pwssword;
  mysql --host=**** -u user1@tenant_1 -D db1@tenant_1 -p pwssword;

  After the connection is successful, the user is restricted by the resources of the corresponding tenant.

User-level Resource Configurations

By default, users under a tenant share the resources of the tenant. To restrict user-level resources, you can call the API in this section.

• Managing resource consumer groups (consumer_group)

  Connect to a database as a user of a tenant to manage resource consumer groups.

  Creating a consumer group

  dbms_resource_manager.create_consumer_group (
     consumer_group     CHAR(128),
     comment            CHAR(2000));

  Binding a consumer group

  If the value of consumer_group is not '', the user is bound to a consumer group.

  dbms_resource_manager.set_consumer_group_mapping (
     attribute        CHAR(128), 
     value            varbinary(128), 
     consumer_group   CHAR(128));

  Unbinding a consumer group

  If the value of consumer_group is '', the user is unbound from the consumer group.

  dbms_resource_manager.set_consumer_group_mapping (
     attribute        CHAR(128), 
     value            varbinary(128), 
     '');

  Deleting a consumer group

  dbms_resource_manager.delete_consumer_group (
      consumer_group   CHAR(128));

  Querying a consumer group

  The DBA_RSRC_CONSUMER_GROUPS view records the association between tenants and consumer groups.

  select * from information_schema.DBA_RSRC_CONSUMER_GROUPS;

  Viewing consumer group mappings

  The DBA_RSRC_GROUP_MAPPINGS view records the association between tenants, users, and consumer groups.

  select * from information_schema.DBA_RSRC_GROUP_MAPPINGS;

  

  • Sharded tenants cannot use the APIs for creating, binding, unbinding, or deleting consumer groups.
  • Parameter description:

    consumer_group: consumer group name. Only uppercase letters, lowercase letters, digits, and underscores (_) are allowed. You can set this parameter to '' when unbinding a consumer group.

    comment: description of the resource consumer group. The value can be ''.

    attribute: mapping attribute to be added or modified. The current version supports only USER.

    value: mapping attribute to be added or modified. The current version supports only usernames.

  • When a consumer group is deleted, the plan directive and consumer group mapping entries corresponding to the resource consumer group are also deleted.
  • If multi-tenancy is enabled, when a user is deleted, the consumer group items associated with the user are also deleted.
  • If multi-tenancy is enabled, when the user is renamed, the consumer group items associated with a user are also updated.
  • If multi-tenancy is enabled again, if the corresponding user is invalid, the invalid items associated with the user are automatically deleted.
  • After multi-tenancy is disabled, if a user is deleted and then a user with the same name is created, the items corresponding to the user will be retained when multi-tenancy is enabled later.

Resource Plan Management

Connect to a database as a user of a tenant to manage resource plans.

• Create a resource plan.

  dbms_resource_manager.create_plan (
     plan_name         VARCHAR(128), 
     comment           VARCHAR(2000));

• Enable a resource plan.

  dbms_resource_manager.set_resource_manager_plan(
      plan_name     VARCHAR(128));

• Disable a resource plan.

  dbms_resource_manager.set_resource_manager_plan ('');

• Delete a resource plan.

  dbms_resource_manager.delete_plan (
      plan_name    VARCHAR(128));

• Query a resource plan.

  The DBA_RSRC_PLANS view records details about the current plan.

  SELECT * FROM information_schema.DBA_RSRC_PLANS;

  

  • Sharded tenants cannot use the APIs for creating, enabling, disabling, or deleting resource plans.
  • Parameter description:

    plan_name: resource plan name. Only uppercase letters, lowercase letters, digits, and underscores (_) are allowed.

    comment: description of the resource plan. The value can be ''.

  • If you delete an enabled resource plan, it will be left empty and the corresponding resource plan directive will be deleted.
  • mt_resource_plan_num: number of plans. By default, there are up to 128 plans.

Resource Plan Directive Management

Connect to a database as a user of a tenant to manage resource plan directives.

• Creating a resource plan directive

  dbms_resource_manager.create_plan_directive (
     plan                    CHAR(128), 
     group_or_subplan        CHAR(128), 
     comment                 VARCHAR(2000), 
     mgmt_p1                 bigint(20),
     utilization_limit       bigint(20));

• Updating a resource plan directive

  dbms_resource_manager.update_plan_directive (
     plan                    CHAR(128), 
     group_or_subplan        CHAR(128), 
     new_comment             VARCHAR(2000), 
     new_mgmt_p1             bigint(20),
     new_utilization_limit   bigint(20));

• Deleting a resource plan directive

  dbms_resource_manager.delete_plan_directive (
      plan                   CHAR(128), 
      group_or_subplan       VARCHAR(128));

• Querying a resource plan directive

  DBA_RSRC_PLAN_DIRECTIVES records the association between the plans and consumer groups as well as the resource configurations of the consumer groups.

  SELECT * FROM information_schema.DBA_RSRC_PLAN_DIRECTIVES;

  

  • Sharded tenants cannot use the APIs for creating, updating, or deleting resource plan directives.
  • Parameter description:

    plan: plan name.

    group_or_subplan: consumer group name.

    comment: description of the resource plan directive. The value can be ''.

    mgmt_p1: committed vCPU percentage allocated to the consumer group when the system is fully loaded. The value range is [0, 100] (100: 100% vCPUs of the tenant are used).

    utilization_limit: upper limit of vCPUs used by the consumer group. The value range is [1, 100]. The value 100 indicates that a maximum of all vCPUs of a tenant can be used. The value 70 indicates that a maximum of 70% vCPUs of a tenant can be used.

  • If you delete a plan directive that has been enabled, the resource configuration of the corresponding user will become invalid.
  • The total resources used by all users in a consumer group cannot exceed the resource limit of the current consumer group. For example, if a tenant has users user1 and user2 (they belong to consumer_group1) and UTILIZATION_LIMIT of consumer_group1 is 70, user1 and user2 can use up to 70% of the vCPU resources of the current tenant.

User-level Configuration Clearing

dbms_resource_manager.clear_all_configs();

vCPU Usage Statistics

• User-level vCPU usage

  The information_schema.cpu_summary_by_user table is added to display the vCPU usage of each user under a tenant.

  SELECT * FROM information_schema.cpu_summary_by_user;

  

  • A consumer group must be configured for the current user in advance.
  • The column names in the query result are described as follows:

    TENANT_NAME: name of the tenant the user belongs to.

    USER_NAME: username.

    CPU_USAGE: vCPU usage of the user, that is, the ratio of the vCPUs used by the user to the MAX_CPU value configured for the tenant. For example, if the MAX_CPU value configured for the current tenant is 4 and the user actually uses 2 vCPUs, the value of CPU_USAGE is 50%.

• Tenant-level vCPU usage

  The information_schema.cpu_summary_by_tenant table is added to display the vCPU usage of each tenant.

  SELECT * FROM information_schema.cpu_summary_by_tenant;

  

  The column names in the query result are described as follows:

  TENANT_NAME: tenant name.

  CPU_USAGE: vCPU usage of the tenant, that is, the ratio of the vCPUs used by the tenant to the MAX_CPU value. For example, if the MAX_CPU value configured for the current tenant is 4 and the tenant actually uses 2 vCPUs, the value of CPU_USAGE is 50%.