Configuring a GaussDB Data Source

Scenario

This section describes how to add a GaussDB JDBC data source on the HSConsole page.

Prerequisites

• The domain name of the cluster where the data source is located must be different from the HetuEngine cluster domain name.
• The cluster where the data source is located and the HetuEngine cluster nodes can communicate with each other.
• A HetuEngine compute instance has been created.

Procedure

1. Log in to FusionInsight Manager as a HetuEngine administrator and choose Cluster > Services > HetuEngine. The HetuEngine service page is displayed.
2. In the Basic Information area on the Dashboard page, click the link next to HSConsole WebUI. The HSConsole page is displayed.
3. Choose Data Source.
4. Click Add Data Source. Configure parameters on the Add Data Source page.
   1. Configure Basic Information. For details, see Table 1.

      Table 1 Basic Information

      Parameter	Description	Example Value
      Name	Name of the data source to be connected. The value can contain only letters, digits, and underscores (_) and must start with a letter.	gaussdb_1
      Data Source Type	Type of the data source to be connected. Choose JDBC > GAUSSDB-A.	GAUSSDB-A
      Description	Description of the data source. The value can contain only letters, digits, commas (,), periods (.), underscores (_), spaces, and line breaks.	-

   2. Configure parameters in the GAUSSDB-A Configuration area. For details, see Table 2.

      Table 2 GAUSSDB-A Configuration

      Parameter	Description	Example Value
      Driver	The default value is gaussdba.	gaussdba
      JDBC URL	JDBC URL for connecting to GaussDB A. The format is as follows: jdbc:postgresql://CN service IP address:Port number/Database name	jdbc:postgresql://10.0.136.1:25308/postgres
      Username	Username for connecting to the GaussDB data source.	Change the value based on the username being connected with the data source.
      Password	Password for connecting to the GaussDB data source.	Change the value based on the username and password for connecting to the data source.
      GaussDB User Information Configuration	Configure multiple GaussDB usernames and passwords in the format of dataSourceUser and password key-value pairs.	-
      HetuEngine-GaussDB User Mapping Configuration	Multiple HetuEngine accounts are configured in the format of hetuUser and dataSourceUser key-value pairs, corresponding to one of the users configured in the GaussDB User Information Configuration area. When different HetuEngine users are used to access GaussDB, different GaussDB usernames and passwords can be used.	-

   3. Configure the GaussDB data source user information. For details, see Table 3.
      GaussDB User Information Configuration and HetuEngine-GaussDB User Mapping Configuration must be used together. When HetuEngine is connected to the GaussDB data source, HetuEngine users can have the same permissions of the mapped GaussDB data source user through mapping. Multiple HetuEngine users can correspond to one GaussDB user.

      Table 3 GaussDB User Information Configuration

      Parameter	Description	Example Value
      Data Source User	Data Source User The value can contain only letters, digits, underscores (_), hyphens (-), and periods (.), and must start with a letter or underscore (_). The minimum length is 2 characters and the maximum length is 100 characters.	If the data source user is set to gaussuser1, a HetuEngine user mapped to gaussuser1 must exist. For example, create hetuuser1 and map it to gaussuser1.
      Password	User authentication password of the corresponding data source.	-

   4. Configure the HetuEngine-GaussDB user mapping. For details, see Table 4.

      Table 4 HetuEngine-GaussDB User Mapping Configuration

      Parameter	Description	Example Value
      HetuEngine User	HetuEngine username. The value can contain only letters, digits, underscores (_), hyphens (-), and periods (.), and must start with a letter or underscore (_). The minimum length is 2 characters and the maximum length is 100 characters.	hetuuser1
      Data Source User	Data source user. The value can contain only letters, digits, underscores (_), hyphens (-), and periods (.), and must start with a letter or underscore (_). The minimum length is 2 characters and the maximum length is 100 characters.	gaussuser1 (data source user configured in Table 3)

   5. Modify custom configurations.
      • You can click Add to add custom configuration parameters. Configure custom parameters of the GaussDB data source. For details, see Table 5.

        Table 5 Custom parameters of the GaussDB data source

        Parameter	Description	Example Value
        use-connection-pool	Whether to use the JDBC connection pool.	true
        jdbc.connection.pool.maxTotal	Maximum number of connections in the JDBC connection pool.	8
        jdbc.connection.pool.maxIdle	Maximum number of idle connections in the JDBC connection pool.	8
        jdbc.connection.pool.minIdle	Minimum number of idle connections in the JDBC connection pool.	0
        jdbc.pushdown-enabled	true: SQL statements can be pushed down to the data source for execution. false: SQL statements are not pushed down to the data source for execution. As a result, more network and computing resources are consumed.	true
        jdbc.pushdown-module	The push-down function should be enabled in advance. DEFAULT: No operator is pushed down. BASE_PUSHDOWN: Only operators such as Filter, Aggregation, Limit, TopN, and Projection are pushed down. FULL_PUSHDOWN: All supported operators are pushed down.	DEFAULT
        source-encoding	GaussDB data source encoding mode.	UTF-8
        multiple-cnn-enabled	Whether to use the GaussDB multi-CN configuration. To use it, ensure that the JDBC connection pool is disabled and the JDBC URL format is as follows: jdbc:postgresql://host:port/database,jdbc:postgresql://host:port/database,jdbc:postgresql://host:port/database.	false
        parallel-read-enabled	Whether to use the parallel data read function. If the parallel data read function is enabled, the actual number of splits is determined based on the node distribution and the value of max-splits. Multiple connections to the data source will be created for parallel read operations. The dependent data source should support the load.	false
        split-type	Type of the parallel data read function. NODE: The degree of parallelism (DOP) is categorized based on the GaussDB data source DataNodes. PARTITION: The DOP is categorized based on table partitions. INDEX: The DOP is categorized based on table indexes.	NODE
        max-splits	Maximum degree of parallelism.	5
        use-copymanager-for-insert	Whether to use CopyManager for batch import.	false
        unsupported-type-handling	If the connector does not support the data of a certain type, convert it to VARCHAR. After the CONVERT_TO_VARCHAR parameter is configured, the data of BIT VARYING, CIDR, MACADDR, INET, OID, REGTYPE, REGCONFIG and POINT types are converted to the varchar type during query and data of these types can only be read. The default value is IGNORE, indicating that unsupported types will be not displayed in the result.	CONVERT_TO_VARCHAR
        max-bytes-in-a-batch-for-copymanager-in-mb	Maximum volume of data imported by CopyManager in a batch, in MB.	10

      • You can click Delete to delete custom configuration parameters.
   6. Click OK.
   

   • The UPDATE and DELETE syntaxes do not support filtering clauses containing cross-catalog conditions, for example, UPDATE mppdb.table SET column1=value WHERE column2 IN (SELECT column2 from hive.table).
   • To use the DELETE syntax, set jdbc.pushdown-enabled to true and unsupported-type-handling to CONVERT_TO_VARCHAR.
   • The DELETE syntax does not support filtering clauses containing subqueries, for example, DELETE FROM mppdb.table WHERE column IN (SELECT column FROM mppdb.table1).
   • HetuEngine supports a maximum precision of 38 digits for GaussDB data sources of the NUMBER data type