Help Center/ ROMA Connect/ API Reference/ Data Integration APIs/ Data Source Management/ Querying Data Sources
Updated on 2023-06-29 GMT+08:00
View PDF
Share

Querying Data Sources

Function

This API is used to query a data source.

URI

GET /v2/{project_id}/fdi/instances/{instance_id}/datasources

Table 1 Path Parameters

Parameter

Mandatory

Type

Description

project_id

Yes

String

Project ID. For details about how to obtain the project ID, see Appendix > Obtaining a Project ID in the ROMA Connect API Reference.

Minimum: 1

Maximum: 64

instance_id

Yes

String

Instance ID.

Minimum: 1

Maximum: 64
Table 2 Query Parameters

Parameter

Mandatory

Type

Description

limit

No

Integer

Number of records displayed on each page. The maximum value is 999. Excessive records are not returned.

Minimum: 0

Maximum: 500

offset

No

Integer

Offset of pagination in pagination query. The query starts from this offset. If the offset is less than 0, the value is automatically converted to 0.

Minimum: 0

Maximum: 999999

datasource_type

No

String

Data source type.

  • DB2

  • GAUSS100

  • GAUSS200

  • MYSQL

  • ORACLE

  • POSTGRESQL/OPENGAUSS

  • SQL SERVER

  • GAUSSDB(FOR MYSQL)

  • DIS

  • DWS

  • HIVE

  • MRS HBASE

  • MRS HDFS

  • MRS HIVE

  • MRS KAFKA

  • FTP

  • OBS

  • ACTIVEMQ

  • ARTEMISMQ

  • IBM MQ

  • KAFKA

  • MQS

  • RABBITMQ

  • HANA

  • MONGODB

  • REDIS

  • API

  • WEBSOCKET

  • LDAP

  • SAP

  • SNMP

  • AOMDP

  • ROCKETMQ

  • DM

  • CLICKHOUSE

Minimum: 0

Maximum: 30

sort_field

No

String

Sorting field (CREATED_DATE).

Minimum: 0

Maximum: 64

sort_type

No

String

Sorting type of the queried data source. Sorting can be in ascending or descending order. The value can be empty.

Minimum: 0

Maximum: 10

name

No

String

Data source name. Fuzzy match is supported.

Minimum: 0

Maximum: 30

app_id

No

String

Integration application ID.

Minimum: 0

Maximum: 64

custom_plugin_id

No

String

Connector ID.

Minimum: 0

Maximum: 64

customized

No

String

Whether the data source is customized.

Minimum: 0

Maximum: 5

Request Parameters

Table 3 Request header parameters

Parameter

Mandatory

Type

Description

X-Auth-Token

Yes

String

User token, which can be obtained by calling the IAM API (value of X-Subject-Token in the response header).

Response Parameters

Status code: 200

Table 4 Response body parameters

Parameter

Type

Description

total

Long

Number of objects that meet the search criteria.

Minimum: 0

Maximum: 1000000

size

Integer

Size of the returned object.

Minimum: 0

Maximum: 200

entities

Array of DataSourceRsp objects

Returned entity object.
Table 5 DataSourceRsp

Parameter

Type

Description

datasource_id

String

Data source ID.

Minimum: 4

Maximum: 64

datasource_name

String

Data source name

Minimum: 4

Maximum: 255

datasource_type

String

Data source type.

  • DB2

  • GAUSS100

  • MYSQL

  • ORACLE

  • POSTGRESQL/OPENGAUSS

  • SQL SERVER

  • GAUSSDB(FOR MYSQL)

  • DIS

  • DWS

  • HIVE

  • MRS HBASE

  • MRS HDFS

  • MRS HIVE

  • MRS KAFKA

  • FTP

  • OBS

  • ACTIVEMQ

  • ARTEMISMQ

  • IBM MQ

  • KAFKA

  • MQS

  • RABBITMQ

  • HANA

  • MONGODB

  • REDIS

  • API

  • WEBSOCKET

  • LDAP

  • SAP

  • SNMP

Minimum: 1

Maximum: 32

vpc_id

String

ID of the VPC to which the data source belongs.

Minimum: 4

Maximum: 64

app_id

String

ID of the application to which the data source belongs.

Minimum: 4

Maximum: 255

app_name

String

Name of the application to which the data source belongs.

Minimum: 4

Maximum: 255

instance_id

String

ID of the instance to which the data source belongs.

Minimum: 4

Maximum: 255

create_time

Integer

Data source creation time.

Minimum: 0

Maximum: 99999999999999999

update_time

Integer

Data source modification time.

Minimum: 0

Maximum: 99999999999999999

custom_plugin_id

String

ID of the connector to which the data source belongs.

Minimum: 0

Maximum: 64

content

Content object

Data source details.

description

String

Data source description.

Minimum: 4

Maximum: 2147483647

app_permission

Array of strings

Integration application permission.

  • read

  • access

  • delete

  • modify
Table 6 Content

Parameter

Type

Description

gauss100_version

String

Gauss100 version.

  • V100R003C20

  • V300R001C00

Minimum: 1

Maximum: 64

host

String

Host IP address.

  • This parameter is mandatory when the data source is DWS, HANA, RABBITMQ, SAP, SNMP or IBMMQ.

  • This parameter is mandatory when the data source is MYSQL, ORACLE, SQLSERVER, DB2, GAUSS100, GaussDB(for MySQL), ARTEMISMQ, POSTGRESQL/OPENGAUSS or HIVE and mode is set to default.

  • The initial value is empty. The host is generated after the configuration task is started.

Minimum: 4

Maximum: 255

port

String

Port number. The value ranges from 0 to 65535.

  • This parameter is mandatory when the data source is DWS, HANA, RABBITMQ, SAP, SNMP, IBMMQ or OBS.

  • This parameter is mandatory when the data source is MYSQL, ORACLE, SQLSERVER, DB2, GAUSS100, GaussDB(for MySQL), ARTEMISMQ, POSTGRESQL/OPENGAUSS or HIVE and mode is set to default.

Minimum: 1

Maximum: 65535

database_name

String

Database name.

  • This parameter is mandatory when the data source is DWS, HANA, RABBITMQ, SAP, SNMP, IBMMQ or OBS.

  • This parameter is mandatory when the data source is MYSQL, SQLSERVER, DB2, GAUSS100, GaussDB(for MySQL), ARTEMISMQ, POSTGRESQL/OPENGAUSS or HIVE and mode is set to default.

  • This parameter is mandatory when the data source is ORACLE and mode is set to default or multiAddress.

Minimum: 1

Maximum: 2147483647

redis_database

String

Redis data source type, which is a database ID containing only digits.

Minimum: 1

Maximum: 1000

user_name

String

User name for accessing the service.

  • This parameter is mandatory when the data source is MYSQL, DWS, FTP, ORACLE, MONGODB, HANA, SQLSERVER, DB2, GAUSS100, GaussDB(for MySQL), ACTIVEMQ, ARTEMISMQ, POSTGRESQL/OPENGAUSS, RABBITMQ, SAP, IBMMQ or HIVE.

  • This parameter is mandatory when the data source is WEBSOCKET and the authentication mode is basicauth.

  • This parameter is mandatory when the data source is LDAP and the security authentication type (security_auth_type) is simple.

Minimum: 4

Maximum: 2147483647

password

String

Password for accessing the service.

  • This parameter is mandatory when the data source is MYSQL, DWS, FTP, ORACLE, MONGODB, HANA, SQLSERVER, DB2, GAUSS100, GaussDB(for MySQL), ACTIVEMQ, ARTEMISMQ, POSTGRESQL/OPENGAUSS, RABBITMQ, SAP, IBMMQ or HIVE.

  • This parameter is mandatory when the data source is WEBSOCKET type and the authentication mode is basicauth.

  • This parameter is mandatory when the data source is LDAP and the security authentication type (security_auth_type) is simple.

Minimum: 4

Maximum: 2147483647

mode

String

Data source connection mode. The value can be default, professional, or multiAddress.

  • Set to default when the data source is DWS, MONGODB, REDIS, or HANA.

  • Set to default or professional when the data source is MYSQL, SQLSERVER, DB2, GAUSS100, GaussDB(for MySQL), POSTGRESQL/OPENGAUSS or HIVE.

  • Set to multiAddress when the data source is ORACLE.

Minimum: 4

Maximum: 255

cdc_mode

String

CDC mode, which is used only for composite tasks.

Minimum: 4

Maximum: 10

multi_oracle_address

Array of MultiOracleAddress objects

IP address of the Oracle cluster. This parameter is mandatory when mode is set to multiAddress.

oracle_service_name

String

Service name of the Oracle cluster.

Minimum: 4

Maximum: 64

ftp_connect_mode

String

Connection mode for accessing the FTP service. The value can be active or passive.

Minimum: 4

Maximum: 2147483647

ftp_protocol

String

Protocol for accessing the FTP service. The value can be SFTP or FTP.

Minimum: 4

Maximum: 2147483647

address

String

Address.

  • OBS (OBS remote address, which is used by the OBS data source)

  • MONGODB (addresses of the host of the MongoDB data source type, in the format of IP address:Port number, and separated by commas (,))

  • REDIS (addresses of the host of the Redis data source type, in the format of IP address:Port number, and separated by commas (,))

Minimum: 0

Maximum: 256

ak

String

Access key ID.

  • This parameter is mandatory when the data source is OBS or DIS.

Minimum: 0

Maximum: 256

sk

String

Secret access key.

  • This parameter is mandatory when the data source is OBS or DIS.

Minimum: 0

Maximum: 256

bucket_name

String

Bucket name. This parameter is mandatory when the data source is OBS.

Minimum: 0

Maximum: 256

https

Boolean

Whether to use HTTPS. This parameter is mandatory when the data source is OBS. Generally, HTTPS is used by default.

url

String

Connection string, indicating the access URL.

  • This parameter is mandatory when the data source is API, LDAP, or WEBSOCKE.

  • This parameter is mandatory when the data source is MYSQL, ORACLE, DB2, GAUSS100, GaussDB(for MySQL) or POSTGRESQL/OPENGAUSS and mode is set to professional.

Minimum: 0

Maximum: 256

api_method

String

API access request mode.

  • POST

  • PUT

  • DELETE

  • PATCH

  • GET

Minimum: 0

Maximum: 10

auth_method

String

Authentication mode for accessing the WEBSOCKET service.

  • none

  • basicauth

Minimum: 0

Maximum: 256

api_auth_detail

ApiAuthDetail object

API authentication field details.

broker

String

Addresses of the Kafka, MQS and ActiveMQ servers, in the format of IP address:Port number. Use commas (,) to separate multiple IP addresses and port numbers.

Minimum: 0

Maximum: 256

ssl

Boolean

Whether to enable SSL authentication.

  • If the internal MQS address is connected and MQS has enabled SSL, enable SSL authentication for the data source.

ssl_enable

Boolean

Whether to enable SSL authentication.

  • This parameter is mandatory when the data source is KAFKA.

  • This parameter is mandatory when the data source is ARTEMISMQ, ACTIVEMQ, RABBITMQ, or IBMMQ.

ssl_username

String

SSL user name or application key.

  • This parameter is mandatory when the data source is KAFKA or MQS and SSL authentication is enabled.

Minimum: 0

Maximum: 256

ssl_password

String

SSL password or application secret.

  • This parameter is mandatory when the data source is KAFKA or MQS and SSL authentication is enabled.

Minimum: 0

Maximum: 256

mongodb_auth_source

String

MONGODB authentication source.

Minimum: 0

Maximum: 60

mongodb_cluster_enable

Boolean

MONGODB cluster mode.

  • true (cluster mode)

  • false (non-cluster mode)

mongodb_replica_set

String

MONGODB replica set. This parameter is mandatory when the MONGODB is in non-cluster mode.

Minimum: 0

Maximum: 256

encoding

String

Encoding format.

  • This parameter is set to big5 when the data source is GAUSS100 or POSTGRESQL/OPENGAUSS.

  • This parameter is mandatory when the data source is MYSQL or GaussDB(for MySQL) and mode is set to default.

Minimum: 0

Maximum: 10

mysql_timeout

Integer

MySQL connection timeout time (seconds).

Minimum: 1

Maximum: 20000

trust_store_password

String

Password of the public key library.

  • This parameter is mandatory when the data source is ACTIVEMQ, ARTEMISMQ, RABBITMQ, or IBMMQ and SSL authentication is enabled.

Minimum: 0

Maximum: 256

trust_store

String

Public key library file.

  • This parameter is mandatory when the data source is ACTIVEMQ, ARTEMISMQ, RABBITMQ, or IBMMQ and SSL authentication is enabled.

Minimum: 0

Maximum: 1024

trust_store_file_type

String

Type of a public key library file.

  • This parameter is mandatory when the data source is ACTIVEMQ, ARTEMISMQ, RABBITMQ, or IBMMQ and SSL authentication is enabled.

Minimum: 0

Maximum: 10

ssl_auth_method

String

Authentication method. This parameter is mandatory when the data source is ACTIVEMQ or ARTEMISMQ and SSL authentication is enabled.

  • one-way

  • two-way

Minimum: 0

Maximum: 10

key_store

String

Content of the private key library file.

  • This parameter is mandatory when the data source is ACTIVEMQ or ARTEMISMQ and the SSL authentication is enabled and the authentication mode is two-way.

Minimum: 0

Maximum: 1024

key_store_file_type

String

Type of the private key library file.

  • This parameter is mandatory when the data source is ACTIVEMQ or ARTEMISMQ, SSL authentication is enabled, and the authentication mode is two-way.

Minimum: 0

Maximum: 10

key_store_password

String

Private key library password.

  • This parameter is mandatory when the data source is ACTIVEMQ or ARTEMISMQ, SSL authentication is enabled, and the authentication mode is two-way.

Minimum: 0

Maximum: 256

key_store_key_password

String

Private key of the private key library.

  • This parameter is mandatory when the data source is ACTIVEMQ or ARTEMISMQ, SSL authentication is enabled, and the authentication mode is two-way.

Minimum: 0

Maximum: 256

dis_tunnel_name

String

DIS tunnel name.

Minimum: 0

Maximum: 256

dis_data_type

String

DIS data type.

  • JSON

Minimum: 0

Maximum: 256

dis_setting_type

String

DIS configuration type.

  • senior

  • basic

Minimum: 0

Maximum: 256

dis_endpoint

String

DIS endpoint. This parameter is mandatory when setting_type is set to senior.

Minimum: 0

Maximum: 256

dis_region

String

DIS region. This parameter is mandatory when setting_type is set to senior.

Minimum: 0

Maximum: 256

dis_source_project_id

String

Project ID of the DIS source. This parameter is mandatory when setting_type is set to senior.

Minimum: 0

Maximum: 256

hl7_position

String

HL7 data source direction.

  • source

  • target

Minimum: 0

Maximum: 256

hl7_whitelist_enable

Boolean

Whether to enable the HL7 whitelist setting.

hl7_whitelist

String

HL7 whitelist. IP address of the server that allows data synchronization to the HL7 source end. Set this parameter when HL7 is the source (position set to source) and the whitelist setting is enabled (open_whitelist set to true).

Minimum: 0

Maximum: 256

ldap_security_auth_type

String

LDAP security authentication type.

Minimum: 0

Maximum: 256

rabbitmq_virtual_host

String

RabbitMQ virtual host.

Minimum: 0

Maximum: 256

rabbitmq_ssl_protocol

String

RabbitMQ SSL authentication protocol.

  • TLS

Minimum: 0

Maximum: 10

sap_client

String

SAP client ID.

Minimum: 0

Maximum: 10

sap_sysnr

String

SAP instance ID.

Minimum: 0

Maximum: 256

snmp_network_protocol

String

SNMP network protocol.

  • udp

  • tcp

Minimum: 0

Maximum: 10

snmp_version

Integer

SNMP version number.

Minimum: 0

Maximum: 3

snmp_community

String

SNMP community name, which is used as an authentication password for access to the SNMP management agent.

Minimum: 0

Maximum: 256

ibmmq_ccs_id

String

IBMMQ character set ID.

Minimum: 0

Maximum: 65535

ibmmq_queue_manager

String

IBMMQ queue manager.

Minimum: 0

Maximum: 256

ibmmq_channel

String

IBMMQ channel name.

Minimum: 0

Maximum: 256

ibmmq_cipher_suite

String

IBMMQ cipher algorithm suite.

Minimum: 0

Maximum: 256

hdfs_path

String

HDFS URL. This parameter is mandatory when the data source is MRSHIVE or MRSHDFS.

Minimum: 0

Maximum: 256

principal_name

String

Machine-machine interaction username. This parameter is mandatory when the data source is MRSHIVE, MRSHDFS, MRSHBASE or MRSKAFKA.

Minimum: 0

Maximum: 256

config_file_name

String

User authentication file. For details about how to obtain the file, see Appendix > Obtaining the Data Source Configuration File in the ROMA Connect API Reference.

  • Compress the obtained files into a .zip package, set config_file_name to the file name, and place the file content in config_file_content in Base64 encoding mode.

  • This parameter is mandatory when the data source is MRSHIVE, MRSHDFS, MRSHBASE, MRSKAFKA, FIHDFS, FIHIVE or FIKAFKA.

Minimum: 0

Maximum: 256

config_file_content

String

Content of the user authentication file. the Content of the config_file_name file is encoded in Base64. This parameter is mandatory when the data source is MRSHIVE, MRSHDFS, MRSHBASE or MRSKAFKA.

Minimum: 0

Maximum: 256

connection_instance_id

String

Connector instance ID, which is generated when a connector is published

Minimum: 0

Maximum: 256

connector_params

Object

Data source parameter corresponding to the connector. Set this parameter based on actual requirements.
Table 7 MultiOracleAddress

Parameter

Type

Description

oracle_address

String

Oracle address.

Minimum: 1

Maximum: 30

oracle_port

String

Oracle port number.

Minimum: 1

Maximum: 10
Table 8 ApiAuthDetail

Parameter

Type

Description

auth_method

String

Authentication mode for accessing the API service.

  • none

  • basicauth

  • oauth2.0

  • hmac

  • secret

  • md5

  • apiGateway, which corresponds to AppKey Auth authentication on the console.

  • keyTop

  • hikVision

  • huaweiNetworkManagement

  • liHe

Minimum: 0

Maximum: 256

app_auth_type

String

App authentication mode for accessing the API service. This parameter is mandatory when the authentication mode is apiGateway.

  • default

  • secret

  • jwt

Minimum: 0

Maximum: 2000

user_name

String

Username for accessing the API service.

  • This parameter is mandatory when the authentication mode is LiHe, huaweiNetworkManagement, or basicauth.

Minimum: 4

Maximum: 2000

password

String

Password for accessing the API service.

  • This parameter is mandatory when the authentication mode is LiHe, huaweiNetworkManagement, basicauth, secret, md5 or hmac.

Minimum: 4

Maximum: 2000

app_key

String

AppKey for accessing the API service.

  • This parameter is mandatory when the authentication mode is apiGateway.

Minimum: 0

Maximum: 256

app_secret

String

AppSecret for accessing the API service.

  • This parameter is mandatory when the authentication mode is apiGateway.

Minimum: 0

Maximum: 256

secret

String

Secret for accessing the API service.

  • This parameter is mandatory when the authentication mode is KeyTop, HiKVision, Secret, HMAC, or MD5.

Minimum: 0

Maximum: 256

alt_ip

String

Alternative IP address for accessing the API service.

  • This parameter is mandatory when the authentication mode is HuaweiNetworkManagement.

Minimum: 0

Maximum: 256

access_token_url

String

AccessTokenUrl for accessing the API service.

  • This parameter is mandatory when the authentication mode is liHe or oauth2.0 huaweiNetworkManagement.

Minimum: 0

Maximum: 256

client_id

String

Client ID for accessing the API service.

  • This parameter is mandatory when the authentication mode is Oauth2.

Minimum: 0

Maximum: 256

client_secret

String

Client secret for accessing the API service.

  • This parameter is mandatory when the authentication mode is Oauth2.

Minimum: 0

Maximum: 256

scope

String

Scope for accessing the API service.

  • This parameter is mandatory when the authentication mode is LiHe or Oauth2.

Minimum: 0

Maximum: 256

authorization

String

Authorization for accessing the API service.

  • This parameter is mandatory when the authentication mode is LiHe.

Minimum: 0

Maximum: 256

grant_type

String

Authorization type for accessing the API service.

  • This parameter is mandatory when the authentication mode is LiHe or Oauth2.

  • The value is client_credentials. This value is used in OAuth2.0.

Minimum: 0

Maximum: 256

Status code: 400

Table 9 Response body parameters

Parameter

Type

Description

error_code

String

Error code.

Minimum: 8

Maximum: 36

error_msg

String

Error message.

Minimum: 2

Maximum: 512

Status code: 500

Table 10 Response body parameters

Parameter

Type

Description

error_code

String

Error code.

Minimum: 8

Maximum: 36

error_msg

String

Error message.

Minimum: 2

Maximum: 512

Example Requests

None

Example Responses

Status code: 200

OK

{
  "total" : "658,",
  "size" : "1,",
  "entities" : [ {
    "datasource_id" : "acd5502992254b3ab2dfd5b6900159db",
    "datasource_name" : "TY_KAFKA",
    "datasource_type" : "KAFKA",
    "content" : {
      "project_id" : "c53626012ba84727b938ca8bf03108ef",
      "type" : "KAFKA",
      "instance_id" : "05173009-2e5f-4773-8303-8e05cc9c9302",
      "broker" : "10.01.01.01:9092",
      "ssl_username" : "1",
      "ssl_password" : "******",
      "ssl_enable" : true
    },
    "create_time" : 1616784907101,
    "update_time" : 1619650507230,
    "app_id" : "141bbdf5-131e-4111-898d-5a6292a2113b",
    "app_name" : "app-zhe",
    "instance_id" : "05173009-2e5f-4773-8303-8e05cc9c9302",
    "remark" : "No change.",
    "app_permission" : [ "read", "access", "delete", "modify" ]
  } ]
}

Status Codes

Status Code

Description

200

OK

400

Bad Request

500

Internal Server Error

Error Codes

See Error Codes.

Parent topic: Data Source Management