Help Center/ Cloud Data Migration/ User Guide/ Creating a Job in a CDM Cluster/ Configuring CDM Source Job Parameters/ From Oracle
Updated on 2024-10-24 GMT+08:00
View PDF
Share

From Oracle

If the source link of a job is an Oracle link, configure the source job parameters based on Table 1.

Table 1 Parameter description

Parameter

Description

Example Value

Use SQL Statement

Whether you can use SQL statements to export data from a relational database

No

SQL Statement

When Use SQL Statement is set to Yes, enter an SQL statement here. CDM exports data based on the SQL statement.

NOTE:
  • SQL statements can only be used to query data. Join and nesting are supported, but multiple query statements are not allowed, for example, select * from table a; select * from table b.
  • With statements are not supported.
  • Comments, such as -- and /*, are not supported.
  • Addition, deletion, and modification operations are not supported, including but not limited to the following:
    • load data
    • delete from
    • alter table
    • create table
    • drop table
    • into outfile
  • If the SQL statement is too long, the request fails to be delivered. If you continue to create a job, the system displays an error message indicating that the request is incorrect. In this case, you need to simplify or clear the SQL statement and try again.

select id,name from sqoop.user;

Schema/Tablespace

Name of the schema or tablespace from which data will be extracted. This parameter is displayed when Use SQL Statement is set to No. Click the icon next to the text box to go to the page for selecting a schema or directly enter a schema or tablespace.

If the desired schema or tablespace is not displayed, check whether the login account has the permissions required to query metadata.

NOTE:
The parameter value can contain wildcard characters (*), which is used to export all databases whose names start with a certain prefix or end with a certain suffix. For example:
  • SCHEMA* indicates that all databases whose names starting with SCHEMA are exported.
  • *SCHEMA indicates that all databases whose names ending with SCHEMA are exported.
  • *SCHEMA* indicates that all databases whose names containing SCHEMA are exported.

SCHEMA_E

Table Name

Name of the table from which data will be extracted. This parameter is displayed when Use SQL Statement is set to No. Click the icon next to the text box to go to the page for selecting the table or directly enter a table name.

If the desired table is not displayed, check whether the table exists or whether the login account has the permission to query metadata.

This parameter can be configured as a macro variable of date and time and a path name can contain multiple macro variables. When the macro variable of date and time works with a scheduled job, the incremental data can be synchronized periodically. For details, see Incremental Synchronization Using the Macro Variables of Date and Time.

NOTE:

If you have configured a macro variable of date and time and schedule a CDM job through DataArts Studio DataArts Factory, the system replaces the macro variable of date and time with (Planned start time of the data development job – Offset) rather than (Actual start time of the CDM job – Offset).

NOTE:
The table name can contain wildcard characters (*), which is used to export all tables whose names start with a certain prefix or end with a certain suffix. The number and types of fields in the tables must be the same. The examples are as follows:
  • table* indicates that all tables whose names starting with table are exported.
  • *table indicates that all tables whose names ending with table are exported.
  • *table* indicates that all tables whose names containing table are exported.

table

Partition Column

This parameter is displayed when Extract by Partition is set to No, indicating a field used to split data during data extraction. CDM splits a job into multiple tasks based on this field and executes the tasks concurrently. Fields with data distributed evenly are used, such as the sequential number field.

Click the icon next to the text box to go to the page for selecting a field or directly enter a field.

NOTE:

The following types of partition columns are supported: TINYINT, SMALLINT, INTEGER, BIGINT, REAL, FLOAT, DOUBLE, NUMERIC, DECIMAL, BIT, BOOLEAN, DATE, TIME, and TIMESTAMP. It is recommended that the partition column have an index.

id

Where Clause

WHERE clause used to specify the data extraction range. This parameter is displayed when Use SQL Statement is set to No. If this parameter is not set, the entire table is extracted.

You can set a date macro variable to extract data generated on a specific date. For details, see Incremental Migration of Relational Databases.

NOTE:

If you have configured a macro variable of date and time and schedule a CDM job through DataArts Studio DataArts Factory, the system replaces the macro variable of date and time with (Planned start time of the data development job – Offset) rather than (Actual start time of the CDM job – Offset).

DS='${dateformat(yyyy-MM-dd,-1,DAY)}'

Null in Partition Column

Whether the partition field can contain null values. This parameter is displayed when Extract by Partition is set to No.

Yes

Extract by Partition
When data is exported from an Oracle database, data can be extracted from each partition in the partitioned table. If this function is enabled, you can configure Table Partition to specify specific Oracle table partitions from which data is extracted.
  • This function does not support non-partitioned tables.
  • The database user must have the SELECT permission on the system views dba_tab_partitions and dba_tab_subpartitions.

No

Table Partition

Oracle table partition from which data is migrated. Separate multiple partitions with ampersands (&). If you do not set this parameter, all partitions will be migrated.

If there is a subpartition, enter the partition in the Partition.Subpartition format, for example, P2.SUBP1.

P0&P1&P2.SUBP1&P2.SUBP3

Split Job

If this parameter is set to Yes, the job is split into multiple subjobs based on the value of Job Split Field, and the subjobs are executed concurrently.

NOTE:

This parameter and parameters Job Split Field, Minimum Split Field Value, Maximum Split Field Value, and Number of subjobs are available only when the destination link is a DLI or Hive link.

Yes

Job Split Field

Field used to split a job into multiple subjobs for concurrent execution. This parameter is available when Split Job is set to Yes.

-

Minimum Split Field Value

Minimum value of Job Split Field during data extraction. This parameter is available when Split Job is set to Yes.

-

Maximum Split Field Value

Maximum value of Job Split Field during data extraction. This parameter is available when Split Job is set to Yes.

-

Number of subjobs

Number of subjobs split from a job for concurrent execution based on the data range specified by the minimum and maximum values of Job Split Field. This parameter is available when Split Job is set to Yes.

-

When an Oracle database is the migration source, if Partitioning Field or Extract by Partition is not configured, CDM automatically uses the ROWIDs to partition data.