Notes and Constraints

Notes and Constraints

1. The metadata of all tables involved in lineages must be collected in DataArts Catalog. Otherwise, data catalog lineages cannot be generated.
2. DataArts Catalog does not support the CREATE TABLE syntax. Data Map only supports table-level lineages.
3. Field lineages must be explicitly specified in SQL statements. The wildcard (*) is not supported, for example, insert into a select * from b;.
4. Lineages generated by the mapping between constants, tables, and fields are not supported, for example, insert into a values ('CONSTANT');. Only lineages generated by the mapping between tables and between fields are supported.
5. Data lineages are parsed based on individual SQL statements:
   • The set statement is not supported. For example, you cannot set the schema to dws using set search_path = dws.
   • The transfer lineages in the temporary table scenario cannot be parsed, for example, the transfer lineage from a to b cannot be parsed from with temp as select * from a; insert into b select * from temp;.
6. If you want to specify the schema of a table, the table name in the SQL statement must be in the {schema}.{table} format. Otherwise, the lineage will be on the table in the default schema.
7. Field lineages do not support fields modified by window functions, for example, insert into a select rank() over(id) from b;.
8. Field lineages do not support the Cartesian product, for example, insert into a select * from b, c;.
9. Lineage parsing does not support subqueries embedded in case when, for example, insert into a select case when true then (select * from b) end;.
10. DWS SQL lineage parsing does not support syntax that is not supported by the DWS syntax parsing package, such as partition for.
11. The lineage of a migration job cannot be parsed if a custom SQL statement is configured for extracting data at the source.
12. In MRS Spark lineage parsing, the MRS Spark connection in Management Center must have the same configuration as MRS Hive.