Help Center/ DataArts Studio/ User Guide/ DataArts Factory/ Node Reference/ Node Lineages/ Notes and Constraints
Updated on 2026-08-21 GMT+08:00
View PDF
Share

Notes and Constraints

During data management and analysis, data lineage parsing is important to ensure data accuracy and traceability. Before using this function, you should understand the constraints and properly plan data collection and SQL compilation so that data lineages can be correctly and completely parsed.

Notes and Constraints

  1. The metadata of all tables involved in lineages must be collected in DataArts Catalog. Otherwise, data catalog lineages cannot be generated.
  2. DataArts Catalog does not support the CREATE TABLE syntax. Data Map only supports table-level lineages.
  3. Field lineages must be explicitly specified in SQL statements. The wildcard (*) is not supported, for example, insert into a select * from b;.
  4. Lineages generated by the mapping between constants, tables, and fields are not supported, for example, insert into a values ('CONSTANT');. Only lineages generated by the mapping between tables and between fields are supported.
  5. Data lineages are parsed based on individual SQL statements:
    • The set statement is not supported. For example, you cannot set the schema to dws using set search_path = dws.
    • The transfer lineages in the temporary table scenario cannot be parsed, for example, the transfer lineage from a to b cannot be parsed from with temp as select * from a; insert into b select * from temp;.
  6. If you want to specify the schema of a table, the table name in the SQL statement must be in the {schema}.{table} format. Otherwise, the lineage will be on the table in the default schema.
  7. Field lineages do not support fields modified by window functions, for example, insert into a select rank() over(id) from b;.
  8. Field lineages do not support the Cartesian product, for example, insert into a select * from b, c;.
  9. Lineage parsing does not support subqueries embedded in case when, for example, insert into a select case when true then (select * from b) end;.
  10. DWS SQL lineage parsing does not support syntax that is not supported by the DWS syntax parsing package, such as partition for.
  11. The lineage of a migration job cannot be parsed if a custom SQL statement is configured for extracting data at the source.
  12. In MRS Spark lineage parsing, the MRS Spark connection in Management Center must have the same configuration as MRS Hive.
  13. Hetu table and view lineages can be parsed, but field lineages cannot. Currently, only Hive tables are supported.
  14. Hetu lineages cannot be displayed in DataArts Catalog. They can only be displayed in Data Map.
  15. DataArts Catalog cannot collect Hetu metadata offline. It can only collect Hive metadata offline.
  16. The MRS cluster used for a Hetu data connection must be the same as the MRS cluster used for Hive metadata collection.
  17. The database name must be explicitly specified in the Hetu lineage SQL statement.
  18. MRS Flink SQL lineages can be parsed only when the script contains create with and insert SQL statements.
  19. There are constraints on the connectors supported by MRS Flink SQL. Currently, flinkCreateSQL only supports the Hudi connector.
Parent topic: Node Lineages