Using DataArts Fabric SQL to Query Data
Description
DataArts Fabric SQL is a cloud-native serverless version of DataArts Fabric. By leveraging the resource pooling and massive storage capabilities provided by the cloud infrastructure, it combines parallel execution, metadata decoupling, and compute-storage persistent decoupling architecture for robust elasticity and lake warehouse integration.
This section describes how to quickly enable DataArts Fabric SQL and perform simple data queries.
Prerequisites
- You have registered an account and completed real-name authentication. The account is not in arrears or frozen.
- You have enabled LakeFormation and OBS permissions and confirmed the agency.
- You have a workspace available.
Step 1: Plan and Create an OBS Bucket and Import Data
DataArts Fabric SQL uses OBS to store data. You need to create a bucket and folder on the OBS console and import sample data.
- Log in to the management console.
- Click
in the upper left corner of the page and choose Storage > Object Storage Service to access the Object Storage Service console.
- Use a parallel file system as an example.
- On the Parallel File System page, click the name of the created file system, for example, fabric-serverless.
- Click Files in the navigation pane, click Create Folder, enter a folder name, and click OK. Click the folder name and click Create Folder to create a subfolder.
- Repeat this step to create paths for storing metadata in sequence. The following paths are examples:
- Catalog storage path: fabric-serverless/catalog1
- Database storage path: fabric-serverless/catalog1/database1
- Data table storage path: fabric-serverless/catalog1/database1/table1
Step 2: Plan and Create a LakeFormation Instance, Catalog, and Database
DataArts Fabric SQL manages data sources using LakeFormation. You need to purchase a LakeFormation instance and configure its catalog, database, and table information.
- Log in to the management console.
- In the upper left corner of the page, choose Analytics > DataArts Lake Formation.
- On the Overview page, purchase an instance.
- In the upper left corner, select the instance to display its details.
- Create a catalog.
- In the navigation pane on the left, choose Metadata > Catalog.
- Click Create. On the displayed page, configure the following parameters, and click Submit.
- Catalog Name: catalog1
- Select Location: Click
, select a storage location, for example, obs://fabric-serverless/catalog1, and click OK.
- Catalog Type: DEFAULT
- Retain the default settings for other parameters.
- After the catalog is created, check the catalog information on the Catalog page.
- Create a database.
- In the navigation pane on the left, choose Metadata > Database.
- Select catalog1 from the drop-down list box next to Catalog in the upper right corner.
If the database default already exists, skip this step.
- Click Create, configure related parameters, and click Submit.
- Database Name: database1
- Catalog: catalog1
- Select Location: Click
, select a location, for example, obs://fabric-serverless/catalog1/database1, and click OK.
- Retain the default settings for other parameters.
- After the database is created, check the database information on the Database page.
Step 3: Use DataArts Fabric SQL
- Log in to the Huawei Cloud DataArts Fabric console and click Access Workspace.
- In the navigation pane on the left, choose Development and Production > SQL Editor. Select a LakeFormation instance and a LakeFormation catalog.
- Select an SQL endpoint to run the SQL statements.
1 2 3 4 5 6 7 8 9 10
CREATE TABLE database1.iceberg_table ( col_id INT, col_tinyint SMALLINT, col_smallint SMALLINT, col_int INTEGER, col_bigint BIGINT ) store AS iceberg; SELECT * FROM database1.iceberg_table;
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot