On this page

Question
Answer

Show all

Help Center/ MapReduce Service/ Component Operation Guide (LTS)/ Using CarbonData/ CarbonData FAQ/ How Do I Logically Split Data Across Different Namespaces?

How Do I Logically Split Data Across Different Namespaces?

Updated on 2022-11-18 GMT+08:00

View PDF

Question

How do I logically split data across different namespaces?

Answer

Configuration:
To logically split data across different namespaces, you must update the following configuration in the core-site.xml file of HDFS, Hive, and Spark.

NOTE:

Changing the Hive component will change the locations of carbonstore and warehouse.
- Configuration in HDFS
  - fs.defaultFS: Name of the default file system. The URI mode must be set to viewfs. When viewfs is used, the permission part must be ClusterX.
  - fs.viewfs.mountable.ClusterX.homedir: Home directory base path. You can use the getHomeDirectory() method defined in FileSystem/FileContext to access the home directory.
  - fs.viewfs.mountable.default.link.<dir_name>: ViewFS mount table.
  Example:
```
<property>
<name>fs.defaultFS</name>
<value>viewfs://ClusterX/</value>
</property>
<property>
<name>fs.viewfs.mounttable.ClusterX.link./folder1</name>
<value>hdfs://NS1/folder1</value>
</property>
<property>
<name>fs.viewfs.mounttable.ClusterX.link./folder2</name>
<value>hdfs://NS2/folder2</value>
</property>
```
- Configurations in Hive and Spark
  fs.defaultFS: Name of the default file system. The URI mode must be set to viewfs. When viewfs is used, the permission part must be ClusterX.

Syntax:
LOAD DATA INPATH 'path to data' INTO TABLE table_name OPTIONS ('...');

NOTE:

When Spark is configured with the viewFS file system and attempts to load data from HDFS, users must specify a path such as viewfs:// or a relative path as the file path in the LOAD statement.
Example:
- Sample viewFS path:
  LOAD DATA INPATH 'viewfs://ClusterX/dir/data.csv' INTO TABLE table_name OPTIONS ('...');
- Sample relative path:
  LOAD DATA INPATH '/apps/input_data1.txt' INTO TABLE table_name;

Parent topic: CarbonData FAQ

Previous topic: Why CarbonData tables created in V100R002C50RC1 not reflecting the privileges provided in Hive Privileges for non-owner?

Next topic: Why Missing Privileges Exception is Reported When I Perform Drop Operation on Databases?

Feedback

Was this page helpful?

Helpful Not helpful

Provide feedback

Thank you very much for your feedback. We will continue working to improve the documentation.

The system is busy. Please try again later.

Which of the following issues have you encountered?

Content is inconsistent with the product UI

Unclear descriptions

Lack of examples or code

Incorrect steps

Can't find what I need

Lack of best practices

Feedback (optional)

0/500

Select at least one type of issue, and enter your comments or suggestions.

Enter a maximum of 500 characters.

Submit Cancel

Feedback

Feedback

Feedback

0/500

Selected Content

Submit selected content with the feedback