How Do I Logically Split Data Across Different Namespaces?
Question
How do I logically split data across different namespaces?
Answer
- Configuration:
To logically split data across different namespaces, you must update the following configuration in the core-site.xml file of HDFS, Hive, and Spark.
Changing the Hive component will change the locations of carbonstore and warehouse.
- Configuration in HDFS
- fs.defaultFS: Name of the default file system. The URI mode must be set to viewfs. When viewfs is used, the permission part must be ClusterX.
- fs.viewfs.mountable.ClusterX.homedir: Home directory base path. You can use the getHomeDirectory() method defined in FileSystem/FileContext to access the home directory.
- fs.viewfs.mountable.default.link.<dir_name>: ViewFS mount table.
Example:
<property> <name>fs.defaultFS</name> <value>viewfs://ClusterX/</value> </property> <property> <name>fs.viewfs.mounttable.ClusterX.link./folder1</name> <value>hdfs://NS1/folder1</value> </property> <property> <name>fs.viewfs.mounttable.ClusterX.link./folder2</name> <value>hdfs://NS2/folder2</value> </property>
- Configurations in Hive and Spark
fs.defaultFS: Name of the default file system. The URI mode must be set to viewfs. When viewfs is used, the permission part must be ClusterX.
- Configuration in HDFS
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.