Converting Data into a Parquet file and Exporting the Data to OBS
Scenarios
New data inserted by users on GeminiDB Influx instances in a performance-enhanced cluster can be converted into a Parquet file and automatically uploaded to a specified OBS bucket. You can access or download a Parquet file on OBS.
Usage Notes
- To export data, choose Service Tickets > Create Service Ticket in the upper right corner of the console and contact the customer service.
- Data can be converted into a Parquet file by table. By default, data in all tables is converted. To export data in a specified table, choose Service Tickets > Create Service Ticket in the upper right corner of the console and contact the customer service.
- This function is available only for GeminiDB Influx instances with classic storage in a performance-enhanced cluster.
- To use this function, you need to enable Export Data.
- OBS supports parallel file systems and standard buckets.
- You can specify a target folder, which must be created in advance.
- When you export data, a policy named data-dump-access is created on OBS. This policy provides only the PUT permission and applies only to the selected folder.
- You need to set the shard duration in the retention policy to one day.
- Conversion into a Parquet file is an asynchronous process. The duration depends on multiple factors such as the concurrency level and system load. The real-time performance cannot be guaranteed.
- The converted Parquet file is uploaded on the hour, for example, at 02:00, every two hours.
- Historical data cannot be converted into a Parquet file or exported to OBS.
Procedure
- Log in to the Huawei Cloud console.
- In the service list, choose Databases > GeminiDB.
- On the Instances page, locate the target instance and choose More > Export Data in the Operation column.
Figure 1 Export Data
You can also click the target instance to go to the basic information page. Click Export Data next to Bucket Configuration in the DB Information area.
Figure 2 Export Data - On the Export Data page, enable Export Data.
Figure 3 Export Data
- Select Parallel file system or Bucket list for Resource Type and select a file system name or an OBS bucket name.
- Click Select Folder.
Figure 4 Select Folder
- Click OK.
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot