Help Center/ GeminiDB/ GeminiDB Influx API/ Working with GeminiDB Influx API/ Migrating Data
Updated on 2024-12-02 GMT+08:00
View PDF
Share

Migrating Data

InfluxDB Community Edition is a popular time series database that focuses on high-performance query and storage of time series data.

GeminiDB Influx API is a cloud-native NoSQL time-series database with a decoupled compute and storage architecture developed by Huawei and is compatible with InfluxDB. This high availability database is secure and scalable, can be deployed, backed up, or restored quickly, and offers monitoring and alarm management capabilities. You can also add storage or compute resources separately. GeminiDB Influx API has better query, write, and data compression performance than InfluxDB Community Edition.

This section describes how to migrate data from InfluxDB Community Edition to GeminiDB Influx API.

Migration Principles

Use open-source migration tool data-migration-tools to parse the tsm and wal files of the InfluxDB Community Edition and write the files to a line protocol file. Then, the line protocol file data is parsed and migrated to the destination.

The migration process is divided into two phases:

  • Export: tsm files of InfluxDB Community Edition are concurrently parsed, and the parsed data is written into memory.
  • Import: The read data is sent to the GeminiDB Influx cluster.

You can specify a migration period while the migration tool is running.

Download and decompress the release package of data-migration-tools.

Usage Notes

  • Deploy the migration tool on the same server as InfluxDB Community Edition and prepare a configuration file.
  • The migration tool needs to extract data from tsm to the local line protocol file, obtain data from the line protocol file, and send the data to the destination GeminiDB Influx database. This process may affect the performance of the source side. You are advised to run the migration tool during off-peak hours.
  • The migration tool supports only InfluxDB 1.X Community Edition.

Prerequisites

  • Ensure that the network connection between the source and destination is normal.
  • The corresponding database has been created and the retention policy (RP) has been configured in the destination GeminiDB Influx.

Procedure

For details about how to migrate data from InfluxDB Community Edition to GeminiDB Influx API, see Data Migration Tool Usage Guide.

Migration Performance Reference

  • Migration environment
    • Source: Deploy InfluxDB and the migration tool on an ECS with 4 vCPUs and 16 GB of memory.
    • Destination: three-node GeminiDB Influx instance with 4 vCPUs and 16 GB of memory
  • Migration performance
    • The data migration rate of a single process on the source database is 1 GB/min.