Help Center/ GeminiDB/ GeminiDB Influx API/ Working with GeminiDB Influx API/ Migrating Data
Updated on 2023-11-21 GMT+08:00
View PDF

Migrating Data

InfluxDB Community Edition is a popular time series database that focuses on high-performance query and storage of time series data.

GeminiDB Influx is a cloud-native NoSQL time-series database with a decoupled compute and storage architecture developed by Huawei and full compatibility with InfluxDB. This high availability database is secure and scalable, can be deployed, backed up, or restored quickly, and includes monitoring and alarm management. You can also expand storage or compute resources separately. GeminiDB Influx has better query, write, and data compression performance than InfluxDB Community Edition.

This section describes how to migrate data from InfluxDB Community Edition to GeminiDB Influx.

Migration Principles

Use the migration tool to parse the tsm and wal files of the InfluxDB community edition and write the files to the line protocol file. Then, the line protocol file data is parsed and migrated to the destination side.

The migration process is divided into two phases: export and import.

  • In the export phase, the tsm and wal files of the InfluxDB community edition are concurrently parsed and the parsed data is written into the line protocol file.
  • In the import phase, the line protocol file is concurrently read and the read data is sent to each node in the GeminiDB Influx cluster.

The migration tool supports full migration and incremental migration, which can be configured in the configuration file.

Precautions

  • Migration tool, which is deployed on the same server as the InfluxDB community edition. Prepare the configuration file.
  • The migration tool needs to extract data from tsm and wal to the local line protocol file, obtain data from the line protocol file, and send the data to the destination GeminiDB Influx database. This process may affect the performance of the source side. You are advised to run the migration tool during off-peak hours.
  • Reserve sufficient disk space because the .tsm/wal file data needs to be extracted to the line protocol file.
  • The migration tool supports only the InfluxDB 1.X community edition.

Prerequisites

  • Ensure that the network connection between the source and destination is normal.
  • The corresponding database has been created and the retention policy (RP) has been configured in the destination GeminiDB Influx.

Procedure

To migrate data from InfluxDB to GeminiDB Influx, in the upper right corner of the management console, choose Service Tickets > Create Service Ticket to contact technical support.

Migration Performance Reference

  • Migration environment
    • Source: Deploy InfluxDB and the migration tool on an ECS with 2 vCPUs and 16 memory.
    • Destination: Three-node GeminiDB Influx instance with 2 vCPUs and 16 GB memory
  • Migration performance
    • The data export rate of a single process on the source is 1 GB/min.
    • The single-thread import rate of the destination is 1 GB/min.