Migrating Data to RDS for PostgreSQL Using psql

Preparing for Data Migration

PostgreSQL supports logical backups. You can use the pg_dump logical backup function to export backup files and then import them to RDS using psql.

You can access RDS DB instances through an EIP or through an ECS.

Preparations

1. Prepare an ECS for accessing DB instances in the same VPC or prepare a device for accessing RDS through an EIP.
   • To connect to a DB instance through an ECS, you need to create an ECS first.
   • To connect to a DB instance through an EIP, you must:
     1. Bind an EIP to the DB instance. For details, see Binding an EIP.
     2. Ensure that the local device can access the EIP that has been bound to the DB instance.
2. Install a PostgreSQL client on the prepared ECS or device.

   For details, see How Can I Install the PostgreSQL Client?

   

   The PostgreSQL client version must be the same as the DB engine version of your RDS for PostgreSQL instance. A PostgreSQL database or client will provide pg_dump and psql.

Exporting Data

Before migrating an existing PostgreSQL database to RDS, you need to export data first.

• The export tool must match the DB engine version.
• Database migration is performed offline. Before the migration, you have to stop all applications using the source database.

1. Log in to the ECS or the device that can access RDS.
2. Use the pg_dump tool to export the source database into an SQL file.

   pg_dump--username=<DB_USER> --host=<DB_ADDRESS> --port=<DB_PORT> --format=plain --file=<BACKUP_FILE><DB_NAME>

   • DB_USER indicates the database username.
   • DB_ADDRESS indicates the database address.
   • DB_PORT indicates the database port.
   • BACKUP_FILE indicates the name of the file to which the data will be exported.
   • DB_NAME indicates the name of the database to be migrated.

   Enter the database password as prompted.

   

   If the exported SQL file uses INSERT statements, you can easily edit and modify the file. However, the speed of importing data may be slower than that of using COPY statements. You are advised to select a right statement format as needed.

   • If both the source and destination databases are PostgreSQL databases, you are advised to export COPY statements (default). For details, see Example 1: Exporting the source database to an SQL file (COPY).
   • If either of the source and destination databases is a non-PostgreSQL database, you are advised to export INSERT statements. For details, see Example 2: Exporting the source database to an SQL file (INSERT).

   For more information, see pg_dump options.

   Examples:

   • Example 1: Exporting the source database to an SQL file (COPY)

     $ pg_dump --username=root --host=192.168.151.18 --port=5432 --format=plain --file=backup.sql my_db

     Password for user root:

   • Example 2: Exporting the source database to an SQL file (INSERT)

     $ pg_dump --username=root --host=192.168.151.18 --port=5432 --format=plain --inserts --file=backup.sql my_db

     Password for user root:

   • Example 3: Exporting all table structures from the source database to an SQL file

     $ pg_dump --username=root --host=192.168.151.18 --port=5432 --format=plain --schema-only --file=backup.sql my_db

     Password for user root:

   • Example 4: Exporting all table data from the source database to an SQL file

     $ pg_dump --username=root --host=192.168.151.18 --port=5432 --format=plain --data-only --file=backup.sql my_db

     Password for user root:

   After the commands in any of the above examples are executed, a backup.sql file will be generated as follows:

   [rds@localhost ~]$ ll backup.sql
   -rw-r-----. 1 rds rds 2714 Sep 21 08:23 backup.sql

3. Use pg_dump to export tables from the source database to an SQL file.

   pg_dump --username=<DB_USER> --host=<DB_ADDRESS> --port=<DB_PORT> --format=plain --file=<BACKUP_FILE> <DB_NAME> --table=<TABLE_NAME>

   • DB_USER indicates the database username.
   • DB_ADDRESS indicates the database address.
   • DB_PORT indicates the database port.
   • BACKUP_FILE indicates the name of the file to be exported.
   • DB_NAME indicates the name of the database to be migrated.
   • TABLE_NAME indicates the name of the specified table in the database to be migrated.

   Enter the database password as prompted.

   Examples:

   • Example 1: Exporting one table from the source database to an SQL file

     $ pg_dump --username=root --host=192.168.151.18 --port=5432 --format=plain --file=backup.sql my_db --table=test

     Password for user root:

   • Example 2: Exporting multiple tables from the source database to an SQL file

     $ pg_dump --username=root --host=192.168.151.18 --port=5432 --format=plain --file=backup.sql my_db --table=test1 --table=test2

     Password for user root:

   • Example 3: Exporting all tables starting with ts_ from the source database to an SQL file

     $ pg_dump --username=root --host=192.168.151.18 --port=5432 --format=plain --file=backup.sql my_db --table=ts_*

     Password for user root:

   • Example 4: Exporting all tables except those starting with ts_ from the source database to an SQL file

     $ pg_dump --username=root --host=192.168.151.18 --port=5432 --format=plain --file=backup.sql my_db -T=ts_*

     Password for user root:

   After the commands in any of the above examples are executed, a backup.sql file will be generated as follows:

   [rds@localhost ~]$ ll backup.sql
   -rw-r-----. 1 rds rds 2714 Sep 21 08:23 backup.sql

Importing Data

1. Log in to the ECS or the device that can access RDS.
2. Ensure that the destination database to which data is to be imported exists.

   If the destination database does not exist, run the following command to create a database:

   # psql --host=<RDS_ADDRESS>--port=<DB_PORT>--username=root--dbname=postgres-c "create database<DB_NAME>;"

   • RDS_ADDRESS indicates the IP address of the RDS DB instance.
   • DB_PORT indicates the RDS DB instance port.
   • DB_NAME indicates the name of the database to be imported.

3. Import the exported file to RDS.

   # psql --host=<RDS_ADDRESS> --port=<DB_PORT>--username=root--dbname=<DB_NAME>--file=<BACKUP_DIR>/backup.sql

   • RDS_ADDRESS indicates the IP address of the RDS DB instance.
   • DB_PORT indicates the RDS DB instance port.
   • DB_NAME indicates the name of the database to which data is to be imported. Ensure that the database exists.
   • BACKUP_DIR indicates the directory where the backup.sql file is stored.

   Enter the password for the RDS DB instance when prompted.

   Example:

   # psql --host=172.16.66.198 --port=5432 --username=root --dbname=my_db --file=backup.sql

   Password for user root:

4. View the import result.

   my_db=> \l my_db

   In this example, the database named my_db has been imported.

   my_db=> \l my_db
   List of databases
   Name  | Owner | Encoding | Collate     | Ctype       | Access privileges
   ------+-------+----------+-------------+-------------+-----------
   my_db | root  | UTF8     | en_US.UTF-8 | en_US.UTF-8 | 
   (1 row)