Help Center/DataArts Studio/Getting Started/Novices: DWS-powered Data Integration and Development Based on Movie Scores/Step 1: Prepare Data
Updated on 2026-03-20 GMT+08:00
View PDF
Share

Step 1: Prepare Data

Preparations

If you are new to DataArts Studio, registera Huawei account, buy a DataArts Studio instance, create workspaces, and make other preparations. For details, see Buying and Configuring a DataArts Studio Instance. Then you can go to the created workspace and start using DataArts Studio.

Preparing Data Sources

This practice uses the 100,000 scores given by 1,000 users to 1,700 movies. The scores are available at https://grouplens.org/datasets/movielens/100k/. Obtain the .zip package from the link and extract the u.item and u.data files from it. They contain the movie information and rating information, respectively.

To facilitate demonstration, this practice provides some data used to simulate the original data. To integrate the source data into the cloud, you need to store the sample data in CSV files and upload them to an OBS bucket.

  1. Create CSV files (UTF-8 without BOM), name the files with the corresponding data table names, copy the sample data to different CSV files, and save the files.

    To generate a CSV file in Windows, you can perform the following steps:
    1. Use a text editor (for example, Notepad) to create a .txt document and copy the sample data to the document. Then check the total number of rows and check whether the data of rows is correctly separated. (If the sample data is copied from a PDF document, the data in a single row will be wrapped if the data is too long. In this case, you must manually adjust the data to ensure that it is in a single row.)
    2. Choose File > Save as. In the displayed dialog box, set Save as type to All files (*.*), enter the file name with the .csv suffix for File name, and select the UTF-8 encoding format (without BOM) to save the file in CSV format.

  2. Upload the CSV file to OBS.

    1. Log in to the management console and choose Storage > Object Storage Service to access the OBS console.
    2. Click Create Bucket and set parameters as prompted to create an OBS bucket named fast-demo.

      To ensure network connectivity, select the same region for OBS bucket as that for the DataArts Studio instance. If an enterprise project is required, select the enterprise project that is the same as that of the DataArts Studio instance.

      For details about how to create a bucket on the OBS console, see Creating a Bucket in Object Storage Service Console Operation Guide.

    3. Upload data to OBS bucket fast-demo.

      For details about how to upload a file on the OBS console, see Uploading a File in Object Storage Service Console Operation Guide.

This practice involves movie data (movies.csv) and rating data (ratings.csv). Descriptions of the data are as follows:
  • movies.csv: 
    movieId,movieTitle,videoReleaseDate,IMDbURL,unknown,Action,Adventure,Animation,Children,Comedy,Crime,Documentary,Drama,Fantasy,FilmNoir,Horror,Musical,Mystery,Romance,SciFi,Thriller,War,Western1,Toy Story (1995),1-Jan-95,http://us.imdb.com/M/title-exact?Toy%20Story%20(1995),0,0,0,1,1,1,0,0,0,0,0,0,0,0,0,0,0,0,02,GoldenEye (1995),1-Jan-95,http://us.imdb.com/M/title-exact?GoldenEye%20(1995),0,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,03,Four Rooms (1995),1-Jan-95,http://us.imdb.com/M/title-exact?Four%20Rooms%20(1995),0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,04,Get Shorty (1995),1-Jan-95,http://us.imdb.com/M/title-exact?Get%20Shorty%20(1995),0,1,0,0,0,1,0,0,1,0,0,0,0,0,0,0,0,0,05,Copycat (1995),1-Jan-95,http://us.imdb.com/M/title-exact?Copycat%20(1995),0,0,0,0,0,0,1,0,1,0,0,0,0,0,0,0,1,0,06,Shanghai Triad (Yao a yao yao dao waipo qiao) (1995),1-Jan-95,http://us.imdb.com/Title?Yao+a+yao+yao+dao+waipo+qiao+(1995),0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,07,Twelve Monkeys (1995),1-Jan-95,http://us.imdb.com/M/title-exact?Twelve%20Monkeys%20(1995),0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,0,08,Babe (1995),1-Jan-95,http://us.imdb.com/M/title-exact?Babe%20(1995),0,0,0,0,1,1,0,0,1,0,0,0,0,0,0,0,0,0,09,Dead Man Walking (1995),1-Jan-95,http://us.imdb.com/M/title-exact?Dead%20Man%20Walking%20(1995),0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,010,Richard III (1995),22-Jan-96,http://us.imdb.com/M/title-exact?Richard%20III%20(1995),0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,1,011,Seven (Se7en) (1995),1-Jan-95,http://us.imdb.com/M/title-exact?Se7en%20(1995),0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,1,0,012,"Usual Suspects, The (1995)",14-Aug-95,"http://us.imdb.com/M/title-exact?Usual%20Suspects,%20The%20(1995)",0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,1,0,013,Mighty Aphrodite (1995),30-Oct-95,http://us.imdb.com/M/title-exact?Mighty%20Aphrodite%20(1995),0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,014,"Postino, Il (1994)",1-Jan-94,"http://us.imdb.com/M/title-exact?Postino,%20Il%20(1994)",0,0,0,0,0,0,0,0,1,0,0,0,0,0,1,0,0,0,015,Mr. Holland's Opus (1995),29-Jan-96,http://us.imdb.com/M/title-exact?Mr.%20Holland's%20Opus%20(1995),0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,016,French Twist (Gazon maudit) (1995),1-Jan-95,http://us.imdb.com/M/title-exact?Gazon%20maudit%20(1995),0,0,0,0,0,1,0,0,0,0,0,0,0,0,1,0,0,0,017,From Dusk Till Dawn (1996),5-Feb-96,http://us.imdb.com/M/title-exact?From%20Dusk%20Till%20Dawn%20(1996),0,1,0,0,0,1,1,0,0,0,0,1,0,0,0,0,1,0,018,"White Balloon, The (1995)",1-Jan-95,http://us.imdb.com/M/title-exact?Badkonake%20Sefid%20(1995),0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,019,Antonia's Line (1995),1-Jan-95,http://us.imdb.com/M/title-exact?Antonia%20(1995),0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,020,Angels and Insects (1995),1-Jan-95,http://us.imdb.com/M/title-exact?Angels%20and%20Insects%20(1995),0,0,0,0,0,0,0,0,1,0,0,0,0,0,1,0,0,0,021,Muppet Treasure Island (1996),16-Feb-96,http://us.imdb.com/M/title-exact?Muppet%20Treasure%20Island%20(1996),0,1,1,0,0,1,0,0,0,0,0,0,1,0,0,0,1,0,022,Braveheart (1995),16-Feb-96,http://us.imdb.com/M/title-exact?Braveheart%20(1995),0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,1,023,Taxi Driver (1976),16-Feb-96,http://us.imdb.com/M/title-exact?Taxi%20Driver%20(1976),0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,1,0,024,Rumble in the Bronx (1995),23-Feb-96,http://us.imdb.com/M/title-exact?Hong%20Faan%20Kui%20(1995),0,1,1,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,025,"Birdcage, The (1996)",8-Mar-96,"http://us.imdb.com/M/title-exact?Birdcage,%20The%20(1996)",0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,026,"Brothers McMullen, The (1995)",1-Jan-95,"http://us.imdb.com/M/title-exact?Brothers%20McMullen,%20The%20(1995)",0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,027,Bad Boys (1995),1-Jan-95,http://us.imdb.com/M/title-exact?Bad%20Boys%20(1995),0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,028,Apollo 13 (1995),1-Jan-95,http://us.imdb.com/M/title-exact?Apollo%2013%20(1995),0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,0,1,0,029,Batman Forever (1995),1-Jan-95,http://us.imdb.com/M/title-exact?Batman%20Forever%20(1995),0,1,1,0,0,1,1,0,0,0,0,0,0,0,0,0,0,0,030,Belle de jour (1967),1-Jan-67,http://us.imdb.com/M/title-exact?Belle%20de%20jour%20(1967),0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,031,Crimson Tide (1995),1-Jan-95,http://us.imdb.com/M/title-exact?Crimson%20Tide%20(1995),0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,1,1,032,Crumb (1994),1-Jan-94,http://us.imdb.com/M/title-exact?Crumb%20(1994),0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,033,Desperado (1995),1-Jan-95,http://us.imdb.com/M/title-exact?Desperado%20(1995),0,1,0,0,0,0,0,0,0,0,0,0,0,0,1,0,1,0,034,"Doom Generation, The (1995)",1-Jan-95,"http://us.imdb.com/M/title-exact?Doom%20Generation,%20The%20(1995)",0,0,0,0,0,1,0,0,1,0,0,0,0,0,0,0,0,0,035,Free Willy 2: The Adventure Home (1995),1-Jan-95,http://us.imdb.com/M/title-exact?Free%20Willy%202:%20The%20Adventure%20Home%20(1995),0,0,1,0,1,0,0,0,1,0,0,0,0,0,0,0,0,0,036,Mad Love (1995),1-Jan-95,http://us.imdb.com/M/title-exact?Mad%20Love%20(1995),0,0,0,0,0,0,0,0,1,0,0,0,0,0,1,0,0,0,037,Nadja (1994),1-Jan-94,http://us.imdb.com/M/title-exact?Nadja%20(1994),0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,038,"Net, The (1995)",1-Jan-95,"http://us.imdb.com/M/title-exact?Net,%20The%20(1995)",0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,0,039,Strange Days (1995),1-Jan-95,http://us.imdb.com/M/title-exact?Strange%20Days%20(1995),0,1,0,0,0,0,1,0,0,0,0,0,0,0,0,1,0,0,040,"To Wong Foo, Thanks for Everything! Julie Newmar (1995)",1-Jan-95,"http://us.imdb.com/M/title-exact?To%20Wong%20Foo,%20Thanks%20for%20Everything!%20Julie%20Newmar%20(1995)",0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,041,Billy Madison (1995),1-Jan-95,http://us.imdb.com/M/title-exact?Billy%20Madison%20(1995),0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,042,Clerks (1994),1-Jan-94,http://us.imdb.com/M/title-exact?Clerks%20(1994),0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,043,Disclosure (1994),1-Jan-94,http://us.imdb.com/M/title-exact?Disclosure%20(1994),0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,1,0,044,Dolores Claiborne (1994),1-Jan-94,http://us.imdb.com/M/title-exact?Dolores%20Claiborne%20(1994),0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,1,0,045,Eat Drink Man Woman (1994),1-Jan-94,http://us.imdb.com/M/title-exact?Yinshi%20Nan%20Nu%20(1994),0,0,0,0,0,1,0,0,1,0,0,0,0,0,0,0,0,0,046,Exotica (1994),1-Jan-94,http://us.imdb.com/M/title-exact?Exotica%20(1994),0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,047,Ed Wood (1994),1-Jan-94,http://us.imdb.com/M/title-exact?Ed%20Wood%20(1994),0,0,0,0,0,1,0,0,1,0,0,0,0,0,0,0,0,0,048,Hoop Dreams (1994),1-Jan-94,http://us.imdb.com/M/title-exact?Hoop%20Dreams%20(1994),0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,049,I.Q. (1994),1-Jan-94,http://us.imdb.com/M/title-exact?I.Q.%20(1994),0,0,0,0,0,1,0,0,0,0,0,0,0,0,1,0,0,0,050,Star Wars (1977),1-Jan-77,http://us.imdb.com/M/title-exact?Star%20Wars%20(1977),0,1,1,0,0,0,0,0,0,0,0,0,0,0,1,1,0,1,0

    The following table describes the data.

    Table 1 Movie data description

    Field

    Type

    Description

    movieId

    INT

    Movie ID

    movieTitle

    VARCHAR

    Movie name

    videoReleaseDate

    VARCHAR

    Release date

    IMDbURL

    VARCHAR

    IMDb link

    unknown

    INT

    Whether the movie type is unknown. If yes, the value is 1; otherwise, the value is 0.

    Action

    INT

    Whether the movie type is action. If yes, the value is 1; otherwise, the value is 0.

    Adventure

    INT

    Whether the movie type is adventure. If yes, the value is 1; otherwise, the value is 0.

    Animation

    INT

    Whether the movie type is animation. If yes, the value is 1; otherwise, the value is 0.

    Children

    INT

    Whether the movie type is children. If yes, the value is 1; otherwise, the value is 0.

    Comedy

    INT

    Whether the movie type is comedy. If yes, the value is 1; otherwise, the value is 0.

    Crime

    INT

    Whether the movie type is crime. If yes, the value is 1; otherwise, the value is 0.

    Documentary

    INT

    Whether the movie type is documentary. If yes, the value is 1; otherwise, the value is 0.

    Drama

    INT

    Whether the movie type is drama. If yes, the value is 1; otherwise, the value is 0.

    Fantasy

    INT

    Whether the movie type is fantasy. If yes, the value is 1; otherwise, the value is 0.

    FilmNoir

    INT

    Whether the movie type is noir. If yes, the value is 1; otherwise, the value is 0.

    Horror

    INT

    Whether the movie type is horror. If yes, the value is 1; otherwise, the value is 0.

    Musical

    INT

    Whether the movie type is musical. If yes, the value is 1; otherwise, the value is 0.

    Mystery

    INT

    Whether the movie type is mystery. If yes, the value is 1; otherwise, the value is 0.

    Romance

    INT

    Whether the movie type is romance. If yes, the value is 1; otherwise, the value is 0.

    SciFi

    INT

    Whether the movie type is science fiction. If yes, the value is 1; otherwise, the value is 0.

    Thriller

    INT

    Whether the movie type is thriller. If yes, the value is 1; otherwise, the value is 0.

    War

    INT

    Whether the movie type is war. If yes, the value is 1; otherwise, the value is 0.

    Western

    INT

    Whether the movie type is western. If yes, the value is 1; otherwise, the value is 0.

  • ratings.csv: 
    userId,movieId,rating,timestamp210,40,3,891035994224,29,3,888104457308,1,4,8877365327,32,4,89135093210,16,4,87788887799,4,5,886519097115,20,3,881171009138,26,5,879024232243,15,3,879987440293,5,3,888906576162,25,4,877635573135,23,4,87985776562,21,3,87937346059,23,5,88820530043,14,2,88395574519,4,4,8854128405,2,3,87563605372,48,4,880036718224,26,3,888104153299,14,4,877877775151,10,5,8795249216,14,5,883599249250,7,4,878089716268,2,2,875744173292,11,5,881104093181,3,2,878963441145,15,2,8752706551,33,4,878542699276,2,4,87479243618,26,4,88012973187,40,3,879876917272,12,5,879455254296,20,5,8841969215,17,4,875636198128,15,4,879968827287,1,5,87533408865,47,2,8792166721,20,4,887431883290,50,5,88047358245,25,4,881014015109,8,3,880572642157,25,3,886890787301,33,4,88207822862,12,4,879373613276,40,3,874791871269,22,1,89144807210,7,4,877892210244,17,2,880607205222,26,3,878183043185,23,4,883524249207,13,3,8755068398,22,5,879362183222,49,3,878183512200,11,5,88412954290,25,5,89138478915,25,3,879456204234,10,3,891227851295,39,4,879518279217,2,3,889069782189,20,5,89326446642,44,3,881108548268,21,3,875742822262,28,3,87979222090,22,4,891384357270,25,5,876954456194,23,4,879522819161,48,1,89117074558,9,4,88430432879,50,4,891271545221,48,5,875245462223,11,3,891550649292,9,4,88110414816,8,5,87772273617,13,3,885272654148,1,4,877019411280,1,4,891700426110,38,3,88698857490,12,5,891383241239,9,5,889180446311,9,4,884963365151,13,3,8795426882,50,5,8885520848,50,5,879362124286,44,3,87753217385,25,2,879452769274,50,5,878944679217,27,1,889070011181,14,1,878962392297,25,4,8749544971,47,4,8750721256,23,4,883601365222,22,5,878183285314,28,5,877888346291,15,5,87483366894,24,4,88587342383,43,4,88030869043,40,3,88395646844,15,4,878341343158,24,4,880134261151,12,5,87952436866,1,3,8836013245,1,4,875635748207,25,4,876079113109,1,4,880563619227,50,4,879035347181,1,3,878962392213,13,4,878955139121,14,5,891390014117,15,5,88012588785,13,3,879452866313,22,3,89101487043,5,4,87598142111,38,3,89190593672,28,4,880036824115,8,5,88117198295,1,5,879197329145,22,5,87527302166,7,3,883601355267,17,4,87897177325,25,5,885853415103,24,4,88041584787,9,4,87987793149,47,5,888068715135,39,3,879857931269,13,4,89144666299,50,5,885679998306,14,5,876503995291,7,5,874834481312,28,4,891698300184,36,3,889910195305,11,1,886323237198,7,4,884205317104,7,3,888465972293,39,3,888906804256,25,5,88215055292,15,3,8756401891,17,3,875073198214,42,5,89266813082,14,4,876311280305,50,5,886321799223,8,2,89155068491,28,4,891439243315,13,4,879821158269,9,4,891446246217,7,4,88906974149,7,4,88806730787,2,4,879876074268,1,3,875742341262,47,2,87979459984,12,5,883452874264,33,3,886122644224,20,1,888104487200,24,2,88412737092,24,3,875640448276,38,3,874792574286,34,5,87753470149,38,1,888068289311,5,3,884365853269,47,4,891448386194,4,4,87952139757,28,4,883698324108,50,4,879879739207,4,4,876198457181,16,1,87896299694,9,5,885872684234,20,4,89122797968,7,3,87697409613,14,4,88453872798,47,4,88049889853,24,3,879442538239,10,5,88918033863,20,3,875748004276,43,1,874791383272,48,4,879455143116,7,2,87645391526,25,3,89137372762,24,4,879372633295,47,5,87951816663,50,4,87574729249,17,2,888068651310,24,4,8794362427,44,5,891351728326,22,4,879874989213,12,5,878955409222,29,3,878184571249,11,5,879640868217,22,5,889069741189,1,5,893264174234,50,4,892079237296,48,5,88419709181,3,4,876592546151,15,4,87952487959,12,5,888204260246,8,3,884921245276,34,2,87793426497,50,5,884239471244,7,4,880602558298,8,5,8841827487,28,5,89135234141,28,4,890687353

    The following table describes the data.

    Table 2 Rating data description

    Field

    Type

    Description

    userId

    INT

    User ID

    movieId

    INT

    Movie ID

    rating

    INT

    Rating. The total score is 5 points.

    timestamp

    VARCHAR

    Timestamp

Preparing a Data Lake

This practice uses DWS as the data foundation.

For details about how to create a DWS cluster, see Creating a Cluster. The DWS cluster must meet the following requirements so that it can communicate with the DataArts Studio instance:
  • If the CDM cluster in the DataArts Studio instance and the DWS cluster are in different regions, a public network or a dedicated connection is required.
  • If the CDM cluster in the DataArts Studio instance and the DWS cluster are in the same region, VPC, subnet, and security group, they can communicate with each other by default. If they are in the same VPC but in different subnets or security groups, you must configure routing rules and security group rules. For details about how to configure routing rules, see Configuring Routing Rules. For details about how to configure security group rules, see Configuring Security Group Rules.
  • The DWS cluster and the DataArts Studio workspace belong to the same enterprise project. If they do not, you can modify the enterprise project of the workspace.

After creating a DWS cluster, you need to create a DWS connection in Management Center, create a database and schema through the DataArts Factory module, and run an SQL statement to create a DWS table. The procedure is as follows:

  1. Log in to the DataArts Studio console by following the instructions in Accessing the DataArts Studio Instance Console.
  2. On the DataArts Studio console, locate a workspace and click Management Center.
  3. On the displayed Manage Data Connections page, click Create Data Connection.

    Figure 1 Creating a data connection

  4. Configure the following parameters and click Save. Figure 2 shows the important parameters.

    • Data Connection Type: Select DWS.
    • Name: Enter dws_link.
    • Tag: This parameter is optional. You can enter the name of a new tag or select an existing tag from the drop-down list box.
    • Applicable Modules: Retain the default settings.
    • Workspace Member Visibility: Retain the default setting.
    • SSL Encryption: Retain the same setting as that for the source DWS cluster.
    • Manual: Select Cluster Name Mode. IP and Port are automatically set.
    • DWS Cluster Name: Select the DWS cluster that you have created.
    • JDBC IP Address or Domain Name: Select a DWS ELB domain name or ELB IP address. The former is recommended. If it is not supported, select an ELB IP address.
    • KMS encryption key: Select a KMS key used to encrypt sensitive data. If no KMS key is available, click Access KMS to go to the KMS console and create one.
    • Agent: Select a CDM cluster as the connection agent. The CDM cluster must be able to communicate with the DWS cluster. In this example, you can select the DataArts Migration cluster that is automatically created when the DataArts Studio instance is created.
    • Username: Enter the database username that you specified when creating the DWS cluster. The default username is dbadmin.
    • Password: Enter the password that you specified when creating the DWS cluster for accessing the database.
    • Real-time Metadata Synchronization: Retain the default setting.
    Figure 2 DWS connection parameters

  5. Go to the DataArts Factory page.

    Figure 3 DataArts Factory page

  6. Create a DWS database and a database schema.

    1. Right-click the DWS connection and select Create Database to create a database named demo for storing data tables.
      Figure 4 Creating a database
    2. Expand the DWS connection directory to the database schema level, right-click schemas, and select Create Schema to create a schema named dgc for storing data tables.
      Figure 5 Creating a database schema

  7. Create a DWS SQL script used to create data tables by entering DWS SQL statements in the editor.

    Figure 6 Creating a script

  8. In the SQL editor, enter the following SQL statements and click Execute to create data tables. Among them, movies_item and ratings_item are original data tables, to which data will be migrated from OBS through CDM. top_rating_movie and top_active_movie are result tables which store analysis results.

    SET SEARCH_PATH TO dgc;CREATE TABLE IF NOT EXISTS movies_item(    movieId INT,    movieTitle VARCHAR,    videoReleaseDate VARCHAR,    IMDbURL VARCHAR,    unknown INT,    Action INT,    Adventure INT,    Animation INT,    Children INT,    Comedy INT,    Crime INT,    Documentary INT,    Drama INT,    Fantasy INT,    FilmNoir INT,    Horror INT,    Musical INT,    Mystery INT,    Romance INT,    SciFi INT,    Thriller INT,    War INT,    Western INT);CREATE TABLE IF NOT EXISTS ratings_item(    userId INT,  movieId INT,  rating INT,  timestamp VARCHAR);CREATE TABLE IF NOT EXISTS top_rating_movie(  movieTitle VARCHAR,  avg_rating float,  rating_user_number int); CREATE TABLE IF NOT EXISTS top_active_movie(  movieTitle VARCHAR,  avg_rating float,  rating_user_number int);
    Figure 7 Creating data tables
    The key parameters are as follows:
    • Data Connection: DWS data connection created in Step 4
    • Database: database created in Step 6

  9. After the script is executed successfully, run the following script to check whether the data tables are created successfully. After confirming that the data tables are created successfully, you can close the script as it is no longer needed.

    SELECT * FROM pg_tables;

Preparing Authentication Data

If you want to migrate OBS data using CDM, you need AK/SK authentication. Therefore, you must create an AK/SK pair.
  • Access Key ID (AK): indicates the ID of the access key, which is a unique identifier associated with a secret access key and is used in conjunction with a secret access key to sign requests cryptographically.
  • Secret Access Key (SK): indicates the key used with its associated AK to cryptographically sign requests and identify request senders to prevent requests from being modified.
To obtain an access key, perform the following steps:
  1. Log in to the management console, move the cursor to the username in the upper right corner, and select My Credentials from the drop-down list.
  2. On the My Credentials page, choose Access Keys, and click Create Access Key. See Figure 8.
    Figure 8 Clicking Create Access Key
  3. Click OK and save the access key file as prompted. The access key file will be saved to your browser's configured download location. Open the credentials.csv file to view Access Key Id and Secret Access Key.
    • Only two access keys can be added for each user.
    • To ensure access key security, the access key is automatically downloaded only when it is generated for the first time and cannot be obtained from the management console later. Keep them properly.