Commissioning the Doris Application in the Linux Environment

Scenario

After the program code is developed, you can upload the JAR package to the Linux environment to run the application.

Prerequisite

The JDK has been installed in the Linux environment. The version number must be the same as the version of the JDK used by the IntelliJ IDEA to export JAR packages.
If the Linux host is not a node in the cluster, you need to set the mapping between the host name and IP address in the hosts file of the node. Host names and IP addresses must be in one-to-one mapping.

Compiling and Running JDBC and SpringBoot Programs

Export a JAR package.

You can build a JAR file in either of the following ways:
- Method 1:
  Choose Maven > Sample projectname > Lifecycle > clean, double click clean to run the clean command of Maven.
  
  Choose Maven > Sample project name > Lifecycle > install, double click install to run the install command of Maven.
  Figure 1 Maven tools: clean and install
- Method 2: Go to the directory where the pom.xml file is located in the Terminal window in the lower part of the IDEA, and run the mvn clean install command to compile the pom.xml file.
  Figure 2 Enter mvn clean compile in the IDEA Terminal text box.
  
  After the compilation is completed, the message BUILD SUCCESS is displayed, the target directory is generated, and the generated JAR file is stored in the target directory.
Check whether Doris is connected to Spring Boot.
- If yes, perform the following steps to run the sample:
  1. Create a running directory in the Linux environment and place doris-rest-client-example-*.jar in the target directory to the directory.
  2. Switch to the running directory and run the following command to run the JAR package:
    java -jar doris-rest-client-example-*.jar
- If no, go to 4.
Export the JAR package on which the sample project depends.

Access the directory where the pom.xml file is located in the Terminal window or other command-line tools. Run the following command:

mvn dependency:copy-dependencies -DoutputDirectory=lib

The lib folder is generated in the directory where the pom.xml file is located. The lib folder contains the JAR packages on which the sample project depends.
Obtain the JAR packages and configuration files required by the.
1. In the Linux environment, create a directory, for example, /opt/test, and create a subdirectory lib. Export the JAR package on which the sample project depends 1 Generated JAR packages, and 3 Upload the generated JAR package to the lib directory of the Linux operating system.
2. Create the run.sh script in the /opt/test root directory, modify the following content, and save the modification:
```
#!/bin/sh
BASEDIR=`cd $(dirname $0);pwd`
cd ${BASEDIR}
for file in ${BASEDIR}/lib/*.jar
do
i_cp=$i_cp:$file
echo "$file"
done

java -cp .${i_cp} com.huawei.bigdata.doris.example.JDBCExample
```
  In the preceding command, com.huawei.bigdata.doris.example.JDBCExample is used as an example.
Switch to /opt/test and run the following command to run the JAR package:

sh run.sh

Compiling and Running the Stream Load Program (MRS 3.3.1-LTS and Later Versions)

Create a new directory in the Linux environment, for example, /opt/test, and upload the test.csv file from the ../src/main/resources/ directory of the doris-stream-load-example sample project obtained in 1 to this directory.
Change the value of the getHttpPost parameter in the doris-stream-load-example sample code to /opt/test/test.csv.
Export a JAR package.

You can build a JAR file in either of the following ways:
- Method 1:
  Choose Maven > Sample projectname > Lifecycle > clean, double click clean to run the clean command of Maven.
  
  Choose Maven > Sample project name > Lifecycle > install, double click install to run the install command of Maven.
  Figure 3 Maven tools: clean and install
- Method 2: Go to the directory where the pom.xml file is located in the Terminal window in the lower part of the IDEA, and run the mvn clean install command to compile the pom.xml file.
  Figure 4 Enter mvn clean compile in the IDEA Terminal text box.
  
  After the compilation is completed, the message BUILD SUCCESS is displayed, the target directory is generated, and the generated JAR file is stored in the target directory.

Place doris-stream-load-example-*.jar in the target directory to the directory created in 1, for example, /opt/test.
Switch to the JDK directory and run the following command:

java -jar doris-stream-load-example-*-jar-with-dependencies.jar

Compiling and Running a Flink Jar Program to Write Data to Doris (MRS 3.6.0-LTS and later versions)

Export the JAR package.
1. Choose Maven > Sample Project Name > Lifecycle > clean, and double-click clean to run the Maven clean command.
2. Choose Maven > Sample Project Name > Lifecycle > package, and double-click package to compile the program.
3. After the compilation is complete, the message BUILD SUCCESS is displayed, and the target directory is generated. The JAR package is generated in the target directory.
Log in to the Flink WebUI as a user with the FlinkServer administrator permissions. On the Job Management page, click Create Job, set the following parameters, and click OK.
- Type: Select Flink Jar.
- Name: Enter a custom job name.
- Job Type: Select Batch Job.
After the job is created, the job development page is displayed. On the page for uploading the JAR file, click Select on the right of Local JAR File to upload the JAR package exported in 1. You can set other parameters as required. For details, see Creating a Job.
Click Save to save the configuration, and then click Submit to run the job.

Viewing Linux Commissioning Results

If the doris-jdbc-example example is successfully executed, the following information is displayed:

2023-08-17 23:13:13,473 | INFO  | main | Start execute doris example. | com.huawei.bigdata.doris.example.JDBCExample.main(JDBCExample.java:41)
2023-08-17 23:13:13,885 | INFO  | main | Start create database. | com.huawei.bigdata.doris.example.JDBCExample.main(JDBCExample.java:44)
2023-08-17 23:13:13,949 | INFO  | main | Database created successfully. | com.huawei.bigdata.doris.example.JDBCExample.main(JDBCExample.java:46)
2023-08-17 23:13:13,950 | INFO  | main | Start create table. | com.huawei.bigdata.doris.example.JDBCExample.main(JDBCExample.java:49)
2023-08-17 23:13:14,132 | INFO  | main | Table created successfully. | com.huawei.bigdata.doris.example.JDBCExample.main(JDBCExample.java:51)
2023-08-17 23:13:14,133 | INFO  | main | Start to insert data into the table. | com.huawei.bigdata.doris.example.JDBCExample.main(JDBCExample.java:54)
2023-08-17 23:13:14,733 | INFO  | main | Inserting data to the table succeeded. | com.huawei.bigdata.doris.example.JDBCExample.main(JDBCExample.java:56)
2023-08-17 23:13:14,733 | INFO  | main | Start to query table data. | com.huawei.bigdata.doris.example.JDBCExample.main(JDBCExample.java:59)
2023-08-17 23:13:15,079 | INFO  | main | Start to print query result. | com.huawei.bigdata.doris.example.JDBCExample.query(JDBCExample.java:121)
2023-08-17 23:13:15,079 | INFO  | main | c1  c2  c3   | com.huawei.bigdata.doris.example.JDBCExample.query(JDBCExample.java:126)
2023-08-17 23:13:15,079 | INFO  | main | 0  0  0   | com.huawei.bigdata.doris.example.JDBCExample.query(JDBCExample.java:134)
2023-08-17 23:13:15,080 | INFO  | main | 1  10  100   | com.huawei.bigdata.doris.example.JDBCExample.query(JDBCExample.java:134)
2023-08-17 23:13:15,080 | INFO  | main | 2  20  200   | com.huawei.bigdata.doris.example.JDBCExample.query(JDBCExample.java:134)
2023-08-17 23:13:15,080 | INFO  | main | 3  30  300   | com.huawei.bigdata.doris.example.JDBCExample.query(JDBCExample.java:134)
2023-08-17 23:13:15,080 | INFO  | main | 4  40  400   | com.huawei.bigdata.doris.example.JDBCExample.query(JDBCExample.java:134)
2023-08-17 23:13:15,080 | INFO  | main | 5  50  500   | com.huawei.bigdata.doris.example.JDBCExample.query(JDBCExample.java:134)
2023-08-17 23:13:15,080 | INFO  | main | 6  60  600   | com.huawei.bigdata.doris.example.JDBCExample.query(JDBCExample.java:134)
2023-08-17 23:13:15,080 | INFO  | main | 7  70  700   | com.huawei.bigdata.doris.example.JDBCExample.query(JDBCExample.java:134)
2023-08-17 23:13:15,081 | INFO  | main | 8  80  800   | com.huawei.bigdata.doris.example.JDBCExample.query(JDBCExample.java:134)
2023-08-17 23:13:15,081 | INFO  | main | 9  90  900   | com.huawei.bigdata.doris.example.JDBCExample.query(JDBCExample.java:134)
2023-08-17 23:13:15,081 | INFO  | main | Querying table data succeeded. | com.huawei.bigdata.doris.example.JDBCExample.main(JDBCExample.java:61)
2023-08-17 23:13:15,081 | INFO  | main | Start to delete the table. | com.huawei.bigdata.doris.example.JDBCExample.main(JDBCExample.java:64)
2023-08-17 23:13:15,114 | INFO  | main | Table deleted successfully. | com.huawei.bigdata.doris.example.JDBCExample.main(JDBCExample.java:66)
2023-08-17 23:13:15,124 | INFO  | main | Doris example execution successfully. | com.huawei.bigdata.doris.example.JDBCExample.main(JDBCExample.java:71)

Result of interconnecting Doris with SpringBoot:
- Run the Doris JDBC sample program through Spring Boot.
  Enter http://IP address of the node that runs the sample:8080/doris/example/executesql in the address box of the browser. IDEA prints logs. The following figure shows the returned information.
  
  Figure 5 Doris JDBC sample runtime information
- Run the DBalancer sample program through Spring Boot.
  Enter http://IP address of the node that runs the sample:8080/doris/example/dbalancer/executesql in the address box of the browser. IDEA prints logs. The following figure shows the returned information.
  
  Figure 6 DBalancer sample runtime information

After the doris-stream-load-example sample runs successfully, the following information is displayed:

Versions prior to MRS 3.6.0-LTS:

Start create database.
Database created successfully.
Start create table.
Table created successfully.
{
    "TxnId": 27,
    "Label": "3661e112-f0e9-4aaa-80c3-4c2b7aefeefb",
    "Comment": "",
    "TwoPhaseCommit": "false",
    "Status": "Success",
    "Message": "OK",
    "NumberTotalRows": 4,
    "NumberLoadedRows": 4,
    "NumberFilteredRows": 0,
    "NumberUnselectedRows": 0,
    "LoadBytes": 141,
    "LoadTimeMs": 387,
    "BeginTxnTimeMs": 58,
    "StreamLoadPutTimeMs": 137,
    "ReadDataTimeMs": 0,
    "WriteDataTimeMs": 94,
    "CommitAndPublishTimeMs": 95
}
curl{
    "TxnId": 27,
    "Label": "3661e112-f0e9-4aaa-80c3-4c2b7aefeefb",
    "Comment": "",
    "TwoPhaseCommit": "false",
    "Status": "Success",
    "Message": "OK",
    "NumberTotalRows": 4,
    "NumberLoadedRows": 4,
    "NumberFilteredRows": 0,
    "NumberUnselectedRows": 0,
    "LoadBytes": 141,
    "LoadTimeMs": 387,
    "BeginTxnTimeMs": 58,
    "StreamLoadPutTimeMs": 137,
    "ReadDataTimeMs": 0,
    "WriteDataTimeMs": 94,
    "CommitAndPublishTimeMs": 95
}

After successful execution, you can also connect to Doris from the MySQL client and run the following command to view the imported:

select * from doris_test_sink;

Click to enlarge

MRS 3.6.0-LTS and later versions:

... Stream load job result: {
    "TxnId": 17,
    "Label": "label_stream_load_400c9155-c74e-4c1c-bcfc-0aa5991d309b",
    "Comment": "",
    "TwoPhaseCommit": "false",
    "Status": "Success",
    "Message": "OK",
    "NumberTotalRows": 1,
    "NumberLoadedRows": 1,
    "NumberFilteredRows": 0,
    "NumberUnselectedRows": 0,
    "LoadBytes": 34,
    "LoadTimeMs": 361,
    "BeginTxnTimeMs": 64,
    "StreamLoadPutTimeMs": 171,
    "ReadDataTimeMs": 0,
    "WriteDataTimeMs": 47,
    "ReceiveDataTimeMs": 278,
    "CommitAndPublishTimeMs": 77
}
 | com.example.stream.load.dorisstreamloadexample.DorisSecurityStreamLoader.testHttpsClient(DorisSecurityStreamLoader.java:198)
Process finished with exit code 0

Connect to Doris on the MySQL client and run the following command to view the imported data:

select * from test_2.doris_test_sink;