Configuring the JOIN between Tables and Streams

Procedure

1. Ensure that clusters including HDFS, Yarn, secure Redis, and Flink are successfully installed.
   

   Redis in normal mode can be installed to secure clusters. To install Redis in security mode, set REDIS_SECURITY_ENABLED to true during installation. To install Redis in normal mode, set REDIS_SECURITY_ENABLED to false. When Redis is in security mode, users are authenticated using Kerberos, which affects performance. When Redis is in normal mode, users can directly access intranet clients without the need to be authenticated. Redis in security mode is used as an example in this section.

2. Create a Redis cluster, add the Redis user and grant permissions, and download the user.keytab and krb5.conf files. For detailed information about Redis, see Using Redis.
3. Modify the import.properties and read.properties files, which are in the config directory specified in the sample code.
   • A configuration example of the import.properties file is provided as follows:

     #path to read csv files, it can be file or directory
     CsvPath=config/configtable.csv
     
     #csv file headers exist in file first line or not
     CsvHeaderExist=true
     #csv file headers, also the redis field names
     #Notice: if CsvHeaderExist false, you must set it, if CsvHeaderExist true, it read from csv file
     ColumnNames=username,age,company,workLocation,educational,workYear,phone,nativeLocation,school
     
     #redis security mode open or not
     Redis_Security=true
     #redis hostname/ip and port when you need to connect to redis
     Redis_IP_Port=SZV1000064084:22400,SZV1000064082:22400,SZV1000064085:22400
     #redis user principal
     Redis_Principal=test11@<System domain name>
     #redis keytab file path
     Redis_KeytabFile=config/user.keytab
     #redis krb5 file path
     Redis_Krb5File=config/krb5.conf

     

     You can log in to FusionInsight Manager, choose System > Permission > Domain and Mutual Trust, and check the value of Local Domain, which is the current system domain name.

   • The following is an example of configurations in the read.properties file:

     #the redis field names, configure which fields you need to read, Notice you need English field names
     ReadFields=username,age,company,workLocation,educational,workYear,phone,nativeLocation,school
     #redis security mode open or not
     Redis_Security=true
     #redis hostname/ip and port when you need to connect to redis
     Redis_IP_Port=SZV1000064084:22400,SZV1000064082:22400,SZV1000064085:22400
     #redis user principal
     Redis_Principal=test11@<System domain name>
     #redis keytab file path
     Redis_KeytabFile=config/user.keytab
     #redis krb5 file path
     Redis_Krb5File=config/krb5.conf

4. On the Flink client, create a config directory, and copy the user.keytab, krb5.conf, log.txt, configtable.csv, import.properties, and read.properties files to the config directory, for example, /opt/client/Flink/flink/config/configtable.csv.
   

   log.txt and configtable.csv files are in the data directory specified in the sample code.