Updated on 2024-04-02 GMT+08:00

Accessing Hive Data Sources Using HSBroker

This section applies to MRS 3.3.0 or later.

This section describes how to use HSBroker to connect to HetuEngine, assemble SQL statements, and send the them to HetuEngine for execution to add, delete, modify, and query Hive data sources.

import jaydebeapi
 
driver = "io.XXX.jdbc.XXXDriver"
 
# need to change the value based on the cluster information
url = "jdbc:XXX://192.168.37.61:29861,192.168.37.62:29861/hive/default?serviceDiscoveryMode=hsbroker"
user = "YourUserName"
tenant = "YourTenant"
jdbc_location = "Your file path of the jdbc jar"
 
sql = "show catalogs"
 
if __name__ == '__main__':
    conn = jaydebeapi.connect(driver, url, {"user": user,
                                            "SSL": "false",
                                            "tenant": tenant},
                              [jdbc_location])
    curs = conn.cursor()
    curs.execute(sql)
    result = curs.fetchall()
    print(result)
    curs.close()
    conn.close()

The following table describes the parameters in the preceding code.

Table 1 Parameter description

Parameter

Description

url

jdbc:XXX://HSBroker1_IP:HSBroker1_Port,HSBroker2_IP:HSBroker2_Port,HSBroker3_IP:HSBroker3_Port/catalog/schema?serviceDiscoveryMode=hsbroker

NOTE:
  • xxx: driver name, which is subjective to the real-world code you use.
  • catalog and schema indicate the names of the catalog and schema to be connected to the JDBC client, respectively.
  • HSBroker_IP:HSBroker_Port indicates the HSBroker URL. Use commas (,) to separate multiple URLs, for example, 192.168.81.37:2181,192.168.195.232:2181,192.168.169.84:2181.

user

Username for accessing HetuServer, that is, the username of the human-machine user created in the cluster.

tenant

Tenant resource queue for accessing HetuEngine compute instances

jdbc_location

Full path of the hetu-jdbc-XXX.jar package obtained in Configuring the Python3 Sample Project.

  • Example path in Windows: D:\\hetu-examples-python3\\hetu-jdbc-XXX.jar
  • Example path in Linux: /opt/hetu-examples-python3/hetu-jdbc-XXX.jar