Python Example Code

Function

In a Spark application, use Spark to call a Hive API to operate a Hive table, and write the data analysis result of the Hive table to an HBase table.

Example Code

PySpark does not provide HBase related APIs. In this example, Python with the Java programming language invoked is used.

The following code snippets are used as an example. For complete codes, see SparkHivetoHbasePythonExample.

# -*- coding:utf-8 -*-

from py4j.java_gateway import java_import
from pyspark.sql import SparkSession

# Create the SparkSession
spark = SparkSession\
        .builder\
        .appName("SparkHivetoHbase") \
        .getOrCreate()

# Import the class that will run into sc._jvm. 
java_import(spark._jvm, 'com.huawei.bigdata.spark.examples.SparkHivetoHbase')

# Create a class instance and invoke the method.
spark._jvm.SparkHivetoHbase().hivetohbase(spark._jsc)

# Stop the SparkSession
spark.stop()