Compiling and Running an HBase Application

Updated on 2024-08-16 GMT+08:00

View PDF

You can run applications in the Windows development environment after application code is developed.

Procedure

When you use REST APIs to operate HBase clusters on Windows, the JDK version must be jdk1.8.0_60 or later. Obtain the cacerts file of JDK from the cluster environment and copy the /opt/Bigdata/jdk/jre/lib/security/cacerts file to C:\Program Files\Java\jdk1.8.0_60\jre\lib\security in the JDK environment on Windows. (Skip this step if you do not use REST APIs to operate HBase clusters.)
Configure a mapping between the cluster IP address and host name on Windows. Log in to the cluster background, run the cat /etc/hosts command, and copy the mapping between IP addresses and host names in the hosts file to C:\Windows\System32\drivers\etc\hosts. The host name is subject to the query result.
```
192.168.0.90 node-master1BedB.089d8c43-12d5-410c-b980-c2728a305be3.com
192.168.0.129 node-ana-corezLaR.089d8c43-12d5-410c-b980-c2728a305be3.com
```
NOTE:
You can use either of the following method to access an MRS cluster to operate HBase on Windows.
- Apply for a Windows ECS to access the MRS cluster to operate HBase. Run the sample code after the development environment is installed. To apply for ECS to access the MRS cluster, perform the following steps:
  1. On the Active Clusters page, click the name of an existing cluster.
    On the cluster details page, record the AZ, VPC, Cluster Manager IP Address of the cluster, and Default Security Group of the Master node.
  2. On the ECS management console, create an ECS.
    The AZ, VPC, and security group of ECS must be the same as those of the cluster to be accessed.
    
    Select a Windows public image.
    
    For details about other configuration parameters, see Elastic Cloud Server > Quick Start > Purchasing and Logging In to a Windows ECS.
- Use the local host to access the MRS cluster to operate HBase. Bind an EIP to all HBase nodes in the MRS cluster. When configuring the mapping between the cluster IP address and host name on the local host (Windows host), replace the IP address with the EIP corresponding to the host name, change the IP addresses of the kdc, admin_server, kpasswd_server, kdc_listen, kadmind_listen, and kpasswd_listen parameters in the krb5.conf file. (Skip this step if the cluster with a single master does not have the last three parameters.) Ensure that the cluster corresponds to the EIP in KrbServer. (Skip this step if the common cluster does not have the Kerberos function enabled.) Then run the sample code. To bind an EIP, perform the following steps:
  1. On the VPC management console, apply for an EIP and bind it to ECS.
    For details, see Virtual Private Cloud > User Guide > Elastic IP Address > Assigning an EIP and Binding It to an ECS.
  2. Open security group rules for the MRS cluster.
    Add security group rules to the security groups of the Master and Core nodes in the cluster to enable the ECS to access the cluster. If the cluster is a security cluster, add UDP ports 21731 and 21732, TCP ports 21730, 21731, and 21732, RPC ports of HBase HMaster and RegionServer instances, and ZooKeeper service ports to the inbound rule of the security group. For details, see Virtual Private Cloud > User Guide > Security > Security Group > Adding a Security Group Rule.
Modify running environment configuration.
Modify the following parameters in the hbase-site.xml file built in sample code based on actual requirements:
- hbase.zookeeper.quorum: hostname of the ZooKeeper instance. Use commas (,) to separate hostnames of multiple instances. This parameter is mandatory.
- hbase.regionserver.kerberos.principal: principal of the RegionServer, which must be the same as the principal of the Master. This parameter is mandatory only for a cluster with the Kerberos function enabled.
- hbase.master.kerberos.principal: principal of the HMaster, which must be the same as the principal of the RegionServer. This parameter is mandatory only for a cluster with the Kerberos function enabled.
- hadoop.security.authentication: Hadoop authentication mode. Set this parameter to kerberos only for a cluster with the Kerberos function enabled. Otherwise, this parameter does not need to be set.
- hbase.security.authentication: HBase authentication mode. Set this parameter to kerberos only for a cluster with the Kerberos function enabled. Otherwise, this parameter does not need to be set.
NOTE:

You can log in to any Master node and obtain the values of the preceding parameters from the HBase client configuration (/opt/client/HBase/hbase/conf). For example, to obtain the value of hbase.zookeeper.quorum, log in to any Master node and run the following command:

grep "hbase.zookeeper.quorum" /opt/client/HBase/hbase/conf/* -R -A1
Modify the sample code.
1. Currently, there are three types of HBase APIs in the sample code: common API, HFS API (no longer supported in MRS 1.9.x), and RESTful API. When commissioning different APIs to operate HBase, you can comment out the invoking of other APIs. In this example, common APIs are used to operate HBase and the main method contains only the following code snippet.
```
public static void main(String[] args) {
  try {
    init();
    login();
  } catch (IOException e) {
      LOG.error("Failed to login because ", e);
      return;
  }
  // getDefaultConfiguration();
  conf = HBaseConfiguration.create();
  // test hbase API
  HBaseExample oneSample;
  try {
    oneSample = new HBaseExample(conf);
    oneSample.test();
  } catch (Exception e) {
    LOG.error("Failed to test HBase because ", e);
  }
  LOG.info("-----------finish HBase-------------------");
}
```
2. Change the value of ZOOKEEPER_DEFAULT_SERVER_PRINCIPAL in the com.huawei.bigdata.hbase.examples.TestMain class of the sample project. Skip this step for a cluster with Kerberos authentication disabled.
```
private static final String ZOOKEEPER_DEFAULT_SERVER_PRINCIPAL = "zookeeper/hadoop.4a049bf4_e74e_4545_9291_6fc6098d3723.com";
```
  NOTE:
  - During ZooKeeper authentication in a security cluster, a four-letter command is used to obtain the principal of the ZooKeeper server. If the ZooKeeper instance bears heavy loads or is unstable, the authentication fails because the principal of the ZooKeeper server fails to be obtained using the four-letter command. In this case, you need to use the client to transfer the value to the environment to avoid authentication failures.
  - You can log in to any Master node and run the following command to obtain the value:
    grep "CLIENT_ZOOKEEPER_PRINCIPAL" /opt/client/HBase/hbase/conf/*
3. Modify the hbaseclient.properties file in the src/main/resources directory of the sample project. userKeytab.path and krb5.conf.path indicate the file addresses obtained in Preparing an HBase Application Development User. Skip this step for a cluster with Kerberos authentication disabled.
```
user.name=hbaseuser
userKeytabName=userKeytab.path
krb5ConfName=krb5.conf.path
```
4. If you need to modify rest.server.info in hbaseclient.properties when using REST APIs, make it correspond to ip:port (default port: 21309) of the rest server.
```
rest.server.info=10.10.10.10:21309
```
  NOTE:
  Obtain the core-site.xml, hdfs-site.xml, and hbase-site.xml files from the Master node of the cluster and save the files to the resources directory of the sample project, that is, src/main/resources. The path for obtaining the files is /opt/client/HBase/hbase/conf. For MRS 1.9.2 or later, you only need to modify the configurations in the built-in hbase-site.xml file.
  
  If you use ECS to access HBase, you can use the IP address of RESTServer directly. If you use the local host to access HBase, use the EIP bound to RESTServer as the IP address of RESTServer.
  
  The HIndexExample sample project is supported only in MRS 1.9.2 or later. Pay attention to the current cluster version.
  
  The HFSSample sample project is removed from MRS 1.9.x. Pay attention to the current cluster version.
Run the sample project.

In the development environment (for example, Eclipse), right-click TestMain.java and choose Run as > Java Application from the shortcut menu to run the corresponding application project.