Submitting a Topology in IntelliJ IDEA Remotely

Updated on 2022-08-16 GMT+08:00

View PDF

Scenario

Storm supports remote topology submitting in IntelliJ IDEA. In current example code, only WordCountTopology can be submitted remotely. If you want to submit other topologies remotely, implement the remote submission function first. For details, see WordCountTopology.

Prerequisites

Packaging IntelliJ IDEA Code is complete.
Ensure that the time difference between the client and the Storm cluster is less than 5 minutes.
When the host where the Linux OS runs is not a node of the cluster, you are required to set the mapping between the host name and IP address in the hosts file of the node where the Linux OS runs. The host name must be correctly mapped to the IP address.

Procedure

Modify the WordCountTopology.java class and change the application submission mode to the remoteSubmit mode. Replace the user keytab file name, user principal name, and the JAR file address.

Submit the application in remoteSubmit mode.

public static void main(String[] args) 
         throws Exception 
     { 
         TopologyBuilder builder = buildTopology(); 
          
         /* 
          * The task submitting modes are as follows: 
          * 1. Command submitting mode: In this mode, you need to copy an application JAR package to the client machine and run related commands for submitting. 
          * 2. Remote submitting mode: In this mode, you need to package related content into an application JAR package and run the main method in IntelliJ IDEA for submitting. 
          * 3. Local submitting mode: In this mode, you need to run an application locally. This mode is usually used for tests. 
          * The command submitting mode and remote submitting mode are supported in both the security and normal modes. 
          * The local submitting mode is supported only in the normal mode. 
          *  
          * The user can only select one mode for task submitting. The command submitting mode is used by default. If another mode is used, delete the code comments. 
          */ 
          
         submitTopology(builder, SubmitType.REMOTE); 
     }

Change the value of userJarFilePath to the actual topology JAR package address based on the actual situation.

private static void remoteSubmit(TopologyBuilder builder) 
         throws AlreadyAliveException, InvalidTopologyException, NotALeaderException, AuthorizationException, 
         IOException 
     { 
         Config config = createConf(); 
          
         String userJarFilePath = "D:\\example.jar"; 
         System.setProperty(STORM_SUBMIT_JAR_PROPERTY, userJarFilePath); 
          
         //Preparations in security mode 
         if (isSecurityModel()) 
         { 
             securityPrepare(config); 
         } 
         config.setNumWorkers(1); 
         StormSubmitter.submitTopologyWithProgressBar(TOPOLOGY_NAME, config, builder.createTopology()); 
     }