Creating a Dump Task
Initialize a DIS SDK client instance named dic. For details, see Initializing a DIS SDK Client Instance.
When using the DIS SDK to create a dump task, you need to specify the stream name, dump task name, dump interval, and dump destination.
Creating an OBS Dump Task
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 | CreateTransferTaskRequest request = new CreateTransferTaskRequest(); //Configure the stream name. You can create streams can be created on the DIS console. request.setStreamName(streamName); //Configure the dump task name. OBSDestinationDescriptorRequest descriptor = new OBSDestinationDescriptorRequest(); descriptor.setTransferTaskName(taskName); //Configure the OBS bucket name and folder name. You can create OBS buckets and files on the OBS console or client. descriptor.setObsBucketPath("obs-dis"); descriptor.setFilePrefix("transfertask"); //Configure the dump interval that is expressed in units of seconds. descriptor.setDeliverTimeInterval(900); //(Optional) Create an IAM agency named dis_admin_agency on the DIS management page and use it to access specific cloud services. For the first time to create an IAM agency, you have to authorize it. descriptor.setAgencyName("dis_admin_agency"); //(Optional) Configure the dump file format. By default, the value is Text. Other available options are Parquet and CarbonData. descriptor.setDestinationFileType(DestinationFileTypeEnum.TEXT.getType()); //Configure the initial offset when data is pulled from the DIS stream. The value can be LATEST or TRIM_HORIZON. LATEST is the default value, indicating that data is read from the latest uploaded records in the stream. TRIM_HORIZON indicates that data is read from the earliest unexpired records in the stream. descriptor.setConsumerStrategy(PartitionCursorTypeEnum.LATEST.name()); request.setObsDestinationDescriptor(descriptor); |
After configuring CreateTransferTaskRequest, you can call the createTransferTask method to create the dump task.
1 | dic.createTransferTask(request); |
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot
