SET/RESET

Updated on 2022-09-22 GMT+08:00

View PDF

Function

This command is used to dynamically add, update, display, or reset the CarbonData properties without restarting the driver.

Syntax

Add or Update parameter value:
SET parameter_name=parameter_value

This command is used to add or update the value of parameter_name.
Display property value:
SET parameter_name

This command is used to display the value of parameter_name.
Display session parameter:
SET

This command is used to display all supported session parameters.
Display session parameters along with usage details:
SET -v

This command is used to display all supported session parameters and their usage details.
Reset parameter value:
RESET

This command is used to clear all session parameters.

Parameter Description

**Table 1** SET parameters
Parameter	Description
parameter_name	Name of the parameter whose value needs to be dynamically added, updated, or displayed
parameter_value	New value of parameter_name to be set

Precautions

The following table lists the properties which you can set or clear using the SET or RESET command.

**Table 2** Properties
Property	Description
carbon.options.bad.records.logger.enable	Whether to enable bad record logger.
carbon.options.bad.records.action	Operations on bad records, for example, force, redirect, fail, or ignore. For more information, see •Bad record handling.
carbon.options.is.empty.data.bad.record	Whether the empty data is considered as a bad record. For more information, see Bad record handling.
carbon.options.sort.scope	Scope of the sort during data loading.
carbon.options.bad.record.path	HDFS path where bad records are stored.
carbon.custom.block.distribution	Whether to enable Spark or CarbonData block distribution.
enable.unsafe.sort	Whether to use unsafe sort during data loading. Unsafe sort reduces the garbage collection during data loading, thereby achieving better performance.
carbon.si.lookup.partialstring	If this is set to TRUE, the secondary index uses the starts-with, ends-with, contains, and LIKE partition condition strings. If this is set to FALSE, the secondary index uses only the starts-with partition condition string.
carbon.input.segments	Segment ID to be queried. This property allows you to query a specified segment of a specified table. CarbonScan reads data only from the specified segment ID. Syntax: carbon.input.segments. <database_name>. <table_name> = < list of segment ids > If you want to query a specified segment in multi-thread mode, you can use CarbonSession.threadSet instead of the SET statement. Syntax: CarbonSession.threadSet ("carbon.input.segments. <database_name>. <table_name>","< list of segment ids >"); NOTE: You are advised not to set this property in the carbon.properties file because all sessions contain the segment list unless session-level or thread-level overwriting occurs.

Examples

Add or Update:
SET enable.unsafe.sort=true
Display property value:
SET enable.unsafe.sort
Show the segment ID list, segment status, and other required details, and specify the segment list to be read:
SHOW SEGMENTS FOR TABLE carbontable1;

SET carbon.input.segments.db.carbontable1 = 1, 3, 9;
Query a specified segment in multi-thread mode:
CarbonSession.threadSet ("carbon.input.segments.default.carbon_table_MulTI_THread", "1,3");

Use CarbonSession.threadSet to query segments in a multi-thread environment (Scala code is used as an example):

def main(args: Array[String]) {
 Future {              CarbonSession.threadSet("carbon.input.segments.default.carbon_table_MulTI_THread", "1")
      spark.sql("select count(empno) from carbon_table_MulTI_THread").show()
    }
}

Reset:
RESET

System Response

Success will be recorded in the driver log.
Failure will be displayed on the UI.

Parent topic: DML

Previous topic: CLEAN FILES

Next topic: Concurrent CarbonData Table Operations

Feedback

Was this page helpful?

Helpful Not helpful

Provide feedback

Thank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.

The system is busy. Please try again later.

Which of the following issues have you encountered?

Content is inconsistent with the product UI

Unclear descriptions

Lack of examples or code

Incorrect steps

Can't find what I need

Lack of best practices

Feedback (optional)

0/500

Select at least one type of issue, and enter your comments or suggestions.

Enter a maximum of 500 characters.

Submit Cancel

For any further questions, feel free to contact us through the chatbot.

Chatbot