Help Center/ DataArts Lake Formation/ API Reference/ API/ LakeCat/ Zone-based Management/ Deleting Partitions in Batches

Updated on 2024-02-21 GMT+08:00

View PDF

Deleting Partitions in Batches

Function

Delete partition data immediately for non-transaction tables if data deletion is enabled. Data in transaction table is retained but is kept invisible. The data will be deleted when it expires.

URI

POST /v1/{project_id}/instances/{instance_id}/catalogs/{catalog_name}/databases/{database_name}/tables/{table_name}/partitions/batch-drop

**Table 1** Path Parameters
Parameter	Mandatory	Type	Description
project_id	Yes	String	Project ID. For how to obtain the project ID, see Obtaining a Project ID (lakeformation_04_0026.xml).
instance_id	Yes	String	LakeFormation instance ID. The value is automatically generated when the instance is created, for example, 2180518f-42b8-4947-b20b-adfc53981a25.
catalog_name	Yes	String	Catalog name. The value should contain 1 to 256 characters. Only letters, numbers, and underscores (_) are allowed.
database_name	Yes	String	Database name. The value should contain 1 to 128 characters. Only letters, numbers, hyphens (-), and underscores (_) are allowed.
table_name	Yes	String	Table name. The value should contain 1 to 256 characters. Only letters, numbers, hyphens (-), and underscores (_) are allowed.

Request Parameters

**Table 2** Request header parameters
Parameter	Mandatory	Type	Description
X-Auth-Token	Yes	Array of strings	Tenant token.

**Table 3** Request body parameters
Parameter	Mandatory	Type	Description
if_exist	No	Boolean	Whether to skip an unallocated or hidden partition. The default value is false.
delete_data	No	Boolean	Delete partition data in non-transaction tables. If if_purge is set to true, space is immediately released. Data in transaction tables is retained but is kept invisible. The data will be deleted when it expires. The default value is false.
partition_values	Yes	Array<Array<String>>	Delete the partition value list in a partition.

Response Parameters

Status code: 200

**Table 4** Response body parameters
Parameter	Type	Description
[items]	Array of Partition objects	List of deleted partitions.

**Table 5** Partition
Parameter	Type	Description
catalog_name	String	Catalog name.
catalog_id	String	catalogID.
database_name	String	Database name.
database_id	String	Database ID.
table_name	String	Table name.
table_id	String	Table ID.
partition_id	String	Partition ID.
partition_values	Array of strings	Partition values.
create_time	String	Creation time.
last_access_time	String	Last access time.
parameters	Map<String,String>	Parameter table.
storage_descriptor	StorageDescriptor object	Data storage.

**Table 6** StorageDescriptor
Parameter	Type	Description
columns	Array of Column objects	All fields except partition columns.
location	String	Path, for example, obs://location/uri/.
compressed	Boolean	Whether to enable data compression. The default value is false.
input_format	String	Input format.
output_format	String	Output format.
number_of_buckets	Integer	Number of buckets. The default value is 0.
bucket_columns	Array of strings	Bucket field.
sort_columns	Array of Order objects	A list that specifies the bucket sorting method.
serde_info	SerDeInfo object	Serialization/Deserialization information.
parameters	Map<String,String>	Descriptor storage parameter. key can contain 255 characters at most. value can contain a maximum of 4,000 characters.
skewed_info	SkewedInfo object	Information about values (skewed values) that often appear in a column.
stored_as_sub_directories	Boolean	Whether to store data in subdirectories.

**Table 7** Column
Parameter	Type	Description
column_type	String	Column type, including array, bigint, binary, boolean, char, date, decimal, double, float, int, interval, map, set, smallint, string, struct, timestamp, tinyint, union, and varchar.
column_name	String	Column name. The value can contain 1 to 767 characters. Only letters, digits, and special characters (_-+*(),) are allowed.
comment	String	Description of a column. Enter a maximum of 4,000 characters.

**Table 8** Order
Parameter	Type	Description
column	String	Column name. The value can contain 1 to 767 characters. Only letters, digits, and special characters (_-+*(),) are allowed.
sort_order	Integer	Sorting method. The values are ==1 (ascending order) and ==0 (descending order).

**Table 9** SerDeInfo
Parameter	Type	Description
name	String	Names.
serialization_library	String	Class that implements serialization/deserialization.
parameters	Map<String,String>	Parameters. The value of key ranges from 1 to 255. The maximum value of value is 4000.

**Table 10** SkewedInfo
Parameter	Type	Description
skewed_column_names	Array of strings	List of skewed columns.
skewed_column_value_location_maps	Map<String,String>	Mapping between skew values and addresses.
skewed_column_values	Array<Array<String>>	List of skewed values.

Status code: 400

**Table 11** Response body parameters
Parameter	Type	Description
error_code	String	Error code.
error_msg	String	Error message.
solution_msg	String	Solution.

Status code: 404

**Table 12** Response body parameters
Parameter	Type	Description
error_code	String	Error code.
error_msg	String	Error message.
solution_msg	String	Solution.

Status code: 500

**Table 13** Response body parameters
Parameter	Type	Description
error_code	String	Error code.
error_msg	String	Error message.
solution_msg	String	Solution.

Example Requests

POST https://{endpoint} /v1/{project_id}/instances/{instance_id}/catalogs/{catalog_name}/databases/{database_name}/tables/{table_name}/partitions/batch-drop

{
  "if_exist" : true,
  "delete_data" : true,
  "partition_values" : [ [ "54fd5f39855e43e29630196af18cfb22", "ae64345374854efc819fac880541465e" ], [ "c085eeb64df04982a4c93d3a33fc8cf6", "e9c380472d974182b973f46443e8dc3b" ], [ "ca39a2137c8548d0a49842802f2dedb8", "c1d9679369494c52933c31ecd5483550" ] ]
}

Example Responses

Status code: 200

List of deleted partitions.

[ {
  "catalog_name" : "catcd6a92bfc2c74a509dc8501b8124505c",
  "database_name" : "db4cfc99ac3cea421abc157c7bd49f6f16",
  "table_name" : "tblbb79f3733c0747adbb94981b412fe48f",
  "partition_values" : [ "48e1f86dc4394a218b832d94411805fe", "5a2ab07a663c49a98cc606e6ffa0b178" ],
  "create_time" : "2023-05-30T09:06:41.372+00:00",
  "last_access_time" : "2023-05-30T09:06:41.000+00:00",
  "parameters" : {
    "key1" : "value1",
    "transient_lastDdlTime" : "120",
    "classification" : "other"
  },
  "storage_descriptor" : {
    "columns" : [ {
      "column_type" : "string",
      "column_name" : "column_prefix0",
      "comment" : "83e99b4daff04e75b28263bf44495a3c"
    }, {
      "column_type" : "string",
      "column_name" : "column_prefix1",
      "comment" : "d947669d90884014b60264b2770849ae"
    }, {
      "column_type" : "string",
      "column_name" : "column_prefix2",
      "comment" : "21edd8e92f2448deb036f8e4682ec694"
    }, {
      "column_type" : "string",
      "column_name" : "column_prefix3",
      "comment" : "fc25f38e8b694aebb34a2f84ef4a62f7"
    }, {
      "column_type" : "string",
      "column_name" : "column_prefix4",
      "comment" : "3443079e616d447f8f35da2014754701"
    }, {
      "column_type" : "string",
      "column_name" : "column_prefix5",
      "comment" : "5f7dda02ac8e4813b3eb4858abb7c520"
    }, {
      "column_type" : "string",
      "column_name" : "column_prefix6",
      "comment" : "2a6ae2e5bc3f4add99133106f2cd526c"
    }, {
      "column_type" : "string",
      "column_name" : "column_prefix7",
      "comment" : "84d50424269a41fbb5528b350e9d0b13"
    }, {
      "column_type" : "string",
      "column_name" : "column_prefix8",
      "comment" : "e3f1d2fa80454554a22e8564ec9c4832"
    }, {
      "column_type" : "string",
      "column_name" : "column_prefix9",
      "comment" : "f11315794e6d40daa2cb8347c61e5a46"
    } ],
    "location" : "obs://location/test/database/b593fc09871e4c9886ed931a3e51f9e5",
    "compressed" : false,
    "input_format" : "8c470072d43344a3a3bfabd2c7f9d6ec",
    "output_format" : "74ee5be43fb548e7b988390df05134fa",
    "number_of_buckets" : 0,
    "bucket_columns" : [ ],
    "sort_columns" : [ ],
    "serde_info" : {
      "name" : "9b3ea84748854c2d86a71ae37e246293",
      "serialization_library" : "e9632ffbbfbe425d8eac91e765e046e1",
      "parameters" : {
        "e5ae426044054a5f8e362b9b5729dbab" : "33d8296203f94760b1e4b7db55edce0d"
      }
    },
    "parameters" : {
      "7e27b28f439e40bea098dd65bc857334" : "ab893cac08bb4374beb8ff00a02e9a73",
      "c698760d6de74d0c92eee65bea283215" : "8ba9a058a5cc4ebaac6f43c9771e167a"
    },
    "skewed_info" : {
      "skewed_column_names" : [ ],
      "skewed_column_value_location_maps" : { },
      "skewed_column_values" : [ ]
    },
    "stored_as_sub_directories" : false
  }
}, {
  "catalog_name" : "catcd6a92bfc2c74a509dc8501b8124505c",
  "database_name" : "db4cfc99ac3cea421abc157c7bd49f6f16",
  "table_name" : "tblbb79f3733c0747adbb94981b412fe48f",
  "partition_values" : [ "e4af64d55308403a9abf1999afbbe2be", "079f4cd2ee5b4de6922ef8f0c92c82ae" ],
  "create_time" : "2023-05-30T09:06:41.372+00:00",
  "last_access_time" : "2023-05-30T09:06:41.000+00:00",
  "parameters" : {
    "key1" : "value1",
    "transient_lastDdlTime" : "120",
    "classification" : "other"
  },
  "storage_descriptor" : {
    "columns" : [ {
      "column_type" : "string",
      "column_name" : "column_prefix0",
      "comment" : "b40e5a6b6b3c4a538c4b7daa8bc49300"
    }, {
      "column_type" : "string",
      "column_name" : "column_prefix1",
      "comment" : "d7de59704feb4218ad6c3899f16f72e3"
    }, {
      "column_type" : "string",
      "column_name" : "column_prefix2",
      "comment" : "215d3961dd0c4858b884e50157da1d4b"
    }, {
      "column_type" : "string",
      "column_name" : "column_prefix3",
      "comment" : "b8e4888680ca449492be2344a81aba1d"
    }, {
      "column_type" : "string",
      "column_name" : "column_prefix4",
      "comment" : "1ac15b288c354834830062e48b04e3c8"
    }, {
      "column_type" : "string",
      "column_name" : "column_prefix5",
      "comment" : "5e65189b3f28434c8b96a56fb19b3005"
    }, {
      "column_type" : "string",
      "column_name" : "column_prefix6",
      "comment" : "7944cf9158514a96905712d686bc7c18"
    }, {
      "column_type" : "string",
      "column_name" : "column_prefix7",
      "comment" : "c6b9396476e541e9ab34a8eb5fda9953"
    }, {
      "column_type" : "string",
      "column_name" : "column_prefix8",
      "comment" : "978fe9b8a1d1437391e83553ae361bfc"
    }, {
      "column_type" : "string",
      "column_name" : "column_prefix9",
      "comment" : "4f7ed315c7c44c5683500104464f4776"
    } ],
    "location" : "obs://location/test/database/3971910eb87a41ee9d7096d538232c33",
    "compressed" : false,
    "input_format" : "620befc9b9a447fb95ecfb0d33fb636c",
    "output_format" : "fc943b36eaec4d1da525734e56374d32",
    "number_of_buckets" : 0,
    "bucket_columns" : [ ],
    "sort_columns" : [ ],
    "serde_info" : {
      "name" : "9286f98baef44841883166345635faed",
      "serialization_library" : "ac9cdbe5004d468798bc57b4ed10f602",
      "parameters" : {
        "cef1b8d74bdd43459793b00b4deff6ae" : "e66acd0f816047ae9e00d01a4045c340"
      }
    },
    "parameters" : {
      "0546e622cf2042598ed4f31e0b3b3276" : "be46b5accfdc41ba93f50fbe16b782c2",
      "a5b63ea213ab48be8c3c361fe93f75d2" : "c0f1215c944040cfad2e69fbe7cb4b0b"
    },
    "skewed_info" : {
      "skewed_column_names" : [ ],
      "skewed_column_value_location_maps" : { },
      "skewed_column_values" : [ ]
    },
    "stored_as_sub_directories" : false
  }
}, {
  "catalog_name" : "catcd6a92bfc2c74a509dc8501b8124505c",
  "database_name" : "db4cfc99ac3cea421abc157c7bd49f6f16",
  "table_name" : "tblbb79f3733c0747adbb94981b412fe48f",
  "partition_values" : [ "e5ade865888a4bb0a22b71505dd0989d", "788dfdb29db342f488a6155a55875f5b" ],
  "create_time" : "2023-05-30T09:06:41.372+00:00",
  "last_access_time" : "2023-05-30T09:06:41.000+00:00",
  "parameters" : {
    "key1" : "value1",
    "transient_lastDdlTime" : "120",
    "classification" : "other"
  },
  "storage_descriptor" : {
    "columns" : [ {
      "column_type" : "string",
      "column_name" : "column_prefix0",
      "comment" : "a4398b0a8f404ad1a2d8d350efdcd812"
    }, {
      "column_type" : "string",
      "column_name" : "column_prefix1",
      "comment" : "c2d17faf55de41ddb9f098a684dd8473"
    }, {
      "column_type" : "string",
      "column_name" : "column_prefix2",
      "comment" : "5f2de67835ff49aba7b661712ef059a6"
    }, {
      "column_type" : "string",
      "column_name" : "column_prefix3",
      "comment" : "26ce9cc09f3545df80181cbed87d4da2"
    }, {
      "column_type" : "string",
      "column_name" : "column_prefix4",
      "comment" : "f5b65b8db66349b3a1f8eace0ea33334"
    }, {
      "column_type" : "string",
      "column_name" : "column_prefix5",
      "comment" : "3f9b100dbc2f45d58d6fbcfa63e0d806"
    }, {
      "column_type" : "string",
      "column_name" : "column_prefix6",
      "comment" : "22c5f279e9d04cada8bdee6ddbd38af7"
    }, {
      "column_type" : "string",
      "column_name" : "column_prefix7",
      "comment" : "7ac1468020094057bf2e1e41f3e035b8"
    }, {
      "column_type" : "string",
      "column_name" : "column_prefix8",
      "comment" : "8c8178e2fe694ace80e78d5c985f7165"
    }, {
      "column_type" : "string",
      "column_name" : "column_prefix9",
      "comment" : "3f8d23d533fe44b295067de73a8f1b68"
    } ],
    "location" : "obs://location/test/database/8c351a1aca6b426a936be4453da30c35",
    "compressed" : false,
    "input_format" : "9fb9f9d02b3044febe447a1ad4ca5297",
    "output_format" : "ba8acae75da24d858ebe99f4ef2fce84",
    "number_of_buckets" : 0,
    "bucket_columns" : [ ],
    "sort_columns" : [ ],
    "serde_info" : {
      "name" : "6e4b2479321842d191f1b540b348a4f1",
      "serialization_library" : "22209422df114f589b4ec28d19d86e52",
      "parameters" : {
        "a5c4f39d83d047748dcd1218764c7a1c" : "9e2256acd089474ba469d329d35ace13"
      }
    },
    "parameters" : {
      "997ebfa4c6e343789a92d20f7961c865" : "2a1899624e084fb4b26595469cd23f32",
      "86887611dd7345c88f3de16538cb3587" : "ac4aab973d50429d9803cb418c9d44e1"
    },
    "skewed_info" : {
      "skewed_column_names" : [ ],
      "skewed_column_value_location_maps" : { },
      "skewed_column_values" : [ ]
    },
    "stored_as_sub_directories" : false
  }
} ]

Status code: 400

Bad Request

{
  "error_code" : "common.01000001",
  "error_msg" : "failed to read http request, please check your input, code: 400, reason: Type mismatch., cause: TypeMismatchException"
}

Status code: 401

Unauthorized

{
  "error_code": 'APIG.1002',
  "error_msg": 'Incorrect token or token resolution failed'
}

Status code: 403

Forbidden

{
  "error" : {
    "code" : "403",
    "message" : "X-Auth-Token is invalid in the request",
    "error_code" : null,
    "error_msg" : null,
    "title" : "Forbidden"
  },
  "error_code" : "403",
  "error_msg" : "X-Auth-Token is invalid in the request",
  "title" : "Forbidden"
}

Status code: 404

Not Found

{
  "error_code" : "common.01000001",
  "error_msg" : "response status exception, code: 404"
}

Status code: 408

Request Timeout

{
  "error_code" : "common.00000408",
  "error_msg" : "timeout exception occurred"
}

Status code: 500

Internal Server Error

{
  "error_code" : "common.00000500",
  "error_msg" : "internal error"
}

Status Codes

Status Code	Description
200	List of deleted partitions.
201	Created
400	Bad Request
401	Unauthorized
403	Forbidden
404	Not Found
408	Request Timeout
500	Internal Server Error

Error Codes

See Error Codes.

Parent topic: Zone-based Management

Previous topic: Adding Partitions in Batches

Next topic: Obtaining Partitions in Batches

Feedback

Was this page helpful?

Helpful Not helpful

Provide feedback

Thank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.

The system is busy. Please try again later.

Which of the following issues have you encountered?

Content is inconsistent with the product UI

Unclear descriptions

Lack of examples or code

Incorrect steps

Can't find what I need

Lack of best practices

Feedback (optional)

0/500

Select at least one type of issue, and enter your comments or suggestions.

Enter a maximum of 500 characters.

Submit Cancel

For any further questions, feel free to contact us through the chatbot.

Chatbot