Updated on 2024-02-21 GMT+08:00

Deleting Partitions in Batches

Function

Delete partition data immediately for non-transaction tables if data deletion is enabled. Data in transaction table is retained but is kept invisible. The data will be deleted when it expires.

URI

POST /v1/{project_id}/instances/{instance_id}/catalogs/{catalog_name}/databases/{database_name}/tables/{table_name}/partitions/batch-drop

Table 1 Path Parameters

Parameter

Mandatory

Type

Description

project_id

Yes

String

Project ID. For how to obtain the project ID, see Obtaining a Project ID (lakeformation_04_0026.xml).

instance_id

Yes

String

LakeFormation instance ID. The value is automatically generated when the instance is created, for example, 2180518f-42b8-4947-b20b-adfc53981a25.

catalog_name

Yes

String

Catalog name. The value should contain 1 to 256 characters. Only letters, numbers, and underscores (_) are allowed.

database_name

Yes

String

Database name. The value should contain 1 to 128 characters. Only letters, numbers, hyphens (-), and underscores (_) are allowed.

table_name

Yes

String

Table name. The value should contain 1 to 256 characters. Only letters, numbers, hyphens (-), and underscores (_) are allowed.

Request Parameters

Table 2 Request header parameters

Parameter

Mandatory

Type

Description

X-Auth-Token

Yes

Array of strings

Tenant token.

Table 3 Request body parameters

Parameter

Mandatory

Type

Description

if_exist

No

Boolean

Whether to skip an unallocated or hidden partition. The default value is false.

delete_data

No

Boolean

Delete partition data in non-transaction tables. If if_purge is set to true, space is immediately released. Data in transaction tables is retained but is kept invisible. The data will be deleted when it expires. The default value is false.

partition_values

Yes

Array<Array<String>>

Delete the partition value list in a partition.

Response Parameters

Status code: 200

Table 4 Response body parameters

Parameter

Type

Description

[items]

Array of Partition objects

List of deleted partitions.

Table 5 Partition

Parameter

Type

Description

catalog_name

String

Catalog name.

catalog_id

String

catalogID.

database_name

String

Database name.

database_id

String

Database ID.

table_name

String

Table name.

table_id

String

Table ID.

partition_id

String

Partition ID.

partition_values

Array of strings

Partition values.

create_time

String

Creation time.

last_access_time

String

Last access time.

parameters

Map<String,String>

Parameter table.

storage_descriptor

StorageDescriptor object

Data storage.

Table 6 StorageDescriptor

Parameter

Type

Description

columns

Array of Column objects

All fields except partition columns.

location

String

Path, for example, obs://location/uri/.

compressed

Boolean

Whether to enable data compression. The default value is false.

input_format

String

Input format.

output_format

String

Output format.

number_of_buckets

Integer

Number of buckets. The default value is 0.

bucket_columns

Array of strings

Bucket field.

sort_columns

Array of Order objects

A list that specifies the bucket sorting method.

serde_info

SerDeInfo object

Serialization/Deserialization information.

parameters

Map<String,String>

Descriptor storage parameter. key can contain 255 characters at most. value can contain a maximum of 4,000 characters.

skewed_info

SkewedInfo object

Information about values (skewed values) that often appear in a column.

stored_as_sub_directories

Boolean

Whether to store data in subdirectories.

Table 7 Column

Parameter

Type

Description

column_type

String

Column type, including array, bigint, binary, boolean, char, date, decimal, double, float, int, interval, map, set, smallint, string, struct, timestamp, tinyint, union, and varchar.

column_name

String

Column name. The value can contain 1 to 767 characters. Only letters, digits, and special characters (_-+*(),) are allowed.

comment

String

Description of a column. Enter a maximum of 4,000 characters.

Table 8 Order

Parameter

Type

Description

column

String

Column name. The value can contain 1 to 767 characters. Only letters, digits, and special characters (_-+*(),) are allowed.

sort_order

Integer

Sorting method. The values are ==1 (ascending order) and ==0 (descending order).

Table 9 SerDeInfo

Parameter

Type

Description

name

String

Names.

serialization_library

String

Class that implements serialization/deserialization.

parameters

Map<String,String>

Parameters. The value of key ranges from 1 to 255. The maximum value of value is 4000.

Table 10 SkewedInfo

Parameter

Type

Description

skewed_column_names

Array of strings

List of skewed columns.

skewed_column_value_location_maps

Map<String,String>

Mapping between skew values and addresses.

skewed_column_values

Array<Array<String>>

List of skewed values.

Status code: 400

Table 11 Response body parameters

Parameter

Type

Description

error_code

String

Error code.

error_msg

String

Error message.

solution_msg

String

Solution.

Status code: 404

Table 12 Response body parameters

Parameter

Type

Description

error_code

String

Error code.

error_msg

String

Error message.

solution_msg

String

Solution.

Status code: 500

Table 13 Response body parameters

Parameter

Type

Description

error_code

String

Error code.

error_msg

String

Error message.

solution_msg

String

Solution.

Example Requests

POST https://{endpoint} /v1/{project_id}/instances/{instance_id}/catalogs/{catalog_name}/databases/{database_name}/tables/{table_name}/partitions/batch-drop

{
  "if_exist" : true,
  "delete_data" : true,
  "partition_values" : [ [ "54fd5f39855e43e29630196af18cfb22", "ae64345374854efc819fac880541465e" ], [ "c085eeb64df04982a4c93d3a33fc8cf6", "e9c380472d974182b973f46443e8dc3b" ], [ "ca39a2137c8548d0a49842802f2dedb8", "c1d9679369494c52933c31ecd5483550" ] ]
}

Example Responses

Status code: 200

List of deleted partitions.

[ {
  "catalog_name" : "catcd6a92bfc2c74a509dc8501b8124505c",
  "database_name" : "db4cfc99ac3cea421abc157c7bd49f6f16",
  "table_name" : "tblbb79f3733c0747adbb94981b412fe48f",
  "partition_values" : [ "48e1f86dc4394a218b832d94411805fe", "5a2ab07a663c49a98cc606e6ffa0b178" ],
  "create_time" : "2023-05-30T09:06:41.372+00:00",
  "last_access_time" : "2023-05-30T09:06:41.000+00:00",
  "parameters" : {
    "key1" : "value1",
    "transient_lastDdlTime" : "120",
    "classification" : "other"
  },
  "storage_descriptor" : {
    "columns" : [ {
      "column_type" : "string",
      "column_name" : "column_prefix0",
      "comment" : "83e99b4daff04e75b28263bf44495a3c"
    }, {
      "column_type" : "string",
      "column_name" : "column_prefix1",
      "comment" : "d947669d90884014b60264b2770849ae"
    }, {
      "column_type" : "string",
      "column_name" : "column_prefix2",
      "comment" : "21edd8e92f2448deb036f8e4682ec694"
    }, {
      "column_type" : "string",
      "column_name" : "column_prefix3",
      "comment" : "fc25f38e8b694aebb34a2f84ef4a62f7"
    }, {
      "column_type" : "string",
      "column_name" : "column_prefix4",
      "comment" : "3443079e616d447f8f35da2014754701"
    }, {
      "column_type" : "string",
      "column_name" : "column_prefix5",
      "comment" : "5f7dda02ac8e4813b3eb4858abb7c520"
    }, {
      "column_type" : "string",
      "column_name" : "column_prefix6",
      "comment" : "2a6ae2e5bc3f4add99133106f2cd526c"
    }, {
      "column_type" : "string",
      "column_name" : "column_prefix7",
      "comment" : "84d50424269a41fbb5528b350e9d0b13"
    }, {
      "column_type" : "string",
      "column_name" : "column_prefix8",
      "comment" : "e3f1d2fa80454554a22e8564ec9c4832"
    }, {
      "column_type" : "string",
      "column_name" : "column_prefix9",
      "comment" : "f11315794e6d40daa2cb8347c61e5a46"
    } ],
    "location" : "obs://location/test/database/b593fc09871e4c9886ed931a3e51f9e5",
    "compressed" : false,
    "input_format" : "8c470072d43344a3a3bfabd2c7f9d6ec",
    "output_format" : "74ee5be43fb548e7b988390df05134fa",
    "number_of_buckets" : 0,
    "bucket_columns" : [ ],
    "sort_columns" : [ ],
    "serde_info" : {
      "name" : "9b3ea84748854c2d86a71ae37e246293",
      "serialization_library" : "e9632ffbbfbe425d8eac91e765e046e1",
      "parameters" : {
        "e5ae426044054a5f8e362b9b5729dbab" : "33d8296203f94760b1e4b7db55edce0d"
      }
    },
    "parameters" : {
      "7e27b28f439e40bea098dd65bc857334" : "ab893cac08bb4374beb8ff00a02e9a73",
      "c698760d6de74d0c92eee65bea283215" : "8ba9a058a5cc4ebaac6f43c9771e167a"
    },
    "skewed_info" : {
      "skewed_column_names" : [ ],
      "skewed_column_value_location_maps" : { },
      "skewed_column_values" : [ ]
    },
    "stored_as_sub_directories" : false
  }
}, {
  "catalog_name" : "catcd6a92bfc2c74a509dc8501b8124505c",
  "database_name" : "db4cfc99ac3cea421abc157c7bd49f6f16",
  "table_name" : "tblbb79f3733c0747adbb94981b412fe48f",
  "partition_values" : [ "e4af64d55308403a9abf1999afbbe2be", "079f4cd2ee5b4de6922ef8f0c92c82ae" ],
  "create_time" : "2023-05-30T09:06:41.372+00:00",
  "last_access_time" : "2023-05-30T09:06:41.000+00:00",
  "parameters" : {
    "key1" : "value1",
    "transient_lastDdlTime" : "120",
    "classification" : "other"
  },
  "storage_descriptor" : {
    "columns" : [ {
      "column_type" : "string",
      "column_name" : "column_prefix0",
      "comment" : "b40e5a6b6b3c4a538c4b7daa8bc49300"
    }, {
      "column_type" : "string",
      "column_name" : "column_prefix1",
      "comment" : "d7de59704feb4218ad6c3899f16f72e3"
    }, {
      "column_type" : "string",
      "column_name" : "column_prefix2",
      "comment" : "215d3961dd0c4858b884e50157da1d4b"
    }, {
      "column_type" : "string",
      "column_name" : "column_prefix3",
      "comment" : "b8e4888680ca449492be2344a81aba1d"
    }, {
      "column_type" : "string",
      "column_name" : "column_prefix4",
      "comment" : "1ac15b288c354834830062e48b04e3c8"
    }, {
      "column_type" : "string",
      "column_name" : "column_prefix5",
      "comment" : "5e65189b3f28434c8b96a56fb19b3005"
    }, {
      "column_type" : "string",
      "column_name" : "column_prefix6",
      "comment" : "7944cf9158514a96905712d686bc7c18"
    }, {
      "column_type" : "string",
      "column_name" : "column_prefix7",
      "comment" : "c6b9396476e541e9ab34a8eb5fda9953"
    }, {
      "column_type" : "string",
      "column_name" : "column_prefix8",
      "comment" : "978fe9b8a1d1437391e83553ae361bfc"
    }, {
      "column_type" : "string",
      "column_name" : "column_prefix9",
      "comment" : "4f7ed315c7c44c5683500104464f4776"
    } ],
    "location" : "obs://location/test/database/3971910eb87a41ee9d7096d538232c33",
    "compressed" : false,
    "input_format" : "620befc9b9a447fb95ecfb0d33fb636c",
    "output_format" : "fc943b36eaec4d1da525734e56374d32",
    "number_of_buckets" : 0,
    "bucket_columns" : [ ],
    "sort_columns" : [ ],
    "serde_info" : {
      "name" : "9286f98baef44841883166345635faed",
      "serialization_library" : "ac9cdbe5004d468798bc57b4ed10f602",
      "parameters" : {
        "cef1b8d74bdd43459793b00b4deff6ae" : "e66acd0f816047ae9e00d01a4045c340"
      }
    },
    "parameters" : {
      "0546e622cf2042598ed4f31e0b3b3276" : "be46b5accfdc41ba93f50fbe16b782c2",
      "a5b63ea213ab48be8c3c361fe93f75d2" : "c0f1215c944040cfad2e69fbe7cb4b0b"
    },
    "skewed_info" : {
      "skewed_column_names" : [ ],
      "skewed_column_value_location_maps" : { },
      "skewed_column_values" : [ ]
    },
    "stored_as_sub_directories" : false
  }
}, {
  "catalog_name" : "catcd6a92bfc2c74a509dc8501b8124505c",
  "database_name" : "db4cfc99ac3cea421abc157c7bd49f6f16",
  "table_name" : "tblbb79f3733c0747adbb94981b412fe48f",
  "partition_values" : [ "e5ade865888a4bb0a22b71505dd0989d", "788dfdb29db342f488a6155a55875f5b" ],
  "create_time" : "2023-05-30T09:06:41.372+00:00",
  "last_access_time" : "2023-05-30T09:06:41.000+00:00",
  "parameters" : {
    "key1" : "value1",
    "transient_lastDdlTime" : "120",
    "classification" : "other"
  },
  "storage_descriptor" : {
    "columns" : [ {
      "column_type" : "string",
      "column_name" : "column_prefix0",
      "comment" : "a4398b0a8f404ad1a2d8d350efdcd812"
    }, {
      "column_type" : "string",
      "column_name" : "column_prefix1",
      "comment" : "c2d17faf55de41ddb9f098a684dd8473"
    }, {
      "column_type" : "string",
      "column_name" : "column_prefix2",
      "comment" : "5f2de67835ff49aba7b661712ef059a6"
    }, {
      "column_type" : "string",
      "column_name" : "column_prefix3",
      "comment" : "26ce9cc09f3545df80181cbed87d4da2"
    }, {
      "column_type" : "string",
      "column_name" : "column_prefix4",
      "comment" : "f5b65b8db66349b3a1f8eace0ea33334"
    }, {
      "column_type" : "string",
      "column_name" : "column_prefix5",
      "comment" : "3f9b100dbc2f45d58d6fbcfa63e0d806"
    }, {
      "column_type" : "string",
      "column_name" : "column_prefix6",
      "comment" : "22c5f279e9d04cada8bdee6ddbd38af7"
    }, {
      "column_type" : "string",
      "column_name" : "column_prefix7",
      "comment" : "7ac1468020094057bf2e1e41f3e035b8"
    }, {
      "column_type" : "string",
      "column_name" : "column_prefix8",
      "comment" : "8c8178e2fe694ace80e78d5c985f7165"
    }, {
      "column_type" : "string",
      "column_name" : "column_prefix9",
      "comment" : "3f8d23d533fe44b295067de73a8f1b68"
    } ],
    "location" : "obs://location/test/database/8c351a1aca6b426a936be4453da30c35",
    "compressed" : false,
    "input_format" : "9fb9f9d02b3044febe447a1ad4ca5297",
    "output_format" : "ba8acae75da24d858ebe99f4ef2fce84",
    "number_of_buckets" : 0,
    "bucket_columns" : [ ],
    "sort_columns" : [ ],
    "serde_info" : {
      "name" : "6e4b2479321842d191f1b540b348a4f1",
      "serialization_library" : "22209422df114f589b4ec28d19d86e52",
      "parameters" : {
        "a5c4f39d83d047748dcd1218764c7a1c" : "9e2256acd089474ba469d329d35ace13"
      }
    },
    "parameters" : {
      "997ebfa4c6e343789a92d20f7961c865" : "2a1899624e084fb4b26595469cd23f32",
      "86887611dd7345c88f3de16538cb3587" : "ac4aab973d50429d9803cb418c9d44e1"
    },
    "skewed_info" : {
      "skewed_column_names" : [ ],
      "skewed_column_value_location_maps" : { },
      "skewed_column_values" : [ ]
    },
    "stored_as_sub_directories" : false
  }
} ]

Status code: 400

Bad Request

{
  "error_code" : "common.01000001",
  "error_msg" : "failed to read http request, please check your input, code: 400, reason: Type mismatch., cause: TypeMismatchException"
}

Status code: 401

Unauthorized

{
  "error_code": 'APIG.1002',
  "error_msg": 'Incorrect token or token resolution failed'
}

Status code: 403

Forbidden

{
  "error" : {
    "code" : "403",
    "message" : "X-Auth-Token is invalid in the request",
    "error_code" : null,
    "error_msg" : null,
    "title" : "Forbidden"
  },
  "error_code" : "403",
  "error_msg" : "X-Auth-Token is invalid in the request",
  "title" : "Forbidden"
}

Status code: 404

Not Found

{
  "error_code" : "common.01000001",
  "error_msg" : "response status exception, code: 404"
}

Status code: 408

Request Timeout

{
  "error_code" : "common.00000408",
  "error_msg" : "timeout exception occurred"
}

Status code: 500

Internal Server Error

{
  "error_code" : "common.00000500",
  "error_msg" : "internal error"
}

Status Codes

Status Code

Description

200

List of deleted partitions.

201

Created

400

Bad Request

401

Unauthorized

403

Forbidden

404

Not Found

408

Request Timeout

500

Internal Server Error

Error Codes

See Error Codes.