Help Center/ Object Storage Service/ SDK Reference/ Java/ FAQs (SDK for Java)/ Does the SDK Support Uploading, Downloading, or Copying Objects in a Batch? (Java SDK)
Updated on 2024-06-18 GMT+08:00

Does the SDK Support Uploading, Downloading, or Copying Objects in a Batch? (Java SDK)

No.

Currently, the SDK does not provide such APIs. You need to encapsulate the service codes for uploading, downloading, or copying objects in a batch by yourself. The procedure is as follows:

  1. List all objects to be uploaded, downloaded, or copied. For details about how to list objects to be downloaded, see Listing Objects (SDK for Java).
  2. Call the API for uploading, downloading, or copying a single object for the listed objects.

Sample code:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
// Replace the following region with the one in use. EU-Dublin is used here as an example.
String endPoint = "https://obs.eu-west-101.myhuaweicloud.eu";
// Hard-coded or plaintext AK/SK are risky. For security purposes, encrypt your AK/SK and store them in the configuration file or environment variables. In this example, the AK/SK are stored in environment variables for identity authentication. Before running this example, configure environment variables ACCESS_KEY_ID and SECRET_ACCESS_KEY_ID.
// Obtain an AK/SK pair on the management console. For details, see https://support.huaweicloud.com/eu/usermanual-ca/ca_01_0003.html.
String ak = System.getenv("ACCESS_KEY_ID");
String sk = System.getenv("SECRET_ACCESS_KEY_ID");
final String bucketName = "bucketname";
// Define the prefix of objects in a bucket.
final String objectPre = "object/";
// Folder to be uploaded
final String localDirPath = "localDirPath";
final List<File> list = new ArrayList<>();
// Scan all objects in the folder.
static void listFiles(File file){
    File[] fs = file.listFiles();
    assert fs != null;
    if (fs.length < 1){
        // If an empty folder needs to be uploaded, add it to the list.
        list.add(file);
    }else{
        for (File f:fs){
            if (f.isDirectory()){
                listFiles(f);
            }
            if (f.isFile()){
                // Add objects to be uploaded to the list.
                list.add(f);
            }
        }
    }
}
// Traverse the folder to be uploaded and obtain all objects to be uploaded.
File file = new File(localDirPath);
listFiles(file);

// Create an instance of ObsClient.
final ObsClient obsClient = new ObsClient(ak, sk, endPoint);

// Initialize the thread pool.
ExecutorService executorService = Executors.newFixedThreadPool(20);

// Concurrently upload parts.
for (File f:list){
    executorService.execute(() -> {
        if (f.isDirectory()){
            // For empty folders, create empty folder objects in the bucket.
            String remoteObjectKey = objectPre + f.getPath().substring(localDirPath.length() + 1) + "/";
            obsClient.putObject(bucketName, remoteObjectKey, new ByteArrayInputStream(new byte[0]));
        }else{
            String remoteObjectKey = objectPre + f.getPath().substring(localDirPath.length() + 1);
            obsClient.putObject(bucketName, remoteObjectKey, new File(f.getPath()));
        }
    });
}

// Wait until the upload is complete.
executorService.shutdown();
while (!executorService.isTerminated())
{
    try
    {
        executorService.awaitTermination(5, TimeUnit.SECONDS);
    }
    catch (InterruptedException e)
    {
        e.printStackTrace();
    }
}
// Close ObsClient.
try {
    obsClient.close();
} catch (IOException e) {
    e.printStackTrace();
}

You can use multiple threads to concurrently upload, download, and copy data to improve efficiency.