Using OBS Buckets Across Regions

Notes and Constraints

• The CCE Container Storage (Everest) add-on version must be 1.2.42 or later.
• The node to which the storage is mounted must be able to access OBS buckets. Generally, the Internet or Direct Connect is used to access OBS buckets across regions. You can ping the endpoint of OBS on the node where OBS is located to check whether OBS is accessible.
• Only PVs can use OBS buckets across regions, and then are bound to PVCs. The PV reclaim policy must be Retain. StorageClass cannot be used to dynamically create PVCs for using OBS buckets across regions.

Procedure

1. Create the paas-obs-endpoint ConfigMap and configure the region and endpoint of OBS.

   The ConfigMap name is fixed to paas-obs-endpoint, and the namespace is fixed to kube-system.

   Region names and endpoints are in key-value pairs. Replace <region_name> and <endpoint_address> with specific values. Use commas (,) to separate multiple values.

   For details about its value, see Regions and Endpoints.

   Example: {"ap-southeast-1": "https://obs.ap-southeast-1.myhuaweicloud.com:443", "ap-southeast-3": "https://obs.ap-southeast-3.myhuaweicloud.com:443"}

   apiVersion: v1
   kind: ConfigMap
   metadata:
     name: paas-obs-endpoint   # The value must be paas-obs-endpoint.
     namespace: kube-system    # The value must be kube-system.
   data:
     obs-endpoint: |
       {"<region_name>": "<endpoint_address>"}

2. Create a PV.
   Set everest.io/region to the region where OBS is located.

   kind: PersistentVolume
   apiVersion: v1
   metadata:
     name: testing-abc
     annotations:
       pv.kubernetes.io/bound-by-controller: 'yes'
       pv.kubernetes.io/provisioned-by: everest-csi-provisioner
   spec:
     capacity:
       storage: 1Gi
     csi:
       driver: obs.csi.everest.io
       volumeHandle: testing-abc             # OBS bucket name
       fsType: s3fs                          # obsfs indicates a parallel file system, and s3fs indicates an OBS bucket.
       volumeAttributes:
         everest.io/obs-volume-type: STANDARD
         everest.io/region: <region_name>       # Region where the OBS bucket resides. Replace it with a specific value.
         storage.kubernetes.io/csiProvisionerIdentity: everest-csi-provisioner
       nodePublishSecretRef:                 # AK/SK used for mounting an OBS bucket
         name: test-user
         namespace: default
     accessModes:
       - ReadWriteMany
     persistentVolumeReclaimPolicy: Retain   # The value must be Retain.
     storageClassName: csi-obs
     volumeMode: Filesystem

   nodePublishSecretRef is the access key (AK/SK) used for mounting the object storage volume. Use the AK/SK to create a secret, which will be used when creating a PV. For details, see Using a Custom Access Key (AK/SK) to Mount an OBS Volume.

3. Create a PVC.

   apiVersion: v1
   kind: PersistentVolumeClaim
   metadata:
     name: pvc-test-abc
     namespace: default
     annotations:
       everest.io/obs-volume-type: STANDARD                 # Bucket type, which can be STANDARD or WARM when an OBS bucket is used
       csi.storage.k8s.io/fstype: s3fs                      # File type. obsfs indicates a parallel file system, and s3fs indicates an OBS bucket.
       volume.beta.kubernetes.io/storage-provisioner: everest-csi-provisioner
   spec:
     accessModes:
     - ReadWriteMany             # The value must be ReadWriteMany for object storage.
     resources:
       requests:
         storage: 1Gi            # Storage capacity of a PVC. This parameter is valid only for verification (fixed to 1, cannot be empty or 0). The value setting does not take effect for OBS buckets.
     storageClassName: csi-obs   # StorageClass name. For object storage, the value is fixed at csi-obs.
     volumeName: testing-abc     # PV name

4. Create a workload, select the PVC in the data storage option of the container settings, and add the created PVC. If the workload is successfully created, the OBS bucket can be used across regions.

   apiVersion: apps/v1 
   kind: Deployment 
   metadata: 
      name: obs-deployment-example                       # Workload name
     namespace: default 
   spec: 
     replicas: 1 
     selector: 
       matchLabels: 
         app: obs-deployment-example 
     template: 
       metadata: 
         labels: 
           app: obs-deployment-example 
       spec: 
         containers: 
         - image: nginx
           name: container-0 
           volumeMounts: 
           - mountPath: /tmp                       # Mount path
             name: pvc-obs-example 
         restartPolicy: Always
         imagePullSecrets:
           - name: default-secret
         volumes: 
         - name: pvc-obs-example  
           persistentVolumeClaim: 
             claimName: pvc-test-abc               # PVC name