# 打包到package目录
Helm模板或者operator包有两个修改点：
- 标记服务的来源，即是来源于开源、华为或者生态伙伴；
- 统一镜像地址，OSC格式包引用lifecycle.yaml及csd.yaml中配置，Helm模板引用values.yaml中的配置，operator引用\*.clusterserviceversion.yaml中的配置。
![](https://support.huaweicloud.com/accessg-osc/public_sys-resources/notice_3.0-zh-cn.png)
提供商上传服务包后，镜像会保存到OSC的仓库中，提供商在上传前无法获取镜像的实际保存地址，OSC服务会将value.yaml或者\*.clusterserviceversion.yaml中默认的镜像地址替换成实际的镜像地址，其它地方的配置不予替换，因此如果其它地方没有引用lifecycle.yaml、csd.yaml、values.yaml或者\*.clusterserviceversion.yaml中的配置，在部署服务实例时，拉取镜像会失败。
#### 标识服务来源
OSC提供来自开源、华为自研以及生态伙伴的服务，生态伙伴需要在服务包中固定字段进行标记说明服务来自生态伙伴。
- operator服务
  以etcd operator为例，etcdoperator.v0.9.4.clusterserviceversion.yaml的内容为：
  ```
  apiVersion: operators.coreos.com/v1alpha1
  kind: ClusterServiceVersion
  metadata:
    annotations:
      capabilities: Full Lifecycle
      categories: Database
      containerImage: quay.io/coreos/etcd-operator@sha256:66a37fd61a06a43969854ee6d3e21087a98b93838e284a6086b13917f96b0d9b
      createdAt: 2019-02-28 01:03:00
      description: Create and maintain highly-available etcd clusters on Kubernetes
      repository: https://github.com/coreos/etcd-operator
      tectonic-visibility: ocs
    name: etcdoperator.v0.9.4
    namespace: placeholder
  ...
  ```
  
1. 编辑文件。 
   在metadata/annotations添加source字段，取值为ISV。
   ```
   apiVersion: operators.coreos.com/v1alpha1
   kind: ClusterServiceVersion
   metadata:
     annotations:
       source: ISV
       capabilities: Full Lifecycle
       categories: Database
       containerImage: quay.io/coreos/etcd-operator@sha256:66a37fd61a06a43969854ee6d3e21087a98b93838e284a6086b13917f96b0d9b
       createdAt: 2019-02-28 01:03:00
       description: Create and maintain highly-available etcd clusters on Kubernetes
       repository: https://github.com/coreos/etcd-operator
       tectonic-visibility: ocs
     name: etcdoperator.v0.9.4
     namespace: placeholder
   ...
   ```
   
   
2. 打包到package目录。 
   把operator包压缩成zip格式，放至package目录下。
   ![](https://support.huaweicloud.com/accessg-osc/public_sys-resources/caution_3.0-zh-cn.png)
   package目录下只能包含一个Operator压缩包，确保使用这个包可以将整个服务部署起来。
   ```
   etcd/
       | --- package/
           | --- etcd-operator-0.9.4.zip
       | --- images/
           | --- etcd-3.5.0-x86_64.tar
           | --- etcd-3.5.0-aarch64.tar
       | --- extends/
   ```
   
   
- Helm服务
  以etcd helm为例，Chart.yaml的内容为：
  ```
  annotations:
    category: Database
  apiVersion: v2
  appVersion: 3.4.14
  dependencies:
    - name: common
      repository: https://charts.bitnami.com/bitnami
      tags:
        - bitnami-common
      version: 1.x.x
  description: etcd is a distributed key value store that provides a reliable way to store data across a cluster of machines
  ...
  ```
  
1. 编辑Chart.yaml文件。 
   如果有annotations属性，则添加子属性source，取值为ISV，如果没有annotations属性，则先添加annotations属性，再添加子属性source。
   ```
   annotations:
     source: ISV
     category: Database
   apiVersion: v2
   appVersion: 3.4.14
   dependencies:
     - name: common
       repository: https://charts.bitnami.com/bitnami
       tags:
         - bitnami-common
       version: 1.x.x
   description: etcd is a distributed key value store that provides a reliable way to store data across a cluster of machines
   ...
   ```
   
   
2. 打包放置到package目录： 
   ```
   etcd/
       | --- package/
           | --- etcd-helm-6.7.0.tgz
       | --- images/
           | --- etcd-3.5.0-x86_64.tar
           | --- etcd-3.5.0-aarch64.tar
           | --- mapping.yaml
   ```
   ![](https://support.huaweicloud.com/accessg-osc/public_sys-resources/caution_3.0-zh-cn.png)
   package目录下只能包含一个Helm模板包，确保使用这个包可以将整个服务部署起来。
   
   
 
 #### 排查和编辑镜像地址
提供商上传服务包后，镜像会保存到OSC的仓库中，提供商在上传前无法知道镜像的实际保存地址，OSC在提供商上传服务后只把value.yaml或者\*.clusterserviceversion.yaml中配置的镜像地址替换成实际的镜像地址。
![](https://support.huaweicloud.com/accessg-osc/public_sys-resources/notice_3.0-zh-cn.png)
提供商需排查Helm模板或者Operator服务包中的其它镜像地址都是引用values.yaml或者\*.clusterserviceversion.yaml中的配置。如果满足，可直接查看[替换镜像地址配置](https://support.huaweicloud.com/accessg-osc/osc_sp_0012.html)。
以etcd helm为例，Helm模板目录结构如下所示：
```
etcd/
    | --- templates/
        | --- secrets.yaml
        | --- servicemonitor.yaml
        | --- snapshot-pvc.yaml
        | --- statefulset.yaml
        | --- svc-headless.yaml
        | --- svc.yaml
    | --- Chart.lock
    | --- Chart.yaml
    | --- README.md
    | --- values.yaml
```
其中values.yaml文件内容是：
```
image:
  registry: docker.io
  repository: bitnami/etcd
  tag: 3.4.14-debian-10-r44
  debug: false
volumePermissions:
  enabled: false
  image:
    registry: docker.io
    repository: bitnami/minideb
    tag: buster
    pullPolicy: Always
  resources:
    limits: {}
    #   cpu: 100m
    #   memory: 128Mi
    requests: {}
    #   cpu: 100m
    #   memory: 128Mi
...
```
templates/statefulset.yaml中引用了values.yaml配置镜像地址的变量：
```
containers:
  - name: etcd-snapshotter
    image: {{ include "etcd.image" . }}
    imagePullPolicy: {{ .Values.image.pullPolicy | quote }}
```
服务包上传后实际的镜像地址是swr.cn-east-3.myhuaweicloud.com/osc-opensource/etcd:3.4.14-debian-10-r44，OSC会自动替换服务包中values.yaml中的镜像地址，保证服务部署时能从OSC的仓库中拉取镜像，values.yaml会被替换成：
```
image:
  registry: swr.cn-east-3.myhuaweicloud.com
  repository: osc-opensource/etcd
  tag: 3.4.14-debian-10-r44
```
因为Helm模板中其它镜像地址是引用values.yaml的配置，因此OSC替换了values.yaml后能保证部署时拉取到正确的镜像。
![](https://support.huaweicloud.com/accessg-osc/public_sys-resources/note_3.0-zh-cn.png)
对于Operator服务，请排查服务包中的镜像地址都是引用\*.clusterserviceversion.yaml中的配置。
#### 配置实例版本定义信息(可选)
OSC提供配置实例版本定义信息以支持实例升级的能力。
- Operator类型实例
  以redis为例， redis的cr内容如下所示：
  ```
  apiVersion: redis.osc/v1
  kind: Redis
  metadata:
    annotations:
      osc.huawei.com/package-source: public
      osc.io/occupied-port: 135,139
    creationTimestamp: '2021-09-29T03:21:57Z'
    finalizers:
      - storage.finalizers.redis.cluster
    generation: 2
    name: redis-fwpydh
    namespace: default
    resourceVersion: '91743432'
    selfLink: /apis/redis.osc/v1/namespaces/default/redises/redis-fwpydh
    uid: 96f0203c-0ae0-48bb-b2b8-b08d2055b0e2
  spec:
    config:
      name: default-redis-fwpydh-unvu7g
      properties:
        aof-load-truncated: 'yes'
        aof-use-rdb-preamble: 'no'
        appendfsync: everysec
        appendonly: 'no'
        hash-max-ziplist-entries: 512
        hash-max-ziplist-value: 64
        latency-monitor-threshold: 100
        list-max-ziplist-size: -2
        loglevel: notice
        maxauthfailtimes: 100
        maxclients: 10000
        maxmemory-policy: noeviction
        repl-diskless-sync: 'yes'
        set-max-intset-entries: 512
        slowlog-log-slower-than: 10000
        stop-writes-on-bgsave-error: 'no'
        timeout: 0
        zset-max-ziplist-entries: 128
        zset-max-ziplist-value: 64
    image: swr.cn-east-3.myhuaweicloud.com/osc-official/redis:21.9.18_20210918221431
    masterSize: 1
    mode: RedisHA
    ......
    ......
    此处中间省略
    ......
    ......
    phase: Available
    serviceAddr: redis-ha-redis-fwpydh.default.svc.cluster.local:6379
    serviceAddrReadonly: redis-ha-redis-fwpydh-readonly.default.svc.cluster.local:6379
    version: 21.9.18_20210918221431
  ```
  
1. 编辑文件 
   实例版本定义配置是在csd文件中配置versionDefinition，实例版本定义支持operator类型和Helm类型。
   ```
   versionDefinition:
     mode: url
     path: spec.image
     tags:    
       - 0.0.1    
       - 0.0.2    
       - 0.0.3
   ```
   
   

2. 打包到package目录。 
   把operator包压缩成zip格式的压缩包，放到package目录下。
   
   
- Helm服务 以clickhouse helm为例，values.yaml的内容如下：
  ```
  ## Timezone
  timezone: "Asia/Shanghai"
  ## Cluster domain
  clusterDomain: "cluster.local"
  ##
  ## Clickhouse Node selectors and tolerations for pod assignment
  ## ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#nodeselector
  ## ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#taints-and-tolerations-beta-feature
  ##
  # nodeSelector: {"beta.kubernetes.io/arch": "amd64"}
  # tolerations: []
  ## Clickhouse pod/node affinity/anti-affinity
  ## 
  #affinity:
  #  nodeAffinity:
  #    requiredDuringSchedulingIgnoredDuringExecution:
  #      nodeSelectorTerms:
  #      - matchExpressions:
  #        - key: "application/clickhouse"
  #          operator: In
  #          values:
  #          - "true"
  clickhouse:
    ## StatefulSet controller supports relax its ordering guarantees while preserving its uniqueness and identity guarantees. There are two valid pod management policies: OrderedReady and Parallel
    ## ref: https://kubernetes.io/docs/tutorials/stateful-application/basic-stateful-set/#pod-management-policy
    ##
    podManagementPolicy: "Parallel"
    ## StatefulSet controller supports automated updates. There are two valid update strategies: RollingUpdate and OnDelete
    ## ref: https://kubernetes.io/docs/tutorials/stateful-application/basic-stateful-set/#updating-statefulsets
    ##
    updateStrategy: "RollingUpdate"
    ## Partition update strategy
    ## https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#partitions
    ##
    # rollingUpdatePartition:
    ##
    ## The path to the directory containing data.
    ## Default value: /var/lib/clickhouse
    path: "/var/lib/clickhouse"
    ##
    ## The port for connecting to the server over HTTP
    http_port: "8123"
    ##
    ## Port for communicating with clients over the TCP protocol.
    tcp_port: "9000"
    ##
    ## Port for exchanging data between ClickHouse servers.
    interserver_http_port: "9009"
    ## 
    ## The instance number of Clickhouse
    replicas: "3"
    ## Clickhouse image configuration.
    image: "swr.cn-east-3.myhuaweicloud.com/osctest/clickhouse-server"
    imageVersion: "0.0.1"
    imagePullPolicy: "IfNotPresent"
    imageBusybox: "swr.cn-north-7.myhuaweicloud.com/osctest/busybox:1.26.2"
    #imagePullSecrets: 
    ## Periodic probe of container liveness. Container will be restarted if the probe fails. Cannot be updated. 
    ## More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes
    livenessProbe:
      enabled: true
      initialDelaySeconds: "30"
      periodSeconds: "30"
      timeoutSeconds: "5"
      failureThreshold: "3"
      successThreshold: "1"
    ## Periodic probe of container service readiness. Container will be removed from service endpoints if the probe fails. Cannot be updated. 
    ## More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes
    readinessProbe:
      enabled: true
      initialDelaySeconds: "30"
      periodSeconds: "30"
      timeoutSeconds: "5"
      failureThreshold: "3"
      successThreshold: "1"
    ## volumeClaimTemplates is a list of claims that pods are allowed to reference. 
    ## The StatefulSet controller is responsible for mapping network identities to claims in a way that maintains the identity of a pod. 
    ## Every claim in this list must have at least one matching (by name) volumeMount in one container in the template. 
    ## A claim in this list takes precedence over any volumes in the template, with the same name.
    persistentVolumeClaim:
      enabled: false
      ## Clickhouse data volume
      dataPersistentVolume: 
        enabled: false
        accessModes: 
        - "ReadWriteOnce"
        storageClassName: "-"
        storage: "500Gi"
      ## Clickhouse logs volume
      logsPersistentVolume:
        enabled: false
        accessModes: 
        - "ReadWriteOnce"
        storageClassName: "csi-disk"
        storage: "50Gi"
    ##
    ## An API object that manages external access to the services in a cluster, typically HTTP.
    ## Ingress can provide load balancing, SSL termination and name-based virtual hosting.
    ingress: 
      enabled: false
    #  host: "clickhouse.domain.com"
    #  path: "/"
    #  tls: 
    #    enabled: false
    #    hosts: 
    #    - "clickhouse.domain.com"
    #    - "clickhouse.domain1.com"
    #    secretName: "clickhouse-secret"
    ## 
    ## Clickhouse config.xml and metrica.xml
    ......
    ......
    后续省略
    ......
    ......
  ```
  
1. 编辑文件。 
   实例版本定义配置是在csd文件中配置versionDefinition，实例版本定义支持operator类型和Helm类型。
   ```
   versionDefinition:
     mode: tag
     path: clickhouse.imageVersion
     tags:    
       - 0.0.1    
       - 0.0.2    
       - 0.0.3
   ```
   
   

2. 打包到package目录。 
   把Helm包压缩成zip格式的压缩包，放到package目录下。