Cloud Container Engine
Cloud Container Engine
Todos os resultados de "
" dentro deste produto
Todos os resultados de "
" dentro deste produto
Visão geral de serviço
Infográfico do CCE
O que é o Cloud Container Engine?
Vantagens do produto
Cenários de aplicação
Gerenciamento de infraestrutura e da aplicação conteinerizada
Dimensionamento automático em segundos
Gerenciamento de tráfego de micros serviços
DevOps e CI/CD
Arquitetura de nuvem híbrida
Agendamento de alto desempenho
Observações e restrições
Detalhes de preço
Gerenciamento de permissões
Regiões e as AZs
Serviços relacionados
Primeiros passos
Introdução
Preparações
Criação um cluster do Kubernetes
Criação de uma Implementação (Nginx)
Implementação de WordPress e MySQL que dependem um do outro
Visão geral
Criação de uma carga de trabalho do MySQL
Criação de uma carga de trabalho do WordPress
Implementação de WordPress usando o Helm
Guia de usuário
Operações de alto risco e soluções
Clusters
Visão geral de cluster
Informações básicas do cluster
Observações de versão do Kubernetes
Observações de versão Kubernetes 1.25
Observações de versão Kubernetes 1.23
Observações de versão Kubernetes 1.21
Observações de versão Kubernetes 1.19
Observações de lançamento para versões de cluster do CCE
Compra de um cluster
Clusters do CCE Turbo e clusters do CCE
Compra de um cluster do CCE
Comparação entre iptables e IPVS
Conexão a um cluster
Conexão a um cluster usando o kubectl
Conexão a um cluster usando o CloudShell
Conexão a um cluster usando um certificado X.509
Acesso a um cluster usando um nome de domínio personalizado
Atualização de um cluster
Visão geral de atualização
Antes de começar
Atualização in-loco
Execução de verificação pós-atualização
Verificação de pod
Verificação de rede de nó e contêiner
Verificação de rótulo e mancha do nó
Verificação de ignoração de nó para redefinição
Verificação do serviço
Verificação de novo nó
Verificação de pod novo
Migração de serviços em clusters de versões diferentes
Solução de problemas para exceções de verificação de pré-atualização
Verificação de pré-atualização
Restrições de nó
Gerenciamento de atualização
Complementos
Gráficos do Helm
Conectividade SSH de nós principais
Pools de nós
Grupos de segurança
Restrições de nó Arm
Nós a serem migrados
Recursos do Kubernetes descartados
Riscos de compatibilidade
Versões do agente do CCE de nó
Uso da CPU do nó
CRDs
Discos de nó
DNS do nó
Permissões de arquivo de diretório principal de nó
Kubelet
Memória do nó
Servidor de sincronização de relógio de nó
Sistema operacional do nó
CPUs do nó
Comandos Python do nó
Versão do ASM
Prontidão do nó
Nó journald
containerd.sock
Erros internos
Pontos de montagem do nó
Manchas de nós do Kubernetes
Restrições de everest
Restrições de cce-hpa-controller
Políticas de CPU aprimorada
Integridade dos componentes do nó de trabalho
Integridade dos componentes do nó principal
Limite de recursos de memória dos componentes do Kubernetes
APIs do Kubernetes descartadas
Capacidades de IPv6 de um cluster do CCE Turbo
NetworkManager de nó
Arquivo de ID do nó
Consistência da configuração do nó
Arquivo da configuração de nó
Consistência da configuração de CoreDNS
Comandos sudo de um nó
Comandos principais dos nós
A montagem do arquivo sock em um nó
Consistência do certificado do balanceador de carga de HTTPS
Montagem do nó
Permissões de logon de usuário paas em um nó
Endereços IPv4 privados de balanceadores de carga
Registros históricos de atualização
Bloco CIDR do plano de gerenciamento do cluster
Complemento da GPU
Configurações de parâmetro do sistema dos nós
Versões de pacotes residuais
Comandos do nó
Troca de nó
Gerenciamento de um cluster
Gerenciamento de configuração de cluster
Controle de sobrecarga do cluster
Alteração de escala do cluster
Alteração do grupo de segurança padrão de um nó
Exclusão de um cluster
Hibernação e despertar de um cluster (pagamento por uso)
Renovação de um cluster de cobrança anual/mensal
Alteração do modo de cobrança de pagamento por uso para anual/mensal
Nós
Visão geral do nó
Mecanismo de contêiner
Sistema operacional do nó
Criação de um nó
Adição de nós para gerenciamento
Logon em um nó
Nós de gerenciamento
Gerenciamento de rótulos de nó
Gerenciamento de manchas de nó
Redefinição de um nó
Remoção de um nó
Sincronização de dados com servidores em nuvem
Drenagem de um nó
Exclusão de um nó
Alteração de pagamento por uso para anual/mensal
Interrupção de um nó
Execução de atualização contínua para nós
O&M do nó
Política de reserva de recursos de nó
Alocação de espaço em disco de dados
Número máximo de pods que podem ser criados em um nó
Migração de nós do Docker para containerd
Otimização de parâmetros do sistema de nó
Lista de parâmetros do sistema de nós que podem ser otimizados
Alteração de RuntimeMaxUse da memória usada pelo cache de log em um nó
Alteração do número máximo de identificadores de arquivo
Modificação de parâmetros do nó de kernel
Alteração de limites de ID de processo (kernel.pid_max)
Política de detecção de falhas de nó
Pools de nós
Visão geral do pool de nós
Criação de um pool de nós
Gerenciamento de um pool de nós
Atualização de um pool de nós
Atualização de uma configuração de AS
Configuração de um pool de nós
Cópia de um pool de nós
Sincronização de pools de nós
Atualização de um sistema operacional
Migração de um nó
Exclusão de um pool de nós
Cargas de trabalho
Visão geral
Criação de uma carga de trabalho
Criação de uma Implantação
Criação de um StatefulSet
Criação de um DaemonSet
Criação de uma tarefa
Criação de uma tarefa cronometrada
Configuração de um contêiner
Configuração da sincronização de fuso horário
Configuração de uma política de extração de imagem
Uso de imagens de terceiros
Configuração de especificações do contêiner
Definição dos parâmetros do ciclo de vida do contêiner
Configuração da verificação de integridade para um contêiner
Configuração de uma variável de ambiente
Configuração de configurações de APM para análise de gargalo de desempenho
Configuração da política de atualização da carga de trabalho
Política de agendamento (afinidade/antiafinidade)
Manchas e tolerâncias
Rótulos e anotações
Acesso de um contêiner
Gerenciamento de cargas de trabalho e tarefas
Tempo de execução do Kata e tempo de execução comum
Agendamento
Visão geral
Agendamento de CPU
Política de CPU
Política de CPU aprimorada
Agendamento de GPU
Agendamento de GPU padrão no Kubernetes
Agendamento de NPU
Agendamento de Volcano
Binpack
Reagendador
Afinidade do pool de nós
Agendamento baseado em prioridade e preempção
DRF
Gang
Agendamento de afinidade NUMA
Implementação híbrida da nuvem nativa
Excesso de assinaturas de recursos dinâmicos
Intermitência de CPU
Garantia de largura de banda de rede de egress
Rede
Visão geral
Modelos de rede de contêineres
Visão geral
Rede de túneis de contêineres
Rede da VPC
Cloud Native Network 2.0
Serviço
Visão geral
ClusterIP
NodePort
LoadBalancer
Criação de um Serviço LoadBalancer
Uso de anotações para configurar o balanceamento de carga
Serviço usando HTTP
Configuração da verificação de integridade para várias portas
Configuração do status pronto para o pod por meio da verificação de integridade do ELB
Configuração do tempo limite para um Serviço LoadBalancer
Configuração de rede de passagem para um Serviço de LoadBalancer
Ativação de regras de grupo de segurança ICMP
DNAT
Serviços headless
Ingresses
Visão geral
Ingresses do ELB
Criação de um ingress do ELB no console
Uso do kubectl para criar um ingress do ELB
Configuração de ingresses do ELB usando anotações
Configuração de certificados HTTPS para ingresses do ELB
Configuração da Indicação de nome do servidor (SNI) para ingresses de ELB
Roteamento de ingresses do ELB para vários Serviços
Ingresses do ELB usando HTTP/2
Interconexão de ingresses do ELB com serviços de back-end HTTPS
Configuração do tempo limite para um ingress do ELB
Ingresses de Nginx
Criação de ingresses de Nginx no console
Uso do kubectl para criar um ingress de Nginx
Configuração de certificados HTTPS para ingresses do Nginx
Configuração de regras de reescrita de URL para ingresses de Nginx
Interconexão de ingresses do Nginx com serviços de back-end HTTPS
Ingresses de Nginx usar hashing consistente para balanceamento de carga
Configuração de ingresses de Nginx usando anotações
DNS
Visão geral
Configuração de DNS
Uso de CoreDNS para resolução de nome de domínio personalizado
Uso do DNSCache do NodeLocal para melhorar o desempenho do DNS
Configurações de rede de contêiner
Rede host
Configuração da limitação da taxa de QoS para acesso entre pods
Configurações de rede de túnel de contêiner
Políticas de rede
Configurações de Cloud Native Network 2.0
Políticas de grupo de segurança
NetworkAttachmentDefinition
Configuração de um endereço IP estático para um pod
Configuração de um EIP para um pod
Configuração de um EIP estático para um pod
Configurações da rede do cluster
Adição de um bloco CIDR secundário de VPC a um cluster
Alternação de uma sub-rede de nó
Adição de um bloco CIDR de contêiner para um cluster
Configuração do acesso dentro da VPC
Acesso a redes públicas a partir de um contêiner
Armazenamento
Visão geral
Noções básicas de armazenamento
Elastic Volume Service
Visão geral
Uso de um disco EVS existente através de um PV estático
Uso de um disco EVS por meio de um PV dinâmico
Montagem dinâmica de um disco EVS para um StatefulSet
Snapshots e backups
Scalable File Service
Visão geral
Uso de um sistema de arquivos do SFS existente por meio de um PV estático
Uso de um sistema de arquivos do SFS através de um PV dinâmico
Configuração de opções de montagem de volume do SFS
SFS Turbo
Visão geral
Uso de um sistema de arquivos do SFS Turbo existente por meio de um PV estático
Configuração de opções de montagem do SFS Turbo
Criação dinâmica de um subdiretório do SFS Turbo usando StorageClass
Object Storage Service
Visão geral
Uso de um bucket do OBS existente através de um PV estático
Uso de um bucket do OBS através de um PV dinâmico
Configuração das opções de montagem do OBS
Uso de uma chave de acesso (AK/SK) personalizada para montar um volume do OBS
Uso de buckets do OBS entre regiões
Volumes persistentes locais
Visão geral
Importação de um PV para um pool de armazenamento
Uso de um PV local através de um PV dinâmico
Montagem dinâmica de um PV local para um StatefulSet
Volumes efêmeros
Visão geral
Importação de um EV para um pool de armazenamento
Uso de um EV local
Uso de um caminho temporário
hostPath
StorageClass
Observabilidade
Registro em logs
Visão geral
Uso do ICAgent para coletar logs de contêiner
Uso do log-agent para coletar logs de contêiner
Exibição de logs do plano de controle de cluster
Monitoramento
Visão geral do monitoramento
Monitoramento de métricas personalizadas no AOM
Monitoramento de métricas personalizadas usando o Prometheus
Monitoramento de métricas dos componentes do nó principais
Centro de monitoramento
Visão geral
Insight de contêiner
Cluster
Nós
Cargas de trabalho
Pods
Eventos
Diagnóstico de integridade
Painel
Gerenciamento de alarmes
Alarm Assistant
Configurações de alarme personalizadas
Logs do CTS
Operações do CCE suportadas pelo CTS
Consulta de logs do CTS
Namespaces
Criação de um namespace
Gerenciamento de namespaces
Configuração de uma cota de recurso
ConfigMaps e segredos
Criação de um ConfigMap
Uso de um ConfigMap
Criação de um segredo
Uso de um segredo
Segredos do cluster
Auto Scaling
Visão geral
Dimensionamento de uma carga de trabalho
Mecanismos de dimensionamento da carga de trabalho
Criação de uma política HPA para dimensionamento automático da carga de trabalho
Políticas CronHPA
Criação de uma política CustomedHPA para o dimensionamento automático da carga de trabalho
Gerenciamento de políticas de dimensionamento de carga de trabalho
Dimensionamento de um nó
Mecanismos de dimensionamento de nós
Criação de uma política de dimensionamento de nós
Gerenciamento de políticas de dimensionamento de nós
Uso de HPA e CA para dimensionamento automático de cargas de trabalho e nós
Complementos
Visão geral
CoreDNS
Armazenamento do contêiner do CCE (Everest)
Detector de problema de nó do CCE
Kubernetes Dashboard
Autoscaler de cluster do CCE
Nginx Ingress controller
Kubernetes Metrics Server
HPA de CCE avançado
Mecanismo de estouro de nuvem do CCE para CCI
Suíte IA do CCE (GPU NVIDIA)
Suíte de IA do CCE (Ascend NPU)
Volcano scheduler
Secrets Manager do CCE para DEW
Exportador de métricas de rede do CCE
NodeLocal DNSCache
Monitoramento de cluster da nuvem nativa
Registro de logs da nuvem nativa
e-backup (EOM)
web-terminal (EOM)
Prometheus (EOM)
FlexVolume (preterido)
Gráfico do Helm
Visão geral
Implementação de uma aplicação a partir de um gráfico
Diferenças entre Helm v2 e Helm v3 e soluções de adaptação
Implementação de uma aplicação através do cliente de Helm v2
Implementação de uma aplicação através do cliente de Helm v3
Conversão um release do Helm v2 para v3
Permissões
Visão geral de permissões
Permissões de cluster (baseadas no IAM)
Permissões de namespace (com base no RBAC do Kubernetes)
Exemplo: projetar e configurar permissões para usuários em um departamento
Dependência de permissão do console do CCE
Segurança de pod
Configuração de uma política de segurança de pod
Configuração de admissão de segurança do pod
Melhoria da segurança do token da conta de serviço
Descrição da atribuição do sistema
Gerenciamento do armazenamento: FlexVolume (preterido)
Visão geral de FlexVolume
Alteração da classe de armazenamento usada por um cluster de v1.15 de FlexVolume para CSI Everest
Uso de discos EVS como volumes de armazenamento
Visão geral
(kubectl) Criação automática de um disco EVS
(kubectl) Criação de um PV a partir de um disco EVS existente
(kubectl) Criação de um pod montado com um volume do EVS
Usar sistemas de arquivos do SFS Turbo como volumes de armazenamento
Visão geral
(kubectl) Criação de um PV a partir de um sistema de arquivos do SFS Turbo existente
(kubectl) Criação de uma Implementação montada com um volume do SFS Turbo
(kubectl) Criação de um StatefulSet montado com um volume do SFS Turbo
Uso de buckets do OBS como volumes de armazenamento
Visão geral
(kubectl) Criação automática de um volume do OBS
(kubectl) Criação de um PV a partir de um bucket do OBS existente
(kubectl) Criação de uma Implementação montada com um volume do OBS
(kubectl) Criação de um StatefulSet montado com um volume do OBS
Usar sistemas de arquivos do SFS como volumes de armazenamento
Visão geral
(kubectl) Criação automática de um volume do SFS
(kubectl) Criação de um PV a partir de um sistema de arquivos do SFS existente
(kubectl) Criação de uma Implementação montada com um volume do SFS
(kubectl) Criação de um StatefulSet montado com um volume do SFS
Referência de API
Antes de começar
Visão geral
Chamada de API
Pontos de extremidade
Restrições
Conceitos
Visão geral de API
Chamada das APIs
Fazer uma solicitação de API
Autenticação
Resposta
APIs
URL da API
Gerenciamento de cluster
Criação de um cluster
Leitura de um cluster especificado
Listagem de clusters em um projeto especificado
Atualização de um cluster especificado
Exclusão de um cluster
Hibernação de um cluster
Despertar de um cluster
Obtenção de um certificado de cluster
Modificação de especificações do cluster
Consulta de uma tarefa
Vinculação/desvinculação endereço do servidor da API público
Obtenção de endereço de acesso ao cluster
Gerenciamento de nó
Criação de um nó
Leitura de um nó especificado
Listagem de todos os nós em um cluster
Atualização de um nó especificado
Exclusão de um nó
Aceitação de um nó
Redefinição de um nó
Remoção de um nó
Migração de um nó
Gerenciamento de pool de nós
Criação de um pool de nós
Leitura de um pool de nós especificado
Listagem de todos os pools de nós em um cluster especificado
Atualização de um pool de nós especificado
Exclusão de um pool de nós
Gerenciamento de armazenamento
Criação de uma PVC (a ser descartada)
Exclusão de uma PVC (a ser descartada)
Gerenciamento de complementos
Instalação de uma instância de complemento
Listagem de modelos de complemento
Atualização de uma instância de complemento
Reversão de uma instância de complemento
Exclusão de uma instância de complemento
Consulta de uma instância de complemento
Listagem de instâncias do complemento
Gerenciamento de cota
Consulta de cotas de recursos
Gerenciamento de tags
Adição de tags de recurso a um cluster especificado em lotes
Exclusão de tags de recurso de um cluster especificado em lotes
Atualização de cluster
Atualização de um cluster
Obtenção de detalhes da tarefa de atualização do cluster
Nova tentativa de uma tarefa de atualização de cluster
Suspensão de uma tarefa de atualização de cluster
Retomada de uma tarefa de atualização de cluster
Versões de API
Obtenção de versões da API
Descrição de parâmetros de instância do complemento
virtual-kubelet
APIs do Kubernetes
APIs desatualizadas
Obtenção de certificados de cluster
Criação de um PersistentVolume
Exclusão de um PersistentVolume
Políticas de permissões e ações suportadas
Apêndice
Código de status
Códigos de erro
Obtenção de um ID de projeto
Obtenção de um ID de conta
Especificação dos complementos a serem instalados durante a criação do cluster
Como obter parâmetros no URI da API
Criação de uma VPC e uma sub-rede
Criação de um par de chaves
Descrição de flavor de nó
Adição de um sal no campo de senha ao criar um nó
Número máximo de pods que podem ser criados em um nó
Sistema operacional do nó
Alocação de espaço em disco de dados
Anexação de discos a um nó
Perguntas frequentes
Perguntas comuns
Cobrança
Como o CCE é faturado/cobrado?
Como alterar o modo de cobrança de um cluster do CCE de pagamento por uso para anual/mensal?
Posso alterar o modo de cobrança dos nós do CCE de pagamento por uso para anual/mensal?
Quais modos de fatura são suportados pela HUAWEI CLOUD?
Será notificado quando meu saldo for insuficiente?
Será notificado quando o saldo da minha conta mudar?
Posso excluir diretamente um cluster do CCE com cobrança anual/mensal quando ele expirar?
Como cancelar a assinatura do CCE?
Reembolso para CCE é suportado?
Cluster
Criação de cluster
Por que não conseguir criar um cluster do CCE?
A escala de gerenciamento de um cluster está relacionada ao número de nós principais?
Como atualizar o certificado raiz ao criar um cluster do CCE?
Quais cotas de recursos devo prestar atenção ao usar o CCE?
Execução de cluster
Como corrigir a falha quando o status do cluster está Unavailable?
Como redefinir ou reinstalar um cluster do CCE?
Como verificar se um cluster está no modo de vários principais?
Posso me conectar diretamente ao nó principal de um cluster?
Como recuperar dados após um cluster é excluído?
Como atualizar um namespace no estado Terminating?
Por que o CCE indica o uso do disco do nó inconsistente com o Cloud Eye?
Exclusão de cluster
Falha ao excluir um cluster: as ENIs residuais
Atualização do cluster
O que fazer se um complemento de cluster falhar ao ser atualizado durante atualização de cluster do CCE?
Nó
Criação de nó
Como solucionar problemas ocorridos ao adicionar nós a um cluster do CCE?
Como solucionar problemas ocorridos ao aceitar nós em um cluster do CCE?
O que fazer se um nó não for aceito porque não for instalado?
Execução de nó
O que fazer se um cluster estiver disponível, mas alguns nós não estiverem disponíveis?
Como solucionar a falha de logon remoto a um nó em um cluster do CCE?
Como fazer logon em um nó usando uma senha e redefinir a senha?
Como coletar logs de nós em um cluster do CCE?
O que fazer se a rede de contêiner ficar indisponível após yum update ser usado para actualizar o SO?
O que fazer se o disco vdb de um nó estiver danificado e o nó não puder ser recuperado após a redefinição?
Quais portas são usadas para instalar o kubelet em nós de cluster do CCE?
Como configurar um pod para usar a capacidade de aceleração de um nó de GPU?
O que fazer se ocorrer ocasionalmente uma suspensão de I/O quando discos SCSI EVS forem usados?
O que fazer se logs de auditoria excessivos do Docker afetarem a I/O do disco?
Como corrigir um contêiner ou nó anormal devido a nenhum espaço em disco de thin pool?
Quais portas que um nó escuta?
Como corrigir falhas quando o driver de NVIDIA é usado para iniciar contêineres em nós de GPU?
O horário NTP dos nós não está sincronizado
Alteração de especificação
Como alterar as especificações do nó em um cluster do CCE?
O que fazer se falhar ao reiniciar ou criar cargas de trabalho em um nó após modificar as especificações do nó?
Posso alterar o endereço IP de um nó em um cluster do CCE?
Kernel do nó
Quando as aplicações são repetidamente criadas e excluídas em um nó de CentOS com uma versão anterior do kernel, o vazamento do cgroup kmem ocorre ocasionalmente
Por que os pods são removidos pelo kubelet devido a estatísticas anormais do cgroup?
Quando OOM do contêiner ocorre no nó de CentOS com uma versão anterior do kernel, o sistema de arquivos Ext4 é ocasionalmente suspenso
Pool de nós
O que fazer se nenhum registro de criação de nós for exibido quando o pool de nós estiver sendo expandido?
Carga de trabalho
Anormalidades da carga de trabalho
Como usar eventos para corrigir cargas de trabalho anormais?
O que fazer se o agendamento do pod falhar?
O que fazer se um pod não conseguir extrair a imagem?
O que fazer se a inicialização do contêiner falhar?
O que fazer se um pod não for removido?
O que fazer se um volume de armazenamento não puder ser montado ou o tempo de montagem expirar?
O que fazer se uma carga de trabalho permanece no estado Creating?
O que fazer se os pods no estado Terminating não puderem ser excluídos?
O que fazer se uma carga de trabalho for interrompida causada pela exclusão de pods?
O que fazer se ocorrer um erro ao implementar um serviço no nó da GPU?
O que fazer se os erros relacionados à sandbox forem relatados quando o pod permanecer no estado Creating?
Por que o pod falha ao gravar dados?
Por que a criação ou exclusão de pods é suspensa em um nó onde o armazenamento de arquivos é montado?
Código indicando que o contêiner sai anormalmente
Configuração do contêiner
Quando o processamento pré-parado é usado?
Como definir um FQDN para acessar um contêiner especificado no mesmo namespace?
O que fazer se as sondas de verificação de saúde falharem ocasionalmente?
Como definir o valor de umask para um contêiner?
O que fazer se um erro for relatado quando um contêiner implementado for iniciado após o parâmetro de memória de heap de inicialização da JVM ser especificado para ENTRYPOINT no Dockerfile?
Qual é o mecanismo de repetição quando o CCE falha ao iniciar um pod?
Monitoramento de alarmes
Por quanto tempo os eventos de uma carga de trabalho são armazenados?
Políticas de agendamento
Como distribuir uniformemente vários pods para cada nó?
Como evitar que um contêiner em um nó seja removido?
Por que os pods não são uniformemente distribuídos aos nós?
Como remover todos os pods em um nó?
Outros
O que fazer se uma tarefa agendada não puder ser reiniciada depois de ser interrompida por um período de tempo?
O que é um Serviço headless quando criar um StatefulSet?
O que fazer se a mensagem de erro "Auth is empty" é exibida quando uma imagem privada é extraída?
Por que um pod não pode ser agendado para um nó?
Qual é a política de extração de imagem para contêineres em um cluster do CCE?
Por que o ponto de montagem de um contêiner do Docker no cluster do Kunpeng é desinstalado?
O que fazer se uma camada estiver faltando durante a extração da imagem?
Por que a permissão de arquivo e o usuário no contêiner são pontos de interrogação?
Redes
Planejamento de rede
Qual é a relação entre clusters, VPCs e sub-redes?
Como visualizar o bloco CIDR da VPC?
Como configurar o bloco CIDR da VPC e o bloco CIDR da sub-rede para um cluster do CCE?
Como definir um bloco CIDR de contêiner para um cluster do CCE?
Quando usar a Cloud Native Network 2.0?
O que é uma ENI?
Configuração de regras do grupo de segurança do cluster
Como definir um bloco CIDR de serviço IPv6?
Falha na rede
Como localizar uma falha de rede de carga de trabalho?
Por que o endereço do ELB não pode ser usado para acessar cargas de trabalho em um cluster?
Por que ingress não pode ser acessado fora do cluster?
Por que o navegador retorna o código de erro 404 quando acesso uma aplicação implementada?
O que fazer se um contêiner falhar ao se conectar à Internet?
O que fazer se uma sub-rede da VPC não puder ser excluída?
Como restaurar uma NIC de contêiner com defeito?
O que fazer se um nó falhar ao se conectar à Internet (rede pública)?
Como resolver um conflito entre o bloco CIDR da VPC e o bloco CIDR do contêiner?
O que fazer se o erro de Java "Connection reset by peer" for relatado durante a verificação de integridade do ELB da camada 4
Como localizar o evento de Serviço que indica que nenhum nó está disponível para vinculação?
Por que "Dead loop on virtual device gw_11cbf51a, fix it urgently" ocorre de forma intermitente quando fazer logon em uma VM usando o VNC?
Por que um pânico ocorre ocasionalmente quando uso políticas de rede em um nó de cluster?
Por que muitos logs de ip_type de origem são gerados no VNC?
Fortalecimento de segurança
Como evitar que nós de cluster sejam expostos a redes públicas?
Configuração de rede
Como o CCE se comunica com outros serviços da HUAWEI CLOUD através de uma intranet?
Como configurar a porta ao configurar o modo de acesso da carga de trabalho no CCE?
Como obter compatibilidade entre a propriedade do ingress e o cliente-go do Kubernetes?
Outros
Como obter um certificado de chave TLS?
Várias NICs podem ser vinculadas a um nó em um cluster do CCE?
Por que o grupo de servidores back-end de um ELB é excluído automaticamente depois que um Serviço é publicado no ELB?
Por que um ingress não pode ser criado depois que o namespace é alterado?
Como obter o endereço IP de origem real de um cliente depois que um serviço é adicionado ao Istio?
Como alterar o grupo de segurança de nós em um cluster em lotes?
Armazenamento
Quais são as diferenças entre as classes de armazenamento do CCE em termos de armazenamento persistente e montagem de vários nós?
Posso adicionar um nó sem um disco de dados de 100 GB?
Posso restaurar um disco EVS usado como um volume persistente em um cluster do CCE depois que o disco é excluído ou expira?
O que fazer se o host não puder ser encontrado quando os arquivos precisarem ser carregados para o OBS durante o acesso ao Serviço do CCE de uma rede pública?
Em quantos nós (ECSs) um sistema de arquivos do SFS pode ser montado?
Como alcançar a compatibilidade entre o ExtendPathMode e o client-go de Kubernetes?
O que fazer se um volume de armazenamento falhar ao ser criado?
PVCs do CCE podem detectar falhas de armazenamento subjacentes?
Namespace
Por que não consigo excluir um namespace devido a uma falha de acesso a objetos do APIService?
Gráfico e complemento
O que fazer se o complemento nginx-ingress falhar ao ser instalado em um cluster e permanecer no estado Creating?
O que fazer se os recursos de processo residual existirem devido a uma versão anterior do complemento npd?
O que fazer se uma release de gráfico não puder ser excluída porque o formato do gráfico está incorreto?
O CCE oferece suporte ao nginx-ingress?
Por que a instalação do complemento falha e exibe "The release name is already exist"?
Por que falha a criação ou atualização de release e exibe "rendered manifests contain a resource that already exists"?
Perguntas frequentes sobre API & kubectl
Como acessar um cluster do CCE?
Os recursos criados usando APIs ou kubectl podem ser exibidos no console do CCE?
Como baixar o kubeconfig para conectar-se a um cluster usando o kubectl?
Como corrigir o erro relatado ao executar o comando kubectl top node?
Por que "Error from server (Forbidden)" é exibido quando uso o kubectl?
Perguntas frequentes sobre DNS
O que fazer se a resolução de nome de domínio falhar?
Por que um contêiner em um cluster do CCE falha ao executar a resolução de DNS?
Por que o nome de domínio da zona do locatário não pode ser resolvido depois que a configuração de DNS de sub-rede é modificada?
Como otimizar a configuração se a definição de nome de domínio externo é lenta ou expirar o tempo limite?
Como configurar uma política de DNS para um contêiner?
Perguntas frequentes sobre o repositório de imagens
Como criar uma imagem do Docker e resolver o problema da extração lenta da imagem?
Como fazer uplpload das minhas imagens para o CCE?
Permissões
Posso configurar apenas permissões de namespace sem permissões de gerenciamento de cluster?
Posso usar APIs do CCE se as permissões de gerenciamento de cluster não estiverem configuradas?
Posso usar o kubectl se as permissões de gerenciamento de cluster não estiverem configuradas?
Referência
Como expandir a capacidade de armazenamento de um contêiner?
Como os endereços IP de contêiner podem sobreviver a uma reinicialização de contêiner?
Quais são as diferenças entre CCE e CCI?
Quais são as diferenças entre CCE e ServiceStage?
What's New
Function Overview
Product Bulletin
Latest Notices
Product Change Notices
EOM of CentOS
Billing Changes for Huawei Cloud CCE Autopilot Data Plane
CCE Autopilot for Commercial Use on September 30, 2024, 00:00 GMT+08:00
Reliability Hardening for Cluster Networks and Storage Functions
Support for Docker
Service Account Token Security Improvement
Upgrade of Helm v2 to Helm v3
Problems Caused by conn_reuse_mode Settings in the IPVS Forwarding Mode of CCE Clusters
Optimized Key Authentication of the everest Add-on
Cluster Version Release Notes
End of Maintenance for Clusters 1.28
End of Maintenance for Clusters 1.27
End of Maintenance for Clusters 1.25
End of Maintenance for Clusters 1.23
End of Maintenance for Clusters 1.21
End of Maintenance for Clusters 1.19
End of Maintenance for Clusters 1.17
End of Maintenance for Clusters 1.15
End of Maintenance for Clusters 1.13
Creation of CCE Clusters 1.13 and Earlier Not Supported
Upgrade for Kubernetes Clusters 1.9
Vulnerability Notices
Vulnerability Fixing Policies
Notice of Kubernetes Security Vulnerability (CVE-2025-7342)
Notice of Grafana Security Vulnerability (CVE-2025-4123)
Notice of the NVIDIA Container Toolkit Container Escape Vulnerabilities (CVE-2025-23266 and CVE-2025-23267)
Notice of the NGINX Ingress Controller Vulnerabilities (CVE-2025-1974, CVE-2025-1097, CVE-2025-1098, CVE-2025-24513, and CVE-2025-24514)
Notice of Kubernetes Security Vulnerability (CVE-2025-0426)
Notice of Kubernetes Security Vulnerability (CVE-2024-10220)
Notice of Kubernetes Security Vulnerabilities (CVE-2024-9486 and CVE-2024-9594)
Notice of Container Escape Vulnerability in NVIDIA Container Toolkit (CVE-2024-0132)
Notice of Linux Remote Code Execution Vulnerability in CUPS (CVE-2024-47076, CVE-2024-47175, CVE-2024-47176, and CVE-2024-47177)
Notice of the NGINX Ingress Controller Vulnerability That Allows Attackers to Bypass Annotation Validation (CVE-2024-7646)
Notice of Docker Engine Vulnerability That Allows Attackers to Bypass AuthZ (CVE-2024-41110)
Notice of Linux Kernel Privilege Escalation Vulnerability (CVE-2024-1086)
Notice of OpenSSH Remote Code Execution Vulnerability (CVE-2024-6387)
Notice of Fluent Bit Memory Corruption Vulnerability (CVE-2024-4323)
Notice of runC systemd Attribute Injection Vulnerability (CVE-2024-3154)
Notice of the Impact of runC Vulnerability (CVE-2024-21626)
Notice of Kubernetes Security Vulnerability (CVE-2022-3172)
Notice of Privilege Escalation Vulnerability in Linux Kernel openvswitch Module (CVE-2022-2639)
Notice of nginx-ingress Add-on Security Vulnerability (CVE-2021-25748)
Notice of nginx-ingress Security Vulnerabilities (CVE-2021-25745 and CVE-2021-25746)
Notice of containerd Process Privilege Escalation Vulnerability (CVE-2022-24769)
Notice of CRI-O Container Runtime Engine Arbitrary Code Execution Vulnerability (CVE-2022-0811)
Notice of Container Escape Vulnerability Caused by the Linux Kernel (CVE-2022-0492)
Notice of Non-Security Handling Vulnerability of containerd Image Volumes (CVE-2022-23648)
Notice of Linux Kernel Integer Overflow Vulnerability (CVE-2022-0185)
Notice of Linux Polkit Privilege Escalation Vulnerability (CVE-2021-4034)
Notice of Vulnerability of Kubernetes subPath Symlink Exchange (CVE-2021-25741)
Notice of runC Vulnerability That Allows a Container Filesystem Breakout via Directory Traversal (CVE-2021-30465)
Notice of Docker Resource Management Vulnerability (CVE-2021-21285)
Notice of NVIDIA GPU Driver Vulnerability (CVE-2021-1056)
Notice of the Sudo Buffer Vulnerability (CVE-2021-3156)
Notice of the Kubernetes Security Vulnerability (CVE-2020-8554)
Notice of Apache containerd Security Vulnerability (CVE-2020-15257)
Notice of Docker Engine Input Verification Vulnerability (CVE-2020-13401)
Notice of Kubernetes kube-apiserver Input Verification Vulnerability (CVE-2020-8559)
Notice of Kubernetes kubelet Resource Management Vulnerability (CVE-2020-8557)
Notice of Kubernetes kubelet and kube-proxy Authorization Vulnerability (CVE-2020-8558)
Notice of Fixing the Kubernetes HTTP/2 Vulnerability
Notice of Fixing the Linux Kernel SACK Vulnerabilities
Notice of Fixing the Docker Command Injection Vulnerability (CVE-2019-5736)
Notice of Fixing the Kubernetes Permission and Access Control Vulnerability (CVE-2018-1002105)
Notice of Fixing the Kubernetes Dashboard Security Vulnerability (CVE-2018-18264)
Product Release Notes
Cluster Versions
Kubernetes Version Policy
Kubernetes Version Release Notes
Kubernetes 1.32 Release Notes
Kubernetes 1.31 Release Notes
Kubernetes 1.30 Release Notes
Kubernetes 1.29 Release Notes
Kubernetes 1.28 Release Notes
Kubernetes 1.27 Release Notes
Kubernetes 1.25 (EOM) Release Notes
Kubernetes 1.23 (EOM) Release Notes
Kubernetes 1.21 (EOM) Release Notes
Kubernetes 1.19 (EOM) Release Notes
Kubernetes 1.17 (EOM) Release Notes
Kubernetes 1.15 (EOM) Release Notes
Kubernetes 1.13 (EOM) Release Notes
Kubernetes 1.11 (EOM) Release Notes
Kubernetes 1.9 (EOM) and Earlier Versions Release Notes
Patch Versions
OS Images
OS Version Support Mechanism
OS Image Tag Release Notes
Add-on Versions
CoreDNS Release History
CCE Container Storage (Everest) Release History
CCE Node Problem Detector Release History
Kubernetes Dashboard Release History
CCE Cluster Autoscaler Release History
NGINX Ingress Controller Release History
Kubernetes Metrics Server Release History
CCE Advanced HPA Release History
CCE Cloud Bursting Engine for CCI Release History
CCE AI Suite (NVIDIA GPU) Release History
CCE AI Suite (Ascend NPU) Release History
Volcano Scheduler Release History
CCE Secrets Manager for DEW Release History
CCE Network Metrics Exporter Release History
NodeLocal DNSCache Release History
Cloud Native Cluster Monitoring Release History
Cloud Native Log Collection Release History
Container Image Signature Verification Release History
Grafana Release History
OpenKruise Release History
Gatekeeper Release History
Kuberay Release History
Vertical Pod Autoscaler Release History
CCE Cluster Backup & Recovery (End of Maintenance) Release History
Kubernetes Web Terminal (End of Maintenance) Release History
Prometheus (End of Maintenance) Release History
Billing
Billing Overview
Billed Items
Billing Modes
Overview
Yearly/Monthly Billing
Pay-per-Use Billing
Spot Pricing
Billing Examples
Modifying Resource Specifications
Billing Mode Changes
Overview
Pay-per-Use to Yearly/Monthly
Yearly/Monthly to Pay-per-Use
Renewing Subscriptions
Overview
Manually Renewing a CCE Resource
Automatically Renewing a CCE Resource
Bills
Arrears
Billing Termination
Billing FAQ
Kubernetes Basics
Overview
Basic Concepts
Containers and Kubernetes
Containers
Kubernetes
Using kubectl to Operate a Cluster
Pods, Liveness Probes, Labels, and Namespaces
Pods
Liveness Probes
Labels
Namespaces
Pod Orchestration and Scheduling
Deployments
StatefulSets
Jobs and CronJobs
DaemonSets
Affinity and Anti-Affinity
Configuration Management
ConfigMaps
Secrets
Kubernetes Networks
Container Networks
Services
Ingresses
Readiness Probes
Network Policies
Persistent Storage
Volumes
PVs, PVCs, and Storage Classes
Authentication and Authorization
Service Accounts
RBAC
Auto Scaling
Best Practices
Checklist for Deploying Containerized Applications in the Cloud
Containerization
Containerizing an Enterprise Application (ERP)
Solution Overview
Resource and Cost Planning
Procedure
Containerizing an Entire Application
Containerization Process
Analyzing the Application
Preparing the Application Runtime
Compiling a Startup Script
Compiling the Dockerfile
Building and Uploading an Image
Creating a Container Workload
Migration
Migrating Container Images
Solution Overview
Migrating Images to SWR Using Docker Commands
Migrating Images to SWR Using image-migrator
Migrating Images Across Clouds from Harbor to SWR
Migrating Kubernetes Clusters to CCE
Solution Overview
Resource Planning for the Target Cluster
Procedure
Migrating Resources Outside a Cluster
Installing the Migration Tool
Migrating Resources in a Cluster (Velero)
Updating Resources Accordingly
Performing Additional Tasks
Troubleshooting
DevOps
Installing and Deploying Jenkins on CCE
Solution Overview
Resource and Cost Planning
Procedure
Installing and Deploying Jenkins Master
Configuring Jenkins Agent
Using Jenkins to Build a Pipeline
Interconnecting Jenkins with RBAC of Kubernetes Clusters (Example)
Interconnecting GitLab with SWR and CCE for CI/CD
Continuous Delivery Using Argo CD
Implementing Separate DevOps Processes for Multiple Clusters Using Jenkins and GitLab
Solution Overview
Resource Planning
Procedure
Setting Up the Jenkins and GitLab Environments
Configuring Cluster Environments
Configuring a GitLab Project
Implementing Continuous Integration and Deployment
Disaster Recovery
Recommended Configurations for HA CCE Clusters
Implementing High Availability for Applications in CCE
Implementing High Availability for Add-ons in CCE
Security
Overview
Configuration Suggestions on CCE Cluster Security
Configuration Suggestions on CCE Node Security
Configuration Suggestions on CCE Container Runtime Security
Configuration Suggestions on CCE Container Security
Configuration Suggestions on CCE Container Image Security
Configuration Suggestions on CCE Secret Security
Configuration Suggestions on CCE Workload Identity Security
Auto Scaling
Using HPA and CA for Auto Scaling of Workloads and Nodes
Elastic Scaling of CCE Pods to CCI
Auto Scaling Based on Prometheus Metrics
Auto Scaling Based on ELB Monitoring Metrics
Auto Scaling of Multiple Applications Using Nginx Ingresses
Monitoring
Monitoring Multiple Clusters Using Prometheus
Monitoring GPU Metrics Using DCGM-Exporter
Reporting Prometheus Monitoring Data to a Third-Party Monitoring Platform
Obtaining Prometheus Data Using PromQL Statements
Collecting GPU Pod Monitoring Metrics and Setting Up a Grafana Dashboard
Cluster
Suggestions on CCE Cluster Selection
Creating an IPv4/IPv6 Dual-Stack Cluster in CCE
Creating a Custom CCE Node Image
Executing the Pre- or Post-installation Commands During Node Creation
Using OBS Buckets to Implement Custom Script Injection During Node Creation
Connecting to Multiple Clusters Using kubectl
Selecting a Data Disk for the Node
Implementing Cost Visualization for a CCE Cluster
Creating a CCE Turbo Cluster Using a Shared VPC
Protecting a CCE Cluster Against Overload
Managing Costs for a Cluster
Networking
Planning CIDR Blocks for a Cluster
Selecting a Network Model
Enabling Cross-VPC Network Communications Between CCE Clusters
Implementing Network Communications Between Containers and IDCs Using VPC and Direct Connect
Enabling a CCE Cluster to Resolve Domain Names on Both On-Premises IDCs and Huawei Cloud
Solution Overview
Solution 1: Using a DNS Endpoint for Cascading Resolution
Solution 2: Changing the CoreDNS Configurations
Implementing Sticky Session Through Load Balancing
Obtaining the Client Source IP Address for a Container
Increasing the Listening Queue Length by Configuring Container Kernel Parameters
Configuring Passthrough Networking for a LoadBalancer Service
Accessing an External Network from a Pod
Accessing the Internet from a Pod
Accessing Cloud Services from a Pod in the Same VPC
Accessing Cloud Services from a Pod in a Different VPC
Deploying Nginx Ingress Controllers Using a Chart
Deploying NGINX Ingress Controller in Custom Mode
Advanced Configuration of Nginx Ingress Controller
CoreDNS Configuration Optimization
CoreDNS Optimization Overview
Client
Optimizing Domain Name Resolution Requests
Selecting a Proper Image
Avoiding Occasional DNS Resolution Timeout Caused by IPVS Defects
Using NodeLocal DNSCache
Upgrading the CoreDNS in the Cluster Timely
Adjusting the DNS Configuration of the VPC and VM
Server
Monitoring the coredns Add-on
Adjusting the CoreDNS Deployment Status
Configuring CoreDNS
Pre-Binding Container Elastic Network Interfaces for CCE Turbo Clusters
Connecting a Cluster to the Peer VPC Through an Enterprise Router
Accessing an IP Address Outside of a Cluster That Uses a VPC Network by Using Source Pod IP Addresses Within the Cluster
Using Kmesh on Huawei Cloud CCE
Storage
Expanding the Storage Space
Mounting Object Storage Across Accounts
Dynamically Creating an SFS Turbo Subdirectory Using StorageClass
Changing the Storage Class Used by a Cluster of v1.15 from FlexVolume to CSI Everest
Using Custom Storage Classes
Scheduling EVS Disks Across AZs Using csi-disk-topology
Automatically Collecting JVM Dump Files That Exit Unexpectedly Using a General Purpose File System (SFS 3.0 Capacity-Oriented)
Deploying Storage Volumes in Multiple AZs
Container
Recommended Configurations for Workloads
Properly Allocating Container Computing Resources
Upgrading Pods Without Interrupting Services
Modifying Kernel Parameters Using a Privileged Container
Using Init Containers to Initialize an Application
Setting Time Zone Synchronization
Configuration Suggestions on Container Network Bandwidth Limit
Configuring the /etc/hosts File of a Pod Using hostAliases
Configuring Domain Name Resolution for CCE Containers
Using Dual-Architecture Images (x86 and Arm) in CCE
Locating Container Faults Using the Core Dump File
Configuring Parameters to Delay the Pod Startup in a CCE Turbo Cluster
Automatically Updating a Workload Version Using SWR Triggers
Effective Troubleshooting in Kubernetes with Temporary Containers
Developing and Testing a Microservice Locally Using Telepresence
Containerizing and Using QingTian Enclave on CCE
Permission
Configuring kubeconfig for Fine-Grained Management on Cluster Resources
Configuring Namespace-level Permissions for an IAM User
Performing RBAC Authentication on a Namespace Using kubectl Commands
Release
Overview
Using Services to Implement Simple Grayscale Release and Blue-Green Deployment
Using Nginx Ingress to Implement Grayscale Release and Blue-Green Deployment
Batch Computing
Deploying and Using Kubeflow in a CCE Cluster
Deploying Kubeflow
Training a TensorFlow Model
Using Kubeflow and Volcano to Train an AI Model
Deploying and Using Caffe in a CCE Cluster
Prerequisites
Preparing Resources
Caffe Classification Example
Deploying and Using TensorFlow in a CCE Cluster
Deploying and Using Flink in a CCE Cluster
Deploying and Using ClickHouse in a CCE Cluster
Deploying and Using Spark in a CCE Cluster
Installing Spark
Using Spark
SDK Reference
SDK Overview
Videos
More Documents
User Guide (ME-Abu Dhabi Region)
Service Overview
What Is CCE?
Product Advantages
Application Scenarios
Containerized Application Management
Auto Scaling in Seconds
DevOps and CI/CD
Hybrid Cloud
Notes and Constraints
Permissions
Related Services
Regions and AZs
Product Bulletin
Kubernetes Version Policy
EOM of CentOS
Support for Docker
Getting Started
Introduction
Preparations
Creating a Kubernetes Cluster
Deploying a Deployment (Nginx)
Deploying WordPress and MySQL That Depend on Each Other
Overview
Step 1: Deploying MySQL
Step 2: Deploying WordPress
High-Risk Operations
Clusters
Cluster Overview
Cluster Version Release Notes
Kubernetes Version Release Notes
Kubernetes 1.30 Release Notes
Kubernetes 1.29 Release Notes
Kubernetes 1.28 Release Notes
Kubernetes 1.27 Release Notes
Kubernetes 1.25 Release Notes
Kubernetes 1.23 Release Notes
Kubernetes 1.21 (EOM) Release Notes
Kubernetes 1.19 (EOM) Release Notes
Kubernetes 1.17 (EOM) Release Notes
Patch Version Release Notes
Buying a Cluster
Comparison Between Cluster Types
Buying a CCE Standard/Turbo Cluster
Comparing iptables and IPVS
Connecting to a Cluster
Accessing a Cluster Using kubectl
Accessing a Cluster Using an X.509 Certificate
Accessing a Cluster Using a Custom Domain Name
Configuring a Cluster's API Server for Internet Access
Revoking a Cluster Access Credential
Managing a Cluster
Modifying Cluster Configurations
Enabling Overload Control for a Cluster
Changing a Cluster Scale
Changing the Default Security Group of a Node
Deleting a Cluster
Preventing Cluster Deletion
Hibernating or Waking Up a Cluster
Upgrading a Cluster
Cluster Upgrade Overview
Before You Start
Performing Post-Upgrade Verification
Cluster Status Check
Node Status Check
Node Skipping Check
Service Check
New Node Check
New Pod Check
Migrating Services Across Clusters of Different Versions
Troubleshooting for Pre-upgrade Check Exceptions
Pre-upgrade Check
Node Restrictions
Upgrade Management
Add-ons
Helm Charts
SSH Connectivity of Master Nodes
Node Pools
Security Groups
Arm Node Restrictions
Residual Nodes
Discarded Kubernetes Resources
Compatibility Risks
CCE Agent Versions
Node CPU Usage
CRDs
Node Disks
Node DNS
Node Key Directory File Permissions
kubelet
Node Memory
Node Clock Synchronization Server
Node OS
Node CPU Cores
Node Python Commands
ASM Version
Node Readiness
Node journald
containerd.sock
Internal Error
Node Mount Points
Kubernetes Node Taints
Everest Restrictions
cce-hpa-controller Limitations
Enhanced CPU Policies
Health of Worker Node Components
Health of Master Node Components
Memory Resource Limit of Kubernetes Components
Discarded Kubernetes APIs
IPv6 Support in CCE Turbo Clusters
NetworkManager
Node ID File
Node Configuration Consistency
Node Configuration File
CoreDNS Configuration Consistency
sudo
Key Node Commands
Mounting of a Sock File on a Node
HTTPS Load Balancer Certificate Consistency
Node Mounting
Login Permissions of User paas on a Node
Private IPv4 Addresses of Load Balancers
Historical Upgrade Records
CIDR Block of the Cluster Management Plane
CCE AI Suite (NVIDIA GPU)
Nodes' System Parameters
Residual Package Version Data
Node Commands
Node Swap
NGINX Ingress Controller
containerd Pod Restart Risks
Key CCE AI Suite (NVIDIA GPU) Parameters
GPU Pod Rebuild Risks
ELB Listener Access Control
Master Node Flavor
Subnet Quota of Master Nodes
Node Runtime
Node Pool Runtime
Number of Node Images
OpenKruise Compatibility Check
Compatibility Check of Secret Encryption
Compatibility Between the Ubuntu Kernel and GPU Driver
Drainage Tasks
Image Layers on a Node
Cluster Rolling Upgrade
Rotation Certificates
Ingress and ELB Configuration Consistency
Nodes
Node Overview
Container Engines
Node OSs
Creating a Node
Accepting Nodes for Management
Logging In to a Node
Management Nodes
Managing Node Labels
Managing Node Taints
Resetting a Node
Removing a Node
Synchronizing the Data of Cloud Servers
Draining a Node
Deleting a Node
Stopping a Node
Performing Rolling Upgrade for Nodes
Node O&M
Node Resource Reservation Policy
Space Allocation of a Data Disk
Maximum Number of Pods That Can Be Created on a Node
Differences in kubelet and Runtime Component Configurations Between CCE and the Native Community
Migrating Nodes from Docker to containerd
Configuring Node Fault Detection Policies
Executing the Pre- or Post-installation Commands During Node Creation
Node Pools
Node Pool Overview
Creating a Node Pool
Scaling a Node Pool
Managing a Node Pool
Updating a Node Pool
Updating an AS Configuration
Modifying Node Pool Configurations
Accepting Nodes in a Node Pool
Copying a Node Pool
Synchronizing Node Pools
Migrating a Node
Deleting a Node Pool
Workloads
Overview
Creating a Workload
Creating a Deployment
Creating a StatefulSet
Creating a DaemonSet
Creating a Job
Creating a CronJob
Configuring a Workload
Secure Runtime and Common Runtime
Configuring Time Zone Synchronization
Configuring an Image Pull Policy
Using Third-Party Images
Configuring Container Specifications
Configuring Container Lifecycle Parameters
Configuring Container Health Check
Configuring Environment Variables
Configuring APM
Configuring Workload Upgrade Policies
Configuring Tolerance Policies
Configuring Labels and Annotations
Scheduling a Workload
Overview
Configuring Specified Node Scheduling (nodeSelector)
Configuring Node Affinity Scheduling (nodeAffinity)
Configuring Workload Affinity or Anti-affinity Scheduling (podAffinity or podAntiAffinity)
Logging In to a Container
Managing Workloads
Managing Custom Resources
Pod Security
Configuring a Pod Security Policy
Configuring Pod Security Admission
Scheduling
Overview
CPU Scheduling
CPU Policy
Enhanced CPU Policy
GPU Scheduling
GPU Driver Version
Selecting a GPU Driver Version for Nodes
Recommended GPU Driver Versions for CCE
Manually Upgrading the Driver Version of a GPU Node
Upgrading the Driver Version of a GPU Node Using a Node Pool
Default GPU Scheduling in Kubernetes
GPU Metrics
Volcano Scheduling
Overview
Scheduling Workloads
Resource Usage-based Scheduling
Bin Packing
Descheduling
Node Pool Affinity
Priority-based Scheduling
Priority-based Scheduling
AI Performance-based Scheduling
DRF
Gang
NUMA Affinity Scheduling
Cloud Native Hybrid Deployment
Dynamic Resource Oversubscription
Network
Overview
Container Network
Overview
Cloud Native Network 2.0 Settings
Cloud Native Network 2.0
Configuring a Default Container Subnet for a CCE Turbo Cluster
Binding a Security Group to a Pod Using an Annotation
Binding a Security Group to a Workload Using a Security Group Policy
Binding a Subnet and Security Group to a Namespace or Workload Using a Container Network Configuration
Configuring an EIP for a Pod
VPC Network Settings
VPC Network Model
Adding a Container CIDR Block for a Cluster
Tunnel Network Settings
Tunnel Network Model
Pod Network Settings
Configuring hostNetwork for Pods
Configuring QoS for a Pod
Configuring Network Policies to Restrict Pod Access
Service
Overview
ClusterIP
NodePort
LoadBalancer
Creating a LoadBalancer Service
Configuring LoadBalancer Services Using Annotations
Configuring HTTP/HTTPS for a LoadBalancer Service
Configuring SNI for a LoadBalancer Service
Configuring HTTP/2 for a LoadBalancer Service
Configuring Timeout for a LoadBalancer Service
Configuring Health Check on Multiple LoadBalancer Service Ports
Configuring Passthrough Networking for a LoadBalancer Service
Changing a Custom EIP for a LoadBalancer Service
Setting the Pod Ready Status Through the ELB Health Check
Enabling ICMP Security Group Rules
DNAT
Headless Services
Ingresses
Overview
LoadBalancer Ingresses
Creating a LoadBalancer Ingress on the Console
Creating a LoadBalancer Ingress Using kubectl
Annotations for Configuring LoadBalancer Ingresses
Advanced Setting Examples of LoadBalancer Ingresses
Configuring an HTTPS Certificate for a LoadBalancer Ingress
Updating the HTTPS Certificate for a LoadBalancer Ingress
Configuring SNI for a LoadBalancer Ingress
Configuring Multiple Forwarding Policies for a LoadBalancer Ingress
Configuring HTTP/2 for a LoadBalancer Ingress
Configuring HTTPS Backend Services for a LoadBalancer Ingress
Configuring Timeout for a LoadBalancer Ingress
Configuring a Slow Start for a LoadBalancer Ingress
Configuring a Range of Listening Ports for a LoadBalancer Ingress
Configuring the Priorities of Forwarding Rules for LoadBalancer Ingresses
Configuring a Custom Header Forwarding Policy for a LoadBalancer Ingress
Configuring a Custom EIP for a LoadBalancer Ingress
Configuring Advanced Forwarding Rules for a LoadBalancer Ingress
Nginx Ingresses
Creating an Nginx Ingress on the Console
Creating an Nginx Ingress Using kubectl
Annotations for Configuring Nginx Ingresses
Advanced Setting Examples of Nginx Ingresses
Configuring an HTTPS Certificate for an Nginx Ingress
Configuring HTTPS Backend Services for an Nginx Ingress
Configuring Consistent Hashing for Load Balancing of an Nginx Ingress
Optimizing NGINX Ingress Controller in High-Traffic Scenarios
DNS
Overview
DNS Configuration
Using CoreDNS for Custom Domain Name Resolution
Using NodeLocal DNSCache to Improve DNS Performance
Configuring Intra-VPC Access
Accessing the Internet from a Container
Storage
Overview
Storage Basics
Elastic Volume Service
Overview
Using an Existing EVS Disk Through a Static PV
Using an EVS Disk Through a Dynamic PV
Dynamically Mounting an EVS Disk to a StatefulSet
Encrypting EVS Disks
Expanding the Capacity of an EVS Disk
Snapshots and Backups
SFS Turbo
Overview
Using an Existing SFS Turbo File System Through a Static PV
Configuring SFS Turbo Mount Options
(Recommended) Creating an SFS Turbo Subdirectory Using a Dynamic PV
Dynamically Creating an SFS Turbo Subdirectory Using StorageClass
Object Storage Service
Overview
Using an Existing OBS Bucket Through a Static PV
Using an OBS Bucket Through a Dynamic PV
Configuring OBS Mount Options
Using a Custom Access Key (AK/SK) to Mount an OBS Volume
DSS
Overview
Using DSS Through a Static PV
Using DSS Through a Dynamic PV
Dynamically Mounting a DSS Disk to a StatefulSet
Local PVs
Overview
Importing a PV to a Storage Pool
Using a Local PV Through a Dynamic PV
Dynamically Mounting a Local PV to a StatefulSet
Expanding a Local PV
Ephemeral Volumes
Overview
Importing an EV to a Storage Pool
Local EV
Temporary Path
hostPath
StorageClass
Auto Scaling
Overview
Scaling a Workload
Workload Scaling Rules
Creating an HPA Policy
Creating a Scheduled CronHPA Policy
Creating a CustomedHPA Policy
Creating a VPA Policy
Managing Workload Scaling Policies
Scaling a Node
Node Scaling Rules
Priorities for Scaling Node Pools
Creating a Node Scaling Policy
Managing Node Scaling Policies
Using HPA and CA for Auto Scaling of Workloads and Nodes
O&M
Logging
Overview
Collecting Container Logs
Collecting Container Logs Using ICAgent
Log Auditing
CCE Operations Supported by Cloud Trace Service
Viewing CTS Traces in the Trace List
O&M Best Practices
Monitoring Custom Metrics Using Prometheus
Monitoring Custom Metrics on AOM
Namespaces
Creating a Namespace
Managing Namespaces
Configuring Resource Quotas
ConfigMaps and Secrets
Creating a ConfigMap
Using a ConfigMap
Creating a Secret
Using a Secret
Cluster Secrets
Add-ons
Overview
Scheduling and Elasticity Add-ons
Volcano Scheduler
CCE Cluster Autoscaler
CCE Advanced HPA
Vertical Pod Autoscaler
Cloud Native Observability Add-ons
CCE Node Problem Detector
CCE Network Metrics Exporter
Kubernetes Metrics Server
Prometheus
Cloud Native Heterogeneous Computing Add-ons
CCE AI Suite (NVIDIA GPU)
Container Network Add-ons
CoreDNS
NGINX Ingress Controller
NodeLocal DNSCache
Container Storage Add-ons
CCE Container Storage (Everest)
Container Security Add-ons
CCE Secrets Manager for DEW
Other Add-ons
Kubernetes Dashboard
OpenKruise
Gatekeeper
Helm Chart
Overview of a Chart
Deploying an Application from a Chart
Differences Between Helm v2 and Helm v3 and Adaptation Solutions
Deploying an Application Through the Helm v2 Client
Deploying an Application Through the Helm v3 Client
Converting a Release from Helm v2 to v3
Permissions
Permissions Overview
Granting Cluster Permissions to an IAM User
Namespace Permissions (Kubernetes RBAC-based)
Example: Designing and Configuring Permissions for Users in a Department
Permission Dependency of the CCE Console
Service Account Token Security Improvement
System Agencies
Settings
Dashboard
Cluster Access
Network
Scheduling
Auto Scaling
Monitoring
Kubernetes
Heterogeneous Resources
Best Practices
Checklist for Deploying Containerized Applications in the Cloud
Containerization
Containerizing an Enterprise Application (ERP)
Solution Overview
Procedure
Containerizing an Entire Application
Containerization Process
Analyzing the Application
Preparing the Application Runtime
Compiling a Startup Script
Compiling the Dockerfile
Building and Uploading an Image
Creating a Container Workload
Disaster Recovery
Recommended Configurations for HA CCE Clusters
Implementing High Availability for Applications in CCE
Implementing High Availability for Add-ons in CCE
Security
Configuration Suggestions on CCE Cluster Security
Configuration Suggestions on CCE Node Security
Configuration Suggestions on CCE Container Runtime Security
Configuration Suggestions on CCE Container Security
Configuration Suggestions on CCE Container Image Security
Configuration Suggestions on CCE Secret Security
Configuration Suggestions on CCE Workload Identity Security
Auto Scaling
Using HPA and CA for Auto Scaling of Workloads and Nodes
Monitoring
Monitoring Multiple Clusters Using Prometheus
Cluster
Suggestions on CCE Cluster Selection
Creating a Custom CCE Node Image
Executing the Pre- or Post-installation Commands During Node Creation
Connecting to Multiple Clusters Using kubectl
Selecting a Data Disk for the Node
Protecting a CCE Cluster Against Overload
Networking
Planning CIDR Blocks for a Cluster
Selecting a Network Model
Implementing Sticky Session Through Load Balancing
Obtaining the Client Source IP Address for a Container
Deploying Nginx Ingress Controllers Using a Chart
Deploying NGINX Ingress Controller in Custom Mode
Advanced Configuration of Nginx Ingress Controller
CoreDNS Configuration Optimization
CoreDNS Optimization Overview
Client
Optimizing Domain Name Resolution Requests
Selecting a Proper Image
Avoiding Occasional DNS Resolution Timeout Caused by IPVS Defects
Using NodeLocal DNSCache
Upgrading the CoreDNS in the Cluster Timely
Adjusting the DNS Configuration of the VPC and VM
Server
Monitoring the coredns Add-on
Adjusting the CoreDNS Deployment Status
Configuring CoreDNS
Pre-Binding Container ENI for CCE Turbo Clusters
Accessing an IP Address Outside of a Cluster That Uses a VPC Network by Using Source Pod IP Addresses Within the Cluster
Storage
Expanding the Storage Space
Mounting Object Storage Across Accounts
Dynamically Creating an SFS Turbo Subdirectory Using StorageClass
Using Custom Storage Classes
Scheduling EVS Disks Across AZs Using csi-disk-topology
Container
Properly Allocating Container Computing Resources
Modifying Kernel Parameters Using a Privileged Container
Using Init Containers to Initialize an Application
Configuring the /etc/hosts File of a Pod Using hostAliases
Locating Container Faults Using the Core Dump File
Permission
Configuring kubeconfig for Fine-Grained Management on Cluster Resources
Release
Overview
Using Services to Implement Simple Grayscale Release and Blue-Green Deployment
Using Nginx Ingress to Implement Grayscale Release and Blue-Green Deployment
FAQs
Common FAQ
Billing
How Is CCE Billed?
Cluster
Cluster Creation
Why Cannot I Create a CCE Cluster?
Is Management Scale of a Cluster Related to the Number of Master Nodes?
Which Resource Quotas Should I Pay Attention To When Using CCE?
Cluster Running
How Do I Locate the Fault When a Cluster Is Unavailable?
How Do I Retrieve Data After a CCE Cluster Is Deleted?
Cluster Deletion
What Can I Do If a Cluster Deletion Fails Due to Residual Resources in the Security Group?
How Do I Clear Residual Resources After Deleting a Non-Running Cluster?
Cluster Upgrade
What Do I Do If a Cluster Add-On Fails to be Upgraded During the CCE Cluster Upgrade?
Node
Node Creation
How Do I Troubleshoot Problems Occurred When Adding Nodes to a CCE Cluster?
How Do I Troubleshoot Problems Occurred When Accepting Nodes into a CCE Cluster?
What Should I Do If a Node Cannot Be Managed and an Error Message Appears Saying That the Node Failed to Install?
Node Running
What Should I Do If a Cluster Is Available But Some Nodes in It Are Unavailable?
How Do I Log In to a Node Using a Password and Reset the Password?
How Do I Collect Logs of Nodes in a CCE Cluster?
What Should I Do If the vdb Disk of a Node Is Damaged and the Node Cannot Be Recovered After Reset?
What Should I Do If I/O Suspension Occasionally Occurs When SCSI EVS Disks Are Used?
How Do I Fix an Abnormal Container or Node Due to No Thin Pool Disk Space?
How Do I Rectify Failures When the NVIDIA Driver Is Used to Start Containers on GPU Nodes?
Specification Change
How Do I Change the Node Specifications in a CCE Cluster?
What Are the Impacts of Changing the Flavor of a Node in a CCE Node Pool?
What Should I Do If I Fail to Restart or Create Workloads on a Node After Modifying the Node Specifications?
OSs
What Should I Do If There Is a Service Access Failure After a Backend Service Upgrade or a 1-Second Latency When a Service Accesses a CCE Cluster?
Node Pool
What Should I Do If a Node Pool Is Abnormal?
What Should I Do If No Node Creation Record Is Displayed When the Node Pool Is Being Expanding?
What Should I Do If a Node Pool Scale-Out Fails?
How Do I Modify ECS Configurations When an ECS Cannot Be Managed by a Node Pool?
Workload
Workload Exception Troubleshooting
How Can I Locate the Root Cause If a Workload Is Abnormal?
What Should I Do If the Scheduling of a Pod Fails?
What Should I Do If a Pod Fails to Pull the Image?
What Should I Do If Container Startup Fails?
What Should I Do If a Pod Fails to Be Evicted?
What Should I Do If a Storage Volume Cannot Be Mounted or the Mounting Times Out?
What Should I Do If a Workload Remains in the Creating State?
What Should I Do If a Pod Remains in the Terminating State?
What Should I Do If a Workload Is Stopped Caused by Pod Deletion?
What Should I Do If an Error Occurs When I Deploy a Service on a GPU Node?
How Can I Locate Faults Using an Exit Code?
Container Configuration
When Is Pre-stop Processing Used?
How Do I Set an FQDN for Accessing a Specified Container in the Same Namespace?
What Should I Do If Health Check Probes Occasionally Fail?
How Do I Set the umask Value for a Container?
What Is the Retry Mechanism When CCE Fails to Start a Pod?
Scheduling Policies
How Do I Evenly Distribute Multiple Pods to Each Node?
How Do I Prevent a Container on a Node from Being Evicted?
Why Are Pods Not Evenly Distributed on Nodes?
How Do I Evict All Pods on a Node?
Why Cannot a Pod Be Scheduled to a Node?
How Do I Troubleshoot a Pod Exit Caused by a Node Label Update?
Others
What Should I Do If a Cron Job Cannot Be Restarted After Being Stopped for a Period of Time?
What Is a Headless Service When I Create a StatefulSet?
What Should I Do If Error Message "Auth is empty" Is Displayed When a Private Image Is Pulled?
What Is the Image Pull Policy for Containers in a CCE Cluster?
What Can I Do If a Layer Is Missing During Image Pull?
Networking
Network Exception Troubleshooting
How Do I Locate a Workload Networking Fault?
Why Does the Browser Return Error Code 404 When I Access a Deployed Application?
What Should I Do If a Container Fails to Access the Internet?
What Should I Do If a Node Fails to Connect to the Internet (Public Network)?
What Should I Do If Nginx Ingress Access in the Cluster Is Abnormal After the NGINX Ingress Controller Add-on Is Upgraded?
What Could Cause Access Exceptions After Configuring an HTTPS Certificate for a LoadBalancer Ingress?
Network Planning
What Is the Relationship Between Clusters, VPCs, and Subnets?
How Can I Configure a Security Group Rule for a Cluster?
Security Hardening
How Do I Prevent Cluster Nodes from Being Exposed to Public Networks?
How Do I Configure an Access Policy for a Cluster?
How Do I Obtain a TLS Key Certificate?
How Do I Change the Security Group of Nodes in a Cluster in Batches?
Network Configuration
How Can Container IP Addresses Survive a Container Restart?
How Can I Check Whether an ENI Is Used by a Cluster?
How Can I Delete a Security Group Rule Associated with a Deleted Subnet?
How Can I Determine Which Ingress the Listener Settings Have Been Applied To?
Storage
How Do I Expand the Storage Capacity of a Container?
What Are the Differences Among CCE Storage Classes in Terms of Persistent Storage and Multi-Node Mounting?
Can I Create a CCE Node Without Adding a Data Disk to the Node?
What Should I Do If the Host Cannot Be Found When Files Need to Be Uploaded to OBS During the Access to the CCE Service from a Public Network?
How Can I Achieve Compatibility Between ExtendPathMode and Kubernetes client-go?
Can CCE PVCs Detect Underlying Storage Faults?
Why Cannot I Delete a PV or PVC Using the kubectl delete Command?
Namespace
What Should I Do If a Namespace Fails to Be Deleted Due to an APIService Object Access Failure?
Chart and Add-on
How Can I Troubleshoot Exceptions That Occur with an Add-on?
What Should I Do If Installation of an Add-on Fails and "The release name is already exist" Is Displayed?
How Do I Configure the Add-on Resource Quotas Based on Cluster Scale?
How Can I Clean Up Residual Resources After the NGINX Ingress Controller Add-on in the Unknown State Is Deleted?
Why TLS v1.0 or v1.1 Cannot Be Used After the NGINX Ingress Controller Add-on Is Upgraded?
API & kubectl FAQs
How Can I Access a Cluster API Server?
Can the Resources Created Using APIs or kubectl Be Displayed on the CCE Console?
How Do I Download kubeconfig for Connecting to a Cluster Using kubectl?
How Do I Rectify the Error Reported When Running the kubectl top node Command?
Why Is "Error from server (Forbidden)" Displayed When I Use kubectl?
DNS FAQs
What Should I Do If Domain Name Resolution Fails in a CCE Cluster?
Why Does a Container in a CCE Cluster Fail to Perform DNS Resolution?
How Do I Optimize the Configuration If the External Domain Name Resolution Is Slow or Times Out?
How Do I Configure a DNS Policy for a Container?
Image Repository FAQs
How Do I Upload My Images to CCE?
Permissions
Can I Configure Only Namespace Permissions Without Cluster Management Permissions?
Can I Use CCE APIs If the Cluster Management Permissions Are Not Configured?
Can I Use kubectl If the Cluster Management Permissions Are Not Configured?
API Reference (ME-Abu Dhabi Region)
Before You Start
API Overview
Calling APIs
Making an API Request
Authentication
Response
APIs
API URL
Cluster Management
Creating a Cluster
Reading a Specified Cluster
Listing Clusters in a Specified Project
Updating a Specified Cluster
Deleting a Cluster
Hibernating a Cluster
Waking Up a Cluster
Obtaining a Cluster Certificate
Modifying Cluster Specifications
Querying a Job
Binding/Unbinding Public API Server Address
Obtaining Cluster Access Address
Node Management
Creating a Node
Reading a Specified Node
Listing All Nodes in a Cluster
Updating a Specified Node
Deleting a Node
Enabling Scale-In Protection for a Node
Disabling Scale-In Protection for a Node
Synchronizing Nodes
Accepting a Node
Managing a Node in a Customized Node Pool
Resetting a Node
Removing a Node
Migrating a Node
Node Pool Management
Creating a Node Pool
Reading a Specified Node Pool
Listing All Node Pools in a Specified Cluster
Updating a Specified Node Pool
Deleting a Node Pool
Scaling a Node Pool
Storage Management
Creating a PVC (to be discarded)
Deleting a PVC (to be discarded)
Add-on Management
Installing an Add-on Instance
Listing Add-on Templates
Updating an Add-on Instance
Rolling Back an Add-on Instance
Deleting an Add-on Instance
Querying an Add-on Instance
Listing Add-on Instances
Quota Management
Querying Resource Quotas
API Versions
Obtaining API Versions
Tag Management
Adding Resource Tags to a Specified Cluster in Batches
Deleting Resource Tags of a Specified Cluster in Batches
Configuration Management
Obtaining the Parameters That Can Be Configured for a Node Pool
Obtaining the List of Parameters That Can Be Configured for a Cluster
Obtaining the Parameters That Can Be Configured for a Node Pool
Changing the Values of Configuration Parameters of a Node Pool
Chart Management
Uploading a Chart
Obtaining a Chart List
Obtaining a Release List
Updating a Chart
Creating a Release
Deleting a Chart
Updating a Release
Obtaining a Chart
Deleting a Release
Downloading a Chart
Obtaining a Release
Obtaining Chart Values
Obtaining Historical Records of a Release
Obtaining the Quota of a User Chart
Add-on Instance Parameters
CoreDNS
CCE Container Storage (Everest)
CCE Node Problem Detector
Kubernetes Dashboard
CCE Cluster Autoscaler
NGINX Ingress Controller
Kubernetes Metrics Server
CCE Advanced HPA
CCE AI Suite (NVIDIA GPU)
Volcano Scheduler
CCE Secrets Manager for DEW
CCE Network Metrics Exporter
NodeLocal DNSCache
Kubernetes APIs
Permissions and Supported Actions
Appendix
Status Code
Error Codes
Obtaining a Project ID
Obtaining an Account ID
Specifying Add-ons to Be Installed During Cluster Creation
How to Obtain Parameters in the API URI
Creating a VPC and Subnet
Creating a Key Pair
Node Flavor Description
Adding a Salt in the password Field When Creating a Node
Maximum Number of Pods That Can Be Created on a Node
Node OS
Space Allocation of a Data Disk
Attaching Disks to a Node
User Guide (Paris Regions)
Service Overview
What Is CCE?
Product Advantages
Application Scenarios
Containerized Application Management
Auto Scaling in Seconds
DevOps and CI/CD
Hybrid Cloud
Notes and Constraints
Billing
Permissions
Related Services
Regions and AZs
Product Bulletin
Risky Operations on Cluster Nodes
CCE Security Guide
Cluster Node OS Patch Notes
Vulnerability Notice
Notice on the Kubernetes Security Vulnerability (CVE-2022-3172)
Privilege Escalation Vulnerability in Linux openvswitch Kernel Module (CVE-2022-2639)
Notice on CRI-O Container Runtime Engine Arbitrary Code Execution Vulnerability (CVE-2022-0811)
Notice on the Container Escape Vulnerability Caused by the Linux Kernel (CVE-2022-0492)
Linux Kernel Integer Overflow Vulnerability (CVE-2022-0185)
CCE Console Upgrade
Kubernetes Basics
Overview
Container and Kubernetes
Containers
Kubernetes
Pod, Label, and Namespace
Pod: the Smallest Scheduling Unit in Kubernetes
Liveness Probe
Label for Managing Pods
Namespaces: Grouping Resources
Pod Orchestration and Scheduling
Deployment
StatefulSet
Job and Cron Job
DaemonSet
Affinity and Anti-Affinity Scheduling
Configuration Management
ConfigMap
Secret
Kubernetes Networking
Container Networking
Services
Ingresses
Readiness Probe
NetworkPolicy
Persistent Storage
Volume
PersistentVolumes, PersistentVolumeClaims, and StorageClasses
Authentication and Authorization
ServiceAccounts
RBAC
Auto Scaling
Getting Started
Introduction
Preparations
Creating a Kubernetes Cluster
Creating a Deployment (Nginx)
Deploying WordPress and MySQL That Depend on Each Other
Overview
Creating a MySQL Workload
Creating a WordPress Workload
High-Risk Operations
Clusters
Cluster Overview
Basic Cluster Information
Kubernetes Version Release Notes
Kubernetes 1.29 Release Notes
Kubernetes 1.28 Release Notes
Kubernetes 1.27 Release Notes
Kubernetes 1.25 Release Notes
Kubernetes 1.23 Release Notes
Kubernetes 1.21 (EOM) Release Notes
Kubernetes 1.19 (EOM) Release Notes
Kubernetes 1.17 (EOM) Release Notes
Patch Version Release Notes
Creating a Cluster
Comparison Between Cluster Types
Creating a CCE Standard Cluster
Comparing iptables and IPVS
Connecting to a Cluster
Connecting to a Cluster Using kubectl
Accessing a Cluster Using an X.509 Certificate
Accessing a Cluster Using a Custom Domain Name
Configuring a Cluster's API Server for Internet Access
Managing a Cluster
Modifying Cluster Configurations
Enabling Overload Control for a Cluster
Changing Cluster Scale
Deleting a Cluster
Hibernating or Waking Up a Cluster
Upgrading a Cluster
Process and Method of Upgrading a Cluster
Before You Start
Performing Post-Upgrade Verification
Cluster Status Check
Node Status Check
Node Skipping Check
Service Check
New Node Check
New Pod Check
Migrating Services Across Clusters of Different Versions
Troubleshooting for Pre-upgrade Check Exceptions
Pre-upgrade Check
Node Restrictions
Upgrade Management
Add-ons
Helm Charts
SSH Connectivity of Master Nodes
Node Pools
Security Groups
Arm Node Restrictions
Residual Nodes
Discarded Kubernetes Resources
Compatibility Risks
CCE Agent Versions
Node CPU Usage
CRDs
Node Disks
Node DNS
Node Key Directory File Permissions
kubelet
Node Memory
Node Clock Synchronization Server
Node OS
Node CPU Cores
Node Python Commands
ASM Version
Node Readiness
Node journald
containerd.sock
Internal Error
Node Mount Points
Kubernetes Node Taints
Everest Restrictions
cce-hpa-controller Limitations
Enhanced CPU Policies
Health of Worker Node Components
Health of Master Node Components
Memory Resource Limit of Kubernetes Components
Discarded Kubernetes APIs
IPv6 Support in CCE Turbo Clusters
NetworkManager
Node ID File
Node Configuration Consistency
Node Configuration File
CoreDNS Configuration Consistency
sudo
Key Node Commands
Mounting of a Sock File on a Node
HTTPS Load Balancer Certificate Consistency
Node Mounting
Login Permissions of User paas on a Node
Private IPv4 Addresses of Load Balancers
Historical Upgrade Records
CIDR Block of the Cluster Management Plane
GPU Add-on
Nodes' System Parameters
Residual Package Version Data
Node Commands
Node Swap
nginx-ingress Upgrade
containerd Pod Restart Risks
Key GPU Add-on Parameters
GPU Pod Rebuild Risks
ELB Listener Access Control
Master Node Flavor
Subnet Quota of Master Nodes
Node Runtime
Node Pool Runtime
Number of Node Images
Nodes
Node Overview
Container Engines
Creating a Node
Accepting Nodes for Management
Logging In to a Node
Management Nodes
Managing Node Labels
Managing Node Taints
Resetting a Node
Removing a Node
Synchronizing the Data of Cloud Servers
Draining a Node
Deleting a Node
Stopping a Node
Performing Rolling Upgrade for Nodes
Node O&M
Node Resource Reservation Policy
Data Disk Space Allocation
Maximum Number of Pods That Can Be Created on a Node
Differences Between CCE Node mountPath Configurations and Community Native Configurations
Migrating Nodes from Docker to containerd
Configuring Node Fault Detection Policies
Node Pools
Node Pool Overview
Creating a Node Pool
Scaling a Node Pool
Managing a Node Pool
Updating a Node Pool
Updating an AS Configuration
Modifying Node Pool Configurations
Accepting Nodes in a Node Pool
Copying a Node Pool
Synchronizing Node Pools
Upgrading an OS
Migrating a Node
Deleting a Node Pool
Workloads
Overview
Creating a Workload
Creating a Deployment
Creating a StatefulSet
Creating a DaemonSet
Creating a Job
Creating a Cron Job
Configuring a Workload
Configuring Time Zone Synchronization
Configuring an Image Pull Policy
Using Third-Party Images
Configuring Container Specifications
Configuring Container Lifecycle Parameters
Configuring Container Health Check
Configuring Environment Variables
Configuring Workload Upgrade Policies
Scheduling Policies (Affinity/Anti-affinity)
Configuring Tolerance Policies
Configuring Labels and Annotations
Logging In to a Container
Managing Workloads
Managing Custom Resources
Pod Security
Configuring a Pod Security Policy
Configuring Pod Security Admission
Scheduling
Overview
CPU Scheduling
CPU Policy
GPU Scheduling
Default GPU Scheduling in Kubernetes
Volcano Scheduling
Overview
Scheduling Workloads
Resource Usage-based Scheduling
Bin Packing
Descheduling
Node Pool Affinity
Priority-based Scheduling
Priority-based Scheduling
AI Performance-based Scheduling
DRF
Gang
NUMA Affinity Scheduling
Cloud Native Hybrid Deployment
Dynamic Resource Oversubscription
Network
Overview
Container Network
Overview
VPC Network Settings
VPC Network Model
Adding a Container CIDR Block for a Cluster
Tunnel Network Settings
Tunnel Network Model
Configuring Network Policies to Restrict Pod Access
Pod Network Settings
Configuring hostNetwork for Pods
Configuring QoS for a Pod
Service
Overview
ClusterIP
NodePort
LoadBalancer
Creating a LoadBalancer Service
Using Annotations to Balance Load
Configuring HTTP/HTTPS for a LoadBalancer Service
Configuring SNI for a LoadBalancer Service
Configuring HTTP/2 for a LoadBalancer Service
Configuring Timeout for a LoadBalancer Service
Configuring a Blocklist/Trustlist Access Policy for a LoadBalancer Service
Configuring Health Check on Multiple Ports of a LoadBalancer Service
Configuring Passthrough Networking for a LoadBalancer Service
Enabling ICMP Security Group Rules
Headless Services
Ingresses
Overview
LoadBalancer Ingresses
Creating a LoadBalancer Ingress on the Console
Using kubectl to Create a LoadBalancer Ingress
Configuring a LoadBalancer Ingress Using Annotations
Configuring an HTTPS Certificate for a LoadBalancer Ingress
Configuring SNI for a LoadBalancer Ingress
Routing a LoadBalancer Ingress to Multiple Services
Configuring HTTP/2 for a LoadBalancer Ingress
Configuring HTTPS Backend Services for a LoadBalancer Ingress
Configuring Timeout for a LoadBalancer Ingress
Configuring a Blocklist/Trustlist Access Policy for a LoadBalancer Ingress
Configuring a Custom Header Forwarding Policy for a LoadBalancer Ingress
Nginx Ingresses
Creating Nginx Ingresses on the Console
Using kubectl to Create an Nginx Ingress
Configuring Nginx Ingresses Using Annotations
Configuring an HTTPS Certificate for an Nginx Ingress
Configuring Redirection Rules for an Nginx Ingress
Configuring URL Rewriting Rules for an Nginx Ingress
Configuring HTTPS Backend Services for an Nginx Ingress
Configuring Consistent Hashing for Load Balancing of an Nginx Ingress
DNS
Overview
DNS Configuration
Using CoreDNS for Custom Domain Name Resolution
Using NodeLocal DNSCache to Improve DNS Performance
Configuring Intra-VPC Access
Accessing the Internet from a Container
Storage
Overview
Storage Basics
Elastic Volume Service
Overview
Using an Existing EVS Disk Through a Static PV
Using an EVS Disk Through a Dynamic PV
Dynamically Mounting an EVS Disk to a StatefulSet
Encrypting EVS Disks
Snapshots and Backups
Scalable File Service
Overview
Using an Existing SFS File System Through a Static PV
Using an SFS File System Through a Dynamic PV
Configuring SFS Volume Mount Options
SFS Turbo
Overview
Using an Existing SFS Turbo File System Through a Static PV
Configuring SFS Turbo Mount Options
(Recommended) Creating an SFS Turbo Subdirectory Using a Dynamic PV
Dynamically Creating an SFS Turbo Subdirectory Using StorageClass
Object Storage Service
Overview
Using an Existing OBS Bucket Through a Static PV
Using an OBS Bucket Through a Dynamic PV
Configuring OBS Mount Options
Using a Custom Access Key (AK/SK) to Mount an OBS Volume
DSS
Overview
Using DSS Through a Static PV
Using DSS Through a Dynamic PV
Dynamically Mounting a DSS Disk to a StatefulSet
Local PVs
Overview
Importing a PV to a Storage Pool
Using a Local PV Through a Dynamic PV
Dynamically Mounting a Local PV to a StatefulSet
Ephemeral Volumes
Overview
Importing an EV to a Storage Pool
Using a Local EV
Using a Temporary Path
hostPath
StorageClass
Observability
Logging
Overview
Collecting Container Logs
Collecting Container Logs Using ICAgent
Log Auditing
CCE Operations Supported by Cloud Trace Service
Viewing CTS Traces in the Trace List
Best Practices
Monitoring Custom Metrics on AOM
Auto Scaling
Overview
Scaling a Workload
Workload Scaling Rules
Creating an HPA Policy
Creating a Scheduled CronHPA Policy
Managing Workload Scaling Policies
Scaling a Node
Node Scaling Rules
Priorities for Scaling Node Pools
Creating a Node Scaling Policy
Managing Node Scaling Policies
Using HPA and CA for Auto Scaling of Workloads and Nodes
Namespaces
Creating a Namespace
Managing Namespaces
Configuring Resource Quotas
ConfigMaps and Secrets
Creating a ConfigMap
Using a ConfigMap
Creating a Secret
Using a Secret
Cluster Secrets
Add-ons
Overview
Scheduling and Elasticity Add-ons
Volcano Scheduler
CCE Cluster Autoscaler
CCE Advanced HPA
Cloud Native Observability Add-ons
CCE Node Problem Detector
CCE Network Metrics Exporter
Kubernetes Metrics Server
Cloud Native Heterogeneous Computing Add-ons
CCE AI Suite (NVIDIA GPU)
CCE AI Suite (Ascend NPU)
Container Network Add-ons
CoreDNS
NGINX Ingress Controller
NodeLocal DNSCache
Container Storage Add-ons
CCE Container Storage (Everest)
FlexVolume (Discarded)
Container Security Add-ons
CCE Secrets Manager for DEW
Other Add-ons
Kubernetes Dashboard
Helm Chart
Overview
Deploying an Application from a Chart
Differences Between Helm v2 and Helm v3 and Adaptation Solutions
Deploying an Application Through the Helm v2 Client
Deploying an Application Through the Helm v3 Client
Converting a Release from Helm v2 to v3
Permissions
Permissions Overview
Granting Cluster Permissions to an IAM User
Namespace Permissions (Kubernetes RBAC-based)
Example: Designing and Configuring Permissions for Users in a Department
Permission Dependency of the CCE Console
Service Account Token Security Improvement
Settings
Dashboard
Cluster Access
Network
Scheduling
Auto Scaling
Kubernetes
Heterogeneous Resources
FAQs
Common Questions
Billing
How Is CCE Billed/Charged?
Will I Be Notified When My Balance Is Insufficient?
Will I Be Notified When My Account Balance Changes?
Cluster
Cluster Creation
Why Cannot I Create a CCE Cluster?
Is Management Scale of a Cluster Related to the Number of Master Nodes?
Which Resource Quotas Should I Pay Attention To When Using CCE?
Cluster Running
How Do I Locate the Fault When a Cluster Is Unavailable?
How Do I Retrieve Data After a CCE Cluster Is Deleted?
Cluster Deletion
What Can I Do If a Cluster Deletion Fails Due to Residual Resources in the Security Group?
How Do I Clear Residual Resources After Deleting a Non-Running Cluster?
Cluster Upgrade
What Do I Do If a Cluster Add-On Fails to be Upgraded During the CCE Cluster Upgrade?
Node
Node Creation
How Do I Troubleshoot Problems Occurred When Adding Nodes to a CCE Cluster?
How Do I Troubleshoot Problems Occurred When Accepting Nodes into a CCE Cluster?
What Should I Do If a Node Fails to Be Accepted Because It Fails to Be Installed?
Node Running
What Should I Do If a Cluster Is Available But Some Nodes Are Unavailable?
How Do I Log In to a Node Using a Password and Reset the Password?
How Do I Collect Logs of Nodes in a CCE Cluster?
What Should I Do If the vdb Disk of a Node Is Damaged and the Node Cannot Be Recovered After Reset?
What Should I Do If I/O Suspension Occasionally Occurs When SCSI EVS Disks Are Used?
How Do I Fix an Abnormal Container or Node Due to No Thin Pool Disk Space?
How Do I Rectify Failures When the NVIDIA Driver Is Used to Start Containers on GPU Nodes?
Specification Change
How Do I Change the Node Specifications in a CCE Cluster?
What Are the Impacts of Changing the Flavor of a Node in a CCE Node Pool?
What Should I Do If I Fail to Restart or Create Workloads on a Node After Modifying the Node Specifications?
OSs
What Should I Do If There Is a Service Access Failure After a Backend Service Upgrade or a 1-Second Latency When a Service Accesses a CCE Cluster?
Node Pool
What Should I Do If a Node Pool Is Abnormal?
What Should I Do If No Node Creation Record Is Displayed When the Node Pool Is Being Expanding?
What Should I Do If a Node Pool Scale-Out Fails?
How Do I Modify ECS Configurations When an ECS Cannot Be Managed by a Node Pool?
Workload
Workload Abnormalities
How Do I Use Events to Fix Abnormal Workloads?
What Should I Do If Pod Scheduling Fails?
What Should I Do If a Pod Fails to Pull the Image?
What Should I Do If Container Startup Fails?
What Should I Do If a Pod Fails to Be Evicted?
What Should I Do If a Storage Volume Cannot Be Mounted or the Mounting Times Out?
What Should I Do If a Workload Remains in the Creating State?
What Should I Do If Pods in the Terminating State Cannot Be Deleted?
What Should I Do If a Workload Is Stopped Caused by Pod Deletion?
What Should I Do If an Error Occurs When Deploying a Service on the GPU Node?
Container Configuration
When Is Pre-stop Processing Used?
How Do I Set an FQDN for Accessing a Specified Container in the Same Namespace?
What Should I Do If Health Check Probes Occasionally Fail?
How Do I Set the umask Value for a Container?
What Is the Retry Mechanism When CCE Fails to Start a Pod?
Scheduling Policies
How Do I Evenly Distribute Multiple Pods to Each Node?
How Do I Prevent a Container on a Node from Being Evicted?
Why Are Pods Not Evenly Distributed on Nodes?
How Do I Evict All Pods on a Node?
Why Cannot a Pod Be Scheduled to a Node?
Others
What Should I Do If a Scheduled Task Cannot Be Restarted After Being Stopped for a Period of Time?
What Is a Headless Service When I Create a StatefulSet?
What Should I Do If Error Message "Auth is empty" Is Displayed When a Private Image Is Pulled?
What Is the Image Pull Policy for Containers in a CCE Cluster?
What Can I Do If a Layer Is Missing During Image Pull?
Networking
Network Planning
What Is the Relationship Between Clusters, VPCs, and Subnets?
How Can I Configure a Security Group Rule in a Cluster?
Network Fault
How Do I Locate a Workload Networking Fault?
Why Does the Browser Return Error Code 404 When I Access a Deployed Application?
What Should I Do If a Container Fails to Access the Internet?
What Should I Do If a Node Fails to Connect to the Internet (Public Network)?
What Should I Do If an Nginx Ingress Access in the Cluster Is Abnormal After the Add-on Is Upgraded?
Security Hardening
How Do I Prevent Cluster Nodes from Being Exposed to Public Networks?
How Do I Configure an Access Policy for a Cluster?
How Do I Obtain a TLS Key Certificate?
Network Configuration
How Can Container IP Addresses Survive a Container Restart?
Storage
How Do I Expand the Storage Capacity of a Container?
What Are the Differences Among CCE Storage Classes in Terms of Persistent Storage and Multi-node Mounting?
Can I Create a CCE Node Without Adding a Data Disk to the Node?
What Should I Do If the Host Cannot Be Found When Files Need to Be Uploaded to OBS During the Access to the CCE Service from a Public Network?
How Can I Achieve Compatibility Between ExtendPathMode and Kubernetes client-go?
Can CCE PVCs Detect Underlying Storage Faults?
Namespace
What Should I Do If a Namespace Fails to Be Deleted Due to an APIService Object Access Failure?
Chart and Add-on
What Should I Do If Installation of an Add-on Fails and "The release name is already exist" Is Displayed?
How Do I Configure the Add-on Resource Quotas Based on Cluster Scale?
How Can I Clean Up Residual Resources After the NGINX Ingress Controller Add-on in the Unknown State Is Deleted?
Why TLS v1.0 and v1.1 Cannot Be Used After the NGINX Ingress Controller Add-on Is Upgraded?
API & kubectl FAQs
How Can I Access a Cluster API Server?
Can the Resources Created Using APIs or kubectl Be Displayed on the CCE Console?
How Do I Download kubeconfig for Connecting to a Cluster Using kubectl?
How Do I Rectify the Error Reported When Running the kubectl top node Command?
Why Is "Error from server (Forbidden)" Displayed When I Use kubectl?
DNS FAQs
What Should I Do If Domain Name Resolution Fails in a CCE Cluster?
Why Does a Container in a CCE Cluster Fail to Perform DNS Resolution?
How Do I Optimize the Configuration If the External Domain Name Resolution Is Slow or Times Out?
How Do I Configure a DNS Policy for a Container?
Image Repository FAQs
How Do I Upload My Images to CCE?
Permissions
Can I Configure Only Namespace Permissions Without Cluster Management Permissions?
Can I Use CCE APIs If the Cluster Management Permissions Are Not Configured?
Can I Use kubectl If the Cluster Management Permissions Are Not Configured?
Best Practices
Checklist for Deploying Containerized Applications in the Cloud
Containerization
Containerizing an Enterprise Application (ERP)
Solution Overview
Procedure
Containerizing an Entire Application
Containerization Process
Analyzing the Application
Preparing the Application Runtime
Compiling a Startup Script
Compiling the Dockerfile
Building and Uploading an Image
Creating a Container Workload
Disaster Recovery
Recommended Configurations for HA CCE Clusters
Implementing High Availability for Applications in CCE
Implementing High Availability for Add-ons in CCE
Security
Configuration Suggestions on CCE Cluster Security
Configuration Suggestions on CCE Node Security
Configuration Suggestions on CCE Container Security
Configuration Suggestions on CCE Secret Security
Auto Scaling
Using HPA and CA for Auto Scaling of Workloads and Nodes
Monitoring
Monitoring Multiple Clusters Using Prometheus
Cluster
Suggestions on CCE Cluster Selection
Executing the Pre- or Post-installation Commands During Node Creation
Connecting to Multiple Clusters Using kubectl
Selecting a Data Disk for the Node
Protecting a CCE Cluster Against Overload
Networking
Planning CIDR Blocks for a Cluster
Selecting a Network Model
Implementing Sticky Session Through Load Balancing
Obtaining the Client Source IP Address for a Container
CoreDNS Configuration Optimization
Overview
Client
Optimizing Domain Name Resolution Requests
Selecting a Proper Image
Avoiding Occasional DNS Resolution Timeout Caused by IPVS Defects
Using NodeLocal DNSCache
Upgrading the CoreDNS in the Cluster Timely
Adjusting the DNS Configuration of the VPC and VM
Server
Monitoring the coredns Add-on
Adjusting the CoreDNS Deployment Status
Configuring CoreDNS
Retaining the Original IP Address of a Pod
Storage
Expanding the Storage Space
Mounting an Object Storage Bucket of a Third-Party Tenant
Dynamically Creating an SFS Turbo Subdirectory Using StorageClass
Changing the Storage Class Used by a Cluster of v1.15 from FlexVolume to CSI Everest
Using Custom Storage Classes
Scheduling EVS Disks Across AZs Using csi-disk-topology
Container
Properly Allocating Container Computing Resources
Modifying Kernel Parameters Using a Privileged Container
Using Init Containers to Initialize an Application
Configuring the /etc/hosts File of a Pod Using hostAliases
Locating Container Faults Using the Core Dump File
Permission
Configuring kubeconfig for Fine-Grained Management on Cluster Resources
Release
Overview
Using Services to Implement Simple Grayscale Release and Blue-Green Deployment
Migrating Data from CCE 1.0 to CCE 2.0
Differences Between CCE 1.0 and CCE 2.0
Migrating Images
Migrating Clusters
Migrating Applications
Applications Created Through APIs or kubectl
Applications Created Through Component Templates
Applications Created Through App Designer
API Reference (Paris Regions)
Before You Start
API Overview
Calling APIs
Making an API Request
Authentication
Response
APIs
API URL
Cluster Management
Creating a Cluster
Reading a Specified Cluster
Listing Clusters in a Specified Project
Updating a Specified Cluster
Deleting a Cluster
Hibernating a Cluster
Waking Up a Cluster
Obtaining a Cluster Certificate
Modifying Cluster Specifications
Querying a Job
Binding/Unbinding Public API Server Address
Obtaining Cluster Access Address
Node Management
Creating a Node
Reading a Specified Node
Listing All Nodes in a Cluster
Updating a Specified Node
Deleting a Node
Accepting a Node
Resetting a Node
Removing a Node
Migrating a Node
Node Pool Management
Creating a Node Pool
Reading a Specified Node Pool
Listing All Node Pools in a Specified Cluster
Updating a Specified Node Pool
Deleting a Node Pool
Add-on Management
Installing an Add-on Instance
Listing Add-on Templates
Updating an Add-on Instance
Rolling Back an Add-on Instance
Deleting an Add-on Instance
Querying an Add-on Instance
Listing Add-on Instances
Quota Management
Querying Resource Quotas
API Versions
Obtaining API Versions
Tag Management
Adding Resource Tags to a Specified Cluster in Batches
Deleting Resource Tags of a Specified Cluster in Batches
Configuration Management
Obtaining the Parameters That Can Be Configured for a Node Pool
Obtaining the List of Parameters That Can Be Configured for a Cluster
Obtaining the Parameters That Can Be Configured for a Node Pool
Changing the Values of Configuration Parameters of a Node Pool
Chart Management
Uploading a Chart
Obtaining a Chart List
Obtaining a Release List
Updating a Chart
Creating a Release
Deleting a Chart
Updating a Release
Obtaining a Chart
Deleting a Release
Downloading a Chart
Obtaining a Release
Obtaining Chart Values
Obtaining Historical Records of a Release
Obtaining the Quota of a User Chart
Add-on Instance Parameters
CoreDNS
CCE Container Storage (Everest)
CCE Node Problem Detector
Kubernetes Dashboard
CCE Cluster Autoscaler
NGINX Ingress Controller
Kubernetes Metrics Server
CCE Advanced HPA
CCE AI Suite (NVIDIA GPU)
Volcano Scheduler
CCE Secrets Manager for DEW
NodeLocal DNSCache
Kubernetes APIs
Permissions and Supported Actions
Appendix
Status Code
Error Codes
Obtaining a Project ID
Obtaining an Account ID
Specifying Add-ons to Be Installed During Cluster Creation
How to Obtain Parameters in the API URI
Creating a VPC and Subnet
Creating a Key Pair
Node Flavor Description
Adding a Salt in the password Field When Creating a Node
Maximum Number of Pods That Can Be Created on a Node
Node OS
Space Allocation of a Data Disk
Attaching Disks to a Node
User Guide (Kuala Lumpur Region)
Service Overview
What Is CCE?
Product Advantages
Application Scenarios
Containerized Application Management
Auto Scaling in Seconds
DevOps and CI/CD
Hybrid Cloud
Permissions
Notes and Constraints
Related Services
Regions and AZs
Product Bulletin
EOM of CentOS
Getting Started
Introduction
Preparations
Creating a Kubernetes Cluster
Deploying a Deployment (Nginx)
Deploying WordPress and MySQL That Depend on Each Other
Overview
Step 1: Deploying MySQL
Step 2: Deploying WordPress
High-Risk Operations
Clusters
Cluster Overview
Cluster Version Release Notes
Kubernetes Version Release Notes
Kubernetes 1.31 Release Notes
Kubernetes 1.30 Release Notes
Kubernetes 1.29 Release Notes
Kubernetes 1.28 Release Notes
Kubernetes 1.27 Release Notes
Kubernetes 1.25 (EOM) Release Notes
Kubernetes 1.23 (EOM) Release Notes
Kubernetes 1.21 (EOM) Release Notes
Kubernetes 1.19 (EOM) Release Notes
Kubernetes 1.17 (EOM) Release Notes
Patch Version Release Notes
Buying a Cluster
Comparison Between Cluster Types
Buying a CCE Standard Cluster
Comparing iptables and IPVS
Connecting to a Cluster
Accessing a Cluster Using kubectl
Accessing a Cluster Using an X.509 Certificate
Accessing a Cluster Using a Custom Domain Name
Configuring a Cluster's API Server for Internet Access
Revoking a Cluster Access Credential
Managing a Cluster
Modifying Cluster Configurations
Enabling Overload Control for a Cluster
Changing a Cluster Scale
Changing the Default Security Group of a Node
Deleting a Cluster
Preventing Cluster Deletion
Hibernating or Waking Up a Cluster
Upgrading a Cluster
Cluster Upgrade Overview
Before You Start
Performing Post-Upgrade Verification
Cluster Status Check
Node Status Check
Node Skipping Check
Service Check
New Node Check
New Pod Check
Migrating Services Across Clusters of Different Versions
Troubleshooting for Pre-upgrade Check Exceptions
Pre-upgrade Check
Node Restrictions
Upgrade Management
Add-ons
Helm Charts
SSH Connectivity of Master Nodes
Node Pools
Security Groups
Arm Node Restrictions
Residual Nodes
Discarded Kubernetes Resources
Compatibility Risks
CCE Agent Versions
Node CPU Usage
CRDs
Node Disks
Node DNS
Node Key Directory File Permissions
kubelet
Node Memory
Node Clock Synchronization Server
Node OS
Node CPU Cores
Node Python Commands
ASM Version
Node Readiness
Node journald
containerd.sock
Internal Error
Node Mount Points
Kubernetes Node Taints
Everest Restrictions
cce-hpa-controller Limitations
Enhanced CPU Policies
Health of Worker Node Components
Health of Master Node Components
Memory Resource Limit of Kubernetes Components
Discarded Kubernetes APIs
NetworkManager
Node ID File
Node Configuration Consistency
Node Configuration File
CoreDNS Configuration Consistency
sudo
Key Node Commands
Mounting of a Sock File on a Node
HTTPS Load Balancer Certificate Consistency
Node Mounting
Login Permissions of User paas on a Node
Private IPv4 Addresses of Load Balancers
Historical Upgrade Records
CIDR Block of the Cluster Management Plane
CCE AI Suite (NVIDIA GPU)
Nodes' System Parameters
Residual Package Version Data
Node Commands
Node Swap
NGINX Ingress Controller
Upgrade of Cloud Native Cluster Monitoring
containerd Pod Restart Risks
Key CCE AI Suite (NVIDIA GPU) Parameters
GPU Pod Rebuild Risks
ELB Listener Access Control
Master Node Flavor
Subnet Quota of Master Nodes
Node Runtime
Node Pool Runtime
Number of Node Images
OpenKruise Compatibility Check
Compatibility Check of Secret Encryption
Compatibility Between the Ubuntu Kernel and GPU Driver
Drainage Tasks
Image Layers on a Node
Cluster Rolling Upgrade
Rotation Certificates
Ingress and ELB Configuration Consistency
Network Policies of Cluster Network Components
Cluster and Node Pool Configurations
Time Zone of Master Nodes
SNATIPRanges
Add-on Configuration Consistency
Nodes
Node Overview
Container Engines
Node OSs
Creating a Node
Accepting Nodes for Management
Logging In to a Node
Management Nodes
Managing Node Labels
Managing Node Taints
Resetting a Node
Removing a Node
Synchronizing the Data of Cloud Servers
Draining a Node
Deleting a Node
Stopping a Node
Performing Rolling Upgrade for Nodes
Node O&M
Node Resource Reservation Policy
Space Allocation of a Data Disk
Maximum Number of Pods That Can Be Created on a Node
Differences in kubelet and Runtime Component Configurations Between CCE and the Native Community
Migrating Nodes from Docker to containerd
Configuring Node Fault Detection Policies
Executing the Pre- or Post-installation Commands During Node Creation
Node Pools
Node Pool Overview
Creating a Node Pool
Scaling a Node Pool
Managing a Node Pool
Updating a Node Pool
Updating an AS Configuration
Modifying Node Pool Configurations
Accepting Nodes in a Node Pool
Copying a Node Pool
Synchronizing Node Pools
Upgrading an OS
Migrating a Node
Deleting a Node Pool
Workloads
Overview
Creating a Workload
Creating a Deployment
Creating a StatefulSet
Creating a DaemonSet
Creating a Job
Creating a CronJob
Configuring a Workload
Configuring Time Zone Synchronization
Configuring an Image Pull Policy
Using Third-Party Images
Configuring Container Specifications
Configuring Container Lifecycle Parameters
Configuring Container Health Check
Configuring Environment Variables
Configuring Workload Upgrade Policies
Configuring Tolerance Policies
Configuring Labels and Annotations
Scheduling a Workload
Overview
Configuring Specified Node Scheduling (nodeSelector)
Configuring Node Affinity Scheduling (nodeAffinity)
Configuring Workload Affinity or Anti-affinity Scheduling (podAffinity or podAntiAffinity)
Logging In to a Container
Managing Workloads
Managing Custom Resources
Pod Security
Configuring a Pod Security Policy
Configuring Pod Security Admission
Scheduling
Overview
CPU Scheduling
CPU Policy
Enhanced CPU Policy
GPU Scheduling
Default GPU Scheduling in Kubernetes
GPU Virtualization
Overview
Preparing Virtualized GPU Resources
Using GPU Virtualization
Supporting Kubernetes' Default GPU Scheduling
Monitoring GPU Metrics
Configuring Workload Scaling Based on GPU Monitoring Metrics
Configuring Auto Scaling for xGPU Nodes
GPU Metrics
Volcano Scheduling
Overview
Scheduling Workloads
Resource Usage-based Scheduling
Bin Packing
Descheduling
Node Pool Affinity
Load-aware Scheduling
Configuration Cases for Resource Usage-based Scheduling
Priority-based Scheduling
Priority-based Scheduling
AI Performance-based Scheduling
DRF
Gang
NUMA Affinity Scheduling
Network
Overview
Container Network
Overview
VPC Network Settings
VPC Network Model
Adding a Container CIDR Block for a Cluster
Tunnel Network Settings
Tunnel Network Model
Pod Network Settings
Configuring hostNetwork for Pods
Configuring QoS for a Pod
Configuring Network Policies to Restrict Pod Access
Service
Overview
ClusterIP
NodePort
LoadBalancer
Creating a LoadBalancer Service
Configuring LoadBalancer Services Using Annotations
Configuring HTTP/HTTPS for a LoadBalancer Service
Configuring SNI for a LoadBalancer Service
Configuring HTTP/2 for a LoadBalancer Service
Configuring Timeout for a LoadBalancer Service
Configuring Health Check on Multiple LoadBalancer Service Ports
Configuring Passthrough Networking for a LoadBalancer Service
Changing a Custom EIP for a LoadBalancer Service
Enabling ICMP Security Group Rules
DNAT
Headless Services
Ingresses
Overview
LoadBalancer Ingresses
Creating a LoadBalancer Ingress on the Console
Creating a LoadBalancer Ingress Using kubectl
Annotations for Configuring LoadBalancer Ingresses
Advanced Setting Examples of LoadBalancer Ingresses
Configuring an HTTPS Certificate for a LoadBalancer Ingress
Updating the HTTPS Certificate for a LoadBalancer Ingress
Configuring SNI for a LoadBalancer Ingress
Configuring Multiple Forwarding Policies for a LoadBalancer Ingress
Configuring HTTP/2 for a LoadBalancer Ingress
Configuring HTTPS Backend Services for a LoadBalancer Ingress
Configuring Timeout for a LoadBalancer Ingress
Configuring a Slow Start for a LoadBalancer Ingress
Configuring a Range of Listening Ports for a LoadBalancer Ingress
Configuring URL Redirection for a LoadBalancer Ingress
Configuring URL Rewriting for a LoadBalancer Ingress
Redirecting HTTP to HTTPS for a LoadBalancer Ingress
Configuring the Priorities of Forwarding Rules for LoadBalancer Ingresses
Configuring a Custom Header Forwarding Policy for a LoadBalancer Ingress
Configuring a Custom EIP for a LoadBalancer Ingress
Configuring Advanced Forwarding Rules for a LoadBalancer Ingress
Configuring Advanced Forwarding Actions for a LoadBalancer Ingress
Forwarding Policy Priorities of LoadBalancer Ingresses
Configuring Multiple Ingresses to Use the Same External ELB Port
Nginx Ingresses
Creating an Nginx Ingress on the Console
Creating an Nginx Ingress Using kubectl
Annotations for Configuring Nginx Ingresses
Advanced Setting Examples of Nginx Ingresses
Configuring an HTTPS Certificate for an Nginx Ingress
Configuring Redirection Rules for an Nginx Ingress
Configuring URL Rewriting Rules for an Nginx Ingress
Configuring HTTPS Backend Services for an Nginx Ingress
Configuring Consistent Hashing for Load Balancing of an Nginx Ingress
Nginx Ingress Usage Suggestions
Optimizing NGINX Ingress Controller in High-Traffic Scenarios
Configuring an ELB Certificate for NGINX Ingress Controller
NGINX Ingress Controller Upgrade Compatibility
DNS
Overview
DNS Configuration
Using CoreDNS for Custom Domain Name Resolution
Using NodeLocal DNSCache to Improve DNS Performance
Changing the Default NodeLocal DNSCache Port
Configuring Intra-VPC Access
Accessing the Internet from a Container
Storage
Overview
Storage Basics
Elastic Volume Service
Overview
Using an Existing EVS Disk Through a Static PV
Using an EVS Disk Through a Dynamic PV
Dynamically Mounting an EVS Disk to a StatefulSet
Encrypting EVS Disks
Expanding the Capacity of an EVS Disk
Snapshots and Backups
SFS Turbo
Overview
Using an Existing SFS Turbo File System Through a Static PV
Configuring SFS Turbo Mount Options
(Recommended) Creating an SFS Turbo Subdirectory Using a Dynamic PV
Dynamically Creating an SFS Turbo Subdirectory Using StorageClass
Object Storage Service
Overview
Using an Existing OBS Bucket Through a Static PV
Using an OBS Bucket Through a Dynamic PV
Configuring OBS Mount Options
Using a Custom Access Key (AK/SK) to Mount an OBS Volume
Local PVs
Overview
Importing a PV to a Storage Pool
Using a Local PV Through a Dynamic PV
Dynamically Mounting a Local PV to a StatefulSet
Ephemeral Volumes
Overview
Importing an EV to a Storage Pool
Local EV
Temporary Path
hostPath
StorageClass
Auto Scaling
Overview
Scaling a Workload
Workload Scaling Rules
Creating an HPA Policy
Creating an HPA Policy with Custom Metrics
Creating a Scheduled CronHPA Policy
Creating a CustomedHPA Policy
Creating a VPA Policy
Creating an AHPA Policy
Managing Workload Scaling Policies
Scaling a Node
Node Scaling Rules
Priorities for Scaling Node Pools
Creating a Node Scaling Policy
Managing Node Scaling Policies
Specifying Node Scale-in Conditions for a Node Pool
Using HPA and CA for Auto Scaling of Workloads and Nodes
O&M
Logging
Overview
Collecting Container Logs
Collecting Container Logs Using ICAgent
Log Auditing
CCE Operations Supported by Cloud Trace Service
Viewing CTS Traces in the Trace List
O&M Best Practices
Monitoring Custom Metrics Using Cloud Native Cluster Monitoring
Monitoring Custom Metrics on AOM
Monitoring Master Node Components Using Prometheus
Monitoring Metrics of NGINX Ingress Controller
Migrating Custom Views of Grafana
Namespaces
Creating a Namespace
Managing Namespaces
Configuring Resource Quotas
ConfigMaps and Secrets
Creating a ConfigMap
Using a ConfigMap
Creating a Secret
Using a Secret
Cluster Secrets
Add-ons
Overview
Scheduling and Elasticity Add-ons
Volcano Scheduler
CCE Cluster Autoscaler
CCE Advanced HPA
Vertical Pod Autoscaler
Cloud Native Observability Add-ons
Cloud Native Cluster Monitoring
Cloud Native Log Collection
CCE Node Problem Detector
CCE Network Metrics Exporter
Kubernetes Metrics Server
Grafana
Prometheus
Cloud Native Heterogeneous Computing Add-ons
CCE AI Suite (NVIDIA GPU)
Container Network Add-ons
CoreDNS
NGINX Ingress Controller
NodeLocal DNSCache
Container Storage Add-ons
CCE Container Storage (Everest)
Container Security Add-ons
CCE Secrets Manager for DEW
Other Add-ons
Kubernetes Dashboard
OpenKruise
Gatekeeper
Kubernetes Web Terminal (EOM)
Helm Chart
Overview of a Chart
Deploying an Application from a Chart
Differences Between Helm v2 and Helm v3 and Adaptation Solutions
Deploying an Application Through the Helm v2 Client
Deploying an Application Through the Helm v3 Client
Converting a Release from Helm v2 to v3
Permissions
Permissions Overview
Granting Cluster Permissions to an IAM User
Namespace Permissions (Kubernetes RBAC-based)
Example: Designing and Configuring Permissions for Users in a Department
Permission Dependency of the CCE Console
Service Account Token Security Improvement
System Agencies
Settings
Dashboard
Cluster Access
Network
Scheduling
Auto Scaling
Monitoring
Kubernetes
Heterogeneous Resources
Best Practices
Checklist for Deploying Containerized Applications in the Cloud
Containerization
Containerizing an Enterprise Application (ERP)
Solution Overview
Procedure
Containerizing an Entire Application
Containerization Process
Analyzing the Application
Preparing the Application Runtime
Compiling a Startup Script
Compiling the Dockerfile
Building and Uploading an Image
Creating a Container Workload
Disaster Recovery
Recommended Configurations for HA CCE Clusters
Implementing High Availability for Applications in CCE
Implementing High Availability for Add-ons in CCE
Security
Configuration Suggestions on CCE Cluster Security
Configuration Suggestions on CCE Node Security
Configuration Suggestions on CCE Container Runtime Security
Configuration Suggestions on CCE Container Security
Configuration Suggestions on CCE Container Image Security
Configuration Suggestions on CCE Secret Security
Auto Scaling
Using HPA and CA for Auto Scaling of Workloads and Nodes
Monitoring
Monitoring Multiple Clusters Using Prometheus
Reporting Prometheus Monitoring Data to a Third-Party Monitoring Platform
Cluster
Suggestions on CCE Cluster Selection
Creating an IPv4/IPv6 Dual-Stack Cluster in CCE
Executing the Pre- or Post-installation Commands During Node Creation
Connecting to Multiple Clusters Using kubectl
Selecting a Data Disk for the Node
Protecting a CCE Cluster Against Overload
Networking
Planning CIDR Blocks for a Cluster
Selecting a Network Model
Implementing Sticky Session Through Load Balancing
Obtaining the Client Source IP Address for a Container
CoreDNS Configuration Optimization
CoreDNS Optimization Overview
Client
Optimizing Domain Name Resolution Requests
Selecting a Proper Image
Avoiding Occasional DNS Resolution Timeout Caused by IPVS Defects
Using NodeLocal DNSCache
Upgrading the CoreDNS in the Cluster Timely
Adjusting the DNS Configuration of the VPC and VM
Server
Monitoring the coredns Add-on
Adjusting the CoreDNS Deployment Status
Configuring CoreDNS
Accessing an IP Address Outside of a Cluster That Uses a VPC Network by Using Source Pod IP Addresses Within the Cluster
Storage
Expanding the Storage Space
Mounting Object Storage Across Accounts
Dynamically Creating an SFS Turbo Subdirectory Using StorageClass
Using Custom Storage Classes
Scheduling EVS Disks Across AZs Using csi-disk-topology
Container
Properly Allocating Container Computing Resources
Modifying Kernel Parameters Using a Privileged Container
Using Init Containers to Initialize an Application
Configuring the /etc/hosts File of a Pod Using hostAliases
Locating Container Faults Using the Core Dump File
Permission
Configuring kubeconfig for Fine-Grained Management on Cluster Resources
Release
Overview
Using Services to Implement Simple Grayscale Release and Blue-Green Deployment
Using Nginx Ingress to Implement Grayscale Release and Blue-Green Deployment
FAQs
Common FAQ
Cluster
Cluster Creation
Why Cannot I Create a CCE Cluster?
Is Management Scale of a Cluster Related to the Number of Master Nodes?
Which Resource Quotas Should I Pay Attention To When Using CCE?
Cluster Running
How Do I Locate the Fault When a Cluster Is Unavailable?
How Do I Retrieve Data After a CCE Cluster Is Deleted?
Cluster Deletion
What Can I Do If a Cluster Deletion Fails Due to Residual Resources in the Security Group?
How Do I Clear Residual Resources After Deleting a Non-Running Cluster?
Cluster Upgrade
What Do I Do If a Cluster Add-On Fails to be Upgraded During the CCE Cluster Upgrade?
Node
How Can I Locate a Fault That Occurs with a Node?
Node Creation
How Do I Troubleshoot Problems Occurred When Adding Nodes to a CCE Cluster?
How Do I Troubleshoot Problems Occurred When Accepting Nodes into a CCE Cluster?
What Should I Do If a Node Cannot Be Managed and an Error Message Appears Saying That the Node Failed to Install?
Node Running
What Should I Do If a Cluster Is Available But Some Nodes in It Are Unavailable?
How Do I Log In to a Node Using a Password and Reset the Password?
How Do I Collect Logs of Nodes in a CCE Cluster?
What Should I Do If the vdb Disk of a Node Is Damaged and the Node Cannot Be Recovered After Reset?
What Should I Do If I/O Suspension Occasionally Occurs When SCSI EVS Disks Are Used?
How Do I Fix an Abnormal Container or Node Due to No Thin Pool Disk Space?
How Do I Rectify Failures When the NVIDIA Driver Is Used to Start Containers on GPU Nodes?
Specification Change
How Do I Change the Node Specifications in a CCE Cluster?
What Are the Impacts of Changing the Flavor of a Node in a CCE Node Pool?
What Should I Do If I Fail to Restart or Create Workloads on a Node After Modifying the Node Specifications?
OSs
What Should I Do If There Is a Service Access Failure After a Backend Service Upgrade or a 1-Second Latency When a Service Accesses a CCE Cluster?
Node Pool
What Should I Do If a Node Pool Is Abnormal?
What Should I Do If No Node Creation Record Is Displayed When the Node Pool Is Being Expanding?
What Should I Do If a Node Pool Scale-Out Fails?
How Do I Modify ECS Configurations When an ECS Cannot Be Managed by a Node Pool?
Workload
Workload Exception Troubleshooting
How Can I Locate the Root Cause If a Workload Is Abnormal?
What Should I Do If the Scheduling of a Pod Fails?
What Should I Do If a Pod Fails to Pull the Image?
What Should I Do If Container Startup Fails?
What Should I Do If a Pod Fails to Be Evicted?
What Should I Do If a Storage Volume Cannot Be Mounted or the Mounting Times Out?
What Should I Do If a Workload Remains in the Creating State?
What Should I Do If a Pod Remains in the Terminating State?
What Should I Do If a Workload Is Stopped Caused by Pod Deletion?
What Should I Do If an Error Occurs When I Deploy a Service on a GPU Node?
How Can I Locate Faults Using an Exit Code?
Container Configuration
When Is Pre-stop Processing Used?
How Do I Set an FQDN for Accessing a Specified Container in the Same Namespace?
What Should I Do If Health Check Probes Occasionally Fail?
How Do I Set the umask Value for a Container?
What Is the Retry Mechanism When CCE Fails to Start a Pod?
Scheduling Policies
How Do I Evenly Distribute Multiple Pods to Each Node?
How Do I Prevent a Container on a Node from Being Evicted?
Why Are Pods Not Evenly Distributed on Nodes?
How Do I Evict All Pods on a Node?
Why Cannot a Pod Be Scheduled to a Node?
How Do I Troubleshoot a Pod Exit Caused by a Node Label Update?
Others
What Should I Do If a Cron Job Cannot Be Restarted After Being Stopped for a Period of Time?
What Is a Headless Service When I Create a StatefulSet?
What Should I Do If Error Message "Auth is empty" Is Displayed When a Private Image Is Pulled?
What Is the Image Pull Policy for Containers in a CCE Cluster?
What Can I Do If a Layer Is Missing During Image Pull?
Networking
Network Exception Troubleshooting
How Do I Locate a Workload Networking Fault?
Why Does the Browser Return Error Code 404 When I Access a Deployed Application?
What Should I Do If a Container Fails to Access the Internet?
What Should I Do If a Node Fails to Connect to the Internet (Public Network)?
What Should I Do If Nginx Ingress Access in the Cluster Is Abnormal After the NGINX Ingress Controller Add-on Is Upgraded?
What Could Cause Access Exceptions After Configuring an HTTPS Certificate for a LoadBalancer Ingress?
Network Planning
What Is the Relationship Between Clusters, VPCs, and Subnets?
How Can I Configure a Security Group Rule for a Cluster?
Security Hardening
How Do I Prevent Cluster Nodes from Being Exposed to Public Networks?
How Do I Configure an Access Policy for a Cluster?
How Do I Obtain a TLS Key Certificate?
How Do I Change the Security Group of Nodes in a Cluster in Batches?
Network Configuration
How Can Container IP Addresses Survive a Container Restart?
How Can I Determine Which Ingress the Listener Settings Have Been Applied To?
Storage
How Do I Expand the Storage Capacity of a Container?
What Are the Differences Among CCE Storage Classes in Terms of Persistent Storage and Multi-Node Mounting?
Can I Create a CCE Node Without Adding a Data Disk to the Node?
What Should I Do If the Host Cannot Be Found When Files Need to Be Uploaded to OBS During the Access to the CCE Service from a Public Network?
How Can I Achieve Compatibility Between ExtendPathMode and Kubernetes client-go?
Can CCE PVCs Detect Underlying Storage Faults?
Why Cannot I Delete a PV or PVC Using the kubectl delete Command?
Namespace
What Should I Do If a Namespace Fails to Be Deleted Due to an APIService Object Access Failure?
Chart and Add-on
What Should I Do If Residual Process Resources Exist Due to an Earlier npd Add-on Version?
What Should I Do If Installation of an Add-on Fails and "The release name is already exist" Is Displayed?
How Do I Configure the Add-on Resource Quotas Based on Cluster Scale?
How Can I Clean Up Residual Resources After the NGINX Ingress Controller Add-on in the Unknown State Is Deleted?
Why TLS v1.0 or v1.1 Cannot Be Used After the NGINX Ingress Controller Add-on Is Upgraded?
API & kubectl FAQs
How Can I Access a Cluster API Server?
Can the Resources Created Using APIs or kubectl Be Displayed on the CCE Console?
How Do I Download kubeconfig for Connecting to a Cluster Using kubectl?
How Do I Rectify the Error Reported When Running the kubectl top node Command?
Why Is "Error from server (Forbidden)" Displayed When I Use kubectl?
DNS FAQs
What Should I Do If Domain Name Resolution Fails in a CCE Cluster?
Why Does a Container in a CCE Cluster Fail to Perform DNS Resolution?
How Do I Optimize the Configuration If the External Domain Name Resolution Is Slow or Times Out?
How Do I Configure a DNS Policy for a Container?
Image Repository FAQs
How Do I Upload My Images to CCE?
Permissions
Can I Configure Only Namespace Permissions Without Cluster Management Permissions?
Can I Use CCE APIs If the Cluster Management Permissions Are Not Configured?
Can I Use kubectl If the Cluster Management Permissions Are Not Configured?
What Is an OBS Global Access Key and How Do I Check Whether a Global Access Key Is Used in a Cluster?
API Reference (Kuala Lumpur Region)
Before You Start
API Overview
Calling APIs
Making an API Request
Authentication
Response
APIs
API URL
Cluster Management
Creating a Cluster
Reading a Specified Cluster
Listing Clusters in a Specified Project
Updating a Specified Cluster
Deleting a Cluster
Hibernating a Cluster
Waking Up a Cluster
Obtaining a Cluster Certificate
Revoking a Cluster Certificate of a User
Modifying Cluster Specifications
Querying a Job
Binding/Unbinding Public API Server Address
Node Management
Creating a Node
Reading a Specified Node
Listing All Nodes in a Cluster
Updating a Specified Node
Deleting a Node
Enabling Scale-In Protection for a Node
Disabling Scale-In Protection for a Node
Synchronizing Nodes
Accepting a Node
Managing a Node in a Customized Node Pool
Resetting a Node
Removing a Node
Migrating a Node
Migrating a Node to a Custom Node Pool
Node Pool Management
Creating a Node Pool
Reading a Specified Node Pool
Listing All Node Pools in a Specified Cluster
Updating a Specified Node Pool
Deleting a Node Pool
Scaling a Node Pool
Synchronizing Node Configurations in a Node Pool
Storage Management
Creating a PVC (to be discarded)
Deleting a PVC (to be discarded)
Add-on Management
Installing an Add-on Instance
Listing Add-on Templates
Updating an Add-on Instance
Rolling Back an Add-on Instance
Deleting an Add-on Instance
Querying an Add-on Instance
Listing Add-on Instances
Tag Management
Adding Resource Tags to a Specified Cluster in Batches
Deleting Resource Tags of a Specified Cluster in Batches
Configuration Management
Obtaining the List of Parameters That Can Be Configured for a Node Pool
Obtaining the List of Parameters That Can Be Configured for a Cluster
Obtaining the Parameters That Can Be Configured for a Node Pool
Changing the Values of Configuration Parameters of a Node Pool
Chart Management
Uploading a Chart
Obtaining a Chart List
Obtaining a Release List
Updating a Chart
Creating a Release
Deleting a Chart
Updating a Release
Obtaining a Chart
Deleting a Release
Downloading a Chart
Obtaining a Release
Obtaining Chart Values
Obtaining Historical Records of a Release
Obtaining the Quota of a User Chart
Add-on Instance Parameters
CoreDNS
CCE Container Storage (Everest)
CCE Node Problem Detector
Kubernetes Dashboard
CCE Cluster Autoscaler
NGINX Ingress Controller
Kubernetes Metrics Server
CCE Advanced HPA
CCE AI Suite (NVIDIA GPU)
Volcano Scheduler
CCE Secrets Manager for DEW
CCE Network Metrics Exporter
NodeLocal DNSCache
Cloud Native Cluster Monitoring
Cloud Native Log Collection
Kubernetes APIs
Permissions and Supported Actions
Appendix
Status Code
Error Codes
Obtaining a Project ID
Obtaining an Account ID
Specifying Add-ons to Be Installed During Cluster Creation
How to Obtain Parameters in the API URI
Creating a VPC and Subnet
Creating a Key Pair
Node Flavor Description
Adding a Salt in the password Field When Creating a Node
Maximum Number of Pods That Can Be Created on a Node
Node OS
Space Allocation of a Data Disk
Attaching Disks to a Node
User Guide (Ankara Region)
Service Overview
What Is CCE?
Product Advantages
Application Scenarios
Containerized Application Management
Auto Scaling in Seconds
DevOps and CI/CD
Hybrid Cloud
Constraints
Permissions
Related Services
Regions and AZs
Product Bulletin
Kubernetes Version Policy
Getting Started
Introduction
Preparations
Creating a Kubernetes Cluster
Creating a Deployment (Nginx)
Deploying WordPress and MySQL That Depend on Each Other
Overview
Creating a MySQL Workload
Creating a WordPress Workload
High-Risk Operations and Solutions
Clusters
Cluster Overview
Basic Cluster Information
Kubernetes Version Release Notes
Kubernetes 1.28 Release Notes
Kubernetes 1.27 Release Notes
Kubernetes 1.25 Release Notes
Patch Version Release Notes
Creating a Cluster
Creating a CCE Standard Cluster
Comparing iptables and IPVS
Connecting to a Cluster
Connecting to a Cluster Using kubectl
Connecting to a Cluster Using an X.509 Certificate
Accessing a Cluster Using a Custom Domain Name
Upgrading a Cluster
Upgrade Overview
Before You Start
Performing Post-Upgrade Verification
Cluster Status Check
Node Status Check
Node Skipping Check
Service Check
New Node Check
New Pod Check
Migrating Services Across Clusters of Different Versions
Troubleshooting for Pre-upgrade Check Exceptions
Pre-upgrade Check
Node Restrictions
Upgrade Management
Add-ons
Helm Charts
SSH Connectivity of Master Nodes
Node Pools
Security Groups
Arm Node Restrictions
To-Be-Migrated Nodes
Discarded Kubernetes Resources
Compatibility Risks
CCE Agent Versions
Node CPU Usage
CRDs
Node Disks
Node DNS
Node Key Directory File Permissions
Kubelet
Node Memory
Node Clock Synchronization Server
Node OS
Node CPU Cores
Node Python Commands
ASM Version
Node Readiness
Node journald
containerd.sock
Internal Error
Node Mount Points
Kubernetes Node Taints
Everest Restrictions
cce-hpa-controller Limitations
Enhanced CPU Policies
Health of Worker Node Components
Health of Master Node Components
Memory Resource Limit of Kubernetes Components
Discarded Kubernetes APIs
Node NetworkManager
Node ID File
Node Configuration Consistency
Node Configuration File
CoreDNS Configuration Consistency
sudo Commands of a Node
Key Commands of Nodes
Mounting of a Sock File on a Node
HTTPS Load Balancer Certificate Consistency
Node Mounting
Login Permissions of User paas on a Node
Private IPv4 Addresses of Load Balancers
Historical Upgrade Records
CIDR Block of the Cluster Management Plane
GPU Add-on
Nodes' System Parameters
Residual Package Versions
Node Commands
Node Swap
nginx-ingress Upgrade
Upgrade of Cloud Native Cluster Monitoring
containerd Pod Restart Risks
Key GPU Add-on Parameters
GPU or NPU Pod Rebuild Risks
ELB Listener Access Control
Master Node Flavor
Subnet Quota of Master Nodes
Node Runtime
Node Pool Runtime
Number of Node Images
Managing a Cluster
Cluster Configuration Management
Cluster Overload Control
Changing Cluster Scale
Changing the Default Security Group of a Node
Deleting a Cluster
Hibernating and Waking Up a Cluster
Nodes
Node Overview
Container Engine
Creating a Node
Accepting Nodes for Management
Management Nodes
Managing Node Labels
Managing Node Taints
Resetting a Node
Removing a Node
Synchronizing the Data of Cloud Servers
Draining a Node
Deleting a Node
Stopping a Node
Performing Rolling Upgrade for Nodes
Node O&M
Node Resource Reservation Policy
Data Disk Space Allocation
Maximum Number of Pods That Can Be Created on a Node
Migrating Nodes from Docker to containerd
Node Fault Detection Policy
Node Pools
Node Pool Overview
Creating a Node Pool
Managing a Node Pool
Updating a Node Pool
Scaling a Node Pool
Updating an AS Configuration
Configuring a Node Pool
Accepting Nodes in a Node Pool
Copying a Node Pool
Synchronizing Node Pools
Upgrading an OS
Migrating a Node
Deleting a Node Pool
Workloads
Overview
Creating a Workload
Creating a Deployment
Creating a StatefulSet
Creating a DaemonSet
Creating a Job
Creating a Cron Job
Configuring a Container
Configuring Time Zone Synchronization
Configuring an Image Pull Policy
Using Third-Party Images
Configuring Container Specifications
Configuring Container Lifecycle Parameters
Configuring Container Health Check
Configuring Environment Variables
Workload Upgrade Policies
Scheduling Policies (Affinity/Anti-affinity)
Taints and Tolerations
Labels and Annotations
Accessing a Container
Managing Workloads and Jobs
Managing Custom Resources
Scheduling
Overview
CPU Scheduling
CPU Policy
Enhanced CPU Policy
GPU Scheduling
Default GPU Scheduling in Kubernetes
GPU Virtualization
Overview
Preparing xGPU Resources
Using GPU Virtualization
Supporting Kubernetes' Default GPU Scheduling
Monitoring GPU Metrics
GPU-based HPA Practice
GPU Fault Handling
NPU Scheduling
Volcano Scheduling
Overview
Scheduling Workloads
Resource Usage-based Scheduling
Bin Packing
Descheduling
Node Pool Affinity
Load-aware Scheduling
Configuration Cases for Resource Usage-based Scheduling
Priority-based Scheduling
Priority-based Scheduling
AI Performance-based Scheduling
DRF
Gang
NUMA Affinity Scheduling
Cloud Native Hybrid Deployment
Dynamic Resource Oversubscription
Network
Overview
Container Network Models
Overview
Container Tunnel Network
VPC Network
Service
Overview
ClusterIP
NodePort
LoadBalancer
Creating a LoadBalancer Service
Using Annotations to Balance Load
Configuring an HTTP or HTTPS Service
Configuring SNI for a Service
Configuring HTTP/2 for a Service
Configuring Timeout for a Service
Configuring Health Check on Multiple Service Ports
Enabling Passthrough Networking for LoadBalancer Services
Enabling ICMP Security Group Rules
Headless Services
Ingresses
Overview
LoadBalancer Ingresses
Creating a LoadBalancer Ingress on the Console
Using kubectl to Create a LoadBalancer Ingress
Configuring a LoadBalancer Ingress Using Annotations
Configuring an HTTPS Certificate for a LoadBalancer Ingress
Configuring SNI for a LoadBalancer Ingress
LoadBalancer Ingresses to Multiple Services
Configuring HTTP/2 for a LoadBalancer Ingress
Configuring URL Redirection for a LoadBalancer Ingress
Configuring URL Rewriting for a LoadBalancer Ingress
Configuring Timeout for a LoadBalancer Ingress
Configuring a Custom Header Forwarding Policy for a LoadBalancer Ingress
Nginx Ingresses
Creating Nginx Ingresses on the Console
Using kubectl to Create an Nginx Ingress
Configuring Nginx Ingresses Using Annotations
Configuring HTTPS Certificates for Nginx Ingresses
Configuring Redirection Rules for an Nginx Ingress
Configuring URL Rewriting Rules for Nginx Ingresses
Nginx Ingresses Using Consistent Hashing for Load Balancing
DNS
Overview
DNS Configuration
Using CoreDNS for Custom Domain Name Resolution
Using NodeLocal DNSCache to Improve DNS Performance
Container Network Settings
Host Network
Configuring QoS for a Pod
Container Tunnel Network Settings
Network Policies
Cluster Network Settings
Adding a Secondary VPC CIDR Block for a Cluster
Switching a Node Subnet
Adding a Container CIDR Block for a Cluster
Configuring Intra-VPC Access
Accessing the Internet from a Container
Storage
Overview
Storage Basics
Elastic Volume Service
Overview
Using an Existing EVS Disk Through a Static PV
Using an EVS Disk Through a Dynamic PV
Dynamically Mounting an EVS Disk to a StatefulSet
Snapshots and Backups
Object Storage Service
Overview
Using an Existing OBS Bucket Through a Static PV
Using an OBS Bucket Through a Dynamic PV
Configuring OBS Mount Options
Using a Custom Access Key (AK/SK) to Mount an OBS Volume
Local Persistent Volumes
Overview
Importing a PV to a Storage Pool
Using a Local PV Through a Dynamic PV
Dynamically Mounting a Local PV to a StatefulSet
Ephemeral Volumes
Overview
Importing an EV to a Storage Pool
Using a Local EV
Using a Temporary Path
hostPath
StorageClass
Observability
Logging
Overview
Collecting Container Logs
Collecting Container Logs Using ICAgent
Best Practices
Monitoring Custom Metrics Using Cloud Native Cluster Monitoring
Monitoring Custom Metrics on AOM
Monitoring Metrics of Master Node Components Using Prometheus
Monitoring Metrics of NGINX Ingress Controller
Namespaces
Creating a Namespace
Managing Namespaces
Configuring Resource Quotas
ConfigMaps and Secrets
Creating a ConfigMap
Using a ConfigMap
Creating a Secret
Using a Secret
Cluster Secrets
Auto Scaling
Overview
Scaling a Workload
Workload Scaling Rules
HPA Policies
CronHPA Policies
CustomedHPA Policies
Managing Workload Scaling Policies
Scaling a Node
Node Scaling Rules
Creating a Node Scaling Policy
Managing Node Scaling Policies
Using HPA and CA for Auto Scaling of Workloads and Nodes
Add-ons
Overview
CoreDNS
CCE Container Storage (Everest)
CCE Node Problem Detector
Kubernetes Dashboard
CCE Cluster Autoscaler
Nginx Ingress Controller
Kubernetes Metrics Server
CCE Advanced HPA
CCE AI Suite (NVIDIA GPU)
CCE AI Suite (Ascend NPU)
Volcano Scheduler
NodeLocal DNSCache
Cloud Native Cluster Monitoring
Cloud Native Logging
Grafana
Prometheus
Helm Chart
Overview
Deploying an Application from a Chart
Differences Between Helm v2 and Helm v3 and Adaptation Solutions
Deploying an Application Through the Helm v2 Client
Deploying an Application Through the Helm v3 Client
Converting a Release from Helm v2 to v3
Permissions
Permissions Overview
Granting Cluster Permissions to an IAM User
Namespace Permissions (Kubernetes RBAC-based)
Example: Designing and Configuring Permissions for Users in a Department
Permission Dependency of the CCE Console
Pod Security
Configuring a Pod Security Policy
Configuring Pod Security Admission
Service Account Token Security Improvement
FAQs
Common Questions
Cluster
Cluster Creation
Why Cannot I Create a CCE Cluster?
Is Management Scale of a Cluster Related to the Number of Master Nodes?
Which Resource Quotas Should I Pay Attention To When Using CCE?
Cluster Running
How Do I Locate the Fault When a Cluster Is Unavailable?
How Do I Retrieve Data After a Cluster Is Deleted?
Cluster Deletion
What Can I Do If a Cluster Deletion Fails Due to Residual Resources in the Security Group?
How Do I Clear Residual Resources After Deleting a Non-Running Cluster?
Node
Node Creation
How Do I Troubleshoot Problems Occurred When Adding Nodes to a CCE Cluster?
How Do I Troubleshoot Problems Occurred When Accepting Nodes into a CCE Cluster?
What Should I Do If a Node Fails to Be Accepted Because It Fails to Be Installed?
Node Running
What Should I Do If a Cluster Is Available But Some Nodes Are Unavailable?
How Do I Log In to a Node Using a Password and Reset the Password?
How Do I Collect Logs of Nodes in a CCE Cluster?
What Should I Do If the vdb Disk of a Node Is Damaged and the Node Cannot Be Recovered After Reset?
What Should I Do If I/O Suspension Occasionally Occurs When SCSI EVS Disks Are Used?
How Do I Fix an Abnormal Container or Node Due to No Thin Pool Disk Space?
How Do I Rectify Failures When the NVIDIA Driver Is Used to Start Containers on GPU Nodes?
Specification Change
How Do I Change the Node Specifications in a CCE Cluster?
What Should I Do If I Fail to Restart or Create Workloads on a Node After Modifying the Node Specifications?
OSs
What Should I Do If There Is a Service Access Failure After a Backend Service Upgrade or a 1-Second Latency When a Service Accesses a CCE Cluster?
Node Pool
What Should I Do If a Node Pool Is Abnormal?
What Should I Do If No Node Creation Record Is Displayed When the Node Pool Is Being Expanding?
What Should I Do If a Node Pool Scale-Out Fails?
How Do I Modify ECS Configurations When an ECS Cannot Be Managed by a Node Pool?
Workload
Workload Abnormalities
How Do I Use Events to Fix Abnormal Workloads?
What Should I Do If Pod Scheduling Fails?
What Should I Do If a Pod Fails to Pull the Image?
What Should I Do If Container Startup Fails?
What Should I Do If a Pod Fails to Be Evicted?
What Should I Do If a Storage Volume Cannot Be Mounted or the Mounting Times Out?
What Should I Do If a Workload Remains in the Creating State?
What Should I Do If Pods in the Terminating State Cannot Be Deleted?
What Should I Do If a Workload Is Stopped Caused by Pod Deletion?
What Should I Do If an Error Occurs When Deploying a Service on the GPU Node?
Container Configuration
When Is Pre-stop Processing Used?
How Do I Set an FQDN for Accessing a Specified Container in the Same Namespace?
What Should I Do If Health Check Probes Occasionally Fail?
How Do I Set the umask Value for a Container?
What Is the Retry Mechanism When CCE Fails to Start a Pod?
Scheduling Policies
How Do I Evenly Distribute Multiple Pods to Each Node?
How Do I Prevent a Container on a Node from Being Evicted?
Why Are Pods Not Evenly Distributed on Nodes?
How Do I Evict All Pods on a Node?
Others
What Should I Do If a Scheduled Task Cannot Be Restarted After Being Stopped for a Period of Time?
What Is a Headless Service When I Create a StatefulSet?
What Should I Do If Error Message "Auth is empty" Is Displayed When a Private Image Is Pulled?
Why Cannot a Pod Be Scheduled to a Node?
What Is the Image Pull Policy for Containers in a CCE Cluster?
What Can I Do If a Layer Is Missing During Image Pull?
Networking
Network Planning
What Is the Relationship Between Clusters, VPCs, and Subnets?
How Can I Configure a Security Group Rule in a Cluster?
Network Fault
How Do I Locate a Workload Networking Fault?
Why Does the Browser Return Error Code 404 When I Access a Deployed Application?
What Should I Do If a Container Fails to Access the Internet?
What Should I Do If a Node Fails to Connect to the Internet (Public Network)?
What Should I Do If an Nginx Ingress Access in the Cluster Is Abnormal After the Add-on Is Upgraded?
Security Hardening
How Do I Prevent Cluster Nodes from Being Exposed to Public Networks?
How Do I Configure an Access Policy for a Cluster?
Others
How Do I Change the Security Group of Nodes in a Cluster in Batches?
Storage
What Are the Differences Among CCE Storage Classes in Terms of Persistent Storage and Multi-node Mounting?
Can I Add a Node Without a Data Disk?
What Should I Do If the Host Cannot Be Found When Files Need to Be Uploaded to OBS During the Access to the CCE Service from a Public Network?
How Can I Achieve Compatibility Between ExtendPathMode and Kubernetes client-go?
Can CCE PVCs Detect Underlying Storage Faults?
Namespace
What Should I Do If a Namespace Fails to Be Deleted Due to an APIService Object Access Failure?
Chart and Add-on
Why Does Add-on Installation Fail and Prompt "The release name is already exist"?
How Do I Configure the Add-on Resource Quotas Based on Cluster Scale?
What Should I Do If the Helm Chart Uploaded Before the Tenant Account Name Is Changed Is Abnormal?
How Can I Clean Up Residual Resources After the NGINX Ingress Controller Add-on in the Unknown State Is Deleted?
Why TLS v1.0 and v1.1 Cannot Be Used After the NGINX Ingress Controller Add-on Is Upgraded?
API & kubectl FAQs
How Can I Access a Cluster API Server?
Can the Resources Created Using APIs or kubectl Be Displayed on the CCE Console?
How Do I Download kubeconfig for Connecting to a Cluster Using kubectl?
How Do I Rectify the Error Reported When Running the kubectl top node Command?
Why Is "Error from server (Forbidden)" Displayed When I Use kubectl?
DNS FAQs
What Should I Do If Domain Name Resolution Fails?
Why Does a Container in a CCE Cluster Fail to Perform DNS Resolution?
How Do I Optimize the Configuration If the External Domain Name Resolution Is Slow or Times Out?
How Do I Configure a DNS Policy for a Container?
Permissions
Can I Configure Only Namespace Permissions Without Cluster Management Permissions?
Can I Use CCE APIs If the Cluster Management Permissions Are Not Configured?
Can I Use kubectl If the Cluster Management Permissions Are Not Configured?
Reference
How Do I Expand the Storage Capacity of a Container?
How Can Container IP Addresses Survive a Container Restart?
Best Practices
Checklist for Deploying Containerized Applications in the Cloud
Containerization
Containerizing an Enterprise Application (ERP)
Solution Overview
Procedure
Containerizing an Entire Application
Containerization Process
Analyzing the Application
Preparing the Application Runtime
Compiling a Startup Script
Compiling the Dockerfile
Building and Uploading an Image
Creating a Container Workload
Disaster Recovery
Recommended Configurations for Cluster HA
Implementing High Availability for Applications in CCE
Implementing High Availability for Add-ons in CCE
Security
Configuration Suggestions on CCE Cluster Security
Configuration Suggestions on CCE Node Security
Configuration Suggestions on CCE Container Security
Configuration Suggestions on CCE Secret Security
Auto Scaling
Using HPA and CA for Auto Scaling of Workloads and Nodes
Monitoring
Using Prometheus for Multi-cluster Monitoring
Cluster
Configuring a CCE Cluster
Creating an IPv4/IPv6 Dual-Stack Cluster in CCE
Executing the Post-installation Command During Node Creation
Connecting to Multiple Clusters Using kubectl
Selecting a Data Disk for the Node
Protecting a CCE Cluster Against Overload
Networking
Planning CIDR Blocks for a Cluster
Selecting a Network Model
Implementing Sticky Session Through Load Balancing
Obtaining the Client Source IP Address for a Container
CoreDNS Configuration Optimization
Overview
Client
Optimizing Domain Name Resolution Requests
Selecting a Proper Image
Avoiding Occasional DNS Resolution Timeout Caused by IPVS Defects
Using NodeLocal DNSCache
Upgrading the CoreDNS in the Cluster Timely
Adjusting the DNS Configuration of the VPC and VM
Server
Monitoring the coredns Add-on
Adjusting the coredns Deployment Status
Configuring coredns
Retaining the Original IP Address of a Pod
Storage
Expanding the Storage Space
Mounting an Object Storage Bucket of a Third-Party Tenant
Custom Storage Classes
Enabling Automatic Topology for EVS Disks When Nodes Are Deployed in Different AZs (csi-disk-topology)
Container
Properly Allocating Container Computing Resources
Modifying Kernel Parameters Using a Privileged Container
Using Init Containers to Initialize an Application
Using hostAliases to Configure /etc/hosts in a Pod
Configuring Core Dumps
Permission
Configuring kubeconfig for Fine-Grained Management on Cluster Resources
Release
Overview
Using Services to Implement Simple Grayscale Release and Blue-Green Deployment
Using Nginx Ingress to Implement Grayscale Release and Blue-Green Deployment
API Reference (Ankara Region)
Before You Start
API Overview
Calling APIs
Making an API Request
Authentication
Response
APIs
API URL
Cluster Management
Creating a Cluster
Reading a Specified Cluster
Listing Clusters in a Specified Project
Updating a Specified Cluster
Deleting a Cluster
Hibernating a Cluster
Waking Up a Cluster
Obtaining a Cluster Certificate
Modifying Cluster Specifications
Querying a Job
Binding/Unbinding Public API Server Address
Obtaining Cluster Access Address
Node Management
Creating a Node
Reading a Specified Node
Listing All Nodes in a Cluster
Updating a Specified Node
Deleting a Node
Accepting a Node
Resetting a Node
Removing a Node
Migrating a Node
Node Pool Management
Creating a Node Pool
Reading a Specified Node Pool
Listing All Node Pools in a Specified Cluster
Updating a Specified Node Pool
Deleting a Node Pool
Storage Management
Creating a PVC (to be discarded)
Deleting a PVC (to be discarded)
Add-on Management
Installing an Add-on Instance
Listing Add-on Templates
Updating an Add-on Instance
Rolling Back an Add-on Instance
Deleting an Add-on Instance
Querying an Add-on Instance
Listing Add-on Instances
Quota Management
Querying Resource Quotas
API Versions
Obtaining API Versions
Tag Management
Adding Resource Tags to a Specified Cluster in Batches
Deleting Resource Tags of a Specified Cluster in Batches
Configuration Management
Obtaining the Parameters That Can Be Configured for a Node Pool
Obtaining the List of Parameters That Can Be Configured for a Cluster
Obtaining the Parameters That Can Be Configured for a Node Pool
Changing the Values of Configuration Parameters of a Node Pool
Chart Management
Uploading a Chart
Obtaining a Chart List
Obtaining a Release List
Updating a Chart
Creating a Release
Deleting a Chart
Updating a Release
Obtaining a Chart
Deleting a Release
Downloading a Chart
Obtaining a Release
Obtaining Chart Values
Obtaining Historical Records of a Release
Obtaining the Quota of a User Chart
Kubernetes APIs
Permissions and Supported Actions
Appendix
Status Code
Error Codes
Obtaining a Project ID
Obtaining an Account ID
Specifying Add-ons to Be Installed During Cluster Creation
How to Obtain Parameters in the API URI
Creating a VPC and Subnet
Creating a Key Pair
Node Flavor Description
Adding a Salt in the password Field When Creating a Node
Maximum Number of Pods That Can Be Created on a Node
Node OS
Data Disk Space Allocation
Attaching Disks to a Node