Updated on 2025-10-17 GMT+08:00

Zero-Privilege O&M

As mentioned in QingTian Threat Assumptions and Security Methods, cloud vendors' internal personnel may be potential attacks that pose threats to tenant data security through logical or physical access. The QingTian system introduces the zero-privilege O&M concept to eliminate such security risks by combining best technical practices and O&M security management.

  • Zero SSH permission: O&M personnel cannot obtain control permissions on servers through common remote login methods. All O&M operations on servers are performed through APIs. All O&M APIs have strict identity authentication, authorization, recording, and auditing. These APIs do not allow O&M personnel to access customer data on servers.
  • Zero packet capture tool: The OS of the server node has been streamlined, with common packet capture tools such as tcpdump removed, to prevent tenant I/O data from being listened to or stolen.
  • Zero memory permission: VRAM, a QingTian memory management component, independently manages VM memory. The memory cannot be exported via traditional virsh dump.

These technical restrictions are built into the QingTian system. Even the system administrator with the highest permissions cannot bypass these controls and protections. As common login access is disabled, the production environment does not support in-place debugging. This is inconvenient for technical personnel, but we believe that this is a good trade-off for our customers. We must maintain high system quality and testing standards before production release.

Cloud infrastructure security also depends on the root key protection of multiple key systems. Root key protection needs to handle various threats, including potential threats from internal personnel with the highest permissions. Hardware security modules (HSMs) cannot avoid individual security issues. Only relying on this key protection technology cannot address increasingly severe attack challenges. The QingTian system uses the following methods and measures to protect the keys of key systems:

  • Threshold digital signature: Some key systems use threshold digital signature algorithms to split signature keys and completely destroy the original keys after the splitting is complete. Once split, a number of N key fragments are separately held by multiple independent nodes. A number of T compute nodes can collaborate with each other to sign a message according to the threshold signature protocol. If the number of nodes is less than T, the message cannot be signed. Complete keys are not displayed throughout the signature process, and no single point holds the complete keys. This method not only improves the security of key management but also improves the availability of the system.
  • Multiple key materials: Some key systems use multiple digital signature technologies (depending on multiple signature keys) for anti-tampering. Other systems synthesize data encryption and decryption keys based on multiple independent key materials. Each independent signature key or key material is protected by security hardware to prevent systematic security risks caused by the compromise of a single key.
  • Key export prevention: Working keys are encrypted using hardware identity public keys and imported to the HSMs built into QingTian cards. They cannot be exported from the hardware in plaintext. Instead, they can only be used through module APIs with limited authentication and authorization.
  • Fast key rotation: The system strictly ensures that the key rotation period is inversely proportional to the key usage frequency. Frequently used working keys (such as end entity certificate keys) are rotated on an hourly basis.
  • Tenant-level key isolation: The system allocates completely independent random key materials to different tenants and derives tenant-level keys based on the key materials. Derived keys are only valid for single tenants, preventing global security impact caused by the compromise of a single key material.

Key protection has always been a challenge. We continuously focus on the latest cryptography technology progress (such as threshold cryptography technology), introduce these new technologies in cloud engineering best practices to improve security, and update the current engineering security best practices.