Rolling Upgrades for Real-Time Service Deployment

Overview

Rolling upgrade is a core capability of ModelArts real-time inference services that enables smooth iteration and zero-disruption updates. By gradually replacing deployment replicas, it allows for service version upgrades, image updates, model replacements, and configuration changes, keeping the entire process transparent to the services and ensuring requests are never interrupted.

Typical scenario:

• Model version iteration: Upgrading large or small models after updating weights or fine-tuning, preventing a complete service outage.
• Image updates: Patching vulnerabilities, optimizing performance, or upgrading dependency packages by replacing instances with zero downtime.
• Configuration changes: Adjusting resource specifications, environment variables, or health check parameters with smooth effectuation.
• Gray releases: Launching new versions in batches and rolling them out completely only after validating stability, thereby minimizing risk.
• High-availability O&M: Rolling restarts of instances when service anomalies occur, ensuring service continuity.

Constraints

• Resource pool constraints: Both dedicated resource pools and public resource pools support rolling upgrades.
• Image constraints: Custom images and preset images are both supported; however, the image must be able to start normally without fatal errors.
• Replica count constraints: True zero-disruption upgrade can only be achieved when the unavailable replica count ≥ 0. If the unavailable replica count reaches 100%, a brief interruption will occur.
• Health check constraints: You are advised to configure health checks beforehand to prevent new instances from receiving traffic before they are fully ready.
• Upgrade interruption constraints: If insufficient resources or image errors are encountered during the upgrade process, the upgrade will be terminated, and the old instances will continue to run.

Configuring a Rolling Upgrade

Log in to the ModelArts console and choose Model Inference > Real-Time Inference. When deploying a real-time service, set Max Surge Replicas and Max Unavailable Replicas under Deployment Management Settings > More Settings. For details, see Deploying a Real-Time Inference Service Using a Single Node.

Max Surge Replicas: The maximum percentage by which the number of deployment replicas can exceed the target replica count during each rolling upgrade step.

Max Unavailable Replicas: The maximum percentage by which the number of deployment replicas can fall below the target replica count during each rolling upgrade step. If Max Unavailable Replicas equals the target replica count, there is a risk of downtime (Min Available Replicas = Total Replicas – Max Unavailable Replicas).