Overview

Service governance is a broad concept. Generally, it refers to some measures that ensure the reliable system running and are independent of business logic. The following assurance measures are provided to deal with the common fault modes in microservice scenarios:

• Load balancing management: provides load balancing policy management in multi-instance scenarios. For example, the polling mode is used to ensure that traffic is balanced among different instances. When an instance is faulty, the instance can be temporarily isolated to prevent access to the instance from causing request timeout.
• Rate limiting: provides load protection and prevents the system from breaking down when the external traffic exceeds the processing capability of the system. Rate limiting is also used to smooth requests so that requests are evenly distributed to services, preventing the impact of burst traffic on the system.
• Retry: prevents random failures, which often occur in the microservice system, which often occur in the microservice system due to many reasons. Take the request timeout of a Java microservice application as an example. This may occur due to network fluctuation or software/hardware upgrade, which may interrupt services for several seconds. The increasing latency due to JVM garbage collection and thread scheduling may also be the cause. The system is more prone to time out when traffic is not even, such as 1000 concurrent requests and 1000 requests within 1s. The interaction between applications, systems, and networks can also cause random failures. The burst traffic of an application may affect the bandwidth and consequently the running of other applications. In other application-related scenarios, for example, SSL needs to obtain the OS entropy. If the entropy is too low, a latency of several seconds will occur. The system must be capable of protection against the inevitable random faults.
• Bulkhead: protects resource-consuming services. For example, if a time-consuming service shares a thread pool with other services, other services will wait when the service receives a large number of sudden requests, compromising the performance of the entire system. The bulkhead allocates an independent resource pool (usually implemented through the semaphore or thread pool) to resource-intensive services to prevent other services from being affected.
• Degradation: During peak hours, access to the target service needs to be temporarily reduced to decrease the load of the target service. Alternatively, access to non-key services needs to be shielded to maintain the core processing capability of the service.

However, no governance measure is applicable to all scenarios. That is, a governance measure that works well in one scenario may cause problems in another. Therefore, it is important to dynamically update the governance policy based on the service running status and metrics.

To use service governance in a service system, perform the following steps:

1. Develop a service. This step focuses more on service function delivery than on service governance. The microservice development framework provides assurance measures by default against common system faults. Selecting an appropriate microservice development framework can save the DFx time.
2. Conduct performance tests and fault drills. Many system instability issues are found in this process. The service governance policies are applied to resolving these issues and written to the configuration file as the default values of the application.
3. Bring the service online. If an unexpected scenario occurs during service rollout, you need to use the configuration center to dynamically adjust governance parameters for stable service running.

The preceding three steps will be continuously optimized throughout the software lifecycle. ServiceComb engines provide unified service governance capability based on request markers for different microservice development frameworks. If a microservice framework is used to develop an application, the microservice is automatically registered with the corresponding ServiceComb engine after the application is hosted and started, and you can perform service governance on the CSE console. For details, see Governing Microservices.

This section describes how to use the service governance capability based on request markers.