Will Enhanced JavaAgents Affect User Performance?
APM Agents use the bytecode enhancement technology to dynamically collect performance data, including method calls, exception information, and distributed traces, helping development and O&M teams monitor and optimize system performance in real time. Agents inevitably cause certain performance loss. By evaluating the performance overhead (such as CPU, memory, and latency) of Agents and the stability and data collection integrity in heavy load scenarios, we can ensure that Agents run reliably in the production environment and their impact on performance is controllable. This evaluation provides important basis for optimization and deployment.
Test Environments
| Tool/Service | Version/Specifications | Description |
|---|---|---|
| JVisualVM | 1.8.0_216 | JVisualVM is a performance monitoring tool provided by Java. It monitors and manages JConsole. It can provide the memory, thread, class loading, JVM overview, and real-time information of a Java process. |
| JMeter | 5.3 | Apache JMeter is a Java-based pressure test tool developed by Apache. In this test, it is used to simulate multiple users to concurrently call an APM API to query graphs. |
| JavaAgent | 2.4.11-profiler | Stable JavaAgent version |
| ECS | 2u4g | General computing-plus | 2 vCPUs | 4 GiB | c7.large.2, CentOS 7.9 |
| Demo application | benchmark.jar | Sends requests based on the pressure test source and accesses the MySQL and Redis services at the same time. Corresponding values will be returned. Spring Cloud and Dubbo are used in this process. |
Constraints
- The Agent version that supports Profiler must be 2.4.5 or later. For details, see JavaAgent Updates.
- The Agent version that allows users to set sampling policies must be 2.4.11 or later. For details, see JavaAgent Updates.
Test Procedure
- Respectively use 1 TPS, 500 TPS, 1,000 TPS, and 2,000 TPS samples for pressure tests when no Agent is installed. Each pressure test lasts for 30 minutes. The pressure test results are used as the performance baseline.
- Install an Agent, respectively set the sampling policy to intelligent sampling and 100% sampling, repeat the pressure test process in 1, and compare differences in CPU, memory, and RT.
- Install an Agent that supports Profiler, disable the performance profiling function, respectively set the sampling policy to intelligent sampling and 100% sampling, repeat the pressure test process in 1, and compare the differences in CPU, memory, and RT.
Performance limit metrics when no Agent is installed
| No. | Pressure Test Samples | RT (ms) | CPU (%) | Memory (MB) |
|---|---|---|---|---|
| 1 | 1 TPS | 77.05 | 0.4 | 200 |
| 2 | 500 TPS | 77.42 | 11 | 250 |
| 3 | 1,000 TPS | 79.17 | 23 | 300 |
| 4 | 2,000 TPS | 83.19 | 45 | 350 |
Performance limit metrics when an Agent is installed
| No. | Pressure Test Samples | RT (ms) | CPU (%) | Memory (MB) |
|---|---|---|---|---|
| 1 | 1 TPS | 78.36 | 0.4 | 250 |
| 2 | 500 TPS | 79.05 | 18 | 300 |
| 3 | 1,000 TPS | 81.84 | 31 | 350 |
| 4 | 2,000 TPS | 86.81 | 55 | 400 |
| No. | Pressure Test Samples | RT (ms) | CPU (%) | Memory (MB) |
|---|---|---|---|---|
| 1 | 1 TPS | 78.38 | 0.4 | 250 |
| 2 | 500 TPS | 80.39 | 20 | 300 |
| 3 | 1,000 TPS | 84.51 | 33 | 450 |
| 4 | 2,000 TPS | 88.72 | 65 | 500 |
Agent Performance Overhead Comparison
| No. | Pressure Test Samples | Intelligent Sampling | 100% Sampling | ||||
|---|---|---|---|---|---|---|---|
| - | RT | CPU | Memory | RT | CPU | Memory | |
| 1 | 1 TPS | +1.31 ms | +0% | +50 MB | +1.33 ms | +0% | +50 MB |
| 2 | 500 TPS | +1.63 ms | +7% | +50 MB | +2.97 ms | +9% | +50 MB |
| 3 | 1,000 TPS | +2.67 ms | +8% | +50 MB | +5.34 ms | +10% | +150 MB |
| 4 | 2,000 TPS | +3.62 ms | +10% | +50 MB | +5.53 ms | +20% | +150 MB |
Conclusion
- Enhanced JavaAgents have very little impact on RT.
- Intelligent sampling increases the CPU and memory overhead, but the increase is less than 10% of the total host overhead. The overhead may increase along with the application complexity.
- The performance overhead of 100% sampling is slightly higher than that of intelligent sampling. Do not enable 100% sampling in heavy load scenarios.
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.See the reply and handling status in My Cloud VOC.
For any further questions, feel free to contact us through the chatbot.
Chatbot
