Optimizing Serverless Performance

Serverless function configuration

• Runtime

Compiled languages (such as Java and C#) may incur higher cold start latency due to initialization overhead, but offer lower execution latency after initialization. Use interpreted languages (such as Node.js and Python) for scenarios with uneven traffic or strict cold start requirements. For stable traffic or average latency-sensitive workloads, choose compiled languages like Java or Go.

• Memory

CPU resources scale with allocated memory. Larger memory sizes improve performance for CPU/memory-intensive tasks. Test different memory configurations to balance latency and cost.

Use dynamic memory allocation if a function requires varying resources across different scenarios. For details, see Configuring Dynamic Memory.

• Maximum instances

Ensure the maximum instance count matches actual concurrency demands. Insufficient instances may cause queuing delays or errors.

• Reserved instances

Reserved instances can be created and released by you as required. After you create reserved instances for a function, FunctionGraph preferentially forwards requests to the reserved instances. If the number of requests exceeds the processing capability of the reserved instances, FunctionGraph will forward the excessive requests to on-demand instances and automatically allocates execution resources to these requests. If your service traffic is uneven and cold start latency is required, you can configure a certain number of reserved instances. If your service traffic has a periodic pattern, you can configure scheduled scaling and intelligent recommendation policies for reserved instances to improve resource utilization. For details, see Reserved Instance Management.

Network

Public access: Functions can be accessed over the public network. However, the bandwidth is limited and shared by all tenants. This access mode is suitable for test services that have low requirements on bandwidth and reliability.

VPC access: Functions can be accessed from a specified VPC. However, the cold start latency increases because the network link to the VPC is initialized during cold start.

For production with high bandwidth needs, use a VPC with a NAT gateway. If functions do not need to access the network, you are advised not to configure a VPC. For details, see Configuring the Network.

• Timeout

Avoid excessively long timeout to prevent service interruption from code exception. Set reasonable timeout aligned with service logic to enable failfast behavior and improve user experience.

Serverless function code

If the service logic can be implemented asynchronously, performance optimization (except for cost) is less critical. FunctionGraph performance largely depends on the executed logic.

Strategies:

• Use connection pools (HTTP, databases, Redis) to reuse established connections.

• Maintain HTTP connection pools to reduce initialization time if functions are invoked via APIs.
• Use global static variables and singletons to avoid reinitializing variable objects.
• Use interpretive languages (Node.js and Python) instead of compiled languages (Java and Go).
• Simplify the function code package to accelerate download from Huawei Cloud OBS before calling the functions.
• When calling other cloud service resources (such as DIS and OBS), optimize the SDK parameters, such as the timeout, number of connections, and retries
• When FunctionGraph accesses a third-party service or Huawei Cloud service, ensure the service has no performance bottleneck.

Cold start optimization

Serverless is the next-generation cloud computing paradigm due to its pay-per-use, auto scaling, and complexity shielding features. However, serverless also has a challenge in real-time applications: cold start. If the cold start and web service initialization time exceed 5 seconds, user experience will be greatly affected. Therefore, reducing the cold start time and improving user experience are urgent problems to be solved when building a serverless architecture.

The lifecycle of a serverless instance can be divided into three phases: initialization, execution, and shutdown.

When FunctionGraph is triggered, if no activated function instance is available, the function code is downloaded and an execution environment is created. The period from the time when a function is triggered to the time when a new FunctionGraph environment is created is called cold start. In the serverless architecture, the cold start cannot be avoided.

Currently, FunctionGraph has optimized the cold start on the system side. For details about the cold start on the user side, see the following solutions.

• Proper runtime

FunctionGraph supports.NET Core, Go, Java, Node.js, and Python. Cold start time (512 MB memory): Java > Go > .NET > Node.js > Python.

• Memory

Given a fixed level of request concurrency, higher function memory leads to better cold start performance with more CPU resources allocated.

• Snapshot-based cold start

Cold start is quite prominent in Java applications. FunctionGraph has proposed a process snapshot–based cold start acceleration solution, which helps you break through the performance bottleneck while involving zero or few code changes. In this solution, the execution environment is restored from a snapshot captured after initialization, avoiding complex framework startup and service initialization. The startup latency of Java applications is significantly reduced, and the performance is improved by over 90%.

Performance metrics

FunctionGraph provides a series of metrics to help you optimize your functions. Once a function is called, you can view the metric changes on the monitoring page and log page of the function to further optimize the function configuration and code.

FunctionGraph provides two types of metrics: overview metrics and function metrics. For details, see the official metric document.