Creating an Endpoint

Constraints

• This function is only available in the CN-Hong Kong regions. Resources cannot be called across regions.
• This function allows you to create custom endpoints based exclusively on built-in services.
• Each account can have up to 30 custom endpoints.
• Each endpoint must have a unique name under an account. A deleted endpoint name cannot be used when you create a new endpoint.
• After an endpoint is created, the model service cannot be modified.
• The created endpoints must comply with the rules and specifications of the platform and cannot be called in violation of regulations.

Billing

Endpoints are free to create. However, you may be charged for calling the model service or using resources. Check service costs in the Billing Center by endpoint name.

• Calling a custom MaaS endpoint: billed by MaaS. Token-based billing is supported. For details, see Billing Items for MaaS Built-in Model Services.

Creating an Endpoint

1. Log in to the MaaS console and select the target region on the top navigation bar.
2. In the navigation pane, choose Model Inference > Real-time inference.
3. Click the Endpoint tab and click Create Endpoint in the upper right corner of the page.
4. On the Create Endpoint dialog box, configure parameters.

   Table 1 Parameters for creating an endpoint

   Parameter		Description
   Basic information	Parameter	Name of an endpoint. The endpoint name must be unique and cannot contain special characters. Enter 1 to 64 characters. The name must start with an uppercase or lowercase letter, and can only contain letters, digits, hyphens (-), underscores (_), and dots (.).
   	Description	Description of an endpoint. The value can contain up to 256 characters.
   	Please select a model	You can select MaaS. MaaS: built-in models provided by MaaS. Click Please select a model. In the Select Model dialog box, select the model and version as required, and click OK. All built-in services, whether enabled or disabled, are supported by default. The supported access mode is displayed on the right of the model version.
   Billing Information	Billing Mode	The following billing modes are available:
   Rate Limiting	Model flow limiting	After you select a built-in model, the total traffic limit for the model under the current account is displayed. Requests Per Minute (RPM): The number of requests processed per minute. Tokens Per Minute (TPM): The number of input and output tokens processed per minute.
   	Endpoint Flow Control	You can manually set the traffic limit for an endpoint. If the combined rate limit of all endpoints accessing the same model under an account equals the model's total quota, it effectively prevents competition for traffic allocation among different endpoints. You can configure different RPM and TPM traffic controls for each endpoint, as long as they stay within your account's traffic limits. The values of RPM and TPM must be positive integers.
   	Designated flow control period	This parameter is displayed only when Endpoint Flow Control is enabled. After Designated flow control period is enabled, you can set the traffic limit for a specific period of each day for the endpoint. The traffic control settings for other periods remain unchanged. You can click Add Period to set the traffic limit for multiple periods. When setting multiple periods, ensure that the time ranges do not overlap. After setting a specified traffic control period, you can click View time period-based flow control in the Model flow limiting column on the Endpoint tab page to view the specified traffic control periods. NOTE: Traffic control is prioritized as follows: specified periods first, then endpoint controls, and finally model-level controls. If both endpoint-based controls and specified period-based controls are configured, the specified period-based controls take precedence. Endpoint-based controls apply outside those periods. If only specified period-based controls are configured, these controls take precedence. Model-level controls apply outside those periods.
   More configurations	ModelArts Guard	Configure content guard if you have a V2 or higher account level. Content guard is enabled by default. NOTE: To view your customer tier, log in to the MaaS console, click the username in the upper right corner, click Basic Information, and view the tier next to the account name. ModelArts Guard blocks harmful content in inputs and outputs during model inference. Enabling it might slow down processing. When disabled, the models rely on their native security features. To disable ModelArts Guard, deselect ModelArts Guard, read the notice, and select I have read and agree to the disclaimer above. You can enable or disable ModelArts Guard when editing an endpoint. For details, see Editing an Endpoint.

5. On the right of the page, view the configuration summary, select I have read and agree to MaaS Service Statement, confirm the billing information at the bottom of the page, and click Create Now.
6. Once you create the endpoint, its details appear in the Endpoint tab. From there, you can see that its status is In use and call or experience it online.
   Figure 1 Endpoint created
   

   To check bill details by ID, click next to the endpoint resource ID to copy the ID. Go to the Billing Center, choose Billing > Transactions and Detailed Bills, and click Bill Details.

   Figure 2 Copying an ID
   

Viewing Endpoint Details

1. On the MaaS console, choose Model Inference > Real-Time Inference. On the displayed page, click the Endpoints tab. In the Endpoint Name/Resource ID column of the target endpoint, click the endpoint name or resource ID. The Overview page is displayed by default. You can view details about the endpoint, including basic information, billing information, rate limiting information, and more configurations. For details about the parameters, see Table 2.

   Table 2 Parameters about an endpoint

   Parameter		Description
   Basic information	Endpoint Name	Name of an endpoint.
   	ID	ID of an endpoint.
   	Model	Model selected for the endpoint. Click a model name to view the model information.
   	Source	Source of the model selected for the endpoint. Currently, MaaS are supported.
   	Model Parameter	Model parameter selected for the endpoint.
   	Created At	Time when an endpoint was created.
   	Description	Description of an endpoint.
   Billing Information	Billing Mode	Billing mode of the endpoint. It is either token-based billing or model unit-based billing. For details, see Table 1.
   	Inference Price	Inference pricing unit. You can switch between 1M tokens and 1K tokens.
   Flow Control	Model Flow Limiting	This parameter is displayed only when Select Model is set to MaaS. After you select a built-in model, the total traffic limit for accessing the model service under the current account is displayed. Requests Per Minute (RPM): The number of requests processed per minute. Tokens Per Minute (TPM): The number of input and output tokens processed per minute.
   	Endpoint Flow Control	You can manually set the traffic limit for an endpoint. If the combined rate limit of all endpoints accessing the same model under an account equals the model's total quota, it effectively prevents competition for traffic allocation among different endpoints. You can configure different RPM and TPM traffic controls for each endpoint, as long as they stay within your account's traffic limits. The values of RPM and TPM must be positive integers.
   More Configurations	ModelArts Guard	Displays the current status of the content safety guardrails. It is On or Off.
   	Implicit Watermark	This parameter is displayed only when Select Model is set to MaaS and a built-in video or image generation service is selected. This function is disabled by default. Once enabled, you must add metadata-based implicit identifiers to AI-generated content.
   	Explicit Watermark	This parameter is displayed only when Select Model is set to MaaS and a built-in video or image generation service is selected. This function is disabled by default. Once enabled, model outputs will include an AI watermark.

   • Endpoints can be edited, deleted, enabled, disabled, and invoked for inference.
   • To check bill details by ID, click next to the endpoint resource ID to copy the ID. Go to the Billing Center, choose Billing > Transactions and Detailed Bills, and click Bill Details.
   • You can also click the icon next to the model parameter to copy the model parameter. On the API Calling tab page, you can call different access points based on different model parameters.
2. Click the View Call Description tab to view the process of calling an endpoint. For details, see Calling an Endpoint.
3. Click the Monitoring tab to view the call details of the endpoint.
   • On the Monitoring tab, select the time range, time granularity, calling method, and IP address as required.

     Table 3 Filters for endpoint monitoring

     Parameter	Description
     Time Range	Service call statistics can be viewed for today, yesterday, the past 3, 7, or 14 days, or across a custom date range. Time range and precision filtering rules: For time ranges of 1–2 day, metrics can be queried at minute or hour precision. For time ranges of 3–7 days, metrics can be queried at hour or day precision. For time ranges longer than 7 days, metrics can be queried at day precision only.
     Time granularity	The calling statistics can be collected by minute or hour.
     Calling Method	API key calls and online experience are supported. All API keys are used by default to authenticate calls to a model service on MaaS. You can choose specific ones if needed. For more information, see Deploying a Model Service and Manage API Keys.
     IP Address	The client source IP address (public IP) that has generated calls is derived from the http_x_forwarded_for field in the APIG logs. When this field contains multiple values, the system will use the first value; if the field value is -, it will be displayed as an empty string. The IP address is displayed by default as All, but you can also select specific IP addresses as needed.

   • Usage: Track endpoints' token usage to prevent overuse.

     Table 4 Usage metrics

     Parameter	Description
     Completed Tasks	Number of completed tasks at the endpoint.
     Total Inferences	Number of inferences at the endpoint.
     Total Requests	The total number of requests at the endpoint.
     Total Tokens Used (K Tokens)	The total number of tokens used by the endpoint within each unit of time.
     TPM (K Tokens/Minute)	The number of input and output tokens processed per minute.
     RPM	The number of requests processed per minute.
     QPS	The number of queries processed per second. When Time Precision is set to By the minute, the value displayed is the peak QPS recorded during that minute.
     Input Tokens (K Tokens)	The length of input tokens. AVG: The average length of input tokens. MAX: The maximum length of input tokens. P50: 50% of the input tokens are below this value. P80: 80% of the input tokens are below this value. P90: 90% of the input tokens are below this value. P99: 99% of the input tokens are below this value.
     Output Tokens (K Tokens)	The length of output tokens. AVG: The average length of output tokens. MAX: The maximum length of output tokens. P50: 50% of the output tokens are below this value. P80: 80% of the output tokens are below this value. P90: 90% of the output tokens are below this value. P99: 99% of the output tokens are below this value.

   • Performance metrics: View performance metrics to identify opportunities for improvement.

     Table 5 Performance metrics

     Parameter	Description
     Average Generation Time (s)	The average time taken to generate each image/video.
     Average Response Latency (ms)	The end-to-end latency of successful requests per unit time AVG: The average end-to-end latency. MAX: The maximum end-to-end latency. P50: 50% of the end-to-end latencies are below this value. P80: 80% of the end-to-end latencies are below this value. P90: 90% of the end-to-end latencies are below this value. P99: 99% of the end-to-end latencies are below this value.
     Time to First Token (ms)	The time from receiving a request to generating the first output token. The statistics are collected only for streaming responses. Due to model version constraints, certain versions cannot display these metrics in non-streaming scenarios. Upgrade to the latest version to view these metrics. AVG: The average TTFT. MAX: The maximum TTFT. P50: 50% of the TTFTs are below this value. P80: 80% of the TTFTs are below this value. P90: 90% of the TTFTs are below this value. P99: 99% of the TTFTs are below this value.
     TPOT (ms)	The interval between two output tokens. The statistics are collected only for streaming responses. Due to model version constraints, certain versions cannot display these metrics in non-streaming scenarios. Upgrade to the latest version to view these metrics. AVG: The average TPOT. MAX: The maximum TPOT. P50: 50% of the TPOTs are below this value. P80: 80% of the TPOTs are below this value. P90: 90% of the TPOTs are below this value. P99: 99% of the TPOTs are below this value.
     Average Task Processing Duration (min)	Average time spent on processing tasks.

   • Cache: View the cache hit metrics of an endpoint to evaluate the system performance.

     Table 6 Cache hit metrics

     Parameter	Description
     Cache Hits (K Tokens)	The total number of tokens served from the cache at the endpoint.
     Cache Hit Rate (%)	The proportion of input tokens served from the endpoint cache, expressed as a percentage of total input tokens.

   • Troubleshooting: Quickly identify issues such as sudden increases in calls, abnormal consumption, and call failures during specific time periods.

     Table 7 Abnormal metrics

     Parameter	Description
     Request Failure Rate (%)	The proportion of failed requests to the total number of requests.
     Total Failed Requests	Indicates the number of failed requests. An error message is displayed when a request fails.
     Inference Failure Rate (%)	The proportion of failed inferences to the total number of inferences.

   • For more information, see Viewing the Call Data and Monitoring Metrics of Real-Time Inference on MaaS.

Calling an Endpoint

An endpoint can be called only when it is in the In use status. AI generates content from service calls. It does not reflect MaaS's views. The platform does not guarantee the legality, authenticity, and accuracy of the content, and does not assume any legal liability.

1. On the MaaS console, choose Model Inference > Real-time inference and click the Endpoints tab. In the Operation column of the target endpoint, click View Call Description.
2. On the View Call Description page, obtain the API key as prompted, copy the call example, replace the API information and API key, and call the API.
   • The model Parameter column in the Endpoint tab shows the model's name used in the code during service calls. You can call different endpoints based on different model parameters.
   • For details about how to create an API key, see Creating an API Key.
   • For details about the parameters in the call example, see MaaS Standard API V2.

Managing an Endpoint

You can view call data and metric data, and edit, enable, or disable endpoints.

FAQs

• What should I do if the number of endpoints I have created reaches the limit?

  You can delete the endpoints that are no longer needed and then create new ones.

• How can I determine the number of tokens consumed?

  On the Call Statistics page, check the total number of tokens used, number of input tokens, and number of output tokens. For details, see Viewing the Call Statistics of an Endpoint.

• How long does it take for changes to the traffic limiting settings of an endpoint to take effect?

  After you save the changes, the traffic limiting settings will take effect immediately, and subsequent calls will be executed according to the new rules.

• What should I do if the billing status of my endpoint changes to invalid?

  If the billing status of your endpoint becomes invalid, it indicates that the resource was frozen due to account arrears and exceeded the resource retention period, resulting in deletion. You may proceed to delete this endpoint and recreate it as needed.