Certification Notes

Before starting the certification test, fill in the required material for product certification tests based on the product solution and upload it as an attachment during solution registration. Fill in the reliability checklist and upload it as an attachment of the last reliability test case after the reliability test case is executed.

Huawei Cloud Certification Test and Requirements

This document describes the certification tests of joint operations products, partner-led, and advanced cloud software. Each test verifies the product functions, performance, security, reliability, and integration. For details, see the following table.

**Table 1** Requirement description
Test Item		Test Requirement
Function	Service function	1) Test cases must cover all level-2 functions in the function list. 2) Operation steps are required for test cases. 3) End to end test cases are required for main scenarios. 4) If hardware products are involved, partners need to provide the product description, quality test report (third-party or self-test report), and function test report (if any).
	Service control	1) A notification is sent when the license is about to expire. 2) The system is unavailable when the license expires. 3) After the license is renewed, the system can be used normally and the original data is not deleted.
	Maintainability	1) Centralized Event Management Platform Check 2) Log Management System Check 3) Product Monitoring and Alarm Management System Check 4) Certificate Expiration Alarm Check
Performance		1) Select three to five basic core functions, determine the pressure load (for example, 100 concurrent users) based on the currently used resource specifications, and perform continuous concurrency tests (5 to 10 minutes). 2) Select a hybrid service scenario with two to four system core functions and perform continuous concurrency tests (0.5 to 2 hours). 3) Select any scenario and perform a fatigue pressure test (4 to 6 hours). 4) If hardware products are involved, select three to four hardware service scenarios and perform continuous concurrency tests (0.5 to 2 hours) on the hardware product performance such as a camera's resolution and the number of simultaneous video streaming.
Security	Manual security test cases	1) Execute 34 manual security test cases. 2) API products use tokens or dynamic signatures for authentication. Session- and password-related test cases in manual test cases are not involved.
	Web scanning	1) The open-sourced tool OWASP Zed Attack Proxy (OWASP ZAP) is used for web scanning, and Huawei Cloud Host Security Service (HSS) is used for host scanning. 2) Rectify a fault based on the scan results and suggestions.
	Host scanning
	Binary scanning
Reliability		1) Deploy the HA test environment (data HA, application HA, and security services) based on the reliability checklist (SaaS reliability requirements and License reliability requirements). 2) Design test cases based on the reliability checklist and test services at each fault point. 3) The reliability requirements of the backend service corresponding to the API offering are the same as those of the SaaS.
Integration		1) Based on the solution architecture diagram, test the connectivity of the integration path of Huawei Cloud services involved in the solution to ensure that the service flows and data flows in core service scenarios can be streamlined with the dependent Huawei Cloud services in an end-to-end manner. 2) If hardware products are involved, select three to six core business scenarios of the solution and perform connectivity tests on their service and data flows.

Function, performance, security, reliability, and integration tests have been released on HaydnCSF Case Templates. Select and modify a template by referring to Designing Test Cases.
If hardware products are involved, pay attention to the requirements related to the function test, performance test, and integration test of hardware products in the preceding table.
The certification test is completed on Haydn Cloud Solution Factory (HaydnCSF). Click here to access the HaydnCSF console using Chrome browser (recommended). HaydnCSF is a self-service tool platform that enables partners to quickly build and verify solutions. For details, see What Is HaydnCSF?

Reliability Requirements

Reliability requirements are classified into two levels based on the importance of software systems. You can determine and evaluate reliability levels based on the importance of application systems or the following table.

**Table 2** Reliability requirement levels evaluation
Reliability Requirement Level	Recovery Time Objective (RTO)	Recovery Point Objective (RPO)
Medium	2 hours < RTO < 12 hours	RPO asynchronization in minutes
High	RTO < 2 hours	RPO synchronization in seconds
Description	Indicates the time required for recovering services after a service interruption. For example, if a disaster occurs at 14:00 and the RTO is 2 hours, the DR system must recover services to an acceptable service level before 16:00.	Indicates the amount of acceptable data loss. For example, if a disaster occurs at 14:00 and the RPO is 2 hours, only data loss within this period (between 12:00 and 14:00) is acceptable.

Reliability requirements are classified into SaaS and license reliability requirements based on the deployment mode. See the following table.

SaaS Reliability Requirements

**Table 3** SaaS reliability requirements
Reliability Requirement Item			Medium	High	Description
Data HA (mandatory)	Distributed			√	Database cluster or active/standby. Use the distributed or active/standby mode of the Huawei Cloud database service. You can also use an Elastic Cloud Server (ECS) to build a distributed or active/standby database. In the self-built architecture, the ECS icon must indicate the database to be deployed and the cluster or active/standby relationship.
	Active/standby		√
	Data backup		√	√	If the automated backup policy is enabled for Huawei Cloud databases, the data backup storage location must be specified for self-built databases.
Application HA	Cluster	Single AZ	√		1. A CCE cluster or a self-built ECS cluster (more than two ECSs where the same application is deployed) is used. 2. Middleware (such as Redis) must be displayed in the architecture regardless of whether Huawei advanced services or self-built middleware is used. Middleware must be deployed in active/standby or cluster mode.
	Cluster	Cross-AZ		√
	Data backup		√	√	1. CCE/CCI container deployment + Cloud Backup and Recovery (CBR) 2. ECS + CBR
Security Service	Host security		√	√	Huawei Cloud Host Security Service (HSS), enterprise edition. One ECS for one HSS.
	Anti-DDoS		√	√	Huawei Cloud Anti-DDoS can be used in the test environment. Advanced Anti-DDoS-BGP Pro must be used in the production environment.
	WAF		√	√	In the test environment, you can apply for resources of the basic edition for testing. The production and release requirements are as follows: Medium: WAF standard edition Advanced: WAF professional edition Note: APIs that released through API gateway are not involved.
	Bastion host		√	√	For Huawei Cloud Bastion Host (CBH), use the minimum specifications during the test: 10 hosts, which can be self-built.
System/Audit Log			√	√	Operation logs, security logs, system logs can be recorded and audited. 1. An operation log records operations performed by operators on the system, such as adding, deleting, and modifying resource information. 2. A security log records operations and running information related to system security, such as user login, normal user logout, user locking, user/role addition, deletion, and modification, and system security configuration modification. 3. A system log records operations performed by background services on user resources during system running, such as automatically locking users and periodically clearing user data by the system timer. 4. For the checks that are included in the test cases, Huawei Cloud services are not necessarily required.

License Reliability Requirements

**Table 4** License reliability requirements
Reliability Requirement Item			Medium	High	Description
Data HA (mandatory)	Distributed				Database cluster or active/standby. Use the distributed or active/standby mode of the Huawei Cloud database service. You can also use an ECS to build a distributed or active/standby database. In the self-built architecture, the ECS icon must indicate the database to be deployed and the cluster or active/standby relationship.
	Active/standby		√	√
	Data backup		√	√	If the automated backup policy is enabled for Huawei Cloud databases, the data backup storage location must be specified for self-built databases.
Application HA	Cluster	Single AZ		√	1. A CCE cluster or a self-built ECS cluster (more than two ECSs where the same application is deployed) is used. 2. Middleware (such as Redis) must be displayed in the architecture regardless of whether Huawei advanced services or self-built middleware is used. Middleware must be deployed in active/standby or cluster mode. 3. If the license reliability requirement level is medium, single-node deployment is allowed but the alarm monitoring service must be integrated. You can use Huawei Cloud Eye (CES), Application Operations Management (AOM), third-party, or self-built alarm monitoring services. 4. The license reliability level is high, and the application HA meets either the cluster or active/standby requirements.
	Cluster	Cross-AZ
	Active/standby			√
	Data backup		√	√	CCE/CCI container deployment + CBR Or ECS + CBR
Security Service	Host security		√	√	Huawei Cloud Host Security Service (HSS), enterprise edition. One ECS for one HSS.
	Anti-DDoS		√	√	Huawei Cloud Anti-DDoS
	Bastion host		√	√	1. If resources belong to tenants, no bastion host is required. If resources are maintained by ISVs in a unified manner, configure bastion hosts. 2. For CBH, use the minimum specifications during the test: 10 hosts, which can be self-built.
System/Audit Log			√	√	Operation logs, security logs, and system logs are recorded. 1. An operation log records operations performed by operators on the system, such as adding, deleting, and modifying resource information. 2. A security log records operations and running information related to system security, such as user login, normal user logout, user locking, user/role addition, deletion, and modification, and system security configuration modification. 3. A system log records operations performed by background services on user resources during system running, such as automatically locking users and periodically clearing user data by the system timer. 4. For the checks that are included in the test cases, Huawei Cloud services are not necessarily required.

Example Work Plan for a Certification Test

To ensure that the certification test can be completed efficiently and quickly, arrange R&D personnel in advance to support the test and rectify faults. Understand the test process and make a proper plan before the certification test starts. The following test work plan is for reference only.

**Table 5** Example work plan
Test Phase	Content		Planned Start	Planned End	Workload Reference (Person-Day)	Description
Test Preparations	Register and design a solution.		October 19, 2022	October 19, 2022	1
	Create test requirements, formulate test plans, and design test cases.		October 20, 2022	October 21, 2022	2
	Deploy a test environment.		October 21, 2022	October 21, 2022	1	The test environment must meet reliability requirements. For details, see Reliability Requirements.
Test Execution	Security test	Binary package scanning (independent of the deployment environment)	October 24, 2022	October 24, 2022	0.5	Different types of tests can be performed concurrently. It takes a long time to fix security vulnerabilities. Therefore, use security tools (binary, web, and host scanning) to scan and report security vulnerabilities to R&D engineers for fixing first. If a product has passed other types of Huawei Cloud tests, provide related test materials and evaluate the reuse test conclusion after confirming with the certification test team.
		Web scanning
		Host scanning
		Basic manual test cases	October 24, 2022	October 25, 2022	2
	Reliability test (basic cases)		October 24, 2022	October 24, 2022	0.5
	Performance test		October 24, 2022	October 24, 2022	1
	Function test	Function	October 24, 2022	October 27, 2022	50 cases/person-day. Evaluate based on the actual situation.
		Service control	October 24, 2022	October 27, 2022
		Maintainability	October 24, 2022	October 27, 2022
	Integration test		October 24, 2022	October 27, 2022
Closure	Issue list closure		October 24, 2022	October 27, 2022	/	Evaluate based on the number and difficulty of problems found in the test. Report the problems found during the test to the R&D engineers for rectification.

User Roles

Roles on HaydnCSF:
Enterprise administrators and common users
- Enterprise administrator can use the Operations Center to manage the users of your enterprise, view reports of your enterprise, and analyze verification efficiency.
- Common users are people in different positions of your enterprise, for example, project managers, architects, and test engineers.
Roles on HaydnCSF workspace
A user in a specific HaydnCSF workspace is isolated from each other. A user can have different roles in different workspaces.
- Administrator: The administrator of a workspace. An administrator can manage users in the workspace.
- Project manager: A project manager manages a workspace and the users in the workspace, makes test plans, and reviews solutions, requirements, test cases and reports.
- Architect: An architect registers, designs, and reviews solutions, creates test requirements, and handles test issues on HaydnCSF.
- Test engineer: A test engineer designs and performs test cases, creates test resources, handles test issues, and provides test reports.
- Visitor: A visitor can only view a workspace. You can assign the role to a user when inviting them into the workspace.

For details about roles in HaydnCSF, see HaydnCSF User Access Permission. To add a user and set a role for the user, see Setting an Enterprise Administrator.

Preparations

Before certification, partners need to complete the following operations:

Register with HaydnCSF by referring to Accessing HaydnCSF in the user guide.
Create a workspace by referring to Creating a Workspace.
Add an enterprise user and assign a role to the user.
Invite Huawei engineers to join the workspace.

For details about how to add an enterprise user, assign a role to the user, and invite a Huawei engineer to join the space, see Managing Workspace Members

Parent topic: Certification Tests on HaydnCSF

Previous topic: Certification Tests on HaydnCSF

Next topic: Certification Process and Instructions