Updated on 2024-11-20 GMT+08:00

Drill Tasks

Scenarios

Manage chaos drill tasks and view drill records.

Creating a Drill Task

  1. Log in to COC.
  2. In the navigation pane on the left, choose Resilience Center > Chaos Drill. On the displayed page, click the Drill Tasks tab.
  3. Click Create Task. Or you can accept a drill plan to access the page for creating a drill task by following the instructions in Drill Plan.

    Figure 1 Creating a drill task

  4. Enter the basic information about the drill task, including the drill task name and expected recovery duration (in minutes).

    Expected Recovery Duration (Minutes): indicates the expected time for the application to automatically recover or recover to the normal state during emergency plan execution after a fault is injected. This time does not affect the drill task.
    Figure 2 Basic information of a drill task

  5. Select an attack task. By default, there is one attack task group. You can click Create Task Group to add a task group or click Create Attack Task to access the page for creating an attack task.

    Figure 3 Selecting an attack task

    1. Tasks between different task groups are executed in serial mode, and tasks in a task group are executed in parallel mode.

    2. Currently, multiple fault injections for the same resource in a task group are not supported.

  6. Add an attack task. You can create an attack task or select an existing attack task. If you have not created an attack task before, you need to click Create Attack Task. However, if you have created attack tasks previously, you can select Select from Existing.
  7. To create an attack task, you need to select an attack target, an attack scenario, and configure a monitoring task (optional). Different attack targets correspond to different attack scenarios. Enter the attack task name. The attack target can be Elastic Cloud Server (ECS), Cloud Container Engine (CCE), Relational Database Service (RDS), Distributed Cache Service (DCS), or Document Database Service (DDS). Click Next. (The following uses an ECS as an example. Select an ECS instance of the application to be attacked.)

    Figure 4 Selecting ECS as the attack target source

  8. Select an attack scenario, set attack parameters, and click OK. The scenarios include Host Resource, Host Process, and Host Network.

    Figure 5 ECS attack scenarios

  9. You can configure drill monitoring task metric, including stable-status metrics and monitoring metrics. Stable-status metrics are key metrics used to measure whether applications are running properly during the drill. If the stable-status metrics are not within the upper and lower limits before or during the drill, the drill automatically stops. Monitoring metrics are used to monitor some service metrics during the drill. You can determine drill risks and whether applications are running properly based on the monitoring data. You can configure a monitoring task by specifying the host in the attack target, the name of the monitored metric, and the upper and lower limits of the metric.

    Figure 6 ECS attack scenario drill monitoring configuration

  10. If you select Cloud Container Engine (CCE) as the attack target source, you need to select an application and POD (select a cluster, namespace, workload type, and workload in sequence). You can specify PODs or the number of PODs. If the number of PODs are specified, the random policy is used. For example, if you set the quantity to 10, 10 PODs will be randomly selected for fault injection.) Then, click Next.

    Figure 7 Selecting CCE as the attack target source and specifying a pod
    Figure 8 Selecting CCE as the attack target source and specifying the quantity

  11. Select a CCE attack scenario, set attack parameters, and click OK. The scenarios include Weapons Attacking POD Instances, Weapons Attacking POD Processes, and Weapons Attacking the POD Network.

    Figure 9 CCE attack scenarios

  12. If you select RDS as the attack source, select an RDS DB instance and click Next.

    Figure 10 Selecting RDS as the attack target

  13. Select an RDS attack scenario, set attack parameters, and click OK.

    Figure 11 Cloud Database (RDS) attack scenarios

  14. If you select DCS as the attack source, select a DCS instance and click Next.

    Figure 12 Selecting Distributed Cache Service (DCS) as the attack target

  15. Select the DCS attack scenario, set required parameters, and click OK.

    Figure 13 DCS attack scenarios

  16. If you select DDS as the attack target, select a DDS instance and click Next.

    Figure 14 Selecting Document Database Service (DDS) as the attack target

  17. Select the DDS attack scenario and click OK.

    Figure 15 Document Database Service (DDS) attack scenario

  18. If you select Select from Existing, select the created attack task from the task list below and click OK.

    Figure 16 Selecting an existing attack task

  19. Click OK. The drill task is created.

    Figure 17 Clicking OK

Editing a Drill Task

You can edit a drill task. However, if a drill record has been generated for the drill task, the task cannot be edited.

  1. Log in to COC.
  2. In the navigation pane on the left, choose Resilience Center > Chaos Drill. On the displayed page, click the Drill Tasks tab.
  3. Locate the target task, choose More > Modify in the Operation column to modify the basic information about the drill task.

    Figure 18 Clicking Modify

  4. You can add a task group, add an attack task, or delete an existing attack task. An existing attack task cannot be modified.
  5. Click OK.

    Figure 19 Modifying a drill task

Deleting a Drill Task

Delete a created drill task. A task that has generated drill records or has associated with drill plans cannot be deleted.

  1. Log in to COC.
  2. In the navigation pane on the left, choose Resilience Center > Chaos Drill. On the displayed page, click the Drill Tasks tab.
  3. Locate the target drill task, choose More > Delete in the Operation column.

    Figure 20 Drill task list

  4. In the displayed dialog box, click OK.

    Figure 21 Deleting a drill task

Starting a Drill Task

Start a drill task.

  1. Log in to COC.
  2. In the navigation pane on the left, choose Resilience Center > Chaos Drill. On the displayed page, click the Drill Tasks tab.
  3. Locate the target drill task, click Start Drill in the Operation column.

    Figure 22 Starting a drill task

  4. Click Drill Record in the Operation column to view the attack progress, including probe installation, drill execution, and environment clearance. The system automatically executes the drill task. The execution time depends on the attack time of the weapon.

    Figure 23 Attack progress
    Figure 24 Attack completed

  5. During the drill task execution, you can click Terminate Drill to end the drill task, click Retry to retry the current step, or click Skip to skip the current step and go to the next step. If you have configured a drill monitoring task when creating the attack task, you can click Monitor to view the real-time monitoring data of the attack target.

    Figure 25 Drill monitoring data

  6. Click Details to view attack details.

    Figure 26 Attack details

Viewing Drill Records

View the drill records of a drill task. A drill task that has not been drilled does not contain drill record.

  1. Log in to COC.
  2. In the navigation pane on the left, choose Resilience Center > Chaos Drill. On the displayed page, click the Drill Tasks tab.
  3. Locate the target drill task and click Drill Record in the Operation column.

    Figure 27 Drill task list

  4. The basic information about the drill task includes the drill task name, drill task ID, attack details, and failure mode. All drill records include the drill record ID, execution status, executor, drill start time, and drill end time.

    Figure 28 Drill Records

  5. Click View Progress to view the attack progress and attack details of the current drill task.

    Figure 29 Attack progress

  6. Click Generate Drill Report to create or view a drill report. For details, see Drill Report.

    Figure 30 Viewing a drill report