Creating a Prompt Evaluation Dataset

Before evaluating the prompt outcomes, upload a prompt variable data file to create an evaluation dataset.

Prompt variables are placeholders within a prompt that can be replaced with specific value. They are used for generating different content based on different scenarios or user inputs. The variable name can be any text, pertaining to the meaning or function of the variable.

Constraints on the Prompt Evaluation Dataset

The file to be uploaded must be in XLSX format.
The number of rows in the dataset must range from 10 to 50.
The table headers in the dataset must be unique, and the number of table headers must be less than 20.
The length of a single text data record cannot exceed 1,000 characters.

The platform verifies the dataset file you upload according to the preceding constraints during creation.

The following shows a reference format and example.

**Table 1** Reference format
key1	key2	key3	result
Group 1 - v1	Group 1 - v2	Group 1 - v3	Group 1 - r
Group 2 - v1	Group 2 - v2	Group 2 - v3	Group 2 - r
Group 3 - v1	Group 3 - v2	Group 3 - v3	Group 3 - r

**Table 2** Example
comment	result
How to correct misspelled words?	Neutral
This effect is not good.	Negative
I like this style.	Positive
Are you worried about something?	Neutral

Creating a Prompt Evaluation Dataset

Log in to ModelArts Studio and access a workspace.
In the navigation pane, choose Agent Dev. On the displayed page, choose Prompt Engineering > Dataset Management. On the displayed page, click Create a Prompt Case Set in the upper right corner.
Figure 1 Prompt case management
Upload the dataset on the Creating Datasets page.
Figure 2 Creating a prompt evaluation dataset