| Document | docx and pdf | Original document content. |
| Pre-trained text | jsonl | text indicates the text data used for pre-training. The following is an example: {
{"text": "The Road is a 2006 post-apocalyptic novel by American writer Cormac McCarthy. The book details the grueling journey of a father and his young son over a period of several months across a landscape blasted by an unspecified cataclysm that has destroyed industrial civilization and almost all life."
} |
| Single-turn Q&A | jsonl | If the data is in JSONL format, the Alpaca, ShareGPT, and Standard formats are supported. The following are examples of datasets in different formats. - Alpaca format
Non-CoT data: The data consists of Q&A pairs. instruction indicates the question, and output indicates the answer. The following is an example: {
"instruction": "Please recommend a book.",
"input": "",
"output": "Certainly! I recommend you read The Future of Autonomous Driving."
} CoT data: The data consists of Q&A pairs. instruction indicates the question, and output indicates the answer. The output must contain the think tag pair to indicate the thinking process. The following is an example: {
"instruction": "Can you recommend some books to me?"
"input": "",
"output": "<think>The user requests book recommendations but does not provide specific preferences. It is suitable to recommend a technology book that covers a wide range of topics and has a forward-looking perspective.</think>I recommend The Future of Autonomous Driving."
} - ShareGPT format
Non-CoT data: The data consists of Q&A pairs. In a conversation, the values from the human and gpt roles represent the question and answer, respectively. The following is an example: {
"conversations": [
{
"from": "human",
"value": "Can you recommend some books to me?"
},
{
"from": "gpt",
"value": "As an expert in book recommendations, I recommend you read The Future of Autonomous Driving."
}
]
} CoT data: The data consists of Q&A pairs. In a conversation, the values from the human and gpt roles indicate the question and answer, respectively. The value of the gpt role must contain the think tag pair to indicate the thinking process. The following is an example: {
"conversations": [
{
"from": "human",
"value": "Can you recommend some books to me?"
},
{
"from": "gpt",
"value": "<think>As an expert in book recommendations, I should recommend books based on current technological trends and popular reading preferences.</think>I recommend you read The Future of Autonomous Driving."
}
]
} - Standard format
Non-CoT data: The data consists of Q&A pairs. context indicates the question, and target indicates the answer. The following is an example: {
"context": "Hello, please introduce yourself.",
"target": "I am a Pangu model."
} CoT data: The data consists of Q&A pairs. context indicates the question, and target indicates the answer. target must contain the think tag pair to indicate the thinking process. The following is an example: {
"context": "Hello, please introduce yourself.",
"target": "<think>OK. The user asks me to introduce myself. First, I need to clarify the user's identity and usage scenario...</think>I am a Pangu model."
} |
| csv | - Non-CoT data: The first column in the CSV file corresponds to context, and the second column corresponds to target. The following is an example:
"Hello, please introduce yourself.","I am a Pangu model." - CoT data: In the CSV file, the first column corresponds to context, the second column corresponds to target, and target must contain the think tag pair. The following is an example:
"Hello, please introduce yourself","<think> The user asks me to introduce myself. First, I need to clarify the user's identity and usage scenario...</think>I am a Pangu model." |
| Single-turn Q&A (with a system persona) | jsonl | If the data is in JSONL format, the Alpaca, ShareGPT, and Standard formats are supported. The following are examples of datasets in different formats. - Alpaca format
Non-CoT data: system indicates the role, and instruction and output indicate the question and answer, respectively. The following is an example: {
"system": "You are a book recommendation expert.",
"instruction": "Please recommend books based on the user's requirements.",
"input": "",
"output": "As a book recommendation expert, I recommend you read The Future of Autonomous Driving."
} CoT data: system indicates the role, instruction indicates the question, and output indicates the answer. The output must contain the think tag pair to indicate the thinking process. The following is an example: {
"system": "You are a book recommendation expert.",
"instruction": "Please recommend books based on the user's requirements.",
"input": "",
"output": "<think>As a book recommendation expert, I should consider both technological trends and popular reading preferences.</think>I recommend you read The Future of Autonomous Driving."
} - ShareGPT format
Non-CoT data: The data consists of Q&A pairs. system_prompt indicates the role. In a conversation, the values of from the human and gpt roles indicate the question and answer, respectively. The following is an example: {
"system_prompt": "Role name: book recommendation expert.",
"conversations": [
{
"from": "human",
"value": "Can you recommend some books to me?"
},
{
"from": "gpt",
"value": "As an expert in book recommendations, I recommend you read The Future of Autonomous Driving."
}
]
} CoT data: The data consists of Q&A pairs. system_prompt indicates the role. In a conversation, the values corresponding to the human and gpt roles indicate the question and answer, respectively. The value of the gpt role must contain the think tag pair to indicate the thinking process. The following is an example: {
"system_prompt": "Role name: book recommendation expert.",
"conversations": [
{
"from": "human",
"value": "Can you recommend some books to me?"
},
{
"from": "gpt",
"value": "<think>As an expert in book recommendations, I should recommend books based on current technological trends and popular reading preferences.</think>I recommend you read The Future of Autonomous Driving."
}
]
} - Standard format
Non-CoT data: The data consists of Q&A pairs. system indicates the role. context indicates the question, and target indicates the answer. The following is an example: {
"system": "Witty and humorous",
"context": "Hello, please introduce yourself.",
"target": "Haha, hello. I'm your smart assistant."
} CoT data: The data consists of Q&A pairs. system indicates the role. context indicates the question, and target indicates the answer. target must contain the think tag pair to indicate the thinking process. The following is an example: {
"system": "Witty and humorous",
"context": "Hello, please introduce yourself.",
"target": "<think>OK. The user asks me to introduce myself. First, I need to clarify the user's identity and usage scenario...</think>I am a Pangu model."
} |
| csv | - CSV non-CoT data: The first column in the CSV file corresponds to system, and the second and third columns correspond to context and target, respectively. The following is an example:
"You're a smart and humorous Q&A assistant.","Hello, please introduce yourself.","Hello. I'm your smart assistant." - CSV CoT data: In the CSV file, the first column corresponds to system, and the second and third columns correspond to context and target, respectively. target must contain the think tag pair to indicate the thinking process. The following is an example:
"You are a smart and humorous Q&A assistant.","context":"Hello, please introduce yourself.","<think>The user asks me to introduce yourself. First, I need to clarify the user's identity and usage scenario.</think>Hello. I'm your smart assistant." |
| Multi-turn Q&A | jsonl | If the data is in JSONL format, the Alpaca, ShareGPT, and Standard formats are supported. The following are examples of datasets in different formats. - Alpaca format
Non-CoT data: Array format. It consists of one or more turns of Q&A pairs. instruction indicates the question, output indicates the answer, and history indicates the historical conversation. The following is an example: {
"instruction": "I'm looking for books related to autonomous driving.",
"input": "",
"output": "I recommend you read The Future of Autonomous Driving.",
"history": [
[
"I'm looking for a book that can help me understand future technology trends.",
"Future technologies cover a wide range. Which areas are you more interested in?"
]
]
} CoT data: Array format. It consists of one or more turns of Q&A pairs. instruction and output indicate the question and answer, respectively. history indicates the historical conversation. The think tag pair indicates the thinking process. The following is an example: {
"instruction": "I'm looking for books related to autonomous driving.",
"input": "",
"output": "<think>Since the user has specified the field, I should recommend a representative book with a complete structure and a balanced perspective on both technology and industry.</think>I recommend you read The Future of Autonomous Driving.",
"history": [
[
"I'm looking for a book that can help me understand future technology trends.",
"<think>As a book recommendation expert, the first step should be to confirm the specific technology field the user is interested in, rather than directly providing a book title.</think>Future technologies cover a wide range of fields. Could you tell me which genres or topics interest you most?"
]
]
} - ShareGPT format
Non-CoT data: Array format. It consists of one or more turns of Q&A pairs. In a conversation, the values corresponding to the human and gpt roles indicate the question and answer, respectively. The following is an example: {
"conversations": [
{
"from": "human",
"value": "Hello"
},
{
"from": "gpt",
"value": "Hi! How can I help you?"
},
{
"from": "human",
"value": "Can you recommend some books to me?"
},
{
"from": "gpt",
"value": "Certainly! Based on your interests, I recommend you read The Future of Autonomous Driving."
}
]
} CoT data: Array format. It consists of one or more turns of Q&A pairs. The values corresponding to human and gpt roles in a conversation indicate the question and answer, respectively. The value corresponding to the gpt role contains the think tag pair to indicate the thinking process. The following is an example: {
"conversations": [
{
"from": "human",
"value": "I want to read a book about future technology trends."
},
{
"from": "gpt",
"value": "<think>I should confirm the specific field first.</think>Future technologies cover a wide range. Which areas are you more interested in?"
},
{
"from": "human",
"value": "Autonomous driving."
},
{
"from": "gpt",
"value": "<think>I should recommend a representative book in the specified field. </think>I recommend The Future of Autonomous Driving."
}
]
} - Standard format
Non-CoT data: Array format. It consists of one or more turns of Q&A pairs. context indicates the question, and target indicates the answer. The following is an example: [
{
"context": "Hello",
"target": "Hello. How can I help you?"
},
{
"context": "Please introduce Huawei Cloud products.",
"target": "Huawei Cloud provides products and services including but not limited to compute, storage, and network."
}
] CoT data: Array format. It consists of one or more turns of Q&A pairs. context and target indicate the question and answer, respectively. target of at least one turn of Q&A contains the think tag pair, indicating the thinking process. The following is an example: [
{
"context": "Hello",
"target": "Hello. How can I help you?"
},
{
"context": "Please introduce Huawei Cloud products.",
"target": "<think>Okay, the user asked me to introduce Huawei Cloud products. First, I need to recall...</think>Huawei Cloud provides products and services including but not limited to compute, storage, and networking."
}
] |
| Multi-turn Q&A (with a system persona) | jsonl | If the data is in JSONL format, the Alpaca, ShareGPT, and Standard formats are supported. The following are examples of datasets in different formats. - Alpaca format
Non-CoT data: Array format. It consists of one or more turns of Q&A pairs. system indicates the role, instruction indicates the question, output indicates the answer, and history indicates the historical conversation. The following is an example: {
"system": "You are a book recommendation expert.",
"instruction": "I'm looking for books related to autonomous driving.",
"input": "",
"output": "I recommend you read The Future of Autonomous Driving.",
"history": [
[
"I'm looking for a book that can help me understand future technology trends.",
"Future technologies cover a wide range. Which areas are you more interested in?"
]
]
} CoT data: Array format. It consists of one or more turns of Q&A pairs. system indicates the role. instruction and output indicate the question and answer, respectively. history indicates the historical conversation. output must contain the think tag pair to indicate the thinking process. The format is as follows: {
"system": "You are a book recommendation expert.",
"instruction": "I'm looking for books related to autonomous driving.",
"input": "",
"output": "<think>Since the user has specified the field, I should recommend a representative book with a complete structure and a balanced perspective on both technology and industry.</think>I recommend you read The Future of Autonomous Driving.",
"history": [
[
"I'm looking for a book that can help me understand future technology trends.",
"<think>As a book recommendation expert, the first step should be to confirm the specific technology field the user is interested in, rather than directly providing a book title.</think>Future technologies cover a wide range of fields. Could you tell me which genres or topics interest you most?"
]
]
} - ShareGPT format
Non-CoT data: Array format. It consists of one or more turns of Q&A pairs. system_prompt indicates the role. In the conversation, the values corresponding to the human and gpt roles indicate the question and answer, respectively. The following is an example: {
"system_prompt": "Book recommendation expert",
"conversations": [
{
"from": "human",
"value": "Hello"
},
{
"from": "gpt",
"value": "Hello, what can I help you with?"
},
{
"from": "human",
"value": "What book do you recommend?"
},
{
"from": "gpt",
"value": "I recommend The Future of Autonomous Driving."
}
]
} CoT data: Array format. It consists of one or more turns of Q&A pairs. system_prompt indicates the role. The values corresponding to human and gpt roles in a conversation indicate the question and answer, respectively. The value corresponding to the gpt role contains the think tag pair to indicate the thinking process. The following is an example: {
"system_prompt": "Book recommendation expert",
"conversations": [
{
"from": "human",
"value": "I want to read a book about future technology trends."
},
{
"from": "gpt",
"value": "<think>I should confirm the specific field first.</think>Future technologies cover a wide range. Which areas are you more interested in?"
},
{
"from": "human",
"value": "Autonomous driving."
},
{
"from": "gpt",
"value": "<think>I should recommend a representative book in the specified field. </think>I recommend The Future of Autonomous Driving."
}
]
} - Standard format
Non-CoT data: Array format. It consists of one or more turns of Q&A pairs. system indicates the persona, context indicates the question, and target indicates the answer. The following is an example: [
{
"system": "Witty and humorous"
},
{
"context": "Hello, please introduce yourself.",
"target": "Haha, hello. I'm your smart assistant."
}
] CoT data: Array format. It consists of one or more turns of Q&A pairs. system indicates the role. context and target indicate the question and answer, respectively. target of at least one turn of Q&A contains the think tag pair, indicating the thinking process. The following is an example: [
{
"system": "Witty and humorous"
},
{
"context": "Hello, please introduce yourself.",
"target": "<think>Okay, the user asks me to introduce myself. First, I need to clarify the user's identity and usage scenario...</think> Haha, hello! I'm your smart assistant."
}
] |
| Q&A sorting | jsonl | context indicates the question. The order of targets 1, 2, and 3 represent the order of human-preferred answers. The most preferred answer is placed at the forefront. {
"context": "context content,"
"targets": [
"a",
"b",
"c"
]
} |
| csv | - CSV format: The first column in the CSV file corresponds to context, and the other columns are answers.
"How do you understand the line 'A thousand sails pass by the sunken ship'?","This line is more than a beautiful image that describes the scene where new ships still pass by the sunken ship like a thousand sails. It contains profound philosophy and expresses the natural law that new things will inevitably replace old things.","This line can be used to describe natural scenery, expressing the grandeur of nature and the vitality of life." |
| Direct Preference Optimization (DPO) | jsonl | - Non-CoT data: context indicates the question, target indicates the expected correct answer, and bad_target indicates the incorrect answer that does not meet the expectation. The following is an example:
Single-turn DPO {
"context": [
"Hello, please introduce yourself."
],
"target": "I am a Pangu model,"
"bad_target": "I cannot answer this question."
} Multi-turn DPO {
"context": [
"Hello, please introduce yourself.",
"I am a Pangu model,"
"Please introduce Huawei Cloud products."
],
"target": "Huawei Cloud provides products and services including but not limited to compute, storage, and network.",
"bad_target": "I cannot answer this question."
} - CoT data: context indicates the question, target indicates the expected correct answer, and bad_target indicates the incorrect answer that does not meet the expectation. At least one answer contains the think tag pair, indicating the thinking process. The following is an example:
Single-turn DPO {
"context": [
"Hello, please introduce yourself."
],
"target": "<think>OK. The user asks me to introduce myself. First, I need to clarify the user's identity and usage scenario...</think>I am a Pangu model.",
"bad_target": "<think>OK. The user asks me to introduce myself...</think>I cannot answer this question."
} Multi-turn DPO {
"context": [
"Hello, please introduce yourself.",
"I am a Pangu model,"
"Please introduce Huawei Cloud products."
],
"target": "<think>Okay, the user asked me to introduce Huawei Cloud products. First, I need to recall...</think>Huawei Cloud provides products and services including but not limited to compute, storage, and networking.",
"bad_target": "<think> Okay, the user asked me to introduce Huawei Cloud products...</think>I cannot answer this question."
} |
| DPO (with a system persona) | jsonl | - Non-CoT data: system indicates the role. context indicates the question, target indicates the expected correct answer, and bad_target indicates the incorrect answer that does not meet the expectation. The following is an example:
Single-turn DPO with persona {
"system": "You are a witty and humorous Q&A assistant.",
"context": [
"Hello, please introduce yourself."
],
"target": "Haha, hello. I'm your smart assistant. How can I help you?",
"bad_target": "I cannot answer this question."
} Multi-turn DPO with persona {
"system": "You are a witty and humorous Q&A assistant.",
"context": [
"Hello, please introduce yourself.",
"Haha, hello. I'm your smart assistant. How can I help you?",
"Please introduce your products."
],
"target": "We offer a wide range of products, including compute, storage, and network products.",
"bad_target": "I cannot answer this question."
} - CoT data: system indicates the role. context indicates the question, target indicates the expected correct answer, and bad_target indicates the incorrect answer that does not meet the expectation. At least one correct answer contains the think tag pair, indicating the thinking process. The following is an example:
Single-turn DPO with persona {
"system": "You are a witty and humorous Q&A assistant.",
"context": [
"Hello, please introduce yourself."
],
"target": "<think> The user asked me to introduce myself. First, I need to clarify the user's identity and usage scenario.</think>Haha, hello! I'm your smart assistant. How can I help you?",
"bad_target": "I cannot answer this question."
} Multi-turn DPO with persona {
"system": "You are a witty and humorous Q&A assistant.",
"context": [
"Hello, please introduce yourself.",
"Haha, hello. I'm your smart assistant. How can I help you?",
"Please introduce your products."
],
"target": "<think>The customer wants to know about our products.</think>We offer a wide range of products, including compute, storage, and network products.",
"bad_target": "I cannot answer this question."
} |
