Help Center/ PanguLargeModels/ FAQs/ FAQs Related to LLM Fine-Tuning and Training/ Why Is the Performance of the Fine-Tuned Pangu Model Unsatisfactory in Multi-Turn Dialogues?
Updated on 2025-11-04 GMT+08:00
View PDF
Share

Why Is the Performance of the Fine-Tuned Pangu Model Unsatisfactory in Multi-Turn Dialogues?

A model has been fine-tuned on multi-turn dialogue data, but its performance on multi-turn dialogues is still satisfactory. Locate the fault as follows:

  • Check the data format. The question field in a new turn multi-turn dialogue data must contain all Q&A pairs of the historical turns of the dialogue. For example, the question field in the third turn must contain the Q&A pairs of the first and second turns as well as the question of the third turn. The answer field is the answer of the third turn. The following provides examples of multi-turn dialogue data:

    Example of the original dialogue:

    A: Who are you?
B: Hello, I'm a Pangu model.
A: What can you do?
B: I can perform various language-based tasks, such as xxxx.
A: Can you tell a joke?
B: Yes, of course. xxx
A: Can you change this joke to xxxx?
B: OK. Here is the modified version: xxxx.

    The following is an example of the data format required for Pangu model training based on ModelArts Studio:

    [{"context": "Who are you?", "target": "Hello, I am a Pangu model."}, {"context": "What can you do?", "target": "I can perform various language-based tasks, such as xxxx."}, {"context": "Can you tell a joke?", "target": "Yes, of course. xxx"}, {"context": "Can you change this joke to xxxx?", "target": "OK. Here is the modified version: xxxx."}]

    The input of the multi-turn Q&A scenario is in array format and consists of at least one Q&A pair. The format is [{"context":"context content 1","target":"target content 1"},{"context":"context content 2","target":"target content 2"}]. context and target indicate the question and answer, respectively. The following are examples:

    Example of the original dialogue:

    A: Agent xxx is at your service.
A: Hello, sir. What can I do for you?
B: Hello.
B: I can't access the Internet at home.
B: The Internet is not connected.
A: Sir, you couldn't connect the Internet at home, right?
A: Did you try to remove and insert the network cable again?
B: Yes, I did.
B: But it still doesn't work.

    The following is an example of the data format required for Pangu model training based on ModelArts Studio:

    [{"context": "Agent xxx is at your service. Hello, sir. What can I do for you?", "target": "Hello. I can't access the Internet at home. The Internet is not connected."}, {"context": "Sir, you couldn't connect the Internet at home, right? Did you try to remove and insert the network cable again?", "target": "Yes, I did. But it still doesn't work."}]
  • Check the data quality. If the data format is correct, you can improve the data quality based on identified problems. For example, as the number of turns increases, the model may forget certain historical data. This is because too few turns are included in most dialogue examples in the constructed training data. You are advised to increase the number of dialogue turns in the dataset as required.