Help Center/ ModelArts/ Best Practices/ LLM Training/ Adapting Mainstream Open-Source Models to AscendFactory NPU Training Based on Lite Server/ Training Service Configurations/ Modifying the Tokenizer File
Updated on 2025-11-04 GMT+08:00
View PDF
Share

Modifying the Tokenizer File

Adjust the model's tokenizer file before starting the training, depending on your chosen framework.

For now, you only need to update the tokenizer for the GLM4-9B model and the InternVL2_5 series within the LlaMA-Factory framework. Edit the tokenizer file directly.

LlaMA-Factory

  • glm4-9b Model

    Before the training starts, modify the tokenizer file modeling_chatglm.py in the glm4-9b model. The procedure is as follows:

    Go to the tokenizer (weight) directory and modify the content of the modeling_chatglm.py file. 
    vim modeling_chatglm.py
# Comment out the following content:
# if attention_mask is not None
      # attention_mask = ~attention_mask

    Example

  • InternVL2_5 series Models

    Install the patch manually since the InternVL2_5 series models are not available in the Transformers repository. Follow these steps:

    • InternVL2_5-8B: 
      # Download the model file using git clone. If the model file has been downloaded, skip this step.
git clone https://huggingface.co/OpenGVLab/InternVL2_5-8B
# Go to the weight directory and install the patch file. ${work_dir} indicates the working directory. Change it based on the site requirements.
cd InternVL2_5-8B && git apply ${work_dir}/llm_train/AscendFactory/third-party/internvl25-8b.patch
    • InternVL2_5-38B: 
      # Download the model file using git clone. If the model file has been downloaded, skip this step.
git clone https://huggingface.co/OpenGVLab/nternVL2_5-38B
# Go to the weight directory and install the patch file. ${work_dir} indicates the working directory. Change it based on the site requirements.
cd InternVL2_5-38B && git apply ${work_dir}/llm_train/AscendFactory/third-party/internvl25-38b.patch
    • InternVL2_5-78B: 
      # Download the model file using git clone. If the model file has been downloaded, skip this step.
git clone https://huggingface.co/OpenGVLab/InternVL2_5-78B
# Go to the weight directory and install the patch file. ${work_dir} indicates the working directory. Change it based on the site requirements.
cd InternVL2_5-78B && git apply ${work_dir}/llm_train/AscendFactory/third-party/internvl25-8b.patch