Help Center/ ModelArts/ Data Preparation/ Data Refining/ Smart Refining/ Smart Refining Templates
Updated on 2026-08-28 GMT+08:00
View PDF
Share

Smart Refining Templates

Description

Smart refining templates on ModelArts are best practices for data processing. They combine complex operations into simple workflows you can use with one click.

Core functions:

  1. You do not need to start from scratch. You can reuse tested solutions from experienced engineers to ensure your data processing is professional and logical.
  2. You only need to select a template that fits your service scenario when creating a task. The system will automatically load the full operator link. You can run the workflow directly or fine-tune it based on data characteristics, reducing data preparation time from hours to minutes.
  3. The template-based system standardizes data processing, prevents quality issues from personal settings, and ensures high-quality training sets.

Built-in Templates

ModelArts provides the following built-in templates, covering the processing of text, image, and video data. For details about how to use these templates, see Table 1.

Table 1 Built-in templates

Template Name

Supported Dataset Modality

Supported Dataset Type

Use Case

Involved Operators

Text generation SFT processing workflow

Text

Text generation SFT

Use built-in operators to process text generation SFT data and generate cleaned fine-tuning data.

Personal Data Anonymization

Switching Between Simplified Chinese and Traditional Chinese

Symbol Standardization

Sensitive Word Filtering

Filtering of the Incomplete Sentence at the End of a Paragraph

N-gram Feature Filtering

Text Length Filtering

Word processing workflow

Text

Base text

Use built-in operators to extract and process Word files and generate pre-trained text data.

Word Document Content Extraction

Personal Data Anonymization

Switching Between Simplified Chinese and Traditional Chinese

Symbol Standardization

Sensitive Word Filtering

Filtering of the Incomplete Sentence at the End of a Paragraph

N-gram Feature Filtering

Text Length Filtering

Pre-trained Text Classification

PDF processing workflow

Text

Base text

Use built-in operators to extract and process PDF files and generate pre-trained text data.

PDF Content Extraction

Personal Data Anonymization

Switching Between Simplified Chinese and Traditional Chinese

Symbol Standardization

Sensitive Word Filtering

Filtering of the Incomplete Sentence at the End of a Paragraph

N-gram feature filtering

Text Length Filtering

Pre-trained Text Classification

Constructing fine-tuning data using PDF documents

Text

Base text

Intelligently parse PDF document content, and obtain appropriate data and generate fine-tuning data through text cleaning, deduplication, and quality filtering, providing high-quality samples for subsequent LLM training.

PDF Content Extraction

Personal Data Anonymization

Switching Between Simplified Chinese and Traditional Chinese

Symbol Standardization

Filtering of the Incomplete Sentence at the End of a Paragraph

Text Length Filtering

Data Generation

Pre-trained text processing workflow

Text

Pre-trained text

Use built-in operators to process pre-trained text and generate cleaned pre-trained text data.

Switching between simplified Chinese and traditional Chinese

Symbol Standardization

N-gram feature filtering

Text Length Filtering

Pre-trained Text Classification

Image processing workflow

Image

Base image

Use built-in operators to extract, deduplicate, label, and filter images, and generate summaries to produce processed image and text data.

Image and Text Extraction

Image Metadata Filtering

Image Deduplication

Dangerous Situation Image Detection

Pornographic Image Detection

Violent and Terrorism Image Detection

Video processing workflow

Video

Base video

Use built-in operators to split, deduplicate, label, and filter video clips, and generate processed video data.

Video Clipping

Video Metadata Filtering

Video Aspect Ratio Filtering

Pornographic Video Detection

Terrorism Video Detection

Political Video Detection

Motion Range Scoring

Aesthetics Scoring

Single-turn Q&A process

Text

Single-turn Q&A

Use built-in operators to process single-turn Q&A data and generate cleaned single-turn Q&A data.

Personal Data Anonymization

Switching between simplified Chinese and traditional Chinese

Symbol Standardization

Sensitive Word Filtering

Filtering of the Incomplete Sentence at the End of a Paragraph

N-gram feature filtering

Text Length Filtering
Parent Topic: Smart Refining