Help Center> ModelArts> Troubleshooting> Training Jobs> Service Code Issues> Error Message "pandas.errors.ParserError: Error tokenizing data. C error: Expected .* fields" Displayed in Logs
Updated on 2022-12-08 GMT+08:00
View PDF

Error Message "pandas.errors.ParserError: Error tokenizing data. C error: Expected .* fields" Displayed in Logs

Symptom

When pandas is used to read CSV data, the following error is displayed in logs, and the training job failed: 
pandas.errors.ParserError: Error tokenizing data. C error: Expected 4 field

Possible Causes

The number of columns in each row of the CSV file is different.

Solution

Use either of the following methods to resolve this issue:

  • Check the CSV file and delete the lines with extra columns.
  • Run the following commands to ignore the lines with extra columns:
    import pandas as pd
pd.read_csv(filePath,error_bad_lines=False)

Summary and Suggestions

Before creating a training job, use the ModelArts development environment to debug the training code to maximally eliminate errors in code migration.
Parent topic: Service Code Issues