Help Center>
Data Lake Insight>
FAQs>
Problems Related to Spark Jobs>
Job Development>
How Does a Spark Job Access a MySQL Database?
Updated on 2023-08-03 GMT+08:00
How Does a Spark Job Access a MySQL Database?
You can use DLI Spark jobs to access data in the MySQL database using either of the following methods:
- Solution 1: Buy a yearly/monthly or pay-per-use dedicated queue, create an enhanced datasource connection, and read data from the MySQL database through a datasource table. You need to write Java or Scala code to implement this solution.
- Solution 2: Use CDM to import data from the MySQL database to an OBS bucket, and then use a Spark job to read data from the OBS bucket. If you already have a CDM cluster, this solution is simpler than solution 1 and does not involve any other database.
Parent topic: Job Development
Job Development FAQs
- How Do I Use Spark to Write Data into a DLI Table?
- How Do I Set the AK/SK for a Queue to Operate an OBS Table?
- How Do I View the Resource Usage of DLI Spark Jobs?
- How Do I Use Python Scripts to Access the MySQL Database If the pymysql Module Is Missing from the Spark Job Results Stored in MySQL?
- How Do I Run a Complex PySpark Program in DLI?
- How Does a Spark Job Access a MySQL Database?
- How Do I Use JDBC to Set the spark.sql.shuffle.partitions Parameter to Improve the Task Concurrency?
- How Do I Read Uploaded Files for a Spark Jar Job?
Feedback
Was this page helpful?
Provide feedbackThank you very much for your feedback. We will continue working to improve the documentation.
The system is busy. Please try again later.
more