What Should I Do If Some or All Historical Database Records Are Missing During Full Synchronization from Oracle and Error Message "The db history topic or its content is fully or partially missing. Please check database history topic configuration and re-execute the snapshot." Is Displayed?

Symptom

During full synchronization from Oracle, some or all historical database records are missing. The JobManager or TaskManager log contains error message "The db history topic or its content is fully or partially missing. Please check database history topic configuration and re-execute the snapshot."

on flink-69782-taskmanager-1-1-00b332b4-fbef-48a1-a584-c15fde32f140 @ 10.16.32.89 (dataPort=38433).java.lang.RuntimeException: One or more fetchers have encountered exceptionat org.apache.flink.connector.base.source.reader.fetcher.SplitFetcherManager.checkErrors(SplitFetcherManager.java:225) ~[flink-connector-files-1.15.0-h0.cbu.dli.330.r19.jar:1.15.0-h0.cbu.dli.330.r19]at org.apache.flink.connector.base.source.reader.SourceReaderBase.getNextFetch(SourceReaderBase.java:169) ~[flink-connector-files-1.15.0-h0.cbu.dli.330.r19.jar:1.15.0-h0.cbu.dli.330.r19]at org.apache.flink.connector.base.source.reader.SourceReaderBase.pollNext(SourceReaderBase.java:130) ~[flink-connector-files-1.15.0-h0.cbu.dli.330.r19.jar:1.15.0-h0.cbu.dli.330.r19]at org.apache.flink.streaming.api.operators.SourceOperator.emitNext(SourceOperator.java:386) ~[flink-dist-1.15.0-h0.cbu.dli.330.r19.jar:1.15.0-h0.cbu.dli.330.r19]at org.apache.flink.streaming.runtime.io.StreamTaskSourceInput.emitNext(StreamTaskSourceInput.java:68) ~[flink-dist-1.15.0-h0.cbu.dli.330.r19.jar:1.15.0-h0.cbu.dli.330.r19]at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:65) ~[flink-dist-1.15.0-h0.cbu.dli.330.r19.jar:1.15.0-h0.cbu.dli.330.r19]at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:539) ~[flink-dist-1.15.0-h0.cbu.dli.330.r19.jar:1.15.0-h0.cbu.dli.330.r19]at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:216) ~[flink-dist-1.15.0-h0.cbu.dli.330.r19.jar:1.15.0-h0.cbu.dli.330.r19]at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:829) ~[flink-dist-1.15.0-h0.cbu.dli.330.r19.jar:1.15.0-h0.cbu.dli.330.r19]at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:778) ~[flink-dist-1.15.0-h0.cbu.dli.330.r19.jar:1.15.0-h0.cbu.dli.330.r19]at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:958) ~[flink-dist-1.15.0-h0.cbu.dli.330.r19.jar:1.15.0-h0.cbu.dli.330.r19]at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:937) ~[flink-dist-1.15.0-h0.cbu.dli.330.r19.jar:1.15.0-h0.cbu.dli.330.r19]at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:751) ~[flink-dist-1.15.0-h0.cbu.dli.330.r19.jar:1.15.0-h0.cbu.dli.330.r19]at org.apache.flink.runtime.taskmanager.Task.run(Task.java:573) ~[flink-dist-1.15.0-h0.cbu.dli.330.r19.jar:1.15.0-h0.cbu.dli.330.r19]at java.lang.Thread.run(Thread.java:750) ~[?:1.8.0_372]Caused by: java.lang.RuntimeException: SplitFetcher thread 0 received unexpected exception while polling the recordsat org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher.runOnce(SplitFetcher.java:150) ~[flink-connector-files-1.15.0-h0.cbu.dli.330.r19.jar:1.15.0-h0.cbu.dli.330.r19]at org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher.run(SplitFetcher.java:105) ~[flink-connector-files-1.15.0-h0.cbu.dli.330.r19.jar:1.15.0-h0.cbu.dli.330.r19]at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_372]at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_372]at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_372]at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_372]... 1 moreCaused by: org.apache.inlong.sort.cdc.oracle.shaded.io.debezium.DebeziumException: The db history topic or its content is fully or partially missing. Please check database history topic configuration and re-execute the snapshot.at org.apache.inlong.sort.cdc.oracle.shaded.io.debezium.relational.HistorizedRelationalDatabaseSchema.recover(HistorizedRelationalDatabaseSchema.java:47) ~[?:?]at org.apache.inlong.sort.cdc.oracle.source.reader.fetch.OracleSourceFetchTaskContext.validateAndLoadDatabaseHistory(OracleSourceFetchTaskContext.java:245) ~[?:?]at org.apache.inlong.sort.cdc.oracle.source.reader.fetch.OracleSourceFetchTaskContext.configure(OracleSourceFetchTaskContext.java:110) ~[?:?]at org.apache.inlong.sort.cdc.oracle.shaded.org.apache.inlong.sort.cdc.base.source.reader.external.IncrementalSourceStreamFetcher.submitTask(IncrementalSourceStreamFetcher.java:81) ~[?:?]at org.apache.inlong.sort.cdc.oracle.shaded.org.apache.inlong.sort.cdc.base.source.reader.IncrementalSourceSplitReader.checkSplitOrStartNext(IncrementalSourceSplitReader.java:142) ~[?:?]at org.apache.inlong.sort.cdc.oracle.shaded.org.apache.inlong.sort.cdc.base.source.reader.IncrementalSourceSplitReader.fetch(IncrementalSourceSplitReader.java:73) ~[?:?]at org.apache.flink.connector.base.source.reader.fetcher.FetchTask.run(FetchTask.java:58) ~[flink-connector-files-1.15.0-h0.cbu.dli.330.r19.jar:1.15.0-h0.cbu.dli.330.r19]at org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher.runOnce(SplitFetcher.java:142) ~[flink-connector-files-1.15.0-h0.cbu.dli.330.r19.jar:1.15.0-h0.cbu.dli.330.r19]at org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher.run(SplitFetcher.java:105) ~[flink-connector-files-1.15.0-h0.cbu.dli.330.r19.jar:1.15.0-h0.cbu.dli.330.r19]at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_372]at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_372]at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_372]at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_372]... 1 more

Possible Causes

• The user used by the Oracle connection does not have the select permission on the synchronized table.
• The synchronized table has been deleted or their names has been changed at the source.

Solution

• Check whether the synchronized table is still contained in the source Oracle database. If the table is not contained, delete it from the job.
• If the synchronized table is contained at the source, check whether the Oracle connection user has the select permissions on the table. If the user does not have the select permission on the table, grant the permission to the user.