Oracle作为源端时在全量同步的过程中，报数据库历史记录内容完全或者部分缺失， 报错信息包含“The db history topic or its content is fully or partially missing. Please check database history topic configuration and re-execute the snapshot.”怎么办？

问题描述

Oracle作为源端时在全量同步的过程中，报数据库历史记录内容完全或者部分缺失，Jobmanager或者task manager日志报错“The db history topic or its content is fully or partially missing. Please check database history topic configuration and re-execute the snapshot.”

on flink-69782-taskmanager-1-1-00b332b4-fbef-48a1-a584-c15fde32f140 @ 10.16.32.89 (dataPort=38433).
java.lang.RuntimeException: One or more fetchers have encountered exception
at org.apache.flink.connector.base.source.reader.fetcher.SplitFetcherManager.checkErrors(SplitFetcherManager.java:225) ~[flink-connector-files-1.15.0-h0.cbu.dli.330.r19.jar:1.15.0-h0.cbu.dli.330.r19]
at org.apache.flink.connector.base.source.reader.SourceReaderBase.getNextFetch(SourceReaderBase.java:169) ~[flink-connector-files-1.15.0-h0.cbu.dli.330.r19.jar:1.15.0-h0.cbu.dli.330.r19]
at org.apache.flink.connector.base.source.reader.SourceReaderBase.pollNext(SourceReaderBase.java:130) ~[flink-connector-files-1.15.0-h0.cbu.dli.330.r19.jar:1.15.0-h0.cbu.dli.330.r19]
at org.apache.flink.streaming.api.operators.SourceOperator.emitNext(SourceOperator.java:386) ~[flink-dist-1.15.0-h0.cbu.dli.330.r19.jar:1.15.0-h0.cbu.dli.330.r19]
at org.apache.flink.streaming.runtime.io.StreamTaskSourceInput.emitNext(StreamTaskSourceInput.java:68) ~[flink-dist-1.15.0-h0.cbu.dli.330.r19.jar:1.15.0-h0.cbu.dli.330.r19]
at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:65) ~[flink-dist-1.15.0-h0.cbu.dli.330.r19.jar:1.15.0-h0.cbu.dli.330.r19]
at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:539) ~[flink-dist-1.15.0-h0.cbu.dli.330.r19.jar:1.15.0-h0.cbu.dli.330.r19]
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:216) ~[flink-dist-1.15.0-h0.cbu.dli.330.r19.jar:1.15.0-h0.cbu.dli.330.r19]
at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:829) ~[flink-dist-1.15.0-h0.cbu.dli.330.r19.jar:1.15.0-h0.cbu.dli.330.r19]
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:778) ~[flink-dist-1.15.0-h0.cbu.dli.330.r19.jar:1.15.0-h0.cbu.dli.330.r19]
at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:958) ~[flink-dist-1.15.0-h0.cbu.dli.330.r19.jar:1.15.0-h0.cbu.dli.330.r19]
at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:937) ~[flink-dist-1.15.0-h0.cbu.dli.330.r19.jar:1.15.0-h0.cbu.dli.330.r19]
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:751) ~[flink-dist-1.15.0-h0.cbu.dli.330.r19.jar:1.15.0-h0.cbu.dli.330.r19]
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:573) ~[flink-dist-1.15.0-h0.cbu.dli.330.r19.jar:1.15.0-h0.cbu.dli.330.r19]
at java.lang.Thread.run(Thread.java:750) ~[?:1.8.0_372]
Caused by: java.lang.RuntimeException: SplitFetcher thread 0 received unexpected exception while polling the records
at org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher.runOnce(SplitFetcher.java:150) ~[flink-connector-files-1.15.0-h0.cbu.dli.330.r19.jar:1.15.0-h0.cbu.dli.330.r19]
at org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher.run(SplitFetcher.java:105) ~[flink-connector-files-1.15.0-h0.cbu.dli.330.r19.jar:1.15.0-h0.cbu.dli.330.r19]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_372]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_372]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_372]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_372]
... 1 more
Caused by: org.apache.inlong.sort.cdc.oracle.shaded.io.debezium.DebeziumException: The db history topic or its content is fully or partially missing. Please check database history topic configuration and re-execute the snapshot.
at org.apache.inlong.sort.cdc.oracle.shaded.io.debezium.relational.HistorizedRelationalDatabaseSchema.recover(HistorizedRelationalDatabaseSchema.java:47) ~[?:?]
at org.apache.inlong.sort.cdc.oracle.source.reader.fetch.OracleSourceFetchTaskContext.validateAndLoadDatabaseHistory(OracleSourceFetchTaskContext.java:245) ~[?:?]
at org.apache.inlong.sort.cdc.oracle.source.reader.fetch.OracleSourceFetchTaskContext.configure(OracleSourceFetchTaskContext.java:110) ~[?:?]
at org.apache.inlong.sort.cdc.oracle.shaded.org.apache.inlong.sort.cdc.base.source.reader.external.IncrementalSourceStreamFetcher.submitTask(IncrementalSourceStreamFetcher.java:81) ~[?:?]
at org.apache.inlong.sort.cdc.oracle.shaded.org.apache.inlong.sort.cdc.base.source.reader.IncrementalSourceSplitReader.checkSplitOrStartNext(IncrementalSourceSplitReader.java:142) ~[?:?]
at org.apache.inlong.sort.cdc.oracle.shaded.org.apache.inlong.sort.cdc.base.source.reader.IncrementalSourceSplitReader.fetch(IncrementalSourceSplitReader.java:73) ~[?:?]
at org.apache.flink.connector.base.source.reader.fetcher.FetchTask.run(FetchTask.java:58) ~[flink-connector-files-1.15.0-h0.cbu.dli.330.r19.jar:1.15.0-h0.cbu.dli.330.r19]
at org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher.runOnce(SplitFetcher.java:142) ~[flink-connector-files-1.15.0-h0.cbu.dli.330.r19.jar:1.15.0-h0.cbu.dli.330.r19]
at org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher.run(SplitFetcher.java:105) ~[flink-connector-files-1.15.0-h0.cbu.dli.330.r19.jar:1.15.0-h0.cbu.dli.330.r19]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_372]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_372]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_372]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_372]
... 1 more

原因分析

• Oracle连接使用的用户没有同步表的select权限。
• 同步的表在源端被清理或者改变了表名。

解决方案

• 在Oracle源端排查同步的表是否存在，如果不存在，需要在作业中去除该表。
• 如果源端存在该同步的表，则排查Oracle连接用户是否对该表有select 权限，如果没有该表select权限, 请增加该表的select权限。