Hudi作为目的端时，如果表路径为OBS路径，报错信息包含“Exception when get temporary security key”怎么办？

问题描述

实时作业启动时异常，jobmanager报错关键字“Exception when get temporary security key”。

com.obs.services.exception.ObsException: 
Exception when get temporary security key
at com.huawei.mrs.MrsObsCredentialsProvider.getSecurityKey(MrsObsCredentialsProvider.java:201) ~[mrs-obs-provider-3.1.1.51.2.jar:?]
at com.obs.services.ObsClient.<init>(ObsClient.java:161) ~[hadoop-huaweicloud-3.1.1-hw-53.5.jar:?]
at org.apache.hadoop.fs.obs.DefaultOBSClientFactory.createObsClient(DefaultOBSClientFactory.java:190) ~[hadoop-huaweicloud-3.1.1-hw-53.5.jar:?]
at org.apache.hadoop.fs.obs.OBSFileSystem.initialize(OBSFileSystem.java:426) ~[hadoop-huaweicloud-3.1.1-hw-53.5.jar:?]
at org.apache.hadoop.fs.LuxorOBSFileSystem.createFileSystem(LuxorOBSFileSystem.java:120) ~[luxor-obs-fs3-2.3.6-20230706.142541-1034.jar:1.15.0-h0.cbu.dli.321.r2]
at org.apache.hadoop.fs.LuxorOBSFileSystem.getFileSystem(LuxorOBSFileSystem.java:88) ~[luxor-obs-fs3-2.3.6-20230706.142541-1034.jar:1.15.0-h0.cbu.dli.321.r2]
at org.apache.hadoop.fs.LuxorOBSFileSystem.initialize(LuxorOBSFileSystem.java:44) ~[luxor-obs-fs3-2.3.6-20230706.142541-1034.jar:1.15.0-h0.cbu.dli.321.r2]
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3548) ~[flink-dist-1.15.0-h0.cbu.dli.321.r2.jar:1.15.0-h0.cbu.dli.321.r2]
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:544) ~[flink-dist-1.15.0-h0.cbu.dli.321.r2.jar:1.15.0-h0.cbu.dli.321.r2]
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:372) ~[flink-dist-1.15.0-h0.cbu.dli.321.r2.jar:1.15.0-h0.cbu.dli.321.r2]
at com.huawei.clouds.dataarts.shaded.org.apache.hudi.common.fs.FSUtils.getFs(FSUtils.java:119) ~[engine-dist-secret-25.5.0.B050-assembly.jar:?]
at com.huawei.clouds.dataarts.shaded.org.apache.hudi.common.table.HoodieTableMetaClient.getFs(HoodieTableMetaClient.java:299) ~[engine-dist-secret-25.5.0.B050-assembly.jar:?]
at com.huawei.clouds.dataarts.shaded.org.apache.hudi.common.table.HoodieTableMetaClient.<init>(HoodieTableMetaClient.java:132) ~[engine-dist-secret-25.5.0.B050-assembly.jar:?]
at com.huawei.clouds.dataarts.shaded.org.apache.hudi.common.table.HoodieTableMetaClient.<init>(HoodieTableMetaClient.java:85) ~[engine-dist-secret-25.5.0.B050-assembly.jar:?]
at com.huawei.clouds.dataarts.shaded.org.apache.hudi.common.table.HoodieTableMetaClient$Builder.build(HoodieTableMetaClient.java:705) ~[engine-dist-secret-25.5.0.B050-assembly.jar:?]
at com.huawei.clouds.dataarts.shaded.org.apache.hudi.util.StreamerUtil.createMetaClient(StreamerUtil.java:391) ~[engine-dist-secret-25.5.0.B050-assembly.jar:?]
at com.huawei.clouds.dataarts.shaded.org.apache.hudi.util.StreamerUtil.metaClientForReader(StreamerUtil.java:383) ~[engine-dist-secret-25.5.0.B050-assembly.jar:?]
at com.huawei.clouds.dataarts.migration.connector.hudi.table.
HoodieTableFactory.getTableMetaClient
(HoodieTableFactory.java:998) ~[engine-dist-secret-25.5.0.B050-assembly.jar:?]
at com.huawei.clouds.dataarts.migration.connector.hudi.table.
HoodieTableFactory.buildColumnMetaForMultiTableTask
(HoodieTableFactory.java:226) ~[engine-dist-secret-25.5.0.B050-assembly.jar:?]
at com.huawei.clouds.dataarts.migration.connector.hudi.table.HoodieTableFactory.lambda$createDynamicTableSink$4(HoodieTableFactory.java:191) ~[engine-dist-secret-25.5.0.B050-assembly.jar:?]

原因分析

作业写出的Hudi表路径为OBS路径，而用户创建的MRS-Hudi连接信息中没有开启“OBS支持”。

解决方案

连接配置里面把OBS支持打开，同步配置AKSK。

图1 配置OBS支持