更新时间:2022-12-07 GMT+08:00

内置依赖包

DLI内置依赖包是平台默认提供的依赖包,用户打包Spark或Flink jar作业jar包时,不需要额外上传这些依赖包,以免与平台内置依赖包冲突。

Spark 2.3.2 依赖包

  • accessors-smart-1.2.jar
  • activation-1.1.1.jar
  • aircompressor-0.8.jar
  • alluxio-2.3.1-luxor-SNAPSHOT-client.jar
  • antlr-2.7.7.jar
  • antlr4-runtime-4.8-1.jar
  • antlr-runtime-3.4.jar
  • aopalliance-1.0.jar
  • aopalliance-repackaged-2.4.0-b34.jar
  • apache-log4j-extras-1.2.17.jar
  • arpack_combined_all-0.1.jar
  • arrow-format-0.8.0.jar
  • arrow-memory-0.8.0.jar
  • arrow-vector-0.8.0.jar
  • asm-5.0.4.jar
  • audience-annotations-0.5.0.jar
  • automaton-1.11-8.jar
  • avro-1.7.7.jar
  • avro-ipc-1.7.7.jar
  • avro-ipc-1.7.7-tests.jar
  • avro-mapred-1.7.7-hadoop2.jar
  • aws-java-sdk-bundle-1.11.271.jar
  • base64-2.3.8.jar
  • bcpkix-jdk15on-1.66.jar
  • bcprov-jdk15on-1.66.jar
  • bonecp-0.8.0.RELEASE.jar
  • breeze_2.11-0.13.2.jar
  • breeze-macros_2.11-0.13.2.jar
  • calcite-avatica-1.2.0-incubating.jar
  • calcite-core-1.2.0-incubating.jar
  • calcite-linq4j-1.2.0-incubating.jar
  • checker-qual-2.11.1.jar
  • chill_2.11-0.8.4.jar
  • chill-java-0.8.4.jar
  • commons-beanutils-1.9.4.jar
  • commons-cli-1.2.jar
  • commons-codec-2.0-20130428.202122-59.jar
  • commons-collections-3.2.2.jar
  • commons-collections4-4.2.jar
  • commons-compiler-3.0.8.jar
  • commons-compress-1.4.1.jar
  • commons-configuration2-2.1.1.jar
  • commons-crypto-1.0.0-20191105.jar
  • commons-daemon-1.0.13.jar
  • commons-dbcp-1.4.jar
  • commons-dbcp2-2.7.0.jar
  • commons-httpclient-3.1.jar
  • commons-io-2.5.jar
  • commons-lang-2.6.jar
  • commons-lang3-3.5.jar
  • commons-logging-1.2.jar
  • commons-math3-3.4.1.jar
  • commons-net-2.2.jar
  • commons-pool-1.5.4.jar
  • commons-pool2-2.8.0.jar
  • commons-text-1.3.jar
  • compress-lzf-1.0.3.jar
  • core-1.1.2.jar
  • curator-client-4.2.0.jar
  • curator-framework-4.2.0.jar
  • curator-recipes-2.7.1.jar
  • datanucleus-api-jdo-3.2.6.jar
  • datanucleus-core-3.2.10.jar
  • datanucleus-rdbms-3.2.9.jar
  • derby-10.12.1.1.jar
  • dnsjava-2.1.7.jar
  • ehcache-3.3.1.jar
  • eigenbase-properties-1.1.5.jar
  • error_prone_annotations-2.3.4.jar
  • failureaccess-1.0.1.jar
  • fastutil-8.2.3.jar
  • ffmpeg-4.3.1-1.5.4.jar
  • ffmpeg-4.3.1-1.5.4-linux-x86_64.jar
  • flatbuffers-1.2.0-3f79e055.jar
  • generex-1.0.2.jar
  • geronimo-jcache_1.0_spec-1.0-alpha-1.jar
  • gson-2.2.4.jar
  • guava-29.0-jre.jar
  • guice-4.0.jar
  • guice-servlet-4.0.jar
  • hadoop-annotations-3.1.1-ei-302002.jar
  • hadoop-auth-3.1.1-ei-302002.jar
  • hadoop-aws-3.1.1-ei-302002.jar
  • hadoop-client-3.1.1-ei-302002.jar
  • hadoop-common-3.1.1-ei-302002.jar
  • hadoop-hdfs-3.1.1-ei-302002.jar
  • hadoop-hdfs-client-3.1.1-ei-302002.jar
  • hadoop-mapreduce-client-common-3.1.1-ei-302002.jar
  • hadoop-mapreduce-client-core-3.1.1-ei-302002.jar
  • hadoop-mapreduce-client-jobclient-3.1.1-ei-302002.jar
  • hadoop-minikdc-3.1.1-hw-ei-302002.jar
  • hadoop-yarn-api-3.1.1-ei-302002.jar
  • hadoop-yarn-client-3.1.1-ei-302002.jar
  • hadoop-yarn-common-3.1.1-ei-302002.jar
  • hadoop-yarn-registry-3.1.1-ei-302002.jar
  • hadoop-yarn-server-common-3.1.1-ei-302002.jar
  • hadoop-yarn-server-web-proxy-3.1.1-ei-302002.jar
  • hamcrest-core-1.3.jar
  • HikariCP-java7-2.4.12.jar
  • hive-common-1.2.1-2.1.0.dli-20201111.064115-91.jar
  • hive-exec-1.2.1-2.1.0.dli-20201111.064444-91.jar
  • hive-metastore-1.2.1-2.1.0.dli-20201111.064230-91.jar
  • hk2-api-2.4.0-b34.jar
  • hk2-locator-2.4.0-b34.jar
  • hk2-utils-2.4.0-b34.jar
  • hppc-0.7.2.jar
  • htrace-core4-4.2.0-incubating-hw-1.0.0.jar
  • httpclient-4.5.4.jar
  • httpcore-4.4.7.jar
  • ivy-2.4.0.jar
  • j2objc-annotations-1.3.jar
  • jackson-annotations-2.10.0.jar
  • jackson-core-2.10.0.jar
  • jackson-core-asl-1.9.13-atlassian-4.jar
  • jackson-databind-2.10.0.jar
  • jackson-dataformat-yaml-2.10.0.jar
  • jackson-datatype-jsr310-2.10.3.jar
  • jackson-jaxrs-base-2.10.3.jar
  • jackson-jaxrs-json-provider-2.10.3.jar
  • jackson-mapper-asl-1.9.13-atlassian-4.jar
  • jackson-module-jaxb-annotations-2.10.3.jar
  • jackson-module-paranamer-2.10.0.jar
  • jackson-module-scala_2.11-2.10.0.jar
  • jakarta.activation-api-1.2.1.jar
  • jakarta.xml.bind-api-2.3.2.jar
  • janino-3.0.8.jar
  • javacpp-1.5.4.jar
  • javacpp-1.5.4-linux-x86_64.jar
  • javacv-1.5.4.jar
  • JavaEWAH-1.1.7.jar
  • javassist-3.18.1-GA.jar
  • javax.annotation-api-1.2.jar
  • javax.inject-1.jar
  • javax.inject-2.4.0-b34.jar
  • javax.servlet-api-3.1.0.jar
  • javax.ws.rs-api-2.0.1.jar
  • java-xmlbuilder-1.1.jar
  • javolution-5.3.1.jar
  • jaxb-api-2.2.11.jar
  • jcip-annotations-1.0-1.jar
  • jcl-over-slf4j-1.7.26.jar
  • jdo-api-3.0.1.jar
  • jersey-client-2.23.1.jar
  • jersey-common-2.23.1.jar
  • jersey-container-servlet-2.23.1.jar
  • jersey-container-servlet-core-2.23.1.jar
  • jersey-guava-2.23.1.jar
  • jersey-media-jaxb-2.23.1.jar
  • jersey-server-2.23.1.jar
  • jets3t-0.9.4.jar
  • jetty-http-9.4.31.v20200723.jar
  • jetty-io-9.4.31.v20200723.jar
  • jetty-security-9.4.31.v20200723.jar
  • jetty-server-9.4.31.v20200723.jar
  • jetty-servlet-9.4.31.v20200723.jar
  • jetty-util-9.4.31.v20200723.jar
  • jetty-util-ajax-9.4.31.v20200723.jar
  • jetty-webapp-9.4.31.v20200723.jar
  • jetty-xml-9.4.31.v20200723.jar
  • joda-time-2.9.3.jar
  • jodd-core-4.2.0.jar
  • json-20200518.jar
  • json4s-ast_2.11-3.2.11.jar
  • json4s-core_2.11-3.2.11.jar
  • json4s-jackson_2.11-3.2.11.jar
  • json-sanitizer-1.2.1.jar
  • json-smart-2.3.jar
  • jsp-api-2.1.jar
  • jsr305-3.0.2.jar
  • jta-1.1.jar
  • jtransforms-2.4.0.jar
  • jul-to-slf4j-1.7.26.jar
  • junit-4.11.jar
  • kerb-admin-1.0.1.jar
  • kerb-client-1.0.1.jar
  • kerb-common-1.0.1.jar
  • kerb-core-1.0.1.jar
  • kerb-crypto-1.0.1.jar
  • kerb-identity-1.0.1.jar
  • kerb-server-1.0.1.jar
  • kerb-simplekdc-1.0.1.jar
  • kerb-util-1.0.1.jar
  • kerby-asn1-1.0.1.jar
  • kerby-config-1.0.1.jar
  • kerby-pkix-1.0.1.jar
  • kerby-util-1.0.1.jar
  • kerby-xdr-1.0.1.jar
  • kryo-shaded-3.0.3.jar
  • kubernetes-client-4.9.2-20200804.jar
  • kubernetes-model-4.9.2-20200804.jar
  • kubernetes-model-common-4.9.2-20200804.jar
  • leveldbjni-all-1.8-20191105.jar
  • libfb303-0.9.3.jar
  • libthrift-0.12.0.jar
  • listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar
  • log4j-1.2.17-cloudera1.jar
  • log4j-rolling-appender-20131024-2017.jar
  • logging-interceptor-3.14.4.jar
  • luxor-encrypt-2.1.0-20201106.065437-53.jar
  • luxor-fs3-2.1.0-20201106.065612-53.jar
  • luxor-obs-fs3-2.1.0-20201106.065616-53.jar
  • luxor-rpc_2.11-2.1.0-20201106.065541-53.jar
  • luxor-rpc-protobuf2-2.1.0-20201106.065551-53.jar
  • lz4-java-1.7.1.jar
  • machinist_2.11-0.6.1.jar
  • macro-compat_2.11-1.1.1.jar
  • metrics-core-3.1.5.jar
  • metrics-graphite-3.1.5.jar
  • metrics-jmx-4.1.12.1.jar
  • metrics-json-3.1.5.jar
  • metrics-jvm-3.1.5.jar
  • minlog-1.3.0.jar
  • mssql-jdbc-6.2.1.jre7.jar
  • netty-3.10.6.Final.jar
  • netty-all-4.1.51.Final.jar
  • nimbus-jose-jwt-8.19.jar
  • objenesis-2.1.jar
  • okhttp-3.14.4.jar
  • okio-1.17.2.jar
  • opencsv-2.3.jar
  • opencsv-4.6.jar
  • opencv-4.3.0-2.jar
  • orc-core-1.4.4-nohive.jar
  • orc-mapreduce-1.4.4-nohive.jar
  • oro-2.0.8.jar
  • osgi-resource-locator-1.0.1.jar
  • paranamer-2.8.jar
  • parquet-column-1.8.3.jar
  • parquet-common-1.8.3.jar
  • parquet-encoding-1.8.3.jar
  • parquet-format-2.3.1.jar
  • parquet-hadoop-1.8.3.jar
  • parquet-hadoop-bundle-1.6.0.jar
  • parquet-jackson-1.8.3.jar
  • parquet-format-2.3.1.jar
  • parquet-hadoop-1.8.3.jar
  • parquet-hadoop-bundle-1.6.0.jar
  • parquet-jackson-1.8.3.jar
  • postgresql-42.2.14.jar
  • protobuf-java-2.5.0.jar
  • py4j-0.10.7.jar
  • pyrolite-4.13.jar
  • re2j-1.1.jar
  • RoaringBitmap-0.5.11.jar
  • scala-compiler-2.11.12.jar
  • scala-library-2.11.12.jar
  • scalap-2.11.0.jar
  • scala-parser-combinators_2.11-1.1.0.jar
  • scala-reflect-2.11.12.jar
  • scala-xml_2.11-1.0.5.jar
  • secComponentApi-1.0.5c.jar
  • shapeless_2.11-2.3.2.jar
  • slf4j-api-1.7.30.jar
  • slf4j-log4j12-1.7.30.jar
  • snakeyaml-1.24.jar
  • snappy-java-1.1.7.5.jar
  • spark-catalyst_2.11-2.3.2.0101-2.1.0.dli-20201111.073826-143.jar
  • spark-core_2.11-2.3.2.0101-2.1.0.dli-20201111.073836-134.jar
  • spark-graphx_2.11-2.3.2.0101-2.1.0.dli-20201111.073847-129.jar
  • spark-hive_2.11-2.3.2.0101-2.1.0.dli-20201111.073854-132.jar
  • spark-kubernetes_2.11-2.3.2.0101-2.1.0.dli-20201111.073916-85.jar
  • spark-kvstore_2.11-2.3.2.0101-2.1.0.dli-20201111.073933-127.jar
  • spark-launcher_2.11-2.3.2.0101-2.1.0.dli-20201111.073940-127.jar
  • spark-mllib_2.11-2.3.2.0101-2.1.0.dli-20201111.073946-127.jar
  • spark-mllib-local_2.11-2.3.2.0101-2.1.0.dli-20201111.073953-127.jar
  • spark-network-common_2.11-2.3.2.0101-2.1.0.dli-20201111.073959-127.jar
  • spark-network-shuffle_2.11-2.3.2.0101-2.1.0.dli-20201111.074007-127.jar
  • spark-om_2.11-2.3.2.0101-2.1.0.dli-20201111.074019-125.jar
  • spark-repl_2.11-2.3.2.0101-2.1.0.dli-20201111.074028-125.jar
  • spark-sketch_2.11-2.3.2.01012.1.0.dli-20201111.074035-125.jar
  • spark-sql_2.11-2.3.2.0101-2.1.0.dli-20201111.074041-126.jar
  • spark-streaming_2.11-2.3.2.0101-2.1.0.dli-20201111.074100-123.jar
  • spark-tags_2.11-2.3.2.0101-2.1.0.dli-20201111.074136-123.jar
  • spark-tags_2.11-2.3.2.0101-2.1.0.dli-20201111.074141-124-tests.jar
  • spark-unsafe_2.11-2.3.2.0101-2.1.0.dli-20201111.074144-123.jar
  • spark-uquery_2.11-2.3.2.0101-2.1.0.dli-20201111.074906-210.jar
  • spark-yarn_2.11-2.3.2.0101-2.1.0.dli-20201111.074151-123.jar
  • spire_2.11-0.13.0.jar
  • spire-macros_2.11-0.13.0.jar
  • ST4-4.3.1.jar
  • stax2-api-3.1.4.jar
  • stax-api-1.0-2.jar
  • stream-2.7.0.jar
  • stringtemplate-3.2.1.jar
  • token-provider-1.0.1.jar
  • univocity-parsers-2.5.9.jar
  • validation-api-1.1.0.Final.jar
  • woodstox-core-5.0.3.jar
  • xbean-asm5-shaded-4.4.jar
  • xercesImpl-2.12.0.jar
  • xml-apis-1.4.01.jar
  • xz-1.0.jar
  • zjsonpatch-0.3.0.jar
  • zookeeper-3.5.6-ei-302002.jar
  • zookeeper-jute-3.5.6-ei-302002.jar
  • zstd-jni-1.4.4-11.jar

Flink 1.7.2 依赖包

  • bcpkix-jdk15on-1.60.jar
  • bcprov-jdk15on-1.60.jar
  • commons-codec-1.9.jar
  • commons-configuration-1.7.jar
  • deeplearning4j-core-0.9.1.jar
  • deeplearning4j-nlp-0.9.1.jar
  • deeplearning4j-nn-0.9.1.jar
  • ejml-cdense-0.33.jar
  • ejml-core-0.33.jar
  • ejml-ddense-0.33.jar
  • ejml-dsparse-0.33.jar
  • ejml-experimental-0.33.jar
  • ejml-fdense-0.33.jar
  • ejml-simple-0.33.jar
  • ejml-zdense-0.33.jar
  • elsa-3.0.0-M7.jar
  • esdk-obs-java-3.1.3.jar
  • flink-cep_2.11-1.7.0.jar
  • flink-cep-scala_2.11-1.7.0.jar
  • flink-dist_2.11-1.7.0.jar
  • flink-gelly_2.11-1.7.0.jar
  • flink-gelly-scala_2.11-1.7.0.jar
  • flink-ml_2.11-1.7.0.jar
  • flink-python_2.11-1.7.0.jar
  • flink-queryable-state-runtime_2.11-1.7.0.jar
  • flink-shaded-curator-1.7.0.jar
  • flink-shaded-hadoop2-uber-1.7.0.jar
  • flink-table_2.11-1.7.0.jar
  • guava-26.0-jre.jar
  • hadoop-cloud-3.1.1-41-20201014.085840-4.jar
  • httpasyncclient-4.1.2.jar
  • httpclient-4.5.12.jar
  • httpcore-4.4.4.jar
  • httpcore-nio-4.4.4.jar
  • java-xmlbuilder-1.1.jar
  • jna-4.1.0.jar
  • libtensorflow-1.12.0.jar
  • log4j-over-slf4j-1.7.21.jar
  • logback-classic-1.2.3.jar
  • logback-core-1.2.3.jar
  • nd4j-api-0.9.1.jar
  • nd4j-native-0.9.1.jar
  • nd4j-native-api-0.9.1.jar
  • nd4j-native-platform-0.9.1.jar
  • okhttp-3.14.8.jar
  • okio-1.14.0.jar
  • slf4j-api-1.7.21.jar
  • tensorflow-1.12.0.jar

Flink 1.10依赖包

2020年12月之后创建的新队列才能使用Flink 1.10依赖包。

  • bcpkix-jdk15on-1.60.jar
  • bcprov-jdk15on-1.60.jar
  • commons-codec-1.9.jar
  • commons-configuration-1.7.jar
  • deeplearning4j-core-0.9.1.jar
  • deeplearning4j-nlp-0.9.1.jar
  • deeplearning4j-nn-0.9.1.jar
  • ejml-cdense-0.33.jar
  • ejml-core-0.33.jar
  • ejml-ddense-0.33.jar
  • ejml-dsparse-0.33.jar
  • ejml-experimental-0.33.jar
  • ejml-fdense-0.33.jar
  • ejml-simple-0.33.jar
  • ejml-zdense-0.33.jar
  • elsa-3.0.0-M7.jar
  • esdk-obs-java-3.20.6.1.jar
  • flink-cep_2.11-1.10.0.jar
  • flink-cep-scala_2.11-1.10.0.jar
  • flink-dist_2.11-1.10.0.jar
  • flink-python_2.11-1.10.0.jar
  • flink-queryable-state-runtime_2.11-1.10.0.jar
  • flink-sql-client_2.11-1.10.0.jar
  • flink-state-processor-api_2.11-1.10.0.jar
  • flink-table_2.11-1.10.0.jar
  • flink-table-blink_2.11-1.10.0.jar
  • guava-26.0-jre.jar
  • hadoop-3.1.1-41.jar
  • httpasyncclient-4.1.2.jar
  • httpclient-4.5.3.jar
  • httpcore-4.4.4.jar
  • httpcore-nio-4.4.4.jar
  • java-xmlbuilder-1.1.jar
  • jna-4.1.0.jar
  • libtensorflow-1.12.0.jar
  • log4j-over-slf4j-1.7.26.jar
  • logback-classic-1.2.3.jar
  • logback-core-1.2.3.jar
  • nd4j-api-0.9.1.jar
  • nd4j-native-0.9.1.jar
  • nd4j-native-api-0.9.1.jar
  • nd4j-native-platform-0.9.1.jar
  • okhttp-3.14.8.jar
  • okio-1.14.0.jar
  • secComponentApi-1.0.5.jar
  • slf4j-api-1.7.26.jar
  • tensorflow-1.12.0.jar