在Kubernetes中,Flink使用的内存比taskmanager.memory.process.size多

在Flink 1.10.0中,我们尝试使用 taskmanager.memory.process.size OOMKilled 使用以下设置。

任何关于如何正确设置库伯内特斯和弗林克的建议 乌姆基勒 ?

• https://ci.apache.org/projects/flink/flink-docs-release-1.10/ops/deployment/kubernetes.html .

• 以下是在Kubernetes中部署taskmanager的资源配置:

resources:
  requests:
    cpu: 1000m
    memory: 4096Mi
  limits:
    cpu: 1000m
    memory: 4096Mi

• 以下是中所有与内存相关的配置 flink-conf.yaml

jobmanager.heap.size: 820m
taskmanager.memory.jvm-metaspace.size: 128m
taskmanager.memory.process.size: 4096m

• 我们使用RocksDB但我们没有设置 state.backend.rocksdb.memory.managed 在里面 弗林克形态山药 .

• any substantial off-heap memory allocations in your application code or its dependencies . 有什么工具可以这么做吗?

  以下是我们的依赖。

val flinkVersion = "1.10.0"

libraryDependencies += "com.squareup.okhttp3" % "okhttp" % "4.2.2"
libraryDependencies += "com.typesafe" % "config" % "1.4.0"
libraryDependencies += "joda-time" % "joda-time" % "2.10.5"
libraryDependencies += "org.apache.flink" %% "flink-connector-kafka" % flinkVersion
libraryDependencies += "org.apache.flink" % "flink-metrics-dropwizard" % flinkVersion
libraryDependencies += "org.apache.flink" %% "flink-scala" % flinkVersion % "provided"
libraryDependencies += "org.apache.flink" %% "flink-statebackend-rocksdb" % flinkVersion % "provided"
libraryDependencies += "org.apache.flink" %% "flink-streaming-scala" % flinkVersion % "provided"
libraryDependencies += "org.json4s" %% "json4s-jackson" % "3.6.7"
libraryDependencies += "org.log4s" %% "log4s" % "1.8.2"
libraryDependencies += "org.rogach" %% "scallop" % "3.3.1"

• 我们在Flink1.9.1中使用的配置如下。它没有 乌姆基勒 .

  • 库伯内特斯

resources:
  requests:
    cpu: 1200m
    memory: 2G
  limits:
    cpu: 1500m
    memory: 2G

jobmanager.heap.size: 820m
taskmanager.heap.size: 1024m