代码之家 › 专栏 › 技术社区 › troynt

如何配置TensorFlow服务,以服务于来自HDF的模型?

tensorflow-serving hdfs tensorflow hadoop

troynt · 技术社区 · 8 年前

我正在尝试使用TensorFlow服务项目从HDF服务TensorFlow模型。

我正在运行TensorFlow服务Docker容器标签1.10.1 https://hub.docker.com/r/tensorflow/serving

我可以在下面看到TensorFlow/Servicing repo引用Hadoop https://github.com/tensorflow/serving/blob/628702e1de1fa3d679369e9546e7d74fa91154d3/tensorflow_serving/model_servers/BUILD#L341

"@org_tensorflow//tensorflow/core/platform/hadoop:hadoop_file_system"

这是对

https://github.com/tensorflow/tensorflow/blob/master/tensorflow/core/platform/hadoop/hadoop_file_system.cc

我已经设置了以下环境变量:

hadoop_hdfs_home指向我的hdfs home(在我的例子中是/etc/hadoop)。
model_base_path设置为“hdfs://tensorflow/models”
模型名称设置为要加载的模型的名称

我将hadoop home装入docker容器,并可以使用docker exec对其进行验证。

当我运行Docker容器时,日志中会显示以下内容:

tensorflow_serving/sources/storage_path/file_system_storage_path_source.cc:369] FileSystemStoragePathSource encountered a file-system access error: Could not find base path hdfs://tensorflow/models/my_model for servable my_model

我发现了TensorFlow使用HDF进行培训的例子,但没有使用TensorFlow服务提供HDF的模型。

TensorFlow服务是否可以服务于HDF的模型? 如果是,你怎么做?

1 回复 | 直到 8 年前

Angus 8 年前

在 BUILD 属于 model_servers ,在 cc_test 对于 get_model_status_impl_test ,添加此行 @org_tensorflow//tensorflow/core/platform/hadoop:hadoop_file_system ,如下所示:

cc_test(
    name = "get_model_status_impl_test",
    size = "medium",
    srcs = ["get_model_status_impl_test.cc"],
    data = [
        "//tensorflow_serving/servables/tensorflow/testdata:saved_model_half_plus_two_2_versions",
    ],
    deps = [
        ":get_model_status_impl",
        ":model_platform_types",
        ":platform_config_util",
        ":server_core",
        "//tensorflow_serving/apis:model_proto",
        "//tensorflow_serving/core:availability_preserving_policy",
        "//tensorflow_serving/core/test_util:test_main",
        "//tensorflow_serving/servables/tensorflow:saved_model_bundle_source_adapter_proto",
        "//tensorflow_serving/servables/tensorflow:session_bundle_config_proto",
        "//tensorflow_serving/servables/tensorflow:session_bundle_source_adapter_proto",
        "//tensorflow_serving/test_util",
        "@org_tensorflow//tensorflow/cc/saved_model:loader",
        "@org_tensorflow//tensorflow/cc/saved_model:signature_constants",
        "@org_tensorflow//tensorflow/contrib/session_bundle",
        "@org_tensorflow//tensorflow/core:test",
        "@org_tensorflow//tensorflow/core/platform/hadoop:hadoop_file_system",
    ],
)

我想这能解决你的问题。

裁判: Fail to load the models from HDFS