代码之家 › 专栏 › 技术社区 › Michael

使用Docker Compose和端口映射进行本地Kafka测试

docker-compose apache-kafka docker

2

Michael · 技术社区 · 7 年前

我正在尝试获取黑盒服务测试的基础结构,并使用 docker-compose . 我想在Jenkins上为多个服务这样做,因此有必要绑定到构建特定的端口(例如50012而不是9092),因为构建应该能够并行运行。

问题是,生成消息失败。

这是我的 docker-compose.yml :

---
version: "3.4"

services:

  zookeeper:
    image: "confluentinc/cp-zookeeper:latest"
    ports:
      - "2181"
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_TICK_TIME: 2000
    extra_hosts:
      - "moby:127.0.0.1"

  kafka:
    image: "confluentinc/cp-kafka:latest"
    environment:
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: INSIDE:PLAINTEXT,OUTSIDE:PLAINTEXT
      KAFKA_PROTOCOL_NAME: INSIDE
      KAFKA_ADVERTISED_PROTOCOL_NAME: OUTSIDE
      KAFKA_INTER_BROKER_LISTENER_NAME: INSIDE
      KAFKA_LISTENERS: "OUTSIDE://localhost:50012,INSIDE://kafka:9092"
      KAFKA_ADVERTISED_LISTENERS: "OUTSIDE://localhost:50012,INSIDE://kafka:9092"
      KAFKA_BROKER_ID: 1
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_ZOOKEEPER_CONNECT: "zookeeper:2181"
    ports:
      - "50012:9092"
    extra_hosts:
      - "moby:127.0.0.1"
    depends_on:
      - zookeeper

除了这个错误(我认为可以忽略),我没有看到任何错误:

ERROR Could not submit metrics to Kafka topic __confluent.support.metrics: Failed to construct kafka producer (io.confluent.support.metrics.BaseMetricsReporter)

当我试图用这样的卡夫卡控制台生产者发送消息时

echo 'my-message' | kafka-console-producer.sh --broker-list localhost:50012 --topic test

我收到这个异常,消息不会被发送:

[2018-07-25 10:56:31,283] ERROR Error when sending message to topic test with key: null, value: 10 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)

org.apache.kafka.common.errors.TimeoutException:60000毫秒后更新元数据失败。

更新: 多亏了罗宾的正确回答,我们可以用下面的docker-compose.yml运行它:

---
version: "3.4"

services:

zookeeper:
    image: "confluentinc/cp-zookeeper:latest"
    environment:
    ZOOKEEPER_CLIENT_PORT: 2181
    ZOOKEEPER_TICK_TIME: 2000
    extra_hosts:
      - "moby:127.0.0.1"

kafka:
    image: "confluentinc/cp-kafka:latest"
    environment:
    KAFKA_BROKER_ID: 1
    KAFKA_ZOOKEEPER_CONNECT: "zookeeper:2181"
    KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: "INSIDE:PLAINTEXT,OUTSIDE:PLAINTEXT"
    KAFKA_INTER_BROKER_LISTENER_NAME: "INSIDE"
    KAFKA_ADVERTISED_LISTENERS: "INSIDE://kafka:29092,OUTSIDE://localhost:50012"
    KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
    KAFKA_AUTO_CREATE_TOPICS_ENABLE: "true"
    ports:
      - "50012:50012"
    extra_hosts:
      - "moby:127.0.0.1"
    depends_on:
      - zookeeper

1 回复 | 直到 7 年前

1

3

Robin Moffatt 7 年前

配置的问题是

      KAFKA_ADVERTISED_LISTENERS: "OUTSIDE://localhost:50012,INSIDE://kafka:9092"
[â¦]
    ports:
      - "50012:9092"

这个 KAFKA_ADVERTISED_LISTENERS 正是那个-经纪人广告的地址。因此,一方面,您将其配置为 localhost:50012 但另一方面你要从你的Docker配置中暴露出来 9092 作为外部端口。

如果你只是使用

 ports:
      - "50012:50012"

那它就可以工作了。

看看这个 docker-compose 例如,在外部和内部公开Kafka的工作配置。