代码之家  ›  专栏  ›  技术社区  ›  Anshul Tripathi

无法将FileBeat发送到弹性搜索

  •  0
  • Anshul Tripathi  · 技术社区  · 7 年前

    我有一个kuberenetes集群,我正试图从集群中的容器收集日志我正在使用Filebeat收集日志并将其发送到elasctic search,然后在Kibana中显示我部署了Kibana和elastic search,效果很好我正在使用守护程序来部署FileBeat。 这是我在部署Filebeat时引用的YAML文件。 我使用这里的清单文件来部署它,并对其进行了一些修改。

    https://www.elastic.co/guide/en/beats/filebeat/master/running-on-kubernetes.html

    ---
    apiVersion: v1
    kind: ConfigMap
    metadata:
      name: filebeat-config
      namespace: kube-system
      labels:
        k8s-app: filebeat
    data:
      filebeat.yml: |-
        filebeat.config:
          inputs:
          - type: log
            # Mounted `filebeat-inputs` configmap:
            paths: /var/lib/docker/containers/*/*.log
            # Reload inputs configs as they change:
            reload.enabled: false
            json.message_key: log
            json.keys_under_root: true
        output.elasticsearch:
          hosts: ['x.x.x.x:9200']
    ---
    apiVersion: v1
    kind: ConfigMap
    metadata:
      name: filebeat-inputs
      namespace: kube-system
      labels:
        k8s-app: filebeat
    data:
      kubernetes.yml: |-
        - type: docker
          containers.ids:
          - "*"
          processors:
            - add_kubernetes_metadata:
                in_cluster: true
    ---
    apiVersion: extensions/v1beta1
    kind: DaemonSet
    metadata:
      name: filebeat
      namespace: kube-system
      labels:
        k8s-app: filebeat
    spec:
      template:
        metadata:
          labels:
            k8s-app: filebeat
        spec:
          serviceAccountName: filebeat
          terminationGracePeriodSeconds: 30
          containers:
          - name: filebeat
            image: docker.elastic.co/beats/filebeat:6.3.1
            args: [
              "-c", "/etc/filebeat.yml",
              "-e",
            ]
            env:
            - name: ELASTICSEARCH_HOST
              value: X.x.x.x
            - name: ELASTICSEARCH_PORT
              value: "9200"
              value:
            securityContext:
              runAsUser: 0
            resources:
              limits:
                memory: 200Mi
              requests:
                cpu: 100m
                memory: 100Mi
            volumeMounts:
            - name: config
              mountPath: /etc/filebeat.yml
              readOnly: true
              subPath: filebeat.yml
            - name: inputs
              mountPath: /usr/share/filebeat/inputs.d
              readOnly: true
            - name: data
              mountPath: /usr/share/filebeat/data
            - name: varlibdockercontainers
              mountPath: /var/lib/docker/containers
              readOnly: true
          volumes:
          - name: config
            configMap:
              defaultMode: 0600
              name: filebeat-config
          - name: varlibdockercontainers
            hostPath:
              path: /var/lib/docker/containers
          - name: inputs
            configMap:
              defaultMode: 0600
              name: filebeat-inputs
          # data folder stores a registry of read status for all files, so we don't send everything again on a Filebeat pod restart
          - name: data
            hostPath:
              path: /var/lib/filebeat-data
              type: DirectoryOrCreate
    --- 
    

    我查看了运行Filebeat的pods,它存储了日志但不知怎么的,它并没有发布到弹性搜索。我的确切配置应该是什么才能发布到elasticsearch我已经坚持了几天了,我没有选择了任何帮助都将不胜感激。

    1 回复  |  直到 7 年前
        1
  •  1
  •   Bal Chua    7 年前

    您的filebeat配置没有获取任何输入类型。

    filebeat.yaml文件输入路径必须指向filebeats-inputs.yaml而不是日志位置。依次委托给docker输入类型。默认containers.path是/var/lib/docker/containers。

    https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-docker.html