代码之家  ›  专栏  ›  技术社区  ›  Scandinave

为什么Kube代理不起作用?没有DNS解析,也没有iptable更新

kube-proxy kubernetes
  •  1
  • Scandinave  · 技术社区  · 4 年前

    我刚刚用kubeadm和kubernetes 1.21建立了一个新的集群。所有吊舱都已标记就绪。但我一个都查不到。在深入研究该问题后,似乎无法进行DNS解析。kube代理似乎不起作用。

    这是kube代理吊舱的日志 

    I0712 05:50:46.511967       1 node.go:172] Successfully retrieved node IP: x.x.x.x
I0712 05:50:46.512039       1 server_others.go:140] Detected node IP x.x.x.x
W0712 05:50:46.512077       1 server_others.go:598] Unknown proxy mode "", assuming iptables proxy
I0712 05:50:46.545626       1 server_others.go:206] kube-proxy running in dual-stack mode, IPv4-primary
I0712 05:50:46.545672       1 server_others.go:212] Using iptables Proxier.
I0712 05:50:46.545692       1 server_others.go:219] creating dualStackProxier for iptables.
W0712 05:50:46.545715       1 server_others.go:512] detect-local-mode set to ClusterCIDR, but no IPv6 cluster CIDR defined, , defaulting to no-op detect-local for IPv6
I0712 05:50:46.546089       1 server.go:643] Version: v1.21.2
I0712 05:50:46.549861       1 conntrack.go:52] Setting nf_conntrack_max to 196608
I0712 05:50:46.550300       1 config.go:224] Starting endpoint slice config controller
I0712 05:50:46.550338       1 shared_informer.go:240] Waiting for caches to sync for endpoint slice config
I0712 05:50:46.550332       1 config.go:315] Starting service config controller
I0712 05:50:46.550354       1 shared_informer.go:240] Waiting for caches to sync for service config
W0712 05:50:46.553020       1 warnings.go:70] discovery.k8s.io/v1beta1 EndpointSlice is deprecated in v1.21+, unavailable in v1.25+; use discovery.k8s.io/v1 EndpointSlice
W0712 05:50:46.555115       1 warnings.go:70] discovery.k8s.io/v1beta1 EndpointSlice is deprecated in v1.21+, unavailable in v1.25+; use discovery.k8s.io/v1 EndpointSlice
I0712 05:50:46.650614       1 shared_informer.go:247] Caches are synced for service config 
I0712 05:50:46.650634       1 shared_informer.go:247] Caches are synced for endpoint slice config 
W0712 05:57:14.556916       1 warnings.go:70] discovery.k8s.io/v1beta1 EndpointSlice is deprecated in v1.21+, unavailable in v1.25+; use discovery.k8s.io/v1 EndpointSlice
W0712 06:06:34.558550       1 warnings.go:70] discovery.k8s.io/v1beta1 EndpointSlice is deprecated in v1.21+, unavailable in v1.25+; use discovery.k8s.io/v1 EndpointSlice

    这是我的吊舱跑步: 

    kube-system     pod/coredns-558bd4d5db-qpf5m                  1/1     Running   1          8h
kube-system     pod/coredns-558bd4d5db-r5jwz                  1/1     Running   0          8h
kube-system     pod/etcd-master2                              1/1     Running   3          20h
kube-system     pod/kube-apiserver-master2                    1/1     Running   3          20h
kube-system     pod/kube-controller-manager-master2           1/1     Running   3          8h
kube-system     pod/kube-flannel-ds-b7xrm                     1/1     Running   0          8h
kube-system     pod/kube-flannel-ds-hcn7f                     1/1     Running   0          8h
kube-system     pod/kube-flannel-ds-rx8j6                     1/1     Running   1          8h
kube-system     pod/kube-flannel-ds-wc2jc                     1/1     Running   0          8h
kube-system     pod/kube-proxy-48wmr                          1/1     Running   0          25m
kube-system     pod/kube-proxy-4gw8t                          1/1     Running   0          25m
kube-system     pod/kube-proxy-h9djp                          1/1     Running   0          25m
kube-system     pod/kube-proxy-r4k9t                          1/1     Running   0          24m
kube-system     pod/kube-scheduler-master2                    1/1     Running   3          20h

    命令 kubectl run -it --rm --restart=Never busybox --image=gcr.io/google-containers/busybox nslookup kubernetes.default 给我: 

    Address 1: x.x.x.x

nslookup: can't resolve 'kubernetes.default'
pod "busybox" deleted
pod default/busybox terminated (Error)

    我的iptables规则: 

    Chain INPUT (policy ACCEPT)
target     prot opt source               destination         
KUBE-NODEPORTS  all  --  anywhere             anywhere             /* kubernetes health check service ports */
KUBE-EXTERNAL-SERVICES  all  --  anywhere             anywhere             ctstate NEW /* kubernetes externally-visible service portals */
KUBE-FIREWALL  all  --  anywhere             anywhere            

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         
KUBE-FORWARD  all  --  anywhere             anywhere             /* kubernetes forwarding rules */
KUBE-SERVICES  all  --  anywhere             anywhere             ctstate NEW /* kubernetes service portals */
KUBE-EXTERNAL-SERVICES  all  --  anywhere             anywhere             ctstate NEW /* kubernetes externally-visible service portals */
DOCKER-USER  all  --  anywhere             anywhere            
DOCKER-ISOLATION-STAGE-1  all  --  anywhere             anywhere            
ACCEPT     all  --  anywhere             anywhere             ctstate RELATED,ESTABLISHED
DOCKER     all  --  anywhere             anywhere            
ACCEPT     all  --  anywhere             anywhere            
ACCEPT     all  --  anywhere             anywhere            
ACCEPT     all  --  10.244.0.0/16        anywhere            
ACCEPT     all  --  anywhere             10.244.0.0/16       

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         
KUBE-SERVICES  all  --  anywhere             anywhere             ctstate NEW /* kubernetes service portals */
KUBE-FIREWALL  all  --  anywhere             anywhere            

Chain DOCKER (1 references)
target     prot opt source               destination         

Chain DOCKER-ISOLATION-STAGE-1 (1 references)
target     prot opt source               destination         
DOCKER-ISOLATION-STAGE-2  all  --  anywhere             anywhere            
RETURN     all  --  anywhere             anywhere            

Chain DOCKER-ISOLATION-STAGE-2 (1 references)
target     prot opt source               destination         
DROP       all  --  anywhere             anywhere            
RETURN     all  --  anywhere             anywhere            

Chain DOCKER-USER (1 references)
target     prot opt source               destination         
RETURN     all  --  anywhere             anywhere            

Chain KUBE-EXTERNAL-SERVICES (2 references)
target     prot opt source               destination         

Chain KUBE-FIREWALL (2 references)
target     prot opt source               destination         
DROP       all  --  anywhere             anywhere             /* kubernetes firewall for dropping marked packets */ mark match 0x8000/0x8000
DROP       all  -- !127.0.0.0/8          127.0.0.0/8          /* block incoming localnet connections */ ! ctstate RELATED,ESTABLISHED,DNAT

Chain KUBE-FORWARD (1 references)
target     prot opt source               destination         
DROP       all  --  anywhere             anywhere             ctstate INVALID
ACCEPT     all  --  anywhere             anywhere             /* kubernetes forwarding rules */ mark match 0x4000/0x4000
ACCEPT     all  --  anywhere             anywhere             /* kubernetes forwarding conntrack pod source rule */ ctstate RELATED,ESTABLISHED
ACCEPT     all  --  anywhere             anywhere             /* kubernetes forwarding conntrack pod destination rule */ ctstate RELATED,ESTABLISHED

Chain KUBE-KUBELET-CANARY (0 references)
target     prot opt source               destination         

Chain KUBE-NODEPORTS (1 references)
target     prot opt source               destination         

Chain KUBE-PROXY-CANARY (0 references)
target     prot opt source               destination         

Chain KUBE-SERVICES (2 references)
target     prot opt source               destination

    有什么想法吗?

    [编辑] 

    #kubectl edit cm -n kube-system kubelet-config-1.21
apiVersion: v1
data:
  kubelet: |
    apiVersion: kubelet.config.k8s.io/v1beta1
    authentication:
      anonymous:
        enabled: false
      webhook:
        cacheTTL: 0s
        enabled: true
      x509:
        clientCAFile: /etc/kubernetes/pki/ca.crt
    authorization:
      mode: Webhook
      webhook:
        cacheAuthorizedTTL: 0s
        cacheUnauthorizedTTL: 0s
    cgroupDriver: systemd
    clusterDNS:
    - 10.96.0.10
    clusterDomain: cluster.local

    #kubectl get svc -n kube-system
NAME       TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)                  AGE
kube-dns   ClusterIP   10.96.0.10   <none>        53/UDP,53/TCP,9153/TCP   22h
    0 回复  |  直到 4 年前
        1
  •  0
  •   vi7a    4 年前

    Kube代理它的网络服务。 负责DNS解析的DNS提供商。如我所见,您已经安装了coredns。 检查kubelet配置。它应该指向正确的服务,并且该服务应该可以在您的播客中访问。 还请检查您的 firewalld 或 iptables 在所有节点上禁用服务。

    这样地: 

    apiVersion: kubelet.config.k8s.io/v1beta1
authentication:
  anonymous:
    enabled: false
  webhook:
    enabled: true
  x509:
    clientCAFile: "/var/lib/kubernetes/ca.pem"
authorization:
  mode: Webhook
clusterDomain: "cluster.local"
clusterDNS:
  - "10.33.0.10"

    kubectl get svc -n kube-system
NAME                        TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)                  AGE
kube-dns                    ClusterIP   10.33.0.10    <none>        53/UDP,53/TCP,9153/TCP   35h

    然后: 

    kubectl exec -ti net-diag-86589fd8f5-r28qq -- nslookup kubernetes.default
Server:         10.33.0.10
Address:        10.33.0.10#53

Name:   kubernetes.default.svc.cluster.local
Address: 10.33.0.1

    UPD。

    我刚刚注意到,Docker是一个容器运行时,flannel是一个网络提供者。根据我的理解,问题可能是Docker在摆弄你的iptables规则,尝试将所有Docker规则设置为prmisseve,看看是否可行。

    我不是iptables配置方面的专家,但类似的内容可能会有所帮助: 

    https://unrouted.io/2017/08/15/docker-firewall/

    此外,如果您使用的是法兰绒,请确保您使用的是正确的 iface 选项如果您正在运行非云安装,这可能非常重要。 

    https://github.com/flannel-io/flannel/blob/master/Documentation/configuration.md#key-command-line-options
    推荐文章
    关于   移动版
    代码之家 - 一站式码农服务社区
    沪ICP备11025650号