Kubernetes FAQs

`crictl images` Command Execution Failed

Applicable Version: Kubernetes 1.35.3, kernel version < 4.11

Error Message:

FATA[0000] validate service connection: validate CRI v1 image API for endpoint "unix:///run/containerd/containerd.sock": rpc error: code = Unimplemented desc = unknown service runtime.v1.ImageService 

Solution:

Disable the enable_unprivileged_ports and enable_unprivileged_icmp settings in containerd:

# Check the current configuration
grep -nE 'unprivileged_(icmp|port)' /etc/containerd/config.toml

# Set both options to false
sed -i \
    -e 's/^\(\s*enable_unprivileged_ports\s*=\s*\)true/\1false/' \
    -e 's/^\(\s*enable_unprivileged_icmp\s*=\s*\)true/\1false/' \
/etc/containerd/config.toml

# Verify the changes
grep -nE 'unprivileged_(icmp|port)' /etc/containerd/config.toml

systemctl restart containerd

After restarting, run the following command to verify plugin status. All plugins showing ok indicates normal operation:

ctr plugin ls | grep -E 'cri|io.containerd.grpc.v1.cri'

Expected output

io.containerd.cri.v1                      images                   -              ok        
io.containerd.cri.v1                      runtime                  linux/amd64    ok        
io.containerd.grpc.v1                     cri                      -              ok

Calico Installation Failed

Applicable Version: Kubernetes 1.35.3

Error Message:

kubectl -n kube-system logs calico-node-xxx -c upgrade-ipam

This program can only be run on AMD64 processors with v2 microarchitecture support.

Cause: The CPU does not support the x86-64-v2 architecture. Starting from Calico v3.31.1, only x86-64-v2 and above architectures are supported. Reference: Calico GitHub Issue #11352

Solution: Switch to Calico 3.30.7

Download and extract the installation package, then import the images

wget https://pdpublic.mingdao.com/private-deployment/offline/common/kubernetes-1.35.3/calico-3.30.7-yaml-images-amd64.tar.gz

tar xzvf calico-3.30.7-yaml-images-amd64.tar.gz
cd calico-3.30.7-yaml-images-amd64
gunzip -d calico-3.30.7-images-amd64.tar.gz
ctr -n k8s.io image import calico-3.30.7-images-amd64.tar

Move calico.yaml to the installation directory

Run this command from within the calico-3.30.7-yaml-images-amd64 directory
```
mv calico.yaml /usr/local/kubernetes/
```

Modify the configuration file

Replace the image registry address

sed -ri 's|image: docker.io/calico|image: 127.0.0.1:5000|g' /usr/local/kubernetes/calico.yaml

grep image: /usr/local/kubernetes/calico.yaml

Example output

    image: 127.0.0.1:5000/cni:v3.30.7
    image: 127.0.0.1:5000/cni:v3.30.7
    image: 127.0.0.1:5000/node:v3.30.7
    image: 127.0.0.1:5000/node:v3.30.7
    image: 127.0.0.1:5000/kube-controllers:v3.30.7

Configure the Pod CIDR

sed -ri '/# - name: CALICO_IPV4POOL_CIDR/,/#   value: ".*"/ {
  s/# - name: CALICO_IPV4POOL_CIDR/- name: CALICO_IPV4POOL_CIDR/
  s/#   value: ".*"/  value: "10.244.0.0\/16"/
}' /usr/local/kubernetes/calico.yaml

grep -C 2 CALICO_IPV4POOL_CIDR /usr/local/kubernetes/calico.yaml

Example output

      # chosen from this range. Changing this value after installation will have
      # no effect. This should fall within `--cluster-cidr`.
      - name: CALICO_IPV4POOL_CIDR
        value: "10.244.0.0/16"
      # Disable file logging so `kubectl logs` works.

Configure the CNI binary path

sed -i '/- name: cni-bin-dir/,/type:/s|path: .*|path: /usr/local/kubernetes/cni/bin|' /usr/local/kubernetes/calico.yaml

grep -C 2 cni-bin-dir /usr/local/kubernetes/calico.yaml

Example output

        name: host-local-net-dir
      - mountPath: /host/opt/cni/bin
        name: cni-bin-dir
    securityContext:
      privileged: true
--
    volumeMounts:
      - mountPath: /host/opt/cni/bin
        name: cni-bin-dir
      - mountPath: /host/etc/cni/net.d
        name: cni-net-dir
--
      path: /proc
  # Used to install CNI.
  - name: cni-bin-dir
    hostPath:
      path: /usr/local/kubernetes/cni/bin

Deploy Calico

kubectl apply -f /usr/local/kubernetes/calico.yaml

calico-node Pod Health Check Failed

Applicable Version: Kubernetes 1.35.3

Error Message:

Warning  Unhealthy  115s  kubelet  Readiness probe errored and resulted in unknown state: rpc error: code = Unknown desc = failed to exec in container: failed to start exec "...": OCI runtime exec failed: exec failed: unable to start container process: error adding pid 23804 to cgroups: Unknown method 'AttachProcessesToUnit' or interface 'org.freedesktop.systemd1.Manager'

Cause: The system does not support the systemd cgroup driver; the cgroup drivers of containerd and kubelet are incompatible.

Solution:

Switch the containerd cgroup driver from systemd to cgroupfs and reinitialize the cluster:

Modify the containerd configuration

sed -i 's/SystemdCgroup = true/SystemdCgroup = false/g' /etc/containerd/config.toml

grep -n "SystemdCgroup" /etc/containerd/config.toml
systemctl restart containerd

Append the kubelet cgroup configuration to kubeadm-config.yaml, then re-run kubeadm init to reinitialize

cd /usr/local/kubernetes/
cat >> kubeadm-config.yaml <<'EOF'

---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
cgroupDriver: cgroupfs
EOF

crictl images Command Execution Failed​

Calico Installation Failed​

calico-node Pod Health Check Failed​

`crictl images` Command Execution Failed

Calico Installation Failed

calico-node Pod Health Check Failed