Updated cluster from 1.29 to 1.32

This commit is contained in:
oriol 2025-02-23 02:28:37 +01:00
parent 6fb2c5f27f
commit 82e41f8edf
2 changed files with 187 additions and 188 deletions

View File

@ -0,0 +1,187 @@
### Links
https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade/
Used this ansible script.
https://gitea.fihome.xyz/ofilter/ansible_update_cluster
### 1.29.14 to 1.30.0
I didn't save the output from that.
### 1.30 to 1.30.10
```text
PLAY RECAP **********************************************************************************************************************************************************************************************************************
masterk.filter.home : ok=26 changed=6 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0
slave01.filter.home : ok=24 changed=5 unreachable=0 failed=0 skipped=3 rescued=0 ignored=0
slave02.filter.home : ok=24 changed=6 unreachable=0 failed=0 skipped=3 rescued=0 ignored=0
slave03.filter.home : ok=24 changed=5 unreachable=0 failed=0 skipped=3 rescued=0 ignored=0
```
### 1.30.10 1.31.0
```shell
root@masterk:/# kubeadm upgrade plan
[preflight] Running pre-flight checks.
[preflight] Some fatal errors occurred:
[ERROR CoreDNSUnsupportedPlugins]: start version '1.11.3' not supported
[ERROR CoreDNSMigration]: CoreDNS will not be upgraded: start version '1.11.3' not supported
[preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...`
To see the stack trace of this error execute with --v=5 or higher
```
Kubeadm 1.30.10 updated CoreDNS to the 1.11.3, meanwhile Kubeadm 1.31 is expecting 1.11.1.
https://github.com/kubernetes/kubernetes/pull/126796/files#diff-b84c5a65e31001a0bf998f9b29f7fbf4e2353c86ada30d39f070bfe8fd23b8e7L136
Downgrading to 1.11.1 allowed this message to not occur again/to `kubeadm upgrade plan` -> `kubeadm upgrade apply v1.31.0` to work.
```text
[upgrade/versions] Target version: v1.31.6
[upgrade/versions] Latest version in the v1.30 series: v1.30.10
Components that must be upgraded manually after you have upgraded the control plane with 'kubeadm upgrade apply':
COMPONENT NODE CURRENT TARGET
kubelet masterk.filter.home v1.30.10 v1.31.6
kubelet slave01.filter.home v1.30.10 v1.31.6
kubelet slave02.filter.home v1.30.10 v1.31.6
kubelet slave03.filter.home v1.30.10 v1.31.6
Upgrade to the latest stable version:
COMPONENT NODE CURRENT TARGET
kube-apiserver masterk.filter.home v1.30.10 v1.31.6
kube-controller-manager masterk.filter.home v1.30.10 v1.31.6
kube-scheduler masterk.filter.home v1.30.10 v1.31.6
kube-proxy 1.30.10 v1.31.6
CoreDNS v1.11.1 v1.11.1
etcd masterk.filter.home 3.5.16-0 3.5.15-0
You can now apply the upgrade by executing the following command:
kubeadm upgrade apply v1.31.6
Note: Before you can perform this upgrade, you have to update kubeadm to v1.31.6.
```
Notice how the etcd is "ahead" of the target.
As well, how meanwhile the kubeadm package is installed as 1.31.0, the version displayed after the upgrade is 1.31.6
```text
root@masterk:/home/klussy# kubeadm version
kubeadm version: &version.Info{Major:"1", Minor:"31", GitVersion:"v1.31.0", GitCommit:"9edcffcde5595e8a5b1a35f88c421764e575afce", GitTreeState:"clean", BuildDate:"2024-08-13T07:35:57Z", GoVersion:"go1.22.5", Compiler:"gc", Platform:"linux/amd64"}
```
```text
➜ ~ kubectl get nodes -owide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
masterk.filter.home Ready control-plane 349d v1.31.6 192.168.1.9 <none> Debian GNU/Linux 12 (bookworm) 6.1.0-31-amd64 containerd://1.7.25
```
#### Result
```text
PLAY RECAP **********************************************************************************************************************************************************************************************************************
masterk.filter.home : ok=26 changed=9 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0
slave01.filter.home : ok=24 changed=9 unreachable=0 failed=0 skipped=3 rescued=0 ignored=0
slave02.filter.home : ok=24 changed=9 unreachable=0 failed=0 skipped=3 rescued=0 ignored=0
slave03.filter.home : ok=24 changed=8 unreachable=0 failed=0 skipped=3 rescued=0 ignored=0
```
```text
➜ ~ kubectl get nodes -owide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
masterk.filter.home Ready control-plane 349d v1.31.6 192.168.1.9 <none> Debian GNU/Linux 12 (bookworm) 6.1.0-31-amd64 containerd://1.7.25
slave01.filter.home Ready <none> 359d v1.31.6 192.168.1.10 <none> Armbian 25.2.1 bookworm 5.10.160-legacy-rk35xx containerd://1.7.25
slave02.filter.home Ready <none> 365d v1.31.6 192.168.1.11 <none> Armbian 24.11.1 jammy 5.10.160-legacy-rk35xx containerd://1.7.25
slave03.filter.home Ready <none> 365d v1.31.6 192.168.1.12 <none> Debian GNU/Linux 12 (bookworm) 6.1.0-31-amd64 containerd://1.7.25
```
### "1.31.0" to 1.32.0
```text
PLAY RECAP **********************************************************************************************************************************************************************************************************************
masterk.filter.home : ok=26 changed=12 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0
slave01.filter.home : ok=24 changed=12 unreachable=0 failed=0 skipped=3 rescued=0 ignored=0
slave02.filter.home : ok=24 changed=12 unreachable=0 failed=0 skipped=3 rescued=0 ignored=0
slave03.filter.home : ok=24 changed=12 unreachable=0 failed=0 skipped=3 rescued=0 ignored=0
```
```text
➜ ~ kubectl get nodes -owide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
masterk.filter.home Ready control-plane 349d v1.32.2 192.168.1.9 <none> Debian GNU/Linux 12 (bookworm) 6.1.0-31-amd64 containerd://1.7.25
slave01.filter.home Ready <none> 359d v1.32.2 192.168.1.10 <none> Armbian 25.2.1 bookworm 5.10.160-legacy-rk35xx containerd://1.7.25
slave02.filter.home Ready <none> 365d v1.32.2 192.168.1.11 <none> Armbian 24.11.1 jammy 5.10.160-legacy-rk35xx containerd://1.7.25
slave03.filter.home Ready <none> 365d v1.32.2 192.168.1.12 <none> Debian GNU/Linux 12 (bookworm) 6.1.0-31-amd64 containerd://1.7.25
```
version 1.32.2
### "1.32.0" to 1.32.2
```text
"[preflight] Running pre-flight checks.",
"[upgrade/config] Reading configuration from the \"kubeadm-config\" ConfigMap in namespace \"kube-system\"...",
"[upgrade/config] Use 'kubeadm init phase upload-config --config your-config.yaml' to re-upload it.",
"[upgrade] Running cluster health checks",
"[upgrade] Fetching available versions to upgrade to",
"[upgrade/versions] Cluster version: 1.32.0",
"[upgrade/versions] kubeadm version: v1.32.2",
"[upgrade/versions] Target version: v1.32.2",
"[upgrade/versions] Latest version in the v1.32 series: v1.32.2",
"",
"Components that must be upgraded manually after you have upgraded the control plane with 'kubeadm upgrade apply':",
"COMPONENT NODE CURRENT TARGET",
"",
"Upgrade to the latest version in the v1.32 series:",
"",
"COMPONENT NODE CURRENT TARGET",
"kube-apiserver masterk.filter.home v1.32.0 v1.32.2",
"kube-controller-manager masterk.filter.home v1.32.0 v1.32.2",
"kube-scheduler masterk.filter.home v1.32.0 v1.32.2",
"kube-proxy 1.32.0 v1.32.2",
"CoreDNS v1.11.3 v1.11.3",
"etcd masterk.filter.home 3.5.16-0 3.5.16-0",
"",
"You can now apply the upgrade by executing the following command:",
"",
"\tkubeadm upgrade apply v1.32.2",
"",
"_____________________________________________________________________",
"",
"",
"The table below shows the current state of component configs as understood by this version of kubeadm.",
"Configs that have a \"yes\" mark in the \"MANUAL UPGRADE REQUIRED\" column require manual config upgrade or",
"resetting to kubeadm defaults before a successful upgrade can be performed. The version to manually",
"upgrade to is denoted in the \"PREFERRED VERSION\" column.",
"",
"API GROUP CURRENT VERSION PREFERRED VERSION MANUAL UPGRADE REQUIRED",
"kubeproxy.config.k8s.io v1alpha1 v1alpha1 no",
"kubelet.config.k8s.io v1beta1 v1beta1 no",
"_____________________________________________________________________"
```
```text
PLAY RECAP **********************************************************************************************************************************************************************************************************************
masterk.filter.home : ok=26 changed=9 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0
slave01.filter.home : ok=24 changed=8 unreachable=0 failed=0 skipped=3 rescued=0 ignored=0
slave02.filter.home : ok=24 changed=8 unreachable=0 failed=0 skipped=3 rescued=0 ignored=0
slave03.filter.home : ok=24 changed=8 unreachable=0 failed=0 skipped=3 rescued=0 ignored=0
```
```text
➜ ~ kubectl get nodes -owide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
masterk.filter.home Ready control-plane 349d v1.32.2 192.168.1.9 <none> Debian GNU/Linux 12 (bookworm) 6.1.0-31-amd64 containerd://1.7.25
slave01.filter.home Ready <none> 359d v1.32.2 192.168.1.10 <none> Armbian 25.2.1 bookworm 5.10.160-legacy-rk35xx containerd://1.7.25
slave02.filter.home Ready <none> 365d v1.32.2 192.168.1.11 <none> Armbian 24.11.1 jammy 5.10.160-legacy-rk35xx containerd://1.7.25
slave03.filter.home Ready <none> 365d v1.32.2 192.168.1.12 <none> Debian GNU/Linux 12 (bookworm) 6.1.0-31-amd64 containerd://1.7.25
```

View File

@ -45,192 +45,4 @@ Upgraded from 1.15.1 to 1.17 ✅
Upgraded from 1.29.14 to 1.32.2 ✅
### Links
https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade/
Used this ansible script.
https://gitea.fihome.xyz/ofilter/ansible_update_cluster
### 1.29.14 to 1.30.0
I didn't save the output from that.
### 1.30 to 1.30.10
```text
PLAY RECAP **********************************************************************************************************************************************************************************************************************
masterk.filter.home : ok=26 changed=6 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0
slave01.filter.home : ok=24 changed=5 unreachable=0 failed=0 skipped=3 rescued=0 ignored=0
slave02.filter.home : ok=24 changed=6 unreachable=0 failed=0 skipped=3 rescued=0 ignored=0
slave03.filter.home : ok=24 changed=5 unreachable=0 failed=0 skipped=3 rescued=0 ignored=0
```
### 1.30.10 1.31.0
```shell
root@masterk:/# kubeadm upgrade plan
[preflight] Running pre-flight checks.
[preflight] Some fatal errors occurred:
[ERROR CoreDNSUnsupportedPlugins]: start version '1.11.3' not supported
[ERROR CoreDNSMigration]: CoreDNS will not be upgraded: start version '1.11.3' not supported
[preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...`
To see the stack trace of this error execute with --v=5 or higher
```
Kubeadm 1.30.10 updated CoreDNS to the 1.11.3, meanwhile Kubeadm 1.31 is expecting 1.11.1.
https://github.com/kubernetes/kubernetes/pull/126796/files#diff-b84c5a65e31001a0bf998f9b29f7fbf4e2353c86ada30d39f070bfe8fd23b8e7L136
Downgrading to 1.11.1 allowed this message to not occur again/to `kubeadm upgrade plan` -> `kubeadm upgrade apply v1.31.0` to work.
```text
[upgrade/versions] Target version: v1.31.6
[upgrade/versions] Latest version in the v1.30 series: v1.30.10
Components that must be upgraded manually after you have upgraded the control plane with 'kubeadm upgrade apply':
COMPONENT NODE CURRENT TARGET
kubelet masterk.filter.home v1.30.10 v1.31.6
kubelet slave01.filter.home v1.30.10 v1.31.6
kubelet slave02.filter.home v1.30.10 v1.31.6
kubelet slave03.filter.home v1.30.10 v1.31.6
Upgrade to the latest stable version:
COMPONENT NODE CURRENT TARGET
kube-apiserver masterk.filter.home v1.30.10 v1.31.6
kube-controller-manager masterk.filter.home v1.30.10 v1.31.6
kube-scheduler masterk.filter.home v1.30.10 v1.31.6
kube-proxy 1.30.10 v1.31.6
CoreDNS v1.11.1 v1.11.1
etcd masterk.filter.home 3.5.16-0 3.5.15-0
You can now apply the upgrade by executing the following command:
kubeadm upgrade apply v1.31.6
Note: Before you can perform this upgrade, you have to update kubeadm to v1.31.6.
```
Notice how the etcd is "ahead" of the target.
As well, how meanwhile the kubeadm package is installed as 1.31.0, the version displayed after the upgrade is 1.31.6
```text
root@masterk:/home/klussy# kubeadm version
kubeadm version: &version.Info{Major:"1", Minor:"31", GitVersion:"v1.31.0", GitCommit:"9edcffcde5595e8a5b1a35f88c421764e575afce", GitTreeState:"clean", BuildDate:"2024-08-13T07:35:57Z", GoVersion:"go1.22.5", Compiler:"gc", Platform:"linux/amd64"}
```
```text
➜ ~ kubectl get nodes -owide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
masterk.filter.home Ready control-plane 349d v1.31.6 192.168.1.9 <none> Debian GNU/Linux 12 (bookworm) 6.1.0-31-amd64 containerd://1.7.25
```
#### Result
```text
PLAY RECAP **********************************************************************************************************************************************************************************************************************
masterk.filter.home : ok=26 changed=9 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0
slave01.filter.home : ok=24 changed=9 unreachable=0 failed=0 skipped=3 rescued=0 ignored=0
slave02.filter.home : ok=24 changed=9 unreachable=0 failed=0 skipped=3 rescued=0 ignored=0
slave03.filter.home : ok=24 changed=8 unreachable=0 failed=0 skipped=3 rescued=0 ignored=0
```
```text
➜ ~ kubectl get nodes -owide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
masterk.filter.home Ready control-plane 349d v1.31.6 192.168.1.9 <none> Debian GNU/Linux 12 (bookworm) 6.1.0-31-amd64 containerd://1.7.25
slave01.filter.home Ready <none> 359d v1.31.6 192.168.1.10 <none> Armbian 25.2.1 bookworm 5.10.160-legacy-rk35xx containerd://1.7.25
slave02.filter.home Ready <none> 365d v1.31.6 192.168.1.11 <none> Armbian 24.11.1 jammy 5.10.160-legacy-rk35xx containerd://1.7.25
slave03.filter.home Ready <none> 365d v1.31.6 192.168.1.12 <none> Debian GNU/Linux 12 (bookworm) 6.1.0-31-amd64 containerd://1.7.25
```
### "1.31.0" to 1.32.0
```text
PLAY RECAP **********************************************************************************************************************************************************************************************************************
masterk.filter.home : ok=26 changed=12 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0
slave01.filter.home : ok=24 changed=12 unreachable=0 failed=0 skipped=3 rescued=0 ignored=0
slave02.filter.home : ok=24 changed=12 unreachable=0 failed=0 skipped=3 rescued=0 ignored=0
slave03.filter.home : ok=24 changed=12 unreachable=0 failed=0 skipped=3 rescued=0 ignored=0
```
```text
➜ ~ kubectl get nodes -owide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
masterk.filter.home Ready control-plane 349d v1.32.2 192.168.1.9 <none> Debian GNU/Linux 12 (bookworm) 6.1.0-31-amd64 containerd://1.7.25
slave01.filter.home Ready <none> 359d v1.32.2 192.168.1.10 <none> Armbian 25.2.1 bookworm 5.10.160-legacy-rk35xx containerd://1.7.25
slave02.filter.home Ready <none> 365d v1.32.2 192.168.1.11 <none> Armbian 24.11.1 jammy 5.10.160-legacy-rk35xx containerd://1.7.25
slave03.filter.home Ready <none> 365d v1.32.2 192.168.1.12 <none> Debian GNU/Linux 12 (bookworm) 6.1.0-31-amd64 containerd://1.7.25
```
version 1.32.2
### "1.32.0" to 1.32.2
```text
"[preflight] Running pre-flight checks.",
"[upgrade/config] Reading configuration from the \"kubeadm-config\" ConfigMap in namespace \"kube-system\"...",
"[upgrade/config] Use 'kubeadm init phase upload-config --config your-config.yaml' to re-upload it.",
"[upgrade] Running cluster health checks",
"[upgrade] Fetching available versions to upgrade to",
"[upgrade/versions] Cluster version: 1.32.0",
"[upgrade/versions] kubeadm version: v1.32.2",
"[upgrade/versions] Target version: v1.32.2",
"[upgrade/versions] Latest version in the v1.32 series: v1.32.2",
"",
"Components that must be upgraded manually after you have upgraded the control plane with 'kubeadm upgrade apply':",
"COMPONENT NODE CURRENT TARGET",
"",
"Upgrade to the latest version in the v1.32 series:",
"",
"COMPONENT NODE CURRENT TARGET",
"kube-apiserver masterk.filter.home v1.32.0 v1.32.2",
"kube-controller-manager masterk.filter.home v1.32.0 v1.32.2",
"kube-scheduler masterk.filter.home v1.32.0 v1.32.2",
"kube-proxy 1.32.0 v1.32.2",
"CoreDNS v1.11.3 v1.11.3",
"etcd masterk.filter.home 3.5.16-0 3.5.16-0",
"",
"You can now apply the upgrade by executing the following command:",
"",
"\tkubeadm upgrade apply v1.32.2",
"",
"_____________________________________________________________________",
"",
"",
"The table below shows the current state of component configs as understood by this version of kubeadm.",
"Configs that have a \"yes\" mark in the \"MANUAL UPGRADE REQUIRED\" column require manual config upgrade or",
"resetting to kubeadm defaults before a successful upgrade can be performed. The version to manually",
"upgrade to is denoted in the \"PREFERRED VERSION\" column.",
"",
"API GROUP CURRENT VERSION PREFERRED VERSION MANUAL UPGRADE REQUIRED",
"kubeproxy.config.k8s.io v1alpha1 v1alpha1 no",
"kubelet.config.k8s.io v1beta1 v1beta1 no",
"_____________________________________________________________________"
```
```text
PLAY RECAP **********************************************************************************************************************************************************************************************************************
masterk.filter.home : ok=26 changed=9 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0
slave01.filter.home : ok=24 changed=8 unreachable=0 failed=0 skipped=3 rescued=0 ignored=0
slave02.filter.home : ok=24 changed=8 unreachable=0 failed=0 skipped=3 rescued=0 ignored=0
slave03.filter.home : ok=24 changed=8 unreachable=0 failed=0 skipped=3 rescued=0 ignored=0
```
```text
➜ ~ kubectl get nodes -owide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
masterk.filter.home Ready control-plane 349d v1.32.2 192.168.1.9 <none> Debian GNU/Linux 12 (bookworm) 6.1.0-31-amd64 containerd://1.7.25
slave01.filter.home Ready <none> 359d v1.32.2 192.168.1.10 <none> Armbian 25.2.1 bookworm 5.10.160-legacy-rk35xx containerd://1.7.25
slave02.filter.home Ready <none> 365d v1.32.2 192.168.1.11 <none> Armbian 24.11.1 jammy 5.10.160-legacy-rk35xx containerd://1.7.25
slave03.filter.home Ready <none> 365d v1.32.2 192.168.1.12 <none> Debian GNU/Linux 12 (bookworm) 6.1.0-31-amd64 containerd://1.7.25
```