ofilter/ansible_kubernetes_cluster
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Slight modifications

Browse Source 
Improved README.md
This commit is contained in:
Oriol 2023-03-13 08:32:24 +00:00
parent 4d2fb39e47
commit 668d66c583
10 changed files with 404 additions and 326 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

108
README.md
View File

@ -12,47 +12,122 @@ arm_initial_setup.yaml: standalone playbook to normalize the initialization of a
run.sh: placeholder script used for testing
ksetup/:
- playbook.yaml
- tasks_prepare_node.yaml
- tasks_master.yaml
- tasks_slave.yaml
- tasks_end.yaml
- kubeadm-join.command
inventory.yaml: Example inventory
```
## Inventory.yaml
Example inventory.
Contains 2 groups, `masters` and `slaves`.
Each group have the variable `is_master` configured.
For more info regarding Ansible Inventory, refer to their [documentation regarding the topic](https://docs.ansible.com/ansible/latest/inventory_guide/index.html).
## arm_initial_setup.yaml
It will:
- Set the loacale for ROOT user
- Set the language for ROOT user
- Create `wheel` group
- Add `wheel` group to sudoers (using password)
- Set the `loacale` for the ROOT user
- Set the `language` for the ROOT user
- Create the `wheel` group
- Add the `wheel` group to sudoers (using password)
- Add a new user with its password
- Add the new user to the `wheel` group
- Change ROOT password
- Disable SSH to the ROOT user
- Executes `sleep 1 && dhclient -r && dhclient && reboot`. The `dhclient -r` it's for my own usage so **modify it if it bothers you**.
- Change the ROOT password
- Disable SSH login to the ROOT user
- Executes `sleep 1 && dhclient -r && dhclient && reboot`. The `dhclient` is for my own usage so **modify it if it bothers you**.
### Vars
A reminder that the point of this script, is to normalize the process of setting up a **fresh** ARMBIAN OS.
```yaml
# Connect
ansible_user: User used to connect
ansible_password: Password used to connect
ansible_become_password: Password used to execute `sudo`
# New values
## Users
new_user_name: Username to create/modify
new_user_pass: Password for the used to create/modify
new_root_pass: New password for root
## Locales
new_locale: New locale desired
new_language: New language desired
```
## ksetup
### playbook.yaml
Used to "orchestrate" the process and call the rest of the **task** playbooks.
Used to "orchestrate" the process and call the rest of the **task** playbooks.
#### Vars
```yaml
# Testing purposes
ansible_user: User used to connect
ansible_password: Password used to connect
ansible_become_password: Password used to execute `sudo`
# Actual vars
set_hostname: New hostname for the device, by default will obtain it's values from the variable `ansible_host` aka. the URL specified to connect to such device.
is_master: This variable determines which playbooks will run.
# Cluster shit
kubeadm_join_path: File path that will store the `kubeadm join` command to be executed by the worker nodes.
_kubeadm_join_command: Placeholder, will be populated at later stages of the script.
```
### TASK playbooks
#### tasks_prepare_node.yaml
- list (TODO)
- Changes the device hostname
- Disables RAM usage.
- Update -> Upgrade.
- Uninstalls `containerd`.
- Installs Docker (Debian) and Kubernetes repos.
- Installs `containerd.io`, `kublet`, `kubeadm`, `kubectl`.
- Sets default config for `containerd` with cGroups enabled.
- Enables some `iptables`.
- "Resets" `/etc/hosts` file
#### tasks_master.yaml
##### Will only be executed if the variable `is_master` is set to `True`
- Executes `kubeadm init` aka initializes the node without further arguments assigned.
- Sets the `kubectl` conf to the user specified.
- Deploys `Calico` network plugin (might change in the future)
- Removes `node-role.kubernetes.io/control-plane` taint to allow deploying containers in the control plane node.
- Generates a "join cluster" command and stores it in a file **LOCALLY** as `kubeadm-join.command`, this file will be later used by the slave nodes, who will execute this file's contents.
#### tasks_slave.yaml
##### Will only be executed if the variable `is_master` is set to `False`
- Executes the "command" stored in the file `kubeadm-join.command` located **LOCALLY**.
#### tasks_end.yaml
- As per the moment, only reboots.
# USAGE
## Setup
On my infrastructure, **I** use a DHCP and DNS to connect / communicate the nodes.
On my infrastructure/environment, **I** use a DHCP and DNS to connect / communicate the nodes.
You **might** need to edit the file `/etc/hosts` and <u>manually</u> point the resources IP addresses.
@ -60,16 +135,16 @@ So my first step is to reserve the MAC addresses and configure the DHCP server.
Afterwards update the DNS server to point to those IPs.
## Set the SD card / SBC (Single Board Computer)
### Set the SD card / SBC (Single Board Computer)
Beware of using the right IMG / device, as you don't want to <u>delete the wrong drive</u>, **right?**
```shell
dd if=Armbian_23.02.2_Orangepi5_bullseye_legacy_5.10.110_minimal.img of=/dev/sdg status=progress bs=1M status=progress
```
## arm_initial_setup.yaml
### arm_initial_setup.yaml
### Change the values of the desired variables
#### Change the values of the desired variables
```yaml
# New values
@ -84,9 +159,9 @@ new_language: "en_US.UTF-8"
```
### Change the connection variables (Optional)
#### Change the connection variables (Optional)
**Optional**, if you are not planning to use root, the playbook might require some slight changes in order to work with an user that's not root, idk, not my problem, this playbook <u>**assumes**</u> you will be using an ARMBIAN image.
**Optional**, if you are not planning to use root, the playbook might require some slight changes in order to work with a user that's not root, idk, not my problem, this playbook <u>**assumes**</u> you will be using an ARMBIAN image.
```yaml
ansible_user: "root"
@ -96,7 +171,7 @@ ansible_password: "1234"
## Run (?)
@ -104,4 +179,5 @@ ansible_password: "1234"
# License
## DWTFUW
Do whatever the fuck you want license ™

8
inventory.yaml
View File

@ -1,8 +1,10 @@
master:
masters:
hosts:
masterk.filter.home:
is_master: yes
vars:
is_master: yes
slaves:
hosts:
slave[01:01].filter.home:
is_master: no
vars:
is_master: no

1
ksetup/kubeadm-join.command
View File

@ -1 +0,0 @@
kubeadm join 192.168.1.10:6443 --token ezekcz.n6hwck49wfvj4h6f --discovery-token-ca-cert-hash sha256:7f78629fddc1310a35d2534d5dafa77761fa9770ff4da871583e32bc549470f7

13
ksetup/playbook.yaml
View File

@ -4,11 +4,12 @@
# Maybe still works for orangepi "official" versions, but I don't care about them unless I used soooooo... gl!
# https://medium.com/karlmax-berlin/how-to-install-kubernetes-on-raspberry-pi-53b4ce300b58
- name: Preparethings
order: inventory
hosts: all
gather_facts: true
vars:
# Testing purpouses
# Testing purposes
ansible_user: "orangepi" # Testing purposes
ansible_password: "orangepi" # Testing purposes
ansible_become_password: "orangepi" # Testing purposes
@ -19,7 +20,7 @@
# Cluster shit
kubeadm_join_path: "./kubeadm-join.command"
kubeadm_join_command: ""
_kubeadm_join_command: "" #placeholder
tasks:
@ -29,19 +30,19 @@
# Init / Basic setup
- name: set up node
import_tasks: set_node_tasks.yaml
import_tasks: tasks_prepare_node.yaml
become: true
# If is_master: init
- name: init cluster
import_tasks: master_tasks.yaml
import_tasks: tasks_master.yaml
when: is_master
# else: join
- name: join cluster
import_tasks: slave_tasks.yaml
import_tasks: tasks_slave.yaml
when: not is_master
# Do other stuff
- name: post setup
import_tasks: end_tasks.yaml
import_tasks: tasks_end.yaml

8
ksetup/slave_tasks.yaml
View File

@ -1,8 +0,0 @@
- name: Populate {{ kubeadm_join_command }}
ansible.builtin.set_fact: kubeadm_join_command="{{ lookup('file', kubeadm_join_path ) }}"
- debug: var=kubeadm_join_command
- name: Join kubeadm
ansible.builtin.command: "{{ kubeadm_join_command }}"
become: yes

10
ksetup/end_tasks.yaml → ksetup/tasks_end.yaml
View File

@ -1,5 +1,5 @@
#reboot
- name: reboot
reboot:
#reboot
- name: reboot
reboot:

170
ksetup/master_tasks.yaml → ksetup/tasks_master.yaml
View File

@ -1,85 +1,85 @@
# Init
- name: Init cluster
ansible.builtin.command: "kubeadm init"
become: true
# Set kubectl tool
- user:
name: "{{ ansible_user_id }}"
state: present
register: user_info_registered
- name: create .kube directory
become: yes
become_user: "{{ ansible_user_id }}"
file:
path: "{{ user_info_registered.home }}/.kube"
state: directory
mode: 0755
- debug: var=user_info_registered.home
- name: copy admin.conf to user's kube config
copy:
src: /etc/kubernetes/admin.conf
remote_src: yes
dest: "{{ user_info_registered.home }}/.kube/config"
owner: "{{ ansible_user_id }}"
become: true
# Network Plugin
# Download and install Flannel
#- name: Download and install Flannel
# ansible.builtin.get_url:
# url: https://github.com/flannel-io/flannel/releases/download/v0.19.2/flanneld-arm64
# dest: /usr/local/bin/flanneld
# owner: root
# group: root
# mode: '0755'
#
#- name: Create Flannel networks directory
# ansible.builtin.file:
# path: /var/lib/k8s/flannel/networks
# state: directory
# recurse: yes
# owner: root
# group: root
# mode: '0755'
#-
#
- name: Calico
ansible.builtin.command: "kubectl create -f https://raw.githubusercontent.com/projectcalico/calico/v3.25.0/manifests/calico.yaml"
#- name: Calico Tigera ?
# ansible.builtin.shell: "kubectl create -f https://raw.githubusercontent.com/projectcalico/calico/v3.25.0/manifests/tigera-operator.yaml"
# become_user: some_user
#- name: Calico Custom resources ?
# ansible.builtin.command: "kubectl create -f https://raw.githubusercontent.com/projectcalico/calico/v3.25.0/manifests/custom-resources.yaml"
#
#
## Remove taints
- name: Remove Taint (allows deployment in control plane)
ansible.builtin.shell: "kubectl taint nodes --all node-role.kubernetes.io/control-plane-"
# Join token / command
- name: Generate join token
shell: kubeadm token create --print-join-command
register: kubeadm_join_cmd
- set_fact:
kubeadm_join_command: "{{ kubeadm_join_cmd.stdout }}"
- debug: var=kubeadm_join_command
- name: Store join command in "{{ kubeadm_join_path }}"
copy:
dest: "{{ kubeadm_join_path }}"
content: |
{{ kubeadm_join_command }}
delegate_to: localhost
# Init
- name: Init cluster
ansible.builtin.command: "kubeadm init"
become: true
# Set kubectl tool
- user:
name: "{{ ansible_user_id }}"
state: present
register: user_info_registered
- name: create .kube directory
become: yes
become_user: "{{ ansible_user_id }}"
file:
path: "{{ user_info_registered.home }}/.kube"
state: directory
mode: 0755
- debug: var=user_info_registered.home
- name: copy admin.conf to user's kube config
copy:
src: /etc/kubernetes/admin.conf
remote_src: yes
dest: "{{ user_info_registered.home }}/.kube/config"
owner: "{{ ansible_user_id }}"
become: true
# Network Plugin
# Download and install Flannel
#- name: Download and install Flannel
# ansible.builtin.get_url:
# url: https://github.com/flannel-io/flannel/releases/download/v0.19.2/flanneld-arm64
# dest: /usr/local/bin/flanneld
# owner: root
# group: root
# mode: '0755'
#
#- name: Create Flannel networks directory
# ansible.builtin.file:
# path: /var/lib/k8s/flannel/networks
# state: directory
# recurse: yes
# owner: root
# group: root
# mode: '0755'
#-
#
- name: Calico
ansible.builtin.command: "kubectl create -f https://raw.githubusercontent.com/projectcalico/calico/v3.25.0/manifests/calico.yaml"
#- name: Calico Tigera ?
# ansible.builtin.shell: "kubectl create -f https://raw.githubusercontent.com/projectcalico/calico/v3.25.0/manifests/tigera-operator.yaml"
# become_user: some_user
#- name: Calico Custom resources ?
# ansible.builtin.command: "kubectl create -f https://raw.githubusercontent.com/projectcalico/calico/v3.25.0/manifests/custom-resources.yaml"
#
#
## Remove taints
- name: Remove Taint (allows deployment in control plane)
ansible.builtin.shell: "kubectl taint nodes --all node-role.kubernetes.io/control-plane-"
# Join token / command
- name: Generate join token
shell: kubeadm token create --print-join-command
register: kubeadm_join_cmd
- set_fact:
kubeadm_join_command: "{{ kubeadm_join_cmd.stdout }}"
- debug: var=kubeadm_join_command
- name: Store join command in "{{ kubeadm_join_path }}"
copy:
dest: "{{ kubeadm_join_path }}"
content: |
{{ kubeadm_join_command }}
delegate_to: localhost

402
ksetup/set_node_tasks.yaml → ksetup/tasks_prepare_node.yaml
View File

@ -1,201 +1,201 @@
# Hostname
## Set hostname
- name: Set a hostname
ansible.builtin.hostname:
name: "{{ set_hostname }}"
# Swap
- name: Swapoff
ansible.builtin.command: swapoff -a
- name: Disable ram on boot (orangepi) # Untested
copy:
dest: "/etc/default/orangepi-zram-config"
content: ENABLED=false
when:
- ansible_distribution | lower == "orangepi"
- ansible_architecture == "aarch64"
- name: Disable ram on boot (armbian) # Untested
copy:
dest: "/etc/default/armbian-zram-config"
content: ENABLED=false
when:
- ansible_architecture == "aarch64"
- ansible_distribution | lower == "ubuntu" or ansible_distribution | lower == "debian"
# INTENDED FOR ARM DISTROS FUCK U
#- name: Sed when x86_64
# ansible.builtin.command: sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab
# when: ansible_architecture == "x86_64"
# Packages
# Delete default containerd
## Looking forward the version 1.6
- name: apt prune containerd
ansible.builtin.apt:
name: containerd
state: absent
purge: true
## BnB
- name: apt update
ansible.builtin.apt:
update_cache: yes
- name: apt upgrade
ansible.builtin.apt:
name: "*"
state: latest
## Keyrings directory
- name: Creating a new directory
file:
path: "/etc/apt/keyrings"
state: directory
recurse: true
mode: '0755'
# ignore_errors: true
## Docker repo
- name: Add Docker GPG key
apt_key:
url: https://download.docker.com/linux/debian/gpg
state: present
- name: Add Docker APT repository
apt_repository:
repo: deb [arch=arm64] https://download.docker.com/linux/debian bullseye stable
state: present
## Kubeshit repo
- name: Download Kubernetes GPG key
ansible.builtin.get_url:
url: "https://packages.cloud.google.com/apt/doc/apt-key.gpg"
dest: "/etc/apt/keyrings/kubernetes-archive-keyring.gpg"
timeout: 10
- name: Add Kubernetes keyring
copy:
dest: "/etc/apt/sources.list.d/kubernetes.list"
content: "deb [signed-by=/etc/apt/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main"
## Install packages
- name: apt update
ansible.builtin.apt:
update_cache: yes
- name: Install Kubelet Kubeadm Kubectl
ansible.builtin.apt:
pkg:
- kubelet
- kubeadm
- kubectl
- name: Hold kubeadm
ansible.builtin.dpkg_selections:
name: kubeadm
selection: hold
- name: Hold kubelet
ansible.builtin.dpkg_selections:
name: kubelet
selection: hold
- name: Hold kubectl
ansible.builtin.dpkg_selections:
name: kubectl
selection: hold
## Containerd
- name: Install Container Runtime
ansible.builtin.apt:
pkg:
- containerd.io
- containernetworking-plugins
- name: Containerd set default config
ansible.builtin.shell: containerd config default | tee /etc/containerd/config.toml
become: yes
- name: SystemdCgroup = true
lineinfile:
dest: /etc/containerd/config.toml
regexp: '^\s*SystemdCgroup = false$'
line: ' SystemdCgroup = true'
backrefs: yes
# Iptables
## Set files
- name: Iptables thingies (not touching specific firewall rules.)
copy:
dest: "/etc/sysctl.d/k8s.conf"
content: |
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward = 1
- name: Iptables thingies
copy:
dest: "/etc/modules-load.d/k8s.conf"
content: |
overlay
br_netfilter
## Modprobe
- name: Add the overlay module
community.general.modprobe:
name: overlay
state: present
- name: Add the br_netfilter module
community.general.modprobe:
name: br_netfilter
state: present
- name: Apply changes (might need to use sysctl module with the reload flag, will try eventually)
ansible.builtin.command: "sysctl --system"
## Systemctl
### Enable
- name: Enable kubelet
ansible.builtin.systemd:
name: kubelet
enabled: true
- name: Enable containerd
ansible.builtin.systemd:
name: containerd
enabled: true
### Restart
- name: Enable kubelet
ansible.builtin.systemd:
name: kubelet
state: restarted
- name: Restart containerd
ansible.builtin.systemd:
name: containerd
state: restarted
## Set /etc/hosts
### This could be better but who cares
- name: Set /etc/hosts file content (template/base)
copy:
dest: "/etc/hosts"
content: |
127.0.0.1 localhost
127.0.1.1 {{ set_hostname }}
::1 localhost {{ set_hostname }} ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
# Hostname
## Set hostname
- name: Set a hostname
ansible.builtin.hostname:
name: "{{ set_hostname }}"
# Swap
- name: Swapoff
ansible.builtin.command: swapoff -a
- name: Disable ram on boot (orangepi) # Untested
copy:
dest: "/etc/default/orangepi-zram-config"
content: ENABLED=false
when:
- ansible_distribution | lower == "orangepi"
- ansible_architecture == "aarch64"
- name: Disable ram on boot (armbian) # Untested
copy:
dest: "/etc/default/armbian-zram-config"
content: ENABLED=false
when:
- ansible_architecture == "aarch64"
- ansible_distribution | lower == "ubuntu" or ansible_distribution | lower == "debian"
# INTENDED FOR ARM DISTROS FUCK U
#- name: Sed when x86_64
# ansible.builtin.command: sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab
# when: ansible_architecture == "x86_64"
# Packages
# Delete default containerd
## Looking forward the version 1.6
- name: apt prune containerd
ansible.builtin.apt:
name: containerd
state: absent
purge: true
## BnB
- name: apt update
ansible.builtin.apt:
update_cache: yes
- name: apt upgrade
ansible.builtin.apt:
name: "*"
state: latest
## Keyrings directory
- name: Creating a new directory
file:
path: "/etc/apt/keyrings"
state: directory
recurse: true
mode: '0755'
# ignore_errors: true
## Docker repo
- name: Add Docker GPG key
apt_key:
url: https://download.docker.com/linux/debian/gpg
state: present
- name: Add Docker APT repository
apt_repository:
repo: deb [arch=arm64] https://download.docker.com/linux/debian bullseye stable
state: present
## Kubeshit repo
- name: Download Kubernetes GPG key
ansible.builtin.get_url:
url: "https://packages.cloud.google.com/apt/doc/apt-key.gpg"
dest: "/etc/apt/keyrings/kubernetes-archive-keyring.gpg"
timeout: 10
- name: Add Kubernetes keyring
copy:
dest: "/etc/apt/sources.list.d/kubernetes.list"
content: "deb [signed-by=/etc/apt/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main"
## Install packages
- name: apt update
ansible.builtin.apt:
update_cache: yes
- name: Install Kubelet Kubeadm Kubectl
ansible.builtin.apt:
pkg:
- kubelet
- kubeadm
- kubectl
- name: Hold kubeadm
ansible.builtin.dpkg_selections:
name: kubeadm
selection: hold
- name: Hold kubelet
ansible.builtin.dpkg_selections:
name: kubelet
selection: hold
- name: Hold kubectl
ansible.builtin.dpkg_selections:
name: kubectl
selection: hold
## Containerd
- name: Install Container Runtime
ansible.builtin.apt:
pkg:
- containerd.io
- containernetworking-plugins
- name: Containerd set default config
ansible.builtin.shell: containerd config default | tee /etc/containerd/config.toml
become: yes
- name: SystemdCgroup = true
lineinfile:
dest: /etc/containerd/config.toml
regexp: '^\s*SystemdCgroup = false$'
line: ' SystemdCgroup = true'
backrefs: yes
# Iptables
## Set files
- name: Iptables thingies (not touching specific firewall rules.)
copy:
dest: "/etc/sysctl.d/k8s.conf"
content: |
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward = 1
- name: Iptables thingies
copy:
dest: "/etc/modules-load.d/k8s.conf"
content: |
overlay
br_netfilter
## Modprobe
- name: Add the overlay module
community.general.modprobe:
name: overlay
state: present
- name: Add the br_netfilter module
community.general.modprobe:
name: br_netfilter
state: present
- name: Apply changes (might need to use sysctl module with the reload flag, will try eventually)
ansible.builtin.command: "sysctl --system"
## Systemctl
### Enable
- name: Enable kubelet
ansible.builtin.systemd:
name: kubelet
enabled: true
- name: Enable containerd
ansible.builtin.systemd:
name: containerd
enabled: true
### Restart
- name: Enable kubelet
ansible.builtin.systemd:
name: kubelet
state: restarted
- name: Restart containerd
ansible.builtin.systemd:
name: containerd
state: restarted
## Set /etc/hosts
### This could be better but who cares
- name: Set /etc/hosts file content (template/base)
copy:
dest: "/etc/hosts"
content: |
127.0.0.1 localhost
127.0.1.1 {{ set_hostname }}
::1 localhost {{ set_hostname }} ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters

8
ksetup/tasks_slave.yaml Executable file
View File

@ -0,0 +1,8 @@
- name: Populate {{ _kubeadm_join_command }}
ansible.builtin.set_fact: _kubeadm_join_command="{{ lookup('file', kubeadm_join_path ) }}"
- debug: var=kubeadm_join_command
- name: Join kubeadm
ansible.builtin.command: "{{ _kubeadm_join_command }}"
become: yes

2
main_issues.md
View File

@ -14,7 +14,7 @@ You can check the version by running `containerd --version`
## CNI plugin initializing
Wait, if after a while still this way, confirm that you actually deployed the network plugin.
Wait, if after a while it still this way, confirm that you actually deployed the (right) network plugin.
```shell
kubectl create -f https://raw.githubusercontent.com/projectcalico/calico/v3.25.0/manifests/calico.yaml