The various scripts in the ./scripts directory currently lack any comments at the top of the file explaining briefly what they do , their arguments etc. There is documentation for them in the README file but I think it would be good to have at least something brief in the script files themselves.
ls -la /home/david/projects/stirlingbridge/machine-provisioning/scripts/ && wc -l /home/david/projects/stirlingbridge/machine-provisioning/scripts/*
total 60
drwxr-xr-x 2 david david 4096 Jul 29 11:06 .
drwxr-xr-x 5 david david 4096 Jul 29 10:38 ..
-rwxr-xr-x 1 david david 2341 Jul 29 11:06 combine.sh
-rwxr-xr-x 1 david david 1076 Jul 29 10:38 docker.sh
-rwxr-xr-x 1 david david 125 Jul 29 10:38 error.sh
-rwxr-xr-x 1 david david 235 Jul 29 10:38 fqdn.sh
-rwxr-xr-x 1 david david 1685 Jul 29 10:38 health.sh
-rwxr-xr-x 1 david david 10691 Jul 29 10:38 k3s-node.sh
-rwxr-xr-x 1 david david 348 Jul 29 10:38 packages.sh
-rwxr-xr-x 1 david david 1060 Jul 29 10:38 podman.sh
-rwxr-xr-x 1 david david 2009 Jul 29 10:38 stack.sh
-rwxr-xr-x 1 david david 7119 Jul 29 10:38 web-shell.sh
98 /home/david/projects/stirlingbridge/machine-provisioning/scripts/combine.sh
33 /home/david/projects/stirlingbridge/machine-provisioning/scripts/docker.sh
7 /home/david/projects/stirlingbridge/machine-provisioning/scripts/error.sh
12 /home/david/projects/stirlingbridge/machine-provisioning/scripts/fqdn.sh
72 /home/david/projects/stirlingbridge/machine-provisioning/scripts/health.sh
346 /home/david/projects/stirlingbridge/machine-provisioning/scripts/k3s-node.sh
14 /home/david/projects/stirlingbridge/machine-provisioning/scripts/packages.sh
37 /home/david/projects/stirlingbridge/machine-provisioning/scripts/podman.sh
90 /home/david/projects/stirlingbridge/machine-provisioning/scripts/stack.sh
224 /home/david/projects/stirlingbridge/machine-provisioning/scripts/web-shell.sh
933 totalREADME.md
examples
scripts
# Provisioning scripts for the machine utility
These provisioning scripts are designed to be used in conjunction with any Linux machine provisioning
tool that can execute a script after first boot (typically via the `cloud-init` mechanism), for example
the [machine](https://github.com/stirlingbridge/machine) utility.
## Scripts
### combine.sh
Supports the execution of several other scripts together (useful because machine provisioning only allows one script to be executed).
### docker.sh
Installs Docker and performs associated system configuration.
### k3s-node.sh
Installs a single-node k8s cluster using k3s.
### podman.sh
Installs podman (only install one of: Docker and podman).
### stack.sh
Installs the [stack](https://github.com/bozemanpass/stack) application deployment utility.
### web-shell.sh
Installs browser-based remote shell access using ttyd, Caddy (with Let's Encrypt TLS), and JWT authentication with asymmetric keys. Supports interactive terminal sessions and programmatic command execution from browser JavaScript. See [examples/web-shell/](examples/web-shell/) for a complete example app and detailed documentation.
## Example
Scripts can be used individually, or together to provision more complex machine configurations specifying arguments as shown in the following `~/.machine/config.yaml` example. It provisions a machine that has the `build-essential` package installed, then podman, the stack utility and finally a single node k8s cluster, with appropriate configuration for hosting applications with TLS:
```yaml
machines:
k8s-stack-host:
new-user-name: bpi
script-dir: /opt/bpi
script-url: https://raw.githubusercontent.com/stirlingbridge/machine-provisioning/refs/heads/main/scripts/combine.sh
script-path: /opt/bpi/combine.sh
script-args: >-
--script-url packages.sh --script-args "build-essential"
--script-url podman.sh
--script-url stack.sh
--script-url k3s-node.sh
--script-args "-y --letsencrypt-email user@example.com --do-dns-access-token ZZZZ --image-registry registry.digitalocean.com --image-registry-username user@example.com --image-registry-password YYYY"
```cd /home/david/projects/stirlingbridge/machine-provisioning/scripts && for f in combine.sh docker.sh error.sh fqdn.sh health.sh packages.sh podman.sh stack.sh; do echo "=========== $f"; cat -n $f; done
=========== combine.sh
1 #!/usr/bin/env bash
2 if [[ -n "$MACHINE_SCRIPT_DEBUG" ]]; then
3 set -x
4 fi
5
6 DEFAULT_SCRIPT_URL_PREFIX="$(dirname ${MACHINE_SCRIPT_URL})"
7 if [[ -z "$DEFAULT_SCRIPT_URL_PREFIX" ]]; then
8 DEFAULT_SCRIPT_URL_PREFIX="https://raw.githubusercontent.com/stirlingbridge/machine-provisioning/refs/heads/main/scripts"
9 fi
10
11 export DEBIAN_FRONTEND=noninteractive
12 export NEEDRESTART_MODE=a
13
14 echo "$0 called with $*"
15
16 set -eo pipefail ## https://vaneyckt.io/posts/safer_bash_scripts_with_set_euxo_pipefail/
17
18 SCRIPTS=()
19 declare -A ARGS
20
21 while (( "$#" )); do
22 case $1 in
23 --script-url)
24 shift&&SCRIPTS+=("$1")||die
25 ;;
26 --script-args)
27 shift&&ARGS[$(( ${#SCRIPTS[@]} ))]="$1"||die
28 ;;
29 *)
30 echo "Unrecognized argument: $1"
31 ;;
32 esac
33 shift
34 done
35
36 function maybe_install {
37 local todo=""
38 while (( "$#" )); do
39 local exists=false
40 which $1 >/dev/null && exists=true || exists=false
41 if [[ "true" != "$exists" ]]; then
42 todo="$todo $1"
43 fi
44 shift
45 done
46 if [[ ! -z "$todo" ]]; then
47 echo "**************************************************************************************"
48 echo "Installing required packages"
49 sudo apt -y update
50 sudo --preserve-env=DEBIAN_FRONTEND,NEEDRESTART_MODE apt -y install $todo
51 fi
52 }
53
54 maybe_install wget
55
56 step=0
57 rc=0
58
59 for script in "${SCRIPTS[@]}"; do
60 step=$((step + 1))
61 echo "**************************************************************************************"
62 echo "$script BEGIN"
63 cmd=""
64
65 # Local path
66 if [[ $script == /* ]]; then
67 cmd="$script"
68 else
69 script_url="$script"
70 if [[ $script_url != http* ]]; then
71 script_url="${DEFAULT_SCRIPT_URL_PREFIX}/${script}"
72 fi
73 echo "Downloading $script_url to /tmp/combine.script.$step ..."
74 wget -q -O /tmp/combine.step.$step "$script_url"
75 chmod 700 /tmp/combine.step.$step
76 cmd=/tmp/combine.step.$step
77 fi
78
79 echo "Running: $cmd ${ARGS["$step"]}"
80 $cmd ${ARGS["$step"]} && rc=$? || rc=$?
81 if [[ $rc -ne 0 ]]; then
82 echo "$script FAILED rc=$rc"
83 fi
84 echo "$script END"
85 echo "#####################################################################################"
86
87 if [[ $rc != 0 ]]; then
88 break
89 fi
90 done
91
92 rm -f /tmp/combine.step.*
93
94 if [[ $rc -eq 0 ]]; then
95 echo "All scripts completed successfully."
96 fi
97
98 exit $rc
=========== docker.sh
1 #!/usr/bin/env bash
2 if [[ -n "$MACHINE_SCRIPT_DEBUG" ]]; then
3 set -x
4 fi
5
6 export DEBIAN_FRONTEND=noninteractive
7 export NEEDRESTART_MODE=a
8
9 APT_INSTALL="sudo --preserve-env=DEBIAN_FRONTEND,NEEDRESTART_MODE apt -y install"
10
11 set -eo pipefail ## https://vaneyckt.io/posts/safer_bash_scripts_with_set_euxo_pipefail/
12
13 which docker >/dev/null && rc=$? || rc=$?
14 if [[ $rc -eq 0 ]]; then
15 echo "docker already installed."
16 if [[ "$1" != "-f" ]]; then
17 exit 0
18 fi
19 fi
20
21 sudo apt update
22 $APT_INSTALL apt-transport-https ca-certificates curl software-properties-common curl
23
24 if [[ ! -f "/usr/share/keyrings/docker-archive-keyring.gpg" ]]; then
25 curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
26 fi
27
28 echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
29
30 sudo apt update
31 $APT_INSTALL docker-ce
32
33 sudo usermod -aG docker ${USER}
=========== error.sh
1 #!/usr/bin/env bash
2 if [[ -n "$MACHINE_SCRIPT_DEBUG" ]]; then
3 set -x
4 fi
5
6 echo "THIS IS AN INTENTIONAL ERROR" 1>&2
7 exit 1
=========== fqdn.sh
1 #!/bin/bash
2
3 if [[ ! -d "/etc/machine" ]]; then
4 sudo mkdir -p /etc/machine
5 fi
6
7 if [[ -n "$MACHINE_FQDN" ]]; then
8 echo "$MACHINE_FQDN" > /tmp/fqdn.??
9 sudo mv /tmp/fqdn.?? /etc/machine/fqdn
10 fi
11
12 sudo chown -R root:root /etc/machine
=========== health.sh
1 #!/usr/bin/env bash
2 if [[ -n "$MACHINE_SCRIPT_DEBUG" ]]; then
3 set -x
4 fi
5
6 export DEBIAN_FRONTEND=noninteractive
7 export NEEDRESTART_MODE=a
8
9 echo "$0 called with $*"
10
11 set -eo pipefail
12
13 PORT=4242
14
15 while (( "$#" )); do
16 case $1 in
17 --port)
18 shift&&PORT="$1"||die
19 ;;
20 *)
21 echo "Unrecognized argument: $1"
22 ;;
23 esac
24 shift
25 done
26
27 function maybe_install {
28 local todo=""
29 while (( "$#" )); do
30 local exists=false
31 which $1 >/dev/null && exists=true || exists=false
32 if [[ "true" != "$exists" ]]; then
33 todo="$todo $1"
34 fi
35 shift
36 done
37 if [[ ! -z "$todo" ]]; then
38 echo "**************************************************************************************"
39 echo "Installing required packages"
40 sudo apt -y update
41 sudo --preserve-env=DEBIAN_FRONTEND,NEEDRESTART_MODE apt -y install $todo
42 fi
43 }
44
45 maybe_install python3
46
47 sudo mkdir -p /var/opt/machine/health/cgi-bin
48
49 cat >/tmp/machine.health.$$ <<EOF
50 #!/bin/bash
51 CLOUD_INIT_LOG=/var/log/cloud-init-output.log
52 STATUS="INITIALIZING"
53
54 sudo grep 'Failed to run module scripts_user' \$CLOUD_INIT_LOG >/dev/null
55 if [ \$? -eq 0 ]; then
56 STATUS="ERROR"
57 else
58 sudo grep '^Cloud-init v' \$CLOUD_INIT_LOG | grep 'finished at' | grep 'Up.*seconds' >/dev/null
59 if [ \$? -eq 0 ]; then
60 STATUS="UP"
61 fi
62 fi
63
64 echo "Content-Type: application/json"
65 echo ""
66 echo "{ \"status\": \"\$STATUS\" }"
67 EOF
68 sudo mv /tmp/machine.health.$$ /var/opt/machine/health/cgi-bin/cloud-init-status
69 sudo chmod -R a+rX /var/opt/machine
70 sudo chmod -R a+x /var/opt/machine/health/cgi-bin/cloud-init-status
71
72 nohup python3 -m http.server --cgi --directory /var/opt/machine/health $PORT &
=========== packages.sh
1 #!/usr/bin/env bash
2 if [[ -n "$MACHINE_SCRIPT_DEBUG" ]]; then
3 set -x
4 fi
5
6 export DEBIAN_FRONTEND=noninteractive
7 export NEEDRESTART_MODE=a
8
9 APT_INSTALL="sudo --preserve-env=DEBIAN_FRONTEND,NEEDRESTART_MODE apt -y install"
10
11 set -eo pipefail ## https://vaneyckt.io/posts/safer_bash_scripts_with_set_euxo_pipefail/
12
13 sudo apt update
14 $APT_INSTALL $*
=========== podman.sh
1 #!/usr/bin/env bash
2 if [[ -n "$MACHINE_SCRIPT_DEBUG" ]]; then
3 set -x
4 fi
5
6 export DEBIAN_FRONTEND=noninteractive
7 export NEEDRESTART_MODE=a
8
9 APT_INSTALL="sudo --preserve-env=DEBIAN_FRONTEND,NEEDRESTART_MODE apt -y install"
10
11 set -eo pipefail ## https://vaneyckt.io/posts/safer_bash_scripts_with_set_euxo_pipefail/
12
13 which podman >/dev/null && rc=$? || rc=$?
14 if [[ $rc -eq 0 ]]; then
15 echo "podman already installed."
16 if [[ "$1" != "-f" ]]; then
17 exit 0
18 fi
19 fi
20
21 which docker >/dev/null && HAS_DOCKER=true || HAS_DOCKER=false
22
23 sudo apt update
24 $APT_INSTALL apt-transport-https ca-certificates curl software-properties-common
25 $APT_INSTALL podman
26
27 if [[ "$HAS_DOCKER" == "false" ]]; then
28 $APT_INSTALL podman-docker
29 sudo touch /etc/containers/nodocker
30 fi
31
32 # enable standard Docker container registry
33 grep '^unqualified-search-registries' /etc/containers/registries.conf >/dev/null && HAS_REG=true || HAS_REG=false
34 if [[ "$HAS_REG" != "true" ]]; then
35 echo 'unqualified-search-registries = ["docker.io"]' | sudo tee -a /etc/containers/registries.conf
36 fi
37
=========== stack.sh
1 #!/usr/bin/env bash
2 if [[ -n "$MACHINE_SCRIPT_DEBUG" ]]; then
3 set -x
4 fi
5
6 export DEBIAN_FRONTEND=noninteractive
7 export NEEDRESTART_MODE=a
8
9 FORCE="false"
10 VER=""
11 BUILD="false"
12
13 while getopts "bfv:" arg; do
14 case $arg in
15 f)
16 FORCE=true
17 ;;
18 b)
19 BUILD=true
20 ;;
21 v)
22 VER=$OPTARG
23 ;;
24 esac
25 done
26
27 set -euo pipefail ## https://vaneyckt.io/posts/safer_bash_scripts_with_set_euxo_pipefail/
28
29 if [[ -x "/usr/local/bin/stack" ]]; then
30 echo "/usr/local/bin/stack already exists"
31 if [[ "$FORCE" != "true" ]]; then
32 exit 0
33 fi
34 fi
35
36 OS_VER=`lsb_release -r | awk '{ print $2 }'`
37
38 if [[ "$OS_VER" != "24.04" ]] && [[ "$OS_VER" != "22.04" ]]; then
39 echo "Build required on $OS_VER"
40 BUILD="true"
41 fi
42
43 function maybe_install {
44 local todo=""
45 while (( "$#" )); do
46 local exists=false
47 which $1 >/dev/null && exists=true || exists=false
48 if [[ "true" != "$exists" ]]; then
49 todo="$todo $1"
50 fi
51 shift
52 done
53 if [[ ! -z "$todo" ]]; then
54 echo "**************************************************************************************"
55 echo "Installing required packages"
56 sudo apt -y update
57 sudo --preserve-env=DEBIAN_FRONTEND,NEEDRESTART_MODE apt -y install $todo
58 fi
59 }
60
61
62 if [[ "true" == "$BUILD" ]]; then
63 maybe_install git python3-venv
64 TMPD=`mktemp -d`
65 cd $TMPD
66
67 git clone https://github.com/bozemanpass/stack.git
68 cd stack
69 if [[ -n "$VER" ]] && [[ "$VER" != "latest" ]]; then
70 git checkout $VER
71 fi
72
73 scripts/developer-mode-setup.sh
74 . venv/bin/activate
75 scripts/build_shiv_package.sh
76
77 sudo mv package/stack-* /usr/local/bin/stack
78
79 rm -rf $TMPD
80 else
81 maybe_install wget
82 if [[ -z "$VER" ]] || [[ "$VER" == "latest" ]]; then
83 wget -O /tmp/stack.$$ https://github.com/bozemanpass/stack/releases/latest/download/stack
84 else
85 wget -O /tmp/stack.$$ https://github.com/bozemanpass/stack/releases/download/${VER}/stack
86 fi
87 sudo mv /tmp/stack.$$ /usr/local/bin/stack
88 fi
89
90 sudo chmod a+x /usr/local/bin/stack 1 #!/usr/bin/env bash
2 if [[ -n "$MACHINE_SCRIPT_DEBUG" ]]; then
3 set -x
4 fi
5
6 export DEBIAN_FRONTEND=noninteractive
7 export NEEDRESTART_MODE=a
8
9 install_dir=~/bin
10
11 DO_TOKEN=""
12 IMAGE_REGISTRY=""
13 IMAGE_REGISTRY_USERNAME=""
14 IMAGE_REGISTRY_PASSWORD=""
15 LETSENCRYPT_EMAIL=""
16 NEEDS_WARN=true
17
18 while (( "$#" )); do
19 case $1 in
20 -y)
21 NEEDS_WARN=false
22 ;;
23 --letsencrypt-email)
24 shift&&LETSENCRYPT_EMAIL="$1"||die
25 ;;
26 --image-registry)
27 shift&&IMAGE_REGISTRY="$1"||die
28 ;;
29 --image-registry-username)
30 shift&&IMAGE_REGISTRY_USERNAME="$1"||die
31 ;;
32 --image-registry-password)
33 shift&&IMAGE_REGISTRY_PASSWORD="$1"||die
34 ;;
35 --do-dns-access-token)
36 shift&&DO_TOKEN="$(echo -n "$1" | base64 -w0)"||die
37 ;;
38 *)
39 echo "Unrecognized argument: $1" 1>&2
40 ;;
41 esac
42 shift
43 done
44
45 function retry {
46 local try=0
47 local max=5
48 local delay=10
49 while [ $try -lt $max ]; do
50 try=$((try + 1))
51 echo "Try $try of $* ..."
52 $* && RC=$? || RC=$?
53 if [ $RC -eq 0 ]; then
54 return 0
55 else
56 sleep $delay
57 fi
58 done
59 return 1
60 }
61
62 # Skip the package install stuff if so directed
63 if ! [[ -n "$BPI_INSTALL_SKIP_PACKAGES" ]]; then
64
65 # First display a reasonable warning to the user unless run with -y
66 if [[ "$NEEDS_WARN" == "true" ]]; then
67 echo "**************************************************************************************"
68 echo "This script requires sudo privilege. It installs utilities"
69 echo "into: ${install_dir}. It also *removes* any existing docker installed on"
70 echo "this machine and then installs the latest docker release as well as other"
71 echo "required packages."
72 echo "Only proceed if you are sure you want to make those changes to this machine."
73 echo "**************************************************************************************"
74 read -p "Are you sure you want to proceed? " -n 1 -r
75 echo
76 if [[ ! $REPLY =~ ^[Yy]$ ]]; then
77 exit 1
78 fi
79 fi
80
81 # Determine if we are on Debian or Ubuntu
82 linux_distro=$(lsb_release -a 2>/dev/null | grep "^Distributor ID:" | cut -f 2)
83 # Some systems don't have lsb_release installed (e.g. ChromeOS) and so we try to
84 # use /etc/os-release instead
85 if [[ -z "$linux_distro" ]]; then
86 if [[ -f "/etc/os-release" ]]; then
87 distro_name_string=$(grep "^NAME=" /etc/os-release | cut -d '=' -f 2)
88 if [[ $distro_name_string =~ Debian ]]; then
89 linux_distro="Debian"
90 elif [[ $distro_name_string =~ Ubuntu ]]; then
91 linux_distro="Ubuntu"
92 fi
93 else
94 echo "Failed to identify distro: /etc/os-release doesn't exist"
95 exit 1
96 fi
97 fi
98 case $linux_distro in
99 Debian)
100 echo "Installing k3s for Debian"
101 ;;
102 Ubuntu)
103 echo "Installing k3s for Ubuntu"
104 ;;
105 *)
106 echo "ERROR: Detected unknown distribution $linux_distro, can't install k3s"
107 exit 1
108 ;;
109 esac
110
111 # dismiss the popups
112 export DEBIAN_FRONTEND=noninteractive
113
114 ## Even though we're installing k3s, which doesn't depend on docker, we still un-install any distro-origin docker components first
115 ## https://docs.docker.com/engine/install/ubuntu/
116 ## https://docs.docker.com/engine/install/debian/
117 ## https://superuser.com/questions/518859/ignore-packages-that-are-not-currently-installed-when-using-apt-get-remove1
118 packages_to_remove="docker docker-engine docker.io containerd runc docker-compose docker-doc podman-docker"
119 installed_packages_to_remove=""
120 for package_to_remove in $(echo $packages_to_remove); do 1 #!/usr/bin/env bash
2 if [[ -n "$MACHINE_SCRIPT_DEBUG" ]]; then
3 set -x
4 fi
5
6 export DEBIAN_FRONTEND=noninteractive
7 export NEEDRESTART_MODE=a
8
9 APT_INSTALL="sudo --preserve-env=DEBIAN_FRONTEND,NEEDRESTART_MODE apt -y install"
10
11 set -eo pipefail ## https://vaneyckt.io/posts/safer_bash_scripts_with_set_euxo_pipefail/
12
13 echo "***********************************************************************"
14 echo "* web-shell.sh"
15 echo "***********************************************************************"
16 echo "$0 called with $*"
17
18 # Defaults
19 TTYD_PORT=7681
20 VERIFY_PORT=9222
21 SHELL_USER="webshell"
22 FQDN=""
23 JWT_PUBLIC_KEY_FILE=""
24 FORCE=false
25
26 while (( "$#" )); do
27 case $1 in
28 --fqdn)
29 shift&&FQDN="$1"||{ echo "Missing --fqdn value"; exit 1; }
30 ;;
31 --jwt-public-key-file)
32 shift&&JWT_PUBLIC_KEY_FILE="$1"||{ echo "Missing --jwt-public-key-file value"; exit 1; }
33 ;;
34 --shell-user)
35 shift&&SHELL_USER="$1"||{ echo "Missing --shell-user value"; exit 1; }
36 ;;
37 -f)
38 FORCE=true
39 ;;
40 *)
41 echo "Unrecognized argument: $1"
42 ;;
43 esac
44 shift
45 done
46
47 # Validate required arguments
48 if [[ -z "$FQDN" ]]; then
49 echo "Error: --fqdn is required (for Let's Encrypt TLS)"
50 exit 1
51 fi
52 if [[ -z "$JWT_PUBLIC_KEY_FILE" ]]; then
53 echo "Error: --jwt-public-key-file is required (PEM-encoded public key for JWT verification)"
54 exit 1
55 fi
56 if [[ ! -f "$JWT_PUBLIC_KEY_FILE" ]]; then
57 echo "Error: JWT public key file not found: $JWT_PUBLIC_KEY_FILE"
58 exit 1
59 fi
60
61 # Check if already installed
62 if [[ "$FORCE" != "true" ]]; then
63 which ttyd >/dev/null 2>&1 && rc=$? || rc=$?
64 if [[ $rc -eq 0 ]]; then
65 echo "ttyd already installed. Use -f to force reinstall."
66 exit 0
67 fi
68 fi
69
70 echo "***********************************************************************"
71 echo "* Installing packages"
72 echo "***********************************************************************"
73
74 sudo apt update
75 $APT_INSTALL ca-certificates curl python3-jwt python3-cryptography
76
77 # Install Caddy from official repository
78 if ! which caddy >/dev/null 2>&1; then
79 echo "***********************************************************************"
80 echo "* Installing Caddy"
81 echo "***********************************************************************"
82 $APT_INSTALL debian-keyring debian-archive-keyring apt-transport-https
83 curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' | sudo gpg --dearmor -o /usr/share/keyrings/caddy-stable-archive-keyring.gpg
84 curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' | sudo tee /etc/apt/sources.list.d/caddy-stable.list
85 sudo apt update
86 $APT_INSTALL caddy
87 fi
88
89 # Install ttyd binary
90 echo "***********************************************************************"grep -n 'echo "\*\|^# \|helm\|kubectl apply\|ClusterIssuer\|traefik\|nginx' k3s-node.sh | head -60
62:# Skip the package install stuff if so directed 65:# First display a reasonable warning to the user unless run with -y 67: echo "**************************************************************************************" 73: echo "**************************************************************************************" 81:# Determine if we are on Debian or Ubuntu 83:# Some systems don't have lsb_release installed (e.g. ChromeOS) and so we try to 84:# use /etc/os-release instead 111:# dismiss the popups 127:# Enable stop on error now, since we needed it off for the code above 131: echo "**************************************************************************************" 136:echo "**************************************************************************************" 141:echo "**************************************************************************************" 147:export INSTALL_K3S_EXEC="--disable=traefik" 151:echo "**************************************************************************************" 152:echo "Installing nginx ingress" 153:sudo kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.12.0/deploy/static/provider/baremetal/deploy.yaml 159: name: ingress-nginx-controller-loadbalancer 160: namespace: ingress-nginx 164: app.kubernetes.io/instance: ingress-nginx 165: app.kubernetes.io/name: ingress-nginx 178:sudo kubectl apply -f /tmp/lb.yml.$$ 181:sudo kubectl annotate ingressclass nginx ingressclass.kubernetes.io/is-default-class=true 183:echo "Installed nginx ingress" 185:echo "**************************************************************************************" 188:sudo kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.16.2/cert-manager.yaml 210:kind: ClusterIssuer 227: class: nginx 232:kind: ClusterIssuer 249: class: nginx 265:kind: ClusterIssuer 286: echo "Adding letsencrypt-prod ClusterIssuer..." 287: retry sudo kubectl apply -f $HOME/letsencrypt-prod.yml 288: echo "Adding letsencrypt-staging ClusterIssuer..." 289: retry sudo kubectl apply -f $HOME/letsencrypt-stage.yml 292: retry sudo kubectl apply -f $HOME/digitalocean-dns.yml 293: echo "Adding letsencrypt-prod-dns ClusterIssuer..." 294: retry sudo kubectl apply -f $HOME/letsencrypt-prod-dns01.yml 296: echo "No DigitalOcean access token specified, so a DNS-based ClusterIssuer's could not be created. Template files created at $HOME/digitalocean-dns.yml and $HOME/letsencrypt-prod-dns01.yml" 299: echo "No e-mail specified, so ClusterIssuer's could not be created. Template files created at $HOME/letsencrypt-prod.yml and $HOME/letsencrypt-stage.yml" 304:echo "**************************************************************************************" 337:# End of long if block: Skip the package install stuff if so directed 340:# Message the user to check docker is working for them
grep -n 'echo "\* \|systemctl enable\|cat >\|verify' web-shell.sh | head -40; echo ---; grep -rn 'MACHINE_' . | grep -v DEBUG
14:echo "* web-shell.sh"
71:echo "* Installing packages"
80: echo "* Installing Caddy"
91:echo "* Installing ttyd"
108: echo "* Creating user: $SHELL_USER"
115:echo "* Writing configuration"
122:sudo tee /etc/web-shell/jwt-verify.py > /dev/null << 'PYEOF'
158:sudo chmod 644 /etc/web-shell/jwt-verify.py
165: uri /verify
189:sudo tee /etc/systemd/system/jwt-verify.service > /dev/null << SVCEOF
197:ExecStart=/usr/bin/python3 /etc/web-shell/jwt-verify.py /etc/web-shell/public.pem ${VERIFY_PORT}
207:echo "* Starting services"
210:sudo systemctl enable --now jwt-verify.service
211:sudo systemctl enable --now ttyd.service
215:echo "* web-shell.sh complete"
220:echo " JWT verify: 127.0.0.1:${VERIFY_PORT} (localhost only)"
---
combine.sh:6:DEFAULT_SCRIPT_URL_PREFIX="$(dirname ${MACHINE_SCRIPT_URL})"
fqdn.sh:7:if [[ -n "$MACHINE_FQDN" ]]; then
fqdn.sh:8: echo "$MACHINE_FQDN" > /tmp/fqdn.??#!/usr/bin/env bash if [[
#!/usr/bin/env bash # # combine.sh -- run several provisioning scripts in sequence. # # Useful because most machine provisioning mechanisms only allow a single # script to be executed after first boot. # # Usage: # combine.sh --script-url <script> [--script-args "<args>"] [--script-url ... ] # # --script-url Script to run. Either an absolute local path, a full URL, or # a bare file name resolved relative to the directory part of # $MACHINE_SCRIPT_URL (falling back to this repo's scripts # directory on GitHub). Repeat for each script to run. # --script-args Arguments for the immediately preceding --script-url. # # Scripts run in the order given, and execution stops at the first failure. # Exits with the exit status of the last script run. # if [[
#!/usr/bin/env bash if [[
#!/usr/bin/env bash # # docker.sh -- install Docker CE from the official Docker apt repository and # add the invoking user to the docker group. # # Usage: # docker.sh [-f] # # -f Reinstall even if docker is already present (otherwise this script # does nothing when docker is already installed). # # Install only one of docker.sh and podman.sh on a given machine. # if [[
#!/usr/bin/env bash if [[
#!/usr/bin/env bash # # error.sh -- always fails. Takes no arguments. # # A test script used to check that provisioning failures are detected and # reported correctly (e.g. that combine.sh stops on error). # if [[
#!/usr/bin/env bash if [[
#!/usr/bin/env bash # # packages.sh -- install arbitrary distro packages with apt. # # Usage: # packages.sh <package> [<package> ...] # # All arguments are passed through to "apt install", e.g. # packages.sh build-essential jq # if [[
#!/usr/bin/env bash if [[
#!/usr/bin/env bash # # podman.sh -- install podman, plus podman-docker (the docker CLI shim) when no # real docker installation is present, and enable docker.io as an unqualified # search registry. # # Usage: # podman.sh [-f] # # -f Reinstall even if podman is already present (otherwise this script # does nothing when podman is already installed). # # Install only one of docker.sh and podman.sh on a given machine. # if [[
The file /home/david/projects/stirlingbridge/machine-provisioning/scripts/combine.sh has been updated successfully. (file state is current in your context — no need to Read it back)
The file /home/david/projects/stirlingbridge/machine-provisioning/scripts/docker.sh has been updated successfully. (file state is current in your context — no need to Read it back)
The file /home/david/projects/stirlingbridge/machine-provisioning/scripts/error.sh has been updated successfully. (file state is current in your context — no need to Read it back)
The file /home/david/projects/stirlingbridge/machine-provisioning/scripts/packages.sh has been updated successfully. (file state is current in your context — no need to Read it back)
The file /home/david/projects/stirlingbridge/machine-provisioning/scripts/podman.sh has been updated successfully. (file state is current in your context — no need to Read it back)
#!/bin/bash
#!/bin/bash # # fqdn.sh -- record the machine's fully qualified domain name on disk. # # Takes no arguments. If $MACHINE_FQDN is set in the environment (normally by # the provisioning tool), its value is written to /etc/machine/fqdn, creating # /etc/machine if necessary. Other scripts and applications can then read the # machine's FQDN from that file. #
#!/usr/bin/env bash if [[
#!/usr/bin/env bash
#
# health.sh -- serve a simple provisioning-status endpoint over HTTP.
#
# Starts a background python3 CGI server rooted at /var/opt/machine/health which
# serves /cgi-bin/cloud-init-status. That endpoint returns JSON of the form
# { "status": "INITIALIZING" | "UP" | "ERROR" }, derived from the cloud-init
# output log, so a remote caller can tell when provisioning has finished.
#
# Usage:
# health.sh [--port <port>]
#
# --port Port to listen on (default 4242).
#
if [[#!/usr/bin/env bash if [[
#!/usr/bin/env bash
#
# stack.sh -- install the stack application deployment utility
# (https://github.com/bozemanpass/stack) as /usr/local/bin/stack.
#
# By default a pre-built release binary is downloaded. On distro versions with
# no pre-built binary (anything other than Ubuntu 22.04 and 24.04), or when -b
# is given, stack is built from source instead.
#
# Usage:
# stack.sh [-f] [-b] [-v <version>]
#
# -f Reinstall even if /usr/local/bin/stack already exists.
# -b Build from source rather than downloading a release binary.
# -v <version> Release tag / git ref to install ("latest" is the default).
#
if [[The file /home/david/projects/stirlingbridge/machine-provisioning/scripts/fqdn.sh has been updated successfully. (file state is current in your context — no need to Read it back)
#!/usr/bin/env bash if [[
#!/usr/bin/env bash # # k3s-node.sh -- install a single-node Kubernetes cluster using k3s, ready for # hosting applications with TLS. # # Installs k3s (with the bundled traefik ingress disabled), the nginx ingress # controller, and cert-manager, then creates Let's Encrypt ClusterIssuers. # Any distro-packaged docker components are removed first. Debian and Ubuntu # only. # # Usage: # k3s-node.sh [-y] [--letsencrypt-email <email>] # [--do-dns-access-token <token>] # [--image-registry <host>] [--image-registry-username <user>] # [--image-registry-password <password>] # # -y Don't prompt for confirmation before making # changes (required for unattended provisioning). # --letsencrypt-email Contact address for Let's Encrypt. Without it # the ClusterIssuers are not created, only written # to $HOME as template files to complete by hand. # --do-dns-access-token DigitalOcean API token, enabling an additional # DNS-01 ClusterIssuer (needed for wildcard certs). # --image-registry Container image registry to authenticate against, # --image-registry-username with these credentials, so the cluster can pull # --image-registry-password private images. # # Environment: # BPI_INSTALL_SKIP_PACKAGES If set, skip the package install phase. # if [[
The file /home/david/projects/stirlingbridge/machine-provisioning/scripts/health.sh has been updated successfully. (file state is current in your context — no need to Read it back)
The file /home/david/projects/stirlingbridge/machine-provisioning/scripts/stack.sh has been updated successfully. (file state is current in your context — no need to Read it back)
The file /home/david/projects/stirlingbridge/machine-provisioning/scripts/k3s-node.sh has been updated successfully. (file state is current in your context — no need to Read it back)
#!/usr/bin/env bash if [[
#!/usr/bin/env bash # # web-shell.sh -- install browser-based remote shell access. # # Installs ttyd (the terminal server), Caddy as a TLS-terminating reverse proxy # using Let's Encrypt, and a small local JWT verification service that Caddy # consults via forward_auth. Callers authenticate with a JWT signed by the # private key matching the public key supplied here, which supports both # interactive terminal sessions and programmatic command execution from browser # JavaScript. See examples/web-shell/ for a worked example. # # Usage: # web-shell.sh --fqdn <hostname> --jwt-public-key-file <path> # [--shell-user <user>] [-f] # # --fqdn Public hostname of this machine, used for the # Let's Encrypt certificate. Required. # --jwt-public-key-file PEM-encoded public key used to verify JWTs. # Required. # --shell-user Unix account the shell runs as, created if it does # not exist (default "webshell"). # -f Reinstall even if ttyd is already present. # if [[
The file /home/david/projects/stirlingbridge/machine-provisioning/scripts/web-shell.sh has been updated successfully. (file state is current in your context — no need to Read it back)
ok combine.sh ok docker.sh ok error.sh ok fqdn.sh ok health.sh ok k3s-node.sh ok packages.sh ok podman.sh ok stack.sh ok web-shell.sh
Added a header comment block to all 10 scripts, placed directly after the shebang (before the existing debug/env preamble), each covering what the script does, its arguments/defaults, and any relevant environment variables. All still pass bash -n.
A few things I documented that weren't in the README:
$MACHINE_FQDN to /etc/machine/fqdn./cgi-bin/cloud-init-status returning INITIALIZING/UP/ERROR, --port default 4242.apt install.Also worth noting: combine.sh resolves bare script names against dirname $MACHINE_SCRIPT_URL, falling back to this repo's GitHub scripts directory — I described that in its header since it isn't obvious from the README example. Say the word if you'd like the README's script list extended to cover error.sh, fqdn.sh, health.sh, and packages.sh too.
{
"file_path": "/home/david/projects/stirlingbridge/machine-provisioning/README.md",
"limit": 25
}1 # Provisioning scripts for the machine utility 2 3 These provisioning scripts are designed to be used in conjunction with any Linux machine provisioning 4 tool that can execute a script after first boot (typically via the `cloud-init` mechanism), for example 5 the [machine](https://github.com/stirlingbridge/machine) utility. 6 7 ## Scripts 8 9 ### combine.sh 10 Supports the execution of several other scripts together (useful because machine provisioning only allows one script to be executed). 11 ### docker.sh 12 Installs Docker and performs associated system configuration. 13 ### k3s-node.sh 14 Installs a single-node k8s cluster using k3s. 15 ### podman.sh 16 Installs podman (only install one of: Docker and podman). 17 ### stack.sh 18 Installs the [stack](https://github.com/bozemanpass/stack) application deployment utility. 19 ### web-shell.sh 20 Installs browser-based remote shell access using ttyd, Caddy (with Let's Encrypt TLS), and JWT authentication with asymmetric keys. Supports interactive terminal sessions and programmatic command execution from browser JavaScript. See [examples/web-shell/](examples/web-shell/) for a complete example app and detailed documentation. 21 22 ## Example 23 Scripts can be used individually, or together to provision more complex machine configurations specifying arguments as shown in the following `~/.machine/config.yaml` example. It provisions a machine that has the `build-essential` package installed, then podman, the stack utility and finally a single node k8s cluster, with appropriate configuration for hosting applications with TLS: 24 ```yaml 25 machines:
### docker.sh Installs Docker and performs associated system configuration. ### k3s-node.sh Installs a single-node k8s cluster using k3s. ### podman.sh
### docker.sh
Installs Docker and performs associated system configuration.
### error.sh
Always fails. A test script for checking that provisioning failures are detected and reported correctly.
### fqdn.sh
Writes the machine's fully qualified domain name (taken from the `MACHINE_FQDN` environment variable) to `/etc/machine/fqdn`, where other scripts and applications can read it.
### health.sh
Serves a provisioning status endpoint at `/cgi-bin/cloud-init-status` (port 4242 by default, set with `--port`), returning JSON of the form `{ "status": "INITIALIZING" | "UP" | "ERROR" }` so a remote caller can tell when provisioning has finished.
### k3s-node.sh
Installs a single-node k8s cluster using k3s.
### packages.sh
Installs the distro packages named in its arguments, which are passed through to `apt install`.
### podman.shThe file /home/david/projects/stirlingbridge/machine-provisioning/README.md has been updated successfully. (file state is current in your context — no need to Read it back)
Added README entries for error.sh, fqdn.sh, health.sh, and packages.sh, keeping the list alphabetical and matching the existing one-line-per-script style. The Scripts section now covers all 10 scripts.
You're welcome! The scripts and README are both up to date — changes are still uncommitted on dboreham/add-script-comments if you want to review before committing.