forked from TrueCloudLab/certificates
Mariano Cano
GitHub
commit
0eb0c3a21b
53 changed files with 3079 additions and 0 deletions
28
.github/ISSUE_TEMPLATE/autocert_bug.md
vendored
Normal file
28
.github/ISSUE_TEMPLATE/autocert_bug.md
vendored
Normal file
|
|
@ -0,0 +1,28 @@
|
|||
---
|
||||
name: Autocert Bug
|
||||
about: Report a bug you found in autocert
|
||||
labels: area/autocert bug
|
||||
---
|
||||
|
||||
### Subject of the issue
|
||||
Describe your issue here
|
||||
|
||||
### Environment
|
||||
* Kubernetes version:
|
||||
* Cloud provider or hardware configuration:
|
||||
* OS (e.g., from /etc/os-release):
|
||||
* Kernel (e.g., `uname -a`):
|
||||
* Install tools:
|
||||
* Other:
|
||||
|
||||
### Steps to reproduce
|
||||
Tell us how to reproduce this issue
|
||||
|
||||
### Expected behaviour
|
||||
Tell us what should happen
|
||||
|
||||
### Actual behaviour
|
||||
Tell us what happens instead
|
||||
|
||||
### Additional context
|
||||
Add any other context about the problem here
|
||||
11
.github/ISSUE_TEMPLATE/autocert_enhancement.md
vendored
Normal file
11
.github/ISSUE_TEMPLATE/autocert_enhancement.md
vendored
Normal file
|
|
@ -0,0 +1,11 @@
|
|||
---
|
||||
name: Autocert Enhancement
|
||||
about: Suggest an enhancement to autocert
|
||||
labels: area/autocert enhancement
|
||||
---
|
||||
|
||||
### What would you like to be added
|
||||
|
||||
|
||||
### Why this is needed
|
||||
|
||||
176
autocert/INSTALL.md
Normal file
176
autocert/INSTALL.md
Normal file
|
|
@ -0,0 +1,176 @@
|
|||
# Installing `autocert`
|
||||
|
||||
### Prerequisites
|
||||
|
||||
To get started you'll need [`kubectl`](https://kubernetes.io/docs/tasks/tools/install-kubectl/#install-kubectl) and a cluster running kubernetes `1.9` or later with [admission webhooks](https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/#admission-webhooks) enabled:
|
||||
|
||||
```bash
|
||||
$ kubectl version --short
|
||||
Client Version: v1.13.1
|
||||
Server Version: v1.10.11
|
||||
$ kubectl api-versions | grep "admissionregistration.k8s.io/v1beta1"
|
||||
admissionregistration.k8s.io/v1beta1
|
||||
```
|
||||
|
||||
### Install
|
||||
|
||||
The easiest way to install `autocert` is to run:
|
||||
|
||||
```bash
|
||||
kubectl run autocert-init -it --rm --image smallstep/autocert-init --restart Never
|
||||
```
|
||||
|
||||
💥 installation complete.
|
||||
|
||||
> You might want to [check out what this command does](init/autocert.sh) before running it.
|
||||
|
||||
## Manual install
|
||||
|
||||
To install manually you'll need to [install step](https://github.com/smallstep/cli#installing) version `0.8.3` or later.
|
||||
|
||||
```
|
||||
$ step version
|
||||
Smallstep CLI/0.8.3 (darwin/amd64)
|
||||
Release Date: 2019-01-16 01:46 UTC
|
||||
```
|
||||
|
||||
### Create a CA
|
||||
|
||||
Set your `STEPPATH` to a working directory where we can stage our CA artifacts before we push them to kubernetes. You can delete this directory once installation is complete.
|
||||
|
||||
```
|
||||
$ export STEPPATH=$(mktemp -d /tmp/step.XXX)
|
||||
$ step path
|
||||
/tmp/step.0kE
|
||||
```
|
||||
|
||||
Run `step ca init` to generate a root certificate and CA configuration for your cluster. You'll be prompted for a password that will be used to encrypt key material.
|
||||
|
||||
```
|
||||
$ step ca init \
|
||||
--name Autocert \
|
||||
--dns "ca.step.svc.cluster.local,127.0.0.1" \
|
||||
--address ":4443" \
|
||||
--provisioner admin \
|
||||
--with-ca-url "ca.step.svc.cluster.local"
|
||||
```
|
||||
|
||||
For older versions of `step` run this command without the flags.
|
||||
|
||||
Add provisioning credentials for use by `autocert`. You'll be prompted for a password for `autocert`.
|
||||
|
||||
```
|
||||
$ step ca provisioner add autocert --create
|
||||
```
|
||||
|
||||
For older versions of `step`:
|
||||
|
||||
* Run `step ca init` and follow prompts
|
||||
* Edit `$(step path)/config/ca.json` and change base paths to `/home/step`
|
||||
* Edit `$(step path)/config/defaults.json` to change base paths to `/home/step` and remove port from CA URL
|
||||
|
||||
```
|
||||
$ sed -i "" "s|$(step path)|/home/step/.step|g" $(step path)/config/ca.json
|
||||
$ sed -i "" "s|$(step path)|/home/step/.step|g" $(step path)/config/defaults.json
|
||||
$ sed -i "" "s|ca.step.svc.cluster.local:4443|ca.step.svc.cluster.local|" $(step path)/config/defaults.json
|
||||
```
|
||||
|
||||
### Install the CA in Kubernetes
|
||||
|
||||
We'll be creating a new kubernetes namespace and setting up some RBAC rules during installation. You'll need appropriate permissions in your cluster (e.g., you may need to be cluster-admin). GKE, in particular, does not give the cluster owner these rights by default. You can give yourself cluster-admin rights on GKE by running:
|
||||
|
||||
```bash
|
||||
kubectl create clusterrolebinding cluster-admin-binding \
|
||||
--clusterrole cluster-admin \
|
||||
--user $(gcloud config get-value account)
|
||||
```
|
||||
|
||||
We'll install our CA and the `autocert` controller in the `step` namespace.
|
||||
|
||||
```
|
||||
$ kubectl create namespace step
|
||||
```
|
||||
|
||||
To install the CA we need to configmap the CA certificates, signing keys, and configuration artifacts. Note that key material is encrypted so we don't need to use secrets.
|
||||
|
||||
```
|
||||
$ kubectl -n step create configmap config --from-file $(step path)/config
|
||||
$ kubectl -n step create configmap certs --from-file $(step path)/certs
|
||||
$ kubectl -n step create configmap secrets --from-file $(step path)/secrets
|
||||
```
|
||||
|
||||
But we will need to create secrets for the CA and autocert to decrypt their keys:
|
||||
|
||||
```
|
||||
$ kubectl -n step create secret generic ca-password --from-literal password=<ca-password>
|
||||
$ kubectl -n step create secret generic autocert-password --from-literal password=<autocert-password>
|
||||
```
|
||||
|
||||
Where `<ca-password>` is the password you entered during `step ca init` and `<autocert-password>` is the password you entered during `step ca provisioner add`.
|
||||
|
||||
Next, we'll install the CA.
|
||||
|
||||
```
|
||||
$ kubectl apply -f https://raw.githubusercontent.com/smallstep/certificates/master/autocert/install/01-step-ca.yaml
|
||||
```
|
||||
|
||||
Once you've done this you can delete the temporary `$STEPPATH` directory and `unset STEPPATH` (though you may want to retain it as a backup).
|
||||
|
||||
### Install `autocert` in Kubernetes
|
||||
|
||||
Install the `autocert` controller.
|
||||
|
||||
```
|
||||
$ kubectl apply -f https://raw.githubusercontent.com/smallstep/certificates/master/autocert/install/02-autocert.yaml
|
||||
```
|
||||
|
||||
Autocert creates secrets containing single-use bootstrap tokens for pods to authenticate with the CA and obtain a certificate. The tokens are automatically cleaned up after they expire. To do this, `autocert` needs permission to create and delete secrets in your cluster.
|
||||
|
||||
If you have RBAC enabled in your cluster, apply `rbac.yaml` to give `autocert` these permissions.
|
||||
|
||||
```
|
||||
$ kubectl apply -f https://raw.githubusercontent.com/smallstep/certificates/master/autocert/install/03-rbac.yaml
|
||||
```
|
||||
|
||||
Finally, register the `autocert` mutation webhook with kubernetes.
|
||||
|
||||
```
|
||||
$ cat <<EOF | kubectl apply -f -
|
||||
apiVersion: admissionregistration.k8s.io/v1beta1
|
||||
kind: MutatingWebhookConfiguration
|
||||
metadata:
|
||||
name: autocert-webhook-config
|
||||
labels: {app: autocert}
|
||||
webhooks:
|
||||
- name: autocert.step.sm
|
||||
clientConfig:
|
||||
service:
|
||||
name: autocert
|
||||
namespace: step
|
||||
path: "/mutate"
|
||||
caBundle: $(cat $(step path)/certs/root_ca.crt | base64)
|
||||
rules:
|
||||
- operations: ["CREATE"]
|
||||
apiGroups: [""]
|
||||
apiVersions: ["v1"]
|
||||
resources: ["pods"]
|
||||
failurePolicy: Ignore
|
||||
namespaceSelector:
|
||||
matchLabels:
|
||||
autocert.step.sm: enabled
|
||||
EOF
|
||||
```
|
||||
|
||||
### Check your work
|
||||
|
||||
If everything worked you should have CA and controller pods running in the `step` namespace and your webhook configuration should be installed:
|
||||
|
||||
```
|
||||
$ kubectl -n step get pods
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
ca-7577d7d667-vtfq5 1/1 Running 0 1m
|
||||
controller-86bd99bd96-s9zlc 1/1 Running 0 28s
|
||||
$ kubectl get mutatingwebhookconfiguration
|
||||
NAME CREATED AT
|
||||
autocert-webhook-config 2019-01-17T22:57:57Z
|
||||
```
|
||||
413
autocert/README.md
Normal file
413
autocert/README.md
Normal file
|
|
@ -0,0 +1,413 @@
|
|||
|
||||
|
||||
# Autocert
|
||||
[](https://github.com/smallstep/certificates/releases)
|
||||
[](https://microbadger.com/images/smallstep/step-ca)
|
||||
[](https://goreportcard.com/report/github.com/smallstep/certificates)
|
||||
|
||||
[](https://github.com/smallstep/certificates/stargazers)
|
||||
[](https://twitter.com/intent/follow?screen_name=smallsteplabs)
|
||||
|
||||
**Autocert** is a kubernetes add-on that automatically injects TLS/HTTPS certificates into your containers.
|
||||
|
||||
To get a certificate **simply annotate your pods** with a name. An X.509 (TLS/HTTPS) certificate is automatically created and mounted at `/var/run/autocert.step.sm/` along with a corresponding private key and root certificate (everything you need for [mTLS](#motivation)).
|
||||
|
||||
We ❤️ feedback. Please [report bugs](https://github.com/smallstep/certificates/issues/new?template=autocert_bug.md) & [suggest enhancements](https://github.com/smallstep/certificates/issues/new?template=autocert_enhancement.md). [Fork](https://github.com/smallstep/certificates/fork) and send a PR. [Give us a ⭐](https://github.com/smallstep/certificates/stargazers) if you like what we're doing.
|
||||
|
||||
|
||||
|
||||
## Motivation
|
||||
|
||||
`Autocert` exists to **make it easy to use mTLS** ([mutual TLS](examples/hello-mtls/README.md#mutual-tls)) to **improve security** within a cluster and to **secure communication into, out of, and between kubernetes clusters**.
|
||||
|
||||
TLS (and HTTPS, which is HTTP over TLS) provides _authenticated encryption_: an _identity dialtone_ and _end-to-end encryption_ for your workloads. It's like a secure line with caller ID. This has all sorts of benefits: better security, compliance, and easier auditability for starters. It **makes workloads identity-aware**, improving observability and enabling granular access control. Perhaps most compelling, mTLS lets you securely communicate with workloads running anywhere, not just inside kubernetes.
|
||||
|
||||
|
||||
|
||||
Unlike VPNs & SDNs, deploying and scaling mTLS is pretty easy. You're (hopefully) already using TLS, and your existing tools and standard libraries will provide most of what you need. If you know how to operate DNS and reverse proxies, you know how to operate mTLS infrastructure.
|
||||
|
||||
There's just one problem: **you need certificates issued by your own certificate authority (CA)**. Building and operating a CA, issuing certificates, and making sure they're renewed before they expire is tricky. `Autocert` does all of this for you.
|
||||
|
||||
## Features
|
||||
|
||||
First and foremost, `autocert` is easy. You can **get started in minutes**.
|
||||
|
||||
`Autocert` uses [`step certificates`](https://github.com/smallstep/certificates) to generate keys and issue certificates. This process is secure and automatic, all you have to do is [install autocert](#install) and [annotate your pods](#enable-autocert-per-namespace).
|
||||
|
||||
Features include:
|
||||
|
||||
* A fully featured private **certificate authority** (CA) for workloads running on kubernetes and elsewhere
|
||||
* [RFC5280](https://tools.ietf.org/html/rfc5280) and [CA/Browser Forum](https://cabforum.org/baseline-requirements-documents/) compliant certificates that work **for TLS**
|
||||
* Namespaced installation into the `step` namespace so it's **easy to lock down** your CA
|
||||
* Short-lived certificates with **fully automated** enrollment and renewal
|
||||
* Private keys are never transmitted across the network and aren't stored in `etcd`
|
||||
|
||||
Because `autocert` is built on [`step certificates`](https://github.com/smallstep/certificates) you can easily [extend access](#connecting-from-outside-the-cluster) to developers, endpoints, and workloads running outside your cluster, too.
|
||||
|
||||
## Getting Started
|
||||
|
||||
> ⚠️ Warning: *this project is in **ALPHA**. DON'T use it for anything mission critical. EXPECT breaking changes in minor revisions with little or no warning. PLEASE [provide feedback](https://github.com/smallstep/certificates/issues/new?template=autocert_enhancement.md).*
|
||||
|
||||
### Prerequisites
|
||||
|
||||
All you need to get started is [`kubectl`](https://kubernetes.io/docs/tasks/tools/install-kubectl/#install-kubectl) and a cluster running kubernetes `1.9` or later with [admission webhooks](https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/#admission-webhooks) enabled:
|
||||
|
||||
```bash
|
||||
$ kubectl version --short
|
||||
Client Version: v1.13.1
|
||||
Server Version: v1.10.11
|
||||
$ kubectl api-versions | grep "admissionregistration.k8s.io/v1beta1"
|
||||
admissionregistration.k8s.io/v1beta1
|
||||
```
|
||||
|
||||
### Install
|
||||
|
||||
To install `autocert` run:
|
||||
|
||||
```bash
|
||||
kubectl run autocert-init -it --rm --image smallstep/autocert-init --restart Never
|
||||
```
|
||||
|
||||
💥 installation complete.
|
||||
|
||||
> You might want to [check out what this command does](init/autocert.sh) before running it. You can also [install `autocert` manually](INSTALL.md#manual-install) if that's your style.
|
||||
|
||||
## Usage
|
||||
|
||||
Using `autocert` is also easy:
|
||||
|
||||
* Enable `autocert` for a namespace by labelling it with `autocert.step.sm=enabled`, then
|
||||
* Inject certificates into containers by annotating pods with `autocert.step.sm/name: <name>`
|
||||
|
||||
### Enable autocert (per namespace)
|
||||
|
||||
To enable `autocert` for a namespace it must be labelled `autocert.step.sm=enabled`.
|
||||
|
||||
To label the `default` namespace run:
|
||||
|
||||
```bash
|
||||
kubectl label namespace default autocert.step.sm=enabled
|
||||
```
|
||||
|
||||
To check which namespaces have `autocert` enabled run:
|
||||
|
||||
```bash
|
||||
$ kubectl get namespace -L autocert.step.sm
|
||||
NAME STATUS AGE AUTOCERT.STEP.SM
|
||||
default Active 59m enabled
|
||||
...
|
||||
```
|
||||
|
||||
### Annotate pods to get certificates
|
||||
|
||||
To get a certificate you need to tell `autocert` your workload's name using the `autocert.step.sm/name` annotation (this name will appear as the X.509 common name and SAN).
|
||||
|
||||
Let's deploy a [simple mTLS server](examples/hello-mtls/go/server.go) named `hello-mtls.default.svc.cluster.local`:
|
||||
|
||||
```yaml
|
||||
cat <<EOF | kubectl apply -f -
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata: {name: hello-mtls, labels: {app: hello-mtls}}
|
||||
spec:
|
||||
replicas: 1
|
||||
selector: {matchLabels: {app: hello-mtls}}
|
||||
template:
|
||||
metadata:
|
||||
annotations:
|
||||
# AUTOCERT ANNOTATION HERE -v ###############################
|
||||
autocert.step.sm/name: hello-mtls.default.svc.cluster.local #
|
||||
# AUTOCERT ANNOTATION HERE -^ ###############################
|
||||
labels: {app: hello-mtls}
|
||||
spec:
|
||||
containers:
|
||||
- name: hello-mtls
|
||||
image: smallstep/hello-mtls-server-go:latest
|
||||
EOF
|
||||
```
|
||||
|
||||
In our new container we should find a certificate, private key, and root certificate mounted at `/var/run/autocert.step.sm`:
|
||||
|
||||
```bash
|
||||
$ export HELLO_MTLS=$(kubectl get pods -l app=hello-mtls -o jsonpath={$.items[0].metadata.name})
|
||||
$ kubectl exec -it $HELLO_MTLS -c hello-mtls -- ls /var/run/autocert.step.sm
|
||||
root.crt site.crt site.key
|
||||
```
|
||||
|
||||
We're done. Our container has a certificate, issued by our CA, which `autocert` will automatically renew.
|
||||
|
||||
✅ Certificates.
|
||||
|
||||
## Hello mTLS
|
||||
|
||||
It's easy to deploy certificates using `autocert`, but it's up to you to use them correctly. To get you started, [`hello-mtls`](examples/hello-mtls) demonstrates the right way to use mTLS with various tools and languages (contributions welcome :). If you're a bit fuzzy on how mTLS works, [the `hello-mtls` README](examples/hello-mtls/README.md) is a great place to start.
|
||||
|
||||
To finish out this tutorial let's keep things simple and try `curl`ing the server we just deployed from inside and outside the cluster.
|
||||
|
||||
### Connecting from inside the cluster
|
||||
|
||||
First, let's expose our workload to the rest of the cluster using a service:
|
||||
|
||||
```
|
||||
kubectl expose deployment hello-mtls --port 443
|
||||
```
|
||||
|
||||
Now let's deploy a client, with its own certificate, that [`curl`s our server in a loop](examples/hello-mtls/curl/client.sh):
|
||||
|
||||
```yaml
|
||||
cat <<EOF | kubectl apply -f -
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata: {name: hello-mtls-client, labels: {app: hello-mtls-client}}
|
||||
spec:
|
||||
replicas: 1
|
||||
selector: {matchLabels: {app: hello-mtls-client}}
|
||||
template:
|
||||
metadata:
|
||||
annotations:
|
||||
# AUTOCERT ANNOTATION HERE -v ######################################
|
||||
autocert.step.sm/name: hello-mtls-client.default.pod.cluster.local #
|
||||
# AUTOCERT ANNOTATION HERE -^ ######################################
|
||||
labels: {app: hello-mtls-client}
|
||||
spec:
|
||||
containers:
|
||||
- name: hello-mtls-client
|
||||
image: smallstep/hello-mtls-client-curl:latest
|
||||
env: [{name: HELLO_MTLS_URL, value: https://hello-mtls.default.svc.cluster.local}]
|
||||
EOF
|
||||
```
|
||||
|
||||
> Note that **the authority portion of the URL** (the `HELLO_MTLS_URL` env var) **matches the name of the server we're connecting to** (both are `hello-mtls.default.svc.cluster.local`). That's required for standard HTTPS and can sometimes require some DNS trickery.
|
||||
|
||||
Once deployed we should start seeing the client log responses from the server [saying hello](examples/hello-mtls/go/server.go#L71-L72):
|
||||
|
||||
```
|
||||
$ export HELLO_MTLS_CLIENT=$(kubectl get pods -l app=hello-mtls-client -o jsonpath={$.items[0].metadata.name})
|
||||
$ kubectl logs $HELLO_MTLS_CLIENT -c hello-mtls-client
|
||||
Thu Feb 7 23:35:23 UTC 2019: Hello, hello-mtls-client.default.pod.cluster.local!
|
||||
Thu Feb 7 23:35:28 UTC 2019: Hello, hello-mtls-client.default.pod.cluster.local!
|
||||
```
|
||||
|
||||
For kicks, let's `exec` into this pod and try `curl`ing ourselves:
|
||||
|
||||
```
|
||||
$ kubectl exec $HELLO_MTLS_CLIENT -c hello-mtls-client -- curl -sS \
|
||||
--cacert /var/run/autocert.step.sm/root.crt \
|
||||
--cert /var/run/autocert.step.sm/site.crt \
|
||||
--key /var/run/autocert.step.sm/site.key \
|
||||
https://hello-mtls.default.svc.cluster.local
|
||||
Hello, hello-mtls-client.default.pod.cluster.local!
|
||||
```
|
||||
|
||||
✅ mTLS inside cluster.
|
||||
|
||||
### Connecting from outside the cluster
|
||||
|
||||
Connecting from outside the cluster is a bit more complicated. We need to handle DNS and obtain a certificate ourselves. These tasks were handled automatically inside the cluster by kubernetes and `autocert`, respectively.
|
||||
|
||||
That said, because our server uses mTLS **only clients that have a certificate issued by our certificate authority will be allowed to connect**. That means it can be safely and easily exposed directly to the public internet using a [LoadBalancer service type](https://kubernetes.io/docs/concepts/services-networking/service/#loadbalancer):
|
||||
|
||||
```
|
||||
kubectl expose deployment hello-mtls --name=hello-mtls-lb --port=443 --type=LoadBalancer
|
||||
```
|
||||
|
||||
To connect we need a certificate. There are a [couple](RUNBOOK.md#federation) [different](RUNBOOK.md#multiple-intermediates) [ways](RUNBOOK.md#exposing-the-ca) to get one, but for simplicity we'll just forward a port.
|
||||
|
||||
```
|
||||
kubectl -n step port-forward $(kubectl -n step get pods -l app=ca -o jsonpath={$.items[0].metadata.name}) 4443:4443
|
||||
```
|
||||
|
||||
In another window we'll use `step` to grab the root certificate, generate a key pair, and get a certificate.
|
||||
|
||||
> To follow along you'll need to [`install step`](https://github.com/smallstep/cli#installing) if you haven't already. You'll also need your admin password and CA fingerprint, which were output during installation (see [here](RUNBOOK.md#recover-admin-and-ca-password) and [here](RUNBOOK.md#recompute-root-certificate-fingerprint) if you already lost them :).
|
||||
|
||||
```bash
|
||||
$ export CA_POD=$(kubectl -n step get pods -l app=ca -o jsonpath={$.items[0].metadata.name})
|
||||
$ step ca root root.crt --ca-url https://127.0.0.1:4443 --fingerprint <fingerprint>
|
||||
$ step ca certificate mike mike.crt mike.key --ca-url https://127.0.0.1:4443 --root root.crt
|
||||
✔ Key ID: H4vH5VfvaMro0yrk-UIkkeCoPFqEfjF6vg0GHFdhVyM (admin)
|
||||
✔ Please enter the password to decrypt the provisioner key: 0QOC9xcq56R1aEyLHPzBqN18Z3WfGZ01
|
||||
✔ CA: https://127.0.0.1:4443/1.0/sign
|
||||
✔ Certificate: mike.crt
|
||||
✔ Private Key: mike.key
|
||||
```
|
||||
|
||||
Now we can simply `curl` the service:
|
||||
|
||||
> If you're using minikube or docker for mac the load balancer's "IP" might be `localhost`, which won't work. In that case, simply `export HELLO_MTLS_IP=127.0.0.1` and try again.
|
||||
|
||||
```
|
||||
$ export HELLO_MTLS_IP=$(kubectl get svc hello-mtls-lb -ojsonpath={$.status.loadBalancer.ingress[0].ip})
|
||||
$ curl --resolve hello-mtls.default.svc.cluster.local:443:$HELLO_MTLS_IP \
|
||||
--cacert root.crt \
|
||||
--cert mike.crt \
|
||||
--key mike.key \
|
||||
https://hello-mtls.default.svc.cluster.local
|
||||
Hello, mike!
|
||||
```
|
||||
|
||||
> Note that we're using `--resolve` to tell `curl` to override DNS and resolve the name in our workload's certificate to its public IP address. In a real production infrastructure you could configure DNS manually, or you could propagate DNS to workloads outside kubernetes using something like [ExternalDNS](https://github.com/kubernetes-incubator/external-dns).
|
||||
|
||||
✅ mTLS outside cluster.
|
||||
|
||||
<!--- TODO: CTA or Further Reading... Move "How it works" maybe? Or put this below that? --->
|
||||
|
||||
### Cleanup & uninstall
|
||||
|
||||
To clean up after running through the tutorial remove the `hello-mtls` and `hello-mtls-client` deployments and services:
|
||||
|
||||
```
|
||||
kubectl delete deployment hello-mtls
|
||||
kubectl delete deployment hello-mtls-client
|
||||
kubectl delete service hello-mtls
|
||||
kubectl delete service hello-mtls-lb
|
||||
```
|
||||
|
||||
See the runbook for instructions on [uninstalling `autocert`](RUNBOOK.md#uninstalling).
|
||||
|
||||
## How it works
|
||||
|
||||
### Architecture
|
||||
|
||||
`Autocert` is an [admission webhook](https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/#admission-webhooks) that intercepts and patches pod creation requests with [some YAML](install/02-autocert.yaml#L26-L44) to inject an [init container](bootstrapper/) and [sidecar](renewer/) that handle obtaining and renewing certificates, respectively.
|
||||
|
||||
|
||||
|
||||
### Enrollment & renewal
|
||||
|
||||
It integrates with [`step certificates`](https://github.com/smallstep/certificates) and uses the [one-time token bootstrap protocol](https://smallstep.com/blog/step-certificates.html#automated-certificate-management) from that project to mutually authenticate a new pod with your certificate authority, and obtain a certificate.
|
||||
|
||||
|
||||
|
||||
Tokens are [generated by the admission webhook](controller/provisioner.go#L46-L72) and [transmitted to the injected init container via a kubernetes secret](controller/main.go#L91-L125). The init container [uses the one-time token](bootstrapper/bootstrapper.sh) to obtain a certificate. A sidecar is also installed to [renew certificates](renewer/Dockerfile#L8) before they expire. Renewal simply uses mTLS with the CA.
|
||||
|
||||
## Further Reading
|
||||
|
||||
* We tweet [@smallsteplabs](https://twitter.com/smallsteplabs)
|
||||
* Read [our blog](https://smallstep.com/blog)
|
||||
* Check out the [runbook](RUNBOOK.md)
|
||||
* Check out [`step` CLI](https://github.com/smallstep/cli)
|
||||
|
||||
## Questions
|
||||
|
||||
#### Wait, so any pod can get a certificate with any identity? How is that secure?
|
||||
|
||||
1. Don't give people `kubectl` access to your production clusters
|
||||
2. Use a deploy pipeline based on `git` artifacts
|
||||
3. Enforce code review on those `git` artifacts
|
||||
|
||||
If that doesn't work for you, or if you have a better idea, we'd love to hear! Please [open an issue](https://github.com/smallstep/certificates/issues/new?template=autocert_enhancement.md)!
|
||||
|
||||
#### Why do I have to tell you the name to put in a certificate? Why can't you automatically bind service names?
|
||||
|
||||
Mostly because monitoring the API server to figure out which services are associated with which workloads is complicated and somewhat magical. And it might not be what you want.
|
||||
|
||||
That said, we're not totally opposed to this idea. If anyone has strong feels and a good design please [open an issue](https://github.com/smallstep/certificates/issues/new?template=autocert_enhancement.md).
|
||||
|
||||
#### Doesn't kubernetes already ship with a certificate authority?
|
||||
|
||||
Yes, it uses [a bunch of CAs](https://jvns.ca/blog/2017/08/05/how-kubernetes-certificates-work/) for different sorts of control plane communication. Technically, kubernetes doesn't _come with_ a CA. It has integration points that allow you to use any CA (e.g., [Kubernetes the hard way](https://github.com/kelseyhightower/kubernetes-the-hard-way) [uses CFSSL](https://github.com/kelseyhightower/kubernetes-the-hard-way/blob/2983b28f13b294c6422a5600bb6f14142f5e7a26/docs/02-certificate-authority.md). You could use [`step certificates`](https://github.com/smallstep/certificates), which `autocert` is based on, instead.
|
||||
|
||||
In any case, these CAs are meant for control plane communication. You could use them for your service-to-service data plane, but it's probably not a good idea.
|
||||
|
||||
#### What permissions does `autocert` require in my cluster and why?
|
||||
|
||||
`Autocert` needs permission to create and delete secrets cluster-wide. You can [check out our RBAC config here](install/03-rbac.yaml). These permissions are needed in order to transmit one-time tokens to workloads using secrets, and to clean up afterwards. We'd love to scope these permissions down further. If anyone has any ideas please [open an issue](https://github.com/smallstep/certificates/issues/new?template=autocert_enhancement.md).
|
||||
|
||||
#### Why does `autocert` create secrets?
|
||||
|
||||
The `autocert` admission webhook needs to securely transmit one-time bootstrap tokens to containers. This could be accomplished without using secrets. The webhook returns a [JSONPatch](https://tools.ietf.org/html/rfc6902) response that's applied to the pod spec. This response could patch the literal token value into our init container's environment.
|
||||
|
||||
Unfortunately, the kubernetes API server does not authenticate itself to admission webhooks by default, and configuring it to do so [requires passing a custom config file](https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/#authenticate-apiservers) at apiserver startup. This isn't an option for everyone (e.g., on GKE) so we opted not to rely on it.
|
||||
|
||||
Since our webhook can't authenticate callers, including bootstrap tokens in patch responses would be dangerous. By using secrets an attacker can still trick `autocert` into generating superflous bootstrap tokens, but they'd also need read access to cluster secrets to do anything with them.
|
||||
|
||||
Hopefully this story will improve with time.
|
||||
|
||||
#### Why not use kubernetes service accounts instead of bootstrap tokens?
|
||||
|
||||
Great idea! This should be pretty easy to add. However, existing service accounts are [somewhat broken](https://github.com/kubernetes/community/pull/1460) for this use case. The upcoming [TokenRequest API](https://github.com/kubernetes/kubernetes/issues/58790) should fix most of these issues.
|
||||
|
||||
TODO: Link to issue for people who want this.
|
||||
|
||||
#### Too. many. containers. Why do you need to install an init container and sidecar?
|
||||
|
||||
We don't. It's just easier for you. Your containers can generate key pairs, exchange them for certificates, and manage renewals themselves. This is pretty easy if you [install `step`](https://github.com/smallstep/cli#installing) in your containers, or integrate with our [golang SDK](https://godoc.org/github.com/smallstep/certificates/ca). To support this we'd need to add the option to inject a bootstrap token without injecting these containers.
|
||||
|
||||
TODO: Link to issue for people who want this.
|
||||
|
||||
That said, the init container and sidecar are both super lightweight.
|
||||
|
||||
#### Why are keys and certificates managed via volume mounts? Why not use a secret or some custom resource?
|
||||
|
||||
Because, by default, kubernetes secrets are stored in plaintext in `etcd` and might even be transmitted unencrypted across the network. Even if secrets were properly encrypted, transmitting a private key across the network violates PKI best practices. Key pairs should always be generated where they're used, and private keys should never be known by anyone but their owners.
|
||||
|
||||
That said, there are use cases where a certificate mounted in a secret resource is desirable (e.g., for use with a kubernetes `Ingress`). We may add support for this in the future. However, we think the current method is easier and a better default.
|
||||
|
||||
TODO: Link to issue for people who want this.
|
||||
|
||||
#### Why not use kubernetes CSR resources for this?
|
||||
|
||||
It's harder and less secure. If any good and simple design exists for securely automating CSR approval using this resource we'd love to see it!
|
||||
|
||||
#### How is this different than [`cert-manager`](https://github.com/jetstack/cert-manager)
|
||||
|
||||
`Cert-manager` is a great project. But it's design is focused on managing Web PKI certificates issued by [Let's Encrypt's](https://letsencrypt.org/) public certificate authority. These certificates are useful for TLS ingress from web browsers. `Autocert` is different. It's purpose-built to manage certificates issued by your own private CA to support the use of mTLS for internal communication (e.g., service-to-service).
|
||||
|
||||
#### What sorts of keys are issued and how often are certificates rotated?
|
||||
|
||||
`Autocert` builds on `step certificates` which issues ECDSA certificates using the P256 curve with ECDSA-SHA256 signatures by default. If this is all Greek to you, rest assured these are safe, sane, and modern defaults that are suitable for the vast majority of environments.
|
||||
|
||||
#### What crypto library is under the hood?
|
||||
|
||||
https://golang.org/pkg/crypto/
|
||||
|
||||
## Building
|
||||
|
||||
This project is based on four docker containers. They use [multi-stage builds](https://docs.docker.com/develop/develop-images/multistage-build/) so all you need in order to build them is `docker`.
|
||||
|
||||
> Caveat: the `controller` container uses [`dep`](https://github.com/golang/dep) and `dep init` isn't run during the build. You'll need to run `dep init` in the `controller/` subdirectory prior to building, and you'll need to run `dep ensure -update` if you change any dependencies.
|
||||
|
||||
Building `autocert-controller` (the admission webhook):
|
||||
|
||||
```
|
||||
cd controller
|
||||
docker build -t smallstep/autocert-controller:latest .
|
||||
```
|
||||
|
||||
Building `autocert-bootstrapper` (the init container that generates a key pair and exchanges a bootstrap token for a certificate):
|
||||
|
||||
```
|
||||
cd bootstrapper
|
||||
docker build -t smallstep/autocert-bootstrapper:latest .
|
||||
```
|
||||
|
||||
Building `autocert-renewer` (the sidecar that renews certificates):
|
||||
|
||||
```
|
||||
cd renewer
|
||||
docker build -t smallstep/autocert-renewer:latest .
|
||||
```
|
||||
|
||||
Building `autocert-init` (the install script):
|
||||
|
||||
```
|
||||
cd init
|
||||
docker build -t smallstep/autocert-init:latest .
|
||||
```
|
||||
|
||||
If you build your own containers you'll probably need to [install manually](INSTALL.md). You'll also need to adjust which images are deployed in the [deployment yaml](install/02-autocert.yaml).
|
||||
|
||||
## Contributing
|
||||
|
||||
If you have improvements to `autocert`, send us your pull requests! For those just getting started, GitHub has a [howto](https://help.github.com/articles/about-pull-requests/). A team member will review your pull requests, provide feedback, and merge your changes. In order to accept contributions we do need you to [sign our contributor license agreement](https://cla-assistant.io/smallstep/certificates).
|
||||
|
||||
If you want to contribute but you're not sure where to start, take a look at the [issues with the "good first issue" label](https://github.com/smallstep/certificates/issues?q=is%3Aopen+label%3A%22good+first+issue%22+label%3Aarea%2Fautocert). These are issues that we believe are particularly well suited for outside contributions, often because we probably won't get to them right now. If you decide to start on an issue, leave a comment so that other people know that you're working on it. If you want to help out, but not alone, use the issue comment thread to coordinate.
|
||||
|
||||
If you've identified a bug or have ideas for improving `autocert` that you don't have time to implement, we'd love to hear about them. Please open an issue to [report a bug](https://github.com/smallstep/certificates/issues/new?template=autocert_bug.md) or [suggest an enhancement](https://github.com/smallstep/certificates/issues/new?template=autocert_enhancement.md)!
|
||||
|
||||
## License
|
||||
|
||||
Copyright 2019 Smallstep Labs
|
||||
|
||||
Licensed under [the Apache License, Version 2.0](https://github.com/smallstep/certificates/blob/master/LICENSE)
|
||||
122
autocert/RUNBOOK.md
Normal file
122
autocert/RUNBOOK.md
Normal file
|
|
@ -0,0 +1,122 @@
|
|||
# Runbook
|
||||
|
||||
## Common admin tasks
|
||||
|
||||
#### Recover `admin` and CA password
|
||||
|
||||
```
|
||||
kubectl -n step get secret ca-password -o jsonpath='{$.data.password}' | base64 -D
|
||||
```
|
||||
|
||||
#### Recover `autocert` password
|
||||
|
||||
```
|
||||
kubectl -n step get secret autocert-password -o jsonpath='{$.data.password}' | base64 -D
|
||||
```
|
||||
|
||||
#### Recompute root certificate fingerprint
|
||||
|
||||
```
|
||||
export CA_POD=$(kubectl -n step get pods -l app=ca -o jsonpath={$.items[0].metadata.name})
|
||||
kubectl -n step exec -it $CA_POD step certificate fingerprint /home/step/.step/certs/root_ca.crt
|
||||
```
|
||||
|
||||
> Tip: Some slight fanciness is necessary to trim this string if you want to put it into an environment variable:
|
||||
>
|
||||
> ```
|
||||
> export FINGERPRINT="$(kubectl -n step exec -it $CA_POD step certificate fingerprint /home/step/.step/certs/root_ca.crt | tr -d '[:space:]')"
|
||||
> ```
|
||||
|
||||
#### Inspect a certificate
|
||||
|
||||
```
|
||||
kubectl exec -it <pod> -c autocert-renewer -- step certificate inspect /var/run/autocert.step.sm/site.crt
|
||||
```
|
||||
|
||||
#### Labelling a namespace (enabling `autocert` for a namespace)
|
||||
|
||||
To enable `autocert` for a namespace it must be labelled. To label an existing namespace run:
|
||||
|
||||
```
|
||||
kubectl label namespace <namespace> autocert.step.sm=enabled
|
||||
```
|
||||
|
||||
#### Checking which namespaces are labelled
|
||||
|
||||
```
|
||||
kubectl get namespace -L autocert.step.sm
|
||||
```
|
||||
|
||||
#### Removing a label from a namespace (disabling `autocert` for a namespace)
|
||||
|
||||
```
|
||||
kubectl label namespace <namespace> autocert.step.sm-
|
||||
```
|
||||
|
||||
#### Naming considerations
|
||||
|
||||
Use hostnames. Must be global. Everyone who connects to the service using mTLS must use the same hostname. For internal communication it's easy enough to use the FQDN of a service. For stuff you expose publicly you'll need to manage DNS yourself...
|
||||
|
||||
In any case, the critical invariant is: ...
|
||||
|
||||
Diagram here?
|
||||
|
||||
#### Cleaning up one-time token secrets
|
||||
|
||||
```
|
||||
for ns in $(kubectl get namespace --selector autocert.step.sm=enabled -o jsonpath='{$.items[*].metadata.name}'); do
|
||||
kubectl -n "$ns" delete secrets --selector="autocert.step.sm/token=true"
|
||||
done
|
||||
```
|
||||
|
||||
### TODO:
|
||||
* Change admin password
|
||||
* Change autocert password
|
||||
* Federating with another CA
|
||||
* DNS tips and tricks
|
||||
* Multiple SANs
|
||||
* Getting rid of the sidecar
|
||||
* Getting logs from the CA (certificates weren't issued)
|
||||
* Getting logs from the init container / renewer (didn't start properly)
|
||||
* Adjusting certificate expiration (default 24h)
|
||||
* Remove label
|
||||
* Clean up secrets
|
||||
* Naming considerations (maybe this should be in hello-mtls)
|
||||
|
||||
## Federation
|
||||
|
||||
TODO: Example of federating a CA running in kubernetes with another CA.
|
||||
|
||||
For now, see https://smallstep.com/blog/step-v0.8.3-federation-root-rotation.html
|
||||
|
||||
## Multiple intermediates
|
||||
|
||||
TODO: Example of creating an additional intermediate signing certificate off of our kubernetes root CA.
|
||||
|
||||
For now, see https://smallstep.com/docs/cli/ca/init/ (specifically, the `--root` flag)
|
||||
|
||||
## Exposing the CA
|
||||
|
||||
Beware that the CA exposes an unauthenticated endpoint that lists your configured provisioners and their encrypted private keys. For this reason, you may not want to expose it directly to the public internet.
|
||||
|
||||
## Uninstalling
|
||||
|
||||
To uninstall `autocert` completely simply delete the mutating webhook configuration, the `step` namespace and the `autocert` RBAC artifacts:
|
||||
|
||||
```
|
||||
kubectl delete mutatingwebhookconfiguration autocert-webhook-config
|
||||
kubectl delete namespace step
|
||||
kubectl delete clusterrolebinding autocert-controller
|
||||
kubectl delete clusterrole autocert-controller
|
||||
```
|
||||
|
||||
Remove any namespace labels and clean up any stray secrets that `autocert` hasn't cleaned up yet:
|
||||
|
||||
```
|
||||
for ns in $(kubectl get namespace --selector autocert.step.sm=enabled -o jsonpath='{$.items[*].metadata.name}'); do
|
||||
kubectl label namespace "$ns" autocert.step.sm-
|
||||
kubectl -n "$ns" delete secrets --selector="autocert.step.sm/token=true"
|
||||
done
|
||||
```
|
||||
|
||||
Any remaining sidecar containers will go away once you remove annotations and re-deploy your workloads.
|
||||
BIN
autocert/autocert-arch.png
Normal file
BIN
autocert/autocert-arch.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 566 KiB |
BIN
autocert/autocert-bootstrap.png
Normal file
BIN
autocert/autocert-bootstrap.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 396 KiB |
BIN
autocert/autocert-logo.png
Normal file
BIN
autocert/autocert-logo.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 112 KiB |
10
autocert/bootstrapper/Dockerfile
Normal file
10
autocert/bootstrapper/Dockerfile
Normal file
|
|
@ -0,0 +1,10 @@
|
|||
FROM smallstep/step-cli:0.8.3
|
||||
|
||||
USER root
|
||||
ENV CRT="/var/run/autocert.step.sm/site.crt"
|
||||
ENV KEY="/var/run/autocert.step.sm/site.key"
|
||||
ENV STEP_ROOT="/var/run/autocert.step.sm/root.crt"
|
||||
|
||||
COPY bootstrapper.sh /home/step/
|
||||
RUN chmod +x /home/step/bootstrapper.sh
|
||||
CMD ["/home/step/bootstrapper.sh"]
|
||||
7
autocert/bootstrapper/bootstrapper.sh
Normal file
7
autocert/bootstrapper/bootstrapper.sh
Normal file
|
|
@ -0,0 +1,7 @@
|
|||
#!/bin/sh
|
||||
|
||||
# Download the root certificate and set permissions
|
||||
step ca certificate $COMMON_NAME $CRT $KEY
|
||||
chmod 644 $CRT $KEY
|
||||
|
||||
step ca root $STEP_ROOT
|
||||
BIN
autocert/connect-with-mtls.png
Normal file
BIN
autocert/connect-with-mtls.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 669 KiB |
19
autocert/controller/Dockerfile
Normal file
19
autocert/controller/Dockerfile
Normal file
|
|
@ -0,0 +1,19 @@
|
|||
# build stage
|
||||
FROM golang:alpine AS build-env
|
||||
RUN apk update && apk upgrade && \
|
||||
apk add --no-cache git
|
||||
RUN go get -u github.com/golang/dep/cmd/dep
|
||||
WORKDIR $GOPATH/src/github.com/step-certificates-k8s/controller
|
||||
# copy dep files and run dep separately from code for better caching
|
||||
COPY Gopkg.toml Gopkg.lock ./
|
||||
RUN dep ensure --vendor-only
|
||||
COPY . ./
|
||||
RUN go build -o /server .
|
||||
|
||||
# final stage
|
||||
FROM smallstep/step-cli:0.8.3
|
||||
ENV STEPPATH="/home/step/.step"
|
||||
ENV PWDPATH="/home/step/password/password"
|
||||
ENV CONFIGPATH="/home/step/autocert/config.yaml"
|
||||
COPY --from=build-env /server .
|
||||
ENTRYPOINT ./server $CONFIGPATH
|
||||
124
autocert/controller/client.go
Normal file
124
autocert/controller/client.go
Normal file
|
|
@ -0,0 +1,124 @@
|
|||
package main
|
||||
|
||||
import (
|
||||
"crypto/tls"
|
||||
"crypto/x509"
|
||||
"fmt"
|
||||
"io/ioutil"
|
||||
"net"
|
||||
"net/http"
|
||||
"os"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
const (
|
||||
serviceAccountToken = "/var/run/secrets/kubernetes.io/serviceaccount/token"
|
||||
serviceAccountCACert = "/var/run/secrets/kubernetes.io/serviceaccount/ca.crt"
|
||||
)
|
||||
|
||||
// Client is minimal kubernetes client interface
|
||||
type Client interface {
|
||||
Do(req *http.Request) (*http.Response, error)
|
||||
GetRequest(url string) (*http.Request, error)
|
||||
PostRequest(url, body, contentType string) (*http.Request, error)
|
||||
DeleteRequest(url string) (*http.Request, error)
|
||||
Host() string
|
||||
}
|
||||
|
||||
type k8sClient struct {
|
||||
host string
|
||||
token string
|
||||
httpClient *http.Client
|
||||
}
|
||||
|
||||
func (kc *k8sClient) GetRequest(url string) (*http.Request, error) {
|
||||
if !strings.HasPrefix(url, kc.host) {
|
||||
url = fmt.Sprintf("%s/%s", kc.host, url)
|
||||
}
|
||||
req, err := http.NewRequest("GET", url, nil)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if len(kc.token) > 0 {
|
||||
req.Header.Set("Authorization", "Bearer "+kc.token)
|
||||
}
|
||||
return req, nil
|
||||
}
|
||||
|
||||
func (kc *k8sClient) PostRequest(url string, body string, contentType string) (*http.Request, error) {
|
||||
if !strings.HasPrefix(url, kc.host) {
|
||||
url = fmt.Sprintf("%s/%s", kc.host, url)
|
||||
}
|
||||
req, err := http.NewRequest("POST", url, strings.NewReader(body))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if len(kc.token) > 0 {
|
||||
req.Header.Set("Authorization", "Bearer "+kc.token)
|
||||
}
|
||||
if contentType != "" {
|
||||
req.Header.Set("Content-Type", contentType)
|
||||
}
|
||||
return req, nil
|
||||
}
|
||||
|
||||
func (kc *k8sClient) DeleteRequest(url string) (*http.Request, error) {
|
||||
if !strings.HasPrefix(url, kc.host) {
|
||||
url = fmt.Sprintf("%s/%s", kc.host, url)
|
||||
}
|
||||
req, err := http.NewRequest("DELETE", url, nil)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if len(kc.token) > 0 {
|
||||
req.Header.Set("Authorization", "Bearer "+kc.token)
|
||||
}
|
||||
return req, nil
|
||||
}
|
||||
|
||||
func (kc *k8sClient) Do(req *http.Request) (*http.Response, error) {
|
||||
return kc.httpClient.Do(req)
|
||||
}
|
||||
|
||||
func (kc *k8sClient) Host() string {
|
||||
return kc.host
|
||||
}
|
||||
|
||||
// NewInClusterK8sClient creates K8sClient if it is inside Kubernetes
|
||||
func NewInClusterK8sClient() (Client, error) {
|
||||
host, port := os.Getenv("KUBERNETES_SERVICE_HOST"), os.Getenv("KUBERNETES_SERVICE_PORT")
|
||||
if len(host) == 0 || len(port) == 0 {
|
||||
return nil, fmt.Errorf("unable to load in-cluster configuration, KUBERNETES_SERVICE_HOST and KUBERNETES_SERVICE_PORT must be defined")
|
||||
}
|
||||
token, err := ioutil.ReadFile(serviceAccountToken)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
ca, err := ioutil.ReadFile(serviceAccountCACert)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
certPool := x509.NewCertPool()
|
||||
certPool.AppendCertsFromPEM(ca)
|
||||
transport := &http.Transport{TLSClientConfig: &tls.Config{
|
||||
MinVersion: tls.VersionTLS10,
|
||||
RootCAs: certPool,
|
||||
}}
|
||||
httpClient := &http.Client{Transport: transport, Timeout: time.Nanosecond * 0}
|
||||
|
||||
return &k8sClient{
|
||||
host: "https://" + net.JoinHostPort(host, port),
|
||||
token: string(token),
|
||||
httpClient: httpClient,
|
||||
}, nil
|
||||
}
|
||||
|
||||
// NewInsecureK8sClient creates an insecure k8s client which is suitable
|
||||
// to connect kubernetes api behind proxy
|
||||
func NewInsecureK8sClient(apiURL string) Client {
|
||||
return &k8sClient{
|
||||
host: apiURL,
|
||||
httpClient: http.DefaultClient,
|
||||
}
|
||||
}
|
||||
601
autocert/controller/main.go
Normal file
601
autocert/controller/main.go
Normal file
|
|
@ -0,0 +1,601 @@
|
|||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/sha256"
|
||||
"encoding/json"
|
||||
"encoding/hex"
|
||||
"fmt"
|
||||
"io/ioutil"
|
||||
"net/http"
|
||||
"os"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/ghodss/yaml"
|
||||
"github.com/pkg/errors"
|
||||
log "github.com/sirupsen/logrus"
|
||||
"github.com/smallstep/certificates/ca"
|
||||
"github.com/smallstep/cli/crypto/pemutil"
|
||||
"k8s.io/api/admission/v1beta1"
|
||||
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
|
||||
corev1 "k8s.io/api/core/v1"
|
||||
"k8s.io/apimachinery/pkg/runtime"
|
||||
"k8s.io/apimachinery/pkg/runtime/serializer"
|
||||
)
|
||||
|
||||
var (
|
||||
runtimeScheme = runtime.NewScheme()
|
||||
codecs = serializer.NewCodecFactory(runtimeScheme)
|
||||
deserializer = codecs.UniversalDeserializer()
|
||||
// GetRootCAPath() is broken; points to secrets not certs. So
|
||||
// we'll hard code instead for now.
|
||||
//rootCAPath = pki.GetRootCAPath()
|
||||
rootCAPath = "/home/step/.step/certs/root_ca.crt"
|
||||
)
|
||||
|
||||
const (
|
||||
admissionWebhookAnnotationKey = "autocert.step.sm/name"
|
||||
admissionWebhookStatusKey = "autocert.step.sm/status"
|
||||
provisionerPasswordFile = "/home/step/password/password"
|
||||
volumeMountPath = "/var/run/autocert.step.sm"
|
||||
tokenSecretKey = "token"
|
||||
tokenSecretLabel = "autocert.step.sm/token"
|
||||
)
|
||||
|
||||
// Config options for the autocert admission controller.
|
||||
type Config struct {
|
||||
LogFormat string `yaml:"logFormat"`
|
||||
CaUrl string `yaml:"caUrl"`
|
||||
CertLifetime string `yaml:"certLifetime"`
|
||||
Bootstrapper corev1.Container `yaml:"bootstrapper"`
|
||||
Renewer corev1.Container `yaml:"renewer"`
|
||||
CertsVolume corev1.Volume `yaml:"certsVolume"`
|
||||
}
|
||||
|
||||
// RFC6902 JSONPatch Operation
|
||||
type PatchOperation struct {
|
||||
Op string `json:"op"`
|
||||
Path string `json:"path"`
|
||||
Value interface{} `json:"value,omitempty"`
|
||||
}
|
||||
|
||||
// RFC6901 JSONPath Escaping -- https://tools.ietf.org/html/rfc6901
|
||||
func escapeJsonPath(path string) string {
|
||||
// Replace`~` with `~0` then `/` with `~1`. Note that the order
|
||||
// matters otherwise we'll turn a `/` into a `~/`.
|
||||
path = strings.Replace(path, "~", "~0", -1)
|
||||
path = strings.Replace(path, "/", "~1", -1)
|
||||
return path
|
||||
}
|
||||
|
||||
func loadConfig(file string) (*Config, error) {
|
||||
data, err := ioutil.ReadFile(file)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
var cfg Config
|
||||
if err := yaml.Unmarshal(data, &cfg); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return &cfg, nil
|
||||
}
|
||||
|
||||
// createTokenSecret generates a kubernetes Secret object containing a bootstrap token
|
||||
// in the specified namespce. The secret name is randomly generated with a given prefix.
|
||||
// A goroutine is scheduled to cleanup the secret after the token expires. The secret
|
||||
// is also labelled for easy identification and manual cleanup.
|
||||
func createTokenSecret(prefix, namespace, token string) (string, error) {
|
||||
secret := corev1.Secret {
|
||||
TypeMeta: metav1.TypeMeta {
|
||||
Kind: "Secret",
|
||||
APIVersion: "v1",
|
||||
},
|
||||
ObjectMeta: metav1.ObjectMeta {
|
||||
GenerateName: prefix,
|
||||
Namespace: namespace,
|
||||
Labels: map[string]string {
|
||||
tokenSecretLabel: "true",
|
||||
},
|
||||
},
|
||||
StringData: map[string]string {
|
||||
tokenSecretKey: token,
|
||||
},
|
||||
Type: corev1.SecretTypeOpaque,
|
||||
}
|
||||
|
||||
client, err := NewInClusterK8sClient()
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
|
||||
body, err := json.Marshal(secret)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
log.WithField("secret", string(body)).Debug("Creating secret")
|
||||
|
||||
req, err := client.PostRequest(fmt.Sprintf("api/v1/namespaces/%s/secrets", namespace), string(body), "application/json")
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
|
||||
resp, err := client.Do(req)
|
||||
if err != nil {
|
||||
log.Errorf("Secret creation error. Response: %v", resp)
|
||||
return "", errors.Wrap(err, "secret creation")
|
||||
}
|
||||
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
|
||||
log.Errorf("Secret creation error (!2XX). Response: %v", resp)
|
||||
var rbody []byte
|
||||
if resp.Body != nil {
|
||||
if data, err := ioutil.ReadAll(resp.Body); err == nil {
|
||||
rbody = data
|
||||
}
|
||||
}
|
||||
log.Error("Error body: ", string(rbody))
|
||||
return "", errors.New("Not 200")
|
||||
}
|
||||
|
||||
var rbody []byte
|
||||
if resp.Body != nil {
|
||||
if data, err := ioutil.ReadAll(resp.Body); err == nil {
|
||||
rbody = data
|
||||
}
|
||||
}
|
||||
if len(rbody) == 0 {
|
||||
return "", errors.New("Empty response body")
|
||||
}
|
||||
|
||||
var created *corev1.Secret
|
||||
if err := json.Unmarshal(rbody, &created); err != nil {
|
||||
return "", errors.Wrap(err, "Error unmarshalling secret response")
|
||||
}
|
||||
|
||||
// Clean up after ourselves by deleting the Secret after the bootstrap
|
||||
// token expires. This is best effort -- obviously we'll miss some stuff
|
||||
// if this process goes away -- but the secrets are also labelled so
|
||||
// it's also easy to clean them up in bulk using kubectl if we miss any.
|
||||
go func() {
|
||||
time.Sleep(tokenLifetime)
|
||||
req, err := client.DeleteRequest(fmt.Sprintf("api/v1/namespaces/%s/secrets/%s", namespace, created.Name))
|
||||
ctxLog := log.WithFields(log.Fields{
|
||||
"name": created.Name,
|
||||
"namespace": namespace,
|
||||
})
|
||||
if err != nil {
|
||||
ctxLog.WithField("error", err).Error("Error deleting expired boostrap token secret")
|
||||
return
|
||||
}
|
||||
resp, err := client.Do(req)
|
||||
if err != nil {
|
||||
ctxLog.WithField("error", err).Error("Error deleting expired boostrap token secret")
|
||||
return
|
||||
}
|
||||
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
|
||||
ctxLog.WithFields(log.Fields{
|
||||
"status": resp.Status,
|
||||
"statusCode": resp.StatusCode,
|
||||
}).Error("Error deleting expired boostrap token secret")
|
||||
return
|
||||
}
|
||||
ctxLog.Info("Deleted expired bootstrap token secret")
|
||||
}()
|
||||
|
||||
return created.Name, err
|
||||
}
|
||||
|
||||
// mkBootstrapper generates a bootstrap container based on the template defined in Config. It
|
||||
// generates a new bootstrap token and mounts it, along with other required coniguration, as
|
||||
// environment variables in the returned bootstrap container.
|
||||
func mkBootstrapper(config *Config, commonName string, namespace string, provisioner Provisioner) (corev1.Container, error) {
|
||||
b := config.Bootstrapper
|
||||
|
||||
token, err := provisioner.Token(commonName)
|
||||
if err != nil {
|
||||
return b, errors.Wrap(err, "token generation")
|
||||
}
|
||||
|
||||
// Generate CA fingerprint
|
||||
crt, err := pemutil.ReadCertificate(rootCAPath)
|
||||
if err != nil {
|
||||
return b, errors.Wrap(err, "CA fingerprint")
|
||||
}
|
||||
sum := sha256.Sum256(crt.Raw)
|
||||
fingerprint := strings.ToLower(hex.EncodeToString(sum[:]))
|
||||
|
||||
secretName, err := createTokenSecret(commonName + "-", namespace, token)
|
||||
if err != nil {
|
||||
return b, errors.Wrap(err, "create token secret")
|
||||
}
|
||||
log.Infof("Secret name is: %s", secretName)
|
||||
|
||||
b.Env = append(b.Env, corev1.EnvVar {
|
||||
Name: "COMMON_NAME",
|
||||
Value: commonName,
|
||||
})
|
||||
b.Env = append(b.Env, corev1.EnvVar {
|
||||
Name: "STEP_TOKEN",
|
||||
ValueFrom: &corev1.EnvVarSource {
|
||||
SecretKeyRef: &corev1.SecretKeySelector {
|
||||
LocalObjectReference: corev1.LocalObjectReference {
|
||||
Name: secretName,
|
||||
},
|
||||
Key: tokenSecretKey,
|
||||
},
|
||||
},
|
||||
})
|
||||
b.Env = append(b.Env, corev1.EnvVar {
|
||||
Name: "STEP_CA_URL",
|
||||
Value: config.CaUrl,
|
||||
})
|
||||
b.Env = append(b.Env, corev1.EnvVar {
|
||||
Name: "STEP_FINGERPRINT",
|
||||
Value: fingerprint,
|
||||
})
|
||||
b.Env = append(b.Env, corev1.EnvVar {
|
||||
Name: "STEP_NOT_AFTER",
|
||||
Value: config.CertLifetime,
|
||||
})
|
||||
return b, nil
|
||||
}
|
||||
|
||||
// mkRenewer generates a new renewer based on the template provided in Config.
|
||||
func mkRenewer(config *Config) (corev1.Container) {
|
||||
r := config.Renewer
|
||||
r.Env = append(r.Env, corev1.EnvVar {
|
||||
Name: "STEP_CA_URL",
|
||||
Value: config.CaUrl,
|
||||
})
|
||||
return r
|
||||
}
|
||||
|
||||
func addContainers(existing, new []corev1.Container, path string) (ops []PatchOperation) {
|
||||
if len(existing) == 0 {
|
||||
return []PatchOperation {
|
||||
PatchOperation {
|
||||
Op: "add",
|
||||
Path: path,
|
||||
Value: new,
|
||||
},
|
||||
}
|
||||
} else {
|
||||
for _, add := range new {
|
||||
ops = append(ops, PatchOperation {
|
||||
Op: "add",
|
||||
Path: path + "/-",
|
||||
Value: add,
|
||||
})
|
||||
}
|
||||
return ops
|
||||
}
|
||||
}
|
||||
|
||||
func addVolumes(existing, new []corev1.Volume, path string) (ops []PatchOperation) {
|
||||
if len(existing) == 0 {
|
||||
return []PatchOperation{
|
||||
PatchOperation {
|
||||
Op: "add",
|
||||
Path: path,
|
||||
Value: new,
|
||||
},
|
||||
}
|
||||
} else {
|
||||
for _, add := range new {
|
||||
ops = append(ops, PatchOperation {
|
||||
Op: "add",
|
||||
Path: path + "/-",
|
||||
Value: add,
|
||||
})
|
||||
}
|
||||
return ops
|
||||
}
|
||||
}
|
||||
|
||||
func addCertsVolumeMount(volumeName string, containers []corev1.Container) (ops []PatchOperation) {
|
||||
volumeMount := corev1.VolumeMount {
|
||||
Name: volumeName,
|
||||
MountPath: volumeMountPath,
|
||||
ReadOnly: true,
|
||||
}
|
||||
for i, container := range containers {
|
||||
if len(container.VolumeMounts) == 0 {
|
||||
ops = append(ops, PatchOperation {
|
||||
Op: "add",
|
||||
Path: fmt.Sprintf("/spec/containers/%v/volumeMounts", i),
|
||||
Value: []corev1.VolumeMount{volumeMount},
|
||||
})
|
||||
} else {
|
||||
ops = append(ops, PatchOperation {
|
||||
Op: "add",
|
||||
Path: fmt.Sprintf("/spec/containers/%v/volumeMounts/-", i),
|
||||
Value: volumeMount,
|
||||
})
|
||||
}
|
||||
}
|
||||
return ops
|
||||
}
|
||||
|
||||
func addAnnotations(existing, new map[string]string) (ops []PatchOperation) {
|
||||
if len(existing) == 0 {
|
||||
return []PatchOperation{
|
||||
PatchOperation {
|
||||
Op: "add",
|
||||
Path: "/metadata/annotations",
|
||||
Value: new,
|
||||
},
|
||||
}
|
||||
}
|
||||
for k, v := range new {
|
||||
if existing[k] == "" {
|
||||
ops = append(ops, PatchOperation {
|
||||
Op: "add",
|
||||
Path: "/metadata/annotations/" + escapeJsonPath(k),
|
||||
Value: v,
|
||||
})
|
||||
} else {
|
||||
ops = append(ops, PatchOperation {
|
||||
Op: "replace",
|
||||
Path: "/metadata/annotations/" + escapeJsonPath(k),
|
||||
Value: v,
|
||||
})
|
||||
}
|
||||
}
|
||||
return ops
|
||||
}
|
||||
|
||||
// patch produces a list of patches to apply to a pod to inject a certificate. In particular,
|
||||
// we patch the pod in order to:
|
||||
// - Mount the `certs` volume in existing containers defined in the pod
|
||||
// - Add the autocert-renewer as a container (a sidecar)
|
||||
// - Add the autocert-bootstrapper as an initContainer
|
||||
// - Add the `certs` volume definition
|
||||
// - Annotate the pod to indicate that it's been processed by this controller
|
||||
// The result is a list of serialized JSONPatch objects (or an error).
|
||||
func patch(pod *corev1.Pod, namespace string, config *Config, provisioner Provisioner) ([]byte, error) {
|
||||
var ops[] PatchOperation
|
||||
|
||||
commonName := pod.ObjectMeta.GetAnnotations()[admissionWebhookAnnotationKey]
|
||||
renewer := mkRenewer(config)
|
||||
bootstrapper, err := mkBootstrapper(config, commonName, namespace, provisioner)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
ops = append(ops, addCertsVolumeMount(config.CertsVolume.Name, pod.Spec.Containers)...)
|
||||
ops = append(ops, addContainers(pod.Spec.Containers, []corev1.Container{renewer}, "/spec/containers")...)
|
||||
ops = append(ops, addContainers(pod.Spec.InitContainers, []corev1.Container{bootstrapper}, "/spec/initContainers")...)
|
||||
ops = append(ops, addVolumes(pod.Spec.Volumes, []corev1.Volume{config.CertsVolume}, "/spec/volumes")...)
|
||||
ops = append(ops, addAnnotations(pod.Annotations, map[string]string{admissionWebhookStatusKey: "injected"})...)
|
||||
|
||||
return json.Marshal(ops)
|
||||
}
|
||||
|
||||
// shouldMutate checks whether a pod is subject to mutation by this admission controller. A pod
|
||||
// is subject to mutation if it's annotated with the `admissionWebhookAnnotationKey` and if it
|
||||
// has not already been processed (indicated by `admissionWebhookStatusKey` set to `injected`).
|
||||
func shouldMutate(metadata *metav1.ObjectMeta) bool {
|
||||
annotations := metadata.GetAnnotations()
|
||||
if annotations == nil {
|
||||
annotations = map[string]string{}
|
||||
}
|
||||
|
||||
// Only mutate if the object is annotated appropriately (annotation key set) and we haven't
|
||||
// mutated already (status key isn't set).
|
||||
if annotations[admissionWebhookAnnotationKey] == "" || annotations[admissionWebhookStatusKey] == "injected" {
|
||||
return false
|
||||
} else {
|
||||
return true
|
||||
}
|
||||
}
|
||||
|
||||
// mutate takes an `AdmissionReview`, determines whether it is subject to mutation, and returns
|
||||
// an appropriate `AdmissionResponse` including patches or any errors that occurred.
|
||||
func mutate(review *v1beta1.AdmissionReview, config *Config, provisioner Provisioner) *v1beta1.AdmissionResponse {
|
||||
ctxLog := log.WithField("uid", review.Request.UID)
|
||||
|
||||
request := review.Request
|
||||
var pod corev1.Pod
|
||||
if err := json.Unmarshal(request.Object.Raw, &pod); err != nil {
|
||||
ctxLog.WithField("error", err).Error("Error unmarshalling pod")
|
||||
return &v1beta1.AdmissionResponse {
|
||||
Allowed: false,
|
||||
UID: request.UID,
|
||||
Result: &metav1.Status {
|
||||
Message: err.Error(),
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
ctxLog = ctxLog.WithFields(log.Fields{
|
||||
"kind": request.Kind,
|
||||
"operation": request.Operation,
|
||||
"name": pod.Name,
|
||||
"generateName": pod.GenerateName,
|
||||
"namespace": request.Namespace,
|
||||
"user": request.UserInfo,
|
||||
})
|
||||
|
||||
if !shouldMutate(&pod.ObjectMeta) {
|
||||
ctxLog.WithField("annotations", pod.Annotations).Info("Skipping mutation")
|
||||
return &v1beta1.AdmissionResponse {
|
||||
Allowed: true,
|
||||
UID: request.UID,
|
||||
}
|
||||
}
|
||||
|
||||
patchBytes, err := patch(&pod, request.Namespace, config, provisioner)
|
||||
if err != nil {
|
||||
ctxLog.WithField("error", err).Error("Error generating patch")
|
||||
return &v1beta1.AdmissionResponse {
|
||||
Allowed: false,
|
||||
UID: request.UID,
|
||||
Result: &metav1.Status {
|
||||
Message: err.Error(),
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
ctxLog.WithField("patch", string(patchBytes)).Info("Generated patch")
|
||||
return &v1beta1.AdmissionResponse {
|
||||
Allowed: true,
|
||||
Patch: patchBytes,
|
||||
UID: request.UID,
|
||||
PatchType: func() *v1beta1.PatchType {
|
||||
pt := v1beta1.PatchTypeJSONPatch
|
||||
return &pt
|
||||
}(),
|
||||
}
|
||||
}
|
||||
|
||||
func main() {
|
||||
if len(os.Args) != 2 {
|
||||
log.Errorf("Usage: %s <config>\n", os.Args[0])
|
||||
os.Exit(1)
|
||||
}
|
||||
|
||||
config, err := loadConfig(os.Args[1])
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
|
||||
log.SetOutput(os.Stdout)
|
||||
if config.LogFormat == "json" {
|
||||
log.SetFormatter(&log.JSONFormatter{})
|
||||
}
|
||||
if config.LogFormat == "text" {
|
||||
log.SetFormatter(&log.TextFormatter{})
|
||||
}
|
||||
|
||||
log.WithFields(log.Fields{
|
||||
"config": config,
|
||||
}).Info("Loaded config")
|
||||
|
||||
provisionerName := os.Getenv("PROVISIONER_NAME")
|
||||
provisionerKid := os.Getenv("PROVISIONER_KID")
|
||||
log.WithFields(log.Fields{
|
||||
"provisionerName": provisionerName,
|
||||
"provisionerKid": provisionerKid,
|
||||
}).Info("Loaded provisioner configuration")
|
||||
|
||||
provisioner, err := NewProvisioner(provisionerName, provisionerKid, config.CaUrl, rootCAPath, provisionerPasswordFile)
|
||||
if err != nil {
|
||||
log.Errorf("Error loading provisioner: %v", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
log.WithFields(log.Fields{
|
||||
"name": provisioner.Name(),
|
||||
"kid": provisioner.Kid(),
|
||||
}).Info("Loaded provisioner")
|
||||
|
||||
namespace := os.Getenv("NAMESPACE")
|
||||
if namespace == "" {
|
||||
log.Errorf("$NAMESPACE not set")
|
||||
os.Exit(1)
|
||||
}
|
||||
|
||||
name := fmt.Sprintf("autocert.%s.svc", namespace)
|
||||
token, err := provisioner.Token(name)
|
||||
if err != nil {
|
||||
log.WithField("error", err).Errorf("Error generating bootstrap token during controller startup")
|
||||
os.Exit(1)
|
||||
}
|
||||
log.WithField("name", name).Infof("Generated bootstrap token for controller")
|
||||
|
||||
// make sure to cancel the renew goroutine
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
defer cancel()
|
||||
|
||||
srv, err := ca.BootstrapServer(ctx, token, &http.Server{
|
||||
Addr: ":4443",
|
||||
Handler: http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
if r.URL.Path == "/healthz" {
|
||||
log.Info("/healthz")
|
||||
fmt.Fprintf(w, "ok")
|
||||
w.WriteHeader(http.StatusOK)
|
||||
return
|
||||
}
|
||||
|
||||
/*
|
||||
var name string
|
||||
if r.TLS != nil && len(r.TLS.PeerCertificates) > 0 {
|
||||
name = r.TLS.PeerCertificates[0].Subject.CommonName
|
||||
}
|
||||
*/
|
||||
|
||||
if r.URL.Path != "/mutate" {
|
||||
log.WithField("path", r.URL.Path).Error("Bad Request: 404 Not Found")
|
||||
http.NotFound(w, r)
|
||||
return
|
||||
}
|
||||
|
||||
var body []byte
|
||||
if r.Body != nil {
|
||||
if data, err := ioutil.ReadAll(r.Body); err == nil {
|
||||
body = data
|
||||
}
|
||||
}
|
||||
if len(body) == 0 {
|
||||
log.Error("Bad Request: 400 (Empty Body)")
|
||||
http.Error(w, "Bad Request (Empty Body)", http.StatusBadRequest)
|
||||
return
|
||||
}
|
||||
|
||||
contentType := r.Header.Get("Content-Type")
|
||||
if contentType != "application/json" {
|
||||
log.WithField("Content-Type", contentType).Error("Bad Request: 415 (Unsupported Media Type)")
|
||||
http.Error(w, fmt.Sprintf("Bad Request: 415 Unsupported Media Type (Expected Content-Type 'application/json' but got '%s')", contentType), http.StatusUnsupportedMediaType)
|
||||
return
|
||||
}
|
||||
|
||||
var response *v1beta1.AdmissionResponse
|
||||
review := v1beta1.AdmissionReview{}
|
||||
if _, _, err := deserializer.Decode(body, nil, &review); err != nil {
|
||||
log.WithFields(log.Fields{
|
||||
"body": body,
|
||||
"error": err,
|
||||
}).Error("Can't decode body")
|
||||
response = &v1beta1.AdmissionResponse {
|
||||
Allowed: false,
|
||||
Result: &metav1.Status {
|
||||
Message: err.Error(),
|
||||
},
|
||||
}
|
||||
} else {
|
||||
response = mutate(&review, config, provisioner)
|
||||
}
|
||||
|
||||
resp, err := json.Marshal(v1beta1.AdmissionReview {
|
||||
Response: response,
|
||||
})
|
||||
if err != nil {
|
||||
log.WithFields(log.Fields{
|
||||
"uid": review.Request.UID,
|
||||
"error": err,
|
||||
}).Info("Marshal error")
|
||||
http.Error(w, fmt.Sprintf("Marshal Error: %v", err), http.StatusInternalServerError)
|
||||
} else {
|
||||
log.WithFields(log.Fields{
|
||||
"uid": review.Request.UID,
|
||||
"response": string(resp),
|
||||
}).Info("Returning review")
|
||||
if _, err := w.Write(resp); err != nil {
|
||||
log.WithFields(log.Fields{
|
||||
"uid": review.Request.UID,
|
||||
"error": err,
|
||||
}).Info("Write error")
|
||||
}
|
||||
}
|
||||
}),
|
||||
}, ca.VerifyClientCertIfGiven())
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
|
||||
log.Info("Listening on :4443 ...")
|
||||
if err := srv.ListenAndServeTLS("", ""); err != nil {
|
||||
panic(err)
|
||||
}
|
||||
}
|
||||
143
autocert/controller/provisioner.go
Normal file
143
autocert/controller/provisioner.go
Normal file
|
|
@ -0,0 +1,143 @@
|
|||
package main
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"time"
|
||||
|
||||
"github.com/pkg/errors"
|
||||
"github.com/smallstep/cli/crypto/pki"
|
||||
"github.com/smallstep/cli/crypto/randutil"
|
||||
"github.com/smallstep/cli/jose"
|
||||
"github.com/smallstep/cli/token"
|
||||
"github.com/smallstep/cli/token/provision"
|
||||
)
|
||||
|
||||
const (
|
||||
tokenLifetime = 5 * time.Minute
|
||||
)
|
||||
|
||||
type Provisioner interface {
|
||||
Name() string
|
||||
Kid() string
|
||||
Token(subject string) (string, error)
|
||||
}
|
||||
|
||||
type provisioner struct {
|
||||
name string
|
||||
kid string
|
||||
caUrl string
|
||||
caRoot string
|
||||
jwk *jose.JSONWebKey
|
||||
tokenLifetime time.Duration
|
||||
}
|
||||
|
||||
// Name returns the provisioner's name.
|
||||
func (p *provisioner) Name() string {
|
||||
return p.name
|
||||
}
|
||||
|
||||
// Kid returns the provisioners key ID.
|
||||
func (p *provisioner) Kid() string {
|
||||
return p.kid
|
||||
}
|
||||
|
||||
// Token generates a bootstrap token for a subject.
|
||||
func (p *provisioner) Token(subject string) (string, error) {
|
||||
// A random jwt id will be used to identify duplicated tokens
|
||||
jwtID, err := randutil.Hex(64) // 256 bits
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
|
||||
notBefore := time.Now()
|
||||
notAfter := notBefore.Add(tokenLifetime)
|
||||
signUrl := fmt.Sprintf("%v/1.0/sign", p.caUrl)
|
||||
|
||||
tokOptions := []token.Options{
|
||||
token.WithJWTID(jwtID),
|
||||
token.WithKid(p.kid),
|
||||
token.WithIssuer(p.name),
|
||||
token.WithAudience(signUrl),
|
||||
token.WithValidity(notBefore, notAfter),
|
||||
token.WithRootCA(p.caRoot),
|
||||
}
|
||||
|
||||
tok, err := provision.New(subject, tokOptions...)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
|
||||
return tok.SignedString(p.jwk.Algorithm, p.jwk.Key)
|
||||
}
|
||||
|
||||
func decryptProvisionerJWK(encryptedKey, passFile string) (*jose.JSONWebKey, error) {
|
||||
decrypted, err := jose.Decrypt("", []byte(encryptedKey), jose.WithPasswordFile(passFile))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
jwk := new(jose.JSONWebKey)
|
||||
if err := json.Unmarshal(decrypted, jwk); err != nil {
|
||||
return nil, errors.Wrap(err, "error unmarshalling provisioning key")
|
||||
}
|
||||
return jwk, nil
|
||||
}
|
||||
|
||||
// loadProvisionerJWKByKid retrieves a provisioner key from the CA by key ID and
|
||||
// decrypts it using the specified password file.
|
||||
func loadProvisionerJWKByKid(kid, caUrl, caRoot, passFile string) (*jose.JSONWebKey, error) {
|
||||
encrypted, err := pki.GetProvisionerKey(caUrl, caRoot, kid)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return decryptProvisionerJWK(encrypted, passFile)
|
||||
}
|
||||
|
||||
// loadProvisionerJWKByName retrieves the list of provisioners and encrypted key then
|
||||
// returns the key of the first provisioner with a matching name that can be successfully
|
||||
// decrypted with the specified password file.
|
||||
func loadProvisionerJWKByName(name, caUrl, caRoot, passFile string) (key *jose.JSONWebKey, err error) {
|
||||
provisioners, err := pki.GetProvisioners(caUrl, caRoot)
|
||||
if err != nil {
|
||||
err = errors.Wrap(err, "error getting the provisioners")
|
||||
return
|
||||
}
|
||||
|
||||
for _, provisioner := range provisioners {
|
||||
if provisioner.Name == name {
|
||||
key, err = decryptProvisionerJWK(provisioner.EncryptedKey, passFile)
|
||||
if err == nil {
|
||||
return
|
||||
}
|
||||
}
|
||||
}
|
||||
return nil, errors.New(fmt.Sprintf("provisioner '%s' not found (or your password is wrong)", name))
|
||||
}
|
||||
|
||||
// NewProvisioner loads and decrypts key material from the CA for the named
|
||||
// provisioner. The key identified by `kid` will be used if specified. If `kid`
|
||||
// is the empty string we'll use the first key for the named provisioner that
|
||||
// decrypts using `passFile`.
|
||||
func NewProvisioner(name, kid, caUrl, caRoot, passFile string) (Provisioner, error) {
|
||||
var jwk *jose.JSONWebKey
|
||||
var err error
|
||||
if kid != "" {
|
||||
jwk, err = loadProvisionerJWKByKid(kid, caUrl, caRoot, passFile)
|
||||
} else {
|
||||
jwk, err = loadProvisionerJWKByName(name, caUrl, caRoot, passFile)
|
||||
}
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return &provisioner{
|
||||
name: name,
|
||||
kid: jwk.KeyID,
|
||||
caUrl: caUrl,
|
||||
caRoot: caRoot,
|
||||
jwk: jwk,
|
||||
tokenLifetime: tokenLifetime,
|
||||
}, nil
|
||||
}
|
||||
BIN
autocert/demo.gif
Normal file
BIN
autocert/demo.gif
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 10 MiB |
103
autocert/examples/hello-mtls/README.md
Normal file
103
autocert/examples/hello-mtls/README.md
Normal file
|
|
@ -0,0 +1,103 @@
|
|||
# hello-mtls
|
||||
|
||||
This repository contains examples of dockerized [m]TLS clients and servers in
|
||||
various languages. There's a lot of confusion and misinformation regarding how
|
||||
to do mTLS properly with an internal public key infrastructure. The goal of
|
||||
this repository is to demonstrate best practices like:
|
||||
|
||||
* Properly configuring TLS to use your internal CA's root certificate
|
||||
* mTLS (client certificates / client authentication)
|
||||
* Short-lived certificate support (clients and servers automatically load
|
||||
renewed certificates)
|
||||
|
||||
Examples use multi-stage docker builds and can be built via without any
|
||||
required local dependencies (except `docker`):
|
||||
|
||||
```
|
||||
docker build -f Dockerfile.server -t hello-mtls-server-<lang> .
|
||||
docker build -f Dockerfile.client -t hello-mtls-client-<lang> .
|
||||
```
|
||||
|
||||
Once built, you should be able to deploy via:
|
||||
|
||||
```
|
||||
kubectl apply -f hello-mtls.server.yaml
|
||||
kubectl apply -f hello-mtls.client.yaml
|
||||
```
|
||||
|
||||
## Mutual TLS
|
||||
|
||||
Unlike the _server auth TLS_ that's typical with web browsers, where the browser authenticates the server but not vice versa, _mutual TLS_ (mTLS) connections have both remote peers (client and server) authenticate to one another by presenting certificates. mTLS is not a different protocol. It's just a variant of TLS that's not usually turned on by default. This respository demonstrates **how to turn on mTLS** with different tools and languages. It also demonstrates other **TLS best practices** like certificate rotation.
|
||||
|
||||
mTLS provides _authenticated encryption_: an _identity dialtone_ and _end-to-end encryption_ for your workloads. It's like a secure line with caller ID. This has [all sorts of benefits](https://smallstep.com/blog/use-tls.html): better security, compliance, and easier auditability for starters. It **makes workloads identity-aware**, improving observability and enabling granular access control. Perhaps most compelling, mTLS lets you securely communicate with workloads running anywhere. Code, containers, devices, people, and anything else can connect securely using mTLS as long as they know one anothers' names and can resolve those names to routable IP addresses.
|
||||
|
||||
With properly configured mTLS, services can be safely exposed directly to the public internet: **only clients that have a certificate issued by the internal certificate authority will be allowed to connect**.
|
||||
|
||||
Here's a rough approximation of how an mTLS handshake works:
|
||||
|
||||
|
||||
|
||||
A few things to note:
|
||||
|
||||
* It's the signing of random numbers that proves we're talking to the right remote. It's the digital equivalent of asking someone to send you a photo of them with today's newspaper.
|
||||
* The client and server need to have prior knowledge of the root certificate(s) used for signing other certificates.
|
||||
* The client and server need to be configured to use the correct certificate and private key (the certificate must have been issued by a CA with a trusted root certificate)
|
||||
* Private keys are never shared. This is the magic of public key cryptography: unlike passwords or access tokens, certificates let you prove who you are without giving anyone the ability to impersonate you.
|
||||
|
||||
## Feature matrix
|
||||
|
||||
This matrix shows the set of features we'd like to demonstrate in each language
|
||||
and where each language is. Bug fixes, improvements, and examples in new
|
||||
languages are appreciated!
|
||||
|
||||
[go/](go/)
|
||||
- [X] Server using autocert certificate & key
|
||||
- [X] mTLS (client authentication using internal root certificate)
|
||||
- [X] Automatic certificate renewal
|
||||
- [X] Restrict to safe ciphersuites and TLS versions
|
||||
- [ ] TLS stack configuration loaded from `step-ca`
|
||||
- [ ] Root certificate rotation
|
||||
- [X] Client using autocert root certificate
|
||||
- [X] mTLS (send client certificate if server asks for it)
|
||||
- [ ] Automatic certificate rotation
|
||||
- [X] Restrict to safe ciphersuites and TLS versions
|
||||
- [ ] TLS stack configuration loaded from `step-ca`
|
||||
- [ ] Root certificate rotation
|
||||
|
||||
[curl/](curl/)
|
||||
- [X] Client
|
||||
- [X] mTLS (send client certificate if server asks for it)
|
||||
- [X] Automatic certificate rotation
|
||||
- [ ] Restrict to safe ciphersuites and TLS versions
|
||||
- [ ] TLS stack configuration loaded from `step-ca`
|
||||
- [ ] Root certificate rotation
|
||||
|
||||
[nginx/](nginx/)
|
||||
- [X] Server
|
||||
- [X] mTLS (client authentication using internal root certificate)
|
||||
- [X] Automatic certificate renewal
|
||||
- [X] Restrict to safe ciphersuites and TLS versions
|
||||
- [ ] TLS stack configuration loaded from `step-ca`
|
||||
- [ ] Root certificate rotation
|
||||
|
||||
[node/](node/)
|
||||
- [X] Server
|
||||
- [X] mTLS (client authentication using internal root certificate)
|
||||
- [X] Automatic certificate renewal
|
||||
- [X] Restrict to safe ciphersuites and TLS versions
|
||||
- [ ] TLS stack configuration loaded from `step-ca`
|
||||
- [ ] Root certificate rotation
|
||||
- [X] Client using autocert root certificate
|
||||
- [X] mTLS (send client certificate if server asks for it)
|
||||
- [X] Automatic certificate rotation
|
||||
- [X] Restrict to safe ciphersuites and TLS versions
|
||||
- [ ] TLS stack configuration loaded from `step-ca`
|
||||
- [ ] Root certificate rotation
|
||||
|
||||
[envoy/](envoy/)
|
||||
- [X] Server
|
||||
- [X] mTLS (client authentication using internal root certificate)
|
||||
- [X] Automatic certificate renewal
|
||||
- [X] Restrict to safe ciphersuites and TLS versions
|
||||
- [ ] TLS stack configuration loaded from `step-ca`
|
||||
- [ ] Root certificate rotation
|
||||
5
autocert/examples/hello-mtls/curl/Dockerfile.client
Normal file
5
autocert/examples/hello-mtls/curl/Dockerfile.client
Normal file
|
|
@ -0,0 +1,5 @@
|
|||
FROM alpine
|
||||
RUN apk add --no-cache bash curl
|
||||
COPY client.sh .
|
||||
RUN chmod +x client.sh
|
||||
ENTRYPOINT ./client.sh
|
||||
11
autocert/examples/hello-mtls/curl/client.sh
Normal file
11
autocert/examples/hello-mtls/curl/client.sh
Normal file
|
|
@ -0,0 +1,11 @@
|
|||
#!/bin/bash
|
||||
while :
|
||||
do
|
||||
response=$(curl -sS \
|
||||
--cacert /var/run/autocert.step.sm/root.crt \
|
||||
--cert /var/run/autocert.step.sm/site.crt \
|
||||
--key /var/run/autocert.step.sm/site.key \
|
||||
${HELLO_MTLS_URL})
|
||||
echo "$(date): ${response}"
|
||||
sleep 5
|
||||
done
|
||||
22
autocert/examples/hello-mtls/curl/hello-mtls.client.yaml
Normal file
22
autocert/examples/hello-mtls/curl/hello-mtls.client.yaml
Normal file
|
|
@ -0,0 +1,22 @@
|
|||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: hello-mtls-client
|
||||
labels: {app: hello-mtls-client}
|
||||
spec:
|
||||
replicas: 1
|
||||
selector: {matchLabels: {app: hello-mtls-client}}
|
||||
template:
|
||||
metadata:
|
||||
annotations:
|
||||
autocert.step.sm/name: hello-mtls-client.default.pod.cluster.local
|
||||
labels: {app: hello-mtls-client}
|
||||
spec:
|
||||
containers:
|
||||
- name: hello-mtls-client
|
||||
image: hello-mtls-client-curl:latest
|
||||
imagePullPolicy: Never
|
||||
resources: {requests: {cpu: 10m, memory: 20Mi}}
|
||||
env:
|
||||
- name: HELLO_MTLS_URL
|
||||
value: https://hello-mtls.default.svc.cluster.local
|
||||
21
autocert/examples/hello-mtls/envoy/Dockerfile.server
Normal file
21
autocert/examples/hello-mtls/envoy/Dockerfile.server
Normal file
|
|
@ -0,0 +1,21 @@
|
|||
FROM envoyproxy/envoy-alpine
|
||||
|
||||
RUN apk update
|
||||
RUN apk add python3
|
||||
RUN apk add inotify-tools
|
||||
RUN mkdir /src
|
||||
|
||||
ADD entrypoint.sh /src
|
||||
ADD certwatch.sh /src
|
||||
ADD hot-restarter.py /src
|
||||
ADD start-envoy.sh /src
|
||||
ADD server.yaml /src
|
||||
|
||||
# Flask app
|
||||
ADD server.py /src
|
||||
ADD requirements.txt /src
|
||||
RUN pip3 install -r /src/requirements.txt
|
||||
|
||||
# app, certificate watcher and envoy
|
||||
ENTRYPOINT ["/src/entrypoint.sh"]
|
||||
CMD ["python3", "/src/hot-restarter.py", "/src/start-envoy.sh"]
|
||||
6
autocert/examples/hello-mtls/envoy/certwatch.sh
Executable file
6
autocert/examples/hello-mtls/envoy/certwatch.sh
Executable file
|
|
@ -0,0 +1,6 @@
|
|||
#!/bin/sh
|
||||
|
||||
while true; do
|
||||
inotifywait -e modify /var/run/autocert.step.sm/site.crt
|
||||
kill -HUP 1
|
||||
done
|
||||
10
autocert/examples/hello-mtls/envoy/entrypoint.sh
Executable file
10
autocert/examples/hello-mtls/envoy/entrypoint.sh
Executable file
|
|
@ -0,0 +1,10 @@
|
|||
#!/bin/sh
|
||||
|
||||
# start hello world app
|
||||
python3 /src/server.py &
|
||||
|
||||
# watch for the update of the cert and reload nginx
|
||||
/src/certwatch.sh &
|
||||
|
||||
# Run docker CMD
|
||||
exec "$@"
|
||||
33
autocert/examples/hello-mtls/envoy/hello-mtls.server.yaml
Normal file
33
autocert/examples/hello-mtls/envoy/hello-mtls.server.yaml
Normal file
|
|
@ -0,0 +1,33 @@
|
|||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
labels: {app: hello-mtls}
|
||||
name: hello-mtls
|
||||
spec:
|
||||
type: ClusterIP
|
||||
ports:
|
||||
- port: 443
|
||||
targetPort: 443
|
||||
selector: {app: hello-mtls}
|
||||
|
||||
---
|
||||
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: hello-mtls
|
||||
labels: {app: hello-mtls}
|
||||
spec:
|
||||
replicas: 1
|
||||
selector: {matchLabels: {app: hello-mtls}}
|
||||
template:
|
||||
metadata:
|
||||
annotations:
|
||||
autocert.step.sm/name: hello-mtls.default.svc.cluster.local
|
||||
labels: {app: hello-mtls}
|
||||
spec:
|
||||
containers:
|
||||
- name: hello-mtls
|
||||
image: hello-mtls-server-envoy:latest
|
||||
imagePullPolicy: Never
|
||||
resources: {requests: {cpu: 10m, memory: 20Mi}}
|
||||
209
autocert/examples/hello-mtls/envoy/hot-restarter.py
Normal file
209
autocert/examples/hello-mtls/envoy/hot-restarter.py
Normal file
|
|
@ -0,0 +1,209 @@
|
|||
#!/usr/bin/env python
|
||||
from __future__ import print_function
|
||||
|
||||
import os
|
||||
import signal
|
||||
import sys
|
||||
import time
|
||||
|
||||
# The number of seconds to wait for children to gracefully exit after
|
||||
# propagating SIGTERM before force killing children.
|
||||
# NOTE: If using a shutdown mechanism such as runit's `force-stop` which sends
|
||||
# a KILL after a specified timeout period, it's important to ensure that this
|
||||
# constant is smaller than the KILL timeout
|
||||
TERM_WAIT_SECONDS = 30
|
||||
|
||||
restart_epoch = 0
|
||||
pid_list = []
|
||||
|
||||
|
||||
def term_all_children():
|
||||
""" Iterate through all known child processes, send a TERM signal to each of
|
||||
them, and then wait up to TERM_WAIT_SECONDS for them to exit gracefully,
|
||||
exiting early if all children go away. If one or more children have not
|
||||
exited after TERM_WAIT_SECONDS, they will be forcibly killed """
|
||||
|
||||
# First uninstall the SIGCHLD handler so that we don't get called again.
|
||||
signal.signal(signal.SIGCHLD, signal.SIG_DFL)
|
||||
|
||||
global pid_list
|
||||
for pid in pid_list:
|
||||
print("sending TERM to PID={}".format(pid))
|
||||
try:
|
||||
os.kill(pid, signal.SIGTERM)
|
||||
except OSError:
|
||||
print("error sending TERM to PID={} continuing".format(pid))
|
||||
|
||||
all_exited = False
|
||||
|
||||
# wait for TERM_WAIT_SECONDS seconds for children to exit cleanly
|
||||
retries = 0
|
||||
while not all_exited and retries < TERM_WAIT_SECONDS:
|
||||
for pid in list(pid_list):
|
||||
ret_pid, exit_status = os.waitpid(pid, os.WNOHANG)
|
||||
if ret_pid == 0 and exit_status == 0:
|
||||
# the child is still running
|
||||
continue
|
||||
|
||||
pid_list.remove(pid)
|
||||
|
||||
if len(pid_list) == 0:
|
||||
all_exited = True
|
||||
else:
|
||||
retries += 1
|
||||
time.sleep(1)
|
||||
|
||||
if all_exited:
|
||||
print("all children exited cleanly")
|
||||
else:
|
||||
for pid in pid_list:
|
||||
print("child PID={} did not exit cleanly, killing".format(pid))
|
||||
force_kill_all_children()
|
||||
sys.exit(1) # error status because a child did not exit cleanly
|
||||
|
||||
|
||||
def force_kill_all_children():
|
||||
""" Iterate through all known child processes and force kill them. Typically
|
||||
term_all_children() should be attempted first to give child processes an
|
||||
opportunity to clean up state before exiting """
|
||||
|
||||
global pid_list
|
||||
for pid in pid_list:
|
||||
print("force killing PID={}".format(pid))
|
||||
try:
|
||||
os.kill(pid, signal.SIGKILL)
|
||||
except OSError:
|
||||
print("error force killing PID={} continuing".format(pid))
|
||||
|
||||
pid_list = []
|
||||
|
||||
|
||||
def shutdown():
|
||||
""" Attempt to gracefully shutdown all child Envoy processes and then exit.
|
||||
See term_all_children() for further discussion. """
|
||||
term_all_children()
|
||||
sys.exit(0)
|
||||
|
||||
|
||||
def sigterm_handler(signum, frame):
|
||||
""" Handler for SIGTERM. """
|
||||
print("got SIGTERM")
|
||||
shutdown()
|
||||
|
||||
|
||||
def sigint_handler(signum, frame):
|
||||
""" Handler for SIGINT (ctrl-c). The same as the SIGTERM handler. """
|
||||
print("got SIGINT")
|
||||
shutdown()
|
||||
|
||||
|
||||
def sighup_handler(signum, frame):
|
||||
""" Handler for SIGUP. This signal is used to cause the restarter to fork and exec a new
|
||||
child. """
|
||||
|
||||
print("got SIGHUP")
|
||||
fork_and_exec()
|
||||
|
||||
|
||||
def sigusr1_handler(signum, frame):
|
||||
""" Handler for SIGUSR1. Propagate SIGUSR1 to all of the child processes """
|
||||
|
||||
global pid_list
|
||||
for pid in pid_list:
|
||||
print("sending SIGUSR1 to PID={}".format(pid))
|
||||
try:
|
||||
os.kill(pid, signal.SIGUSR1)
|
||||
except OSError:
|
||||
print("error in SIGUSR1 to PID={} continuing".format(pid))
|
||||
|
||||
|
||||
def sigchld_handler(signum, frame):
|
||||
""" Handler for SIGCHLD. Iterates through all of our known child processes and figures out whether
|
||||
the signal/exit was expected or not. Python doesn't have any of the native signal handlers
|
||||
ability to get the child process info directly from the signal handler so we need to iterate
|
||||
through all child processes and see what happened."""
|
||||
|
||||
print("got SIGCHLD")
|
||||
|
||||
kill_all_and_exit = False
|
||||
global pid_list
|
||||
pid_list_copy = list(pid_list)
|
||||
for pid in pid_list_copy:
|
||||
ret_pid, exit_status = os.waitpid(pid, os.WNOHANG)
|
||||
if ret_pid == 0 and exit_status == 0:
|
||||
# This child is still running.
|
||||
continue
|
||||
|
||||
pid_list.remove(pid)
|
||||
|
||||
# Now we see how the child exited.
|
||||
if os.WIFEXITED(exit_status):
|
||||
exit_code = os.WEXITSTATUS(exit_status)
|
||||
print("PID={} exited with code={}".format(ret_pid, exit_code))
|
||||
if exit_code == 0:
|
||||
# Normal exit. We assume this was on purpose.
|
||||
pass
|
||||
else:
|
||||
# Something bad happened. We need to tear everything down so that whoever started the
|
||||
# restarter can know about this situation and restart the whole thing.
|
||||
kill_all_and_exit = True
|
||||
elif os.WIFSIGNALED(exit_status):
|
||||
print("PID={} was killed with signal={}".format(ret_pid, os.WTERMSIG(exit_status)))
|
||||
kill_all_and_exit = True
|
||||
else:
|
||||
kill_all_and_exit = True
|
||||
|
||||
if kill_all_and_exit:
|
||||
print("Due to abnormal exit, force killing all child processes and exiting")
|
||||
|
||||
# First uninstall the SIGCHLD handler so that we don't get called again.
|
||||
signal.signal(signal.SIGCHLD, signal.SIG_DFL)
|
||||
|
||||
force_kill_all_children()
|
||||
|
||||
# Our last child died, so we have no purpose. Exit.
|
||||
if not pid_list:
|
||||
print("exiting due to lack of child processes")
|
||||
sys.exit(1 if kill_all_and_exit else 0)
|
||||
|
||||
|
||||
def fork_and_exec():
|
||||
""" This routine forks and execs a new child process and keeps track of its PID. Before we fork,
|
||||
set the current restart epoch in an env variable that processes can read if they care. """
|
||||
|
||||
global restart_epoch
|
||||
os.environ['RESTART_EPOCH'] = str(restart_epoch)
|
||||
print("forking and execing new child process at epoch {}".format(restart_epoch))
|
||||
restart_epoch += 1
|
||||
|
||||
child_pid = os.fork()
|
||||
if child_pid == 0:
|
||||
# Child process
|
||||
os.execl(sys.argv[1], sys.argv[1])
|
||||
else:
|
||||
# Parent process
|
||||
print("forked new child process with PID={}".format(child_pid))
|
||||
pid_list.append(child_pid)
|
||||
|
||||
|
||||
def main():
|
||||
""" Script main. This script is designed so that a process watcher like runit or monit can watch
|
||||
this process and take corrective action if it ever goes away. """
|
||||
|
||||
print("starting hot-restarter with target: {}".format(sys.argv[1]))
|
||||
|
||||
signal.signal(signal.SIGTERM, sigterm_handler)
|
||||
signal.signal(signal.SIGINT, sigint_handler)
|
||||
signal.signal(signal.SIGHUP, sighup_handler)
|
||||
signal.signal(signal.SIGCHLD, sigchld_handler)
|
||||
signal.signal(signal.SIGUSR1, sigusr1_handler)
|
||||
|
||||
# Start the first child process and then go into an endless loop since everything else happens via
|
||||
# signals.
|
||||
fork_and_exec()
|
||||
while True:
|
||||
time.sleep(60)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
1
autocert/examples/hello-mtls/envoy/requirements.txt
Normal file
1
autocert/examples/hello-mtls/envoy/requirements.txt
Normal file
|
|
@ -0,0 +1 @@
|
|||
Flask
|
||||
9
autocert/examples/hello-mtls/envoy/server.py
Normal file
9
autocert/examples/hello-mtls/envoy/server.py
Normal file
|
|
@ -0,0 +1,9 @@
|
|||
from flask import Flask
|
||||
app = Flask(__name__)
|
||||
|
||||
@app.route("/")
|
||||
def hello():
|
||||
return "Hello World!\n"
|
||||
|
||||
if __name__ == "__main__":
|
||||
app.run(host='127.0.0.1', port=8080, debug=False)
|
||||
50
autocert/examples/hello-mtls/envoy/server.yaml
Normal file
50
autocert/examples/hello-mtls/envoy/server.yaml
Normal file
|
|
@ -0,0 +1,50 @@
|
|||
static_resources:
|
||||
listeners:
|
||||
- address:
|
||||
socket_address:
|
||||
address: 0.0.0.0
|
||||
port_value: 443
|
||||
filter_chains:
|
||||
- filters:
|
||||
- name: envoy.http_connection_manager
|
||||
config:
|
||||
codec_type: auto
|
||||
stat_prefix: ingress_http
|
||||
route_config:
|
||||
name: hello
|
||||
virtual_hosts:
|
||||
- name: hello
|
||||
domains:
|
||||
- "hello-mtls.default.svc.cluster.local"
|
||||
routes:
|
||||
- match:
|
||||
prefix: "/"
|
||||
route:
|
||||
cluster: hello-mTLS
|
||||
http_filters:
|
||||
- name: envoy.router
|
||||
config: {}
|
||||
tls_context:
|
||||
common_tls_context:
|
||||
tls_params:
|
||||
tls_minimum_protocol_version: TLSv1_2
|
||||
tls_maximum_protocol_version: TLSv1_3
|
||||
cipher_suites: "[ECDHE-ECDSA-AES128-GCM-SHA256|ECDHE-ECDSA-CHACHA20-POLY1305]"
|
||||
tls_certificates:
|
||||
- certificate_chain:
|
||||
filename: "/var/run/autocert.step.sm/site.crt"
|
||||
private_key:
|
||||
filename: "/var/run/autocert.step.sm/site.key"
|
||||
validation_context:
|
||||
trusted_ca:
|
||||
filename: "/var/run/autocert.step.sm/root.crt"
|
||||
require_client_certificate: true
|
||||
clusters:
|
||||
- name: hello-mTLS
|
||||
connect_timeout: 0.25s
|
||||
type: strict_dns
|
||||
lb_policy: round_robin
|
||||
hosts:
|
||||
- socket_address:
|
||||
address: 127.0.0.1
|
||||
port_value: 8080
|
||||
4
autocert/examples/hello-mtls/envoy/start-envoy.sh
Executable file
4
autocert/examples/hello-mtls/envoy/start-envoy.sh
Executable file
|
|
@ -0,0 +1,4 @@
|
|||
#!/bin/sh
|
||||
|
||||
ulimit -n 65536
|
||||
/usr/local/bin/envoy -c /src/server.yaml --service-cluster hello-mTLS --restart-epoch $RESTART_EPOCH
|
||||
10
autocert/examples/hello-mtls/go/Dockerfile.client
Normal file
10
autocert/examples/hello-mtls/go/Dockerfile.client
Normal file
|
|
@ -0,0 +1,10 @@
|
|||
# build stage
|
||||
FROM golang:alpine AS build-env
|
||||
RUN mkdir /src
|
||||
ADD client.go /src
|
||||
RUN cd /src && go build -o client
|
||||
|
||||
# final stage
|
||||
FROM alpine
|
||||
COPY --from=build-env /src/client .
|
||||
ENTRYPOINT ./client
|
||||
10
autocert/examples/hello-mtls/go/Dockerfile.server
Normal file
10
autocert/examples/hello-mtls/go/Dockerfile.server
Normal file
|
|
@ -0,0 +1,10 @@
|
|||
# build stage
|
||||
FROM golang:alpine AS build-env
|
||||
RUN mkdir /src
|
||||
ADD server.go /src
|
||||
RUN cd /src && go build -o server
|
||||
|
||||
# final stage
|
||||
FROM alpine
|
||||
COPY --from=build-env /src/server .
|
||||
ENTRYPOINT ./server
|
||||
85
autocert/examples/hello-mtls/go/client.go
Normal file
85
autocert/examples/hello-mtls/go/client.go
Normal file
|
|
@ -0,0 +1,85 @@
|
|||
package main
|
||||
|
||||
import (
|
||||
"crypto/tls"
|
||||
"crypto/x509"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io/ioutil"
|
||||
"log"
|
||||
"net/http"
|
||||
"os"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
const (
|
||||
autocertFile = "/var/run/autocert.step.sm/site.crt"
|
||||
autocertKey = "/var/run/autocert.step.sm/site.key"
|
||||
autocertRoot = "/var/run/autocert.step.sm/root.crt"
|
||||
requestFrequency = 5 * time.Second
|
||||
)
|
||||
|
||||
func loadRootCertPool() (*x509.CertPool, error) {
|
||||
root, err := ioutil.ReadFile(autocertRoot)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
pool := x509.NewCertPool()
|
||||
if ok := pool.AppendCertsFromPEM(root); !ok {
|
||||
return nil, errors.New("Missing or invalid root certificate")
|
||||
}
|
||||
|
||||
return pool, nil
|
||||
}
|
||||
|
||||
func main() {
|
||||
url := os.Getenv("HELLO_MTLS_URL")
|
||||
|
||||
// Read our leaf certificate and key from disk
|
||||
cert, err := tls.LoadX509KeyPair(autocertFile, autocertKey)
|
||||
if err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
|
||||
// Read the root certificate for our CA from disk
|
||||
roots, err := loadRootCertPool()
|
||||
if err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
|
||||
// Create an HTTPS client using our cert, key & pool
|
||||
client := &http.Client{
|
||||
Transport: &http.Transport{
|
||||
TLSClientConfig: &tls.Config{
|
||||
RootCAs: roots,
|
||||
Certificates: []tls.Certificate{cert},
|
||||
MinVersion: tls.VersionTLS12,
|
||||
CurvePreferences: []tls.CurveID{tls.CurveP521, tls.CurveP384, tls.CurveP256},
|
||||
CipherSuites: []uint16{
|
||||
tls.TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,
|
||||
tls.TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
for {
|
||||
// Make request
|
||||
r, err := client.Get(url)
|
||||
if err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
|
||||
defer r.Body.Close()
|
||||
body, err := ioutil.ReadAll(r.Body)
|
||||
if err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
|
||||
fmt.Printf("%s: %s\n", time.Now().Format(time.RFC3339), strings.Trim(string(body), "\n"))
|
||||
|
||||
time.Sleep(requestFrequency)
|
||||
}
|
||||
}
|
||||
22
autocert/examples/hello-mtls/go/hello-mtls.client.yaml
Normal file
22
autocert/examples/hello-mtls/go/hello-mtls.client.yaml
Normal file
|
|
@ -0,0 +1,22 @@
|
|||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: hello-mtls-client
|
||||
labels: {app: hello-mtls-client}
|
||||
spec:
|
||||
replicas: 1
|
||||
selector: {matchLabels: {app: hello-mtls-client}}
|
||||
template:
|
||||
metadata:
|
||||
annotations:
|
||||
autocert.step.sm/name: hello-mtls-client.default.pod.cluster.local
|
||||
labels: {app: hello-mtls-client}
|
||||
spec:
|
||||
containers:
|
||||
- name: hello-mtls-client
|
||||
image: hello-mtls-client-go:latest
|
||||
imagePullPolicy: Never
|
||||
resources: {requests: {cpu: 10m, memory: 20Mi}}
|
||||
env:
|
||||
- name: HELLO_MTLS_URL
|
||||
value: https://hello-mtls.default.svc.cluster.local
|
||||
33
autocert/examples/hello-mtls/go/hello-mtls.server.yaml
Normal file
33
autocert/examples/hello-mtls/go/hello-mtls.server.yaml
Normal file
|
|
@ -0,0 +1,33 @@
|
|||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
labels: {app: hello-mtls}
|
||||
name: hello-mtls
|
||||
spec:
|
||||
type: ClusterIP
|
||||
ports:
|
||||
- port: 443
|
||||
targetPort: 443
|
||||
selector: {app: hello-mtls}
|
||||
|
||||
---
|
||||
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: hello-mtls
|
||||
labels: {app: hello-mtls}
|
||||
spec:
|
||||
replicas: 1
|
||||
selector: {matchLabels: {app: hello-mtls}}
|
||||
template:
|
||||
metadata:
|
||||
annotations:
|
||||
autocert.step.sm/name: hello-mtls.default.svc.cluster.local
|
||||
labels: {app: hello-mtls}
|
||||
spec:
|
||||
containers:
|
||||
- name: hello-mtls
|
||||
image: hello-mtls-server-go:latest
|
||||
imagePullPolicy: Never
|
||||
resources: {requests: {cpu: 10m, memory: 20Mi}}
|
||||
136
autocert/examples/hello-mtls/go/server.go
Normal file
136
autocert/examples/hello-mtls/go/server.go
Normal file
|
|
@ -0,0 +1,136 @@
|
|||
package main
|
||||
|
||||
import (
|
||||
"crypto/tls"
|
||||
"crypto/x509"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io/ioutil"
|
||||
"log"
|
||||
"net/http"
|
||||
"sync"
|
||||
"time"
|
||||
)
|
||||
|
||||
const (
|
||||
autocertFile = "/var/run/autocert.step.sm/site.crt"
|
||||
autocertKey = "/var/run/autocert.step.sm/site.key"
|
||||
autocertRoot = "/var/run/autocert.step.sm/root.crt"
|
||||
tickFrequency = 15 * time.Second
|
||||
)
|
||||
|
||||
// Uses techniques from https://diogomonica.com/2017/01/11/hitless-tls-certificate-rotation-in-go/
|
||||
// to automatically rotate certificates when they're renewed.
|
||||
|
||||
type rotator struct {
|
||||
sync.Mutex
|
||||
certificate *tls.Certificate
|
||||
}
|
||||
|
||||
func (r *rotator) getCertificate(hello *tls.ClientHelloInfo) (*tls.Certificate, error) {
|
||||
r.Lock()
|
||||
defer r.Unlock()
|
||||
|
||||
return r.certificate, nil
|
||||
}
|
||||
|
||||
func (r *rotator) loadCertificate(certFile, keyFile string) error {
|
||||
r.Lock()
|
||||
defer r.Unlock()
|
||||
|
||||
c, err := tls.LoadX509KeyPair(certFile, keyFile)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
r.certificate = &c
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func loadRootCertPool() (*x509.CertPool, error) {
|
||||
root, err := ioutil.ReadFile(autocertRoot)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
pool := x509.NewCertPool()
|
||||
if ok := pool.AppendCertsFromPEM(root); !ok {
|
||||
return nil, errors.New("Missing or invalid root certificate")
|
||||
}
|
||||
|
||||
return pool, nil
|
||||
}
|
||||
|
||||
func main() {
|
||||
mux := http.NewServeMux()
|
||||
mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
|
||||
if r.TLS == nil || len(r.TLS.PeerCertificates) == 0 {
|
||||
fmt.Fprintf(w, "Unauthenticated")
|
||||
} else {
|
||||
name := r.TLS.PeerCertificates[0].Subject.CommonName
|
||||
fmt.Fprintf(w, "Hello, %s!\n", name)
|
||||
}
|
||||
})
|
||||
mux.HandleFunc("/healthz", func(w http.ResponseWriter, r *http.Request) {
|
||||
fmt.Fprintf(w, "Ok\n")
|
||||
})
|
||||
|
||||
roots, err := loadRootCertPool()
|
||||
if err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
|
||||
r := &rotator{}
|
||||
cfg := &tls.Config{
|
||||
ClientAuth: tls.RequireAndVerifyClientCert,
|
||||
ClientCAs: roots,
|
||||
MinVersion: tls.VersionTLS12,
|
||||
CurvePreferences: []tls.CurveID{tls.CurveP521, tls.CurveP384, tls.CurveP256},
|
||||
PreferServerCipherSuites: true,
|
||||
CipherSuites: []uint16{
|
||||
tls.TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,
|
||||
tls.TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,
|
||||
},
|
||||
GetCertificate: r.getCertificate,
|
||||
}
|
||||
srv := &http.Server{
|
||||
Addr: ":443",
|
||||
Handler: mux,
|
||||
TLSConfig: cfg,
|
||||
}
|
||||
|
||||
// Load certificate
|
||||
err = r.loadCertificate(autocertFile, autocertKey)
|
||||
if err != nil {
|
||||
log.Fatal("Error loading certificate and key", err)
|
||||
}
|
||||
|
||||
// Schedule periodic re-load of certificate
|
||||
done := make(chan struct{})
|
||||
go func() {
|
||||
ticker := time.NewTicker(tickFrequency)
|
||||
defer ticker.Stop()
|
||||
for {
|
||||
select {
|
||||
case <-ticker.C:
|
||||
fmt.Println("Checking for new certificate...")
|
||||
err := r.loadCertificate(autocertFile, autocertKey)
|
||||
if err != nil {
|
||||
log.Println("Error loading certificate and key", err)
|
||||
}
|
||||
case <- done:
|
||||
return
|
||||
}
|
||||
}
|
||||
}()
|
||||
defer close(done)
|
||||
|
||||
log.Println("Listening no :443")
|
||||
|
||||
// Start serving HTTPS
|
||||
err = srv.ListenAndServeTLS("", "")
|
||||
if err != nil {
|
||||
log.Fatal("ListenAndServerTLS: ", err)
|
||||
}
|
||||
}
|
||||
11
autocert/examples/hello-mtls/nginx/Dockerfile.server
Normal file
11
autocert/examples/hello-mtls/nginx/Dockerfile.server
Normal file
|
|
@ -0,0 +1,11 @@
|
|||
FROM nginx:alpine
|
||||
|
||||
RUN apk add inotify-tools
|
||||
RUN mkdir /src
|
||||
ADD site.conf /etc/nginx/conf.d
|
||||
ADD certwatch.sh /src
|
||||
ADD entrypoint.sh /src
|
||||
|
||||
# Certificate watcher and nginx
|
||||
ENTRYPOINT ["/src/entrypoint.sh"]
|
||||
CMD ["nginx", "-g", "daemon off;"]
|
||||
6
autocert/examples/hello-mtls/nginx/certwatch.sh
Executable file
6
autocert/examples/hello-mtls/nginx/certwatch.sh
Executable file
|
|
@ -0,0 +1,6 @@
|
|||
#!/bin/sh
|
||||
|
||||
while true; do
|
||||
inotifywait -e modify /var/run/autocert.step.sm/site.crt
|
||||
nginx -s reload
|
||||
done
|
||||
7
autocert/examples/hello-mtls/nginx/entrypoint.sh
Executable file
7
autocert/examples/hello-mtls/nginx/entrypoint.sh
Executable file
|
|
@ -0,0 +1,7 @@
|
|||
#!/bin/sh
|
||||
|
||||
# watch for the update of the cert and reload nginx
|
||||
/src/certwatch.sh &
|
||||
|
||||
# Run docker CMD
|
||||
exec "$@"
|
||||
33
autocert/examples/hello-mtls/nginx/hello-mtls.server.yaml
Normal file
33
autocert/examples/hello-mtls/nginx/hello-mtls.server.yaml
Normal file
|
|
@ -0,0 +1,33 @@
|
|||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
labels: {app: hello-mtls}
|
||||
name: hello-mtls
|
||||
spec:
|
||||
type: ClusterIP
|
||||
ports:
|
||||
- port: 443
|
||||
targetPort: 443
|
||||
selector: {app: hello-mtls}
|
||||
|
||||
---
|
||||
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: hello-mtls
|
||||
labels: {app: hello-mtls}
|
||||
spec:
|
||||
replicas: 1
|
||||
selector: {matchLabels: {app: hello-mtls}}
|
||||
template:
|
||||
metadata:
|
||||
annotations:
|
||||
autocert.step.sm/name: hello-mtls.default.svc.cluster.local
|
||||
labels: {app: hello-mtls}
|
||||
spec:
|
||||
containers:
|
||||
- name: hello-mtls
|
||||
image: hello-mtls-server-nginx:latest
|
||||
imagePullPolicy: Never
|
||||
resources: {requests: {cpu: 10m, memory: 20Mi}}
|
||||
16
autocert/examples/hello-mtls/nginx/site.conf
Normal file
16
autocert/examples/hello-mtls/nginx/site.conf
Normal file
|
|
@ -0,0 +1,16 @@
|
|||
server {
|
||||
listen 443 ssl;
|
||||
server_name localhost;
|
||||
ssl_protocols TLSv1.2;
|
||||
ssl_certificate /var/run/autocert.step.sm/site.crt;
|
||||
ssl_certificate_key /var/run/autocert.step.sm/site.key;
|
||||
ssl_client_certificate /var/run/autocert.step.sm/root.crt;
|
||||
ssl_verify_client on;
|
||||
ssl_prefer_server_ciphers on;
|
||||
ssl_ciphers ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256;
|
||||
|
||||
location / {
|
||||
root /usr/share/nginx/html;
|
||||
index index.html index.htm;
|
||||
}
|
||||
}
|
||||
6
autocert/examples/hello-mtls/node/Dockerfile.client
Normal file
6
autocert/examples/hello-mtls/node/Dockerfile.client
Normal file
|
|
@ -0,0 +1,6 @@
|
|||
FROM node:lts-alpine
|
||||
|
||||
RUN mkdir /src
|
||||
ADD client.js /src
|
||||
|
||||
CMD ["node", "/src/client.js"]
|
||||
6
autocert/examples/hello-mtls/node/Dockerfile.server
Normal file
6
autocert/examples/hello-mtls/node/Dockerfile.server
Normal file
|
|
@ -0,0 +1,6 @@
|
|||
FROM node:lts-alpine
|
||||
|
||||
RUN mkdir /src
|
||||
ADD server.js /src
|
||||
|
||||
CMD ["node", "/src/server.js"]
|
||||
44
autocert/examples/hello-mtls/node/client.js
Normal file
44
autocert/examples/hello-mtls/node/client.js
Normal file
|
|
@ -0,0 +1,44 @@
|
|||
const fs = require('fs');
|
||||
const https = require('https');
|
||||
|
||||
const config = {
|
||||
ca: '/var/run/autocert.step.sm/root.crt',
|
||||
key: '/var/run/autocert.step.sm/site.key',
|
||||
cert: '/var/run/autocert.step.sm/site.crt',
|
||||
url: process.env.HELLO_MTLS_URL,
|
||||
requestFrequency: 5000
|
||||
};
|
||||
|
||||
var options = {
|
||||
ca: fs.readFileSync(config.ca),
|
||||
key: fs.readFileSync(config.key),
|
||||
cert: fs.readFileSync(config.cert),
|
||||
ciphers: 'ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256',
|
||||
minVersion: 'TLSv1.2',
|
||||
maxVersion: 'TLSv1.2',
|
||||
// Not necessary as it defaults to true
|
||||
rejectUnauthorized: true
|
||||
};
|
||||
|
||||
fs.watch(config.cert, (event, filename) => {
|
||||
if (event == 'change') {
|
||||
options.cert = fs.readFileSync(config.cert);
|
||||
}
|
||||
});
|
||||
|
||||
function loop() {
|
||||
var req = https.request(config.url, options, function(res) {
|
||||
res.on('data', (data) => {
|
||||
process.stdout.write(options.cert)
|
||||
process.stdout.write(data)
|
||||
setTimeout(loop, config.requestFrequency);
|
||||
});
|
||||
});
|
||||
req.on('error', (e) => {
|
||||
process.stderr.write('error: ' + e.message + '\n');
|
||||
setTimeout(loop, config.requestFrequency);
|
||||
})
|
||||
req.end();
|
||||
}
|
||||
|
||||
loop();
|
||||
22
autocert/examples/hello-mtls/node/hello-mtls.client.yaml
Normal file
22
autocert/examples/hello-mtls/node/hello-mtls.client.yaml
Normal file
|
|
@ -0,0 +1,22 @@
|
|||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: hello-mtls-client
|
||||
labels: {app: hello-mtls-client}
|
||||
spec:
|
||||
replicas: 1
|
||||
selector: {matchLabels: {app: hello-mtls-client}}
|
||||
template:
|
||||
metadata:
|
||||
annotations:
|
||||
autocert.step.sm/name: hello-mtls-client.default.pod.cluster.local
|
||||
labels: {app: hello-mtls-client}
|
||||
spec:
|
||||
containers:
|
||||
- name: hello-mtls-client
|
||||
image: hello-mtls-client-node:latest
|
||||
imagePullPolicy: Never
|
||||
resources: {requests: {cpu: 10m, memory: 20Mi}}
|
||||
env:
|
||||
- name: HELLO_MTLS_URL
|
||||
value: https://hello-mtls.default.svc.cluster.local
|
||||
33
autocert/examples/hello-mtls/node/hello-mtls.server.yaml
Normal file
33
autocert/examples/hello-mtls/node/hello-mtls.server.yaml
Normal file
|
|
@ -0,0 +1,33 @@
|
|||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
labels: {app: hello-mtls}
|
||||
name: hello-mtls
|
||||
spec:
|
||||
type: ClusterIP
|
||||
ports:
|
||||
- port: 443
|
||||
targetPort: 443
|
||||
selector: {app: hello-mtls}
|
||||
|
||||
---
|
||||
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: hello-mtls
|
||||
labels: {app: hello-mtls}
|
||||
spec:
|
||||
replicas: 1
|
||||
selector: {matchLabels: {app: hello-mtls}}
|
||||
template:
|
||||
metadata:
|
||||
annotations:
|
||||
autocert.step.sm/name: hello-mtls.default.svc.cluster.local
|
||||
labels: {app: hello-mtls}
|
||||
spec:
|
||||
containers:
|
||||
- name: hello-mtls
|
||||
image: hello-mtls-server-node:latest
|
||||
imagePullPolicy: Never
|
||||
resources: {requests: {cpu: 10m, memory: 20Mi}}
|
||||
42
autocert/examples/hello-mtls/node/server.js
Normal file
42
autocert/examples/hello-mtls/node/server.js
Normal file
|
|
@ -0,0 +1,42 @@
|
|||
const https = require('https');
|
||||
const tls = require('tls');
|
||||
const fs = require('fs');
|
||||
|
||||
var config = {
|
||||
ca: '/var/run/autocert.step.sm/root.crt',
|
||||
key: '/var/run/autocert.step.sm/site.key',
|
||||
cert: '/var/run/autocert.step.sm/site.crt',
|
||||
ciphers: 'ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256',
|
||||
minVersion: 'TLSv1.2',
|
||||
maxVersion: 'TLSv1.2'
|
||||
};
|
||||
|
||||
function createSecureContext() {
|
||||
return tls.createSecureContext({
|
||||
ca: fs.readFileSync(config.ca),
|
||||
key: fs.readFileSync(config.key),
|
||||
cert: fs.readFileSync(config.cert),
|
||||
ciphers: config.ciphers,
|
||||
});
|
||||
}
|
||||
|
||||
var ctx = createSecureContext()
|
||||
|
||||
fs.watch(config.cert, (event, filename) => {
|
||||
if (event == 'change') {
|
||||
ctx = createSecureContext();
|
||||
}
|
||||
});
|
||||
|
||||
https.createServer({
|
||||
requestCert: true,
|
||||
rejectUnauthorized: true,
|
||||
SNICallback: (servername, cb) => {
|
||||
cb(null, ctx);
|
||||
}
|
||||
}, (req, res) => {
|
||||
res.writeHead(200);
|
||||
res.end('hello nodejs\n');
|
||||
}).listen(443);
|
||||
|
||||
console.log("Listening on :443 ...");
|
||||
18
autocert/init/Dockerfile
Normal file
18
autocert/init/Dockerfile
Normal file
|
|
@ -0,0 +1,18 @@
|
|||
FROM smallstep/step-cli:0.8.4-rc.1
|
||||
|
||||
ENV CA_NAME="Autocert"
|
||||
ENV CA_DNS="ca.step.svc.cluster.local,127.0.0.1"
|
||||
ENV CA_ADDRESS=":4443"
|
||||
ENV CA_DEFAULT_PROVISIONER="admin"
|
||||
ENV CA_URL="ca.step.svc.cluster.local"
|
||||
|
||||
ENV KUBE_LATEST_VERSION="v1.13.2"
|
||||
|
||||
USER root
|
||||
RUN curl -L https://storage.googleapis.com/kubernetes-release/release/${KUBE_LATEST_VERSION}/bin/linux/amd64/kubectl -o /usr/local/bin/kubectl \
|
||||
&& chmod +x /usr/local/bin/kubectl
|
||||
RUN apk --update add expect
|
||||
|
||||
COPY autocert.sh /home/step/
|
||||
RUN chmod +x /home/step/autocert.sh
|
||||
CMD ["/home/step/autocert.sh"]
|
||||
164
autocert/init/autocert.sh
Executable file
164
autocert/init/autocert.sh
Executable file
|
|
@ -0,0 +1,164 @@
|
|||
#!/bin/bash
|
||||
|
||||
#set -x
|
||||
|
||||
echo "Welcome to Autocert configuration. Press return to begin."
|
||||
read ANYKEY
|
||||
|
||||
|
||||
STEPPATH=/home/step/.step
|
||||
|
||||
CA_PASSWORD=$(head /dev/urandom | tr -dc A-Za-z0-9 | head -c 32 ; echo '')
|
||||
AUTOCERT_PASSWORD=$(head /dev/urandom | tr -dc A-Za-z0-9 | head -c 32 ; echo '')
|
||||
|
||||
echo -e "\e[1mChecking cluster permissions...\e[0m"
|
||||
|
||||
function permission_error {
|
||||
# TODO: Figure out the actual service account instead of assuming default.
|
||||
echo
|
||||
echo -e "\033[0;31mPERMISSION ERROR\033[0m"
|
||||
echo "Set permissions by running the following command, then try again:"
|
||||
echo -e "\e[1m"
|
||||
echo " kubectl create clusterrolebinding autocert-init-binding \\"
|
||||
echo " --clusterrole cluster-admin \\"
|
||||
echo " --user \"system:serviceaccount:default:default\""
|
||||
echo -e "\e[0m"
|
||||
echo "Once setup is complete you can remove this binding by running:"
|
||||
echo -e "\e[1m"
|
||||
echo " kubectl delete clusterrolebinding autocert-init-binding"
|
||||
echo -e "\e[0m"
|
||||
|
||||
exit 1
|
||||
}
|
||||
|
||||
echo -n "Checking for permission to create step namespace: "
|
||||
kubectl auth can-i create namespaces
|
||||
if [ $? -ne 0 ]; then
|
||||
permission_error "create step namespace"
|
||||
fi
|
||||
|
||||
echo -n "Checking for permission to create configmaps in step namespace: "
|
||||
kubectl auth can-i create configmaps --namespace step
|
||||
if [ $? -ne 0 ]; then
|
||||
permission_error "create configmaps"
|
||||
fi
|
||||
|
||||
echo -n "Checking for permission to create secrets in step namespace: "
|
||||
kubectl auth can-i create secrets --namespace step
|
||||
if [ $? -ne 0 ]; then
|
||||
permission_error "create secrets"
|
||||
fi
|
||||
|
||||
echo -n "Checking for permission to create deployments in step namespace: "
|
||||
kubectl auth can-i create deployments --namespace step
|
||||
if [ $? -ne 0 ]; then
|
||||
permission_error "create deployments"
|
||||
fi
|
||||
|
||||
echo -n "Checking for permission to create services in step namespace: "
|
||||
kubectl auth can-i create services --namespace step
|
||||
if [ $? -ne 0 ]; then
|
||||
permission_error "create services"
|
||||
fi
|
||||
|
||||
echo -n "Checking for permission to create cluster role: "
|
||||
kubectl auth can-i create clusterrole
|
||||
if [ $? -ne 0 ]; then
|
||||
permission_error "create cluster roles"
|
||||
fi
|
||||
|
||||
echo -n "Checking for permission to create cluster role binding:"
|
||||
kubectl auth can-i create clusterrolebinding
|
||||
if [ $? -ne 0 ]; then
|
||||
permission_error "create cluster role bindings"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Setting this here on purpose, after the above section which explicitly checks
|
||||
# for and handles exit errors.
|
||||
set -e
|
||||
|
||||
step ca init \
|
||||
--name "$CA_NAME" \
|
||||
--dns "$CA_DNS" \
|
||||
--address "$CA_ADDRESS" \
|
||||
--provisioner "$CA_DEFAULT_PROVISIONER" \
|
||||
--with-ca-url "$CA_URL" \
|
||||
--password-file <(echo "$CA_PASSWORD")
|
||||
|
||||
echo
|
||||
echo -e "\e[1mCreating autocert provisioner...\e[0m"
|
||||
|
||||
expect <<EOD
|
||||
spawn step ca provisioner add autocert --create
|
||||
expect "Please enter a password to encrypt the provisioner private key? \\\\\\[leave empty and we'll generate one\\\\\\]: "
|
||||
send "${AUTOCERT_PASSWORD}\n"
|
||||
expect eof
|
||||
EOD
|
||||
|
||||
echo
|
||||
echo -e "\e[1mCreating step namespace and preparing environment...\e[0m"
|
||||
|
||||
kubectl create namespace step
|
||||
|
||||
kubectl -n step create configmap config --from-file $(step path)/config
|
||||
kubectl -n step create configmap certs --from-file $(step path)/certs
|
||||
kubectl -n step create configmap secrets --from-file $(step path)/secrets
|
||||
|
||||
kubectl -n step create secret generic ca-password --from-literal "password=${CA_PASSWORD}"
|
||||
kubectl -n step create secret generic autocert-password --from-literal "password=${AUTOCERT_PASSWORD}"
|
||||
|
||||
# Deploy CA and wait for rollout to complete
|
||||
echo
|
||||
echo -e "\e[1mDeploying certificate authority...\e[0m"
|
||||
|
||||
kubectl apply -f https://raw.githubusercontent.com/smallstep/certificates/autocert/autocert/install/01-step-ca.yaml
|
||||
kubectl -n step rollout status deployment/ca
|
||||
|
||||
# Deploy autocert, setup RBAC, and wait for rollout to complete
|
||||
echo
|
||||
echo -e "\e[1mDeploying autocert...\e[0m"
|
||||
|
||||
kubectl apply -f https://raw.githubusercontent.com/smallstep/certificates/autocert/autocert/install/02-autocert.yaml
|
||||
kubectl apply -f https://raw.githubusercontent.com/smallstep/certificates/autocert/autocert/install/03-rbac.yaml
|
||||
kubectl -n step rollout status deployment/autocert
|
||||
|
||||
# Some `base64`s wrap lines... no thanks!
|
||||
CA_BUNDLE=$(cat $(step path)/certs/root_ca.crt | base64 | tr -d '\n')
|
||||
|
||||
cat <<EOF | kubectl apply -f -
|
||||
apiVersion: admissionregistration.k8s.io/v1beta1
|
||||
kind: MutatingWebhookConfiguration
|
||||
metadata:
|
||||
name: autocert-webhook-config
|
||||
labels: {app: autocert}
|
||||
webhooks:
|
||||
- name: autocert.step.sm
|
||||
clientConfig:
|
||||
service:
|
||||
name: autocert
|
||||
namespace: step
|
||||
path: "/mutate"
|
||||
caBundle: $CA_BUNDLE
|
||||
rules:
|
||||
- operations: ["CREATE"]
|
||||
apiGroups: [""]
|
||||
apiVersions: ["v1"]
|
||||
resources: ["pods"]
|
||||
failurePolicy: Ignore
|
||||
namespaceSelector:
|
||||
matchLabels:
|
||||
autocert.step.sm: enabled
|
||||
EOF
|
||||
|
||||
FINGERPRINT=$(step certificate fingerprint $(step path)/certs/root_ca.crt)
|
||||
|
||||
echo
|
||||
echo -e "\e[1mAutocert installed!\e[0m"
|
||||
echo
|
||||
echo "Store this information somewhere safe:"
|
||||
echo " CA & admin provisioner password: ${CA_PASSWORD}"
|
||||
echo " Autocert password: ${AUTOCERT_PASSWORD}"
|
||||
echo " CA Fingerprint: ${FINGERPRINT}"
|
||||
echo
|
||||
|
||||
87
autocert/install/01-step-ca.yaml
Normal file
87
autocert/install/01-step-ca.yaml
Normal file
|
|
@ -0,0 +1,87 @@
|
|||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
labels:
|
||||
app: ca
|
||||
name: ca
|
||||
namespace: step
|
||||
spec:
|
||||
type: ClusterIP
|
||||
ports:
|
||||
- port: 443
|
||||
targetPort: 4443
|
||||
selector:
|
||||
app: ca
|
||||
|
||||
---
|
||||
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: ca
|
||||
namespace: step
|
||||
labels:
|
||||
app: ca
|
||||
spec:
|
||||
replicas: 1
|
||||
selector:
|
||||
matchLabels:
|
||||
app: ca
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: ca
|
||||
spec:
|
||||
containers:
|
||||
- name: ca
|
||||
image: smallstep/step-ca:0.8.3
|
||||
env:
|
||||
- name: PWDPATH
|
||||
value: /home/step/password/password
|
||||
resources:
|
||||
requests:
|
||||
cpu: 100m
|
||||
memory: 20Mi
|
||||
readinessProbe:
|
||||
httpGet:
|
||||
path: /health
|
||||
port: 4443
|
||||
scheme: HTTPS
|
||||
initialDelaySeconds: 3
|
||||
periodSeconds: 3
|
||||
livenessProbe:
|
||||
httpGet:
|
||||
path: /health
|
||||
port: 4443
|
||||
scheme: HTTPS
|
||||
initialDelaySeconds: 3
|
||||
periodSeconds: 3
|
||||
volumeMounts:
|
||||
- name: config
|
||||
mountPath: /home/step/.step/config
|
||||
readOnly: true
|
||||
- name: certs
|
||||
mountPath: /home/step/.step/certs
|
||||
readOnly: true
|
||||
- name: secrets
|
||||
mountPath: /home/step/.step/secrets
|
||||
readOnly: true
|
||||
- name: ca-password
|
||||
mountPath: /home/step/password
|
||||
readOnly: true
|
||||
securityContext:
|
||||
runAsUser: 1000
|
||||
allowPrivilegeEscalation: false
|
||||
volumes:
|
||||
- name: certs
|
||||
configMap:
|
||||
name: certs
|
||||
- name: config
|
||||
configMap:
|
||||
name: config
|
||||
- name: secrets
|
||||
configMap:
|
||||
name: secrets
|
||||
- name: ca-password
|
||||
secret:
|
||||
secretName: ca-password
|
||||
106
autocert/install/02-autocert.yaml
Normal file
106
autocert/install/02-autocert.yaml
Normal file
|
|
@ -0,0 +1,106 @@
|
|||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
labels: {app: autocert}
|
||||
name: autocert
|
||||
namespace: step
|
||||
spec:
|
||||
type: ClusterIP
|
||||
ports:
|
||||
- port: 443
|
||||
targetPort: 4443
|
||||
selector: {app: autocert}
|
||||
|
||||
---
|
||||
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: autocert-config
|
||||
namespace: step
|
||||
data:
|
||||
config.yaml: |
|
||||
logFormat: json # or text
|
||||
caUrl: https://ca.step.svc.cluster.local
|
||||
certLifetime: 24h
|
||||
renewer:
|
||||
name: autocert-renewer
|
||||
image: smallstep/autocert-renewer:0.8.3
|
||||
resources: {requests: {cpu: 10m, memory: 20Mi}}
|
||||
imagePullPolicy: IfNotPresent
|
||||
volumeMounts:
|
||||
- name: certs
|
||||
mountPath: /var/run/autocert.step.sm
|
||||
bootstrapper:
|
||||
name: autocert-bootstrapper
|
||||
image: smallstep/autocert-bootstrapper:0.8.3
|
||||
resources: {requests: {cpu: 10m, memory: 20Mi}}
|
||||
imagePullPolicy: IfNotPresent
|
||||
volumeMounts:
|
||||
- name: certs
|
||||
mountPath: /var/run/autocert.step.sm
|
||||
certsVolume:
|
||||
name: certs
|
||||
emptyDir: {}
|
||||
|
||||
---
|
||||
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: autocert
|
||||
namespace: step
|
||||
labels: {app: autocert}
|
||||
spec:
|
||||
replicas: 1
|
||||
selector: {matchLabels: {app: autocert}}
|
||||
template:
|
||||
metadata: {labels: {app: autocert}}
|
||||
spec:
|
||||
containers:
|
||||
- name: autocert
|
||||
image: smallstep/autocert-controller:0.8.3
|
||||
resources: {requests: {cpu: 100m, memory: 20Mi}}
|
||||
env:
|
||||
- name: PROVISIONER_NAME
|
||||
value: autocert
|
||||
- name: NAMESPACE
|
||||
valueFrom:
|
||||
fieldRef:
|
||||
fieldPath: metadata.namespace
|
||||
volumeMounts:
|
||||
- name: config
|
||||
mountPath: /home/step/.step/config
|
||||
readOnly: true
|
||||
- name: certs
|
||||
mountPath: /home/step/.step/certs
|
||||
readOnly: true
|
||||
- name: autocert-password
|
||||
mountPath: /home/step/password
|
||||
readOnly: true
|
||||
- name: autocert-config
|
||||
mountPath: /home/step/autocert
|
||||
readOnly: true
|
||||
securityContext:
|
||||
runAsUser: 1000
|
||||
allowPrivilegeEscalation: false
|
||||
livenessProbe:
|
||||
httpGet:
|
||||
path: /healthz
|
||||
port: 4443
|
||||
scheme: HTTPS
|
||||
readinessProbe:
|
||||
httpGet:
|
||||
path: /healthz
|
||||
port: 4443
|
||||
scheme: HTTPS
|
||||
volumes:
|
||||
- name: config
|
||||
configMap: {name: config}
|
||||
- name: certs
|
||||
configMap: {name: certs}
|
||||
- name: autocert-password
|
||||
secret: {secretName: autocert-password}
|
||||
- name: autocert-config
|
||||
configMap: {name: autocert-config}
|
||||
|
||||
36
autocert/install/03-rbac.yaml
Normal file
36
autocert/install/03-rbac.yaml
Normal file
|
|
@ -0,0 +1,36 @@
|
|||
# Create a ClusterRole for managing autocert secrets, which should
|
||||
# only exist in namespaces with autocert enabled and should always
|
||||
# be labeled `autocert.step.sm/token: true`.
|
||||
#
|
||||
# To create this ClusterRole you need cluster-admin privileges. On
|
||||
# GKE you can give yourself cluster-admin privileges using the
|
||||
# following command:
|
||||
#
|
||||
# kubectl create clusterrolebinding cluster-admin-binding \
|
||||
# --clusterrole cluster-admin \
|
||||
# --user $(gcloud config get-value account)
|
||||
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRole
|
||||
metadata:
|
||||
name: autocert-controller
|
||||
rules:
|
||||
- apiGroups: [""]
|
||||
resources: ["secrets"]
|
||||
verbs: ["create", "delete"]
|
||||
|
||||
---
|
||||
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRoleBinding
|
||||
metadata:
|
||||
name: autocert-controller
|
||||
roleRef:
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: ClusterRole
|
||||
name: autocert-controller
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: default
|
||||
namespace: step
|
||||
|
||||
BIN
autocert/mtls-handshake.png
Normal file
BIN
autocert/mtls-handshake.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 1.3 MiB |
8
autocert/renewer/Dockerfile
Normal file
8
autocert/renewer/Dockerfile
Normal file
|
|
@ -0,0 +1,8 @@
|
|||
FROM smallstep/step-cli:0.8.3
|
||||
|
||||
USER root
|
||||
ENV CRT="/var/run/autocert.step.sm/site.crt"
|
||||
ENV KEY="/var/run/autocert.step.sm/site.key"
|
||||
ENV STEP_ROOT="/var/run/autocert.step.sm/root.crt"
|
||||
|
||||
ENTRYPOINT ["/bin/bash", "-c", "step ca renew --daemon $CRT $KEY"]
|
||||
Loading…
Reference in a new issue