.\" Generated by Mmark Markdown Processer - mmark.miek.nl
.TH "COREDNS-KUBERNETES" 7 "March 2021" "CoreDNS" "CoreDNS Plugins"

.SH "NAME"
.PP
\fIkubernetes\fP - enables reading zone data from a Kubernetes cluster.

.SH "DESCRIPTION"
.PP
This plugin implements the Kubernetes DNS-Based Service Discovery
Specification
\[la]https://github.com/kubernetes/dns/blob/master/docs/specification.md\[ra].

.PP
CoreDNS running the kubernetes plugin can be used as a replacement for kube-dns in a kubernetes
cluster.  See the deployment
\[la]https://github.com/coredns/deployment\[ra] repository for details on how
to deploy CoreDNS in Kubernetes
\[la]https://github.com/coredns/deployment/tree/master/kubernetes\[ra].

.PP
stubDomains and upstreamNameservers
\[la]https://kubernetes.io/blog/2017/04/configuring-private-dns-zones-upstream-nameservers-kubernetes/\[ra]
are implemented via the \fIforward\fP plugin. See the examples below.

.PP
This plugin can only be used once per Server Block.

.SH "SYNTAX"
.PP
.RS

.nf
kubernetes [ZONES...]

.fi
.RE

.PP
With only the plugin specified, the \fIkubernetes\fP plugin will default to the zone specified in
the server's block. It will handle all queries in that zone and connect to Kubernetes in-cluster. It
will not provide PTR records for services or A records for pods. If \fBZONES\fP is used it specifies
all the zones the plugin should be authoritative for.

.PP
.RS

.nf
kubernetes [ZONES...] {
    endpoint URL
    tls CERT KEY CACERT
    kubeconfig KUBECONFIG [CONTEXT]
    namespaces NAMESPACE...
    labels EXPRESSION
    pods POD\-MODE
    endpoint\_pod\_names
    ttl TTL
    noendpoints
    fallthrough [ZONES...]
    ignore empty\_service
}

.fi
.RE

.IP \(bu 4
\fB\fCendpoint\fR specifies the \fBURL\fP for a remote k8s API endpoint.
If omitted, it will connect to k8s in-cluster using the cluster service account.
.IP \(bu 4
\fB\fCtls\fR \fBCERT\fP \fBKEY\fP \fBCACERT\fP are the TLS cert, key and the CA cert file names for remote k8s connection.
This option is ignored if connecting in-cluster (i.e. endpoint is not specified).
.IP \(bu 4
\fB\fCkubeconfig\fR \fBKUBECONFIG [CONTEXT]\fP authenticates the connection to a remote k8s cluster using a kubeconfig file.
\fB[CONTEXT]\fP is optional, if not set, then the current context specified in kubeconfig will be used.
It supports TLS, username and password, or token-based authentication.
This option is ignored if connecting in-cluster (i.e., the endpoint is not specified).
.IP \(bu 4
\fB\fCnamespaces\fR \fBNAMESPACE [NAMESPACE...]\fP only exposes the k8s namespaces listed.
If this option is omitted all namespaces are exposed
.IP \(bu 4
\fB\fCnamespace_labels\fR \fBEXPRESSION\fP only expose the records for Kubernetes namespaces that match this label selector.
The label selector syntax is described in the
Kubernetes User Guide - Labels
\[la]https://kubernetes.io/docs/user-guide/labels/\[ra]. An example that
only exposes namespaces labeled as "istio-injection=enabled", would use:
\fB\fClabels istio-injection=enabled\fR.
.IP \(bu 4
\fB\fClabels\fR \fBEXPRESSION\fP only exposes the records for Kubernetes objects that match this label selector.
The label selector syntax is described in the
Kubernetes User Guide - Labels
\[la]https://kubernetes.io/docs/user-guide/labels/\[ra]. An example that
only exposes objects labeled as "application=nginx" in the "staging" or "qa" environments, would
use: \fB\fClabels environment in (staging, qa),application=nginx\fR.
.IP \(bu 4
\fB\fCpods\fR \fBPOD-MODE\fP sets the mode for handling IP-based pod A records, e.g.
\fB\fC1-2-3-4.ns.pod.cluster.local. in A 1.2.3.4\fR.
This option is provided to facilitate use of SSL certs when connecting directly to pods. Valid
values for \fBPOD-MODE\fP:

.RS
.IP \(en 4
\fB\fCdisabled\fR: Default. Do not process pod requests, always returning \fB\fCNXDOMAIN\fR
.IP \(en 4
\fB\fCinsecure\fR: Always return an A record with IP from request (without checking k8s).  This option
is vulnerable to abuse if used maliciously in conjunction with wildcard SSL certs.  This
option is provided for backward compatibility with kube-dns.
.IP \(en 4
\fB\fCverified\fR: Return an A record if there exists a pod in same namespace with matching IP.  This
option requires substantially more memory than in insecure mode, since it will maintain a watch
on all pods.

.RE
.IP \(bu 4
\fB\fCendpoint_pod_names\fR uses the pod name of the pod targeted by the endpoint as
the endpoint name in A records, e.g.,
\fB\fCendpoint-name.my-service.namespace.svc.cluster.local. in A 1.2.3.4\fR
By default, the endpoint-name name selection is as follows: Use the hostname
of the endpoint, or if hostname is not set, use the dashed form of the endpoint
IP address (e.g., \fB\fC1-2-3-4.my-service.namespace.svc.cluster.local.\fR)
If this directive is included, then name selection for endpoints changes as
follows: Use the hostname of the endpoint, or if hostname is not set, use the
pod name of the pod targeted by the endpoint. If there is no pod targeted by
the endpoint, use the dashed IP address form.
.IP \(bu 4
\fB\fCttl\fR allows you to set a custom TTL for responses. The default is 5 seconds.  The minimum TTL allowed is
0 seconds, and the maximum is capped at 3600 seconds. Setting TTL to 0 will prevent records from being cached.
.IP \(bu 4
\fB\fCnoendpoints\fR will turn off the serving of endpoint records by disabling the watch on endpoints.
All endpoint queries and headless service queries will result in an NXDOMAIN.
.IP \(bu 4
\fB\fCfallthrough\fR \fB[ZONES...]\fP If a query for a record in the zones for which the plugin is authoritative
results in NXDOMAIN, normally that is what the response will be. However, if you specify this option,
the query will instead be passed on down the plugin chain, which can include another plugin to handle
the query. If \fB[ZONES...]\fP is omitted, then fallthrough happens for all zones for which the plugin
is authoritative. If specific zones are listed (for example \fB\fCin-addr.arpa\fR and \fB\fCip6.arpa\fR), then only
queries for those zones will be subject to fallthrough.
.IP \(bu 4
\fB\fCignore empty_service\fR returns NXDOMAIN for services without any ready endpoint addresses (e.g., ready pods).
This allows the querying pod to continue searching for the service in the search path.
The search path could, for example, include another Kubernetes cluster.


.PP
Enabling zone transfer is done by using the \fItransfer\fP plugin.

.SH "READY"
.PP
This plugin reports readiness to the ready plugin. This will happen after it has synced to the
Kubernetes API.

.SH "EXAMPLES"
.PP
Handle all queries in the \fB\fCcluster.local\fR zone. Connect to Kubernetes in-cluster. Also handle all
\fB\fCin-addr.arpa\fR \fB\fCPTR\fR requests for \fB\fC10.0.0.0/17\fR . Verify the existence of pods when answering pod
requests.

.PP
.RS

.nf
10.0.0.0/17 cluster.local {
    kubernetes {
        pods verified
    }
}

.fi
.RE

.PP
Or you can selectively expose some namespaces:

.PP
.RS

.nf
kubernetes cluster.local {
    namespaces test staging
}

.fi
.RE

.PP
Connect to Kubernetes with CoreDNS running outside the cluster:

.PP
.RS

.nf
kubernetes cluster.local {
    endpoint https://k8s\-endpoint:8443
    tls cert key cacert
}

.fi
.RE

.SH "STUBDOMAINS AND UPSTREAMNAMESERVERS"
.PP
Here we use the \fIforward\fP plugin to implement a stubDomain that forwards \fB\fCexample.local\fR to the nameserver \fB\fC10.100.0.10:53\fR.
Also configured is an upstreamNameserver \fB\fC8.8.8.8:53\fR that will be used for resolving names that do not fall in \fB\fCcluster.local\fR
or \fB\fCexample.local\fR.

.PP
.RS

.nf
cluster.local:53 {
    kubernetes cluster.local
}
example.local {
    forward . 10.100.0.10:53
}

\&. {
    forward . 8.8.8.8:53
}

.fi
.RE

.PP
The configuration above represents the following Kube-DNS stubDomains and upstreamNameservers configuration.

.PP
.RS

.nf
stubDomains: |
   {“example.local”: [“10.100.0.10:53”]}
upstreamNameservers: |
   [“8.8.8.8:53”]

.fi
.RE

.SH "AUTOPATH"
.PP
The \fIkubernetes\fP plugin can be used in conjunction with the \fIautopath\fP plugin.  Using this
feature enables server-side domain search path completion in Kubernetes clusters.  Note: \fB\fCpods\fR must
be set to \fB\fCverified\fR for this to function properly. Furthermore, the remote IP address in the DNS
packet received by CoreDNS must be the IP address of the Pod that sent the request.

.PP
.RS

.nf
cluster.local {
    autopath @kubernetes
    kubernetes {
        pods verified
    }
}

.fi
.RE

.SH "WILDCARDS"
.PP
Some query labels accept a wildcard value to match any value.  If a label is a valid wildcard (*,
or the word "any"), then that label will match all values.  The labels that accept wildcards are:

.IP \(bu 4
\fIendpoint\fP in an \fB\fCA\fR record request: \fIendpoint\fP.service.namespace.svc.zone, e.g., \fB\fC*.nginx.ns.svc.cluster.local\fR
.IP \(bu 4
\fIservice\fP in an \fB\fCA\fR record request: \fIservice\fP.namespace.svc.zone, e.g., \fB\fC*.ns.svc.cluster.local\fR
.IP \(bu 4
\fInamespace\fP in an \fB\fCA\fR record request: service.\fInamespace\fP.svc.zone, e.g., \fB\fCnginx.*.svc.cluster.local\fR
.IP \(bu 4
\fIport and/or protocol\fP in an \fB\fCSRV\fR request: \fBport_.\fPprotocol_.service.namespace.svc.zone.,
e.g., \fB\fC_http.*.service.ns.svc.cluster.local\fR
.IP \(bu 4
multiple wildcards are allowed in a single query, e.g., \fB\fCA\fR Request \fB\fC*.*.svc.zone.\fR or \fB\fCSRV\fR request \fB\fC*.*.*.*.svc.zone.\fR


.PP
For example, wildcards can be used to resolve all Endpoints for a Service as \fB\fCA\fR records. e.g.: \fB\fC*.service.ns.svc.myzone.local\fR will return the Endpoint IPs in the Service \fB\fCservice\fR in namespace \fB\fCdefault\fR:

.PP
.RS

.nf
*.service.default.svc.cluster.local. 5    IN A    192.168.10.10
*.service.default.svc.cluster.local. 5    IN A    192.168.25.15

.fi
.RE

.SH "METADATA"
.PP
The kubernetes plugin will publish the following metadata, if the \fImetadata\fP
plugin is also enabled:

.IP \(bu 4
\fB\fCkubernetes/endpoint\fR: the endpoint name in the query
.IP \(bu 4
\fB\fCkubernetes/kind\fR: the resource kind (pod or svc) in the query
.IP \(bu 4
\fB\fCkubernetes/namespace\fR: the namespace in the query
.IP \(bu 4
\fB\fCkubernetes/port-name\fR: the port name in an SRV query
.IP \(bu 4
\fB\fCkubernetes/protocol\fR: the protocol in an SRV query
.IP \(bu 4
\fB\fCkubernetes/service\fR: the service name in the query
.IP \(bu 4
\fB\fCkubernetes/client-namespace\fR: the client pod's namespace (see requirements below)
.IP \(bu 4
\fB\fCkubernetes/client-pod-name\fR: the client pod's name (see requirements below)


.PP
The \fB\fCkubernetes/client-namespace\fR and \fB\fCkubernetes/client-pod-name\fR metadata work by reconciling the
client IP address in the DNS request packet to a known pod IP address. Therefore the following is required:
 * \fB\fCpods verified\fR mode must be enabled
 * the remote IP address in the DNS packet received by CoreDNS must be the IP address
   of the Pod that sent the request.

.SH "METRICS"
.PP
If monitoring is enabled (via the \fIprometheus\fP plugin) then the following metrics are exported:

.IP \(bu 4
\fB\fCcoredns_kubernetes_dns_programming_duration_seconds{service_kind}\fR - Exports the
DNS programming latency SLI
\[la]https://github.com/kubernetes/community/blob/master/sig-scalability/slos/dns_programming_latency.md\[ra].
The metrics has the \fB\fCservice_kind\fR label that identifies the kind of the
kubernetes service
\[la]https://kubernetes.io/docs/concepts/services-networking/service\[ra].
It may take one of the three values:

.RS
.IP \(en 4
\fB\fCcluster_ip\fR
.IP \(en 4
\fB\fCheadless_with_selector\fR
.IP \(en 4
\fB\fCheadless_without_selector\fR

.RE


.SH "BUGS"
.PP
The duration metric only supports the "headless_with_selector" service currently.

.SH "SEE ALSO"
.PP
See the \fIautopath\fP plugin to enable search path optimizations. And use the \fItransfer\fP plugin to
enable outgoing zone transfers.