coredns/plugin/metrics
Miek Gieben 6953ab2b4f
Metrics: expand coredns_dns_responses_total with plugin label (#4914)
* Metrics: expand coredns_dns_responses_total with plugin label

This adds (somewhat hacky?) code to add a plugin label to the
coredns_dns_responses_total metric. It's completely obvlious to the
plugin as we just check who called the *recorder.WriteMsg method. We use
runtime.Caller( 1 2 3) to get multiple levels of callers, this should be
deep enough, but it depends on the dns.ResponseWriter wrapping that's
occuring.

README.md of metrics updates and test added in test/metrics_test.go to
check for the label being set.

I went through the plugin to see what metrics could be removed, but
actually didn't find any, the plugin push out metrics that make sense.

Due to the path fiddling to figure out the plugin name I doubt this
works (out-of-the-box) for external plugins, but I haven't tested that.

Signed-off-by: Miek Gieben <miek@miek.nl>

* better comment

Signed-off-by: Miek Gieben <miek@miek.nl>

* Metrics: expand coredns_dns_responses_total with plugin label

This adds (somewhat hacky?) code to add a plugin label to the
coredns_dns_responses_total metric. It's completely obvlious to the
plugin as we just check who called the *recorder.WriteMsg method. We use
runtime.Caller( 1 2 3) to get multiple levels of callers, this should be
deep enough, but it depends on the dns.ResponseWriter wrapping that's
occuring.

README.md of metrics updates and test added in test/metrics_test.go to
check for the label being set.

I went through the plugin to see what metrics could be removed, but
actually didn't find any, the plugin push out metrics that make sense.

Due to the path fiddling to figure out the plugin name I doubt this
works (out-of-the-box) for external plugins, but I haven't tested that.

Signed-off-by: Miek Gieben <miek@miek.nl>

* Update core/dnsserver/server.go

Co-authored-by: dilyevsky <ilyevsky@gmail.com>

* Use [3]string

Signed-off-by: Miek Gieben <miek@miek.nl>

* imports

Signed-off-by: Miek Gieben <miek@miek.nl>

* remove dnstest changes

Signed-off-by: Miek Gieben <miek@miek.nl>

* revert

Signed-off-by: Miek Gieben <miek@miek.nl>

* Add some sleeps to make it less flaky

Signed-off-by: Miek Gieben <miek@miek.nl>

* Revert "Add some sleeps to make it less flaky"

This reverts commit b5c6655196.

* Remove forward when not needed

Signed-off-by: Miek Gieben <miek@miek.nl>

* remove newline

Signed-off-by: Miek Gieben <miek@miek.nl>

Co-authored-by: dilyevsky <ilyevsky@gmail.com>
2021-11-12 16:07:05 +00:00
..
vars Metrics: expand coredns_dns_responses_total with plugin label (#4914) 2021-11-12 16:07:05 +00:00
context.go Remove server addr from the context (#2722) 2019-03-25 10:46:44 -07:00
handler.go Metrics: expand coredns_dns_responses_total with plugin label (#4914) 2021-11-12 16:07:05 +00:00
log_test.go Clean up tests logging (#1979) 2018-07-19 16:23:06 +01:00
metrics.go Metrics: expand coredns_dns_responses_total with plugin label (#4914) 2021-11-12 16:07:05 +00:00
metrics_test.go Speed up testing (#4239) 2020-10-30 10:27:04 +01:00
README.md Metrics: expand coredns_dns_responses_total with plugin label (#4914) 2021-11-12 16:07:05 +00:00
recorder.go Metrics: expand coredns_dns_responses_total with plugin label (#4914) 2021-11-12 16:07:05 +00:00
registry.go plugin/metrcs: fix datarace on listeners (#2835) 2019-05-18 18:34:46 +01:00
setup.go reverse zone: fix Normalize (#4621) 2021-05-17 13:19:54 -07:00
setup_test.go For caddy v1 in our org (#4018) 2020-09-24 18:14:41 +02:00

prometheus

Name

prometheus - enables Prometheus metrics.

Description

With prometheus you export metrics from CoreDNS and any plugin that has them. The default location for the metrics is localhost:9153. The metrics path is fixed to /metrics. The following metrics are exported:

  • coredns_build_info{version, revision, goversion} - info about CoreDNS itself.
  • coredns_panics_total{} - total number of panics.
  • coredns_dns_requests_total{server, zone, proto, family, type} - total query count.
  • coredns_dns_request_duration_seconds{server, zone, type} - duration to process each query.
  • coredns_dns_request_size_bytes{server, zone, proto} - size of the request in bytes.
  • coredns_dns_do_requests_total{server, zone} - queries that have the DO bit set
  • coredns_dns_response_size_bytes{server, zone, proto} - response size in bytes.
  • coredns_dns_responses_total{server, zone, rcode, plugin} - response per zone, rcode and plugin.
  • coredns_plugin_enabled{server, zone, name} - indicates whether a plugin is enabled on per server and zone basis.

Each counter has a label zone which is the zonename used for the request/response.

Extra labels used are:

  • server is identifying the server responsible for the request. This is a string formatted as the server's listening address: <scheme>://[<bind>]:<port>. I.e. for a "normal" DNS server this is dns://:53. If you are using the bind plugin an IP address is included, e.g.: dns://127.0.0.53:53.
  • proto which holds the transport of the response ("udp" or "tcp")
  • The address family (family) of the transport (1 = IP (IP version 4), 2 = IP6 (IP version 6)).
  • type which holds the query type. It holds most common types (A, AAAA, MX, SOA, CNAME, PTR, TXT, NS, SRV, DS, DNSKEY, RRSIG, NSEC, NSEC3, HTTPS, IXFR, AXFR and ANY) and "other" which lumps together all other types.
  • the plugin label holds the name of the plugin that made the write to the client. If the server did the write (on error for instance), the value is empty.

If monitoring is enabled, queries that do not enter the plugin chain are exported under the fake name "dropped" (without a closing dot - this is never a valid domain name).

This plugin can only be used once per Server Block.

Syntax

prometheus [ADDRESS]

For each zone that you want to see metrics for.

It optionally takes a bind address to which the metrics are exported; the default listens on localhost:9153. The metrics path is fixed to /metrics.

Examples

Use an alternative listening address:

. {
    prometheus localhost:9253
}

Or via an environment variable (this is supported throughout the Corefile): export PORT=9253, and then:

. {
    prometheus localhost:{$PORT}
}

Bugs

When reloading, the Prometheus handler is stopped before the new server instance is started. If that new server fails to start, then the initial server instance is still available and DNS queries still served, but Prometheus handler stays down. Prometheus will not reply HTTP request until a successful reload or a complete restart of CoreDNS. Only the plugins that register as Handler are visible in coredns_plugin_enabled{server, zone, name}. As of today the plugins reload and bind will not be reported.