* Default to upstream to self
This is a backwards incompatible change.
This is a massive (cleanup) PR where we default to resolving external
names by the coredns process itself, instead of directly forwarding them
to some upstream.
This ignores any arguments `upstream` may have had and makes it depend
on proxy/forward configuration in the Corefile. This allows resolved
upstream names to be cached and we have better healthchecking of the
upstreams. It also means there is only one way to resolve names, by
either using the proxy or forward plugin.
The proxy/forward lookup.go functions have been removed. This also
lessen the dependency on proxy, meaning deprecating proxy will become
easier. Some tests have been removed as well, or moved to the top-level
test directory as they now require a full coredns process instead of
just the plugin.
For the etcd plugin, the entire StubZone resolving is *dropped*! This
was a hacky (but working) solution to say the least. If someone cares
deeply it can be brought back (maybe)?
The pkg/upstream is now very small and almost does nothing. Also the
New() function was changed to return a pointer to upstream.Upstream. It
also returns only one parameter, so any stragglers using it will
encounter a compile error.
All documentation has been adapted. This affected the following plugins:
* etcd
* file
* auto
* secondary
* federation
* template
* route53
A followup PR will make any upstream directives with arguments an error,
right now they are ignored.
Signed-off-by: Miek Gieben <miek@miek.nl>
* Fix etcd build - probably still fails unit test
Signed-off-by: Miek Gieben <miek@miek.nl>
* Slightly smarter lookup check in upstream
Signed-off-by: Miek Gieben <miek@miek.nl>
* Compilez
Signed-off-by: Miek Gieben <miek@miek.nl>
I think this is causing problem and it will actually clash with the
scrubbing that now happens for all plugins anyway. We're assuming the
returned message will be valid even with tc being set. request.Scrub
follows that same logic.
Signed-off-by: Miek Gieben <miek@miek.nl>
Upgrade to new dns lib version; that saw multiple improvements; some
patch releases are in the pipeline.
The big thing here is the removal of ErrTruncated, so we need to deal
with this slightly different in the forward plugin. It removed the
entire truncated.go logic and just checks the message for .Truncated (if
there is a message) and retries with tcp.
Signed-off-by: Miek Gieben <miek@miek.nl>
* plugin/forward: remove dynamic read timeout
We care about an upstream being there, so we still have a dynamic dial
time out (by way higher then 200ms) of 1s; this should be fairly stable
for an upstream. The read timeout if more variable because of cached and
non cached responses. As such remove his logic entirely.
Drop to 2s read timeout.
Fixes#2306
Signed-off-by: Miek Gieben <miek@miek.nl>
* - ensure plugins that use prometheus.MustRegister, re-register after reload
- removing once.Do on the startup function was simplest way to do it.
* - fix underscored names (advice of bot)
* - tune existing UT for reload, and add a test verifying failing reload does not prevent correct registering for metrics
* - ensure different ports for tests that can run in same time ..
* Move functions from pkg/transport to pkg/parse
Although "parse" is a fairly generic name I believe this is somewhat
better named. pkg/transport keeps a few constants that are uses
throughout for the rest is is renaming a bunch (and the fallout from
there to make things compile again).
Signed-off-by: Miek Gieben <miek@miek.nl>
* Fix tests
Signed-off-by: Miek Gieben <miek@miek.nl>
Create plugin/pkg/transport that holds the transport related functions.
This needed to be a new pkg to prevent cyclic import errors.
This cleans up a bunch of duplicated code in core/dnsserver that also
tried to parse a transport (now all done in transport.Parse).
Signed-off-by: Miek Gieben <miek@miek.nl>
Every plugin needs to deal with EDNS0 and should call Scrub to make a
message fit the client's buffer. Move this functionality into the server
and wrapping the ResponseWriter into a ScrubWriter that handles these
bits for us. Result:
Less code and faster, because multiple chained plugins could all be
calling scrub and SizeAndDo - now there is just one place.
Most tests in file/* and dnssec/* needed adjusting because in those unit
tests you don't see OPT RRs anymore. The DNSSEC signer was also looking
at the returned OPT RR to see if it needed to sign - as those are now
added by the server (and thus later), this needed to change slightly.
Scrub itself still exist (for backward compat reasons), but has been
made a noop. Scrub has been renamed to scrub as it should not be used by
external plugins.
Fixes: #2010
Signed-off-by: Miek Gieben <miek@miek.nl>
* Clean up tests logging
This cleans up the travis logs so you can see the failures better.
Older tests in tests/ would call log.SetOutput(ioutil.Discard) in
a haphazard way. This add log.Discard and put an `init` function in each
package's dir (no way to do this globally). The cleanup in tests/ is
clear.
All plugins also got this init function to have some uniformity and kill
any (future) logging there in the tests as well.
There is a one-off in pkg/healthcheck because that does log.
Signed-off-by: Miek Gieben <miek@miek.nl>
* bring back original log_test.go
Signed-off-by: Miek Gieben <miek@miek.nl>
* suppress logging here as well
Signed-off-by: Miek Gieben <miek@miek.nl>
* plugin/forward: add HealthChecker interface
Make the HealthChecker interface and morph the current DNS health
checker into that interface.
Remove all whole bunch of method on Forward that didn't make sense.
This is done in preparation of adding a DoH client to forward - which
requires a completely different healthcheck implementation (and more,
but lets start here)
Signed-off-by: Miek Gieben <miek@miek.nl>
* Use protocol
Signed-off-by: Miek Gieben <miek@miek.nl>
* Dial doesnt need to be method an Forward either
Signed-off-by: Miek Gieben <miek@miek.nl>
* Address comments
Address various comments on the PR.
Signed-off-by: Miek Gieben <miek@miek.nl>
Allow plugins to dump messages in text pcap to the log. The forward
plugin does this when a reply does not much the query.
If the debug plugin isn't loaded Hexdump and Hexdumpf are noop.
Signed-off-by: Miek Gieben <miek@miek.nl>
After several experiments at SoundCloud we found that the current
minimum read timeout of 10ms is too low. A single request against a
slow/unavailable authoritative server can cause all TCP connections to
get closed. We record a 50th percentile forward/proxy latency of <5ms,
and a 99th percentile latency of 60ms. Using a minimum timeout of 200ms
seems to be a fair trade-off between avoiding unnecessary high
connection churn and reacting to upstream failures in a timely manner.
This change also renames hcDuration to hcInterval to reflect its usage,
and removes the duplicated timeout constant to make code comprehension
easier.
add a test to see if we copy the rcode correctly. Some minor cleanup in
import ordering and renaming NewUpstream to New as we already are in the
upstream package.
* plugin/forward: erase expired connection by timer
- in previous implementation, the expired connections resided in
cache until new request to the same upstream/protocol came. In
case if the upstream was unhealthy new request may come long time
later or may not come at all. All this time expired connections
held system resources (file descriptors, ephemeral ports). In my
fix the expired connections and related resources are released
by timer
- decreased the complexity of taking connection from cache. The list
of connections is treated as stack (LIFO queue), i.e. the connection
is taken from the end of queue (the most fresh connection) and
returned to the end (as it was implemented before). The remarkable
thing is that all connections in the stack appear to be ordered by
'used' field
- the cleanup() method finds the first good (not expired) connection
in stack with binary search, since all connections are ordered by
'used' field
* fix race conditions
* minor enhancement
* add comments
- connManager() goroutine will stop when Proxy is about to be
garbage collected. This means that no queries are in progress,
and no queries are going to come
* Remove Compress by default
Set Compress = true in Scrub only when the message doesn not fit the
advertized buffer. Doing compression is expensive, so try to avoid it.
Master vs this branch
pkg: github.com/coredns/coredns/plugin/cache
BenchmarkCacheResponse-2 50000 24774 ns/op
pkg: github.com/coredns/coredns/plugin/cache
BenchmarkCacheResponse-2 100000 21960 ns/op
* and make it compile
Rework the TestProxyClose - close the proxy in the *same* goroutine
as where we started it. Close channels as long as we don't get dataraces
(this may need another fix).
Move the Dial goroutine out of the connManager - this simplifies things
*and* makes another goroutine go away and removes the need for connErr
channels - can now just be dns.Conn.
Also:
Revert "plugin/forward: gracefull stop (#1701)"
This reverts commit 135377bf77.
Revert "rework TestProxyClose (#1735)"
This reverts commit 9e8893a0b5.
* plugin/forward: gracefull stop
- stop connection manager only when no queries in progress
* minor improvement
* prevent healthcheck on stopped proxy
* revert closing channels
* use standard context
* global: move to context
Move from golang.org/x/net/context to std lib's context.
Change done with:
for i in $(grep -l '/context' **/*.go); do sed -e 's|golang.org/x/net/context|context|' -i $i; echo $i; done
for i in **/*.go; do goimports -w $i; done
* drop from dns.pb.go as well
