* Clean up tests logging
This cleans up the travis logs so you can see the failures better.
Older tests in tests/ would call log.SetOutput(ioutil.Discard) in
a haphazard way. This add log.Discard and put an `init` function in each
package's dir (no way to do this globally). The cleanup in tests/ is
clear.
All plugins also got this init function to have some uniformity and kill
any (future) logging there in the tests as well.
There is a one-off in pkg/healthcheck because that does log.
Signed-off-by: Miek Gieben <miek@miek.nl>
* bring back original log_test.go
Signed-off-by: Miek Gieben <miek@miek.nl>
* suppress logging here as well
Signed-off-by: Miek Gieben <miek@miek.nl>
* DoH: put in pkg/doh
Factor out the DoH stuff into its own package, add function to request
a DoH response. This can be used by forward (and maybe proxy) to
implement DoH client support.
Signed-off-by: Miek Gieben <miek@miek.nl>
* lint
Signed-off-by: Miek Gieben <miek@miek.nl>
* ... and make it compile
Signed-off-by: Miek Gieben <miek@miek.nl>
This revert 17d807f0 and re-adds the metadata plugin as a plugin that
just sets a label to a value function.
Add package documentation on how to use the metadata package. Make it
clear that any caching is up to the Func implemented.
There are now - no in tree users. We could add the request metadata by
default under names that copy request.Request, i.e
request/ip - remote IP
request/port - remote port
Variables.go has been deleted.
Signed-off-by: Miek Gieben <miek@miek.nl>
* plugin/metadata: some cleanups
Name to provider.go as that's what being defined right now in the file.
Use request.Request because that's done in variables.go anyway. Name the
main storage M, because there is no further meaning behind.
Remove superfluous methods
Signed-off-by: Miek Gieben <miek@miek.nl>
* Fix test
Signed-off-by: Miek Gieben <miek@miek.nl>
* Fix max-age in http server
Move the minMsgTTL to dnsutil and rename it MinimalTTL, move some
constants there as well.
Use these new function in server_https to correctly set the max-age
HTTP header.
Fixes: #1823
* Linter
* Add part 1 watch functionality. (squashed)
* add funcs for service/endpoint fqdns
* add endpoints watch
* document exposed funcs
* only send subset deltas
* locking for watch map
* tests and docs
* add pod watch
* remove debugs prints
* feedback part 1
* add error reporting to proto
* inform clients of server stop+errors
* add grpc options param
* use proper context
* Review feedback:
* Removed client (will move to another repo)
* Use new log functions
* Change watchChan to be for string not []string
* Rework how k8s plugin stores watch tracking info to simplify
* Normalize the qname on watch request
* Add blank line back
* Revert another spurious change
* Fix tests
* Add stop channel.
Fix tests.
Better docs for plugin interface.
* fmt.Printf -> log.Warningf
* Move from dnsserver to plugin/pkg/watch
* gofmt
* remove dead client watches
* sate linter
* linter omg
add a test to see if we copy the rcode correctly. Some minor cleanup in
import ordering and renaming NewUpstream to New as we already are in the
upstream package.
* plugin/file: fix local CNAME lookup
Issue #1864 explains it will, when we serve the child zone as well we
should just recursive into ourself (upstream self). Thus relax the
IsSubDomain check in file/lookup.go and just query (even if the query
will hit a remote server).
I've looped over all other plugins that do something similar (CNAME
resolving) and they didn't do the IsSubDomain check; therefor I've
removed it from *file* as well.
Added test in file_upstream_test that shows this failed before but now
results in a reply.
Fixes#1864
* self does not need to be exported
* Fix test
We don't know if we had a valid reply. Check this.
Remove the code and remove the call in etcd and kubernetes handlers.
This does mean we should not add dups in the first place, which means
adding maps in backend_lookup to prevent dups from begin added.
This should cut down on the allocations because dnsutil.Dedup is very
expensive by converting everything to strings, we avoid doing that now.
* Current stage of the log files. Test need to be done as well as formatting of times.
* Finished testing. All altered classes test pass along with my additions
* Updated the replacer package to print the units as well. May take out.
* Changed the time units to be within the rules. Fixed the test as well.
* Fixed some tests, updated the readme, fixed the replacer class.
* Updates of standardizing only to seconds in response duration. Need to revert README.
* Reverted readme.
* Added a small test in new replacer.
* Changed replacer to inline the strconv for duration.
Uppercase all these test errors as well. And extend the presubmit to
check for these in the future. Also do a slightly smarter grep to only
get t.<something>. as (because dump regexp) this also grep over non test
files.
The DoH work (#1619) made changes to pkg/nonwriter.Writer that in
hindsight were not backwards compatible; it added override for the
LocalAddr() and RemoteAddr(). Instead of rolling back that PR, this PR
reverts those changes and creates a DoHWriter for use in the
https-server.go side of things.
This was only caught in the integration test making this hard to catch,
so we add a upstream_file_test.go that tries (doesn't work yet) to test
this in the unit tests as well. Esp. helpful when 'git bisecting'.
Fixes#1826
* WIP: make CoreDNS DoH Server
* It works
* Fix tests
* Review from Tom - on diff. PR
* correct mime type
* Cleanups and use the pkg/nonwriter
* rename and updates
* implement get
* implement GET
* Code review comments
* correct context
* tweaks
* code review
* Probe simplification
- the main reason of rework is that previous implementation hung
when calling Do() after Stop()
* replace atomics with mutex
* access Probe.interval under lock
* update docs
* plugins: use plugin specific logging
Hooking up pkg/log also changed NewWithPlugin to just take a string
instead of a plugin.Handler as that is more flexible and for instance
the Root "plugin" doesn't implement it fully.
Same logging from the reload plugin:
.:1043
2018/04/22 08:56:37 [INFO] CoreDNS-1.1.1
2018/04/22 08:56:37 [INFO] linux/amd64, go1.10.1,
CoreDNS-1.1.1
linux/amd64, go1.10.1,
2018/04/22 08:56:37 [INFO] plugin/reload: Running configuration MD5 = ec4c9c55cd19759ea1c46b8c45742b06
2018/04/22 08:56:54 [INFO] Reloading
2018/04/22 08:56:54 [INFO] plugin/reload: Running configuration MD5 = 9e2bfdd85bdc9cceb740ba9c80f34c1a
2018/04/22 08:56:54 [INFO] Reloading complete
* update docs
* better doc
Add per plugin logging to make it explicit what is logging, if you
include this package under the name clog (coredns log), you can do the
following:
log := clog.NewWithPlugin{whoami{}} // e.g.
And then just log.Info(...); these will then include the plugin ala:
[INFO] plugin/whoami: stuff
So we only need to init the logger and then just use it.
* global: move to context
Move from golang.org/x/net/context to std lib's context.
Change done with:
for i in $(grep -l '/context' **/*.go); do sed -e 's|golang.org/x/net/context|context|' -i $i; echo $i; done
for i in **/*.go; do goimports -w $i; done
* drop from dns.pb.go as well
* Update all plugins to use plugin/pkg/log
I wish this could have been done with sed. Alas manually changed all
callers to use the new plugin/pkg/log package.
* Error -> Info
* Add docs to debug plugin as well
* pkg/log: ability for debug logs
When the debug plugin is enabled all log.Debug calls will print to
standard; if not there are a noop (almost).
The log package wraps some standard log functions as well, so just
replacing "log" with "plugin/pkg/log" should be enough to use this
package.
* docs
* Add docs
* lint
* Test fallthrough to log pkg as well
* simple package - up test coverage
* add other log levels as well
* update docs
Rename the constants to IP4arpa and IP6arpa (shorter and exported) and
make IsReverse return the type of the reverse zone which could be handy
for some callers.
Also add tests for IsReverse()
* doc: some function/vars/const/package level updates
Various update that stood out while reading godoc.org for CoreDNS.
* Fix some misspellings as well
* Revert "pkg/typify: empty messages are OtherError (#1531)"
This reverts commit fc1d73ffa9.
* plugin/cache: add failsafeTTL
If we can not see what TTL we should put on a message to be cached, use
5 seconds as minimal TTL. We used to apply the maximum TTL to these
messages.
Messages with nothing in them are considered OtherError, they can not
serve any purpose for normal clients (i.e. dyn update or notifies might
have a use for them).
Also update a test in the cache plugin, so that we explicitaly test for
this case.
* plugin/forward: on demand healtchecking
Only start doing health checks when we encouner an error (any error).
This uses the new pluing/pkg/up package to abstract away the actual
checking. This reduces the LOC quite a bit; does need more testing, unit
testing and tcpdumping a bit.
* fix tests
* Fix readme
* Use pkg/up for healthchecks
* remove unused channel
* more cleanups
* update readme
* * Again do go generate and go build; still referencing the wrong forward
repo? Anyway fixed.
* Use pkg/up for doing the healtchecks to cut back on unwanted queries
* Change up.Func to return an error instead of a boolean.
* Drop the string target argument as it doesn't make sense.
* Add healthcheck test on failing to get an upstream answer.
TODO(miek): double check Forward and Lookup and how they interact with
HC, and if we correctly call close() on those
* actual test
* Tests here
* more tests
* try getting rid of host
* Get rid of the host indirection
* Finish removing hosts
* moar testing
* import fmt
* field is not used
* docs
* move some stuff
* bring back health_check
* maxfails=0 test
* git and merging, bah
* review
This adds a generic way of start a check function to check a backend.
This package can be used to kick off healthchecks. The package makes
sure only 1 is run at any one time.
It should allow for:
See upstream error -> kick off healthcheck
and not to worry about overwhelming the upstream with a barrage of
queries.
* add new policy always_first to mimic windows dns resolvers
fill documentation, add UT and cleanup fmt
* change name of policy from always_first to first. Update docs
HTTP healthchecking will be implicitely activated for proxy upstream
hosts, even if not configured. The README states that not using the
health_check directive will disable HTTP healthchecks though.
It seems to me that the availability of the HealthCheck.Path attribute
is used as indicator whether HTTP healthchecks should be used or not.
The normalizeCheckURL() function didn't check that attribute though,
always returning a CheckURL. This would increase the healthcheck failure
on every third failure in plugin/proxy, without any possibility for the
upstream host to be marked as healthy again. This would eventually
remove all upstream hosts from the serving pool.
* Add pkg/fall for Fallthrough
Move this into it's own package to facilitate tests. Important bug
was fixed: make the names fully qualified.
Add fall package to hosts, reverse, etcd, and fix kubernetes and any
tests. The k8s tests are still as-is, might need a future cleanup.
* dnstest: add multirecorder
This adds a new recorder that captures all messages written to it. This
can be useful when, for instance, testing AXFR which can write muliple
messages back to the client.
* docs
When we failing the healthcheck we should increate the fails for this
host; which is the *bleeping* point of doing the HC in the first place.
Add the missing atomic.Adds.
Fixes#1247
* core: allow all CIDR ranges in zone specifications
Allow (e.g.) a v4 reverse on a /17. If a zone is specified in such a
way a FilterFunc is set in the config. This filter is checked against
incoming queries.
For all other queries this adds a 'x != nil' check which will not impact
performace too much. Benchmark function is added as well to check for
this as wel.
Add multiple tests in tests/server_reverse_test.go.
Benchmark shows in the non-reverse case this hardly impact the speed:
~~~
classless:
pkg: github.com/coredns/coredns/core/dnsserver
BenchmarkCoreServeDNS-4 1000000 1431 ns/op 16 B/op 1 allocs/op
pkg: github.com/coredns/coredns/core/dnsserver
BenchmarkCoreServeDNS-4 1000000 1429 ns/op 16 B/op 1 allocs/op
master:
pkg: github.com/coredns/coredns/core/dnsserver
BenchmarkCoreServeDNS-4 1000000 1412 ns/op 16 B/op 1 allocs/op
pkg: github.com/coredns/coredns/core/dnsserver
BenchmarkCoreServeDNS-4 1000000 1429 ns/op 16 B/op 1 allocs/op
~~~
* README.md updates
* healthchecks: check on every 3rd failure
Check on every third failure and some cleanups to make this possible. A
failed healthcheck will never increase Fails, a successfull healthceck
will reset Fails to 0. This is a chance this counter now drops below 0,
making the upstream super? healthy.
This removes the okUntil smartness and condences everything back to 1
metrics: Fails; so it's simpler in that regard.
Timout errors are *not* attributed to the local upstream, and don't get
counted into the Fails anymore. Meaning the 'dig any isc.org' won't kill
your upstream.
Added extra test the see if the Fails counter gets reset after 3 failed
connection.
There is still a disconnect beween HTTP healthceck working the proxy (or
lookup) not being able to connect to the upstream.
* Fix tests
Fix file/fuzz.go build and docs in Makefile.fuzz
Each plugin can add a fuzz.go to join the fuzzing craze.
pkg/fuzz/do.go could be made a lot smarter, but is probably good enough
for starters.
$ make -f Makefile.fuzz <plugin>
will build with go-fuzz-build and then execute a go-fuzz run. Each
plugin's fuzz run uses a per-plugin directory to store the fuzz data.
