Commit graph

1250 commits

Author SHA1 Message Date
Miek Gieben
2c80551fdc
plugin/{kubernetes/etcd}: dynamic SOA (#1188)
Add a dynamic SOA record (at least the serial and minttl). This create
another interface that should be implemented by the backends.

For now default to returning epoch in as a uint32 (no change from
before).

Lower the minTTL returned to 30s (from 60s)
2017-11-01 10:11:34 +00:00
Miek Gieben
fa2ae3fb43 docs: updates some, remove others (#1187)
Fix typo in kubernetes/README.md and remove DEV-README.md as it is stale
and information on the website is more up to date.

Remove large sections of text in plugin.md; just talk about how to
structure your plugin and docs.
2017-10-31 09:33:41 -04:00
Pat Moroney
1d4ac4adbb add goroutine to check hosts file for updates (#1180)
* add goroutine to check hosts file for updates

* rename parseFile to parseReader, remove extra error check
2017-10-31 07:40:47 +00:00
Miek Gieben
87c9f00c83
readme: more tests (#1184)
* readme: more tests

Add dnssec and file plugin to the test readme. This requires creating a
bunch of files with the right content. Doing so already unconvered an
unconditional type assertion in DNSSEC. This PR will include the fix for
that as well.

Also extended the snippets in the file plugin README, so that they are
whole Corefile - showing more value and checking all corefile snippets.

Create outliner right now is the kubernetes plugin, because even setting
the right env vars will result in:

open /var/run/secrets/kubernetes.io/serviceaccount/token: no such file or directory":

Which we can't create for a test.

* lint
2017-10-31 07:14:49 +00:00
Miek Gieben
c2d93f7182 plugin/dnstap: some cleanup (#1172)
Some cleanup in proxy and dnstap:
* just use time pkg directly and side step the indirection for Epoch
* Use Set in SetQueryEpoch to be more Go like. (Looked like a reader)
* Don't maintain two sets of time, we already track start, so use that.
* Use time.Time and convert when needed
* dedent the toDnstap function and put in a separate file
2017-10-25 19:46:41 +01:00
Pat Moroney
25367a4329 remove the defer 2017-10-24 14:28:41 -06:00
Pat Moroney
53d9bff707 read lock around ReadHosts() 2017-10-24 13:46:58 -06:00
Pat Moroney
680e6bd5c3 move ReadHosts out of the lock 2017-10-24 10:34:16 -06:00
Pat Moroney
7ad99d0d36 Fix locking for hosts plugin 2017-10-24 10:27:07 -06:00
Miek Gieben
cc490a8912 plugin/kubernetes: lazy initialze EndPointsList (#1168)
If we don't need it, don't initialize it.

Fixes #1156
2017-10-24 12:44:34 +01:00
Miek Gieben
fcd0342e42 CIDR query routing (#1159)
* core: allow all CIDR ranges in zone specifications

Allow (e.g.) a v4 reverse on a /17. If a zone is specified in such a
way a FilterFunc is set in the config. This filter is checked against
incoming queries.

For all other queries this adds a 'x != nil' check which will not impact
performace too much. Benchmark function is added as well to check for
this as wel.

Add multiple tests in tests/server_reverse_test.go.

Benchmark shows in the non-reverse case this hardly impact the speed:

~~~
classless:
pkg: github.com/coredns/coredns/core/dnsserver
BenchmarkCoreServeDNS-4   	 1000000	      1431 ns/op	      16 B/op	       1 allocs/op

pkg: github.com/coredns/coredns/core/dnsserver
BenchmarkCoreServeDNS-4   	 1000000	      1429 ns/op	      16 B/op	       1 allocs/op

master:
pkg: github.com/coredns/coredns/core/dnsserver
BenchmarkCoreServeDNS-4   	 1000000	      1412 ns/op	      16 B/op	       1 allocs/op

pkg: github.com/coredns/coredns/core/dnsserver
BenchmarkCoreServeDNS-4   	 1000000	      1429 ns/op	      16 B/op	       1 allocs/op
~~~

* README.md updates
2017-10-24 10:16:03 +01:00
Miek Gieben
5aa0d55e72 plugin/pkg/cache: smarter locking (#1164)
Make the locking slightly smarter in Evict and add benchmark function.

Seems a bit faster (there was some variance while performing these
benchmarks)

Master:
BenchmarkCache-2   	 1000000	      2317 ns/op	       0 B/op	       0 allocs/op
BenchmarkCache-2   	 1000000	      2032 ns/op	       0 B/op	       0 allocs/op

This branch:
BenchmarkCache-2   	 1000000	      1806 ns/op	       0 B/op	       0 allocs/op
BenchmarkCache-2   	 1000000	      1809 ns/op	       0 B/op	       0 allocs/op
2017-10-23 17:24:48 +01:00
Miek Gieben
d64b684831 plugin/kubernetes: implement HasSynced() (#1155)
* plugin/kubernetes: wait until api is ready

Wait for HasSynced before allowing startup to avoid startup race.

Also do a small refactor in findServices() to pull a check out of the
loop - only needs to be done once.

* sigh
2017-10-20 22:53:17 +01:00
Miek Gieben
c1f67493de docs: less CoreDNS in docs (#1154)
Various other changes.
2017-10-20 09:47:43 +01:00
Miek Gieben
11203e440d plugin/dnssec; insert and sign DS records (#1153)
* plugin/dnssec; insert and sign DS records

Sign a delegation as well and insert DS records.

Fixes #698

* better
2017-10-20 09:22:02 +01:00
Miek Gieben
73d702c052 plugin/metrcs: add env variable example (#1151)
We support ENV variables in config files, add an example in the one for
metric, note that `localhost:` is a valid host (in Go), so the
TestReadme will parse this correctly.

Fixes #1150
2017-10-18 17:20:27 +01:00
Sandeep Rajan
b6b05eae8f Plugin/Kubernetes: Service and Endpoint Indexing (#1149)
* indexing

* corrections
2017-10-17 21:30:54 -04:00
Miek Gieben
70ee39844e plugin/autopath: Add metrics and remove log line (#1143)
* plugin/autopath: Add namespace selector and metrics

Add a namespace, so autopathing only is performed in this namespace.
This will make caching work for the cluster again.

Also export metrics that we've done a successful autopath

* dont shadow

* Fix

* Back the namespacing changes
2017-10-15 19:39:24 +02:00
Miek Gieben
e34e2c251f plugin/proxy: kick of HC on every 3rd failure (#1110)
* healthchecks: check on every 3rd failure

Check on every third failure and some cleanups to make this possible. A
failed healthcheck will never increase Fails, a successfull healthceck
will reset Fails to 0. This is a chance this counter now drops below 0,
making the upstream super? healthy.

This removes the okUntil smartness and condences everything back to 1
metrics: Fails; so it's simpler in that regard.

Timout errors are *not* attributed to the local upstream, and don't get
counted into the Fails anymore. Meaning the 'dig any isc.org' won't kill
your upstream.

Added extra test the see if the Fails counter gets reset after 3 failed
connection.

There is still a disconnect beween HTTP healthceck working the proxy (or
lookup) not being able to connect to the upstream.

* Fix tests
2017-10-15 19:38:39 +02:00
Miek Gieben
c7ff44fb3a plugin/cache: don't update freq is prefetch isn't used. (#1144)
Small optimization.
2017-10-13 17:22:08 +02:00
Miek Gieben
7a64d1bbc3 plugin/cache: add minttl test (#1141)
See https://github.com/kubernetes/kubernetes/pull/53604, explicitaly add
test to make sure we do the right thing.
2017-10-10 18:30:14 +02:00
Miek Gieben
427aed6f5b doc update (#1140)
* doc update

Go through all README and fix mistakes, extend example and let more
corefile snippets be test for validity.

* Cant use spefic addr in test
2017-10-10 09:39:35 +02:00
Miek Gieben
7c6ba3fcbd plugin/proxy: fix metrics (#1137)
Add Counter metrics and fix duration to use upstream name (and only use
it when we have one).

Fix the documentation to reflect this.

Fixes #1134
2017-10-08 04:30:44 -07:00
Isolus
c1b9f74f98 plugin/dnssec: fix NSEC-records (#1138)
* add NSEC-records if entry exists but the requested type is missing

* added test for dnssec fix
2017-10-08 04:28:35 -07:00
Miek Gieben
b6086ca8f1 plugin/whoami (#1132)
Make it work with root zone in the query.
2017-10-05 06:14:02 -07:00
John Belamaric
37d06f382a Warn if the hosts file is a directory (#1126) 2017-09-29 22:28:37 +01:00
Miek Gieben
4276d29b81 Add fuzzing infrastructure (#1118)
Fix file/fuzz.go build and docs in Makefile.fuzz
Each plugin can add a fuzz.go to join the fuzzing craze.
pkg/fuzz/do.go could be made a lot smarter, but is probably good enough
for starters.

$ make -f Makefile.fuzz <plugin>

will build with go-fuzz-build and then execute a go-fuzz run. Each
plugin's fuzz run uses a per-plugin directory to store the fuzz data.
2017-09-29 22:28:13 +01:00
Thong Huynh
2f9c42d82e Enable dnstap plugin to insert other plugin's specific data into extra field of tap.Dnstap message (#1101)
* Add custom data into dnstap context

* Fix error and fix UT compile errors

* Add UTs

* Change as per review comments.  Use boolean to indicate which Dnstap message to send out

* Merge with master and fix lint warning

* Remove newline

* Fix review comments
2017-09-29 16:38:01 -04:00
Chris O'Haver
4b3a430ff2 plugin/kubernetes: Enable protobuf, Update client api package (#1114)
* vendor

* code
2017-09-29 15:58:50 -04:00
varyoo
45b0252c1a plugin/dnstap: don't spam Travis + valid dnstap payload (#1121)
* don't spam Travis + valid dnstap payload

* log instead of fmt

* Revert "log instead of fmt"

This reverts commit 88f09c3939.

* log the right way

* log the final way

* minor enhancements
2017-09-29 20:29:33 +01:00
Miek Gieben
1e71d0e2c1 plugin/cache: don't cache msg with TTL=0 in them (#1116)
Don't cache these - may be lead to weird side effects.

Fixes #1113
2017-09-28 14:22:23 +01:00
Miek Gieben
44a0cb9557 test: fix zone file (#1112)
* test: fix zone file

Add missing $TTL 3600 as the top.

Recent change in Go DNS made this mandatory - old behavior wasn't
standards compliant.

* and here
2017-09-26 19:41:49 +01:00
varyoo
daf8ef0da8 Adds the dnstap I/O routines and should fix some issues (#1083)
* adds the dnstap I/O thread and should fix a lot of mistakes

* docs

* -race test

* oops

* docs
2017-09-26 11:45:33 -04:00
Miek Gieben
2a32cd4159 plugin/proxy: decrease health timeouts (#1107)
Turn down the timeouts and numbers a bit:
FailTimeout 10s -> 5s
Future 60s -> 12s
TryDuration 60s -> 16s
The timeout for decrementing the fails in a host: 10s -> 2s

And the biggest change: don't set fails when the error is Timeout(),
meaning we loop for a bit and may try the same server again, but we
don't mark our upstream as bad, see comments in proxy.go. Testing this
with "ANY isc.org" and "MX miek.nl" we see:

~~~
::1 - [24/Sep/2017:08:06:17 +0100] "ANY IN isc.org. udp 37 false 4096" SERVFAIL qr,rd 37 10.001621221s
24/Sep/2017:08:06:17 +0100 [ERROR 0 isc.org. ANY] unreachable backend: read udp 192.168.1.148:37420->8.8.8.8:53: i/o timeout

::1 - [24/Sep/2017:08:06:17 +0100] "MX IN miek.nl. udp 37 false 4096" NOERROR qr,rd,ra,ad 170 35.957284ms

127.0.0.1 - [24/Sep/2017:08:06:18 +0100] "ANY IN isc.org. udp 37 false 4096" SERVFAIL qr,rd 37 10.002051726s
24/Sep/2017:08:06:18 +0100 [ERROR 0 isc.org. ANY] unreachable backend: read udp 192.168.1.148:54901->8.8.8.8:53: i/o timeout

::1 - [24/Sep/2017:08:06:19 +0100] "MX IN miek.nl. udp 37 false 4096" NOERROR qr,rd,ra,ad 170 56.848416ms
127.0.0.1 - [24/Sep/2017:08:06:21 +0100] "MX IN miek.nl. udp 37 false 4096" NOERROR qr,rd,ra,ad 170 48.118349ms
::1 - [24/Sep/2017:08:06:21 +0100] "MX IN miek.nl. udp 37 false 4096" NOERROR qr,rd,ra,ad 170 1.055172915s
~~~

So the ANY isc.org queries show up twice, because we retry internally -
this is I think WAI.

The `miek.nl MX` queries are just processed normally as no backend is
marked as unreachable.

May fix #1035 #486
2017-09-24 20:05:36 +01:00
Miek Gieben
148a99442d healhcheck: various cleanups (#1106)
* healhcheck: various cleanups

Network wasn't used. IgnorePaths wasn't used. Move checkdown function to
common function shared between proxy protocols. And some naming fixed.

Also reset the Fails on a succesful healthcheck back to 0.

remove newlines from log

* compile

* fix test
2017-09-24 19:37:43 +01:00
Yong Tang
102cfbd7fe Use gometalinter and enforcing go fmt/lint/vet (#1108)
* Use gometalinter and enforcing go fmt/lint/vet

Before this PR go fmt is enabled, go lint is suggest only.
From time to time we have to manually check for go lint and go vet
for any issues.

This fix uses gometalinter and enforcing go fmt/lint/vet.
Several reasons:
- gometalinter could handle multiple linters concurrently
- gometalinter supports suppression with `// nolint[: <linter>]`

Previously one reason we didn't enable go lint was due to the
```
warning: context.Context should be the first parameter of a function (golint)
```
this is now possible with gometalinter and `// nolint: golint` (See changes).

This fix also discovered several go vet issues and fixes it.

Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

* Fix several issues reported by gometalinter (go vet)

This commit fixes several issues reported by gometalinter (go vet).

Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

* Increase deadline

Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
2017-09-24 11:59:04 +01:00
Yong Tang
387ceb480b go lint/vet/misspell cleanup (#1105)
This fix address several issues related to golint/vet/misspell.

Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
2017-09-22 16:13:04 -07:00
John Belamaric
6f19dda0b4 Fix autopath crash when pods verified not enabled (#1099) 2017-09-21 18:06:35 +01:00
Miek Gieben
284061eee7 pkg: add dnstest (#1098)
Add a full test server impl in this new package + tests. Move
dnsrecorder into this package as well and finish up the commented out
tests that were left in the old dnsrecorder package.

Update all callers and tests.
2017-09-21 15:15:47 +01:00
Yong Tang
7109c6715c Add inline support for middleware/hosts (#1072)
This fix add inline support for middleware/hosts so that
it is possible to specify hosts file insides the Corefile:
```
hosts inline example.org {
    10.0.0.1 example.org
    fallthrough
}
```

This fix fixes 999.

Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
2017-09-21 04:18:13 -07:00
Damian Myerscough
aecf916377 Fixing a small typo (#1097) 2017-09-21 07:22:13 +01:00
Thong Huynh
ec21f83425 Modify the rewrite plugin to write multiple EDNS0 options (#936) (#1096)
* Add processing mode

* Add processing mode

* Update UTs

* Update README.md

* Change to use the constant Stop

* Fix README per review comments
2017-09-20 16:06:53 -04:00
Miek Gieben
36c7aa6437 plugin/{file,auto}: drop fsnotify (#1090)
* plugin/{file,auto}: drop fsnotify

Reload every minute. This is more deterministic then fsnotify. Also
other thing cropped up: sharing zone files between zone; there is only
1 fsnotify event and we need to fan out the reload to all zone files.
This is a large rewrite (which could still be done), for now, poll the
zone file on disk.

Give serial no change a special error type so we can check for this.
Improve the logging for reloading:

2017/09/19 07:34:39 [INFO] Successfully reloaded zone "miek.nl." in "db.miek.nl" with serial 128263060
2017/09/19 07:34:45 [INFO] Successfully reloaded zone "miek.nl." in "db.miek.nl" with serial 128263059
2017/09/19 07:34:51 [INFO] Successfully reloaded zone "miek.nl." in "db.miek.nl" with serial 128263060

Fixes #1013

* typo
2017-09-20 17:28:23 +01:00
Miek Gieben
cd5879f866 plugin/cache: cap TTL on first answer (#1092)
Cache would let the first response through and would then cap subsequent
ones to whatever the cache duration was. This would lead to huge drops
in TTL values: 3600 -> 20 for instance, which is not only bad, but can
mess up your careful TTL planning business.

This PR fixes that and applies the cache duration to all replies. As a
bonus I could remove a time.Sleep() from the cache test and just check
for the cache duration as the TTL on the reply.

Fixes #1038
2017-09-20 11:36:41 +01:00
Miek Gieben
be47709270 More Middleware -> Plugin conversions (#1088)
Forgot about these.
2017-09-16 14:13:28 +01:00
Yong Tang
c79a4769e4 golint cleanup (#1085)
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
2017-09-15 23:34:02 +01:00
Miek Gieben
2388e36c2c plugin: README.md updates (#1084)
updates so the look better on coredns.io
2017-09-15 22:27:55 +01:00
Miek Gieben
19d7d207d3 Make proxy examples better (#1080) 2017-09-15 11:30:10 +01:00
Miek Gieben
ae9cae67e3 plugins: expand examples (#1077)
* plugins: expand examples

Expand some examples and test them by annotating it with `corefile`

* Can't start kubernetes

* Can test pprof :(
2017-09-15 09:56:05 +01:00
Miek Gieben
d8714e64e4 Remove the word middleware (#1067)
* Rename middleware to plugin

first pass; mostly used 'sed', few spots where I manually changed
text.

This still builds a coredns binary.

* fmt error

* Rename AddMiddleware to AddPlugin

* Readd AddMiddleware to remain backwards compat
2017-09-14 09:36:06 +01:00