Commit graph

1208 commits

Author SHA1 Message Date
Miek Gieben
f3134da45e
Clean up tests logging (#1979)
* Clean up tests logging

This cleans up the travis logs so you can see the failures better.

Older tests in tests/ would call log.SetOutput(ioutil.Discard) in
a haphazard way. This add log.Discard and put an `init` function in each
package's dir (no way to do this globally). The cleanup in tests/ is
clear.

All plugins also got this init function to have some uniformity and kill
any (future) logging there in the tests as well.

There is a one-off in pkg/healthcheck because that does log.

Signed-off-by: Miek Gieben <miek@miek.nl>

* bring back original log_test.go

Signed-off-by: Miek Gieben <miek@miek.nl>

* suppress logging here as well

Signed-off-by: Miek Gieben <miek@miek.nl>
2018-07-19 16:23:06 +01:00
Miek Gieben
c69bed726b plugin/hosts: add log.Debug (#1985)
Logs the amount of entries we parsed after every parse, but only when
debug is loaded.

Signed-off-by: Miek Gieben <miek@miek.nl>
2018-07-19 07:16:35 -07:00
Jiacheng Xu
6621931620 plugin/etcd: Add SRV and TXT examples. (#1980) 2018-07-19 07:33:31 +01:00
Nitish Tiwari
a5fe423188 Update etcd plugin documentation wrt etcdv3 support (#1974)
Fixes #1960
2018-07-15 22:50:53 +01:00
Miek Gieben
d9b9a955ba plugin/rewrite: prevent illegal names (#1972)
Log and returns an error when the name rewrite creates a name that is
illegal. Add test in name_test.go to see if an error is returned.

Possible followup could be the only check this if a name-rewrite is
done.

Fixes: #1638

Signed-off-by: Miek Gieben <miek@miek.nl>
2018-07-13 09:32:07 -04:00
Miek Gieben
a536833546
plugin/forward: add HealthChecker interface (#1950)
* plugin/forward: add HealthChecker interface

Make the HealthChecker interface and morph the current DNS health
checker into that interface.

Remove all whole bunch of method on Forward that didn't make sense.

This is done in preparation of adding a DoH client to forward - which
requires a completely different healthcheck implementation (and more,
but lets start here)

Signed-off-by: Miek Gieben <miek@miek.nl>

* Use protocol

Signed-off-by: Miek Gieben <miek@miek.nl>

* Dial doesnt need to be method an Forward either

Signed-off-by: Miek Gieben <miek@miek.nl>

* Address comments

Address various comments on the PR.

Signed-off-by: Miek Gieben <miek@miek.nl>
2018-07-09 15:14:55 +01:00
Miek Gieben
4083852b70 Remove trailing whitespace (#1955)
Prevent future; "remove trailing whitespace" PR, but adding a simple
presubmit that checks for this.

This presubmit flagged quite some offenders, remove all trailing
whitespace from. Apart from that there aren't any other changes.

Signed-off-by: Miek Gieben <miek@miek.nl>
2018-07-09 08:08:02 -04:00
Miek Gieben
b27a59c160
plugin/metadata: finish documentation (#1951)
* plugin/metadata: finish documentation

Finish the README.md, add corner case in the IsLabel test and reword
some code comments slightly.

Generate the man-pages and add man/coredns-metadata.7 as well.

Signed-off-by: Miek Gieben <miek@miek.nl>

* Fix test

Signed-off-by: Miek Gieben <miek@miek.nl>
2018-07-09 07:58:14 +01:00
Francois Tur
b3a92f1622 - compliance with metadata contract. If metadata value is empty, ignore rewrite - like if metadata does not exist. (#1953) 2018-07-08 17:38:46 -04:00
Francois Tur
7745462430 plugin/rewrite - extend edns0 local variable support with metadata (#1928)
* - add support of metadata values for edns0 local variables

* - comments from review.

* - simplify label check. Add UT

* - enhance check for Labels, add UT
- remove IsMetadataSet

* - edns0 variable - if variable is not found just ignore the rewrite.
2018-07-08 08:18:01 +01:00
Miek Gieben
6ec1978340
plugin/forward: various cleanup (#1949)
Fix documentation and remove the unused From method.

Signed-off-by: Miek Gieben <miek@miek.nl>
2018-07-07 14:38:05 +01:00
Miek Gieben
41c2871907
plugin/test: Fix documentation (#1948)
Fix documentation and touch up plugin/forward/README.md

Signed-off-by: Miek Gieben <miek@miek.nl>
2018-07-07 08:30:57 +01:00
Miek Gieben
30a788fd3a
Doh: put in pkg/doh (#1946)
* DoH: put in pkg/doh

Factor out the DoH stuff into its own package, add function to request
a DoH response. This can be used by forward (and maybe proxy) to
implement DoH client support.

Signed-off-by: Miek Gieben <miek@miek.nl>

* lint

Signed-off-by: Miek Gieben <miek@miek.nl>

* ... and make it compile

Signed-off-by: Miek Gieben <miek@miek.nl>
2018-07-07 08:22:07 +01:00
Ruslan Drozhdzh
bc50901234 plugin/forward: add prefer_udp option (#1944)
* plugin/forward: add prefer_udp option

* updated according to code review

 - fixed linter warning
 - removed metric parameter in Proxy.Connect()
2018-07-07 08:14:21 +01:00
Miek Gieben
7c41f2ce9f plugin/loadbalance: add parse and tests (#1947)
Automatically submitted.
2018-07-06 21:49:21 +00:00
Miek Gieben
bcc749db04
Docs (#1943)
* Doc updates

Make the name section fit on one line.

Signed-off-by: Miek Gieben <miek@miek.nl>

* Regen docs

Signed-off-by: Miek Gieben <miek@miek.nl>
2018-07-06 11:27:40 +01:00
Miek Gieben
195fe5d1a4
plugin/metadata: adjust doc to latest code. (#1923)
* plugin/metadata: adjust doc to latest code.

Signed-off-by: Miek Gieben <miek@miek.nl>
2018-07-06 09:41:37 +01:00
Marcus André
826d878084 Fix typo in comment (#1939)
Automatically submitted.
2018-07-05 14:29:54 +00:00
Miek Gieben
e6f81ebb31
Add debug.Hexdump (#1902)
Allow plugins to dump messages in text pcap to the log. The forward
plugin does this when a reply does not much the query.

If the debug plugin isn't loaded Hexdump and Hexdumpf are noop.

Signed-off-by: Miek Gieben <miek@miek.nl>
2018-07-04 07:54:17 +01:00
Isolus
063e673bc4 plugin/file: fix zone expiration (#1933) 2018-07-03 17:22:00 +01:00
Eren Güven
ee3c9894d9 fix minor typo (#1932) 2018-07-03 15:57:49 +01:00
Miek Gieben
2aa1bda005 plugin/cache: add extra test for FORMERR (#1930)
* plugin/cache: add extra test for FORMERR

Add extra test that test for not caching a formerr.

Signed-off-by: Miek Gieben <miek@miek.nl>

* govet

Signed-off-by: Miek Gieben <miek@miek.nl>
2018-07-03 04:00:22 -07:00
Joey Espinosa
c2780f42c4 Updating filepath in auto plugin if existing zone's file changes location (#1901) (#1910)
Exporting Zone.File to avoid getters and setters

Updating getter and setter for Zone.File to be less racy

Renaming GetFile to File in zone plugin
2018-07-03 11:20:39 +01:00
Miek Gieben
6dd2cf8c4b plugin/rewrite: use request.Request and other cleanups (#1920)
This was done anyway, but only deep in the functions, just do this
everywhere; allows for shorter code and request.Request allows for
caching as well.

Cleanups, make it more Go like.
* remove unneeded switches
* remove testdir (why was this there??)
* simplify the logic
* remove unneeded variables
* put short functions on a single line
* fix documentation.
* spin off wire funcs in wire.go, make them functions.

Signed-off-by: Miek Gieben <miek@miek.nl>
2018-07-02 07:39:50 -07:00
Miek Gieben
99800a687c
plugin/metadata: metadata is just label=value (#1914)
This revert 17d807f0 and re-adds the metadata plugin as a plugin that
just sets a label to a value function.

Add package documentation on how to use the metadata package. Make it
clear that any caching is up to the Func implemented.

There are now - no in tree users. We could add the request metadata by
default under names that copy request.Request, i.e

request/ip - remote IP
request/port - remote port

Variables.go has been deleted.

Signed-off-by: Miek Gieben <miek@miek.nl>
2018-07-01 20:01:17 +01:00
Miek Gieben
37cdbff203 Add nitisht to etcd OWNERS (#1919)
Signed-off-by: Miek Gieben <miek@miek.nl>
2018-07-01 08:22:37 -07:00
Nitish Tiwari
264c89d7ad Document two lookups by CoreDNS etcdv3 plugin (#1918) 2018-07-01 07:32:16 +01:00
Miek Gieben
fcc32a79d4
plugin/metadata: add OWNERS file (#1915)
Signed-off-by: Miek Gieben <miek@miek.nl>
2018-06-30 22:01:31 +01:00
Miek Gieben
0a43306871
plugin/rewrite: cleanup (#1916)
delete unused tests and fix import lines.

Signed-off-by: Miek Gieben <miek@miek.nl>
2018-06-30 21:55:38 +01:00
Nitish Tiwari
6fe27d99be plugin/etcdv3: Add etcd v3 plugin (#1702)
* Update dependencies and add etcdv3 client

* Update etcd plugin to support etcd v3 clients

Fixes #341
2018-06-30 16:19:13 +01:00
Miek Gieben
2fd31cd3e0
plugin/metadata: some cleanups (#1906)
* plugin/metadata: some cleanups

Name to provider.go as that's what being defined right now in the file.
Use request.Request because that's done in variables.go anyway. Name the
main storage M, because there is no further meaning behind.

Remove superfluous methods

Signed-off-by: Miek Gieben <miek@miek.nl>

* Fix test

Signed-off-by: Miek Gieben <miek@miek.nl>
2018-06-29 15:03:25 +01:00
Eugen Kleiner
17d807f05f plugin/metadata: add metadata plugin (#1894)
* plugin/metadata: add metadata plugin

* plugin/metadata: Add MD struct, refactor code, fix doc

* plugin/metadata: simplify metadata key

* plugin/metadata: improve setup_test

* Support of metadata by rewrite plugin. Move calculated variables to metadata.

* Move variables from metadata to pkg, add UTs, READMEs change, metadata small fixes

* Add client port validation to variables_test

* plugin/metadata: improve README

* plugin/metadata: rename methods

* plugin/metadata: Update Metadataer interface, update doc, cosmetic code changes

* plugin/metadata: move colllisions check to OnStartup(). Fix default variables metadataer.

* plugin/metadata: Fix comment for method setValue

* plugin/metadata: change variables order to fix linter warning

* plugin/metadata: rename Metadataer to Provider
2018-06-29 10:44:16 +01:00
Miek Gieben
dae506b563
Fix max-age in http server (#1890)
* Fix max-age in http server

Move the minMsgTTL to dnsutil and rename it MinimalTTL, move some
constants there as well.
Use these new function in server_https to correctly set the max-age
HTTP header.

Fixes: #1823

* Linter
2018-06-27 21:12:27 +01:00
John Belamaric
99287d091c
Watch feature (#1527)
* Add part 1 watch functionality. (squashed)

* add funcs for service/endpoint fqdns

* add endpoints watch

* document exposed funcs

* only send subset deltas

* locking for watch map

* tests and docs

* add pod watch

* remove debugs prints

* feedback part 1

* add error reporting to proto

* inform clients of server stop+errors

* add grpc options param

* use proper context

* Review feedback:
 * Removed client (will move to another repo)
 * Use new log functions
 * Change watchChan to be for string not []string
 * Rework how k8s plugin stores watch tracking info to simplify
 * Normalize the qname on watch request

* Add blank line back

* Revert another spurious change

* Fix tests

* Add stop channel.
Fix tests.
Better docs for plugin interface.

* fmt.Printf -> log.Warningf

* Move from dnsserver to plugin/pkg/watch

* gofmt

* remove dead client watches

* sate linter

* linter omg
2018-06-27 07:45:32 -07:00
Jun Li
b7480d5d12 plugin/cache: Fix: metric cache miss total shall include ttl case (#1897)
In the case of ttl <= 0, `cacheMisses` should increase its counter.

Signed-off-by: Jun Li <lijun.git@gmail.com>
2018-06-26 09:11:17 +02:00
Ruslan Drozhdzh
1ef9e31262 retry creating dnstest.Server if tcp or udp port is in use (#1799)
* retry creating dnstest.Server if tcp or udp port is in use

* changes according to review
2018-06-22 10:54:40 +01:00
Chris O'Haver
716791aa09
plugin/template: fix CNAME upstream handling (#1886) 2018-06-21 14:38:29 -04:00
Miek Gieben
ad8021230c Makefile fixes: (#1892)
Correctly set the path for the presubmits, fix some typos and make
the goimport target do the linting to the lint target can be removed.

Also don't make it a fatal error because gofmt changes between releases.
2018-06-21 04:34:02 -07:00
Tobias Schmidt
422aec5f5f plugin/forward: Increase minimum read timeout to 200ms (#1889)
After several experiments at SoundCloud we found that the current
minimum read timeout of 10ms is too low. A single request against a
slow/unavailable authoritative server can cause all TCP connections to
get closed. We record a 50th percentile forward/proxy latency of <5ms,
and a 99th percentile latency of 60ms. Using a minimum timeout of 200ms
seems to be a fair trade-off between avoiding unnecessary high
connection churn and reacting to upstream failures in a timely manner.

This change also renames hcDuration to hcInterval to reflect its usage,
and removes the duplicated timeout constant to make code comprehension
easier.
2018-06-21 11:40:19 +01:00
Tobias Schmidt
e3534205c7 Rename forward metrics socket_count_total to sockets_open (#1885)
The prometheus naming convention states only counters should have a
`_total` suffix, so that gagues and counters can be easily
distinguished.
2018-06-20 11:35:57 +01:00
Miek Gieben
41c724780f
remove deprecated code (#1888)
This removes:
* reverse plugin from plugin.cfg
* https_google option from proxy
* the -log flag
2018-06-20 11:35:07 +01:00
Tobias Schmidt
9c2dc7a156 Ensure cache.ResponseWriter can be used asynchronously during prefetch (#1884)
The default dns.Response implementation of a dns.ResponseWriter will
panic if RemoteAddr() is called after the connection to the client has
been closed already. The current cache implementation doesn't create a
new request+responsewriter during an asynchronous prefetch, but
piggybacks on the request triggering the prefetch.

This change copies the RemoteAddr first, so that it's safe to use it
later during the actual prefetch request.

A better implementation would be to completely decouple the prefetch
request from the client triggering a request.
2018-06-19 19:50:08 +01:00
Miek Gieben
ab9efdcac4
Follow up on #1867 : use one map in backend_lookup (#1869)
Reduce map usage and combine two maps into one; add isDuplicate that
tests and adds to make things more readable.
2018-06-18 11:16:56 +01:00
Francois Tur
6fbc1f8990 Plugin/Rewrite - review documentation of "append" (#1877)
* - ensure description of 'append' match what is encoded.

* - fix description based on review comments
2018-06-18 10:02:05 +01:00
Miek Gieben
177e32b62e plugin/forward: add REFUSED test (#1878)
add a test to see if we copy the rcode correctly. Some minor cleanup in
import ordering and renaming NewUpstream to New as we already are in the
upstream package.
2018-06-15 08:12:56 -07:00
Francois Tur
70c957d885 Plugin/Forward - autotune the dialTimeout for connection (#1852)
* - implement an auto-tunable dialTimeout for fallback.

* - fix gofmt

* - factorized timeout computation with readTimeout / updated readme /

* - fix comment
2018-06-15 07:37:22 +01:00
Miek Gieben
26c41a0c17
plugin/file: fix local CNAME lookup (#1866)
* plugin/file: fix local CNAME lookup

Issue #1864 explains it will, when we serve the child zone as well we
should just recursive into ourself (upstream self). Thus relax the
IsSubDomain check in file/lookup.go and just query (even if the query
will hit a remote server).

I've looped over all other plugins that do something similar (CNAME
resolving) and they didn't do the IsSubDomain check; therefor I've
removed it from *file* as well.

Added test in file_upstream_test that shows this failed before but now
results in a reply.

Fixes #1864

* self does not need to be exported

* Fix test

We don't know if we had a valid reply. Check this.
2018-06-12 14:54:37 +01:00
Miek Gieben
6e466d5092 Remove dnsutil.Dedup (#1867)
Remove the code and remove the call in etcd and kubernetes handlers.
This does mean we should not add dups in the first place, which means
adding maps in backend_lookup to prevent dups from begin added.

This should cut down on the allocations because dnsutil.Dedup is very
expensive by converting everything to strings, we avoid doing that now.
2018-06-11 19:23:25 -07:00
Malcolm Akinje
18b11fc851 Normalizing Response Duration in Log Plugin (#1860)
* Current stage of the log files. Test need to be done as well as formatting of times.

* Finished testing. All altered classes test pass along with my additions

* Updated the replacer package to print the units as well. May take out.

* Changed the time units to be within the rules. Fixed the test as well.

* Fixed some tests, updated the readme, fixed the replacer class.

* Updates of standardizing only to seconds in response duration. Need to revert README.

* Reverted readme.

* Added a small test in new replacer.

* Changed replacer to inline the strconv for duration.
2018-06-07 16:21:17 +01:00
Miek Gieben
751a08d6a2
plugin/forward: fix alignment for sync.Atomic (#1855)
These must be alligned on 8 bytes, in Go this means putting them first
in the struct (AFAICT).
2018-06-05 17:21:09 +01:00