Commit graph

128 commits

Author SHA1 Message Date
Maxime Ginters
b1173ed2a5
plugin/forward Add rcode and rtype to request_duration_seconds metric (#4391)
* plugin/forward Add rcode and rtype to request_duration_seconds metric

Signed-off-by: Maxime Ginters <maxime.ginters@shopify.com>

* Control the cardinality of query type

Signed-off-by: Maxime Ginters <maxime.ginters@shopify.com>
2021-01-28 16:37:17 +01:00
Frank Riley
eba74389c4
Fix #4395, fix out of order messages and fix forward perspective. (#4396)
Signed-off-by: Frank Riley <fhriley@gmail.com>
2021-01-21 10:00:27 +01:00
Chris O'Haver
9cb53487ec
respond with REFUSED when max_concurrent is exceeded to avoid caching it (#4326)
Signed-off-by: Chris O'Haver <cohaver@infoblox.com>
2020-12-15 14:02:15 +01:00
Miek Gieben
8759d00edd
forward doc update (#4254)
* forward: add example with multiple DoT upstreams

Remove Bugs section as this is a nice work around.

h/t https://twitter.com/mholt6/status/1284250606673080321

Signed-off-by: Miek Gieben <miek@miek.nl>

* Actually remove bugs section

Signed-off-by: Miek Gieben <miek@miek.nl>
2020-11-03 06:32:49 -08:00
Miek Gieben
c840caf1ef
Speed up testing (#4239)
* Speed up testing

* make notification run in the background, this recudes the test_readme
time from 18s to 0.10s
* reduce time for zone reload

* TestServeDNSConcurrent remove entirely. This took a whopping 58s for
  ... ? A few minutes staring didn't reveal wth it is actually testing.
  Making values smaller revealed race conditions in the tests. Remove
  entirely.

* Move many interval values to variables so we can reset them to short
  values for the tests.

* test_large_axfr: make the zone smaller. The number used 64K has no
  rational, make it 64/10 to speed up.
* TestProxyThreeWay: use client with shorter timeout

A few random tidbits in other tests.

Total time saved: 177s (almost 3m) - which makes it worthwhile again to
run the test locally:

this branch:

~~~
ok  	github.com/coredns/coredns/test	10.437s
cd plugin; time go t ./...
5,51s user 7,51s system 11,15s elapsed 744%CPU (
~~~

master:

~~~
ok  	github.com/coredns/coredns/test	35.252s
cd plugin; time go t ./...
157,64s user 15,39s system 50,05s elapsed 345%CPU ()
~~~
tests/ -25s
plugins/ -40s

This brings the total on 20s, and another 10s can be saved by fixing
dnstapio. Moving this to 5s would be even better, but 10s is also nice.

Signed-off-by: Miek Gieben <miek@miek.nl>

* Also 0.01

Signed-off-by: Miek Gieben <miek@miek.nl>
2020-10-30 10:27:04 +01:00
Miek Gieben
c2e4f2f1ab
docs: move Also See to See Also (#4245)
sed -i 's/Also See/See Also/' plugin/**/README.md

Some plugins did already use 'See Also', so it's all consistent now.

Fixes: #4196

Signed-off-by: Miek Gieben <miek@miek.nl>
2020-10-28 10:56:35 -07:00
ZouYu
c58e4b09fc
Fix golint warnings (#4241)
Include:
1. plugin/forward/type.go:8:2: const typeUdp should be typeUDP
2. plugin/forward/type.go:9:2: const typeTcp should be typeTCP
3. plugin/forward/type.go:10:2: const typeTls should be typeTLS
4. plugin/kubernetes/metrics.go:24:2: var DnsProgrammingLatency should be DNSProgrammingLatency
5. plugin/kubernetes/metrics_test.go:124:102: func parameter clusterIp should be clusterIP

Signed-off-by: zouyu <zouy.fnst@cn.fujitsu.com>
2020-10-28 07:39:56 +01:00
Miek Gieben
b3b8a7e4b7
plugin/dnstap: various cleanups (#4179)
* plugin/dnstap: various cleanups

A recent issue made me look into this plugin, I suspect various other
cleanups (hopefully deletion of code) can be made as well

Remove identical functions ToClientQuery etc, and just use tap.Message
as the base type in plugin. Keep msg/ for a few helper functions that
may proof useful.

This remove the whole test directory as we will just check the things we
are interested in which gives much better feedback and keeps that code
closer together.

tapwr dir is also not needed, writer_test.go was just duplicating the
tests already done. This moves writer.go to the top directory.

Make the only user of dnstap, the forward plugin, use the newer code
also remove the test, a better test there would be a full e2e test to
see the correct thing happens.

Cleanup the Tapper interface and move it to dnstapio where it belongs,
remove higher level interfaces that are not used. This remove
dnstap.Tapper and dnstap.IORoutines.

Use the standard mechanism for getting access to a plugin and remove
shuffling the plugin into the context.

Signed-off-by: Miek Gieben <miek@miek.nl>

* use opts to get the correct proto

Signed-off-by: Miek Gieben <miek@miek.nl>

* Various fixes

Signed-off-by: Miek Gieben <miek@miek.nl>

* Remove bad addr test, as dnstap is only called from within coredns where these fields have been preparsed

Signed-off-by: Miek Gieben <miek@miek.nl>

* dnstap: remove saving the error

all these fields have been preparsed, no need for dnstap to be pedantic
and check (and save!) this error again.

Simplifies it a bit more.

Signed-off-by: Miek Gieben <miek@miek.nl>

* Update plugin/forward/dnstap.go

Co-authored-by: Ruslan Drozhdzh <30860269+rdrozhdzh@users.noreply.github.com>

* Code review

Signed-off-by: Miek Gieben <miek@miek.nl>

* add back in preferUDP

Signed-off-by: Miek Gieben <miek@miek.nl>

* nit

Signed-off-by: Miek Gieben <miek@miek.nl>

Co-authored-by: Ruslan Drozhdzh <30860269+rdrozhdzh@users.noreply.github.com>
2020-10-12 19:10:35 +02:00
Miek Gieben
b003d06003
For caddy v1 in our org (#4018)
* For caddy v1 in our org

This RP changes all imports for caddyserver/caddy to coredns/caddy. This
is the v1 code of caddy.

For the coredns/caddy repo the following changes have been made:

* anything not needed by us is deleted
* all `telemetry` stuff is deleted
* all its import paths are also changed to point to coredns/caddy
* the v1 branch has been moved to the master branch
* a v1.1.0 tag has been added to signal the latest release

Signed-off-by: Miek Gieben <miek@miek.nl>

* Fix imports

Signed-off-by: Miek Gieben <miek@miek.nl>

* Group coredns/caddy with out plugins

Signed-off-by: Miek Gieben <miek@miek.nl>

* remove this file

Signed-off-by: Miek Gieben <miek@miek.nl>

* Relax import ordering

github.com/coredns is now also a coredns dep, this makes
github.com/coredns/caddy fit more natural in the list.

Signed-off-by: Miek Gieben <miek@miek.nl>

* Fix final import

Signed-off-by: Miek Gieben <miek@miek.nl>
2020-09-24 18:14:41 +02:00
Ruslan Drozhdzh
f96a2f1f69
plugin/forward: fix panic when expire is configured as 0s (#4115)
Signed-off-by: Ruslan Drozhdzh <rdrozhdzh@infoblox.com>
2020-09-15 14:36:39 +02:00
Ruslan Drozhdzh
30a4a87eaa
plugin/forward: add hit/miss metrics for connection cache (#4114)
Signed-off-by: Ruslan Drozhdzh <rdrozhdzh@infoblox.com>
2020-09-14 11:42:55 +02:00
Ruslan Drozhdzh
04af1c692c
plugin/forward: init ClientSessionCache in tls.Config (#4108)
* plugin/forward: init ClientSessionCache in tls.Config

 - ClientSessionCache may speed up a TLS handshake in upcoming connections
   to the same TLS server

Signed-off-by: Ruslan Drozhdzh <rdrozhdzh@infoblox.com>

* add comment

Signed-off-by: Ruslan Drozhdzh <rdrozhdzh@infoblox.com>
2020-09-08 21:36:06 +08:00
Hu Shuai
e233f59ee8
Fix some typos in comments. (#4100)
Signed-off-by: Hu Shuai <hus.fnst@cn.fujitsu.com>
2020-09-01 09:10:45 +02:00
Zou Nengren
4166dcc2fe
using promauto package to ensure all created metrics are properly registered (#4025)
Signed-off-by: zounengren <zounengren@cmss.chinamobile.com>
2020-07-25 08:06:28 -07:00
Chris O'Haver
05751bdb59
register HealthcheckBrokenCount (#4021)
Signed-off-by: Chris O'Haver <cohaver@infoblox.com>
2020-07-24 08:28:18 -04:00
Zou Nengren
20c4f28a5c
remove TestProxyClose (#4005)
Signed-off-by: zounengren <zounengren@cmss.chinamobile.com>
2020-07-13 08:26:43 -07:00
Chris O'Haver
47d6e86f58
plugin/cache/forward: Clean up grammar/wording in forward & cache metrics descriptions. (#3971)
* tweak language

Signed-off-by: Chris O'Haver <cohaver@infoblox.com>

* tweak language

Signed-off-by: Chris O'Haver <cohaver@infoblox.com>

* typo

Signed-off-by: Chris O'Haver <cohaver@infoblox.com>
2020-06-24 07:49:42 -07:00
Miek Gieben
55e9c2cd7b
plugin/forward: remove exp backoff stuff (#3970)
we hc every 0.5s, doing exp backoff will create a large gap in the
ability to re-use an upstream. Doing a exp. backoff up to (say) 3s,
isn't really exp backoff either.

Remove the wording from the documentation.

Signed-off-by: Miek Gieben <miek@miek.nl>
2020-06-24 07:49:06 -07:00
Zou Nengren
8647592786
format and add newline (#3969)
Signed-off-by: zounengren <zounengren@cmss.chinamobile.com>
2020-06-24 07:48:39 -07:00
Zou Nengren
73e927d6a8
completed metrics of cache and forward (#3962)
Signed-off-by: zounengren <zounengren@cmss.chinamobile.com>
2020-06-24 06:54:03 +02:00
Chris O'Haver
54fb2112ac
plugin/forward/grpc: Revert forward/grpc policy dedup (#3919)
* revert de-dup

Signed-off-by: Chris O'Haver <cohaver@infoblox.com>

* unit test

Signed-off-by: Chris O'Haver <cohaver@infoblox.com>

* use roundrobin policy in test

Signed-off-by: Chris O'Haver <cohaver@infoblox.com>
2020-05-29 18:30:26 +02:00
Chris O'Haver
f4cb9a1ba3
fix readme (#3889)
Signed-off-by: Chris O'Haver <cohaver@infoblox.com>
2020-05-14 09:58:58 -07:00
Catena cyber
886cdd9dd3
Fix plugin forward fuzz target (#3877)
using new method to start proxies

Signed-off-by: Philippe Antoine <contact@catenacyber.fr>
2020-05-07 10:24:00 +02:00
kadern0
e410760a49
plugin/forward: crash if using https (#3817)
Signed-off-by: kadern0 <kaderno@gmail.com>
2020-04-09 11:26:28 -07:00
Miek Gieben
19cfa2960c
Cleanup metrics (#3776)
Cleanup a variety of metric issues.
* Eliminate department of redundancy "count_total" naming.
* Use the plural of the unit when appropriate. (ex, "requests")
* Remove label names from metric names where appropriate. (ex, "rcode")
* Simplify request metrics by consolidating type label in to the base
request counter.
* Re-generate man pages.

Signed-off-by: Ben Kochie <superq@gmail.com>

Co-authored-by: Ben Kochie <superq@gmail.com>
2020-03-26 09:17:33 +01:00
Chris O'Haver
d18b48e36c
register missing metric (#3756)
Signed-off-by: Chris O'Haver <cohaver@infoblox.com>
2020-03-17 21:19:52 +01:00
Miek Gieben
c52a51fac8
parse.HostPortorFile: return error when 0 found (#3742)
* parse.HostPortorFile: return error when 0 found

Return an error when we haven't found any nameservers. This is the
alternative considered in #3735. It's also slighly less code to be
changing.

Replaces: #3741
Closes: #3741 #3735

Signed-off-by: Miek Gieben <miek@miek.nl>

* Add extra test case here as well

Signed-off-by: Miek Gieben <miek@miek.nl>
2020-03-13 14:23:10 +01:00
Christian Tryti
116bda4d27
Add configuration flag to set if RecursionDesired should be set on health checkers in Forward-plugin (#3679)
* Make the RD-flag in health-checks in the Forward-plugin configurable

Introduces a new configuration flag; `health_check_non_recursive`. This
flag makes the health-checker do non-recursive requests when checking
the health of upstream servers.

Signed-off-by: Geir Haugom <ghagit@haugom.org>
Signed-off-by: Christian Tryti <ctryti@gmail.com>

* Changes after feedback from reviewer

* Better tests of health-checks with and without recursion
* Removed the health_check_non_recursive configuration in favor of
extending the existing health_check configuration. Now supports an
optional `no_rec` argument.

Signed-off-by: Christian Tryti <ctryti@gmail.com>

* Add new test that checks setup of health_check.

Signed-off-by: Christian Tryti <ctryti@gmail.com>
2020-03-06 11:52:43 +01:00
Miek Gieben
c4fc5cb54a
plugin/pkg/up: make default intervals shorter (#3651)
* plugin/pkg/up: make default intervals shorter

I think 15 min is too high, make this lower to react faster.

Signed-off-by: Miek Gieben <miek@miek.nl>

* Update README

Signed-off-by: Miek Gieben <miek@miek.nl>
2020-02-06 19:28:53 +01:00
Chris O'Haver
1818df0d06
Update README.md (#3655) 2020-02-05 10:19:04 -05:00
Ricky S
efbe4ac5e8
Add exponential backoff to healthcheck (#3643)
Move exponential backoff initialization to Start()

Signed-off-by: RickyRajinder <singh.sangh@gmail.com>

Move comment

Increase max interval and update README

Remove trailing whitespace

Change Start() param name back to interval
2020-02-04 14:19:48 +01:00
Chris O'Haver
22cd28a798
plugins/forward: Add max_concurrent option (#3640)
* count and limit concurrent queries

Signed-off-by: Chris O'Haver <cohaver@infoblox.com>

* add option

Signed-off-by: Chris O'Haver <cohaver@infoblox.com>

* return servfail when limit exceeded

Signed-off-by: Chris O'Haver <cohaver@infoblox.com>

* docs

Signed-off-by: Chris O'Haver <cohaver@infoblox.com>

* docs

Signed-off-by: Chris O'Haver <cohaver@infoblox.com>

* docs

Signed-off-by: Chris O'Haver <cohaver@infoblox.com>

* review feedback

Signed-off-by: Chris O'Haver <cohaver@infoblox.com>

* move atomic counter to beginning of struct

Signed-off-by: Chris O'Haver <cohaver@infoblox.com>

* add comment for ErrLimitExceeded

Signed-off-by: Chris O'Haver <cohaver@infoblox.com>

* rename option to max_concurrent

Signed-off-by: Chris O'Haver <cohaver@infoblox.com>

* add metric

Signed-off-by: Chris O'Haver <cohaver@infoblox.com>

* response REFUSED; incl max in error; add more docs

Signed-off-by: Chris O'Haver <cohaver@infoblox.com>

* avoid err setup race

Signed-off-by: Chris O'Haver <cohaver@infoblox.com>

* respond SERVFAIL; doc memory usage

Signed-off-by: Chris O'Haver <cohaver@infoblox.com>
2020-02-04 13:59:08 +01:00
Miek Gieben
5f159ca464 gofmt -w -s **/*.go (#3603)
format and remove trailing white space; makes 'make presubmit' pass
again.

Signed-off-by: Miek Gieben <miek@miek.nl>
2020-01-17 10:47:45 -05:00
Zou Nengren
5e04c27238 Dedup policy implement between grpc and proxy plugin (#3537)
Signed-off-by: zouyee <zounengren@cmss.chinamobile.com>
2019-12-17 08:15:31 +00:00
Miek Gieben
24176a97e6
Move to CODEOWNERS (#3489)
* Move to CODEOWNERS

No change in who own what; just a move to CODEOWNERS. This allows
dreck cleanups.

Added .dreck.yaml for alias and exec.

Fixes: #3486

Signed-off-by: Miek Gieben <miek@miek.nl>

* stickler bot

Signed-off-by: Miek Gieben <miek@miek.nl>

* sort the file

Signed-off-by: Miek Gieben <miek@miek.nl>
2019-11-29 13:17:05 +00:00
yuxiaobo96
bf356e79fc fix spelling mistakes (#3364)
Automatically submitted.
2019-10-09 07:24:18 +00:00
Miek Gieben
65458b2de2
Directive -> plugin (#3363)
Caught my eye, we name things directive still, esp when talking about
the prometheus *plugin*. Rename everything that needs to be plugin to
'plugin'. Also make sure Metrics is a H2 section (not H1).

Signed-off-by: Miek Gieben <miek@miek.nl>
2019-10-08 10:20:48 +01:00
Miek Gieben
575cea4496
Move map to array (#3339)
* Move map to array

The map was not needed move to an array, see #1941 for the original
idea. That of course didn't apply anymore; make a super minimal change
to implements the idea from #1941

Signed-off-by: Miek Gieben <miek@miek.nl>

* Add total count

Signed-off-by: Miek Gieben <miek@miek.nl>
2019-10-01 20:45:52 +01:00
Miek Gieben
2d98d520b5
plugin/forward: make Yield not block (#3336)
* plugin/forward: may Yield not block

Yield may block when we're super busy with creating (and looking) for
connection. Set a small timeout on Yield, to skip putting the connection
back in the queue.

Use persistentConn troughout the socket handling code to be more
consistent.

Signed-off-by: Miek Gieben <miek@miek.nl>

Dont do

Signed-off-by: Miek Gieben <miek@miek.nl>

* Set used in Yield

This gives one central place where we update used in the persistConns

Signed-off-by: Miek Gieben <miek@miek.nl>
2019-10-01 16:39:42 +01:00
Miek Gieben
dbd1c047cb
Run gostaticheck (#3325)
* Run gostaticheck

Run gostaticcheck on the codebase and fix almost all flagged items.

Only keep

* coremain/run.go:192:2: var appVersion is unused (U1000)
* plugin/chaos/setup.go:54:3: the surrounding loop is unconditionally terminated (SA4004)
* plugin/etcd/setup.go:103:3: the surrounding loop is unconditionally terminated (SA4004)
* plugin/pkg/replacer/replacer.go:274:13: argument should be pointer-like to avoid allocations (SA6002)
* plugin/route53/setup.go:124:28: session.New is deprecated: Use NewSession functions to create sessions instead. NewSession has the same functionality as New except an error can be returned when the func is called instead of waiting to receive an error until a request is made.  (SA1019)
* test/grpc_test.go:25:69: grpc.WithTimeout is deprecated: use DialContext and context.WithTimeout instead.  Will be supported throughout 1.x.  (SA1019)

The first one isn't true, as this is set via ldflags. The rest is
minor. The deprecation should be fixed at some point; I'll file some
issues.

Signed-off-by: Miek Gieben <miek@miek.nl>

* Make sure to plug in the plugins

import the plugins, that file that did this was removed, put it in the
reload test as this requires an almost complete coredns server.

Signed-off-by: Miek Gieben <miek@miek.nl>
2019-10-01 07:41:29 +01:00
Miek Gieben
03a3695ea9
plugins: calling Dispenser itself is a mistake (#3323)
Remove all these uses and just make them work on caddy.Controller. Also
don't export parsing functions as their should be private to the plugin.

Signed-off-by: Miek Gieben <miek@miek.nl>
2019-09-28 10:41:12 +01:00
Miek Gieben
f2df37a1fe
plugin/forward: metrics: make docs reflect reality (#3311)
Remove talk about labels that are not added.

Signed-off-by: Miek Gieben <miek@miek.nl>
2019-09-27 11:09:59 +01:00
xieyanker
9a5e4fa1a0 fix mis-spelling (#3310)
Signed-off-by: xieyanker <xjsisnice@gmail.com>
2019-09-26 13:19:45 +01:00
li mengyang
50bac4d3c3 fix: delete unused var and const (#3294)
Signed-off-by: hwdef <hwdef97@gmail.com>
2019-09-24 07:06:37 +01:00
Miek Gieben
004c5fca9d
all: simply registering plugins (#3287)
Abstract the caddy call and make it simpler.

See #3261 for some part of the discussion.

Go from:

~~~ go
func init() {
       caddy.RegisterPlugin("any", caddy.Plugin{
               ServerType: "dns",
               Action:     setup,
       })
}
~~~

To:

~~~ go
func init() { plugin.Register("any", setup) }
~~~

This requires some external documents in coredns.io to be updated as
well; the old way still works, so it's backwards compatible.

Signed-off-by: Miek Gieben <miek@miek.nl>
2019-09-20 08:02:30 +01:00
Miek Gieben
2332467b87
Fuzz: fix forward build (#3200)
* Fuzz: fix forward build

Signed-off-by: Miek Gieben <miek@miek.nl>

* Flag not released yet, wget quiet

Signed-off-by: Miek Gieben <miek@miek.nl>
2019-08-26 09:33:20 +00:00
Miek Gieben
793bd32499
plugin/forward: add fuzzing (#3188)
* plugin/forward: add fuzzing

Add fuzz.go for forward

Signed-off-by: Miek Gieben <miek@miek.nl>

* Make it compile

Signed-off-by: Miek Gieben <miek@miek.nl>
2019-08-25 19:00:59 +00:00
Guangming Wang
7918190901 Cleanup: fix some typos in code comment (#3172)
Signed-off-by: Guangming Wang <guangming.wang@daocloud.io>
2019-08-22 14:35:55 +00:00
zhangguoyan
86edb15c27 plugin/forward: fix a typo (#3133) 2019-08-16 16:34:12 +00:00
Andrey Meshkov
1ef24a8813 Fix handling truncated responses in forward (#3110)
* Fix handling truncated responses in forward

* Removed error check after proxy.Connect
See https://github.com/coredns/coredns/pull/3110/files#r312942826
2019-08-12 18:24:16 +00:00