While adding a parallel performance benchmark I stumbled on a race
condition (another reason to add performance benchmarks!), so this
PR makes sure the msg is created in a race free manor and adds the
parallel benchmark.
* cache: add sharded cache implementation
Add Cache impl and a few tests. This cache is 256-way sharded, mainly
so each shard has it's own lock. The main cache structure is a readonly
jump plane into the right shard.
This should remove the single lock contention on the main lock and
provide more concurrent throughput - Obviously this hasn't been tested
or measured.
The key into the cache was made a uint32 (hash.fnv) and the hashing op
is not using strings.ToLower anymore remove any GC in that code path.
* here too
* Minimum shard size
* typos
* blurp
* small cleanups no defer
* typo
* Add freq based on Johns idea
* cherry-pick conflict resolv
* typo
* update from early code review from john
* add prefetch to the cache
* mw/cache: add prefetch
* remove println
* remove comment
* Fix tests
* Test prefetch in setup
* Add start of cache
* try add diff cache options
* Add hacky testcase
* not needed
* allow the use of a percentage for prefetch
If the TTL falls below xx% do a prefetch, if the record was popular.
Some other fixes and correctly prefetch only popular records.
This checks if the next middleware to be called is nil, and if so returns
ServerFailure and an error. This makes the next calling more robust and
saves some lines of code.
Also prefix the error with the name of the middleware to aid in
debugging.
This PR adds a middleware that talks to dns.google.com over HTTPS,
meaning all your DNS traffic is encrypted when traversing your ISP and
the internet.
The `dns.google.com` address is re-resolved every 30 seconds.
* middleware/metrics: add more metrics
middleware/cache:
Add metrics for number of elements in the cache. Also export the total
size. Update README to detail the new metrics.
middleware/metrics
Move metrics into subpackage called "vars". This breaks the import
cycle and is cleaner. This allows vars.Report to be used in the
the dnsserver to log refused queries.
middleware/metrics: tests
Add tests to the metrics framework. The metrics/test subpackage allows
scraping of the local server. Do a few test scrape of the metrics that
are defined in the metrics middleware.
This also allows metrics integration tests to check if the caching and
dnssec middleware export their metrics correctly.
* update README
* typos
* fix tests
Make the cache memory bounded, by using a LRU cache. Also split the
cache in a positive and negative one - each with its own controls.
Extend the cache stanza to allow for this:
cache {
positive limit [ttl]
negative limit [ttl]
}
is now possible. This also add a cache_test.go in the toplevel test/
directory that exercises the caching path.
Fixes#260
Move all (almost all) Go files in middleware into their
own packages. This makes for better naming and discoverability.
Lot of changes elsewhere to make this change.
The middleware.State was renamed to request.Request which is better,
but still does not cover all use-cases. It was also moved out middleware
because it is used by `dnsserver` as well.
A pkg/dnsutil packages was added for shared, handy, dns util functions.
All normalize functions are now put in normalize.go
Add prometheus metrics to the cache handler. This just used prometheus,
if the metrics middleware does not setup the handler, there is nobody
reading these metrics, but they are still reported. Seems the simplest
solution while keeping the whole middleware separation in tact.
* Add middleware/cache
Add a caching middleware that caches nxdomain, nodata and successful
responses. It differentiates between DNSSEC on normal DNS replies.
Each reply is compress and scrubbed so it will fit the specific client
asking for it.
* first simple test, less exporting of stuff
* more
* Add middleware/cache
Add a caching middleware that caches nxdomain, nodata and successful
responses. It differentiates between DNSSEC on normal DNS replies.
Each reply is compressed and scrubbed so it will fit the specific client
asking for it. The TTL is decremented with the time spend in the cache.
There is syntax that allows you to cap the TTL for all records, no
matter what. This allows for a shortlived cache, just to absorb query
peaks.
+Tests
* cache test infrastructure
* Testing
