restic/internal
Matthew Dawson df2c03a6a4
repository/master_index: Optimize Index.Lookup()
When looking up a blob in the master index, with several
indexes present in the master index, a significant amount of time
is spent generating errors for each failed lookup.  However, these
errors are often used to check if a blob is present, but the contents
are not inspected making the overhead of the error not useful.

Instead, change Index.Lookup (and Index.LookupSize) to instead return
a boolean denoting if the blob was found instead of an error.  Also change
all the calls to these functions to handle the new function signature.

benchmark                                            old ns/op     new ns/op     delta
BenchmarkMasterIndexLookupSingleIndex-6              820           897           +9.39%
BenchmarkMasterIndexLookupMultipleIndex-6            12821         2001          -84.39%
BenchmarkMasterIndexLookupSingleIndexUnknown-6       5378          492           -90.85%
BenchmarkMasterIndexLookupMultipleIndexUnknown-6     17026         1649          -90.31%

benchmark                                            old allocs     new allocs     delta
BenchmarkMasterIndexLookupSingleIndex-6              9              9              +0.00%
BenchmarkMasterIndexLookupMultipleIndex-6            59             19             -67.80%
BenchmarkMasterIndexLookupSingleIndexUnknown-6       22             6              -72.73%
BenchmarkMasterIndexLookupMultipleIndexUnknown-6     72             16             -77.78%

benchmark                                            old bytes     new bytes     delta
BenchmarkMasterIndexLookupSingleIndex-6              160           160           +0.00%
BenchmarkMasterIndexLookupMultipleIndex-6            3200          240           -92.50%
BenchmarkMasterIndexLookupSingleIndexUnknown-6       1232          48            -96.10%
BenchmarkMasterIndexLookupMultipleIndexUnknown-6     4272          128           -97.00%
2018-01-23 22:25:56 -05:00
..
archiver repository/master_index: Optimize Index.Lookup() 2018-01-23 22:25:56 -05:00
backend rest: Add test for external server 2018-01-20 10:25:47 +01:00
cache cache: Simplify cache dir creation 2017-11-24 20:53:26 +01:00
checker checker: Optimize checker.Packs() 2018-01-18 20:50:39 -05:00
crypto crypto: Rework Seal/Open to use sliceForAppend 2017-11-01 10:30:40 +01:00
debug Run goimports 2017-07-23 14:21:03 +02:00
errors errors: Add "Fatal" prefix for Fatalf() 2018-01-06 23:27:47 +01:00
filter filter: document recursive wildcards 2017-09-04 14:38:48 -07:00
fs fs: Make HasPathPrefix work with relative paths 2018-01-05 17:38:52 +01:00
fuse repository/master_index: Optimize Index.Lookup() 2018-01-23 22:25:56 -05:00
hashing Moves files 2017-07-23 14:19:13 +02:00
index Move restic package to internal/restic 2017-07-24 17:43:32 +02:00
limiter Improve limiting HTTP based backends 2017-12-29 12:43:49 +01:00
list Move restic package to internal/restic 2017-07-24 17:43:32 +02:00
migrations Move restic package to internal/restic 2017-07-24 17:43:32 +02:00
mock Move restic package to internal/restic 2017-07-24 17:43:32 +02:00
options errors: Add "Fatal" prefix for Fatalf() 2018-01-06 23:27:47 +01:00
pack Use Seal/Open everywhere 2017-11-01 10:30:40 +01:00
pipe small cleanup: 2017-10-25 12:03:55 -04:00
repository repository/master_index: Optimize Index.Lookup() 2018-01-23 22:25:56 -05:00
restic repository/master_index: Optimize Index.Lookup() 2018-01-23 22:25:56 -05:00
test internal: check error before deferring file Close() 2017-08-13 19:28:13 +02:00
walk replace ad-hoc context.TODO() with gopts.ctx, so that cancellation 2017-12-03 07:22:14 -05:00
worker Run goimports 2017-07-23 14:21:03 +02:00