eaf96bccf7
[ #1461 ] meta: Add lock tests
...
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-07-18 11:42:25 +03:00
9c5ef3bab8
[ #1461 ] node: Allow force LOCK removal
...
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-07-18 11:42:25 +03:00
fed9e6679d
[ #1461 ] node: Unlock locked object on its lock removal
...
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-07-18 11:42:25 +03:00
633b4e7d2d
[ #1483 ] metabase: Add VERSION.md
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-21 17:48:28 +03:00
6f243a2a76
[ #1483 ] metabase: Store version
...
The main problem is to distinguish the case of initial initialization
and update from version 0. We can't do this at `Open`, because of
`resync_metabase` flag. Thus, the following approach was taken:
1. During `Open` check whether the metabase was initialized.
2. Check for the version in `Init` or write the new one if the metabase
is new.
3. Update version in `Reset`.
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-21 17:48:28 +03:00
07e06249d5
[ #1524 ] metabase: Add some bolt parameters to the configuration
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-20 17:04:35 +03:00
81f925d5a0
[ #1516 ] metabase: Optimize `ListWithCursor` for long listings
...
Cache buckets outside of the main loop and allocate memory for the
resulting offset only once.
```
name old time/op new time/op delta
ListWithCursor/1_item-8 6.45µs ±14% 5.79µs ±11% -10.24% (p=0.002 n=10+10)
ListWithCursor/10_items-8 20.9µs ±17% 17.3µs ± 9% -17.27% (p=0.000 n=10+10)
ListWithCursor/100_items-8 153µs ±12% 131µs ± 9% -14.63% (p=0.000 n=10+10)
name old alloc/op new alloc/op delta
ListWithCursor/1_item-8 2.31kB ± 0% 1.91kB ± 0% -17.46% (p=0.000 n=10+10)
ListWithCursor/10_items-8 6.94kB ± 0% 5.50kB ± 0% -20.78% (p=0.000 n=8+8)
ListWithCursor/100_items-8 53.3kB ± 0% 41.5kB ± 0% -22.18% (p=0.000 n=10+10)
name old allocs/op new allocs/op delta
ListWithCursor/1_item-8 40.0 ± 0% 34.0 ± 0% -15.00% (p=0.000 n=10+10)
ListWithCursor/10_items-8 121 ± 0% 100 ± 0% -17.36% (p=0.000 n=10+10)
ListWithCursor/100_items-8 930 ± 0% 758 ± 0% -18.49% (p=0.000 n=10+10)
```
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-15 20:49:41 +03:00
a93373fe71
[ #1516 ] metabase: Cache graveyard buckets in `ListWithCursor`
...
```
name old time/op new time/op delta
ListWithCursor/1_item-8 6.40µs ±13% 6.45µs ±14% ~ (p=0.739 n=10+10)
ListWithCursor/10_items-8 30.9µs ±21% 20.9µs ±17% -32.49% (p=0.000 n=10+10)
ListWithCursor/100_items-8 274µs ±27% 153µs ±12% -44.09% (p=0.000 n=10+10)
name old alloc/op new alloc/op delta
ListWithCursor/1_item-8 2.26kB ± 0% 2.31kB ± 0% +2.46% (p=0.000 n=10+10)
ListWithCursor/10_items-8 10.8kB ± 0% 6.9kB ± 0% -36.07% (p=0.000 n=8+8)
ListWithCursor/100_items-8 96.8kB ± 0% 53.3kB ± 0% -44.98% (p=0.000 n=10+10)
name old allocs/op new allocs/op delta
ListWithCursor/1_item-8 39.0 ± 0% 40.0 ± 0% +2.56% (p=0.000 n=10+10)
ListWithCursor/10_items-8 192 ± 0% 121 ± 0% -36.98% (p=0.000 n=10+10)
ListWithCursor/100_items-8 1.72k ± 0% 0.93k ± 0% -45.93% (p=0.000 n=10+10)
```
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
name old time/op new time/op delta
ListWithCursor/1_item-8 5.23µs ±19% 5.26µs ±15% ~ (p=0.853 n=10+10)
ListWithCursor/10_items-8 27.2µs ±15% 18.0µs ±19% -33.80% (p=0.000 n=10+10)
ListWithCursor/100_items-8 250µs ±13% 139µs ±15% -44.27% (p=0.000 n=10+10)
name old alloc/op new alloc/op delta
ListWithCursor/1_item-8 1.99kB ± 0% 2.04kB ± 0% +2.82% (p=0.000 n=8+8)
ListWithCursor/10_items-8 10.3kB ± 0% 6.4kB ± 0% -37.83% (p=0.000 n=8+10)
ListWithCursor/100_items-8 93.9kB ± 0% 50.4kB ± 0% -46.37% (p=0.000 n=10+10)
name old allocs/op new allocs/op delta
ListWithCursor/1_item-8 35.0 ± 0% 36.0 ± 0% +2.86% (p=0.000 n=10+10)
ListWithCursor/10_items-8 184 ± 0% 113 ± 0% -38.59% (p=0.000 n=10+10)
ListWithCursor/100_items-8 1.67k ± 0% 0.88k ± 0% -47.29% (p=0.000 n=10+10)
```
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-15 20:49:41 +03:00
af4db8a73b
[ #1516 ] metabase: Cache address key and do not decode address twice
...
```
name old time/op new time/op delta
ListWithCursor/1_item-8 10.6µs ± 1% 6.4µs ±13% -39.62% (p=0.000 n=7+10)
ListWithCursor/10_items-8 75.3µs ± 2% 30.9µs ±21% -58.97% (p=0.000 n=10+10)
ListWithCursor/100_items-8 726µs ± 2% 274µs ±27% -62.28% (p=0.000 n=10+10)
name old alloc/op new alloc/op delta
ListWithCursor/1_item-8 3.19kB ± 0% 2.26kB ± 0% -29.21% (p=0.000 n=10+10)
ListWithCursor/10_items-8 20.7kB ± 0% 10.8kB ± 0% -47.68% (p=0.000 n=10+8)
ListWithCursor/100_items-8 196kB ± 0% 97kB ± 0% -50.65% (p=0.000 n=7+10)
name old allocs/op new allocs/op delta
ListWithCursor/1_item-8 55.0 ± 0% 39.0 ± 0% -29.09% (p=0.000 n=10+10)
ListWithCursor/10_items-8 346 ± 0% 192 ± 0% -44.51% (p=0.000 n=10+10)
ListWithCursor/100_items-8 3.25k ± 0% 1.72k ± 0% -47.13% (p=0.000 n=9+10)
```
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-15 20:49:41 +03:00
504f45e9ee
[ #1516 ] metabase: Add benchmark for `ListWithCursor`
...
```
name time/op
ListWithCursor/1_item-8 10.6µs ± 1%
ListWithCursor/10_items-8 75.3µs ± 2%
ListWithCursor/100_items-8 726µs ± 2%
name alloc/op
ListWithCursor/1_item-8 3.19kB ± 0%
ListWithCursor/10_items-8 20.7kB ± 0%
ListWithCursor/100_items-8 196kB ± 0%
name allocs/op
ListWithCursor/1_item-8 55.0 ± 0%
ListWithCursor/10_items-8 346 ± 0%
ListWithCursor/100_items-8 3.25k ± 0%
```
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-15 20:49:41 +03:00
9b2932609b
[ #1460 ] meta: Do not use pointers as the results
...
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-06-06 18:03:12 +03:00
a580429996
[ #1460 ] meta: Add a benchmark on `Get` operation
...
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-06-06 18:03:12 +03:00
6d4d920bad
[ #1418 ] meta: Do not use pointers as parameters
...
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-06-06 18:03:12 +03:00
5073a37930
[ #1433 ] metabase: Optimize `Select`
...
For some filters we can scan only a subset of keys instead of checking
each key.
```
name old time/op new time/op delta
Select/string_equal-8 49.3µs ± 4% 11.0µs ± 4% -77.68% (p=0.000 n=10+10)
Select/string_not_equal-8 7.01ms ± 5% 7.06ms ±10% ~ (p=0.971 n=10+10)
Select/common_prefix-8 118µs ± 6% 79µs ± 5% -33.04% (p=0.000 n=10+9)
Select/unknown-8 21.3µs ± 4% 3.2µs ± 4% -84.88% (p=0.000 n=10+9)
```
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-03 14:25:07 +03:00
5c48588c64
[ #1433 ] metabase: Add more tests for `Select`
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-03 14:25:07 +03:00
ea76c989ca
[ #1433 ] metabase: Add a benchmark for `Select`
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-03 14:25:07 +03:00
e265ce2d52
[ #1418 ] meta: Do not use pointers as parameters
...
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-06-03 07:35:17 +03:00
1c30414a6c
[ #1454 ] Upgrade NeoFS SDK Go module with new IDs
...
Core changes:
* avoid package-colliding variable naming
* avoid using pointers to IDs where unnecessary
* avoid using `idSDK` import alias pattern
* use `EncodeToString` for protocol string calculation and `String` for
printing
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-01 17:41:45 +03:00
883f2f1ac6
[ #1423 ] local storage: Activate skipped unit tests after fix
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-05-27 15:45:33 +03:00
bb25ecbd15
[ #1400 ] owner: Upgrade SDK package
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-05-23 15:33:14 +03:00
f15e6e888f
[ #1377 ] oid, cid: Upgrade SDK package
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-05-16 15:33:22 +03:00
ae92074272
[ #1369 ] checksum: Upgrade SDK package
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-05-12 09:48:31 +03:00
088df0e2a9
[ #1369 ] version: Upgrade SDK package
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-05-12 09:48:31 +03:00
cc7a723d77
[ #1320 ] English Check
...
Signed-off-by: Elizaveta Chichindaeva <elizaveta@nspcc.ru>
2022-05-11 10:40:02 +03:00
9da7df4d42
[ #1318 ] metabase: Init metabase in test
...
`Init()` creates buckets. It prevents NPE in tests.
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-04-29 16:38:52 +03:00
486ec8a2e5
[ #1318 ] metabase: Reuse comparing func from `require`
...
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-04-29 16:38:52 +03:00
1caeb0b648
[ #1318 ] metabase: Add missing error check in tests
...
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-04-29 16:38:52 +03:00
7799f8e4cf
[ #1318 ] engine: Change tombstone clear process
...
- Delete objects physically on tombstone's arrival;
- Store information about tombstones in the Graveyard;
- Clear Graveyard every epoch based on the information about TS in the
network.
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-04-29 16:38:52 +03:00
2583f608e8
[ #1318 ] metabase: Add `DropGraves` method
...
Method clean up Graveyard and should be called by the routine that handles
tombstone expiration.
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-04-29 16:38:52 +03:00
daab30c391
[ #1318 ] metabase: Add iteration offset
...
Add offset element to the iterations over deleted objects (both the
Graveyard and the Garbage buckets).
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-04-29 16:38:52 +03:00
fe8076e60a
[ #1318 ] metabase: Add immediate object deletion
...
Mark objects with GC immediately after a Tombstone is received.
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-04-29 16:38:52 +03:00
8107c8d1a9
[ #1318 ] metabase: Separate buckets with TS and GC marks
...
It allows storing information about object in both ways at the same time:
1. Metabase should know if an object is covered by a tombstone (that is
not expired yet);
2. It should be possible to physically delete objects covered by a
tombstone immediately (mark with GC) but keep tombstone knowledge.
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-04-29 16:38:52 +03:00
629a4f79cd
[ #1293 ] metabase: Fix out-of-range panic in `freePotentialLocks`
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-04-05 15:38:51 +03:00
de5a2f6574
[ #1262 ] metabase: Remove list index in place `Delete`
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-03-28 17:08:11 +03:00
43867a3093
[ #1262 ] metabase: Do not allocate intermediate slices for indices
...
```
name old alloc/op new alloc/op delta
Put/parallel-8 123kB ± 4% 119kB ± 3% -2.72% (p=0.006 n=10+9)
Put/sequential-8 170kB ± 1% 168kB ± 1% -1.42% (p=0.000 n=10+10)
name old allocs/op new allocs/op delta
Put/parallel-8 473 ± 1% 469 ± 0% -0.87% (p=0.000 n=10+10)
Put/sequential-8 792 ± 0% 787 ± 0% -0.58% (p=0.000 n=10+10)
```
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-03-28 17:08:11 +03:00
0e9b6be3fd
[ #1262 ] metabase: Remove intermediate allocations in `decodeList`
...
```
name old time/op new time/op delta
Put/parallel-8 1.57ms ±11% 1.51ms ± 3% -4.06% (p=0.043 n=9+10)
Put/sequential-8 5.16ms ± 2% 5.16ms ± 3% ~ (p=1.000 n=9+10)
name old alloc/op new alloc/op delta
Put/parallel-8 126kB ± 4% 123kB ± 4% -2.54% (p=0.016 n=8+10)
Put/sequential-8 171kB ± 1% 170kB ± 1% ~ (p=0.182 n=9+10)
name old allocs/op new allocs/op delta
Put/parallel-8 565 ± 2% 473 ± 1% -16.18% (p=0.000 n=9+10)
Put/sequential-8 819 ± 1% 792 ± 0% -3.34% (p=0.000 n=9+10)
```
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-03-28 17:08:11 +03:00
d45df614fb
[ #1262 ] metabase: Optimize `decodeList`
...
Prevent additional allocation during `append` in `Put`.
```
name old alloc/op new alloc/op delta
Put/parallel-8 131kB ± 1% 126kB ± 4% -3.87% (p=0.005 n=8+8)
Put/sequential-8 172kB ± 1% 171kB ± 1% -0.73% (p=0.028 n=10+9)
```
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-03-28 17:08:11 +03:00
456e1584d6
[ #1262 ] metabase: Add benchmarks for `Put`
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-03-28 17:08:11 +03:00
318639e5bf
[ #1247 ] *: Clarify docs about returned errors from `apistatus` package
...
`apistatus` package provides types which implement build-in `error`
interface. Add `error of type` pattern when documenting these errors in
order to clarify how these errors should be handled (e.g. `errors.Is` is
not good).
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-03-17 16:34:00 +03:00
70ffdf3478
[ #1247 ] object: Return `NOT_FOUND` and `ALREADY_REMOVED` statuses
...
Replace `ErrNotFound`/`ErrAlreadyRemoved` error from
`pkg/core/object` package with `ObjectNotFound`/`ObjectAlreadyRemoved`
one from `apistatus` package. These errors are returned by storage
node's server as NeoFS API statuses.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-03-17 16:34:00 +03:00
0adb29c035
[ #1214 ] Fix metabase put logging
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-03-16 17:17:18 +03:00
8f476f3c4d
[ #1238 ] Adopt neofs-node for non pointer slices in SDK
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-03-16 15:29:31 +03:00
9dff07200c
[ #1181 ] local storage: Process expired locks similar to tombstones
...
There is a need to process expired `LOCK` objects similar to `TOMBSTONE`
ones: we collect them on `Shard`, notify all other shards about
expiration so they could unlock the objects, and only after that mark
lockers as garbage.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-03-15 13:03:23 +03:00
323dea95c6
[ #1181 ] Do not use deprecated elements
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-03-15 13:03:23 +03:00
e3f4e48d87
[ #1175 ] Fix typo in error return docs
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-03-15 13:03:23 +03:00
ced854bc2e
[ #1175 ] object/fmt: Handle `LOCK` objects
...
Make `FormatValidator.ValidateContent` to verify payload of `LOCK`
objects. Pass locked objects to `Locker` interface. Require from
`Locker.Lock` to return `apistatus.IrregularObjectLock` error on a
corresponding condition.
Also add error return to `DeleteHandler.DeleteObjects` method. Require
from method to return `apistatus.ObjectLocked` error on a corresponding
condition. Adopt implementations.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-03-15 13:03:23 +03:00
07107c8bfd
[ #1175 ] metabase: Update locker list during `Inhume`
...
`Inhume` operation can potentially mark lockers as garbage. There is a
need to update locker list in locked bucket.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-03-15 13:03:23 +03:00
26b8e33ed7
[ #1175 ] metabase: Fix encoding of updated locker list
...
Even single locker must be encoded using `encodeList`, or subsequent
`decodeList` call will fail.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-03-15 13:03:23 +03:00
b585791d6e
[ #1175 ] metabase: Return status error on Lock of irregular object
...
Make `DB.Lock` to return `apistatus.IrregularObjectLock` if at least one
of the locked objects is irregular (not of type REGULAR).
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-03-15 13:03:23 +03:00
23fcacd3f2
[ #1175 ] metabase: Return status error on Inhume of locked object
...
Make `DB.Inhume` to return `apistatus.ObjectLocked` if at least one of
the inhumed objects is locked.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-03-15 13:03:23 +03:00
Pavel Karpy
Evgenii Stratonikov
Leonard Lyubich
Elizaveta Chichindaeva
Alex Vanin