TrueCloudLab/frostfs-node
19
1
Fork 27
Code Issues 91 Pull requests 6 Projects Releases 1 Activity Actions
5064 commits 8 branches 81 tags 52 MiB
Commit graph

47 commits

Author SHA1 Message Date
Dmitrii Stepanov
1efa64ee72
[#1412] metabase: Add search by indexed attributes 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-10-08 11:41:50 +03:00
Dmitrii Stepanov
01e3944b31 [#1408] metabase: Fix tests
All checks were successful
DCO action / DCO (pull_request) Successful in 1m3s
Details
Tests and linters / Run gofumpt (pull_request) Successful in 1m21s
Details
Pre-commit hooks / Pre-commit (pull_request) Successful in 2m27s
Details
Vulncheck / Vulncheck (pull_request) Successful in 2m20s
Details
Build / Build Components (pull_request) Successful in 2m41s
Details
Tests and linters / gopls check (pull_request) Successful in 2m57s
Details
Tests and linters / Staticcheck (pull_request) Successful in 3m5s
Details
Tests and linters / Lint (pull_request) Successful in 3m37s
Details
Tests and linters / Tests (pull_request) Successful in 4m31s
Details
Tests and linters / Tests with -race (pull_request) Successful in 6m22s
Details 
No need to specify container ID for objects created with `testutil.GenerateObjectWithCID`.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-10-02 14:41:40 +03:00
Dmitrii Stepanov
434048e8d9 [#1408] metabase: Fix EC search with slow and fast filters 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-10-02 14:36:59 +03:00
Dmitrii Stepanov
76268e3ea2 [#1385] metabase: Validate that tombstone and target have the same container ID 
Target container ID is taken from tombstone: cmd/frostfs-node/object.go:507
Also object of type `TOMBSTONE` contains objectID, so tombstone and
tombstoned object must have the same containerID.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-09-24 17:09:06 +03:00
Dmitrii Stepanov
e3764c51df [#1347] metabase: Fix EC search 
For EC chunks need to return EC parent object ID as
EC chunks don't have own attributes but inherit parent's.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-09-12 13:23:29 +00:00
Ekaterina Lebedeva
a685fcdc96 [#1317] go.mod: Use range over int
All checks were successful
DCO action / DCO (pull_request) Successful in 2m41s
Details
Tests and linters / Run gofumpt (pull_request) Successful in 2m32s
Details
Vulncheck / Vulncheck (pull_request) Successful in 2m38s
Details
Build / Build Components (1.23) (pull_request) Successful in 3m0s
Details
Build / Build Components (1.22) (pull_request) Successful in 3m3s
Details
Pre-commit hooks / Pre-commit (pull_request) Successful in 3m33s
Details
Tests and linters / Tests (1.22) (pull_request) Successful in 3m34s
Details
Tests and linters / Tests (1.23) (pull_request) Successful in 3m36s
Details
Tests and linters / Staticcheck (pull_request) Successful in 3m35s
Details
Tests and linters / Lint (pull_request) Successful in 4m18s
Details
Tests and linters / Tests with -race (pull_request) Successful in 4m20s
Details
Tests and linters / gopls check (pull_request) Successful in 4m25s
Details 
Since Go 1.22 a "for" statement with a "range" clause is able
to iterate through integer values from zero to an upper limit.

gopatch script:
@@
var i, e expression
@@
-for i := 0; i <= e - 1; i++ {
+for i := range e {
    ...
}

@@
var i, e expression
@@
-for i := 0; i <= e; i++ {
+for i := range e + 1 {
    ...
}

@@
var i, e expression
@@
-for i := 0; i < e; i++ {
+for i := range e {
    ...
}

Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
 2024-09-03 13:00:54 +03:00
Dmitrii Stepanov
15dae8685e [#1323] metabase: Drop ownerID index 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-08-22 08:21:40 +00:00
Evgenii Stratonikov
3f1961157e [#1163] metabase: Handle multiple splitInfos for EC
All checks were successful
DCO action / DCO (pull_request) Successful in 3m5s
Details
Pre-commit hooks / Pre-commit (pull_request) Successful in 4m41s
Details
Vulncheck / Vulncheck (pull_request) Successful in 4m10s
Details
Build / Build Components (1.21) (pull_request) Successful in 5m49s
Details
Build / Build Components (1.22) (pull_request) Successful in 5m46s
Details
Tests and linters / Staticcheck (pull_request) Successful in 6m37s
Details
Tests and linters / Lint (pull_request) Successful in 7m12s
Details
Tests and linters / gopls check (pull_request) Successful in 7m16s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 10m11s
Details
Tests and linters / Tests (1.22) (pull_request) Successful in 10m20s
Details
Tests and linters / Tests with -race (pull_request) Successful in 10m21s
Details 
For REP updating split info is handled explicitly by a high-level PUT logic.
For EC it is trickier, because the address of an object we put is only
distantly related to a split info.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-06-06 16:26:29 +03:00
Evgenii Stratonikov
2e074d3846 [#1163] metabase: Properly save EC parent split ID 
Search by SplitID should return all parts of a complex object.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-06-05 12:40:16 +03:00
Dmitrii Stepanov
47dcfa20f3 [#895] test: Use t.Cleanup only for external resources 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-01-11 12:32:09 +00:00
Evgenii Stratonikov
c62025c836 [#321] metabase/test: execute tests in parallel
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-05-12 09:45:03 +00:00
Evgenii Stratonikov
a9c4ba62c3 [#248] metabase: Remove storage group bucket 
Backwards compatible change, so no version increase.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-04-28 17:00:43 +03:00
Dmitrii Stepanov
d62c6e4ce6 [#242] node: Add tracing spans 
Add tracing spans for PUT requests.
Add tracing spans for DELETE requests.
Add tracing spans for SELECT requests.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-14 10:25:53 +00:00
Ayrat Arifullin
9808dec591 [#86] node: Move testing utils to one package
Some checks failed
ci/woodpecker/push/pre-commit Pipeline failed
Details 
Move testing utils from tests in local_object_storage package to
unified testutil package

Signed-off-by: Airat Arifullin <aarifullin@yadro.com>
 2023-03-23 08:19:15 +00:00
Alex Vanin
20de74a505 Rename package name 
Due to source code relocation from GitHub.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-03-07 16:38:26 +03:00
Pavel Karpy
923f84722a Move to frostfs-node 
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2022-12-28 15:04:29 +03:00
Pavel Karpy
156ba85326 [#1634] node: Do not return expired objects 
If an object has not been marked for removal by the GC in the current epoch
yet but has already expired, respond with `ErrObjectNotFound` api status.
Also, optimize shard iteration: a node must stop any iteration if the object
 is found but gonna be removed soon.
All the checks are performed by the Metabase.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-08-04 16:31:49 +03:00
Evgenii Stratonikov
34d319fed2 [#1559] metabase: Use Set prefix for parameter setting 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-21 17:56:06 +03:00
Evgenii Stratonikov
f58234aa2f [#1559] metabase: Remove public functions 
Reduce public interface of this package. Later each result will contain
an additional status, so it makes more sense to use the same functions
and result processing everywhere.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-21 17:56:06 +03:00
Evgenii Stratonikov
5073a37930 [#1433] metabase: Optimize Select 
For some filters we can scan only a subset of keys instead of checking
each key.

```
name                       old time/op  new time/op  delta
Select/string_equal-8      49.3µs ± 4%  11.0µs ± 4%  -77.68%  (p=0.000 n=10+10)
Select/string_not_equal-8  7.01ms ± 5%  7.06ms ±10%     ~     (p=0.971 n=10+10)
Select/common_prefix-8      118µs ± 6%    79µs ± 5%  -33.04%  (p=0.000 n=10+9)
Select/unknown-8           21.3µs ± 4%   3.2µs ± 4%  -84.88%  (p=0.000 n=10+9)
```

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-06-03 14:25:07 +03:00
Evgenii Stratonikov
5c48588c64 [#1433] metabase: Add more tests for Select 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-06-03 14:25:07 +03:00
Evgenii Stratonikov
ea76c989ca [#1433] metabase: Add a benchmark for Select 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-06-03 14:25:07 +03:00
Leonard Lyubich
1c30414a6c [#1454] Upgrade NeoFS SDK Go module with new IDs 
Core changes:
 * avoid package-colliding variable naming
 * avoid using pointers to IDs where unnecessary
 * avoid using `idSDK` import alias pattern
 * use `EncodeToString` for protocol string calculation and `String` for
  printing

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-06-01 17:41:45 +03:00
Leonard Lyubich
883f2f1ac6 [#1423] local storage: Activate skipped unit tests after fix 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-05-27 15:45:33 +03:00
Leonard Lyubich
f15e6e888f [#1377] oid, cid: Upgrade SDK package 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-05-16 15:33:22 +03:00
Leonard Lyubich
ae92074272 [#1369] checksum: Upgrade SDK package 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-05-12 09:48:31 +03:00
Alex Vanin
8f476f3c4d [#1238] Adopt neofs-node for non pointer slices in SDK 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2022-03-16 15:29:31 +03:00
Leonard Lyubich
9508633a7e [#1175] metabase: Work with LOCK objects 
After introduction of LOCK objects (of type `TypeLock`) complicated
extended its behavior:
  * create `lockers` container bucket (LCB) during PUT;
  * remove object from LCB during DELETE;
  * look up object in LCB during EXISTS;
  * get object from LCB during GET;
  * list objects from LCB during LIST with cursor;
  * select objects from LCB during SELECT with '*'.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-03-15 13:03:23 +03:00
Leonard Lyubich
7ccd1625af [#1214] *: Use single Object type in whole project 
Remove `Object` and `RawObject` types from `pkg/core/object` package.
Use `Object` type from NeoFS SDK Go library everywhere. Avoid using the
deprecated elements.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-03-04 17:45:00 +03:00
Pavel Karpy
1667ec9e6d [#1131] *: Adopt SDK changes 
`object.Address` has been moved to `object/address`
`object.ID` has been moved to `object/id`

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-02-08 09:45:38 +03:00
Leonard Lyubich
6f41cd86ed [#979] Upgrade SDK NeoFS Go dependency 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2021-12-01 17:31:04 +03:00
Evgenii Stratonikov
95893927aa *: replace neofs-api-go with neofs-sdk-go 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2021-11-12 17:29:09 +03:00
Alex Vanin
2271944064 [#945] metabase: Use cleanup instead of defer in tests 
Cleanup is more idiomatic way to clean test data

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2021-10-27 16:10:01 +03:00
Pavel Karpy
4ff98a7e2b [#760] metabase: Support COMMON_PREFIX matchtype 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2021-08-27 15:03:15 +03:00
Leonard Lyubich
3e1463cc76 [#570] *: Use generator of test container IDs from API Go lib 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2021-05-31 20:45:15 +03:00
Leonard Lyubich
ffbf6b922f [#361] metabase: Support new match types 
Support STRING_NOT_EQUAL and NOT_PRESENT match types.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2021-02-03 10:32:15 +03:00
Alex Vanin
b97f818323 [#242] metabase: Make ContainerID a mandatory parameter for Select 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Leonard Lyubich
590745204c [#237] metabase: Structure parameters and results of all operations 
All parameters and resulting values of all metabase operations are
structured in new types. The most popular scenarios for using operations are
moved to auxiliary functions.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Leonard Lyubich
a875d80491 Remove outdated code of metabase and localstore 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-12-11 17:19:37 +03:00
Leonard Lyubich
ffeea2c003 [#177] metabase: Add index by object ID 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-11-23 09:54:59 +03:00
Leonard Lyubich
3de8febe57 [#174] Update to latest neofs-api-go changes 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-11-17 11:56:00 +03:00
Leonard Lyubich
2913aa0fd1 [#149] metabase: Add parent ID index 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-11-06 14:01:01 +03:00
Leonard Lyubich
62bd22a379 [#142] metabase: Fix false-positive select in absence of filtered header 
Fix a bug in the selection when the object without some filtered header
added to the final result.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-11-03 18:42:32 +03:00
Leonard Lyubich
3c39c5a90c [#142] metabase: Fix selection emptying due to deleted object 
Fix a bug in the selection when removed object that matches search query
provoked the return of an empty result.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-11-03 18:42:32 +03:00
Leonard Lyubich
7a8f322d59 [#142] metabase: Fix false-positive select 
Fix a bug in the selection when an object could be added to the result after
a mismatch in the previous filter.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-11-03 18:42:32 +03:00
Leonard Lyubich
7704811654 [#137] metabase: Implement and useful test funcs for work with DB 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-11-03 18:42:32 +03:00
Leonard Lyubich
a61f8d44d1 [#135] metabase: Implement benchmarking tests 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-11-03 18:42:32 +03:00