Commit graph

668 commits

Author SHA1 Message Date
9ee3dd4e91 [#1618] ape: Fix object service request tests
Constant string `testOwnerID` for these tests has an invalid
format. It has 11 bytes instead of required 25 for `user.ID`.
It worked because:
  1. `user.ID` was a byte slice and didn't check length
     and format of byte slices decoded from strings.
  2. in these tests `testOwnerID` was used only to decode
     container owner id and to compare it with owner id encoded
     back to string.
Since `user.ID implementation has changed`, the problem arised.
Now `testOwnerID` is valid.

Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2025-01-30 13:18:37 +00:00
57dc0a8e9e [#1616] getsvc: Move break condition from body to the loop condition
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2025-01-30 06:50:37 +00:00
b309b34bfc [#1616] getsvc: Simplify buildChainRange()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2025-01-30 06:50:37 +00:00
c8acdf40bb [#1616] getsvc: Use slices.Reverse() where possible
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2025-01-30 06:50:37 +00:00
6410542d19 [#1616] getsvc: Move range assembling to a separate file
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2025-01-30 06:50:37 +00:00
c0a341a7f6 [#1616] getsvc: Move head assembling to a separate file
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2025-01-30 06:50:37 +00:00
e1a984e9d8
[#1620] logs: Drop redundant trace_id fields
`trace_id` is taken from context.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2025-01-29 16:13:51 +03:00
c98357606b
[#1606] Use slices.Clone()/bytes.Clone() where possible
gopatch:
```
@@
var from, to expression
@@
+import "bytes"
-to := make([]byte, len(from))
-copy(to, from)
+to := bytes.Clone(from)

@@
var from, to expression
@@
+import "bytes"
-to = make([]byte, len(from))
-copy(to, from)
+to = bytes.Clone(from)

@@
var from, to, typ expression
@@
+import "slices"
-to := make([]typ, len(from))
-copy(to, from)
+to := slices.Clone(from)

@@
var from, to, typ expression
@@
+import "slices"
-to = make([]typ, len(from))
-copy(to, from)
+to = slices.Clone(from)
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2025-01-17 14:50:14 +03:00
4d5ae59a52
[#1598] golangci: Enable unconvert linters
To drop unnecessary conversions.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2025-01-14 09:06:40 +03:00
a9f27e074b [#1243] object: Look for X-Headers within origin before APE check
* X-Headers can be found in `origin` field of `MetaHeader` if the request
  has been forwarded from non-container node.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2025-01-13 12:07:27 +00:00
d7fcc5ce30 [#1586] objsvc: Allow to send search response in multiple messages
Previously, `ln` was only set once, so search has really worked for
small number of objects.

Fix panic:
```
panic: runtime error: slice bounds out of range [:43690] with capacity 21238
goroutine 6859775 [running]:
git.frostfs.info/TrueCloudLab/frostfs-node/pkg/services/object.(*searchStreamMsgSizeCtrl).Send(0xc001eec8d0, 0xc005734000)
        git.frostfs.info/TrueCloudLab/frostfs-node/pkg/services/object/transport_splitter.go:173 +0x1f0
git.frostfs.info/TrueCloudLab/frostfs-node/pkg/services/object/search/v2.(*streamWriter).WriteIDs(0xc000520320, {0xc00eb1a000, 0x4fd9c, 0x7fd6475a9a68?})
        git.frostfs.info/TrueCloudLab/frostfs-node/pkg/services/object/search/v2/streamer.go:28 +0x155
git.frostfs.info/TrueCloudLab/frostfs-node/pkg/services/object/search.(*uniqueIDWriter).WriteIDs(0xc001386420, {0xc00eb1a000?, 0xc0013ea9c0?, 0x113eef3?})
        git.frostfs.info/TrueCloudLab/frostfs-node/pkg/services/object/search/util.go:62 +0x202
git.frostfs.info/TrueCloudLab/frostfs-node/pkg/services/object/search.(*execCtx).writeIDList(0xc00011aa38?, {0xc00eb1a000?, 0xc001eec9f0?, 0xc0008f4380?})
        git.frostfs.info/TrueCloudLab/frostfs-node/pkg/services/object/search/exec.go:68 +0x91
git.frostfs.info/TrueCloudLab/frostfs-node/pkg/services/object/search.(*execCtx).executeLocal(0xc0008f4380, {0x176c538, 0xc001eec9f0})
        git.frostfs.info/TrueCloudLab/frostfs-node/pkg/services/object/search/local.go:18 +0x16b
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-12-28 12:29:22 +00:00
e44782473a [#1512] object: Fix writePart for EC-container
* Immediatly return after `ObjectAlreadyRemoved` error.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-12-26 11:27:55 +00:00
9cd1bcef06 [#1512] object: Make raw PutSingle check status within response
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-12-26 11:27:55 +00:00
ca0a33ea0f [#465] objsvc: Set NETMAP_EPOCH xheader for auxiliary requests
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-12-26 09:17:58 +00:00
1a091ea7bb [#1563] object: Wrap only ChainRouterError erros with ObjectAccessDenied
* Such wrapping helps to differentiate logical check errors and server internal
  errors.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-12-16 15:15:25 +03:00
f0c43c8d80
[#1502] Use zap.Error for logging errors
Use `zap.Error` instead of `zap.String` for logging errors: change all expressions like
`zap.String("error", err.Error())` or `zap.String("err", err.Error())` to `zap.Error(err)`.
Leave similar expressions with other messages unchanged, for example,
`zap.String("last_error", lastErr.Error())` or `zap.String("reason", ctx.Err().Error())`.

This change was made by applying the following patch:
```diff
@@
var err expression
@@
-zap.String("error", err.Error())
+zap.Error(err)

@@
var err expression
@@
-zap.String("err", err.Error())
+zap.Error(err)
```

Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-12-16 11:13:42 +03:00
7df3520d48 [#1540] getSvc: Drop redundant returns
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-12-05 12:39:49 +00:00
5fe78e51d1 [#1540] getSvc: Do not log context canceled errors during EC assemble
Those errors are fired when it is enough chunks retrieved and error group
cancels other requests.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-12-05 12:39:49 +00:00
7429553266
[#1437] node: Fix contextcheck linter
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-11-13 10:36:10 +03:00
6db46257c0
[#1437] node: Use ctx for logging
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-11-13 10:36:07 +03:00
c82c753e9f [#1480] objsvc: Remove useless stream wrappers
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-08 12:01:14 +00:00
f666898e5d [#1480] objsvc: Remove EACL checks
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-08 12:01:14 +00:00
b1a31281e4 [#1480] ape: Remove SoftAPECheck flag
Previous release was EACL-compatible.
Starting from now all EACL should've been migrated to APE chains.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-08 12:01:14 +00:00
9b13a18aac [#1479] go.mod: Bump frostfs-sdk-go version
* Update version within go.mod;
* Fix deprecated frostfs-api-go/v2 package and use frostfs-sdk-go/api
  instead.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-11-08 10:43:19 +03:00
ef64930fef
[#1477] ape: Fix EC chunk test
Initially, this test was a check that only the container node can
assemble an EC object. But the implementation of this test was wrong.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-11-07 16:00:20 +03:00
3cf6ea745d [#1451] ec: Check all parts are saved
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-11-06 08:18:10 +00:00
9a77527f46 [#1451] ape: Drop unused
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-11-06 08:18:10 +00:00
5b1ba8e23d [#1451] ape: Perform strict APE checks for EC parts
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-11-06 08:18:10 +00:00
9902965ff4 [#1451] writer: Sign EC parts with node's private key
As EC put request may be processed only by container node, so sign requests
with current node private to not to perform APE checks.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-11-06 08:18:10 +00:00
c2effcc61c [#1465] Makefile: Update golangci-lint, fix warnings
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-10-31 06:39:59 +00:00
7ac0852364 [#1459] .golangci.yml: Add intrange linter, fix issues
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-10-30 15:18:22 +00:00
87ac3c5279 [#1458] object: Make patch not set key before target construction
* `SignRequestPrivateKey` field should be initialized either within
  `newUntrustedTarget` or within `newTrustedTarget`. Otherwise, all
  requests are signed by local node key that makes impossible to perform
  patch on non-container node.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-10-29 15:20:28 +00:00
b42bcdc6fa
[#1433] services/object: Put object before auxiliary info
Consider the following operations ordering:
1. Inhume(with tombstone A) --> add tombstone mark for an object
2. --> new epoch arives
3. --> GCMark is added for a tombstone A, because it is unavailable
4. Put(A) --> return error, because the object already has a GCMark

It is possible, and I have successfully reproduced it with a test on the
shard level. However, the error is related to the specific
_ordering_ of operations with engine. And triggering race-conditions like
this is only possible on a shard level currently, so no tests are
written.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-10-17 14:43:13 +03:00
d83879d4b8 [#1431] node: Fix comment format
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2024-10-15 08:53:59 +00:00
f6582081a4 [#1431] obj_storage/metabase: Delete unused variable
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2024-10-15 08:53:59 +00:00
714ff784fa [#1431] objsvc: Use specific values in message about address mismatch
This makes troubleshooting failed operations much easier

Signed-off-by: Vitaliy Potyarkin <v.potyarkin@yadro.com>
2024-10-15 07:34:03 +00:00
acd6eb1815 [#1427] object: Fix Put for EC object when node unavailable
There might be situation when context canceled earlier than traverser move to another part of the nodes.
To avoid this, need to wait for the result from concurrent put at each traverser iteration.

Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-10-11 16:53:30 +03:00
4572fa4874
[#1412] searchSvc: Check container is indexed
For non S3 containers it is expected to use attributes index for some
attributes.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-10-08 11:41:50 +03:00
1b520f7973
[#1412] engine: Add IsIndexedContainer flag
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-10-08 11:41:49 +03:00
5f22ba6f38 [#1397] object: Correctly set namespace before APE check
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-09-27 11:43:29 +00:00
a5e1aa22c9 [#1394] putSvc: Fix relay
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-25 17:28:03 +03:00
4fbfffd44c [#1388] putSvc: Drop unused
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-25 08:55:38 +00:00
d1d6e3471c [#1388] signSvc: Drop unused
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-25 08:55:38 +00:00
580cd55180 [#1388] getSvc: Drop unused
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-25 08:55:38 +00:00
e319bf403e [#1388] apeSvc: Drop unused and make annotations
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-25 08:55:38 +00:00
839dead226 [#1297] getSvc: Return AccessDenied instead of ObjectNotFound
Do not replace the access denied error if it was received earlier.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-24 12:05:09 +03:00
3bb65ba820 [#1392] object: Fix target initialization within put streamer
* Remove `relay` field from put streamer as it's no longer used;
* Fix initialization of `Relay` object writer parameter.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-09-24 09:04:13 +00:00
d4493a6d08 [#1390] getSvc: Fix Head EC1.1
If local EC chunk found, but remote node is off, then `HEAD --raw` request
returns object not found.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-23 15:12:06 +03:00
61d5e140e0 [#1383] object: Add restrictions for Patch method
* `Patch` can't be applied for non-regular type object (tombstones,
  locks etc.)
* Complex object parts can't be patched. So, if an object has EC/Split
  header, it won't be patched.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-09-19 13:54:49 +03:00
a812932984 [#1362] ape: Move common APE check logic to separate package
* Tree and object service have the same log for checking APE. So,
  this check should be moved to common package.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-09-10 12:40:34 +00:00