Tree sync is too much different from GC and rebuild to use the same tag for GC and tree sync.
Change-Id: Ib44d5fa9a88daff507d759d0b0410cc9272e236f
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
When we do `object patch` with audit enabled we get several
duplicated entries in logs.
`object patch` request is logged in 2 places:
1. `(*auditPatchStream) CloseAndRecv()` - when the client closes
the request stream or when stream gets aborted.
2. `(*auditPatchStream) Send()` - when stream was NOT aborted.
`Send()` doesn't check if `err != nil` before logging.
It led to to logging on every `Send()` call.
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
Added method `ListShardsForObject` to ControlService and to
StorageEngine. It returns information about shards storing
object on the node.
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
By default, gRPC fetches TXT report while resolving a domain.
0914bba6c5/internal/resolver/dns/dns_resolver.go (L336)
This leads to a hanging dial if DNS is unavailable, even though the host
may be specified in `/etc/hosts` (hello, localhost!).
SDK client for the main API uses these options by default.
Refs TrueCloudLab/frostfs-sdk-go#342
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
When we do `object put` with audit enabled we get several entries
in logs: with and without object id.
`object put` request is logged in 2 places:
1. `(*auditPutStream) CloseAndRecv()` - when the client closes the
request stream or when stream gets aborted.
2. `(*auditPutStream) Send()` - when stream was NOT aborted.
`Send()` does error check for `ErrAbortStream` because if there
is any other error - CloseAndRecv will not be called and there
won't be any audit log about failed request.
It led to logging on every object chunck put, even if `err == nil`.
Added check for `err != nil` in `Send()` to fix it.
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
* A client may open stream to server, not send anything and close
the open stream immediatly. This shouldn't cause a panic;
* Return the error if `s.patcher` is uninitialized. Uninitialized
patcher cannot be closed, this causes a panic.
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
If applyOperationStream() exits prematurely, other goroutines will block
on send and errgroup will never finish waiting. In this commit we also
check whether context is cancelled.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Constant string `testOwnerID` for these tests has an invalid
format. It has 11 bytes instead of required 25 for `user.ID`.
It worked because:
1. `user.ID` was a byte slice and didn't check length
and format of byte slices decoded from strings.
2. in these tests `testOwnerID` was used only to decode
container owner id and to compare it with owner id encoded
back to string.
Since `user.ID implementation has changed`, the problem arised.
Now `testOwnerID` is valid.
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
Current flow is hard to reason about, #1601 is a notorious example of
accidental complexity.
1. Remove multiple nested ifs, use depth=1.
2. Process each status exactly once, hopefully preventing bugs like
#1601.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Consider `REP 1 REP 1` placement (selects/filters are omitted).
The placement is `[1, 2], [1, 0]`. We are the 0-th node.
Node 1 is under maintenance, so we do not replicate object
on the node 2. In the second replication group node 1 is under maintenance,
but current caching logic considers it as "replica holder" and removes
local copy. Voilà, we have DL if the object is missing from the node 1.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
The node can have MAINTENANCE status in the network map, but can also be
ONLINE while responding with MAINTENANCE. These are 2 different code
paths, let's test them separately.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
* X-Headers can be found in `origin` field of `MetaHeader` if the request
has been forwarded from non-container node.
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
Previously, `ln` was only set once, so search has really worked for
small number of objects.
Fix panic:
```
panic: runtime error: slice bounds out of range [:43690] with capacity 21238
goroutine 6859775 [running]:
git.frostfs.info/TrueCloudLab/frostfs-node/pkg/services/object.(*searchStreamMsgSizeCtrl).Send(0xc001eec8d0, 0xc005734000)
git.frostfs.info/TrueCloudLab/frostfs-node/pkg/services/object/transport_splitter.go:173 +0x1f0
git.frostfs.info/TrueCloudLab/frostfs-node/pkg/services/object/search/v2.(*streamWriter).WriteIDs(0xc000520320, {0xc00eb1a000, 0x4fd9c, 0x7fd6475a9a68?})
git.frostfs.info/TrueCloudLab/frostfs-node/pkg/services/object/search/v2/streamer.go:28 +0x155
git.frostfs.info/TrueCloudLab/frostfs-node/pkg/services/object/search.(*uniqueIDWriter).WriteIDs(0xc001386420, {0xc00eb1a000?, 0xc0013ea9c0?, 0x113eef3?})
git.frostfs.info/TrueCloudLab/frostfs-node/pkg/services/object/search/util.go:62 +0x202
git.frostfs.info/TrueCloudLab/frostfs-node/pkg/services/object/search.(*execCtx).writeIDList(0xc00011aa38?, {0xc00eb1a000?, 0xc001eec9f0?, 0xc0008f4380?})
git.frostfs.info/TrueCloudLab/frostfs-node/pkg/services/object/search/exec.go:68 +0x91
git.frostfs.info/TrueCloudLab/frostfs-node/pkg/services/object/search.(*execCtx).executeLocal(0xc0008f4380, {0x176c538, 0xc001eec9f0})
git.frostfs.info/TrueCloudLab/frostfs-node/pkg/services/object/search/local.go:18 +0x16b
```
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
* Separated iteration through container ids from `ContainersOf()`
so that it could be reused.
* When listing containers we used to iterate through the
the whole list of containers twice: first when reading from
a contract, then when sending them. Now we can send batches
of containers when reading from the contract.
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
