aarifullin/frostfs-node
1
0
Fork 0
forked from TrueCloudLab/frostfs-node
Code Issues Pull requests Projects Releases Packages Wiki Activity
3006 commits 130 branches 26 tags 52 MiB
Commit graph

3006 commits

This branch
This branch
All branches
Author SHA1 Message Date
Evgenii Stratonikov
3d882e9f47 [#1806] engine: Allow to flush write-cache 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-28 09:28:01 +03:00
Evgenii Stratonikov
0b4c867ef1 [#1806] writecache: Allow to start flush manually 
Allow user to initiate flushing objects from a writecache.
We need this in 2 cases:
1. During writecache storage schema update, it should be flushed with
   the old version of node and started clean with a new one.
2. During SSD replacement, to avoid data loss.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-28 09:28:01 +03:00
Evgenii Stratonikov
55148404ee [#1806] writecache: Use interfaces for blobstor and metabase 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-28 09:28:01 +03:00
Evgenii Stratonikov
36ab1a2472 [#1731] neofs-cli: Make --endpoint flag optional 
Allow to provide it using config file.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-28 09:28:01 +03:00
Leonard Lyubich
93742d37b7 [#1793] node/netmap: Change interface of the latest network map reader 
Replace `ProcessCurrentNetMap` method of `NodeState` interface with
`ReadCurrentNetMap` one with two changes:
 * Replace network map type from NeoFS SDK package with the
   protocol-generated message. This replaces all the business logic to
   the application layer.
 * Support error return. This allows to cover problem node states.

Return an error from `NodeState.ReadCurrentNetMap` method implemeted
through `atomic.Value` if `Store` method has not been called yet.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-09-27 09:03:17 +03:00
Leonard Lyubich
485a5418d2 [#1793] node: Serve NetmapService.NetmapSnapshot RPC 
There is no more need to serve the same request on Control API.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-09-27 09:03:17 +03:00
Evgenii Stratonikov
59de20fbba [#1764] neofs-node: Allow to check configuration 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-27 08:57:20 +03:00
Evgenii Stratonikov
0fb5c51ac9 [#1764] neofs-node: Validate config before usage 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-27 08:57:20 +03:00
Evgenii Stratonikov
6f45cc81fc [#1764] neofs-node: Use constants for storage types 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-27 08:57:20 +03:00
Evgenii Stratonikov
9113793688 [#1764] neofs-node: Allow to return error from IterateShards 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-27 08:57:20 +03:00
Evgenii Stratonikov
898689ec14 [#1731] services/replicator: Unify Task interface with other parameters 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-24 13:47:48 +03:00
Evgenii Stratonikov
4e043a801c [#1731] services/control: Replicate object over network in EvacuateShard RPC 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-24 13:47:48 +03:00
Evgenii Stratonikov
a49137349b [#1731] engine: Allow to use user handler for evacuated objects 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-24 13:47:48 +03:00
Pavel Karpy
93ae3f0b19 [#1808] .github: Add changelog workflow 
It checks whether the CHANGELOG.md was updated in a just-created PR.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-09-23 10:20:25 +04:00
Evgenii Stratonikov
7cc68cf4d4 [#1810] blobovniczatree: Add a test for blobovnicva usage 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-23 10:19:26 +04:00
Anton Nikiforov
c7f85994e5 [nspcc-dev#1692] cli: Remove --generate-key option in neofs-cli container delete 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2022-09-20 07:19:46 +04:00
Anton Nikiforov
bb02913c39 [nspcc-dev#1128] cli: Remove WIF and NEP2 support in --wallet argument 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2022-09-19 14:21:37 +03:00
Evgenii Stratonikov
3df98ce7ba [#1731] engine: Return the amount of actually moved objects in Evacuate 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-19 11:33:52 +03:00
Evgenii Stratonikov
8fc88487db [#1731] neofs-cli: Add control shards evacuate command 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-19 11:33:52 +03:00
Evgenii Stratonikov
091d7d30f6 [#1731] services/control: Allow to evacuate data from shard 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-19 11:33:52 +03:00
Evgenii Stratonikov
a51b76056e [#1731] engine: Add Evacuate command 
Make it possible to move all data from 1 shard to other shards.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-19 11:33:52 +03:00
Evgenii Stratonikov
7377979e12 [#1731] engine: Move single shard PUT to a separate function 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-19 11:33:52 +03:00
Evgenii Stratonikov
4208f7c0cf [#1741] neofs-adm: Use ValidUntilBlock to wait for tx persist 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-15 14:27:16 +03:00
Evgenii Stratonikov
5321f8ef9c [#1786] engine: Unify parameter setters 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-15 10:28:48 +03:00
Evgenii Stratonikov
b064fb24d8 [#1616] engine: Do not use batches in delete 
Use a simple loop at the callsite. This way we remove as much as we can.
Also, `Delete` metrics is more meaningful now.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-15 10:28:48 +03:00
Evgenii Stratonikov
cda8f9df2e [#1786] services/control: Remove WithDeletedObjectHandler option 
Use storage engine directly instead. It is already provided in the
options.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-15 10:28:48 +03:00
Evgenii Stratonikov
ae1dab29bc [#1482] metabase: Encode database keys in binary 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-15 10:28:26 +03:00
Leonard Lyubich
d6fef68a62 Release v0.32.0 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-09-14 17:52:49 +04:00
Leonard Lyubich
d895a272ad docs: Improve release instructions 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-09-14 17:52:49 +04:00
Leonard Lyubich
ae655b74f0 *: Exec make fmts 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-09-14 17:52:49 +04:00
Evgenii Stratonikov
4b8b575ca4 [#1782] go.mod: Update neofs-sdk-go 
Add fixes for key parameter in `object.Get` operation.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-14 13:18:19 +04:00
Pavel Karpy
532c41ca05 [#1658] meta: Do not check object expiration in counters 
Do not perform operations that produce unused results in `syncCounter`.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-09-13 21:32:37 +04:00
Pavel Karpy
431e331373 [#1658] shard: Update metric counters 
Use meta's operation results to change the metrics. Support typed object
counters.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-09-13 21:32:37 +04:00
Pavel Karpy
ad47e2a985 [#1658] meta: Add logic counter 
- Meta now supports (and requires) inc/dec labeled counters
- The new logic counter is incremented on `Put` operations and is
decremented on `Inhume` and `Delete` operations that are performed on
_stored_ objects only
- Allow force counters sync. "Force" mode should be used on metabase resync
and should not be used on a regular meta start

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-09-13 21:32:37 +04:00
Pavel Karpy
d872862710 [#1658] meta: Force counters resync process 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-09-13 21:32:37 +04:00
Pavel Karpy
beb7f2a048 [#1634] node: Change default epoch in tests 
Do not treat objects with expiration as expired by default.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-09-13 21:32:37 +04:00
Pavel Karpy
edef26a4fd [#1658] engine: Update metrics interfaces 
It now supports typed object counter metrics.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-09-13 21:32:37 +04:00
Pavel Karpy
175e2b9fa0 [#1658] metrics: Add object type label to the counters 
It will allow to use labeled object counters, e.g.: "phy" for all
the physically stored objects, "logic" for only available objects. Also, it
will allow to add typed (regular, TS, SG, LOCK) counters if needed.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-09-13 21:32:37 +04:00
Leonard Lyubich
a789159e5b [#1189] config/example: Mention persistent_sessions.path default 
In current implementation storage node app uses in-memory session
storage if `persistent_sessions.path` value is empty/missing in the
config.

Clarify default behavior in `config/example/node.yaml`.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-09-13 21:29:04 +04:00
Leonard Lyubich
d25b7e177b [#1189] node/config: Fix PersistentSessionsConfig.Path method's docs 
Method never returns `PersistentStatePathDefault` value, and this is
correct.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-09-13 21:29:04 +04:00
Pavel Karpy
4083a62679 [#1628] tree: Pre-allocate errors in ACL checks 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-09-13 10:33:50 +03:00
Pavel Karpy
5adf089c1d [#1628] tree: Log unacceptable bearer attachment 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-09-13 10:33:50 +03:00
Pavel Karpy
2ffcd02ac3 [#1628] tree: Document ACL checks in tree service 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-09-13 10:33:50 +03:00
Pavel Karpy
8d0906c6ab [#1628] tree: Skip eACL filters for tree requests 
Do not call `CalculateAction` for the eACL checks since it requires object
headers that are meaningless in the tree context.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-09-13 10:33:50 +03:00
Pavel Karpy
876e014b5d [#1628] tree: Make ACL checks the same way as for object requests 
1. Do not require a request to be signed by the container owner if a
bearer token is missing
2. Do not check the system role since public requests are not expected to
be signed by IR or a container node (unlike the object requests)

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-09-13 10:33:50 +03:00
Pavel Karpy
4f18893d9b [#1628] node: Move common EACLSource interface to core pkg 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-09-13 10:33:50 +03:00
Pavel Karpy
4afb928ab6 [#1674] *: Expire entities after the expiration epoch 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-09-12 19:56:59 +04:00
Evgenii Stratonikov
f1572a674b [#1775] go.mod: Update neofs-sdk-go 
Make use of fixes for the placement policy parser.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-12 19:10:49 +04:00
Evgenii Stratonikov
d25ec52459 [#1778] neofs-node: Panic if config type is invalid 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-12 17:06:00 +03:00
Evgenii Stratonikov
60b2930417 [#1778] neofs-node: Fix blobovnicza config type 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-12 17:06:00 +03:00