carpawell/frostfs-node
1
0
Fork 0
forked from TrueCloudLab/frostfs-node
Code Issues Pull requests Projects Releases Packages Wiki Activity
3527 commits 27 branches 26 tags 21 MiB
Commit graph

3527 commits

This branch
This branch
All branches
Author SHA1 Message Date
Evgenii Stratonikov
134f2ba02e [#1732] pilorama: Fix backwards log insertion 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-11-19 11:01:04 +03:00
Evgenii Stratonikov
2ef38cfbc4 [#1996] engine: Ignore pilorama.ErrTreeNotFound for write operations 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-11-19 11:01:04 +03:00
Evgenii Stratonikov
a455ec18c3 [#2007] services/object: Allocate memory on-demand in GET_RANGE 
For big objects we want to get OutOfRange error before all the memory is
allocated.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-11-19 11:01:04 +03:00
Evgenii Stratonikov
ff5526038d [#2007] services/object: Fix comment 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-11-19 11:01:04 +03:00
Evgenii Stratonikov
d8d3588e1b [#1996] engine: Always select proper shard for a tree 
Currently there is a possibility for modifying operations to fail
because of I/O errors and a new tree to be created on another shard.
This commit adds existence check for modifying operations.
Read operations remain as they are, not to slow things.
`TreeDrop` is an exception, because this is a tree removal and trying
multiple shards is not an unwanted behaviour.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-11-03 15:29:23 +03:00
Pavel Karpy
08efe6eb11 [#2010] github: Run CI on support branches 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-11-02 14:29:17 +03:00
Evgenii Stratonikov
777fd32d4f [#1818] writecache: Increase error counter on background errors 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-11-02 14:24:02 +03:00
Evgenii Stratonikov
bffb0f894c [#1818] writecache: Update storage ID during flush 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-11-02 14:24:02 +03:00
Evgenii Stratonikov
5cf75404dc [#1818] metabase: Add UpdateStorageID operation 
By default writecache puts the whole object to update storage ID.
This logic comes from the times when we needed to put objects
in the metabase by the writecache itself. Now this is done by the
blobstor at unmarshaling objects during flush only to update storage ID
is an overkill.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-11-02 14:24:02 +03:00
Evgenii Stratonikov
b64b14eb54 [#1818] writecache: Reuse FSTree flushing code between flushes 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-11-02 14:24:02 +03:00
Evgenii Stratonikov
a56927e3d4 [#1818] writecache: Remove unused variable 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-11-02 14:24:02 +03:00
Anton Nikiforov
bb52857b2b [#1338] neofs-cli: Add support to store/restore/delete binary objects 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2022-11-01 15:30:00 +03:00
Pavel Karpy
1f82c583e3 [#1971] cli: Unify CID and OID flags provision 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-11-01 15:07:19 +03:00
Pavel Karpy
7daa57d4d2 [#1991] cli: Refine container placement description 
Not to confuse a user by mixing a replication vector number with its copy
number.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-11-01 15:00:00 +03:00
Anton Nikiforov
51e3810285 [#1689] Add new command morph list-containers in neofs-adm 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2022-11-01 09:43:02 +03:00
Evgenii Stratonikov
98a152256b [#1992] writecache: Allow to open in NOSYNC mode 
Applicable only to FSTree as we cannot handle corrupted databases
properly yet.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-11-01 09:42:26 +03:00
Evgenii Stratonikov
b6930f2219 [#1992] neofs-node: Allow to open fstree in NOSYNC mode 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-11-01 09:42:26 +03:00
Evgenii Stratonikov
148da5fdbb [#1994] docs: Update storage node configuration 
Reflect the reality after a not so recent refactoring.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-11-01 09:42:26 +03:00
Evgenii Stratonikov
f564430b90 [#1992] fstree: Allow working in SYNC mode 
Make O_SYNC the default and allow to opt-out explicitly.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-11-01 09:42:26 +03:00
Pavel Karpy
694d888219 Release v0.34.0 - Marado (마라도, 馬羅島) 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-31 15:07:53 +03:00
Pavel Karpy
d739e06289 [#1980] make: Update linter version in docker command 
To be in sync with GitHub actions.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-31 15:07:53 +03:00
Pavel Karpy
cd2faf29a9 [#1980] go.mod: Update neofs-sdk to rc7 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-31 15:07:53 +03:00
Evgenii Stratonikov
34501685b7 [#1969] local_object_storage: Move ErrObjectIsExpired to another package 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-31 11:41:24 +03:00
Evgenii Stratonikov
56de2f1363 [#1969] local_object_storage: Simplify logic error construction 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-31 11:41:24 +03:00
Evgenii Stratonikov
fcdbf5e509 [#1969] local_object_storage: Add a type for logical errors 
All logic errors are wrapped in `logicerr.Logical` type and do not
affect shard error counter.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-31 11:41:24 +03:00
Leonard Lyubich
98034005f1 [#1900] node: Fix loosing the "maintenance" status 
In previous implementation node lost maintenance status after successful
switching to it. For example, after some period of time node sent
bootstrap requests with the "online" state instead of "maintenance".

Make `startMaintenance` method to set maintenance status in the
`networkState`.

Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
 2022-10-28 18:30:59 +03:00
Leonard Lyubich
db92e96e40 [#1900] node: Refactor bootstrap methods 
In previous implementation bootstrapping state was chosen according to
bool flag which was not convenient.

Create separate method to boostrap with "online" and the current state.

Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
 2022-10-28 18:30:59 +03:00
Leonard Lyubich
e8c5f03c30 [#1905] shard: Don't log read-only errors of write-cache 
There is no need to log `writecache.ErrReadOnly` errors in `Delete`
method of the `Shard`.

Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
 2022-10-28 18:30:45 +03:00
Leonard Lyubich
b1fa084756 [#1905] shard: Decrease severity level of write-cache failure logs 
In previous implementation `Shard.Delete` logged writecache's removal
failures in `error` level. There is a need to decrease severity of these
log records since they aren't critical and don't require individual
review.

Change level of the message to `info`.

Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
 2022-10-28 18:30:45 +03:00
Leonard Lyubich
98ac525272 [#1978] cli/object: Gather all related object in delete session 
Object removal session should reflect all objects related to the
removing one.

Make `OpenSessionViaClient` to gather the split members of the original
object in order to spread the session to them.

Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
 2022-10-28 13:49:41 +03:00
Stanislav Bogatyrev
7653a1f626 [#409] Fix package building doc typos 
Signed-off-by: Stanislav Bogatyrev <stanislav@nspcc.ru>
 2022-10-28 12:58:32 +03:00
Stanislav Bogatyrev
43fe156f7c [#409] Fix more lintian warnings 
Signed-off-by: Stanislav Bogatyrev <stanislav@nspcc.ru>
 2022-10-28 12:58:32 +03:00
Dmitriy Zabolotskiy
c640374d98 [#409] Return OS release suffix to package version 
Signed-off-by: Dmitriy Zabolotskiy <d.zabolotskiy@yadro.com>
 2022-10-28 12:58:32 +03:00
Dmitriy Zabolotskiy
1e0b1919ed [#409] Fix lintian warnings 
Signed-off-by: Dmitriy Zabolotskiy <d.zabolotskiy@yadro.com>
 2022-10-28 12:58:32 +03:00
Stanislav Bogatyrev
bbfc1d9263 [#409] Fix neofs-locode-db dependency 
And update CHANGELOG.md =)

Signed-off-by: Stanislav Bogatyrev <stanislav@nspcc.ru>
 2022-10-28 12:58:32 +03:00
Stanislav Bogatyrev
e1a085ffd5 [#409] Remove docker target 
It would not work this way for non-root users. Let's postpone.

Signed-off-by: Stanislav Bogatyrev <stanislav@nspcc.ru>
 2022-10-28 12:58:32 +03:00
Stanislav Bogatyrev
ff67e903ca [#409] Add default package changelog 
Make it work with simple `dpkg-buildpackage`

Signed-off-by: Stanislav Bogatyrev <stanislav@nspcc.ru>
 2022-10-28 12:58:32 +03:00
Dmitriy Zabolotskiy
92e9782c44 [#409] Debian packaging 
Debian package building, including:

 - users creation if not exist;
 - dirs creation and ownership/ACLs set;
 - man/doc/examples installation;
 - shell completion generation;
 - locode DB binary download

Using:
	make debpackage

Signed-off-by: Dmitriy Zabolotskiy <d.zabolotskiy@yadro.com>
 2022-10-28 12:58:32 +03:00
Evgenii Stratonikov
846ff515e6 [#1812] policer: Do not remove copies if there are maintenance nodes 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-26 19:13:17 +03:00
Evgenii Stratonikov
280e56f4bb [#1893] neofs-node: Do not fail unless all gRPC endpoints are unavailable 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-26 12:48:38 +03:00
Evgenii Stratonikov
1e6588e761 [#1974] shard: Do not panic in degraded mode 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-26 12:41:12 +03:00
Evgenii Stratonikov
7395ab8ef7 [#1944] metabase: Assume static buckets are created on Init 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-26 12:41:12 +03:00
Evgenii Stratonikov
e4bc3d0e9d [#1944] metabase: Recreate static buckets instead of resetting 
From the `Bucket.ForEach` doc:
```
The provided function must not modify the bucket; this will result in undefined behavior.
```

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-26 12:41:12 +03:00
Leonard Lyubich
3c6daa2995 [#1883] cli/shards: Exclude degraded-read-write mode from help 
The mode is not expected to be used by the user. It is supported only
for developers.

Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
 2022-10-26 08:24:40 +03:00
Leonard Lyubich
eb206d6e59 [#1883] config/example: Actualize desc of shard modes 
Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
 2022-10-26 08:24:40 +03:00
Leonard Lyubich
c4a2a283ae [#1883] cli/shards: Make supporting new modes easier 
Add new shard modes as a map entry to automatically parse them in
`set-mode` command. The change also automatically adds new modes to help
message.

Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
 2022-10-26 08:24:40 +03:00
Evgenii Stratonikov
3b939d190c [#1957] engine: Move shard to read-only if cannot move to degraded 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-26 08:20:53 +03:00
Leonard Lyubich
7f54462e60 [#1920] cli: Print used RPC timeout in verbose exec 
Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
 2022-10-26 08:20:36 +03:00
Pavel Karpy
9a0824766c [#1713] node: Add Neo RPC switch interval to the config 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-25 17:38:48 +03:00
Pavel Karpy
0631b38da1 [#1713] ir: Add Neo RPC switch interval to the config 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-25 17:38:48 +03:00