EliChin/frostfs-node
1
0
Fork 0
forked from TrueCloudLab/frostfs-node
Code Issues Pull requests Projects Releases Packages Wiki Activity
3227 commits 12 branches 26 tags 25 MiB
Commit graph

3227 commits

This branch
This branch
All branches
Author SHA1 Message Date
Evgenii Stratonikov
d2cce62934 [#1818] writecache: Increase error counter on background errors 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-11-07 16:07:20 +03:00
Evgenii Stratonikov
a4a6d547a8 [#1818] writecache: Update storage ID during flush 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-11-07 16:07:20 +03:00
Evgenii Stratonikov
681400eed8 [#1818] metabase: Add UpdateStorageID operation 
By default writecache puts the whole object to update storage ID.
This logic comes from the times when we needed to put objects
in the metabase by the writecache itself. Now this is done by the
blobstor at unmarshaling objects during flush only to update storage ID
is an overkill.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-11-07 16:07:20 +03:00
Evgenii Stratonikov
b580846630 [#1818] writecache: Reuse FSTree flushing code between flushes 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-11-07 16:07:20 +03:00
Evgenii Stratonikov
6e2f7e291d [#1818] writecache: Remove unused variable 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-11-07 16:07:20 +03:00
Pavel Karpy
7a75b3aaaf [#1699] meta: Do not return SplitInfoError on Delete 
It is not an error: removing virtual object is expected and should be just
skipped. Getting a virtual object with `raw` flag is considered as an
impossible action, all the virtual objects removals will be handled via
their children's removals implicitly.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-11-07 16:04:09 +03:00
Evgenii Stratonikov
9cd8441dd5 [#1732] pilorama: Fill parent mark correctly 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-11-07 13:02:45 +03:00
Evgenii Stratonikov
163d8d778d [#1732] pilorama: Fix backwards log insertion 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-11-07 13:02:45 +03:00
Evgenii Stratonikov
52d85ca463 [#1732] pilorama: Improve logical error handling 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-11-07 13:02:45 +03:00
Evgenii Stratonikov
ba3db7fed5 [#1996] engine: Always select proper shard for a tree 
Currently there is a possibility for modifying operations to fail
because of I/O errors and a new tree to be created on another shard.
This commit adds existence check for modifying operations.
Read operations remain as they are, not to slow things.
`TreeDrop` is an exception, because this is a tree removal and trying
multiple shards is not an unwanted behaviour.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-11-07 13:02:45 +03:00
Pavel Karpy
2e89176892 [#1971] cli: Unify CID and OID flags provision 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-11-03 15:23:16 +03:00
Evgenii Stratonikov
855de87b62 [#2007] services/object: Allocate memory on-demand in GET_RANGE 
For big objects we want to get OutOfRange error before all the memory is
allocated.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-11-03 15:02:32 +03:00
Evgenii Stratonikov
289a7827c4 [#2007] services/object: Fix comment 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-11-03 15:02:32 +03:00
Pavel Karpy
abf4a63585 [#1991] cli: Refine container placement description 
Not to confuse a user by mixing a replication vector number with its copy
number.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-11-02 15:14:19 +03:00
Pavel Karpy
3ee4260647 [#2009] github: Run CI on support branches 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-11-02 14:27:36 +03:00
Evgenii Stratonikov
245da1e50e [#1992] writecache: Allow to open in NOSYNC mode 
Applicable only to FSTree as we cannot handle corrupted databases
properly yet.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-11-01 15:29:01 +03:00
Evgenii Stratonikov
a386448ab9 [#1992] neofs-node: Allow to open fstree in NOSYNC mode 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-11-01 15:29:01 +03:00
Evgenii Stratonikov
3c1f788642 [#1994] docs: Update storage node configuration 
Reflect the reality after a not so recent refactoring.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-11-01 15:29:01 +03:00
Evgenii Stratonikov
0de9efa685 [#1992] fstree: Allow working in SYNC mode 
Make O_SYNC the default and allow to opt-out explicitly.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-11-01 15:29:01 +03:00
Pavel Karpy
694d888219 Release v0.34.0 - Marado (마라도, 馬羅島) 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-31 15:07:53 +03:00
Pavel Karpy
d739e06289 [#1980] make: Update linter version in docker command 
To be in sync with GitHub actions.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-31 15:07:53 +03:00
Pavel Karpy
cd2faf29a9 [#1980] go.mod: Update neofs-sdk to rc7 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-31 15:07:53 +03:00
Evgenii Stratonikov
34501685b7 [#1969] local_object_storage: Move ErrObjectIsExpired to another package 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-31 11:41:24 +03:00
Evgenii Stratonikov
56de2f1363 [#1969] local_object_storage: Simplify logic error construction 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-31 11:41:24 +03:00
Evgenii Stratonikov
fcdbf5e509 [#1969] local_object_storage: Add a type for logical errors 
All logic errors are wrapped in `logicerr.Logical` type and do not
affect shard error counter.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-31 11:41:24 +03:00
Leonard Lyubich
98034005f1 [#1900] node: Fix loosing the "maintenance" status 
In previous implementation node lost maintenance status after successful
switching to it. For example, after some period of time node sent
bootstrap requests with the "online" state instead of "maintenance".

Make `startMaintenance` method to set maintenance status in the
`networkState`.

Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
 2022-10-28 18:30:59 +03:00
Leonard Lyubich
db92e96e40 [#1900] node: Refactor bootstrap methods 
In previous implementation bootstrapping state was chosen according to
bool flag which was not convenient.

Create separate method to boostrap with "online" and the current state.

Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
 2022-10-28 18:30:59 +03:00
Leonard Lyubich
e8c5f03c30 [#1905] shard: Don't log read-only errors of write-cache 
There is no need to log `writecache.ErrReadOnly` errors in `Delete`
method of the `Shard`.

Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
 2022-10-28 18:30:45 +03:00
Leonard Lyubich
b1fa084756 [#1905] shard: Decrease severity level of write-cache failure logs 
In previous implementation `Shard.Delete` logged writecache's removal
failures in `error` level. There is a need to decrease severity of these
log records since they aren't critical and don't require individual
review.

Change level of the message to `info`.

Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
 2022-10-28 18:30:45 +03:00
Leonard Lyubich
98ac525272 [#1978] cli/object: Gather all related object in delete session 
Object removal session should reflect all objects related to the
removing one.

Make `OpenSessionViaClient` to gather the split members of the original
object in order to spread the session to them.

Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
 2022-10-28 13:49:41 +03:00
Stanislav Bogatyrev
7653a1f626 [#409] Fix package building doc typos 
Signed-off-by: Stanislav Bogatyrev <stanislav@nspcc.ru>
 2022-10-28 12:58:32 +03:00
Stanislav Bogatyrev
43fe156f7c [#409] Fix more lintian warnings 
Signed-off-by: Stanislav Bogatyrev <stanislav@nspcc.ru>
 2022-10-28 12:58:32 +03:00
Dmitriy Zabolotskiy
c640374d98 [#409] Return OS release suffix to package version 
Signed-off-by: Dmitriy Zabolotskiy <d.zabolotskiy@yadro.com>
 2022-10-28 12:58:32 +03:00
Dmitriy Zabolotskiy
1e0b1919ed [#409] Fix lintian warnings 
Signed-off-by: Dmitriy Zabolotskiy <d.zabolotskiy@yadro.com>
 2022-10-28 12:58:32 +03:00
Stanislav Bogatyrev
bbfc1d9263 [#409] Fix neofs-locode-db dependency 
And update CHANGELOG.md =)

Signed-off-by: Stanislav Bogatyrev <stanislav@nspcc.ru>
 2022-10-28 12:58:32 +03:00
Stanislav Bogatyrev
e1a085ffd5 [#409] Remove docker target 
It would not work this way for non-root users. Let's postpone.

Signed-off-by: Stanislav Bogatyrev <stanislav@nspcc.ru>
 2022-10-28 12:58:32 +03:00
Stanislav Bogatyrev
ff67e903ca [#409] Add default package changelog 
Make it work with simple `dpkg-buildpackage`

Signed-off-by: Stanislav Bogatyrev <stanislav@nspcc.ru>
 2022-10-28 12:58:32 +03:00
Dmitriy Zabolotskiy
92e9782c44 [#409] Debian packaging 
Debian package building, including:

 - users creation if not exist;
 - dirs creation and ownership/ACLs set;
 - man/doc/examples installation;
 - shell completion generation;
 - locode DB binary download

Using:
	make debpackage

Signed-off-by: Dmitriy Zabolotskiy <d.zabolotskiy@yadro.com>
 2022-10-28 12:58:32 +03:00
Evgenii Stratonikov
846ff515e6 [#1812] policer: Do not remove copies if there are maintenance nodes 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-26 19:13:17 +03:00
Evgenii Stratonikov
280e56f4bb [#1893] neofs-node: Do not fail unless all gRPC endpoints are unavailable 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-26 12:48:38 +03:00
Evgenii Stratonikov
1e6588e761 [#1974] shard: Do not panic in degraded mode 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-26 12:41:12 +03:00
Evgenii Stratonikov
7395ab8ef7 [#1944] metabase: Assume static buckets are created on Init 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-26 12:41:12 +03:00
Evgenii Stratonikov
e4bc3d0e9d [#1944] metabase: Recreate static buckets instead of resetting 
From the `Bucket.ForEach` doc:
```
The provided function must not modify the bucket; this will result in undefined behavior.
```

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-26 12:41:12 +03:00
Leonard Lyubich
3c6daa2995 [#1883] cli/shards: Exclude degraded-read-write mode from help 
The mode is not expected to be used by the user. It is supported only
for developers.

Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
 2022-10-26 08:24:40 +03:00
Leonard Lyubich
eb206d6e59 [#1883] config/example: Actualize desc of shard modes 
Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
 2022-10-26 08:24:40 +03:00
Leonard Lyubich
c4a2a283ae [#1883] cli/shards: Make supporting new modes easier 
Add new shard modes as a map entry to automatically parse them in
`set-mode` command. The change also automatically adds new modes to help
message.

Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
 2022-10-26 08:24:40 +03:00
Evgenii Stratonikov
3b939d190c [#1957] engine: Move shard to read-only if cannot move to degraded 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-26 08:20:53 +03:00
Leonard Lyubich
7f54462e60 [#1920] cli: Print used RPC timeout in verbose exec 
Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
 2022-10-26 08:20:36 +03:00
Pavel Karpy
9a0824766c [#1713] node: Add Neo RPC switch interval to the config 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-25 17:38:48 +03:00
Pavel Karpy
0631b38da1 [#1713] ir: Add Neo RPC switch interval to the config 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-25 17:38:48 +03:00