carpawell/frostfs-node
1
0
Fork 0
forked from TrueCloudLab/frostfs-node
Code Issues Pull requests Projects Releases Packages Wiki Activity
3205 commits 27 branches 26 tags 21 MiB
Commit graph

3205 commits

This branch
This branch
All branches
Author SHA1 Message Date
Leonard Lyubich
d15a7d8d3d [#1632] node/container: Update GET cache on successful removal 
In previous implementation storage node responded with the removed
container up until cache invalidation due to TTL. In order to avoid
false-positive responses node should update its cache on `DeleteSuccess`
events.

Make node to call `handleRemoval` method of the container cache which
leads to subsequent `apistatus.ErrContainerNotFound` errors.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-09-02 11:29:55 +04:00
Leonard Lyubich
a529149e6f [#1632] node: Cache Sidechain errors along with the values 
In previous implementation failed requests to the Sidechain weren't
cached. It makes sense to cache errors along with the values in order to
decrease potential load spikes onto Sidechain nodes.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-09-02 11:29:55 +04:00
Leonard Lyubich
7c1babb7d6 [#1632] node: Subscribe on the successful container creations/removals 
There is a need to sync container-related caching mechanism with the
actual Sidechain changes. To do this, node should be able to listen
incoming notifications about container ops.

Define `PutSuccess` / `DeleteSuccess` notification event's parsers.
Subscribe to these events in node app. As initial implementation node
will log event receipts. Later handling is going to be practically
complicated.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-09-02 11:29:55 +04:00
Evgenii Stratonikov
d8a00c365a [#1691] blobovniczatree: Fix active blobovnicza caching 
Maintain an invariant that any blobovnicza is present either
in `opened` or in `active` map. Otherwise, the logic becomes too
complicate because it is not obvious when we should close the blobovnicza.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-08-31 19:11:48 +03:00
Evgenii Stratonikov
dfac4e1c0b [#1691] blobovniczatree: Do not fill cache during initialization 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-08-31 19:11:48 +03:00
Evgenii Stratonikov
df88afeef3 [#1738] neofs-adm: Clean tx hash list after awaiting 
Do not query application log for the same transaction more than once.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-08-30 16:53:00 +03:00
Evgenii Stratonikov
ae52d53609 [#1698] pilorama: Add a test for the empty FileName 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-08-30 16:52:51 +03:00
Evgenii Stratonikov
180e5e938f [#1735] go.mod: Update API and SDK 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-08-30 12:27:46 +03:00
Evgenii Stratonikov
78bf17c0b0 [#1686] blobstor/*: Remove unneeded tests 
The blobovniczatree test also checks the amount of objects we can put,
so leave it here.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-08-30 10:02:43 +03:00
Evgenii Stratonikov
b9a2055e1c [#1686] blobstor/*: Return ErrReadOnly for modifying operations 
This check should occur on the shard level, but because
blobstor components expose `Open(readOnly bool)` interface,
it is reasonable to expect an error here.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-08-30 10:02:43 +03:00
Evgenii Stratonikov
6c2d3b020f [#1686] fstree: Return proper error for concurrent Delete 
If the file doesn't exist, return `apistatus.ObjectNotFound`.
First check is still there as a shortcut.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-08-30 10:02:43 +03:00
Evgenii Stratonikov
0b95a21701 [#1686] blobstor: Add generic tests 
This tests check that each blobstor component behaves similarly when
same methods are being used. It is intended to serve as a specification
for all future components.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-08-30 10:02:43 +03:00
Evgenii Stratonikov
b2d4cc556e [#1686] blobovnicza: Remove IterateObjects 
It is used only once, makes sense to inline.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-08-30 10:02:43 +03:00
Evgenii Stratonikov
5f95498364 [#1686] blobstor: Add generic tests 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-08-30 10:02:43 +03:00
Evgenii Stratonikov
9792e6a4ef [#1686] metabase: Add generic tests 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-08-30 10:02:43 +03:00
Evgenii Stratonikov
482a7e7f2f [#1686] pilorama: Add generic tests 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-08-30 10:02:43 +03:00
Evgenii Stratonikov
b8b9d25f9d [#1686] local_object_storage: Add generic tests 
Use them for writecache as a simple example.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-08-30 10:02:43 +03:00
Evgenii Stratonikov
bc20851cd1 [#1686] shard: Open all components in case metabase has failed 
All components must be opened first, before any other operation is
performed.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-08-30 10:02:43 +03:00
Pavel Karpy
c7c1c257e1 [#1658] node: Init object counter on meta's Init 
Includes:
1. Renaming counter key to distinguish logical and physical objects
2. Version update dropping since changes could be done in a compatible way

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-08-25 19:20:33 +03:00
Pavel Karpy
cd6f8e051a [#1658] Update CHANGELOG 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-08-25 19:20:33 +03:00
Pavel Karpy
9c7b3ce799 [#1658] node: Read metrics from meta on startup 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-08-25 19:20:33 +03:00
Pavel Karpy
745f72fff0 [#1658] node: Support object counter metrics 
Increment shard's object counter on successful `Put` calls and decrement on
`Delete`.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-08-25 19:20:33 +03:00
Pavel Karpy
067ab44e0b [#1658] metrics: Add object counter metric 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-08-25 19:20:33 +03:00
Pavel Karpy
626766db08 [#1658] meta: Add object counter 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-08-25 19:20:33 +03:00
Evgenii Stratonikov
61f0d85834 [#1727] neofs-adm: Allow to register custom TLD for contracts 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-08-25 12:31:00 +03:00
Evgenii Stratonikov
ddbe129cb2 [#1729] neofs-adm: Change default domain expiration time 
It is a noop for current contract versions, but will matter in the next
release.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-08-25 12:31:00 +03:00
Evgenii Stratonikov
5827a926c9 [#1729] go.mod: Update neofs-contract to v0.15.5 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-08-25 12:31:00 +03:00
Evgenii Stratonikov
19585c63ed [#1730] go.mod: Update neofs-sdk-go 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-08-24 17:17:42 +03:00
Alex Vanin
e68c80df0a [#1728] Don't describe default value twice 
Cobra will mention default value automatically.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2022-08-24 16:44:44 +03:00
Alex Vanin
31b6af5667 [#1728] Update neofs-adm morph deploy command description 
Add more details about expected file names in contract dir.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2022-08-24 16:44:44 +03:00
Pavel Karpy
b54f34d7df [#1683] adm: Do not deploy custom contract with a group 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-08-24 13:52:55 +03:00
Pavel Karpy
bbd6b9780f [#1683] adm: Minimize registration price 
Register NNS domain in one TX:
1. Set minimal (`1`) registration price;
2. Register domain;
3. Return registration price back.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-08-24 13:52:55 +03:00
Evgenii Stratonikov
d57c57010f [#1726] neofs-cli: Truncate file before writing the object 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-08-23 18:07:12 +03:00
Evgenii Stratonikov
40f1cf36e1 [#1719] neofs-cli: Save object with proper permissions 
`curl` uses 0644, for example.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-08-23 18:07:12 +03:00
Evgenii Stratonikov
871fae8321 [#1705] go.mod: Update neo-go to v0.99.2 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-08-23 11:59:23 +03:00
Evgenii Stratonikov
e4370c4129 [#1702] neofs-adm: Set HighPriority attribute for committee transactions 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-08-23 11:59:23 +03:00
Pavel Karpy
0720d96c9d [#1687] go.mod: Update neofs-sdk-go 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-08-22 18:59:57 +03:00
Pavel Karpy
d40ed00438 [#1687] go.mod: Update neofs-api-go 
It contains the signature fix for go `v1.19`.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-08-22 18:59:57 +03:00
Pavel Karpy
56792d5eaf [#1687] .github: Update actions to the latest version 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-08-22 18:59:57 +03:00
Pavel Karpy
99903e1aba [#1687] Support v1.19 Golang 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-08-22 18:59:57 +03:00
Pavel Karpy
30341f2192 [#1687] *: Perform go fmt using go v1.19 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-08-22 18:59:57 +03:00
Pavel Karpy
37ab26bfa9 [#1296] cli: "ID" -> "OID" 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-08-22 14:29:50 +04:00
Evgenii Stratonikov
e8c6dce466 [#1523] blobstor: Rename compression.CConfig to Config 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-08-22 13:14:19 +03:00
Evgenii Stratonikov
0042c1d4a6 [#1584] blobovniczatree: move public methods to different files 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-08-22 13:14:19 +03:00
Evgenii Stratonikov
26b4a258e0 [#1523] neofs-node: Refactor configuration 
1. Move compression parameters to the `shard` section.
2. Allow to use multiple sub-storage components in the blobstor.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-08-22 13:14:19 +03:00
Evgenii Stratonikov
13cdbde2e2 [#1523] blobstor: Remove unused types 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-08-22 13:14:19 +03:00
Evgenii Stratonikov
0c9e4e6a35 [#1523] blobstor: Unify request dispatch logic 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-08-22 13:14:19 +03:00
Evgenii Stratonikov
266458fe5c [#1523] blobstor: Unify fstree and blobovnicza interfaces 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-08-22 13:14:19 +03:00
Evgenii Stratonikov
5aa3defc67 [#1523] writecache: Remove cloneBytes 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-08-22 13:14:19 +03:00
Evgenii Stratonikov
ddaed283e9 [#1523] writecache: Simplify logic 
1. Remove in-memory cache. It doesn't persist objects and if we want
   more speed, `NoSync` option can be used for the bolt DB.
2. Put to the metabase in a synchronous fashion. This considerably
   simplifies overall logic and plays nicely with the metabase bolt DB
   batch settings.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-08-22 13:14:19 +03:00