Commit graph

1173 commits

Author SHA1 Message Date
68029d756e [#1302] writecache: Allow to specify custom page size
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-08-09 12:12:07 +03:00
fa82854af4 [#1302] writecache: Add put->flush->put benchmark
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-08-09 12:12:07 +03:00
08b1f18bca [#1296] writecache: Add count limit
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-08-09 06:30:32 +00:00
36efccd862 [#1298] writecache: Add shrink flag for Seal command
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-08-08 16:32:29 +03:00
5c01bd5be8 [#1298] writecache: Add restore-mode flag for Seal command
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-08-07 11:20:14 +03:00
8e51d7849a [#1295] getSvc: Assemble complex EC object headers without linking object
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-08-06 16:48:12 +03:00
10602b55b1 [#1295] engine: Resolve funlen linter
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-08-06 09:45:47 +03:00
eeca796d2e [#1295] engine: Log object address in case of error
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-08-06 09:45:43 +03:00
18182e578e [#1261] shard: Fix delete objects from FSTree
Replace nil storageID with empty like by shard.Get.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-22 14:38:09 +03:00
3119f2fbc3 [#1257] metabase: Delete EC gc marks and split info
EC parent and split gc marks should be deleted after last EC chunk delete.
Also should delete split info for EC parent and split parent.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-19 17:14:14 +03:00
27caa8a72f [#1256] metabase: Put split parent object ID for EC chunks
It is required to save split parent ID too, not only split ID.
Otherwise inhume operation works incorrect: shard with last part may be skipped
and parent object will be available.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-17 18:08:07 +03:00
3940bc17c1 [#1251] pilorama: Allow traversing multiple branches in parallel
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-17 11:25:07 +03:00
3bf6e6dde6 [#1238] engine: Add non-blocking send in the shard's notification channel
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-07-10 11:37:11 +03:00
b027a7f91e [#1234] pilorama: Fix GetByPath() on duplicate directories
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-10 06:11:38 +00:00
6ace2f597e [#1234] pilorama: Add test for duplicate directory behaviour
When AddByPath() is called concurrently on 2 different nodes,
internal path components may be created twice. This violates some
of our assumptions in GetByPath() and, indirectly, in S3 handling of
GetSubTree() results.

Add a test for the correct behaviour, fixes will follow.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-10 06:11:38 +00:00
62cbb72a5e [#1226] blobovniczatree: Delete fix db extensions in Init()
Since several releases have been released, this code is not relevant.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-04 12:22:06 +03:00
78b1d9b18d [#1226] blobovniczatree: Drop leaf width limitation
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-04 12:22:06 +03:00
40c9ddb6ba [#1226] blobovniczatree: Drop init in advance option
To make blobovniczatree unlimited.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-04 12:22:06 +03:00
3a797e4682 [#1222] engine: Fix tree evacuation
Do not fail if it is unable to evacuate tree to other node.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-04 10:38:10 +03:00
2bac82cd6f [#1222] engine: Fix object evacuation
Do not fail evacuation if it unable to evacuate object to other node.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-04 10:38:10 +03:00
bbe95dac8b [#1225] engine: Log the error when check object existence
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-07-04 07:22:54 +00:00
56eeb630b6 [#1217] Fix grammar mistakes and misspelling
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-07-01 19:14:25 +03:00
7a8ac4907a [#1213] engine: Drop unused
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-01 06:49:35 +00:00
7085723c6b [#1074] pilorama: Allow empty filenames in SortedByFilename()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-06-28 17:46:24 +03:00
4c7ff159ec [#1201] writecache: Cancel background flush without lock
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-06-28 09:02:36 +03:00
4951babd5f [#1208] blobstor: Fix delete without storage id
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-06-27 11:59:08 +03:00
81ea91de52 [#451] metrics: Move to internal
`metrics` don't look like something others want to import.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-06-25 08:52:37 +00:00
9ac74efc41 [#1173] shard: Use mode from config on reload
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-06-20 11:29:10 +00:00
75eedf71f3 [#1187] pilorama/test: Remove debug print
Introduced in e12fcc041d.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-06-18 15:09:44 +03:00
76cf7a051b [#1178] shard: Check metabase existence before read shard ID
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-06-17 09:59:15 +03:00
68ac490729 [#1174] shard: Update metric mode_info on Init
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-06-13 08:32:59 +00:00
6a39c3d15e [#1086] engine: Do not use metabase if shard looks bad
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-06-13 07:35:22 +00:00
9d73f9c2c6 Reapply "[#446] engine: Move to read-only on blobstor errors"
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-06-13 07:35:22 +00:00
b9fcaad21f [#1168] shard: Set Disabled as default mode for components
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-06-11 15:13:38 +00:00
6cf512e574 [#1166] blobovniczatree: Handle blobovnicza's NoSpaceLeft error
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-06-07 17:15:43 +03:00
e7d479f4c2 [#1166] blobovnicza: Return NoSpaceLeft error instead of syscall.ENOSPC
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-06-07 17:15:43 +03:00
3f1961157e [#1163] metabase: Handle multiple splitInfos for EC
For REP updating split info is handled explicitly by a high-level PUT logic.
For EC it is trickier, because the address of an object we put is only
distantly related to a split info.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-06-06 16:26:29 +03:00
2e074d3846 [#1163] metabase: Properly save EC parent split ID
Search by SplitID should return all parts of a complex object.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-06-05 12:40:16 +03:00
806236da78 [#1121] node: Change mode of shard components
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2024-06-05 05:55:24 +00:00
6f2187a420 [#1121] node: Refactor mods of shard
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2024-06-05 05:55:24 +00:00
cc2449beaf [#1158] metabase: Fix EC storage schema
Do not store EC info twice.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-06-04 17:24:40 +03:00
5aacb8fc86 [#1144] metabase: Save parent attributes for ec-chunks
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-31 19:55:32 +03:00
f8e33f8e3a [#1144] metabase: Proprely choose root OID for EC-splitted objects
* If EC-parent is a part of Split itself, then save to root bucket
  its parent;
* If EC-parent is not a part of Split itself, then save to root bucket
  OID of this EC-parent.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-31 19:53:32 +03:00
f0edebea18 [#1144] metabase: Support ec parent filter for Search
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-31 19:53:32 +03:00
92e19feb57 [#1147] node: Use public fields for shard.ExistsPrm
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-05-30 08:13:04 +00:00
6130650bb6 [#1147] node: Implement Lock\Delete requests for EC object
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-05-30 08:13:04 +00:00
a82c8cc5b8 [#1147] gc: Execute callback for expired tombstones when they exists
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-05-30 08:13:04 +00:00
40b04c00ef [#1141] metabase: Fix IsUserObject method
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-20 13:24:15 +03:00
89a80e9a0f [#1141] metabase: Fix putUniqueIndexItem
* `GetECHeader` is not correct way to determine if an object's got
  EC-header: `ECHeader` must be used for that.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-20 13:24:08 +03:00
d45d086acd [#1129] policer: Add EC chunk replication
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-05-16 16:28:48 +03:00
5c582e96fd [#1136] metabase: Fix creation of ECInfoError
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-05-15 11:04:27 +00:00
6e71ae3bda [#1130] fstree: Remove useless Stat() call
```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/frostfs-node/pkg/local_object_storage/blobstor
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
                                           │     old     │                new                 │
                                           │   sec/op    │   sec/op     vs base               │
SubstorageReadPerf/fstree_nosync-seq100-8    2.689µ ± 2%   2.428µ ± 4%  -9.72% (p=0.000 n=10)
SubstorageReadPerf/fstree_nosync-rand100-8   2.727µ ± 1%   2.497µ ± 2%  -8.42% (p=0.000 n=10)
geomean                                      2.708µ        2.462µ       -9.07%
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-05-14 16:05:45 +03:00
00b2b77b26 [#1112] node: Implement Range\RangeHash requests for EC object
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-05-07 14:47:21 +03:00
e07869a8cf [#1100] Remove unused fields
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-05-06 10:14:36 +03:00
c9efaa5819 [#966] node: Add path of the write_cache to metric labels
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2024-05-02 06:46:46 +00:00
4730ecfdb8 [#966] node: Refactor WriteCacheMetrics interface
Grouping common fields of methods will enhance the readability of the interface.

Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2024-05-02 06:46:46 +00:00
411a8d0245 [#1004] blobovnicza: Use TTL for blobovnicza tree cache
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-04-26 19:54:29 +03:00
112a7c690f [#1103] node: Implement Get\Head requests for EC object
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-04-24 18:15:53 +03:00
167c52a1a9 [#1103] node: Reduce amount of lines for method StorageEngine.head
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-04-24 16:31:04 +03:00
e5e0542482 [#1085] log: Move storage log message to constants package
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-15 07:57:00 +00:00
5be36924e3 [#41] log: Log storage operations in only in Debug
They are mostly useless unless we need to _debug_ a specific issue.
The amount of logs we produce is too big.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-15 07:57:00 +00:00
40781b3a20 [#1086] engine: Change mode in case of errors async
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-04-10 12:29:43 +00:00
5ef5734c4e Reapply "[#972] Drop x/exp/slices dependency"
This reverts commit 946f2ec2bf.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-10 12:09:34 +00:00
2c4b50a71e Reapply "[#972] Use require.ElementsMatch() where possible"
This reverts commit 7627d08914.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-10 12:09:34 +00:00
3dc81cb4fc Reapply "[#972] Use min/max builtins"
This reverts commit dad56d2e98.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-10 12:09:34 +00:00
d864945961 Reapply "[#972] pilorama: Remove removeDuplicatesInPlace()"
This reverts commit 9f68305c2e.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-10 12:09:34 +00:00
1b17258c04 [#1029] metabase: Add refill metrics
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-04-10 13:05:44 +03:00
e3d9dd6ee8 [#1024] blobovnicza: Copy data on iterate
DB value is only valid while the tx is alive.
But handler may to run something in other goroutine.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-04-10 10:21:11 +03:00
57466594fb [#1024] shard: Resync metabase concurrently
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-04-10 10:21:10 +03:00
1005bf4f56 [#1024] shard: Add refill metabase benchmark
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-04-10 10:21:10 +03:00
76398c06b0 [#1080] metabase: Add StorageID metric
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-04-10 10:00:08 +03:00
7b1adfed3e [#1080] metabase: Open bucket for container counter once
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-04-10 10:00:08 +03:00
5b8200de88 [#984] blobovnicza: Do not fail rebuild on big objects
If blobovnicza contains objects larger than object size parameter
value, then rebuild fails with an error, because there is no such
bucket in database. This commit forces to create bucket on rebuild.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-04-09 11:51:18 +00:00
d614f04a0a [#1072] Fix gofumpt issues
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-04-03 22:21:14 +03:00
17af91619a [#1070] pilorama: Fix cycling behaviour for sorted listing
In case there are no items left, return empty slice.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-02 14:41:31 +00:00
d5194ab2a6 [#949] metabase: fix shard.UpdateID()
metabase.Open() now reports metabase mode metric. shard.UpdateID()
needs to read shard ID from metabase => needs to open metabase.
It caused reporting 'shard undefined' metrics. To avoid reporting
wrong metrics metabase.GetShardID() was added which also opens
metabase and does not report metrics.

Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-04-01 17:27:34 +03:00
81a0346a96 [#949] metabase: fix metabase mode metric
It used to always show CLOSED regardless of actual mode.
Now metric represents actual metabase mode of operations.

Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-04-01 17:27:34 +03:00
e12fcc041d [#1059] services/tree: Fast sorted listing
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-01 12:37:34 +00:00
f23e38c285 Revert "[#446] engine: Move to read-only on blobstor errors"
This reverts commit 69df0d21c2.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-01 12:48:30 +03:00
942d83611b [#874] engine: Revert Check object existance concurrently
This reverts commit f526f49995.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-04-01 08:42:34 +00:00
0990a9b0bd [#1055] blobstor: fix mode metric
It used to always show CLOSED after setting shard mode
to read-only regardless of actual mode.
Now metric represents actual blobstor mode of operations.

Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-03-29 20:44:47 +00:00
c09c701613 [#1048] metabase: Fix drop buckets during resync
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-03-19 14:28:31 +03:00
31e2396a5f [#1043] control: Add ResetEvacuationStatus implementation
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-03-13 10:29:45 +00:00
3195142d67 [#959] writecache: Avoid manipulation with cache in DEGRADED mode
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-03-11 18:35:41 +00:00
d433b49265 [#973] node: Resolve perfsprint linter
`fmt.Errorf can be replaced with errors.New` and `fmt.Sprintf can be replaced with string addition`

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-03-11 17:55:50 +03:00
66a26b7775 [#973] node: Resolve revive: unused-parameter linter
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-03-11 17:11:49 +03:00
0882840bf5 [#634] shard: Add writecache inhume tests
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-03-06 13:12:34 +03:00
ae5bb87e70 Revert "[#866] Use TTL for blobovnicza tree cache"
This reverts commit d9cbb16bd3.

Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-03-01 19:29:33 +03:00
d6534fd755 [#1016] frostfs-node: Fix gopls issues
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-03-01 12:13:43 +03:00
918613546f [#1008] metabase: Do not update storageID on put
There may be a race condition between put an object and
flushing the writecache:
1. Put object to the writecache
2. Writecache flushes object to the blobstore and sets blobstore's
storageID
3. Put object to the metabase, set writecache's storageID

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-28 11:01:50 +03:00
2ad433dbcb [#1005] engine: Drop shards weights
Unused.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-26 17:25:05 +03:00
7470c383dd [#997] metabase: Drop toMoveIt bucket
It is not used.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-21 10:06:05 +03:00
adf7ebab5b [#996] metabase: Speed up bucket creation
Most of the time it exits, e.g. when it is per-container and use on each
object PUT. Bbolt implementation first tries to create bucket and then
returns it if it exists. Create operation uses cursor and thus is not
very lightweight, we can avoid it.

```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/frostfs-node/pkg/local_object_storage/metabase
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
                 │     old     │                new                 │
                 │   sec/op    │   sec/op     vs base               │
Put/parallel-8     174.4µ ± 3%   163.3µ ± 3%  -6.39% (p=0.000 n=10)
Put/sequential-8   263.3µ ± 2%   259.0µ ± 1%  -1.64% (p=0.000 n=10)
geomean            214.3µ        205.6µ       -4.05%

                 │     old      │                 new                 │
                 │     B/op     │     B/op      vs base               │
Put/parallel-8     275.3Ki ± 3%   281.1Ki ± 4%       ~ (p=0.063 n=10)
Put/sequential-8   413.0Ki ± 2%   426.6Ki ± 2%  +3.29% (p=0.003 n=10)
geomean            337.2Ki        346.3Ki       +2.70%

                 │     old     │                 new                 │
                 │  allocs/op  │  allocs/op   vs base                │
Put/parallel-8      678.0 ± 1%    524.5 ± 2%  -22.64% (p=0.000 n=10)
Put/sequential-8   1.329k ± 0%   1.183k ± 0%  -10.91% (p=0.000 n=10)
geomean             949.1         787.9       -16.98%
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-20 15:42:58 +00:00
9f68305c2e Revert "[#972] pilorama: Remove removeDuplicatesInPlace()"
This reverts commit 45fd4e4ff1.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-19 15:36:01 +00:00
dad56d2e98 Revert "[#972] Use min/max builtins"
This reverts commit 89784b2e0a.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-19 15:36:01 +00:00
7627d08914 Revert "[#972] Use require.ElementsMatch() where possible"
This reverts commit 6d9707ff1f.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-19 15:36:01 +00:00
946f2ec2bf Revert "[#972] Drop x/exp/slices dependency"
This reverts commit f3e50772fd.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-19 15:36:01 +00:00
f3e50772fd [#972] Drop x/exp/slices dependency
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-19 13:13:09 +00:00
6d9707ff1f [#972] Use require.ElementsMatch() where possible
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-19 13:13:09 +00:00
89784b2e0a [#972] Use min/max builtins
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-19 13:13:09 +00:00