Commit Graph

2983 Commits (7cc368e188dc03e761cad0801e7f269a81b1ed83)

Author SHA1 Message Date
Dmitrii Stepanov 189dbb01be [#733] frostfs-cli: Add `control ir remove-container`
DCO action / DCO (pull_request) Successful in 3m51s Details
Vulncheck / Vulncheck (pull_request) Successful in 3m49s Details
Build / Build Components (1.21) (pull_request) Successful in 4m34s Details
Build / Build Components (1.20) (pull_request) Successful in 6m21s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 9m1s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 9m12s Details
Tests and linters / Staticcheck (pull_request) Successful in 6m40s Details
Tests and linters / Tests with -race (pull_request) Successful in 9m21s Details
Tests and linters / Lint (pull_request) Successful in 9m39s Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-10-19 16:22:18 +03:00
Dmitrii Stepanov f2437f7ae9 [#734] shard: Fix Delete method
DCO action / DCO (pull_request) Successful in 1m36s Details
Build / Build Components (1.20) (pull_request) Successful in 3m46s Details
Vulncheck / Vulncheck (pull_request) Successful in 3m16s Details
Tests and linters / Staticcheck (pull_request) Successful in 4m42s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 6m27s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 6m35s Details
Tests and linters / Tests with -race (pull_request) Successful in 6m33s Details
Build / Build Components (1.21) (pull_request) Successful in 13m7s Details
Tests and linters / Lint (pull_request) Successful in 19m14s Details
Due to the flushing data from the writecache to the storage
and simultaneous deletion, a partial deletion situation is possible.
So as a solution, deletion is allowed only when the object is in storage,
because object will be deleted from writecache by flush goroutine.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-10-16 17:00:18 +03:00
Dmitrii Stepanov f26233b47a [#734] metabase: Include UpdateStorageID in metrics and traces
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-10-16 17:00:18 +03:00
Dmitrii Stepanov 7e0c5a55de [#734] writecache: Fix flush
Now UpdateStorageID doesn't return error in case of logical error.
If object is in graveyard or GC market, it is still required to
update storage ID.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-10-16 17:00:17 +03:00
Anton Nikiforov 3a997d1207 [#680] metrics: Initialize log metrics together with services
DCO action / DCO (pull_request) Successful in 3m31s Details
Vulncheck / Vulncheck (pull_request) Successful in 3m17s Details
Build / Build Components (1.21) (pull_request) Successful in 3m56s Details
Tests and linters / Staticcheck (pull_request) Successful in 5m13s Details
Tests and linters / Lint (pull_request) Successful in 5m50s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 7m31s Details
Build / Build Components (1.20) (pull_request) Successful in 7m44s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 7m32s Details
Tests and linters / Tests with -race (pull_request) Successful in 7m33s Details
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-10-11 17:08:03 +03:00
Anton Nikiforov 994f48f8bb [#680] metrics: Export log and morph with script
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-10-11 17:08:03 +03:00
Evgenii Stratonikov aca11d7474 [#735] policer: Allow to provide metrics from the outside
DCO action / DCO (pull_request) Successful in 3m9s Details
Vulncheck / Vulncheck (pull_request) Successful in 3m28s Details
Build / Build Components (1.21) (pull_request) Successful in 4m17s Details
Build / Build Components (1.20) (pull_request) Successful in 4m35s Details
Tests and linters / Lint (pull_request) Successful in 7m19s Details
Tests and linters / Tests with -race (pull_request) Failing after 9m47s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 8m0s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 11m19s Details
Tests and linters / Staticcheck (pull_request) Successful in 11m4s Details
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-10-11 15:14:13 +03:00
Ekaterina Lebedeva 5e229dc248 [#701] metrics: add metric to evaluate policer performance
Add processed objects counter in policerMetrics,
add policer field to NodeMetrics

Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2023-10-09 19:02:08 +00:00
Evgenii Stratonikov 4caa934eea [#729] containersvc: Remove load announcement
IR code was removed in 8879c6ea.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-10-09 19:01:13 +00:00
Dmitrii Stepanov d07afd803c [#726] writecache: Fix small object flush for Badger
DCO action / DCO (pull_request) Successful in 2m25s Details
Vulncheck / Vulncheck (pull_request) Successful in 2m41s Details
Build / Build Components (1.21) (pull_request) Successful in 3m2s Details
Build / Build Components (1.20) (pull_request) Successful in 3m17s Details
Tests and linters / Staticcheck (pull_request) Successful in 4m26s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 6m9s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 18m15s Details
Tests and linters / Lint (pull_request) Successful in 20m21s Details
Tests and linters / Tests with -race (pull_request) Successful in 18m2s Details
Do not marshal object twice.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-10-06 11:32:50 +03:00
Dmitrii Stepanov 997ac7cd8d [#726] writecache: Fix small object flush for BBolt
Do not marshal object twice.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-10-06 11:32:44 +03:00
Alexander Chuprov c1e4130020 [#146] node: Add trace_id to logs
Vulncheck / Vulncheck (pull_request) Successful in 3m7s Details
DCO action / DCO (pull_request) Successful in 3m36s Details
Build / Build Components (1.21) (pull_request) Successful in 3m29s Details
Build / Build Components (1.20) (pull_request) Successful in 3m37s Details
Tests and linters / Staticcheck (pull_request) Successful in 4m39s Details
Tests and linters / Lint (pull_request) Successful in 5m2s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 6m27s Details
Tests and linters / Tests with -race (pull_request) Successful in 6m29s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 9m19s Details
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2023-09-27 11:05:27 +03:00
Airat Arifullin 3889e829e6 [#667] writecache: Add logs for report error func in tests
Signed-off-by: Airat Arifullin a.arifullin@yadro.com
2023-09-14 17:02:54 +00:00
Evgenii Stratonikov c6af4a3ec8 [#679] engine: Do not increase error counter on meta mismatch
DCO action / DCO (pull_request) Successful in 1m43s Details
Vulncheck / Vulncheck (pull_request) Successful in 2m0s Details
Tests and linters / Staticcheck (pull_request) Successful in 2m54s Details
Build / Build Components (1.20) (pull_request) Successful in 2m49s Details
Build / Build Components (1.21) (pull_request) Successful in 2m44s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 3m43s Details
Tests and linters / Lint (pull_request) Successful in 3m58s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 23m28s Details
Tests and linters / Tests with -race (pull_request) Successful in 29m0s Details
It was introduced in 69e1e6ca to help node determine faulty shards.
However, the situation is possible in a real-life scenario:
1. Object O is evacuated from shard A to B.
2. Shard A is unmounted because of lower-level errors.
3. We now have object in meta on A and in blobstor on B. Technically we
   have it in meta on shard B too, but we still got the error if B goes
   to a degraded mode.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-14 10:39:18 +03:00
Ekaterina Lebedeva 8a81af5a3b [#653] Add context parameter to Open functions
DCO action / DCO (pull_request) Successful in 1m38s Details
Build / Build Components (1.20) (pull_request) Successful in 4m22s Details
Build / Build Components (1.21) (pull_request) Successful in 4m25s Details
Vulncheck / Vulncheck (pull_request) Successful in 4m56s Details
Tests and linters / Lint (pull_request) Successful in 6m1s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 7m43s Details
Tests and linters / Staticcheck (pull_request) Successful in 8m1s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 8m14s Details
Tests and linters / Tests with -race (pull_request) Successful in 8m32s Details
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2023-09-07 18:03:29 +03:00
Evgenii Stratonikov a716db99db [#668] shard/test: Do not alter rootPath option
Supposedly, this was added to allow creating 2 different shards without
subtest. Now we use t.TempDir() everywhere, so this should not be a
problem.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-07 07:39:39 +00:00
Evgenii Stratonikov 36759f8434 [#668] shard/test: Properly check event processing
See https://git.frostfs.info/TrueCloudLab/frostfs-node/actions/runs/1594/jobs/2

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-07 07:39:39 +00:00
Evgenii Stratonikov 39879fa868 [#668] shard/test: Add dontRelease options
Most of the time we would like to close shard with minor exceptions.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-07 07:39:39 +00:00
Evgenii Stratonikov c661ba1312 [#668] shard/test: Use sane defaults in the test constructor
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-07 07:39:39 +00:00
Evgenii Stratonikov 268adb79cb [#668] shard/test: Simplify shard construction
newCustomShard() has many parameters but only the first is obligatory.
`enableWriteCache` is left as-is, because it directly affects the
functionality.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-07 07:39:39 +00:00
Evgenii Stratonikov 429f941cda [#668] shard/test: Release shard in t.Cleanup()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-07 07:39:39 +00:00
Evgenii Stratonikov 382eb8a485 [#668] shard/test: Disable GC where it is not needed
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-07 07:39:39 +00:00
Evgenii Stratonikov 42696016de [#668] shard: Close stopChannel in GC
It is done once, but now we could read it from multiple places.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-07 07:39:39 +00:00
Evgenii Stratonikov bdecfbc1be [#668] shard/test: Move tests to the main package
Semantic patch (also, duplicate definitions are removed):
```
@@
var e identifier
@@
-import "git.frostfs.info/TrueCloudLab/frostfs-node/pkg/local_object_storage/shard"

-shard.e
+e
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-07 07:39:39 +00:00
Evgenii Stratonikov aa23c6a83a [#668] shard/test: Remove subtest from TestCounters
Otherwise, individual tests cannot be run.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-07 07:39:39 +00:00
Evgenii Stratonikov da8f384324 [#668] shard/test: Fix typo in `existence`
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-07 07:39:39 +00:00
Anton Nikiforov aeeb8193d2 [#676] node: Fix header source creation when checking eacl
DCO action / DCO (pull_request) Successful in 2m55s Details
Build / Build Components (1.20) (pull_request) Successful in 4m53s Details
Vulncheck / Vulncheck (pull_request) Successful in 4m36s Details
Tests and linters / Staticcheck (pull_request) Successful in 6m35s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 7m7s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 7m47s Details
Tests and linters / Tests with -race (pull_request) Failing after 10m7s Details
Build / Build Components (1.21) (pull_request) Successful in 11m3s Details
Tests and linters / Lint (pull_request) Successful in 17m34s Details
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-09-06 17:06:54 +03:00
Anton Nikiforov 88d50e4c77 [#656] policer: Add "bad" testcase
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-09-06 08:04:59 +00:00
Evgenii Stratonikov a54b4472de [#674] network: Close connections on address updates
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-04 15:34:24 +03:00
Dmitrii Stepanov 7456c8556a [#536] blobovnicza: Add blobovniczatree DB cache
DCO action / DCO (pull_request) Successful in 1m33s Details
Vulncheck / Vulncheck (pull_request) Successful in 2m43s Details
Build / Build Components (1.20) (pull_request) Successful in 3m40s Details
Tests and linters / Staticcheck (pull_request) Successful in 3m59s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 5m44s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 6m3s Details
Tests and linters / Tests with -race (pull_request) Successful in 6m28s Details
Build / Build Components (1.21) (pull_request) Successful in 12m34s Details
Tests and linters / Lint (pull_request) Successful in 12m51s Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-09-01 13:53:11 +03:00
Dmitrii Stepanov c672f59ab8 [#536] blobovnicza: Drop cache
Each blobovnicza instance is opened
while is in use.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-09-01 13:51:26 +03:00
Evgenii Stratonikov b9b86d2ec8 [#666] shard/test: Fix data race in metrics tests
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-31 08:39:42 +00:00
Anton Nikiforov 4dff9555f1 [#568] writecache: Improve flushing scheme for badger
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-08-30 17:22:28 +00:00
Airat Arifullin 806cc13d9f [#658] client: Refactor PrmObjectGet/Head/Range usage
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-08-30 17:13:23 +00:00
Alejandro Lopez 1daef2ceeb [#660] writecache: Fix remaining addr2key uses
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-08-30 17:12:33 +00:00
Anton Nikiforov fe5aa06a75 [#665] node: Bind length of copies number to number of replicas
Allow to use one digit in copies number array for backward compatibility.

Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-08-30 17:11:55 +00:00
Dmitrii Stepanov 55b82e744b [#529] objectcore: Use common sender classifier
DCO action / DCO (pull_request) Successful in 2m19s Details
Vulncheck / Vulncheck (pull_request) Successful in 3m5s Details
Build / Build Components (1.21) (pull_request) Successful in 4m8s Details
Build / Build Components (1.20) (pull_request) Successful in 4m24s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 4m57s Details
Tests and linters / Staticcheck (pull_request) Successful in 4m43s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 5m2s Details
Tests and linters / Lint (pull_request) Successful in 5m21s Details
Tests and linters / Tests with -race (pull_request) Successful in 6m17s Details
Use common sender classifier for ACL service and format validator.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-08-29 10:33:06 +03:00
Dmitrii Stepanov ae81d6660a [#529] objectcore: Fix object content validation
There are old objects where the owner of the object
may not match the one who issued the token.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-08-29 10:33:06 +03:00
Dmitrii Stepanov ab2614ec2d [#528] objectcore: Validate token issuer
Add token issuer against object owner validation.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-08-29 10:10:10 +03:00
Airat Arifullin 4ea0df77d0 [#574] policer: Check if the container was really removed
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-08-28 14:21:38 +00:00
Airat Arifullin 554ff2c06b [#574] core: Extend Source interface with DeletionInfo method
* Introduce common method EverExisted
* Define DeletionInfo for struct that must implement Source
* Refactor tree srv

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-08-28 14:21:38 +00:00
Evgenii Stratonikov 9072772a09 [#649] shard/test: Increase GC remover interval
This was set in #348 to speed up tests.
It seems 100ms doesn't increase overall test time,
but it reduces the amount of logs by 100x factor.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-25 10:10:25 +00:00
Evgenii Stratonikov c4db8e7690 [#637] shard/test: Fix data race
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-25 10:10:25 +00:00
Evgenii Stratonikov f8ba60aa0c [#648] objsvc/delete: Handle errors in Go style
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-25 09:45:35 +00:00
Evgenii Stratonikov d2084ece41 [#648] objsvc/delete: Remove redundant logs
We never propagate delete requests to the container node, because
tombstone broadcast is done via PUT. No need to pollute logs.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-25 09:45:35 +00:00
Evgenii Stratonikov 40b556fc19 [#647] objsvc/search: Improve testing coverage
DCO action / DCO (pull_request) Successful in 1m32s Details
Build / Build Components (1.20) (pull_request) Successful in 3m54s Details
Tests and linters / Staticcheck (pull_request) Successful in 3m44s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 4m18s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 4m35s Details
Tests and linters / Lint (pull_request) Successful in 4m42s Details
Vulncheck / Vulncheck (pull_request) Successful in 5m11s Details
Tests and linters / Tests with -race (pull_request) Successful in 6m23s Details
Build / Build Components (1.21) (pull_request) Successful in 8m38s Details
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-25 10:40:01 +03:00
Evgenii Stratonikov 4db2cbc927 [#647] objsvc/search: Wrap in uniqueIDWriter during parameter setting
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-25 10:40:01 +03:00
Evgenii Stratonikov 966ad22abf [#647] objsvc/search: Simplify error handling
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-25 10:40:01 +03:00
Evgenii Stratonikov 56f841b022 [#647] objsvc/search: Remove TraverserGenerator wrapper
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-25 10:40:01 +03:00
Evgenii Stratonikov ba58144de1 [#647] objsvc/search: Remove netmap.Source wrapper
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-25 10:40:01 +03:00
Evgenii Stratonikov c9e3c9956e [#643] objsvc/put: Unify extraBroadcastEnabled usage
DCO action / DCO (pull_request) Successful in 1m26s Details
Vulncheck / Vulncheck (pull_request) Successful in 2m44s Details
Build / Build Components (1.20) (pull_request) Successful in 3m26s Details
Tests and linters / Staticcheck (pull_request) Successful in 3m51s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 5m4s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 5m46s Details
Tests and linters / Tests with -race (pull_request) Successful in 6m23s Details
Build / Build Components (1.21) (pull_request) Successful in 13m1s Details
Tests and linters / Lint (pull_request) Successful in 19m46s Details
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-24 11:03:17 +03:00
Evgenii Stratonikov facd3b2c4b [#643] objsvc/put: Unify placement iterators
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-24 11:03:17 +03:00
Evgenii Stratonikov 3fcf56f2fb [#643] objsvc/put: Copy config to `distributedTarget`
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-24 11:03:17 +03:00
Evgenii Stratonikov 96e690883f [#638] Unify test loggers
In some places we have debug=false, in others debug=true.
Let's be consistent.

Semantic patch:
```
@@
@@
-test.NewLogger(..., false)
+test.NewLogger(..., true)
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-23 11:21:05 +00:00
Evgenii Stratonikov 322c1dc273 [#638] Use test.NewLogger() in tests
Semantic patch (restricted to **/*_test.go):
```
@@
@@
+import "git.frostfs.info/TrueCloudLab/frostfs-node/pkg/util/logger/test"
-import "go.uber.org/zap"
-import "git.frostfs.info/TrueCloudLab/frostfs-node/pkg/util/logger"

-&logger.Logger{Logger: zap.L()}
+test.NewLogger(t, false)
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-23 11:21:05 +00:00
Evgenii Stratonikov 02b03d9c4f [#638] logger: Remove sampling from test loggers
Losing logs is always a bad idea, especially when we debug tests.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-23 11:21:05 +00:00
Evgenii Stratonikov 82cc453be9 [#xx] shard: Fix data race in metrics tests
Protect test metric store fields with a mutex. Probably, not every field
should be protected, but better safe than sorry.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-23 10:26:12 +00:00
Alejandro Lopez 345a1a69a2 [#635] Use internal key type when deleting from badger wc
DCO action / DCO (pull_request) Successful in 2m48s Details
Build / Build Components (1.21) (pull_request) Successful in 4m59s Details
Build / Build Components (1.20) (pull_request) Successful in 5m5s Details
Vulncheck / Vulncheck (pull_request) Successful in 4m55s Details
Tests and linters / Staticcheck (pull_request) Successful in 6m51s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 7m21s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 7m53s Details
Tests and linters / Lint (pull_request) Successful in 8m35s Details
Tests and linters / Tests with -race (pull_request) Successful in 9m57s Details
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-08-22 10:53:19 +03:00
Airat Arifullin 23be3eb627 [#574] tree: Check if container is really removed
Vulncheck / Vulncheck (pull_request) Successful in 2m37s Details
Build / Build Components (1.21) (pull_request) Successful in 3m36s Details
Tests and linters / Staticcheck (pull_request) Successful in 3m36s Details
DCO action / DCO (pull_request) Successful in 4m16s Details
Tests and linters / Lint (pull_request) Successful in 5m34s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 5m36s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 5m46s Details
Tests and linters / Tests with -race (pull_request) Successful in 6m21s Details
Build / Build Components (1.20) (pull_request) Successful in 8m1s Details
* Use DeletionInfo method from morph client to check if
  the container has been really removed from neo-go

Signed-off-by: Airat Arifullin a.arifullin@yadro.com
2023-08-21 12:50:20 +03:00
Airat Arifullin 42fb6fb372 [#574] morph: Add DeletionInfo method for morph client
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-08-21 12:49:06 +03:00
Dmitrii Stepanov 354a92ea2c [#602] blobovnicza: Add leaf width implementation
Vulncheck / Vulncheck (pull_request) Successful in 1m14s Details
Build / Build Components (1.21) (pull_request) Successful in 3m36s Details
DCO action / DCO (pull_request) Successful in 4m4s Details
Tests and linters / Staticcheck (pull_request) Successful in 4m5s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 4m49s Details
Tests and linters / Lint (pull_request) Successful in 5m18s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 5m44s Details
Tests and linters / Tests with -race (pull_request) Successful in 5m47s Details
Build / Build Components (1.20) (pull_request) Successful in 10m45s Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-08-21 10:27:32 +03:00
Alexander Chuprov 4d9a6c07fb [#618] core: Replace fmt.Sprintf with strconv.FormatUint
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2023-08-21 07:14:50 +00:00
Dmitrii Stepanov f2811f8585 [#602] metrics: Add blobovnicza items counter
Vulncheck / Vulncheck (pull_request) Successful in 1m29s Details
Build / Build Components (1.21) (pull_request) Successful in 3m23s Details
DCO action / DCO (pull_request) Successful in 3m50s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 4m27s Details
Tests and linters / Lint (pull_request) Successful in 4m48s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 5m6s Details
Tests and linters / Staticcheck (pull_request) Successful in 5m8s Details
Tests and linters / Tests with -race (pull_request) Successful in 5m38s Details
Build / Build Components (1.20) (pull_request) Successful in 7m46s Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-08-18 13:01:27 +03:00
Dmitrii Stepanov c4e1d8eb07 [#602] node: Fix blobovnicza typos
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-08-18 11:14:10 +03:00
Dmitrii Stepanov 10e63537b2 [#602] metrics: Rename blobovnicza size metric
`Size` is not size, but open db size.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-08-17 19:17:35 +00:00
Dmitrii Stepanov 809e97626b [#602] blobovnicza: Fix size counter
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-08-17 19:17:35 +00:00
Dmitrii Stepanov 2e49d7ea7e [#602] blobovnicza: Init before using
Fix blobovnicza size: after restart size metric resets.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-08-17 19:17:35 +00:00
Evgenii Stratonikov f7042c5a6f [#609] Replace zaptest.NewLogger() with zap.L()
Semantic patch:
```
@@
@@
-import "go.uber.org/zap/zaptest"
+import "go.uber.org/zap"

-zaptest.NewLogger(t)
+zap.L()
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-17 16:51:10 +00:00
Evgenii Stratonikov 127c676786 [#607] *: Use keys.PublicKeys.Copy() where possible
Semantic patch:
```
@@
var dst identifier
var src identifier
var keys identifier
@@
 import keys "github.com/nspcc-dev/neo-go/pkg/crypto/keys"

-dst := make(keys.PublicKeys, len(src))
-copy(dst, src)
+dst := src.Copy()
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-17 16:50:02 +00:00
Evgenii Stratonikov e604a3d749 [#607] *: Use zap.Stringer() where possible
Semantic patch:
```
@@
var f expression
var t expression
var a expression
@@
 f(
    ...,
-    zap.String(t, a.String()),
+    zap.Stringer(t, a),
    ...,
)
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-17 16:50:02 +00:00
Evgenii Stratonikov a8de37c8a2 [#607] *: Remove redundant if on error returns
Semantic patch:
```
@@
@@
-if err != nil { return err }
-return nil
+return err
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-17 16:50:02 +00:00
Evgenii Stratonikov 5a51b78946 [#620] object: Send status response for server-side streams
Previously status responses were wrapped in the gRPC error and thus
couldn't be correctly handled on client.

Introduced in c2617baf63, thanks @ale64bit for having found.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-17 12:52:38 +00:00
Anton Nikiforov 6407bb5bd1 [#619] node: Fix object put when copies numbers contains only zeros
DCO action / DCO (pull_request) Successful in 3m59s Details
Build / Build Components (1.21) (pull_request) Successful in 4m53s Details
Vulncheck / Vulncheck (pull_request) Successful in 4m43s Details
Build / Build Components (1.20) (pull_request) Successful in 5m57s Details
Tests and linters / Staticcheck (pull_request) Successful in 8m18s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 8m33s Details
Tests and linters / Tests with -race (pull_request) Successful in 8m41s Details
Tests and linters / Lint (pull_request) Successful in 8m58s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 8m53s Details
In this case object should placement according to replicas.

Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-08-17 11:31:34 +03:00
Evgenii Stratonikov 5335e7089e [#615] pilorama: Speedup TestForest_ApplyRandom()
Some of our pilorama tests fail on CI.
The reasons are not obvious, but one possible improvement
is using `WithNoSync` option for these. It should have much effect,
because we are writing on the tmpfs, but doesn't hurt anyway.

If I replace `t.TempDir()` with a local directory, test execution time
goes down from 5s (sync) to 0.4s (nosync), which is the same time as
with `t.TempDir()`. Maybe we have some strange CI configuration.

```
panic: test timed out after 10m0s
running tests:
	TestForest_ApplyRandom (8m22s)
	TestForest_ApplyRandom/bbolt (8m21s)
...
goroutine 170 [syscall]:
syscall.Syscall(0xc000100000?, 0xc00047b758?, 0x6aff9a?, 0xc00041c1b0?)
	/opt/hostedtoolcache/go/1.20.7/x64/src/syscall/syscall_linux.go:69 +0x27
syscall.Fdatasync(0x9e35c0?)
	/opt/hostedtoolcache/go/1.20.7/x64/src/syscall/zsyscall_linux_amd64.go:418 +0x2a
go.etcd.io/bbolt.fdatasync(0xc000189000?)
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-16 19:09:08 +00:00
Dmitrii Stepanov 2efe9cc1be [#585] writecache: Fix DB counter
DCO action / DCO (pull_request) Successful in 2m14s Details
Vulncheck / Vulncheck (pull_request) Successful in 2m44s Details
Build / Build Components (1.21) (pull_request) Successful in 2m54s Details
Build / Build Components (1.20) (pull_request) Successful in 3m5s Details
Tests and linters / Staticcheck (pull_request) Successful in 3m43s Details
Tests and linters / Lint (pull_request) Successful in 5m12s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 7m44s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 21m5s Details
Tests and linters / Tests with -race (pull_request) Failing after 26m26s Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-08-16 14:47:44 +03:00
Dmitrii Stepanov 58c8722c81 [#585] fstree: Add optional file counter
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-08-16 14:47:44 +03:00
Dmitrii Stepanov baad49990c [#585] fstree: Return logical error if object deleted
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-08-16 14:47:44 +03:00
Dmitrii Stepanov 0c52186572 [#585] fstree: Remove unused method
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-08-16 14:47:44 +03:00
Dmitrii Stepanov eec97d177e [#585] writecache: Count items periodically
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-08-16 14:47:44 +03:00
Evgenii Stratonikov d15199c5d8 [#596] engine: Consider context errors as logical
Vulncheck / Vulncheck (pull_request) Successful in 2m59s Details
DCO action / DCO (pull_request) Successful in 2m54s Details
Build / Build Components (1.20) (pull_request) Successful in 4m2s Details
Build / Build Components (1.21) (pull_request) Successful in 4m51s Details
Tests and linters / Staticcheck (pull_request) Successful in 14m8s Details
Tests and linters / Tests (1.20) (pull_request) Failing after 14m56s Details
Tests and linters / Lint (pull_request) Successful in 15m27s Details
Tests and linters / Tests (1.21) (pull_request) Failing after 15m36s Details
Tests and linters / Tests with -race (pull_request) Failing after 16m18s Details
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-16 10:39:41 +03:00
Alejandro Lopez 88b6755c5e [#598] Fix use-after-close bug in badger writecache
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-08-15 07:04:06 +00:00
Alejandro Lopez ae8be495c8 [#xx] Avoid manual management of files in tests
Vulncheck / Vulncheck (pull_request) Successful in 2m36s Details
Build / Build Components (1.21) (pull_request) Successful in 3m33s Details
DCO action / DCO (pull_request) Successful in 3m33s Details
Tests and linters / Staticcheck (pull_request) Successful in 4m15s Details
Tests and linters / Lint (pull_request) Successful in 5m31s Details
Build / Build Components (1.20) (pull_request) Successful in 7m2s Details
Tests and linters / Tests with -race (pull_request) Failing after 8m51s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 9m55s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 10m6s Details
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-08-14 14:01:39 +03:00
Alejandro Lopez 376f03a445 [#598] Hold mode mutex when setting mode
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-08-14 07:12:27 +00:00
Alexander Chuprov 21800e9fcc [#162] core: Move literals to constants
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2023-08-11 15:48:42 +03:00
Evgenii Stratonikov b44a8dd46c [#597] *: Fix linter warnings
Vulncheck / Vulncheck (pull_request) Successful in 6m26s Details
Tests and linters / Staticcheck (pull_request) Successful in 7m20s Details
Build / Build Components (1.21) (pull_request) Successful in 8m31s Details
Build / Build Components (1.20) (pull_request) Successful in 8m43s Details
Tests and linters / Tests (1.21) (pull_request) Failing after 15m48s Details
Tests and linters / Tests (1.20) (pull_request) Failing after 16m2s Details
Tests and linters / Tests with -race (pull_request) Failing after 16m28s Details
Tests and linters / Lint (pull_request) Successful in 19m8s Details
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-11 09:55:51 +03:00
Evgenii Stratonikov e084c47bd6 [#248] innerring: Remove audit from tests
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-10 16:38:43 +00:00
Evgenii Stratonikov 779da6ec35 [#248] morph: Remove audit contract name
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-10 16:38:43 +00:00
Evgenii Stratonikov 55ce4dc075 [#248] morph: Remove obsolete network parameters
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-10 16:38:43 +00:00
Anton Nikiforov 4ad0ebb32f [#565] Add metrics for current GRPC endpoint status
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-08-10 16:37:49 +00:00
Evgenii Stratonikov c3e23a1448 [#578] gendoc: Allow to override flags
The command is used in multiple places across the whole FrostFS
ecosystem. While we want to have uniform interfaces everywhere,
sometimes we can't: already defined global flags can be harder to change
because of our obligations to the users. Cobra framework doesn't allow
conflicting flags (we can have global ones), so allow to override them.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-10 11:10:11 +00:00
Evgenii Stratonikov 6bcba27757 [#578] gendoc: Remove flag shorthands
Make it harder to encounter conflicts in already existing commands.
Because the command is executed once, I don't think the usability is
worse.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-10 11:10:11 +00:00
Evgenii Stratonikov dca31e8888 [#578] gendoc: Add date to man pages
```
HISTORY
       8-Aug-2023 Auto generated by spf13/cobra
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-10 11:10:11 +00:00
Evgenii Stratonikov b02a1a34c1 [#578] gendoc: Allow to customize man pages
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-10 11:10:11 +00:00
Evgenii Stratonikov 082370583f [#578] gendoc: Replace NSPCC with FrostFS
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-10 11:10:11 +00:00
Alejandro Lopez 8f994163ee [#586] Fix writecache benchmarks and refactor hacky NeedsCompression
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-08-10 08:05:18 +00:00
Alejandro Lopez 023b90342c [#584] Disable compression in badger writecache
Build / Build Components (1.20) (pull_request) Failing after 3s Details
Build / Build Components (1.21) (pull_request) Failing after 2s Details
Tests and linters / Lint (pull_request) Failing after 4s Details
Tests and linters / Tests (1.20) (pull_request) Failing after 3s Details
Tests and linters / Tests (1.21) (pull_request) Failing after 3s Details
Tests and linters / Tests with -race (pull_request) Failing after 3s Details
Tests and linters / Staticcheck (pull_request) Failing after 3s Details
Vulncheck / Vulncheck (pull_request) Failing after 3s Details
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-08-10 10:14:06 +03:00
Evgenii Stratonikov d641cba2fc [#587] Do not use math/rand.Read
Build / Build Components (1.20) (pull_request) Failing after 3s Details
Build / Build Components (1.21) (pull_request) Failing after 3s Details
Tests and linters / Lint (pull_request) Failing after 3s Details
Tests and linters / Tests (1.20) (pull_request) Failing after 3s Details
Tests and linters / Tests (1.21) (pull_request) Failing after 3s Details
Tests and linters / Tests with -race (pull_request) Failing after 5s Details
Tests and linters / Staticcheck (pull_request) Failing after 8s Details
Vulncheck / Vulncheck (pull_request) Failing after 2s Details
Fix staticcheck warnings after go1.20 update.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-09 16:02:44 +03:00
Alejandro Lopez 5b7e4a51b7 [#481] Update frostfs-sdk-go and error pointer receivers
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-08-09 10:26:53 +00:00
Alejandro Lopez ae322e9f73 [#576] Set SyncWrites for badger writecache by default
Build / Build Components (1.19) (pull_request) Failing after 3s Details
Build / Build Components (1.20) (pull_request) Failing after 2s Details
Tests and linters / Tests (1.19) (pull_request) Failing after 3s Details
Tests and linters / Tests with -race (pull_request) Failing after 3s Details
Tests and linters / Lint (pull_request) Failing after 12s Details
Tests and linters / Staticcheck (pull_request) Failing after 3s Details
Vulncheck / Vulncheck (pull_request) Successful in 5m46s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 11m37s Details
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-08-09 11:28:26 +03:00
Anton Nikiforov 8d589314b5 [#560] node: Fix `Put` in multi REP with intersecting sets of nodes
Once the node was processed it skipped, at the step of forming
result in case when all nodes skipped, because processed for
previous REP, service mark the whole request as incomplete.

Example of policies which are unblocked:
- REP 1 REP 1 CBF 1
- REP 4 IN X REP 4 IN Y
  CBF 4
  SELECT 2 FROM FX AS X SELECT 2 FROM FY AS Y
  FILTER Country EQ Russia OR Country EQ Sweden OR Country EQ Finland AS FY
  FILTER Price GE 0 AS FX

Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-08-08 10:22:53 +00:00
Alejandro Lopez 7da4306e38 [#575] writecache: Fix log level for badger writecache
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-08-08 08:16:36 +00:00
Dmitrii Stepanov 0e697266c3 [#563] writecache: Fix metrics and bolt delete
Estimate cache size after delete objects to update metric.
Update counters on small object deletion.
Do not count bbolt DB file as FSTree object.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-08-07 12:53:28 +00:00
Alejandro Lopez 1a0cb0f34a [#421] Try using badger for the write-cache
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-08-07 08:16:57 +00:00
Alejandro Lopez 65c72f3e0b [#559] Remove manual path handling in fstree tests
Build / Build Components (1.19) (pull_request) Successful in 2m8s Details
Build / Build Components (1.20) (pull_request) Successful in 1m54s Details
Tests and linters / Lint (pull_request) Successful in 2m48s Details
Tests and linters / Tests (1.19) (pull_request) Successful in 1m53s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 2m4s Details
Tests and linters / Tests with -race (pull_request) Successful in 5m21s Details
Tests and linters / Staticcheck (pull_request) Successful in 2m16s Details
Vulncheck / Vulncheck (pull_request) Failing after 16m3s Details
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-08-03 10:03:41 +03:00
Evgenii Stratonikov 1e8b4b8a17 [#557] services: Regenerate stable marshalers
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-02 13:37:40 +00:00
Airat Arifullin b3695411d9 [#553] eacl: Fix bug with casting to ObjectAccessDenied error
Signed-off-by: Airat Arifullin a.arifullin@yadro.com
2023-08-02 07:22:48 +00:00
Evgenii Stratonikov 35dc64bd7b [#547] metabase: Fix datarace in tests
Quite an old one bf9e938a3b.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-07-31 15:35:15 +00:00
Evgenii Stratonikov 05ac9e3637 [#547] objectsvc: Work with `traversal` struct from a single thread
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-07-31 15:35:15 +00:00
Evgenii Stratonikov 7b0fdf0202 [#533] services: Assume API supports status codes
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-07-31 15:35:15 +00:00
Evgenii Stratonikov ec8b4fdc48 [#541] writecache/test: Close writecache on exit
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-07-29 10:57:25 +00:00
Evgenii Stratonikov ad5f527bd3 [#541] writecache/test: Remove initWC()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-07-29 10:57:25 +00:00
Dmitrii Stepanov ea32913430 [#543] putsvc: Fix PutSingle implementation
Add Lock and Delete handlers to local PutSingle.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-28 12:09:41 +00:00
Dmitrii Stepanov 99bb488ebd [#539] getsvc: Write payload direct to out stream
Build / Build Components (1.20) (pull_request) Failing after 1s Details
Build / Build Components (1.19) (pull_request) Failing after 3s Details
Tests and linters / Lint (pull_request) Failing after 2s Details
Tests and linters / Tests (1.19) (pull_request) Failing after 1s Details
Tests and linters / Tests (1.20) (pull_request) Failing after 2s Details
Tests and linters / Tests with -race (pull_request) Failing after 1s Details
Vulncheck / Vulncheck (pull_request) Failing after 2s Details
Tests and linters / Staticcheck (pull_request) Failing after 2s Details
To reduce memory allocations.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-27 17:02:08 +03:00
Dmitrii Stepanov 286242cad0 [#539] getsvc: Use buffer to assemble object
To reduce memory consumption.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-27 17:02:08 +03:00
Evgenii Stratonikov 5ff82ff04f [#6] services/object: Simplify local/remote targets
ci/woodpecker/pr/pre-commit Pipeline was successful Details
Tests and linters / Tests (1.19) (pull_request) Successful in 14m18s Details
Vulncheck / Vulncheck (pull_request) Failing after 2m23s Details
Tests and linters / Staticcheck (pull_request) Successful in 4m8s Details
Build / Build Components (1.19) (pull_request) Successful in 3m26s Details
Build / Build Components (1.20) (pull_request) Successful in 2m56s Details
Tests and linters / Tests with -race (pull_request) Successful in 21m39s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 22m36s Details
Tests and linters / Lint (pull_request) Successful in 1m39s Details
ci/woodpecker/push/pre-commit Pipeline was successful Details
We do not use the return result from Close() and we always execute both
methods in succession. It makes sense to unite them.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-07-21 18:39:12 +03:00
Evgenii Stratonikov 448b48287c [#6] services/util: Do not panic in sign function
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-07-21 18:39:12 +03:00
Evgenii Stratonikov c2617baf63 [#6] services/util: Remove remaining stream wrappers
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-07-21 18:39:09 +03:00
Evgenii Stratonikov 372160d048 [#6] services/util: Remove `SignService.HandleUnaryRequest`
There is no need in a wrapper with many from-`interface{}` conversions.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-07-21 18:36:08 +03:00
Evgenii Stratonikov fef172c5b0 [#6] services/util: Simplify `response.Service`
It has only 1 parameter, which is obligatory.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-07-21 18:29:57 +03:00
Dmitrii Stepanov eed594431f [#335] treesvc: Add GetSubTree ordering unit test
Build / Build Components (1.20) (pull_request) Successful in 3m47s Details
Build / Build Components (1.19) (pull_request) Successful in 4m2s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 3m8s Details
ci/woodpecker/pr/pre-commit Pipeline was successful Details
Tests and linters / Staticcheck (pull_request) Successful in 3m48s Details
Tests and linters / Tests with -race (pull_request) Successful in 7m59s Details
Tests and linters / Tests (1.19) (pull_request) Successful in 15m36s Details
Tests and linters / Lint (pull_request) Successful in 18m12s Details
ci/woodpecker/push/pre-commit Pipeline was successful Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-20 10:14:10 +03:00
Dmitrii Stepanov af82c2865e [#335] treesvc: Fix inmemory unit tests and nil meta items
Bolt forest saves empty slice of items. Now inmemory forest
does it the same way.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-20 10:14:10 +03:00
Dmitrii Stepanov b4e72a2dfd [#335] treesvc: Sort nodes by Filename in GetSubTree
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-20 10:14:10 +03:00
Dmitrii Stepanov f0355a453e [#463] policer: Remove capacity rebalance logic
ci/woodpecker/push/pre-commit Pipeline was successful Details
Current implementation has some quirks. For example,
using only half of object.put.pool_size_remote threads
tells replicator that is node is 50% loaded,
but in reality we could be putting lot's of big objects.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-18 10:52:12 +00:00
Dmitrii Stepanov 8b78db74bc [#463] replicator: Add tracing span
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-18 10:52:12 +00:00
Dmitrii Stepanov 8966dd8e35 [#463] putsvc: Use PutSingle RPC for remote target
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-18 10:52:12 +00:00
Alejandro Lopez 3b66f98f27 [#519] Use Address.Equals in policer tests
ci/woodpecker/push/pre-commit Pipeline was successful Details
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-07-17 10:45:07 +00:00
Dmitrii Stepanov 3e8de14e7d [#382] evacuate: Fix unit tests
ci/woodpecker/push/pre-commit Pipeline was successful Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-17 10:24:05 +00:00
Dmitrii Stepanov 397131b0ea [#520] objectcore: Refactor format validator
ci/woodpecker/push/pre-commit Pipeline was successful Details
Remove redundant FIXME.
Move error to consts.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-17 07:07:21 +00:00
Alejandro Lopez 11027945d8 [#479] writecache: Fix writecache fstree flush premature ctx cancel
Build / Build Components (1.20) (pull_request) Successful in 3m49s Details
Build / Build Components (1.19) (pull_request) Successful in 4m2s Details
ci/woodpecker/pr/pre-commit Pipeline was successful Details
Tests and linters / Tests (1.20) (pull_request) Successful in 6m13s Details
Tests and linters / Lint (pull_request) Successful in 11m25s Details
Tests and linters / Tests with -race (pull_request) Successful in 9m4s Details
Tests and linters / Staticcheck (pull_request) Successful in 6m0s Details
Tests and linters / Tests (1.19) (pull_request) Successful in 25m11s Details
ci/woodpecker/push/pre-commit Pipeline was successful Details
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-07-14 10:25:52 +03:00
Evgenii Stratonikov 8a9fc2c372 [#510] treesvc: Rename `tableFromBearer` to `useBearer`
ci/woodpecker/push/pre-commit Pipeline was successful Details
With impersonation, the old name is no longer descriptive.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-07-12 10:23:21 +00:00
Evgenii Stratonikov b8bcfac531 [#510] treesvc: Fix panic in bearer token processing
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-07-12 10:23:21 +00:00
Dmitrii Stepanov 24eb988897 [#294] deletesvc: Drop cast
ci/woodpecker/push/pre-commit Pipeline was successful Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-12 07:42:10 +00:00
Dmitrii Stepanov c83e7c875f [#294] searchsvcv2: Refactor service constructor
Pass required deps as args.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-12 07:42:10 +00:00
Dmitrii Stepanov e8091101c7 [#294] searchsvc: Refactor service constructor
Pass required deps as args.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-12 07:42:10 +00:00
Dmitrii Stepanov ec9b738465 [#294] putsvcv2: Refactor service constructor
Pass required deps as args.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-12 07:42:10 +00:00
Dmitrii Stepanov 800a685e84 [#294] putsvc: Refactor service constructor
Pass required deps as args.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-12 07:42:10 +00:00
Dmitrii Stepanov 1420b8b9ea [#294] deletesvc: Refactor service constructor
Pass required deps as args.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-12 07:42:10 +00:00
Dmitrii Stepanov a476d8285a [#294] deletesvcv2: Refactor service constructor
Pass required deps as args.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-12 07:42:10 +00:00
Dmitrii Stepanov 70a1081988 [#294] aclsvcv2: Refactor service constructor
Pass required deps as args.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-12 07:42:10 +00:00
Dmitrii Stepanov 18d8898b00 [#294] aclsvc: Refactor service constructor
Pass required deps as args.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-12 07:42:10 +00:00
Dmitrii Stepanov 61541eaec2 [#294] aclsvc: Refactor checker constructor
Pass required deps as args.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-12 07:42:10 +00:00
Evgenii Stratonikov 7da284f3e8 [#509] go.mod: Update sdk-go
ci/woodpecker/pr/pre-commit Pipeline was successful Details
Tests and linters / Lint (pull_request) Successful in 2m15s Details
Tests and linters / Tests (1.19) (pull_request) Successful in 5m7s Details
Tests and linters / Tests with -race (pull_request) Successful in 4m50s Details
Tests and linters / Staticcheck (pull_request) Successful in 4m10s Details
Build / Build Components (1.19) (pull_request) Successful in 3m47s Details
Build / Build Components (1.20) (pull_request) Successful in 3m30s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 20m39s Details
ci/woodpecker/push/pre-commit Pipeline was successful Details
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-07-11 17:38:00 +03:00
Evgenii Stratonikov 80481c015c [#509] putsvc: Omit AccessIdentifiers from iteratePlacement()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-07-11 17:38:00 +03:00
Dmitrii Stepanov a65e26878b [#486] put service: Fix error typo
ci/woodpecker/pr/pre-commit Pipeline failed Details
Build / Build Components (1.19) (pull_request) Successful in 4m12s Details
Tests and linters / Tests (1.19) (pull_request) Successful in 2m52s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 3m49s Details
Tests and linters / Staticcheck (pull_request) Successful in 3m13s Details
Tests and linters / Lint (pull_request) Successful in 10m50s Details
Build / Build Components (1.20) (pull_request) Successful in 11m28s Details
Tests and linters / Tests with -race (pull_request) Successful in 7m54s Details
ci/woodpecker/push/pre-commit Pipeline was successful Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-10 15:49:21 +03:00
Dmitrii Stepanov fcbf90d31b [#486] node: Add PutSingle implemetation
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-10 15:49:21 +03:00
Dmitrii Stepanov 7b76527759 [#486] node: Add PutSingle wrappers
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-10 15:49:21 +03:00
Evgenii Stratonikov 140d970a95 [#498] policer: Fix objectSDK import
ci/woodpecker/pr/pre-commit Pipeline failed Details
Build / Build Components (1.19) (pull_request) Successful in 4m23s Details
Tests and linters / Tests (1.19) (pull_request) Successful in 3m27s Details
Tests and linters / Lint (pull_request) Successful in 4m11s Details
Tests and linters / Tests with -race (pull_request) Successful in 5m36s Details
Build / Build Components (1.20) (pull_request) Successful in 10m18s Details
Tests and linters / Staticcheck (pull_request) Successful in 5m41s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 11m33s Details
ci/woodpecker/push/pre-commit Pipeline was successful Details
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-07-07 18:52:40 +03:00
Evgenii Stratonikov 2310a5c7ba [#498] policer: Allow to set sleep duration between iterations
ci/woodpecker/push/pre-commit Pipeline failed Details
Speed up tests on CI.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-07-07 14:52:33 +00:00
Evgenii Stratonikov e858479a74 [#498] policer: Explicitly Rewind() iterator after finish
Previously, we can continue to return `EndOfListing` infinitely.
Reflect iterator reuse via Rewind() method.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-07-07 14:52:33 +00:00
Alexander Chuprov 033eaf77e1 [#496] node: Fix linter importas
Build / Build Components (1.20) (pull_request) Successful in 3m52s Details
Build / Build Components (1.19) (pull_request) Successful in 4m1s Details
ci/woodpecker/pr/pre-commit Pipeline was successful Details
Tests and linters / Tests with -race (pull_request) Successful in 5m36s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 5m55s Details
Tests and linters / Lint (pull_request) Successful in 14m40s Details
Tests and linters / Tests (1.19) (pull_request) Successful in 15m29s Details
ci/woodpecker/push/pre-commit Pipeline was successful Details
Standardize the alias of the
import frostfs-sdk-go/object as objectSDK.

Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2023-07-06 15:36:41 +03:00
Evgenii Stratonikov 6eefe9747e treesvc: Do not provide credentials unless TLS is used
ci/woodpecker/push/pre-commit Pipeline failed Details
4f413fe86e was perfect, except it did the opposite of what we needed.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-07-06 08:04:28 +00:00
Alexey Vanin f354b8a270 [#473] Add tests for rem values of traverser
Build / Build Components (1.20) (pull_request) Successful in 3m52s Details
Build / Build Components (1.19) (pull_request) Successful in 4m8s Details
ci/woodpecker/pr/pre-commit Pipeline was successful Details
Tests and linters / Lint (pull_request) Successful in 9m3s Details
Tests and linters / Tests with -race (pull_request) Successful in 5m27s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 6m18s Details
Tests and linters / Tests (1.19) (pull_request) Successful in 13m54s Details
ci/woodpecker/push/pre-commit Pipeline was successful Details
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-07-05 20:13:23 +03:00
Alexey Vanin c6df6c84ae [#473] Do not modify traverse plan when `[0]` copies number vector is provided
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-07-05 19:54:43 +03:00
Anton Nikiforov 90e9a85acc [#483] Update dependencies
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-07-03 15:44:59 +03:00
Dmitrii Stepanov 8d16d95376 [#484] morph: Add expired tx logging
ci/woodpecker/push/pre-commit Pipeline was successful Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-03 08:03:25 +00:00
Alejandro Lopez 26acf5689e [#92] Ensure policer objects cannot be worked on concurrently
ci/woodpecker/push/pre-commit Pipeline was successful Details
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-07-03 07:05:31 +00:00
Alejandro Lopez 3223402c90 [#92] Embed policer's objectsInWork mutex
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-07-03 07:05:31 +00:00
Alejandro Lopez f9730f090d [#92] Refactor policer and add some unit tests
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-07-03 07:05:31 +00:00
Airat Arifullin f437ab8f15 [#197] object: Make Delete method return correct status
ci/woodpecker/push/pre-commit Pipeline was successful Details
Signed-off-by: Airat Arifullin a.arifullin@yadro.com
2023-06-30 12:58:56 +00:00
Anton Nikiforov d01c064674 [#455] Fix non-informative message when parse node attributes
ci/woodpecker/push/pre-commit Pipeline was successful Details
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-06-28 15:02:29 +00:00
Evgenii Stratonikov d0ab552a90 [#478] *: Fix funlen linter warnings
ci/woodpecker/push/pre-commit Pipeline was successful Details
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-28 15:01:49 +00:00
Evgenii Stratonikov 8a4e250dae [#468] *: replace outdated TODO crypto-related links
ci/woodpecker/push/pre-commit Pipeline was successful Details
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-28 12:13:20 +00:00
Evgenii Stratonikov 4f413fe86e [#1] treesvc: Properly check for secure transport
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-28 12:13:20 +00:00
Evgenii Stratonikov cab51c8cbe [#1] metabase: Rename blindlyProcess()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-28 12:13:20 +00:00
Evgenii Stratonikov f4c71cea65 [#1] *: Replace outdated FIXME/TODO links
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-28 12:13:20 +00:00
Evgenii Stratonikov 0c866f62c5 [#473] placement: Fix backwards compatibility for `copies_number`
ci/woodpecker/pr/pre-commit Pipeline was successful Details
ci/woodpecker/push/pre-commit Pipeline was successful Details
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-27 11:13:16 +03:00
Dmitrii Stepanov 43d263c3d5 [#428] linter: Fix unkeyed assignment
ci/woodpecker/push/pre-commit Pipeline was successful Details
Thanks to gopls.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-26 13:29:39 +00:00
Dmitrii Stepanov cac4ed93d6 [#428] engine: Add low_mem config parameter
Concurrent initialization in case of the metabase resync leads to
high memory consumption and potential OOM.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-26 13:29:39 +00:00
Evgenii Stratonikov 4bf345225c [#447] pilorama: Use named constant for the key size
ci/woodpecker/push/pre-commit Pipeline was successful Details
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-26 07:42:05 +00:00
Evgenii Stratonikov b4ce0b0412 [#447] pilorama: Do not undo log for create ops
```
goos: linux
goarch: amd64
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
         │     old      │                 new                  │
         │    sec/op    │    sec/op     vs base                │
Create-8   36.48µ ± 11%   30.34µ ± 14%  -16.84% (p=0.000 n=10)

         │     old      │                 new                  │
         │     B/op     │     B/op      vs base                │
Create-8   43.01Ki ± 4%   37.78Ki ± 5%  -12.15% (p=0.000 n=10)

         │    old     │                new                 │
         │ allocs/op  │ allocs/op   vs base                │
Create-8   166.0 ± 3%   146.0 ± 3%  -12.05% (p=0.000 n=10)
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-26 07:42:05 +00:00
Evgenii Stratonikov dd3874eff1 [#447] pilorama: Add benchmark for create ops
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-26 07:42:05 +00:00
Anton Nikiforov 71889234b7 [#449] tree: Allow reading requests signed by keys from allow list
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-06-23 14:44:19 +03:00
Evgenii Stratonikov 40eae22109 [#460] services/util: Remove `CreateRequestStreamer`
ci/woodpecker/pr/pre-commit Pipeline was successful Details
ci/woodpecker/push/pre-commit Pipeline was successful Details
There is no need in a wrapper with many from-`interface{}` conversions.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-21 17:08:04 +03:00
Evgenii Stratonikov a64dc9ad70 [#460] services/util: Remove `HandleServerStreamRequest`
There is no need in a wrapper with many from-`interface{}` conversions.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-21 17:08:01 +03:00
Evgenii Stratonikov 167a67f0b8 [#460] services/util: Remove `HandleUnaryRequest`
There is no need in a wrapper with many from-`interface{}` conversions.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-21 17:07:56 +03:00
Evgenii Stratonikov 785d81a68a [#460] services/object: Reduce `distibutedTarget` memory footprint
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-21 17:07:50 +03:00
Dmitrii Stepanov 4d48377cec [#459] blobovniczatree: Fix get error
ci/woodpecker/pr/pre-commit Pipeline was successful Details
ci/woodpecker/push/pre-commit Pipeline was successful Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-21 16:44:53 +03:00
Dmitrii Stepanov 03aa210145 [#373] metrics: Move labels to consts
ci/woodpecker/pr/pre-commit Pipeline was successful Details
ci/woodpecker/push/pre-commit Pipeline was successful Details
To unify label naming all lable keys and other consts are moved to
one file.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-21 15:13:26 +03:00
Dmitrii Stepanov b5d9f4a285 [#373] metrics: Add pilorama metrics
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-21 15:13:26 +03:00
Dmitrii Stepanov e89fa110c7 [#373] metrics: Add metabase metrics
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-21 15:13:26 +03:00
Dmitrii Stepanov af608da952 [#373] metrics: Add blobovnizca metrics
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-21 15:13:26 +03:00
Dmitrii Stepanov 56f320dd85 [#373] metrics: Add blobstor metrics
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-21 15:13:26 +03:00
Dmitrii Stepanov 16a142cd0c [#373] metrics: Add FSTree metrics
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-21 15:13:26 +03:00
Dmitrii Stepanov d8ecc69d00 [#373] local storage: Pass parent ID
This is required to add shard ID as metric label.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-21 15:13:26 +03:00
Dmitrii Stepanov d5aaec1107 [#373] pilorama: Add metrics
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-21 15:13:26 +03:00
Dmitrii Stepanov 059e9e88a2 [#373] metabase: Add metrics
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-21 15:13:26 +03:00
Dmitrii Stepanov f54cc0b607 [#373] blobstor: Add metrics
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-21 15:13:26 +03:00
Dmitrii Stepanov 8318d90ad0 [#373] blobovniczatree: Add metrics
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-21 15:13:26 +03:00
Dmitrii Stepanov 3ae3c8dfdb [#373] fstree: Add metrics
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-21 15:13:26 +03:00
Dmitrii Stepanov a8526d45e9 [#373] blobovnizca: Add missed/fix tracing spans
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-21 15:13:26 +03:00
Dmitrii Stepanov 028d4a8058 [#373] blobovnicza: Add metrics
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-21 15:13:26 +03:00
Dmitrii Stepanov 01a0c97760 [#453] engine: Set Disabled mode to deleted shard
ci/woodpecker/pr/pre-commit Pipeline was successful Details
ci/woodpecker/push/pre-commit Pipeline was successful Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-20 12:04:07 +03:00
Anton Nikiforov 50caa388b0 [#303] ir: Use pub key when validate container deletion
ci/woodpecker/push/pre-commit Pipeline was successful Details
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-06-20 08:02:48 +00:00
Evgenii Stratonikov 69df0d21c2 [#446] engine: Move to read-only on blobstor errors
ci/woodpecker/pr/pre-commit Pipeline was successful Details
ci/woodpecker/push/pre-commit Pipeline was successful Details
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-16 14:53:32 +03:00
Evgenii Stratonikov fe01781811 [#446] los: Wrap SSD errors in a separate type
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-16 14:53:31 +03:00
Evgenii Stratonikov 20b84f183a [#446] engine: Simplify logs for shard mode change
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-16 14:51:29 +03:00
Dmitrii Stepanov 69b788a90b [#424] metrics: Rename mode to mode_info
ci/woodpecker/pr/pre-commit Pipeline was successful Details
ci/woodpecker/push/pre-commit Pipeline was successful Details
To follow best practice.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-14 18:39:49 +03:00
Dmitrii Stepanov 26b305f82b [#424] morph: Fix cache metrics
ci/woodpecker/pr/pre-commit Pipeline was successful Details
Use separate morph cache metrics for node and IR

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-14 18:26:19 +03:00
Dmitrii Stepanov 4449006862 [#424] metrics: Use mode value as metric value for shard
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-14 18:26:19 +03:00
Dmitrii Stepanov 847732605c [#424] metrics: Rename counter to total
Suffix total should be used for a unit-less accumulating count.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-14 14:53:32 +03:00
Dmitrii Stepanov c348ae35b0 [#424] metrics: Drop embedded metrics
It was not obvious where metrics are used.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-14 14:53:32 +03:00
Dmitrii Stepanov 1b364d8cf4 [#424] metrics: Refactor engine metrics
Use histogram vector to measure request duration.
Fix naming like in Prometheus best practice.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-14 14:53:32 +03:00
Dmitrii Stepanov c8023a9c8d [#424] morphcache: Use labels for method duration
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-14 14:53:32 +03:00
Dmitrii Stepanov 85deb12f4d [#424] writecache: Drop metrics when close
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-14 14:52:40 +03:00
Dmitrii Stepanov 07f155ac77 [#424] metrics: Use labels for writecache methods and operations
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-14 14:52:40 +03:00
Dmitrii Stepanov 71bbeddb64 [#424] metrics: Drop unused arg
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-14 14:51:40 +03:00
Alejandro Lopez fb8fee0c8e [#442] Use strconv.FormatBool instead of untyped conversion
ci/woodpecker/push/pre-commit Pipeline was successful Details
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-06-14 11:08:36 +00:00
Alejandro Lopez 344d6b2ae1 [#xx] Fix invalid log metric namespace identifier
ci/woodpecker/pr/pre-commit Pipeline was successful Details
ci/woodpecker/push/pre-commit Pipeline was successful Details
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-06-13 15:36:34 +03:00
Alejandro Lopez 4887f489a1 [#17] Add morph client metrics
ci/woodpecker/pr/pre-commit Pipeline was successful Details
ci/woodpecker/push/pre-commit Pipeline was successful Details
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-06-13 14:06:04 +03:00
Airat Arifullin 90e9247b69 [#371] ir: Add morph cache metrics
ci/woodpecker/push/pre-commit Pipeline was successful Details
Signed-off-by: Airat Arifullin a.arifullin@yadro.com
2023-06-13 10:10:13 +00:00
Dmitrii Stepanov 957a43a124 [#266] services/tree: Add sync check
ci/woodpecker/push/pre-commit Pipeline was successful Details
Do not accept requests until initial sync is finished.
`Apply` is deliberately left out -- we don't want to miss anything new.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-13 10:00:45 +00:00
Dmitrii Stepanov e69a1e8482 [#266] services/tree: Return operation log up to some height
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-13 10:00:45 +00:00
Dmitrii Stepanov 2541d319de [#266] pilorama: Allow to get current tree height
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-13 10:00:45 +00:00
Alejandro Lopez 0c40d98f7a [#375] Add log metrics
ci/woodpecker/push/pre-commit Pipeline was successful Details
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-06-13 09:52:45 +00:00
Dmitrii Stepanov 0400153b7d [#424] object: Do not store large slices in pool
Dynamically growing an unbounded buffers can cause a large
amount of memory to be pinned and never be freed.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-13 07:19:55 +00:00
Evgenii Stratonikov 41ab4d070e [#423] *: Use hrw.StringHash() where possible
ci/woodpecker/push/pre-commit Pipeline was successful Details
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-13 07:18:25 +00:00
Dmitrii Stepanov 9bc1a25c07 [#434] metrics: Fix writecache duration
ci/woodpecker/push/pre-commit Pipeline was successful Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-13 07:18:01 +00:00
Dmitrii Stepanov 5b75432ca2 [#431] metrics: Add missed label
ci/woodpecker/push/pre-commit Pipeline was successful Details
ci/woodpecker/pr/pre-commit Pipeline was successful Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-07 13:51:36 +00:00
Anton Nikiforov 263c6fdc50 [#372] node: Add metrics for the error counter in the engine
ci/woodpecker/push/pre-commit Pipeline was successful Details
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-06-07 13:04:47 +00:00
Evgenii Stratonikov 63473d0806 [#417] go.mod: Update neo-go
ci/woodpecker/push/pre-commit Pipeline was successful Details
Update to master, because after the API update notary signer is being
incorrectly formed.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-02 11:43:48 +00:00
Dmitrii Stepanov 55c28fd5f4 [#412] cache: Pass DialOptions to create new conn
ci/woodpecker/push/pre-commit Pipeline was successful Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-01 13:23:11 +00:00
Dmitrii Stepanov dcdfb6ed41 [#412] node: Use observability interceptors
Use metrics and tracing interceptors.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-01 13:23:11 +00:00
Dmitrii Stepanov c09144ecf1 [#412] node: Replace metrics package
Use observability module.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-01 13:23:11 +00:00
Dmitrii Stepanov 74578052f9 [#412] node: Replace tracing package
Use observability module.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-01 13:23:11 +00:00
Alejandro Lopez f7c4c07453 [#xx] Create innerring metrics before metrics usage
ci/woodpecker/pr/pre-commit Pipeline was successful Details
ci/woodpecker/push/pre-commit Pipeline was successful Details
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-06-01 10:07:48 +03:00
Evgenii Stratonikov 4476a1dbaf [#413] morph/client: Fix govet warnings
ci/woodpecker/pr/pre-commit Pipeline was successful Details
ci/woodpecker/push/pre-commit Pipeline was successful Details
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-31 15:38:17 +03:00
Dmitrii Stepanov dbf41391b5 [#401] engine: Extend evacuation logs
ci/woodpecker/pr/pre-commit Pipeline was successful Details
ci/woodpecker/push/pre-commit Pipeline was successful Details
Add operation-tag to logger.
Log evacuation results.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-05-31 13:24:30 +03:00
Dmitrii Stepanov 3220c4df9f [#376] metrics: Add GC metrics
ci/woodpecker/push/pre-commit Pipeline was successful Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-05-31 10:22:12 +00:00
Alejandro Lopez faca861451 [#411] Remove unnecessary pointers for sync objects
ci/woodpecker/push/pre-commit Pipeline was successful Details
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-05-31 10:19:14 +00:00
Alejandro Lopez ebcc8afbee [#374] Add inner-ring event metrics
ci/woodpecker/push/pre-commit Pipeline was successful Details
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-05-31 10:11:48 +00:00
Anton Nikiforov 8dcd06c587 [#394] node: Use `Context` in `Blobovniczas.Iterate()`
ci/woodpecker/push/pre-commit Pipeline was successful Details
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-05-31 10:09:18 +00:00
Anton Nikiforov 365a7ca0f4 [#366] node: Stop GC once termination signal received
ci/woodpecker/pr/pre-commit Pipeline was successful Details
ci/woodpecker/push/pre-commit Pipeline was successful Details
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-05-29 09:35:08 +03:00
Anton Nikiforov 802168c0c6 [#364] node: Stop flushing big object when termination signal received
ci/woodpecker/pr/pre-commit Pipeline was successful Details
ci/woodpecker/push/pre-commit Pipeline was successful Details
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-05-26 16:46:58 +03:00
Dmitrii Stepanov 271a56c2ab [#395] metrics: Drop redundant metrics
ci/woodpecker/push/pre-commit Pipeline was successful Details
HistogramVec already has labeled counter.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-05-26 13:39:37 +00:00
Alejandro Lopez bc34fee6a7 [#370] Add tree service metrics
ci/woodpecker/push/pre-commit Pipeline was successful Details
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-05-26 13:39:12 +00:00
Evgenii Stratonikov f2e5dead7e [#398] pilorama: Disallow applying same operations
ci/woodpecker/push/pre-commit Pipeline was successful Details
1. In redo() we save the old state.
2. If we do redo() for the same operation twice, the old state will be
   overritten with the new one.
3. This in turn affects undo() and subsequent isAncestor() check.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-26 12:14:29 +00:00
Dmitrii Stepanov 20a489bdb5 [#393] gc: Use defer to mark handler done
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-05-26 12:14:02 +00:00
Dmitrii Stepanov 9119199f6e [#397] cli: Fix evacuation method names
ci/woodpecker/pr/pre-commit Pipeline was successful Details
ci/woodpecker/push/pre-commit Pipeline was successful Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-05-26 11:28:30 +03:00
Dmitrii Stepanov 2613351008 [#387] gc: Cancel GC is change mode requested
ci/woodpecker/pr/pre-commit Pipeline was successful Details
ci/woodpecker/push/pre-commit Pipeline was successful Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-05-25 09:38:16 +03:00
Dmitrii Stepanov 2ce43935f9 [#312] metrics: Add writecache metrcis
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-05-24 10:18:39 +00:00
Dmitrii Stepanov d212d908b5 [#312] wc: Add metrics
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-05-24 10:18:39 +00:00
Dmitrii Stepanov 4503a61997 [#312] wc: Delete unused Iterate method
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-05-24 10:18:39 +00:00
Dmitrii Stepanov 53a1b15693 [#338] ir: Drop notaryless code from governance
ci/woodpecker/push/pre-commit Pipeline was successful Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-05-24 08:44:47 +00:00
Dmitrii Stepanov 0ab589dd52 [#338] ir: Always enable notary on sidechain
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-05-24 08:44:47 +00:00
Dmitrii Stepanov 81718afb39 [#338] ir: Drop named named put fee
Named put fee value used only when notary disabled.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-05-24 08:44:47 +00:00
Dmitrii Stepanov 656fd7f376 [#338] ir: Drop notaryless code from netmap
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-05-24 08:44:47 +00:00
Dmitrii Stepanov fb708b3a2d [#338] ir: Drop container notaryless code
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-05-24 08:44:47 +00:00
Evgenii Stratonikov 4b768fd115 [#381] *: Move to sync/atomic
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-23 08:18:01 +03:00
Evgenii Stratonikov ff570847a4 [#381] go.mod: Update bbolt
Adopt new `ForEachBucket` function where possible.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-22 11:49:14 +03:00
Evgenii Stratonikov 35c9b6b26d [#314] writecache: remove objects right after they are flushed
ci/woodpecker/push/pre-commit Pipeline was successful Details
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-19 09:37:05 +00:00