Dmitrii Stepanov
29550fe600
[ #838 ] shard: Refactor `updateMetrics` method
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-06 15:44:21 +03:00
Alexander Chuprov
b2c63e57ba
[ #651 ] engine/test: Speedup StorageEngine_Inhume
...
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2023-11-30 13:19:43 +00:00
Alexander Chuprov
445ebcc0e7
[ #651 ] shard/test: Speedup Shard_Delete
...
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2023-11-30 13:19:43 +00:00
Alexander Chuprov
2302e5d342
[ #651 ] shard/test: Refactor Shard_Delete
...
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2023-11-30 13:19:43 +00:00
Dmitrii Stepanov
07390ad4e3
[ #715 ] node: Unify config parameter names
...
DCO action / DCO (pull_request) Successful in 2m54s
Details
Vulncheck / Vulncheck (pull_request) Successful in 3m11s
Details
Tests and linters / Staticcheck (pull_request) Successful in 3m55s
Details
Build / Build Components (1.21) (pull_request) Successful in 3m51s
Details
Build / Build Components (1.20) (pull_request) Successful in 4m6s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 5m10s
Details
Tests and linters / Lint (pull_request) Successful in 5m24s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 6m56s
Details
Tests and linters / Tests with -race (pull_request) Successful in 7m46s
Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-22 17:13:50 +03:00
Dmitrii Stepanov
8d18fa159e
[ #667 ] writecache: Fix flush test
...
DCO action / DCO (pull_request) Successful in 1m28s
Details
Vulncheck / Vulncheck (pull_request) Successful in 3m55s
Details
Tests and linters / Staticcheck (pull_request) Successful in 4m15s
Details
Build / Build Components (1.21) (pull_request) Successful in 4m40s
Details
Build / Build Components (1.20) (pull_request) Successful in 4m54s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 5m32s
Details
Tests and linters / Lint (pull_request) Successful in 5m59s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 6m59s
Details
Tests and linters / Tests with -race (pull_request) Successful in 6m57s
Details
Allow to disable background flusher for testing purposes.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-17 17:45:43 +03:00
Dmitrii Stepanov
76ff26039c
[ #96 ] node: Drop neo-go's `slices` package
...
DCO action / DCO (pull_request) Successful in 2m13s
Details
Build / Build Components (1.20) (pull_request) Successful in 5m48s
Details
Build / Build Components (1.21) (pull_request) Successful in 5m47s
Details
Vulncheck / Vulncheck (pull_request) Successful in 5m47s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 7m33s
Details
Tests and linters / Staticcheck (pull_request) Successful in 7m37s
Details
Tests and linters / Lint (pull_request) Successful in 8m8s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 10m38s
Details
Tests and linters / Tests with -race (pull_request) Successful in 11m22s
Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-17 13:24:04 +03:00
Airat Arifullin
47286ebf32
[ #805 ] pilorama: Fix TreeDrop
...
* If treeID is empty then deleting buckets for cursor may get
invalidated. So, buckets should be gathered before deleting.
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-11-17 10:21:35 +00:00
Dmitrii Stepanov
29fe8c41f3
[ #655 ] storage: Drop `ErrorHandler`
...
DCO action / DCO (pull_request) Successful in 1m42s
Details
Build / Build Components (1.21) (pull_request) Successful in 3m23s
Details
Vulncheck / Vulncheck (pull_request) Successful in 3m25s
Details
Build / Build Components (1.20) (pull_request) Successful in 4m16s
Details
Tests and linters / Staticcheck (pull_request) Successful in 4m24s
Details
Tests and linters / Lint (pull_request) Successful in 5m2s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 5m36s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 6m42s
Details
Tests and linters / Tests with -race (pull_request) Successful in 7m18s
Details
The only one usage was for logging.
Now logging performed by storage anyway.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-16 17:27:38 +03:00
Dmitrii Stepanov
137e987a4e
[ #655 ] storage: Drop `LazyHandler`
...
LazyHandler is implemented and used incorrectly.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-16 17:27:38 +03:00
Dmitrii Stepanov
1b22801eed
[ #788 ] engine: Fix flaky tests
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-09 10:25:46 +03:00
Dmitrii Stepanov
70ab1ebd54
[ #763 ] metrics: Add container_objects_total metric
...
DCO action / DCO (pull_request) Successful in 3m54s
Details
Build / Build Components (1.20) (pull_request) Successful in 4m58s
Details
Build / Build Components (1.21) (pull_request) Successful in 5m16s
Details
Vulncheck / Vulncheck (pull_request) Successful in 9m54s
Details
Tests and linters / Lint (pull_request) Successful in 10m57s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 12m40s
Details
Tests and linters / Staticcheck (pull_request) Successful in 12m34s
Details
Tests and linters / Tests with -race (pull_request) Successful in 12m48s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 13m19s
Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-08 12:30:57 +03:00
Dmitrii Stepanov
9c98fa6152
[ #763 ] metabase: Add container objects counter
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-08 12:30:57 +03:00
Dmitrii Stepanov
226e84d782
[ #684 ] node: Add skipped objects count to evacuation result
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-07 12:17:11 +00:00
Dmitrii Stepanov
c80b46fad3
[ #754 ] blobstor: Estimate compressability
...
DCO action / DCO (pull_request) Successful in 1m59s
Details
Vulncheck / Vulncheck (pull_request) Successful in 3m31s
Details
Build / Build Components (1.20) (pull_request) Successful in 4m37s
Details
Build / Build Components (1.21) (pull_request) Successful in 4m33s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 4m54s
Details
Tests and linters / Staticcheck (pull_request) Successful in 4m49s
Details
Tests and linters / Tests with -race (pull_request) Successful in 5m9s
Details
Tests and linters / Lint (pull_request) Successful in 6m4s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 6m9s
Details
Now it is possible to enable compressability estimation.
If data is likely uncompressable, it should reduce CPU time and memory.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-01 11:24:32 +03:00
Dmitrii Stepanov
79088baa06
[ #772 ] node: Apply gofumpt
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-10-31 17:03:03 +03:00
Evgenii Stratonikov
00aa6d9749
[ #633 ] shard/test: Fix TestCounters()
...
Introduced in 362f24953a
, forgotten to be changed because test
generator didn't provide payload size.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-10-31 12:53:28 +00:00
Evgenii Stratonikov
b8f79f4227
[ #633 ] shard/test: Fix race conditions in TestCounters()
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-10-31 12:53:28 +00:00
Dmitrii Stepanov
869518be0a
[ #728 ] writecache: Fix Badger writecache race.
...
Build / Build Components (1.21) (pull_request) Successful in 1m47s
Details
DCO action / DCO (pull_request) Successful in 2m1s
Details
Vulncheck / Vulncheck (pull_request) Successful in 3m5s
Details
Build / Build Components (1.20) (pull_request) Successful in 3m58s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 5m43s
Details
Tests and linters / Staticcheck (pull_request) Successful in 5m49s
Details
Tests and linters / Lint (pull_request) Successful in 6m20s
Details
Tests and linters / Tests with -race (pull_request) Successful in 6m8s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 3m6s
Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-10-30 18:36:41 +03:00
Dmitrii Stepanov
d4b6ebe7e7
[ #725 ] writecache: Fix metric values
...
DCO action / DCO (pull_request) Successful in 1m33s
Details
Build / Build Components (1.21) (pull_request) Successful in 3m11s
Details
Build / Build Components (1.20) (pull_request) Successful in 3m22s
Details
Vulncheck / Vulncheck (pull_request) Successful in 3m34s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 6m0s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 6m1s
Details
Tests and linters / Staticcheck (pull_request) Successful in 6m4s
Details
Tests and linters / Tests with -race (pull_request) Successful in 3m58s
Details
Tests and linters / Lint (pull_request) Successful in 11m43s
Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-10-27 12:22:29 +03:00
Anton Nikiforov
559ad58ab1
[ #642 ] writecache: Remove usage of close channel in bbolt
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-10-24 15:57:50 +00:00
Anton Nikiforov
c0b86f2d93
[ #642 ] writecache: Remove usage of close channel in badger
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-10-24 15:57:50 +00:00
Dmitrii Stepanov
f2437f7ae9
[ #734 ] shard: Fix Delete method
...
DCO action / DCO (pull_request) Successful in 1m36s
Details
Build / Build Components (1.20) (pull_request) Successful in 3m46s
Details
Vulncheck / Vulncheck (pull_request) Successful in 3m16s
Details
Tests and linters / Staticcheck (pull_request) Successful in 4m42s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 6m27s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 6m35s
Details
Tests and linters / Tests with -race (pull_request) Successful in 6m33s
Details
Build / Build Components (1.21) (pull_request) Successful in 13m7s
Details
Tests and linters / Lint (pull_request) Successful in 19m14s
Details
Due to the flushing data from the writecache to the storage
and simultaneous deletion, a partial deletion situation is possible.
So as a solution, deletion is allowed only when the object is in storage,
because object will be deleted from writecache by flush goroutine.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-10-16 17:00:18 +03:00
Dmitrii Stepanov
f26233b47a
[ #734 ] metabase: Include UpdateStorageID in metrics and traces
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-10-16 17:00:18 +03:00
Dmitrii Stepanov
7e0c5a55de
[ #734 ] writecache: Fix flush
...
Now UpdateStorageID doesn't return error in case of logical error.
If object is in graveyard or GC market, it is still required to
update storage ID.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-10-16 17:00:17 +03:00
Dmitrii Stepanov
d07afd803c
[ #726 ] writecache: Fix small object flush for Badger
...
DCO action / DCO (pull_request) Successful in 2m25s
Details
Vulncheck / Vulncheck (pull_request) Successful in 2m41s
Details
Build / Build Components (1.21) (pull_request) Successful in 3m2s
Details
Build / Build Components (1.20) (pull_request) Successful in 3m17s
Details
Tests and linters / Staticcheck (pull_request) Successful in 4m26s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 6m9s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 18m15s
Details
Tests and linters / Lint (pull_request) Successful in 20m21s
Details
Tests and linters / Tests with -race (pull_request) Successful in 18m2s
Details
Do not marshal object twice.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-10-06 11:32:50 +03:00
Dmitrii Stepanov
997ac7cd8d
[ #726 ] writecache: Fix small object flush for BBolt
...
Do not marshal object twice.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-10-06 11:32:44 +03:00
Alexander Chuprov
c1e4130020
[ #146 ] node: Add trace_id to logs
...
Vulncheck / Vulncheck (pull_request) Successful in 3m7s
Details
DCO action / DCO (pull_request) Successful in 3m36s
Details
Build / Build Components (1.21) (pull_request) Successful in 3m29s
Details
Build / Build Components (1.20) (pull_request) Successful in 3m37s
Details
Tests and linters / Staticcheck (pull_request) Successful in 4m39s
Details
Tests and linters / Lint (pull_request) Successful in 5m2s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 6m27s
Details
Tests and linters / Tests with -race (pull_request) Successful in 6m29s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 9m19s
Details
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2023-09-27 11:05:27 +03:00
Airat Arifullin
3889e829e6
[ #667 ] writecache: Add logs for report error func in tests
...
Signed-off-by: Airat Arifullin a.arifullin@yadro.com
2023-09-14 17:02:54 +00:00
Evgenii Stratonikov
c6af4a3ec8
[ #679 ] engine: Do not increase error counter on meta mismatch
...
DCO action / DCO (pull_request) Successful in 1m43s
Details
Vulncheck / Vulncheck (pull_request) Successful in 2m0s
Details
Tests and linters / Staticcheck (pull_request) Successful in 2m54s
Details
Build / Build Components (1.20) (pull_request) Successful in 2m49s
Details
Build / Build Components (1.21) (pull_request) Successful in 2m44s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 3m43s
Details
Tests and linters / Lint (pull_request) Successful in 3m58s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 23m28s
Details
Tests and linters / Tests with -race (pull_request) Successful in 29m0s
Details
It was introduced in 69e1e6ca
to help node determine faulty shards.
However, the situation is possible in a real-life scenario:
1. Object O is evacuated from shard A to B.
2. Shard A is unmounted because of lower-level errors.
3. We now have object in meta on A and in blobstor on B. Technically we
have it in meta on shard B too, but we still got the error if B goes
to a degraded mode.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-14 10:39:18 +03:00
Ekaterina Lebedeva
8a81af5a3b
[ #653 ] Add context parameter to Open functions
...
DCO action / DCO (pull_request) Successful in 1m38s
Details
Build / Build Components (1.20) (pull_request) Successful in 4m22s
Details
Build / Build Components (1.21) (pull_request) Successful in 4m25s
Details
Vulncheck / Vulncheck (pull_request) Successful in 4m56s
Details
Tests and linters / Lint (pull_request) Successful in 6m1s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 7m43s
Details
Tests and linters / Staticcheck (pull_request) Successful in 8m1s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 8m14s
Details
Tests and linters / Tests with -race (pull_request) Successful in 8m32s
Details
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2023-09-07 18:03:29 +03:00
Evgenii Stratonikov
a716db99db
[ #668 ] shard/test: Do not alter rootPath option
...
Supposedly, this was added to allow creating 2 different shards without
subtest. Now we use t.TempDir() everywhere, so this should not be a
problem.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-07 07:39:39 +00:00
Evgenii Stratonikov
36759f8434
[ #668 ] shard/test: Properly check event processing
...
See https://git.frostfs.info/TrueCloudLab/frostfs-node/actions/runs/1594/jobs/2
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-07 07:39:39 +00:00
Evgenii Stratonikov
39879fa868
[ #668 ] shard/test: Add dontRelease options
...
Most of the time we would like to close shard with minor exceptions.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-07 07:39:39 +00:00
Evgenii Stratonikov
c661ba1312
[ #668 ] shard/test: Use sane defaults in the test constructor
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-07 07:39:39 +00:00
Evgenii Stratonikov
268adb79cb
[ #668 ] shard/test: Simplify shard construction
...
newCustomShard() has many parameters but only the first is obligatory.
`enableWriteCache` is left as-is, because it directly affects the
functionality.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-07 07:39:39 +00:00
Evgenii Stratonikov
429f941cda
[ #668 ] shard/test: Release shard in t.Cleanup()
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-07 07:39:39 +00:00
Evgenii Stratonikov
382eb8a485
[ #668 ] shard/test: Disable GC where it is not needed
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-07 07:39:39 +00:00
Evgenii Stratonikov
42696016de
[ #668 ] shard: Close stopChannel in GC
...
It is done once, but now we could read it from multiple places.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-07 07:39:39 +00:00
Evgenii Stratonikov
bdecfbc1be
[ #668 ] shard/test: Move tests to the main package
...
Semantic patch (also, duplicate definitions are removed):
```
@@
var e identifier
@@
-import "git.frostfs.info/TrueCloudLab/frostfs-node/pkg/local_object_storage/shard"
-shard.e
+e
```
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-07 07:39:39 +00:00
Evgenii Stratonikov
aa23c6a83a
[ #668 ] shard/test: Remove subtest from TestCounters
...
Otherwise, individual tests cannot be run.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-07 07:39:39 +00:00
Evgenii Stratonikov
da8f384324
[ #668 ] shard/test: Fix typo in `existence`
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-07 07:39:39 +00:00
Dmitrii Stepanov
7456c8556a
[ #536 ] blobovnicza: Add blobovniczatree DB cache
...
DCO action / DCO (pull_request) Successful in 1m33s
Details
Vulncheck / Vulncheck (pull_request) Successful in 2m43s
Details
Build / Build Components (1.20) (pull_request) Successful in 3m40s
Details
Tests and linters / Staticcheck (pull_request) Successful in 3m59s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 5m44s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 6m3s
Details
Tests and linters / Tests with -race (pull_request) Successful in 6m28s
Details
Build / Build Components (1.21) (pull_request) Successful in 12m34s
Details
Tests and linters / Lint (pull_request) Successful in 12m51s
Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-09-01 13:53:11 +03:00
Dmitrii Stepanov
c672f59ab8
[ #536 ] blobovnicza: Drop cache
...
Each blobovnicza instance is opened
while is in use.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-09-01 13:51:26 +03:00
Evgenii Stratonikov
b9b86d2ec8
[ #666 ] shard/test: Fix data race in metrics tests
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-31 08:39:42 +00:00
Anton Nikiforov
4dff9555f1
[ #568 ] writecache: Improve flushing scheme for badger
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-08-30 17:22:28 +00:00
Alejandro Lopez
1daef2ceeb
[ #660 ] writecache: Fix remaining addr2key uses
...
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-08-30 17:12:33 +00:00
Evgenii Stratonikov
9072772a09
[ #649 ] shard/test: Increase GC remover interval
...
This was set in #348 to speed up tests.
It seems 100ms doesn't increase overall test time,
but it reduces the amount of logs by 100x factor.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-25 10:10:25 +00:00
Evgenii Stratonikov
c4db8e7690
[ #637 ] shard/test: Fix data race
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-25 10:10:25 +00:00
Evgenii Stratonikov
96e690883f
[ #638 ] Unify test loggers
...
In some places we have debug=false, in others debug=true.
Let's be consistent.
Semantic patch:
```
@@
@@
-test.NewLogger(..., false)
+test.NewLogger(..., true)
```
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-23 11:21:05 +00:00
Evgenii Stratonikov
322c1dc273
[ #638 ] Use test.NewLogger() in tests
...
Semantic patch (restricted to **/*_test.go):
```
@@
@@
+import "git.frostfs.info/TrueCloudLab/frostfs-node/pkg/util/logger/test"
-import "go.uber.org/zap"
-import "git.frostfs.info/TrueCloudLab/frostfs-node/pkg/util/logger"
-&logger.Logger{Logger: zap.L()}
+test.NewLogger(t, false)
```
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-23 11:21:05 +00:00
Evgenii Stratonikov
82cc453be9
[#xx] shard: Fix data race in metrics tests
...
Protect test metric store fields with a mutex. Probably, not every field
should be protected, but better safe than sorry.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-23 10:26:12 +00:00
Alejandro Lopez
345a1a69a2
[ #635 ] Use internal key type when deleting from badger wc
...
DCO action / DCO (pull_request) Successful in 2m48s
Details
Build / Build Components (1.21) (pull_request) Successful in 4m59s
Details
Build / Build Components (1.20) (pull_request) Successful in 5m5s
Details
Vulncheck / Vulncheck (pull_request) Successful in 4m55s
Details
Tests and linters / Staticcheck (pull_request) Successful in 6m51s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 7m21s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 7m53s
Details
Tests and linters / Lint (pull_request) Successful in 8m35s
Details
Tests and linters / Tests with -race (pull_request) Successful in 9m57s
Details
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-08-22 10:53:19 +03:00
Dmitrii Stepanov
354a92ea2c
[ #602 ] blobovnicza: Add leaf width implementation
...
Vulncheck / Vulncheck (pull_request) Successful in 1m14s
Details
Build / Build Components (1.21) (pull_request) Successful in 3m36s
Details
DCO action / DCO (pull_request) Successful in 4m4s
Details
Tests and linters / Staticcheck (pull_request) Successful in 4m5s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 4m49s
Details
Tests and linters / Lint (pull_request) Successful in 5m18s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 5m44s
Details
Tests and linters / Tests with -race (pull_request) Successful in 5m47s
Details
Build / Build Components (1.20) (pull_request) Successful in 10m45s
Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-08-21 10:27:32 +03:00
Alexander Chuprov
4d9a6c07fb
[ #618 ] core: Replace fmt.Sprintf with strconv.FormatUint
...
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2023-08-21 07:14:50 +00:00
Dmitrii Stepanov
f2811f8585
[ #602 ] metrics: Add blobovnicza items counter
...
Vulncheck / Vulncheck (pull_request) Successful in 1m29s
Details
Build / Build Components (1.21) (pull_request) Successful in 3m23s
Details
DCO action / DCO (pull_request) Successful in 3m50s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 4m27s
Details
Tests and linters / Lint (pull_request) Successful in 4m48s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 5m6s
Details
Tests and linters / Staticcheck (pull_request) Successful in 5m8s
Details
Tests and linters / Tests with -race (pull_request) Successful in 5m38s
Details
Build / Build Components (1.20) (pull_request) Successful in 7m46s
Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-08-18 13:01:27 +03:00
Dmitrii Stepanov
c4e1d8eb07
[ #602 ] node: Fix blobovnicza typos
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-08-18 11:14:10 +03:00
Dmitrii Stepanov
10e63537b2
[ #602 ] metrics: Rename blobovnicza size metric
...
`Size` is not size, but open db size.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-08-17 19:17:35 +00:00
Dmitrii Stepanov
809e97626b
[ #602 ] blobovnicza: Fix size counter
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-08-17 19:17:35 +00:00
Dmitrii Stepanov
2e49d7ea7e
[ #602 ] blobovnicza: Init before using
...
Fix blobovnicza size: after restart size metric resets.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-08-17 19:17:35 +00:00
Evgenii Stratonikov
f7042c5a6f
[ #609 ] Replace zaptest.NewLogger() with zap.L()
...
Semantic patch:
```
@@
@@
-import "go.uber.org/zap/zaptest"
+import "go.uber.org/zap"
-zaptest.NewLogger(t)
+zap.L()
```
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-17 16:51:10 +00:00
Evgenii Stratonikov
e604a3d749
[ #607 ] *: Use zap.Stringer() where possible
...
Semantic patch:
```
@@
var f expression
var t expression
var a expression
@@
f(
...,
- zap.String(t, a.String()),
+ zap.Stringer(t, a),
...,
)
```
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-17 16:50:02 +00:00
Evgenii Stratonikov
a8de37c8a2
[ #607 ] *: Remove redundant if on error returns
...
Semantic patch:
```
@@
@@
-if err != nil { return err }
-return nil
+return err
```
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-17 16:50:02 +00:00
Evgenii Stratonikov
5335e7089e
[ #615 ] pilorama: Speedup TestForest_ApplyRandom()
...
Some of our pilorama tests fail on CI.
The reasons are not obvious, but one possible improvement
is using `WithNoSync` option for these. It should have much effect,
because we are writing on the tmpfs, but doesn't hurt anyway.
If I replace `t.TempDir()` with a local directory, test execution time
goes down from 5s (sync) to 0.4s (nosync), which is the same time as
with `t.TempDir()`. Maybe we have some strange CI configuration.
```
panic: test timed out after 10m0s
running tests:
TestForest_ApplyRandom (8m22s)
TestForest_ApplyRandom/bbolt (8m21s)
...
goroutine 170 [syscall]:
syscall.Syscall(0xc000100000?, 0xc00047b758?, 0x6aff9a?, 0xc00041c1b0?)
/opt/hostedtoolcache/go/1.20.7/x64/src/syscall/syscall_linux.go:69 +0x27
syscall.Fdatasync(0x9e35c0?)
/opt/hostedtoolcache/go/1.20.7/x64/src/syscall/zsyscall_linux_amd64.go:418 +0x2a
go.etcd.io/bbolt.fdatasync(0xc000189000?)
```
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-16 19:09:08 +00:00
Dmitrii Stepanov
2efe9cc1be
[ #585 ] writecache: Fix DB counter
...
DCO action / DCO (pull_request) Successful in 2m14s
Details
Vulncheck / Vulncheck (pull_request) Successful in 2m44s
Details
Build / Build Components (1.21) (pull_request) Successful in 2m54s
Details
Build / Build Components (1.20) (pull_request) Successful in 3m5s
Details
Tests and linters / Staticcheck (pull_request) Successful in 3m43s
Details
Tests and linters / Lint (pull_request) Successful in 5m12s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 7m44s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 21m5s
Details
Tests and linters / Tests with -race (pull_request) Failing after 26m26s
Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-08-16 14:47:44 +03:00
Dmitrii Stepanov
58c8722c81
[ #585 ] fstree: Add optional file counter
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-08-16 14:47:44 +03:00
Dmitrii Stepanov
baad49990c
[ #585 ] fstree: Return logical error if object deleted
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-08-16 14:47:44 +03:00
Dmitrii Stepanov
0c52186572
[ #585 ] fstree: Remove unused method
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-08-16 14:47:44 +03:00
Dmitrii Stepanov
eec97d177e
[ #585 ] writecache: Count items periodically
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-08-16 14:47:44 +03:00
Evgenii Stratonikov
d15199c5d8
[ #596 ] engine: Consider context errors as logical
...
Vulncheck / Vulncheck (pull_request) Successful in 2m59s
Details
DCO action / DCO (pull_request) Successful in 2m54s
Details
Build / Build Components (1.20) (pull_request) Successful in 4m2s
Details
Build / Build Components (1.21) (pull_request) Successful in 4m51s
Details
Tests and linters / Staticcheck (pull_request) Successful in 14m8s
Details
Tests and linters / Tests (1.20) (pull_request) Failing after 14m56s
Details
Tests and linters / Lint (pull_request) Successful in 15m27s
Details
Tests and linters / Tests (1.21) (pull_request) Failing after 15m36s
Details
Tests and linters / Tests with -race (pull_request) Failing after 16m18s
Details
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-16 10:39:41 +03:00
Alejandro Lopez
88b6755c5e
[ #598 ] Fix use-after-close bug in badger writecache
...
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-08-15 07:04:06 +00:00
Alejandro Lopez
ae8be495c8
[#xx] Avoid manual management of files in tests
...
Vulncheck / Vulncheck (pull_request) Successful in 2m36s
Details
Build / Build Components (1.21) (pull_request) Successful in 3m33s
Details
DCO action / DCO (pull_request) Successful in 3m33s
Details
Tests and linters / Staticcheck (pull_request) Successful in 4m15s
Details
Tests and linters / Lint (pull_request) Successful in 5m31s
Details
Build / Build Components (1.20) (pull_request) Successful in 7m2s
Details
Tests and linters / Tests with -race (pull_request) Failing after 8m51s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 9m55s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 10m6s
Details
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-08-14 14:01:39 +03:00
Alejandro Lopez
376f03a445
[ #598 ] Hold mode mutex when setting mode
...
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-08-14 07:12:27 +00:00
Alexander Chuprov
21800e9fcc
[ #162 ] core: Move literals to constants
...
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2023-08-11 15:48:42 +03:00
Evgenii Stratonikov
b44a8dd46c
[ #597 ] *: Fix linter warnings
...
Vulncheck / Vulncheck (pull_request) Successful in 6m26s
Details
Tests and linters / Staticcheck (pull_request) Successful in 7m20s
Details
Build / Build Components (1.21) (pull_request) Successful in 8m31s
Details
Build / Build Components (1.20) (pull_request) Successful in 8m43s
Details
Tests and linters / Tests (1.21) (pull_request) Failing after 15m48s
Details
Tests and linters / Tests (1.20) (pull_request) Failing after 16m2s
Details
Tests and linters / Tests with -race (pull_request) Failing after 16m28s
Details
Tests and linters / Lint (pull_request) Successful in 19m8s
Details
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-11 09:55:51 +03:00
Alejandro Lopez
8f994163ee
[ #586 ] Fix writecache benchmarks and refactor hacky NeedsCompression
...
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-08-10 08:05:18 +00:00
Alejandro Lopez
023b90342c
[ #584 ] Disable compression in badger writecache
...
Build / Build Components (1.20) (pull_request) Failing after 3s
Details
Build / Build Components (1.21) (pull_request) Failing after 2s
Details
Tests and linters / Lint (pull_request) Failing after 4s
Details
Tests and linters / Tests (1.20) (pull_request) Failing after 3s
Details
Tests and linters / Tests (1.21) (pull_request) Failing after 3s
Details
Tests and linters / Tests with -race (pull_request) Failing after 3s
Details
Tests and linters / Staticcheck (pull_request) Failing after 3s
Details
Vulncheck / Vulncheck (pull_request) Failing after 3s
Details
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-08-10 10:14:06 +03:00
Evgenii Stratonikov
d641cba2fc
[ #587 ] Do not use math/rand.Read
...
Build / Build Components (1.20) (pull_request) Failing after 3s
Details
Build / Build Components (1.21) (pull_request) Failing after 3s
Details
Tests and linters / Lint (pull_request) Failing after 3s
Details
Tests and linters / Tests (1.20) (pull_request) Failing after 3s
Details
Tests and linters / Tests (1.21) (pull_request) Failing after 3s
Details
Tests and linters / Tests with -race (pull_request) Failing after 5s
Details
Tests and linters / Staticcheck (pull_request) Failing after 8s
Details
Vulncheck / Vulncheck (pull_request) Failing after 2s
Details
Fix staticcheck warnings after go1.20 update.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-09 16:02:44 +03:00
Alejandro Lopez
5b7e4a51b7
[ #481 ] Update frostfs-sdk-go and error pointer receivers
...
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-08-09 10:26:53 +00:00
Alejandro Lopez
ae322e9f73
[ #576 ] Set SyncWrites for badger writecache by default
...
Build / Build Components (1.19) (pull_request) Failing after 3s
Details
Build / Build Components (1.20) (pull_request) Failing after 2s
Details
Tests and linters / Tests (1.19) (pull_request) Failing after 3s
Details
Tests and linters / Tests with -race (pull_request) Failing after 3s
Details
Tests and linters / Lint (pull_request) Failing after 12s
Details
Tests and linters / Staticcheck (pull_request) Failing after 3s
Details
Vulncheck / Vulncheck (pull_request) Successful in 5m46s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 11m37s
Details
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-08-09 11:28:26 +03:00
Alejandro Lopez
7da4306e38
[ #575 ] writecache: Fix log level for badger writecache
...
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-08-08 08:16:36 +00:00
Dmitrii Stepanov
0e697266c3
[ #563 ] writecache: Fix metrics and bolt delete
...
Estimate cache size after delete objects to update metric.
Update counters on small object deletion.
Do not count bbolt DB file as FSTree object.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-08-07 12:53:28 +00:00
Alejandro Lopez
1a0cb0f34a
[ #421 ] Try using badger for the write-cache
...
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-08-07 08:16:57 +00:00
Alejandro Lopez
65c72f3e0b
[ #559 ] Remove manual path handling in fstree tests
...
Build / Build Components (1.19) (pull_request) Successful in 2m8s
Details
Build / Build Components (1.20) (pull_request) Successful in 1m54s
Details
Tests and linters / Lint (pull_request) Successful in 2m48s
Details
Tests and linters / Tests (1.19) (pull_request) Successful in 1m53s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 2m4s
Details
Tests and linters / Tests with -race (pull_request) Successful in 5m21s
Details
Tests and linters / Staticcheck (pull_request) Successful in 2m16s
Details
Vulncheck / Vulncheck (pull_request) Failing after 16m3s
Details
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-08-03 10:03:41 +03:00
Evgenii Stratonikov
35dc64bd7b
[ #547 ] metabase: Fix datarace in tests
...
Quite an old one bf9e938a3b
.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-07-31 15:35:15 +00:00
Evgenii Stratonikov
ec8b4fdc48
[ #541 ] writecache/test: Close writecache on exit
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-07-29 10:57:25 +00:00
Evgenii Stratonikov
ad5f527bd3
[ #541 ] writecache/test: Remove initWC()
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-07-29 10:57:25 +00:00
Dmitrii Stepanov
af82c2865e
[ #335 ] treesvc: Fix inmemory unit tests and nil meta items
...
Bolt forest saves empty slice of items. Now inmemory forest
does it the same way.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-20 10:14:10 +03:00
Dmitrii Stepanov
b4e72a2dfd
[ #335 ] treesvc: Sort nodes by Filename in GetSubTree
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-20 10:14:10 +03:00
Dmitrii Stepanov
3e8de14e7d
[ #382 ] evacuate: Fix unit tests
...
ci/woodpecker/push/pre-commit Pipeline was successful
Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-17 10:24:05 +00:00
Alejandro Lopez
11027945d8
[ #479 ] writecache: Fix writecache fstree flush premature ctx cancel
...
Build / Build Components (1.20) (pull_request) Successful in 3m49s
Details
Build / Build Components (1.19) (pull_request) Successful in 4m2s
Details
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 6m13s
Details
Tests and linters / Lint (pull_request) Successful in 11m25s
Details
Tests and linters / Tests with -race (pull_request) Successful in 9m4s
Details
Tests and linters / Staticcheck (pull_request) Successful in 6m0s
Details
Tests and linters / Tests (1.19) (pull_request) Successful in 25m11s
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-07-14 10:25:52 +03:00
Alexander Chuprov
033eaf77e1
[ #496 ] node: Fix linter importas
...
Build / Build Components (1.20) (pull_request) Successful in 3m52s
Details
Build / Build Components (1.19) (pull_request) Successful in 4m1s
Details
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
Tests and linters / Tests with -race (pull_request) Successful in 5m36s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 5m55s
Details
Tests and linters / Lint (pull_request) Successful in 14m40s
Details
Tests and linters / Tests (1.19) (pull_request) Successful in 15m29s
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details
Standardize the alias of the
import frostfs-sdk-go/object as objectSDK.
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2023-07-06 15:36:41 +03:00
Evgenii Stratonikov
cab51c8cbe
[ #1 ] metabase: Rename blindlyProcess()
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-28 12:13:20 +00:00
Evgenii Stratonikov
f4c71cea65
[ #1 ] *: Replace outdated FIXME/TODO links
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-28 12:13:20 +00:00
Dmitrii Stepanov
43d263c3d5
[ #428 ] linter: Fix unkeyed assignment
...
ci/woodpecker/push/pre-commit Pipeline was successful
Details
Thanks to gopls.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-26 13:29:39 +00:00
Dmitrii Stepanov
cac4ed93d6
[ #428 ] engine: Add low_mem config parameter
...
Concurrent initialization in case of the metabase resync leads to
high memory consumption and potential OOM.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-26 13:29:39 +00:00
Evgenii Stratonikov
4bf345225c
[ #447 ] pilorama: Use named constant for the key size
...
ci/woodpecker/push/pre-commit Pipeline was successful
Details
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-26 07:42:05 +00:00
Evgenii Stratonikov
b4ce0b0412
[ #447 ] pilorama: Do not undo log for create ops
...
```
goos: linux
goarch: amd64
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
│ old │ new │
│ sec/op │ sec/op vs base │
Create-8 36.48µ ± 11% 30.34µ ± 14% -16.84% (p=0.000 n=10)
│ old │ new │
│ B/op │ B/op vs base │
Create-8 43.01Ki ± 4% 37.78Ki ± 5% -12.15% (p=0.000 n=10)
│ old │ new │
│ allocs/op │ allocs/op vs base │
Create-8 166.0 ± 3% 146.0 ± 3% -12.05% (p=0.000 n=10)
```
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-26 07:42:05 +00:00
Evgenii Stratonikov
dd3874eff1
[ #447 ] pilorama: Add benchmark for create ops
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-26 07:42:05 +00:00
Dmitrii Stepanov
4d48377cec
[ #459 ] blobovniczatree: Fix get error
...
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-21 16:44:53 +03:00
Dmitrii Stepanov
b5d9f4a285
[ #373 ] metrics: Add pilorama metrics
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-21 15:13:26 +03:00
Dmitrii Stepanov
e89fa110c7
[ #373 ] metrics: Add metabase metrics
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-21 15:13:26 +03:00
Dmitrii Stepanov
af608da952
[ #373 ] metrics: Add blobovnizca metrics
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-21 15:13:26 +03:00
Dmitrii Stepanov
56f320dd85
[ #373 ] metrics: Add blobstor metrics
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-21 15:13:26 +03:00
Dmitrii Stepanov
16a142cd0c
[ #373 ] metrics: Add FSTree metrics
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-21 15:13:26 +03:00
Dmitrii Stepanov
d8ecc69d00
[ #373 ] local storage: Pass parent ID
...
This is required to add shard ID as metric label.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-21 15:13:26 +03:00
Dmitrii Stepanov
d5aaec1107
[ #373 ] pilorama: Add metrics
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-21 15:13:26 +03:00
Dmitrii Stepanov
059e9e88a2
[ #373 ] metabase: Add metrics
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-21 15:13:26 +03:00
Dmitrii Stepanov
f54cc0b607
[ #373 ] blobstor: Add metrics
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-21 15:13:26 +03:00
Dmitrii Stepanov
8318d90ad0
[ #373 ] blobovniczatree: Add metrics
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-21 15:13:26 +03:00
Dmitrii Stepanov
3ae3c8dfdb
[ #373 ] fstree: Add metrics
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-21 15:13:26 +03:00
Dmitrii Stepanov
a8526d45e9
[ #373 ] blobovnizca: Add missed/fix tracing spans
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-21 15:13:26 +03:00
Dmitrii Stepanov
028d4a8058
[ #373 ] blobovnicza: Add metrics
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-21 15:13:26 +03:00
Dmitrii Stepanov
01a0c97760
[ #453 ] engine: Set Disabled mode to deleted shard
...
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-20 12:04:07 +03:00
Evgenii Stratonikov
69df0d21c2
[ #446 ] engine: Move to read-only on blobstor errors
...
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-16 14:53:32 +03:00
Evgenii Stratonikov
fe01781811
[ #446 ] los: Wrap SSD errors in a separate type
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-16 14:53:31 +03:00
Evgenii Stratonikov
20b84f183a
[ #446 ] engine: Simplify logs for shard mode change
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-16 14:51:29 +03:00
Dmitrii Stepanov
4449006862
[ #424 ] metrics: Use mode value as metric value for shard
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-14 18:26:19 +03:00
Dmitrii Stepanov
1b364d8cf4
[ #424 ] metrics: Refactor engine metrics
...
Use histogram vector to measure request duration.
Fix naming like in Prometheus best practice.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-14 14:53:32 +03:00
Dmitrii Stepanov
85deb12f4d
[ #424 ] writecache: Drop metrics when close
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-14 14:52:40 +03:00
Dmitrii Stepanov
07f155ac77
[ #424 ] metrics: Use labels for writecache methods and operations
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-14 14:52:40 +03:00
Dmitrii Stepanov
2541d319de
[ #266 ] pilorama: Allow to get current tree height
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-13 10:00:45 +00:00
Evgenii Stratonikov
41ab4d070e
[ #423 ] *: Use hrw.StringHash() where possible
...
ci/woodpecker/push/pre-commit Pipeline was successful
Details
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-13 07:18:25 +00:00
Anton Nikiforov
263c6fdc50
[ #372 ] node: Add metrics for the error counter in the engine
...
ci/woodpecker/push/pre-commit Pipeline was successful
Details
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-06-07 13:04:47 +00:00
Dmitrii Stepanov
74578052f9
[ #412 ] node: Replace tracing package
...
Use observability module.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-01 13:23:11 +00:00
Dmitrii Stepanov
dbf41391b5
[ #401 ] engine: Extend evacuation logs
...
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details
Add operation-tag to logger.
Log evacuation results.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-05-31 13:24:30 +03:00
Dmitrii Stepanov
3220c4df9f
[ #376 ] metrics: Add GC metrics
...
ci/woodpecker/push/pre-commit Pipeline was successful
Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-05-31 10:22:12 +00:00
Alejandro Lopez
faca861451
[ #411 ] Remove unnecessary pointers for sync objects
...
ci/woodpecker/push/pre-commit Pipeline was successful
Details
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-05-31 10:19:14 +00:00
Anton Nikiforov
8dcd06c587
[ #394 ] node: Use `Context` in `Blobovniczas.Iterate()`
...
ci/woodpecker/push/pre-commit Pipeline was successful
Details
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-05-31 10:09:18 +00:00
Anton Nikiforov
365a7ca0f4
[ #366 ] node: Stop GC once termination signal received
...
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-05-29 09:35:08 +03:00
Anton Nikiforov
802168c0c6
[ #364 ] node: Stop flushing big object when termination signal received
...
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-05-26 16:46:58 +03:00
Dmitrii Stepanov
271a56c2ab
[ #395 ] metrics: Drop redundant metrics
...
ci/woodpecker/push/pre-commit Pipeline was successful
Details
HistogramVec already has labeled counter.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-05-26 13:39:37 +00:00
Evgenii Stratonikov
f2e5dead7e
[ #398 ] pilorama: Disallow applying same operations
...
ci/woodpecker/push/pre-commit Pipeline was successful
Details
1. In redo() we save the old state.
2. If we do redo() for the same operation twice, the old state will be
overritten with the new one.
3. This in turn affects undo() and subsequent isAncestor() check.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-26 12:14:29 +00:00
Dmitrii Stepanov
20a489bdb5
[ #393 ] gc: Use defer to mark handler done
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-05-26 12:14:02 +00:00
Dmitrii Stepanov
2613351008
[ #387 ] gc: Cancel GC is change mode requested
...
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-05-25 09:38:16 +03:00
Dmitrii Stepanov
2ce43935f9
[ #312 ] metrics: Add writecache metrcis
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-05-24 10:18:39 +00:00
Dmitrii Stepanov
d212d908b5
[ #312 ] wc: Add metrics
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-05-24 10:18:39 +00:00
Dmitrii Stepanov
4503a61997
[ #312 ] wc: Delete unused Iterate method
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-05-24 10:18:39 +00:00
Evgenii Stratonikov
4b768fd115
[ #381 ] *: Move to sync/atomic
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-23 08:18:01 +03:00
Evgenii Stratonikov
ff570847a4
[ #381 ] go.mod: Update bbolt
...
Adopt new `ForEachBucket` function where possible.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-22 11:49:14 +03:00
Evgenii Stratonikov
35c9b6b26d
[ #314 ] writecache: remove objects right after they are flushed
...
ci/woodpecker/push/pre-commit Pipeline was successful
Details
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-19 09:37:05 +00:00
Pavel Karpy
bf79d06f03
[ #314 ] writecache: Do not lose small objects on disk errors
...
Do return error if an object could not been stored on WC's disk.
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2023-05-19 09:37:05 +00:00
Pavel Karpy
9e56592be3
[ #314 ] writecache: Simplify background workers naming
...
Also, drop not used arg.
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2023-05-19 09:37:05 +00:00
Dmitrii Stepanov
e4889e06ba
[ #329 ] node: Make evacuate async
...
Now it's possible to run evacuate shard in async.
Also only one evacuate process can be in progress.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-05-19 08:43:52 +00:00
Dmitrii Stepanov
869fcbf591
[ #332 ] gc: Fix expired complex object deletion
...
ci/woodpecker/push/pre-commit Pipeline was successful
Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-05-16 12:44:57 +00:00
Dmitrii Stepanov
ab07bad33d
[ #332 ] gc: Add complex object unit test
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-05-16 12:44:57 +00:00
Evgenii Stratonikov
c62025c836
[ #321 ] metabase/test: execute tests in parallel
...
ci/woodpecker/push/pre-commit Pipeline was successful
Details
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-12 09:45:03 +00:00
Evgenii Stratonikov
945454f60c
[ #321 ] engine/test: Execute tests in parallel
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-12 09:45:03 +00:00
Evgenii Stratonikov
4578d00619
[ #321 ] shard/test: Execute tests in parallel
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-12 09:45:03 +00:00
Evgenii Stratonikov
d35e4c389f
[ #321 ] shard/test: Parallelize TestWriteCacheObjectLoss
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-12 09:45:03 +00:00