Commit Graph

55 Commits (v0.38.5)

Author SHA1 Message Date
Anton Nikiforov c8ce6e9fe4 [#1072] node, ir: Add new config option `kludge_compatibility_mode`
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-04-04 11:24:37 +03:00
Dmitrii Stepanov 6bf77cabd4 [#1044] ape: Add morph chain cache
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-03-18 18:44:10 +03:00
Dmitrii Stepanov d433b49265 [#973] node: Resolve perfsprint linter
DCO action / DCO (pull_request) Successful in 2m40s Details
Vulncheck / Vulncheck (pull_request) Successful in 3m41s Details
Build / Build Components (1.20) (pull_request) Successful in 4m27s Details
Build / Build Components (1.21) (pull_request) Successful in 5m6s Details
Tests and linters / Staticcheck (pull_request) Successful in 6m16s Details
Tests and linters / gopls check (pull_request) Successful in 6m23s Details
Tests and linters / Lint (pull_request) Successful in 6m48s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 9m4s Details
Tests and linters / Tests with -race (pull_request) Successful in 9m9s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 9m23s Details
`fmt.Errorf can be replaced with errors.New` and `fmt.Sprintf can be replaced with string addition`

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-03-11 17:55:50 +03:00
Dmitrii Stepanov 702351a5d1 [#983] blobstor: Allow to specify wait before drop time
DCO action / DCO (pull_request) Successful in 1m11s Details
Build / Build Components (1.21) (pull_request) Successful in 3m29s Details
Vulncheck / Vulncheck (pull_request) Successful in 2m53s Details
Build / Build Components (1.20) (pull_request) Successful in 4m8s Details
Tests and linters / Staticcheck (pull_request) Successful in 4m19s Details
Tests and linters / Lint (pull_request) Successful in 5m50s Details
Tests and linters / gopls check (pull_request) Successful in 5m40s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 7m46s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 8m6s Details
Tests and linters / Tests with -race (pull_request) Successful in 8m50s Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-03-05 09:23:57 +03:00
Anton Nikiforov ae5bb87e70 Revert "[#866] Use TTL for blobovnicza tree cache"
DCO action / DCO (pull_request) Successful in 2m7s Details
Vulncheck / Vulncheck (pull_request) Successful in 3m7s Details
Tests and linters / Staticcheck (pull_request) Successful in 5m12s Details
Tests and linters / Lint (pull_request) Successful in 6m23s Details
Tests and linters / Tests with -race (pull_request) Successful in 7m11s Details
Tests and linters / gopls check (pull_request) Successful in 7m27s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 8m5s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 8m22s Details
Build / Build Components (1.21) (pull_request) Successful in 1m41s Details
Build / Build Components (1.20) (pull_request) Successful in 1m56s Details
This reverts commit d9cbb16bd3.

Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-03-01 19:29:33 +03:00
Dmitrii Stepanov d6534fd755 [#1016] frostfs-node: Fix gopls issues
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-03-01 12:13:43 +03:00
Dmitrii Stepanov 962e5a9c19 [#736] logger: Add journald support
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-12 06:17:22 +00:00
Airat Arifullin 5be2af881a [#934] container: Make container APE middleware read namespaces
* Those methods that can access already existing containers and thus
  can get container properties should read namespace from Zone
  property. If Zone is not set, take a namespace for root.
* Otherwise, define namespaces by owner ID via frostfs-id contract.
* Improve unit-tests, consider more cases.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-02-01 17:38:24 +00:00
Dmitrii Stepanov 8180a0664f [#887] node: Drop badger writecache implementation
Vulncheck / Vulncheck (pull_request) Successful in 6m1s Details
Build / Build Components (1.21) (pull_request) Successful in 7m37s Details
Build / Build Components (1.20) (pull_request) Successful in 7m52s Details
Tests and linters / Staticcheck (pull_request) Successful in 8m56s Details
Tests and linters / Lint (pull_request) Successful in 9m26s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 15m5s Details
Tests and linters / Tests with -race (pull_request) Successful in 15m7s Details
DCO action / DCO (pull_request) Successful in 1m1s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 4m1s Details
Badger implementation isn't tested and works not well,
but requires human resources to maintain.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-22 13:00:54 +03:00
Anton Nikiforov d9cbb16bd3 [#866] Use TTL for blobovnicza tree cache
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-12-19 16:36:28 +00:00
Dmitrii Stepanov f4877e7b42 [#835] grpc: Try to reconnect if endpoint listen failed
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-14 16:38:59 +03:00
Airat Arifullin 0f45e3d344 [#804] ape: Implement boltdb storage for local overrides
DCO action / DCO (pull_request) Successful in 2m10s Details
Vulncheck / Vulncheck (pull_request) Successful in 3m26s Details
Build / Build Components (1.20) (pull_request) Successful in 5m41s Details
Build / Build Components (1.21) (pull_request) Successful in 5m44s Details
Tests and linters / Staticcheck (pull_request) Successful in 7m10s Details
Tests and linters / Lint (pull_request) Successful in 8m14s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 14m24s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 14m41s Details
Tests and linters / Tests with -race (pull_request) Successful in 14m38s Details
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-12-07 19:08:41 +03:00
Dmitrii Stepanov db49ad16cc [#826] blobovniczatree: Do not create DB's on init
DCO action / DCO (pull_request) Successful in 4m10s Details
Build / Build Components (1.21) (pull_request) Successful in 7m30s Details
Vulncheck / Vulncheck (pull_request) Successful in 7m5s Details
Tests and linters / Staticcheck (pull_request) Successful in 10m6s Details
Tests and linters / Lint (pull_request) Successful in 10m26s Details
Build / Build Components (1.20) (pull_request) Successful in 12m39s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 16m55s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 17m13s Details
Tests and linters / Tests with -race (pull_request) Successful in 17m14s Details
Blobovniczas will be created on write requests.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-07 15:37:33 +03:00
Dmitrii Stepanov f1c7905263 [#661] blobovniczatree: Make Rebuild concurrent
Different DBs can be rebuild concurrently.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-07 15:37:33 +03:00
Dmitrii Stepanov da4fee2d0b [#698] blobovniczatree: Init blobovniczas concurrently
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-07 15:37:33 +03:00
Dmitrii Stepanov 07390ad4e3 [#715] node: Unify config parameter names
DCO action / DCO (pull_request) Successful in 2m54s Details
Vulncheck / Vulncheck (pull_request) Successful in 3m11s Details
Tests and linters / Staticcheck (pull_request) Successful in 3m55s Details
Build / Build Components (1.21) (pull_request) Successful in 3m51s Details
Build / Build Components (1.20) (pull_request) Successful in 4m6s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 5m10s Details
Tests and linters / Lint (pull_request) Successful in 5m24s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 6m56s Details
Tests and linters / Tests with -race (pull_request) Successful in 7m46s Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-22 17:13:50 +03:00
Alexander Chuprov 364f835b7e [#740] logs: Add Loki
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2023-11-14 19:01:05 +00:00
Dmitrii Stepanov c80b46fad3 [#754] blobstor: Estimate compressability
DCO action / DCO (pull_request) Successful in 1m59s Details
Vulncheck / Vulncheck (pull_request) Successful in 3m31s Details
Build / Build Components (1.20) (pull_request) Successful in 4m37s Details
Build / Build Components (1.21) (pull_request) Successful in 4m33s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 4m54s Details
Tests and linters / Staticcheck (pull_request) Successful in 4m49s Details
Tests and linters / Tests with -race (pull_request) Successful in 5m9s Details
Tests and linters / Lint (pull_request) Successful in 6m4s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 6m9s Details
Now it is possible to enable compressability estimation.
If data is likely uncompressable, it should reduce CPU time and memory.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-01 11:24:32 +03:00
Dmitrii Stepanov 79088baa06 [#772] node: Apply gofumpt
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-10-31 17:03:03 +03:00
Dmitrii Stepanov ae81d6660a [#529] objectcore: Fix object content validation
There are old objects where the owner of the object
may not match the one who issued the token.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-08-29 10:33:06 +03:00
Dmitrii Stepanov d3904ec599 [#602] config: Add blobovnicza leaf width parameter
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-08-21 10:27:32 +03:00
Alejandro Lopez 1a0cb0f34a [#421] Try using badger for the write-cache
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-08-07 08:16:57 +00:00
Anton Nikiforov ec8a631d31 [#542] Update test for `SizeInBytesSafe` function
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-08-01 13:38:00 +00:00
Anton Nikiforov 9ca63ac8c3 [#542] Fix bug in `SizeInBytesSafe` function
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-08-01 13:38:00 +00:00
Dmitrii Stepanov 32c77f3a23 [#537] node: Add runtime.memory_limit config parameter
Build / Build Components (1.20) (pull_request) Successful in 4m6s Details
Build / Build Components (1.19) (pull_request) Successful in 4m18s Details
ci/woodpecker/pr/pre-commit Pipeline failed Details
Tests and linters / Lint (pull_request) Failing after 2m10s Details
Tests and linters / Tests with -race (pull_request) Has started running Details
Tests and linters / Tests (1.19) (pull_request) Successful in 3m58s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 3m44s Details
Vulncheck / Vulncheck (pull_request) Failing after 3m51s Details
Tests and linters / Staticcheck (pull_request) Successful in 8m3s Details
ci/woodpecker/push/pre-commit Pipeline was successful Details
This parameter allows to set soft memory limit for Go GC.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-25 17:27:09 +03:00
Dmitrii Stepanov d8e37a827f [#497] config: Add examples and unit tests
ci/woodpecker/push/pre-commit Pipeline was successful Details
Add examples and unit tests for tree.authorized_keys section.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-17 08:23:28 +00:00
Evgenii Stratonikov 73a71a71b0 [#1] node: Use a proper validation of a substorage type
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-28 12:13:20 +00:00
Dmitrii Stepanov cac4ed93d6 [#428] engine: Add low_mem config parameter
Concurrent initialization in case of the metabase resync leads to
high memory consumption and potential OOM.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-26 13:29:39 +00:00
Anton Nikiforov 71889234b7 [#449] tree: Allow reading requests signed by keys from allow list
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-06-23 14:44:19 +03:00
Dmitrii Stepanov 74578052f9 [#412] node: Replace tracing package
Use observability module.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-01 13:23:11 +00:00
Airat Arifullin a3e30062df [#325] node: Introduce unsafe_disable param to disable policer
ci/woodpecker/push/pre-commit Pipeline was successful Details
Signed-off-by: Airat Arifullin a.arifullin@yadro.com
2023-05-31 10:09:05 +00:00
Pavel Karpy 14c35d776e [#39] node: Add optional profilers
Include settings for block and mutex profilers.
They are disabled by default, as in Go runtime itself.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-12 15:49:04 +03:00
Anton Nikiforov ef222e2487 [#125] cmd: Refactor `internal/common/viper`
Add `opts.WithViper`, set opts struct as private.

Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-04-26 13:53:51 +03:00
Anton Nikiforov e61aec4a7d [#125] node: Remove unused config.Prm
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-04-26 13:53:51 +03:00
Anton Nikiforov eb7be82e87 [#125] node: Avoid panic when reading config
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-04-26 13:53:51 +03:00
Anton Nikiforov d390f093e0 [#125] node: Move viper creation to `internal/common/config`
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-04-26 13:53:51 +03:00
Anton Nikiforov b2123bfd1a [#125] node: Add new option `WithEnvPrefix`
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-04-26 13:53:51 +03:00
Anton Nikiforov ee01275d25 [#125] node: Remove redundant `env` from `config/internal`
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-04-26 13:53:51 +03:00
Dmitrii Stepanov d757d881d0 [#249] node: Drop subnet from config
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-17 09:46:02 +03:00
Pavel Karpy 960e3c219e [#247] config, doc: Drop reputation references
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2023-04-14 14:44:14 +03:00
Anton Nikiforov 995db117d0 [#238] node: Read cfg from dir even if cfg file not set
ci/woodpecker/push/pre-commit Pipeline was successful Details
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-04-14 10:19:10 +00:00
Dmitrii Stepanov 5af9f58469 [#135] tracing: Add tracing to node gRPC endpoints
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-12 06:52:00 +00:00
Airat Arifullin 221203beeb [#180] node: Refactor panics in unit test
ci/woodpecker/push/pre-commit Pipeline was successful Details
* Replace panics in unit tests by require.NoError and t.Fatalf

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-03-29 12:39:07 +03:00
Dmitrii Stepanov 5059dcc19d [#145] shard-gc: Delete expired objects after locks
GC deletes expired locks and objects sequentially. Expired locks and
objects are now being deleted concurrently in batches. Added a config
parameter that controls the number of concurrent workers and batch size.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-03-21 11:31:08 +03:00
Alexey Vanin 20de74a505 Rename package name
Due to source code relocation from GitHub.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-03-07 16:38:26 +03:00
Anton Nikiforov e9f3c24229 [#65] Use `strings.Cut` instead of `strings.Split*` where possible
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-02-28 13:39:14 +03:00
Alejandro Lopez 73bb590cb1 [#64] node: Use pool_size_local and separate pool for local puts
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-02-22 13:43:19 +03:00
Alejandro Lopez cb5468abb8 [#66] node: Replace interface{} with any
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-02-21 16:47:07 +03:00
Denis Kirillov 87e69b9349 [#44] node: Support multiple configs
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-02-21 10:00:28 +03:00
Evgenii Stratonikov 351fdd9fa2 [#2246] node: Allow to configure tombsone lifetime
Currently, DELETE service sets tombstone expiration epoch to
`current epoch + 5`. This works less than ideal in private networks
where an epoch can be e.g. 10 minutes. In this case, after a node is
unavailable for more than 1 hour, already deleted objects have a chance
to reappear.

After this commit tombstone lifetime can be configured.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-02-20 13:53:27 +03:00