Commit graph

1543 commits

Author SHA1 Message Date
ef07c1a3c9 [#552] Add sysd notifications to storage service
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2023-12-13 17:51:41 +03:00
eca7ac9f0d [#552] Add sdnotify package
To avoid using third-party dependencies.

Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2023-12-13 17:49:26 +03:00
9b2dce5763 [#552] Add status notification to systemd
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2023-12-13 15:02:39 +03:00
05f8f49289 [#552] gofumpt changes
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2023-12-13 15:02:25 +03:00
11add38e87 [#857] golangci: Add protogetter linter
All checks were successful
DCO action / DCO (pull_request) Successful in 1m38s
Vulncheck / Vulncheck (pull_request) Successful in 3m2s
Build / Build Components (1.21) (pull_request) Successful in 3m51s
Build / Build Components (1.20) (pull_request) Successful in 4m3s
Tests and linters / Staticcheck (pull_request) Successful in 5m17s
Tests and linters / Lint (pull_request) Successful in 6m11s
Tests and linters / Tests (1.20) (pull_request) Successful in 11m7s
Tests and linters / Tests (1.21) (pull_request) Successful in 11m32s
Tests and linters / Tests with -race (pull_request) Successful in 12m22s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-12 16:27:02 +03:00
d2746a7d67 [#857] Makefile: Update linter version
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-12 16:27:02 +03:00
0f45e3d344 [#804] ape: Implement boltdb storage for local overrides
All checks were successful
DCO action / DCO (pull_request) Successful in 2m10s
Vulncheck / Vulncheck (pull_request) Successful in 3m26s
Build / Build Components (1.20) (pull_request) Successful in 5m41s
Build / Build Components (1.21) (pull_request) Successful in 5m44s
Tests and linters / Staticcheck (pull_request) Successful in 7m10s
Tests and linters / Lint (pull_request) Successful in 8m14s
Tests and linters / Tests (1.21) (pull_request) Successful in 14m24s
Tests and linters / Tests (1.20) (pull_request) Successful in 14m41s
Tests and linters / Tests with -race (pull_request) Successful in 14m38s
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-12-07 19:08:41 +03:00
39060382a1 [#842] control: Recieve target in gRPC methods for APE managing
* Introduce Target type and pass it to all gRPC methods
  for APE chain managing instead CID.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-12-07 14:21:55 +00:00
db49ad16cc [#826] blobovniczatree: Do not create DB's on init
All checks were successful
DCO action / DCO (pull_request) Successful in 4m10s
Build / Build Components (1.21) (pull_request) Successful in 7m30s
Vulncheck / Vulncheck (pull_request) Successful in 7m5s
Tests and linters / Staticcheck (pull_request) Successful in 10m6s
Tests and linters / Lint (pull_request) Successful in 10m26s
Build / Build Components (1.20) (pull_request) Successful in 12m39s
Tests and linters / Tests (1.20) (pull_request) Successful in 16m55s
Tests and linters / Tests (1.21) (pull_request) Successful in 17m13s
Tests and linters / Tests with -race (pull_request) Successful in 17m14s
Blobovniczas will be created on write requests.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-07 15:37:33 +03:00
8bbfb2df43 [#661] blobovniczatree: Pass object size limit from config
If actual small object size value lower than default
object size limit, then unnecessary buckets created.
If actual small object size value greated than default
object size limit, then error happens.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-07 15:37:33 +03:00
f1c7905263 [#661] blobovniczatree: Make Rebuild concurrent
Different DBs can be rebuild concurrently.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-07 15:37:33 +03:00
da4fee2d0b [#698] blobovniczatree: Init blobovniczas concurrently
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-07 15:37:33 +03:00
484eb59893 [#661] blobovniczatree: Use .db extension for db files
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-07 15:37:32 +03:00
b892feeaf6 [#845] adm: Relax notary-enabled check
Starting from v0.104.0 `NativeActivations` config field is no longer
present and Notary activation height is always 0.

https://github.com/nspcc-dev/neo-go/pull/3212/
TrueCloudLab/frostfs-dev-env#59

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-12-06 11:08:04 +00:00
f1db468d48 [#840] adm: Update FrostFS ID deploy arguments
All checks were successful
Vulncheck / Vulncheck (pull_request) Successful in 1m28s
DCO action / DCO (pull_request) Successful in 3m9s
Build / Build Components (1.21) (pull_request) Successful in 4m15s
Build / Build Components (1.20) (pull_request) Successful in 4m35s
Tests and linters / Staticcheck (pull_request) Successful in 4m58s
Tests and linters / Tests (1.20) (pull_request) Successful in 6m48s
Tests and linters / Tests (1.21) (pull_request) Successful in 7m6s
Tests and linters / Lint (pull_request) Successful in 7m25s
Tests and linters / Tests with -race (pull_request) Successful in 7m9s
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-12-04 17:39:41 +03:00
a982c3df18 [#824] cli: Support passing chain ID in add-rule command
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-11-30 13:13:46 +00:00
7f6852bbd2 [#639] node: Refactor TTL cache
Migrate from internal to external TTL implementation

Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2023-11-30 12:54:51 +00:00
306f12e6c5 [#828] adm: Fix policy contract deploy
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-11-29 06:23:56 +00:00
5521737f0b [#808] cli: Use EnableTraverseRunHooks in cobra
Adopt EnableTraverseRunHooks to get rid of tracing boilerplate in
multiple commands. Now adding `--trace` flag is sufficient for a command
to support tracing. Finally, it looks how it _should_.

Refs #406

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-11-27 09:58:19 +00:00
e81a58b8da [#808] cli: Use MarkFlagsOneRequired after cobra update
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-11-27 09:58:19 +00:00
c516c7c5f4 [#821] node: Pass user.ID by value
All checks were successful
DCO action / DCO (pull_request) Successful in 3m45s
Build / Build Components (1.21) (pull_request) Successful in 5m18s
Build / Build Components (1.20) (pull_request) Successful in 5m28s
Tests and linters / Tests (1.20) (pull_request) Successful in 7m30s
Tests and linters / Tests (1.21) (pull_request) Successful in 7m42s
Tests and linters / Lint (pull_request) Successful in 8m25s
Vulncheck / Vulncheck (pull_request) Successful in 9m22s
Tests and linters / Staticcheck (pull_request) Successful in 10m57s
Tests and linters / Tests with -race (pull_request) Successful in 16m53s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-23 10:21:07 +03:00
07390ad4e3 [#715] node: Unify config parameter names
All checks were successful
DCO action / DCO (pull_request) Successful in 2m54s
Vulncheck / Vulncheck (pull_request) Successful in 3m11s
Tests and linters / Staticcheck (pull_request) Successful in 3m55s
Build / Build Components (1.21) (pull_request) Successful in 3m51s
Build / Build Components (1.20) (pull_request) Successful in 4m6s
Tests and linters / Tests (1.21) (pull_request) Successful in 5m10s
Tests and linters / Lint (pull_request) Successful in 5m24s
Tests and linters / Tests (1.20) (pull_request) Successful in 6m56s
Tests and linters / Tests with -race (pull_request) Successful in 7m46s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-22 17:13:50 +03:00
02454df14a [#809] client: Refactor PrmInit, PrmDial usage
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-11-17 13:37:03 +00:00
5cfb758e4e [#806] morph: Remove container list cache
All checks were successful
DCO action / DCO (pull_request) Successful in 2m48s
Vulncheck / Vulncheck (pull_request) Successful in 3m37s
Tests and linters / Staticcheck (pull_request) Successful in 4m0s
Build / Build Components (1.21) (pull_request) Successful in 4m24s
Build / Build Components (1.20) (pull_request) Successful in 4m48s
Tests and linters / Tests (1.20) (pull_request) Successful in 5m26s
Tests and linters / Tests (1.21) (pull_request) Successful in 5m22s
Tests and linters / Lint (pull_request) Successful in 5m37s
Tests and linters / Tests with -race (pull_request) Successful in 8m7s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-17 10:40:08 +03:00
29fe8c41f3 [#655] storage: Drop ErrorHandler
All checks were successful
DCO action / DCO (pull_request) Successful in 1m42s
Build / Build Components (1.21) (pull_request) Successful in 3m23s
Vulncheck / Vulncheck (pull_request) Successful in 3m25s
Build / Build Components (1.20) (pull_request) Successful in 4m16s
Tests and linters / Staticcheck (pull_request) Successful in 4m24s
Tests and linters / Lint (pull_request) Successful in 5m2s
Tests and linters / Tests (1.20) (pull_request) Successful in 5m36s
Tests and linters / Tests (1.21) (pull_request) Successful in 6m42s
Tests and linters / Tests with -race (pull_request) Successful in 7m18s
The only one usage was for logging.
Now logging performed by storage anyway.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-16 17:27:38 +03:00
4d5be5ccb5 [#811] ape: Update policy-engine module version and rebase
All checks were successful
DCO action / DCO (pull_request) Successful in 4m23s
Vulncheck / Vulncheck (pull_request) Successful in 5m31s
Build / Build Components (1.21) (pull_request) Successful in 7m33s
Build / Build Components (1.20) (pull_request) Successful in 7m40s
Tests and linters / Staticcheck (pull_request) Successful in 8m22s
Tests and linters / Lint (pull_request) Successful in 9m23s
Tests and linters / Tests with -race (pull_request) Successful in 11m20s
Tests and linters / Tests (1.21) (pull_request) Successful in 11m32s
Tests and linters / Tests (1.20) (pull_request) Successful in 11m41s
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-11-16 11:31:37 +03:00
fd9128d051 [#800] node: eACL -> APE converter
All checks were successful
DCO action / DCO (pull_request) Successful in 3m43s
Vulncheck / Vulncheck (pull_request) Successful in 6m9s
Build / Build Components (1.21) (pull_request) Successful in 6m53s
Build / Build Components (1.20) (pull_request) Successful in 7m18s
Tests and linters / Staticcheck (pull_request) Successful in 7m36s
Tests and linters / Tests (1.21) (pull_request) Successful in 8m1s
Tests and linters / Lint (pull_request) Successful in 8m15s
Tests and linters / Tests (1.20) (pull_request) Successful in 9m59s
Tests and linters / Tests with -race (pull_request) Successful in 10m3s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-15 11:55:55 +03:00
364f835b7e [#740] logs: Add Loki
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2023-11-14 19:01:05 +00:00
c1ec6e33b4 [#793] adm: Always use committee as FrostFS ID owner
Committee should be able to authorize everything, there are no other
usecases for the frostfs-adm currently. Also, it somewhat eases
configuration, because committee hash depends on the protocol
configuration.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-11-14 19:00:32 +00:00
f871f5cc6c [#793] adm: Support new FrostFS ID contract
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-11-14 19:00:32 +00:00
b62008daca [#506] node: Invalidate list cache after container add/removal
`update` already has problems mentioned in its doc-comment and the code
itself is not straightforward. Invalidating cache altogether seems like
a better option because we don't construct cache output ourselves (thus, no
"impossible" results).

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-11-14 19:00:11 +00:00
f13f5d3b0f [#506] node: Use DeletionInfo() method to get deleted container owner
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-11-14 19:00:11 +00:00
2393d13e4d [#787] morph: Return VUB for IR service calls
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-13 17:13:03 +03:00
5466e88444 [#787] cli: Add vub for control ir commands
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-13 17:13:03 +03:00
78cfb6aea8 [#796] cli: Fix object nodes command
All checks were successful
DCO action / DCO (pull_request) Successful in 2m23s
Vulncheck / Vulncheck (pull_request) Successful in 2m55s
Build / Build Components (1.21) (pull_request) Successful in 4m18s
Build / Build Components (1.20) (pull_request) Successful in 4m22s
Tests and linters / Tests (1.21) (pull_request) Successful in 6m2s
Tests and linters / Lint (pull_request) Successful in 6m11s
Tests and linters / Staticcheck (pull_request) Successful in 5m55s
Tests and linters / Tests (1.20) (pull_request) Successful in 6m29s
Tests and linters / Tests with -race (pull_request) Successful in 7m47s
Tombstone objects must be present on all container nodes.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-09 13:41:36 +03:00
3ed3e2715b [#xx] adm: Drop notaryDisabled deploy parameter
Refs TrueCloudLab/frostfs-contract#50

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-11-08 13:38:04 +00:00
66848d3288 [#770] cli: Add methods to work with APE rules via control svc
* Add methods to frostfs-cli
* Implement rpc in control service

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-11-08 13:34:03 +00:00
8e11ef46b8 [#770] object: Introduce ape chain checker for object svc
* Introduce Request type converted from RequestInfo type
  to implement policy-engine's Request interface
* Implement basic ape checker to check if a request is
  permitted to be performed
* Make put handlers use APE checker instead EACL

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-11-08 13:34:03 +00:00
5ec73fe8a0 [#770] node: Introduce ape chain source
* Provide methods to access rule chains with access
  policy engine (APE) chain source
* Initialize apeChainSource within object service
  initialization
* Share apeChainSource with control service
* Implement dummy apeChainSource instance based on
  in-memory implementation

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-11-08 13:34:03 +00:00
1e21733ea5 [#684] cli: Add skipped count to evacuation status output
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-07 12:17:11 +00:00
74c91eeef5 [#777] client: Refactor PrmContainerList, PrmObjectSearch usage
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-11-06 06:50:11 +00:00
cae50ecb21 [#716] adm: Add dump policy
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2023-11-06 06:43:56 +00:00
1ab567870a [#779] adm: Support deploying policy contract
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-11-02 09:23:41 +00:00
0b0e5dab24 [#756] adm: Add polling interval increase
All checks were successful
Build / Build Components (1.20) (pull_request) Successful in 3m37s
DCO action / DCO (pull_request) Successful in 5m49s
Vulncheck / Vulncheck (pull_request) Successful in 5m44s
Build / Build Components (1.21) (pull_request) Successful in 6m53s
Tests and linters / Tests (1.20) (pull_request) Successful in 7m40s
Tests and linters / Staticcheck (pull_request) Successful in 7m39s
Tests and linters / Tests with -race (pull_request) Successful in 7m53s
Tests and linters / Lint (pull_request) Successful in 8m16s
Tests and linters / Tests (1.21) (pull_request) Successful in 9m55s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-01 14:24:28 +03:00
c7a7229484 [#764] metrics: Fix epoch metric
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-01 10:57:31 +00:00
a26483fc30 [#749] morph: Fix panic when closing morph client
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-11-01 10:48:10 +00:00
c80b46fad3 [#754] blobstor: Estimate compressability
All checks were successful
DCO action / DCO (pull_request) Successful in 1m59s
Vulncheck / Vulncheck (pull_request) Successful in 3m31s
Build / Build Components (1.20) (pull_request) Successful in 4m37s
Build / Build Components (1.21) (pull_request) Successful in 4m33s
Tests and linters / Tests (1.20) (pull_request) Successful in 4m54s
Tests and linters / Staticcheck (pull_request) Successful in 4m49s
Tests and linters / Tests with -race (pull_request) Successful in 5m9s
Tests and linters / Lint (pull_request) Successful in 6m4s
Tests and linters / Tests (1.21) (pull_request) Successful in 6m9s
Now it is possible to enable compressability estimation.
If data is likely uncompressable, it should reduce CPU time and memory.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-01 11:24:32 +03:00
79088baa06 [#772] node: Apply gofumpt
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-10-31 17:03:03 +03:00
261d281154 [#762] go.mod: Update sdk-go
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-10-31 11:22:29 +00:00
4239f1e817 [#750] adm: Drop deprecated rpcclient.TransferTarget
We do not use `nep17` wrapper, because transfers of different tokens are
possible in a single transaction.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-10-25 07:57:05 +00:00