Commit graph

1550 commits

Author SHA1 Message Date
8a4e250dae [#468] *: replace outdated TODO crypto-related links
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-28 12:13:20 +00:00
73a71a71b0 [#1] node: Use a proper validation of a substorage type
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-28 12:13:20 +00:00
f4c71cea65 [#1] *: Replace outdated FIXME/TODO links
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-28 12:13:20 +00:00
cac4ed93d6 [#428] engine: Add low_mem config parameter
Concurrent initialization in case of the metabase resync leads to
high memory consumption and potential OOM.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-26 13:29:39 +00:00
72fedff7ad [#426] cli: Add object nodes command
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-26 07:41:34 +00:00
ab489265b3 [#449] cli: Allow to exec tree get-by-path without bearer token
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-06-23 14:44:19 +03:00
71889234b7 [#449] tree: Allow reading requests signed by keys from allow list
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-06-23 14:44:19 +03:00
167a67f0b8 [#460] services/util: Remove HandleUnaryRequest
There is no need in a wrapper with many from-`interface{}` conversions.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-21 17:07:56 +03:00
b5d9f4a285 [#373] metrics: Add pilorama metrics
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-21 15:13:26 +03:00
e89fa110c7 [#373] metrics: Add metabase metrics
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-21 15:13:26 +03:00
af608da952 [#373] metrics: Add blobovnizca metrics
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-21 15:13:26 +03:00
56f320dd85 [#373] metrics: Add blobstor metrics
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-21 15:13:26 +03:00
16a142cd0c [#373] metrics: Add FSTree metrics
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-21 15:13:26 +03:00
059e9e88a2 [#373] metabase: Add metrics
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-21 15:13:26 +03:00
26b305f82b [#424] morph: Fix cache metrics
Use separate morph cache metrics for node and IR

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-14 18:26:19 +03:00
c348ae35b0 [#424] metrics: Drop embedded metrics
It was not obvious where metrics are used.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-14 14:53:32 +03:00
344d6b2ae1 [#xx] Fix invalid log metric namespace identifier
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-06-13 15:36:34 +03:00
4887f489a1 [#17] Add morph client metrics
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-06-13 14:06:04 +03:00
4f83ab0fb4 [#409] node: Do not sent initial bootstrap under maintenance
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-13 10:02:36 +00:00
e68384d4e3 [#409] node: Fetch last bootstrap info on startup
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-13 10:02:36 +00:00
898f0686b1 [#409] node: Log maintenance state on startup
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-13 10:02:36 +00:00
0c40d98f7a [#375] Add log metrics
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-06-13 09:52:45 +00:00
508e2064eb [#438] cli: Drop tracing debug print
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-09 14:12:25 +03:00
189a367ef2 [#390] frostfs-cli: Pass bearer token to Tree srv
* Add --bearer flag for "tree" subcommand

Signed-off-by: Airat Arifullin a.arifullin@yadro.com
2023-06-07 11:08:34 +03:00
a770b89fd8 [#422] adm: Fix ValidatorCount problems after neo-go update
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-02 14:48:51 +03:00
f8c1e0639d [#422] cli: Provide context to NetmapSnapshot()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-02 14:48:47 +03:00
f1f56ef871 [#406] cli: Add --trace flag
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-02 11:44:04 +00:00
96c9843591 [#406] cmd/common: Execute PersistentPostRun() on errors
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-02 11:44:04 +00:00
9562123c49 [#406] cli: Pass context to internal client
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-02 11:44:04 +00:00
2dd3fc8b7e [#xx] cli: Add policy-playground command
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-06-02 09:23:40 +00:00
dcdfb6ed41 [#412] node: Use observability interceptors
Use metrics and tracing interceptors.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-01 13:23:11 +00:00
c09144ecf1 [#412] node: Replace metrics package
Use observability module.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-01 13:23:11 +00:00
74578052f9 [#412] node: Replace tracing package
Use observability module.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-01 13:23:11 +00:00
faca861451 [#411] Remove unnecessary pointers for sync objects
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-05-31 10:19:14 +00:00
f64322576a [#402] cli: Add estimated evacuation time left
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-05-31 10:18:44 +00:00
8dcd06c587 [#394] node: Use Context in Blobovniczas.Iterate()
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-05-31 10:09:18 +00:00
a3e30062df [#325] node: Introduce unsafe_disable param to disable policer
Signed-off-by: Airat Arifullin a.arifullin@yadro.com
2023-05-31 10:09:05 +00:00
bc34fee6a7 [#370] Add tree service metrics
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-05-26 13:39:12 +00:00
5983617069 [#393] shard: Create tombstone source when reload
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-05-26 12:14:02 +00:00
9119199f6e [#397] cli: Fix evacuation method names
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-05-26 11:28:30 +03:00
81718afb39 [#338] ir: Drop named named put fee
Named put fee value used only when notary disabled.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-05-24 08:44:47 +00:00
03ab0ca30f [#338] adm: Drop notaryless code
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-05-24 08:44:47 +00:00
4b768fd115 [#381] *: Move to sync/atomic
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-23 08:18:01 +03:00
e3ad3c2965 [#283] cli: Move control healthcheck command under control ir
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-05-22 08:13:52 +00:00
f7c0b50d70 [#329] cli: Add async evacuate commands
Add start, stop evacuate and evacuate status commands.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-05-19 08:43:52 +00:00
aabcd8d3e4 [#362] node: Cancel ctx in the same routine as for signal watcher
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-05-18 09:28:10 +03:00
df9e099fa7 [#352] Add explicit max number of alphabet nodes
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-05-17 15:01:26 +00:00
6f47c75e43 [#125] ir: Set extra wallets on SIGHUP
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-05-16 12:47:43 +00:00
8da6530f41 [#351] cli: Support copies number parameter in object put
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-05-16 08:22:56 +00:00
079b28fa0f [#341] Register candidates in separate transactions
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-05-15 12:46:12 +00:00
d4d921dcaf [#346] adm: fix race in wallet generation
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-15 11:50:20 +03:00
429a87e83b [#39] ir: Use defer for wg.Done()
Not important, but `exitOnErr` can alter control flow, let's be
explicit.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-12 17:49:47 +03:00
Pavel Karpy
f604d6bbdc [#39] ir: Add optional profilers
Includes `block` and `mutex` profiles configuration.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-12 17:49:47 +03:00
f989bc52be [#39] ir: Do not store config keys in httpComponent
Pprof will have specific options, it seems wrong to have them in a
generic struct.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-12 17:49:46 +03:00
61776033c2 [#39] ir: Do not reload services if they are disabled
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-12 15:49:04 +03:00
Pavel Karpy
14c35d776e [#39] node: Add optional profilers
Include settings for block and mutex profilers.
They are disabled by default, as in Go runtime itself.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-12 15:49:04 +03:00
a6ee7a3087 [#324] Add replicator metrics
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-05-12 11:00:17 +00:00
c8c5f14e2e [#321] adm/test: Check wallet correctness in parallel
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-12 09:45:03 +00:00
fe4082799a [#321] adm: Create multisig accounts in parallel
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-12 09:45:03 +00:00
800eb5e983 [#125] ir: Reconfigure pprof and metrics on SIGHUP
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-05-11 12:38:34 +03:00
90799497d3 [#114] Add remove-node IR control command
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-05-10 14:31:44 +00:00
d02950ad63 [#330] morph: Fix linter issue
Revive became smarter after go 1.20.4 upgrade.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-05-05 18:02:38 +03:00
d1661ae7dc [#308] Remove downloadContractsFromGithub from the frostfs-adm
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-05-04 14:08:42 +03:00
30e1b62b67 [#277] getsvc: Fix service deps
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-28 14:03:12 +00:00
3bac5a485d [#248] config: Remove audit-related parameters
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-28 17:00:43 +03:00
8b2aae73c6 [#248] cli: Remove storagegroup commands
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-28 17:00:43 +03:00
ff25521204 [#270] Add IR epoch tick control call
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-04-28 07:57:00 +00:00
dfe4ada838 [#285] lint: Resolve revive/if-return
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-26 17:35:08 +03:00
f07e2d4812 [#285] lint: Fix revive/unused-parameter
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-26 17:35:04 +03:00
14f83b8aa9 [#125] ir: Change log level on SIGHUP
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-04-26 13:54:19 +03:00
563780057d [#125] ir: Use internal/common/config for reading config
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-04-26 13:53:51 +03:00
ef222e2487 [#125] cmd: Refactor internal/common/viper
Add `opts.WithViper`, set opts struct as private.

Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-04-26 13:53:51 +03:00
e61aec4a7d [#125] node: Remove unused config.Prm
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-04-26 13:53:51 +03:00
eb7be82e87 [#125] node: Avoid panic when reading config
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-04-26 13:53:51 +03:00
d390f093e0 [#125] node: Move viper creation to internal/common/config
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-04-26 13:53:51 +03:00
b2123bfd1a [#125] node: Add new option WithEnvPrefix
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-04-26 13:53:51 +03:00
ee01275d25 [#125] node: Remove redundant env from config/internal
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-04-26 13:53:51 +03:00
04be9415d9 [#231] node: Fix race condition in TTL cache
Use key locker to lock by key.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-25 09:54:12 +03:00
ddbc9e255f [#231] node: Invalidate container cache on PutSuccess event
For example: frostfs-cli creates container and makes polling
GetContainer requests. These requests go through container cache,
so not found error stores in container cache.
So container cache can contain not found error when PutSuccess event received.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-25 09:42:02 +03:00
015d62425b [#164] metrics: Fill local registry explicitly
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-24 09:01:13 +00:00
Pavel Karpy
59822f7fb4 [#1248] node: Do not update cache twice
Do not request morph values on morph cache misses concurrently.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2023-04-21 09:45:53 +00:00
3d43b0f7f9 [#265] node: Fix after SDK & API-Go version up
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-18 12:09:19 +03:00
a358255c1b [#265] node: Drop unused
Resolve unused linter.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-18 12:09:16 +03:00
f07d4158f5 [#249] node: Drop subnet from IR and morph
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-17 09:46:02 +03:00
d757d881d0 [#249] node: Drop subnet from config
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-17 09:46:02 +03:00
05c870f39a [#249] cli: Drop subnet support
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-17 09:46:02 +03:00
160147b05d [#249] adm: Drop subnet support
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-17 09:46:02 +03:00
8466894fdf [#250] control: remove DumpShard and RestoreShard RPC
We have `Evacuate` with a cleaner interface.
Also, remove them from CLI and engine.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-14 12:28:49 +00:00
Pavel Karpy
beabed788c [#247] network_config: Drop reputation
Drop the code that was expected to work with global reputation network
parameters.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2023-04-14 14:53:22 +03:00
Pavel Karpy
960e3c219e [#247] config, doc: Drop reputation references
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2023-04-14 14:44:14 +03:00
Pavel Karpy
560f73ab7e [#247] node, ir: Drop reputation related code
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2023-04-14 14:44:14 +03:00
d62c6e4ce6 [#242] node: Add tracing spans
Add tracing spans for PUT requests.
Add tracing spans for DELETE requests.
Add tracing spans for SELECT requests.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-14 10:25:53 +00:00
995db117d0 [#238] node: Read cfg from dir even if cfg file not set
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-04-14 10:19:10 +00:00
299b6a6938 [#100] adm: Use netmap constants from pkg
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-04-14 05:12:50 +00:00
0c6aeaaf18 [#100] adm: Take net settings into account during netmap contract update
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-04-14 05:12:50 +00:00
0e31c12e63 [#240] logs: Move log messages to constants
Drop duplicate entities.
Format entities.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-14 05:06:09 +00:00
d686ab49e8 [#202] adm: Remove deprecated RPC client methods
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-13 13:15:30 +00:00
2f1beddfd3 [#202] adm: Remove deprecated warnings in tests
`VerifyBlocks` is now `SkipBlockVerification` and is false by default.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-13 13:15:30 +00:00
01c0c90a86 [#113] cli: add "name" option for "get container" command
* Make get container command filter out the container by attribute name

Signed-off-by: Airat Arifullin a.arifullin@yadro.com
2023-04-12 18:25:37 +00:00
0920d848d0 [#135] get-object: Add tracing spans
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-12 06:52:00 +00:00
5af9f58469 [#135] tracing: Add tracing to node gRPC endpoints
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-12 06:52:00 +00:00
2c07f831c7 [#223] node: Refactor cache usage
Drop excess type args.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-10 14:16:34 +03:00
b689027d57 [#191] cli: Add control shards doctor command
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-07 17:25:50 +00:00
7ebbfa3358 [#212] reputationsvc: Resolve linters and rename
Resolved containedctx linters.
Renamed context structs and interfaces to more understandble names.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-07 15:35:57 +00:00
e815b19101 [#219] morph: Resolve containedctx linter
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-07 15:43:20 +03:00
e21c5bea21 [#203] cli: Fix error message
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-06 16:33:36 +03:00
68a2f36636 [#203] morph: Mark depracated methods
Skip staticcheck for depracated methods. Will be fixed soon.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-06 16:33:36 +03:00
9e2df4b7c7 [#203] node: Fix double imports
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-06 16:33:36 +03:00
206458c841 [#217] containersvc: Resolve containedctx linter
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-05 14:38:00 +00:00
279261ace3 [#217] containersvc: Refactor route passing
Resolve containedctx for routeCtx.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-05 14:38:00 +00:00
23575e1ac0 [#210] policier: Resolve contextcheck linter
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-05 14:55:52 +03:00
49cc23e03c [#175] adm: pipeline container iteration
Do not accumulate everything in memory.
Also, CLI should be responsive.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-04 07:26:23 +00:00
e85e5382e4 [#175] adm: list containers using containersOf
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-04 07:26:23 +00:00
0948a280fa [#195] morph: use blocking unlimited pool for notifications
With non-blocking pool restricted by 10 in capacity, the probability of
dropping events is unexpectedly big. Notifications are an essential part of the FrostFS,
we should not drop anything, especially new epochs.
```
Mar 31 07:07:03 vedi neofs-ir[19164]: 2023-03-31T07:07:03.901Z        debug        subscriber/subscriber.go:154        new notification event from sidechain        {"name": "NewEpoch"}
Mar 31 07:07:03 vedi neofs-ir[19164]: 2023-03-31T07:07:03.901Z        warn        event/listener.go:248        listener worker pool drained        {"chain": "morph", "capacity": 10}
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-03 15:58:16 +00:00
27bdddc48f [#199] putsvc: Refactor put object
Resolve containedctx linter for streamer and remote target

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-03 15:58:11 +00:00
ed28ce24cd [#168] node: Refactor reputation service
Resolve funlen linter for initReputationService function

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-03-31 09:32:59 +03:00
1f929fdd57 [#168] node: Refactor netmap service init
Resolve funlen linter for initNetmapService function

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-03-31 09:32:59 +03:00
dd572825c7 [#168] node: Refactor object service init
Resolve funlen linter for initObjectService function

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-03-31 09:32:59 +03:00
56161d39b4 [#168] node: Refactor container services
Resolve containedctx for remoteLoadAnnounceWriter struct

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-03-31 09:32:59 +03:00
28dc9e2190 [#168] node: Refactor container service init
Resolve funlen linter for initContainerService function

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-03-31 09:32:59 +03:00
dcd39f8fdd [#168] node: Refactor shard opts initialization
Resolve funlen linter for shardOpts method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-03-31 09:32:59 +03:00
c94372e6f9 [#168] node: Refactor config initialization
Resolve funlen linter for initCfg function

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-03-31 09:32:59 +03:00
3bbb516528 [#168] node: Refactor node config read
Resolve funlen linter for readConfig method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-03-31 09:32:59 +03:00
a7c79c773a [#168] node: Refactor node config
Resolve containedctx linter for cfg

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-03-31 09:32:59 +03:00
341fe1688f [#139] test: Add test storage implementation
This aims to reduce the usage of chmod hackery to induce or simulate
OS-related failures.

Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-03-29 14:28:49 +00:00
Pavel Karpy
ba58a77f8c [#184] node, ir: Do not wait for already made notary deposits
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2023-03-29 10:34:54 +00:00
221203beeb [#180] node: Refactor panics in unit test
* Replace panics in unit tests by require.NoError and t.Fatalf

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-03-29 12:39:07 +03:00
382ecae96a [#172] Use ContainersOf() for container list fetching
Previously we were limited by ~2048 containers because of
neo-go VM limits.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-27 14:32:55 +00:00
5bf1ec348f [#161] adm: Refactor storage-config command
Resolve funlen linter for storageConfig function

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-03-27 12:42:37 +00:00
93cb9e3a94 [#161] morph: Refactor invokeNotary method
Resolve funlen linter

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-03-27 12:42:37 +00:00
512b72591a [#161] adm: Refactor subnet commands
Resolve funlen linter for init method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-03-27 12:42:37 +00:00
e1b99dacad [#161] adm: Refactor netmap add/remove commands
Resolve funlen linter for manageSubnetAdmins function

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-03-27 12:42:37 +00:00
92f8810970 [#161] adm: Refactor commands initialization
Resolve funlen linter for init function

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-03-27 12:42:37 +00:00
b0fefcb21f [#161] adm: Refactor deposit-notary command
Resolve funlen linter for depositNotary function

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-03-27 12:42:37 +00:00
d62b11f5e7 [#161] adm: Refactor init context creation
Resolve funlen linter for newInitializeContext function

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-03-27 12:42:37 +00:00
ecbc211016 [#161] adm: Refactor update-contracts command
Resolve funlen linter for updateContracts method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-03-27 12:42:37 +00:00
5f7d70c59c [#161] adm: Refactor deploy command
Resolve funlen linter for deployContractCmd function

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-03-27 12:42:37 +00:00
9534ade716 [#161] adm: Refactor restore-containers command
Resolve funlen linter for restoreContainers function

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-03-27 12:42:37 +00:00
f2e880465e [#165] ir: Refactor config default init
Resolve funlen linter for defaultConfiguration function

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-03-27 07:43:56 +00:00
04b3d9d068 [#165] cli: Drop redundant nolint comment
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-03-27 07:43:56 +00:00
1d5f2dd681 [#165] cli: Refactor collectObjectRelatives
Resolve funlen linter

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-03-27 07:43:56 +00:00
c2cf708e0e [#165] cli: Refactor put command
Resolve funlen linter for putObject method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-03-27 07:43:56 +00:00
c78e9cc857 [#165] cli: Refactor get command
Resolve funlen linter for getObject method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-03-27 07:43:56 +00:00
Pavel Karpy
3c7ed21f74 [#141] Add big object's parts to a lock object's body
That will prevent part/link object from being removed by both an external
`DELETE` call and the object expiration procedure.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2023-03-27 07:28:32 +00:00
342e571d89 [#159] Add handle __SYSTEM__ sys attributes
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-03-22 17:35:20 +03:00
484ac502ca [#156] frostfs-adm: Make dumpBalances() pass linter checks
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-22 07:14:18 +00:00
5059dcc19d [#145] shard-gc: Delete expired objects after locks
GC deletes expired locks and objects sequentially. Expired locks and
objects are now being deleted concurrently in batches. Added a config
parameter that controls the number of concurrent workers and batch size.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-03-21 11:31:08 +03:00
481a1ca6f3 [#148] linter: Add gocognit linter
Code with high cognitive complexity is hard intuitively to understand

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-03-21 09:54:41 +03:00
97c36ed3ec [#148] linter: Add funlen linter
Long functions are hard to understand and source of errors

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-03-21 09:54:41 +03:00
cc8ff015b4 [#148] linter: Add containedctx linter
Context has to be passed as an argument: https://pkg.go.dev/context

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-03-21 09:52:39 +03:00
db3ccd2762 [#128] innerring: Add GAS pouring mechanism for a configurable list of wallets
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2023-03-20 12:50:05 +00:00
Pavel Karpy
10c419adf0 [#67] node: Fix infinite recursion in SE's wrapper
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2023-03-17 18:47:59 +03:00
Pavel Karpy
64bde68fb9 [#67] node: Accept expired locked objects
Allow replication of any (expired too) locked object. Information about
object locking is considered to be presented on the _container nodes_.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2023-03-16 16:22:19 +03:00
b4582239bf [#130] adm: Fix adding of pub key for group.frostfs at init step
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-03-13 15:05:53 +03:00
861e9ab59a [#83] pre-commit: Add initial configuration
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-13 07:07:29 +00:00
Leonard Lyubich
24a540caa8 [#132] cli/util: Fix basic ACL rendering
In previous implementation pretty-printer of basic ACL in NeoFS CLI had
mistakes:
 * F-bit was set to `Extendable()` property instead of its inversion
 * B-bits were set to `acl.RoleInnerRing` rights

Make `PrettyPrintTableBACL` to correctly render mentioned bits.

Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
2023-03-10 14:55:45 +03:00
9929dcf50b [#126] adm: Exclude group.frostfs key from output of the dump-hashes
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-03-09 18:29:36 +03:00
7486c02bbc [#88] adm: Fix method nnsResolveKey
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-03-09 15:41:28 +03:00
20de74a505 Rename package name
Due to source code relocation from GitHub.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-03-07 16:38:26 +03:00
79ba34714a [#79] cli: Fix panic when setting domain for container
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-03-01 11:11:11 +03:00
e9f3c24229 [#65] Use strings.Cut instead of strings.Split* where possible
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-02-28 13:39:14 +03:00
88e3868f47 [#37] cli: Add nns-name and nns-zone for container create
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-02-28 13:37:23 +03:00
73bb590cb1 [#64] node: Use pool_size_local and separate pool for local puts
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-02-22 13:43:19 +03:00
7eaf159a8b [#63] adm: Fix contract wallet creation
Create contract wallet only by init and update-config command.

Close #63

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-02-22 10:08:37 +03:00
cb5468abb8 [#66] node: Replace interface{} with any
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-02-21 16:47:07 +03:00
633c5a35de [#44] adm: Support multiple configs
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-02-21 10:00:28 +03:00
5f06232d34 [#44] cli: Support multiple configs
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-02-21 10:00:28 +03:00
bed5a36235 [#44] ir: Support multiple configs
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-02-21 10:00:28 +03:00
87e69b9349 [#44] node: Support multiple configs
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-02-21 10:00:28 +03:00
22f3c7d080 [#1868] Reload config for pprof and metrics on SIGHUP
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-02-20 13:53:27 +03:00
2b755ddb12 [#2260] node: Use a separate client cache for PUT service
Currently, under a mixed load one failed PUT can lead to closing
connection for all concurrent GETs. For PUT it does no harm: we have
many other nodes to choose from. For GET we are limited by `REP N`
factor, so in case of failover we can close the connection with the only
node posessing an object, which leads to failing the whole operation.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-02-20 13:53:27 +03:00
351fdd9fa2 [#2246] node: Allow to configure tombsone lifetime
Currently, DELETE service sets tombstone expiration epoch to
`current epoch + 5`. This works less than ideal in private networks
where an epoch can be e.g. 10 minutes. In this case, after a node is
unavailable for more than 1 hour, already deleted objects have a chance
to reappear.

After this commit tombstone lifetime can be configured.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-02-20 13:53:27 +03:00
2272c55c4d [#2238] engine: Add test for component initialization failures
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-02-20 13:53:27 +03:00
59748b7ae8 [#2238] neofs-node: Gracefully handle shard initialization errors
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-02-20 13:53:27 +03:00
236c4af615 [#2224] adm: Use native neo-go sessions in dump-hashes
If we had lots of domains in one zone, `dump-hashes` for all others
can miss some domains, because we need to restrict ourselves with _some_
number.
In this commit we use neo-go sessions by default, with a proper
failback to in-script iterator unwrapping.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-02-20 13:53:27 +03:00
c43b2dbac9 [#1465] Add log entry for morph components shutdown action
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-02-17 12:13:00 +03:00
85cf1f47ac [#1465] node: Prevent process from killing by systemd when shutting down
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-02-17 12:13:00 +03:00
3e5bc394b5 [#48] adm: Add initialize test for 1 node
Single node is used in dev-env, worth testing.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-02-16 09:07:18 +03:00
1d3669232e [#48] adm: Allow using nonzero magic with local client
neo-go actor API uses `getVersion` call which returned incorrect magic.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-02-16 09:07:18 +03:00
f825cfac78 [#1] Fix project name in comments here and there
Signed-off-by: Stanislav Bogatyrev <s.bogatyrev@yadro.com>
2023-02-06 17:41:14 +03:00
c6645ef775 [#1] Documentation rebranding
Signed-off-by: Stanislav Bogatyrev <s.bogatyrev@yadro.com>
2023-02-06 17:41:14 +03:00
1858f11146 [#1] Fix viper env prefix in cli tools
Changing env prefix and corresponding example config files.

Signed-off-by: Stanislav Bogatyrev <s.bogatyrev@yadro.com>
2023-02-06 17:41:14 +03:00
Pavel Karpy
73bc1b0b68 [#38] node: Fix linter warnings
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2023-02-06 17:27:54 +03:00
515c60bdf4 [#1889] adm: Add command morph netmap-candidates
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-02-06 17:26:34 +03:00
ee24815748 [#1889] Move flag --config in cmd/frostfs-adm/internal/commonflags/flags.go
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-02-06 17:26:34 +03:00
2b09564355 [#1889] Move netmap.go and exit.go from cli to cmd/internal/common
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-02-06 17:26:34 +03:00
5a9d6a09d8 [#8] cli: Set flag mode required for control shards set-mode
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-02-06 16:45:03 +03:00
5d64a354cb [#1] Fix naming in FrostFS Adm help output
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-01-31 11:24:50 +03:00
d7e9e2ef9e [#1] Fix naming in FrostFS Lens help output
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-01-31 11:24:50 +03:00
d31d8c5335 [#1] Fix naming in FrostFS CLI help output
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-01-31 11:24:50 +03:00
406ff1360f [#1] Fix version output for all compiled binaries
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-01-31 11:24:50 +03:00
072a7d61ab [#2176] neofs-node: Do not cache full container list
We rarely need to list all containers: as one example
we need it for tree service synchronization once per epoch.
Given that cache TTL has the order of block time it makes no sense
to cache the list of all containers.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-01-25 15:31:47 +03:00
8078af3424 [#2176] neofs-node: Do not invalidate old cache items
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-01-25 15:31:47 +03:00
e5a40b90b6 [#2176] neofs-node: Peek during cache invalidation
`Get` needs write mutex and makes our item move to top.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-01-25 15:31:47 +03:00
93eb72ef44 [#2176] neofs-node: Remove unused field from morphContainerWriter
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-01-25 15:31:47 +03:00
8ee590794f [#1962] cli: common.PrintVerbose prints via cobra.Command.Printf
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-01-25 15:31:47 +03:00
cff4184cd3 [#2128] Add doc for extended headers
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-01-25 15:31:47 +03:00
d4d2a8c865 [#11] Rename NeoFS to FrostFS in docs
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-01-12 08:55:47 +03:00
9cb4b4cc17 [#11] Rename neofsid contract to frostfsid
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-01-12 08:55:47 +03:00
023396e6a4 [#11] adm: Remove NeoFS mentions
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-01-12 08:55:47 +03:00