Support impersonation flag in bearer token #68

Merged
KirillovDenis merged 1 commit from KirillovDenis/poc/impersonate into master 2023-07-26 21:07:56 +00:00
KirillovDenis commented 2023-02-21 09:00:52 +00:00 (Migrated from github.com)

close #229

close #229
fyrchik changed title from Poc/impersonate to WIP: Poc/impersonate 2023-03-09 15:00:23 +00:00
fyrchik added the
blocked
label 2023-04-05 11:14:35 +00:00
Owner

Blocked until we create a support branch for the new release.

Blocked until we create a support branch for the new release.
dkirillov removed the
blocked
label 2023-04-18 12:01:21 +00:00
dkirillov self-assigned this 2023-04-18 12:01:28 +00:00
dkirillov added 248 commits 2023-04-18 12:05:06 +00:00
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
Create contract wallet only by init and update-config command.

Close #63

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Compute shard hash only once

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Update HRW lib and use typed HRW methods to sort shards and nodes

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
Due to source code relocation from GitHub.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
It will prevent test fails with `-race` flag on components that have
background processes and make some actions on test framework.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
If `processNodes` exits earlier for some reason, `needLocalCopy` could
be false.
See https://github.com/nspcc-dev/neofs-node/issues/2267

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
In previous implementation pretty-printer of basic ACL in NeoFS CLI had
mistakes:
 * F-bit was set to `Extendable()` property instead of its inversion
 * B-bits were set to `acl.RoleInnerRing` rights

Make `PrettyPrintTableBACL` to correctly render mentioned bits.

Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Issue templates are still supported by Gitea:
https://docs.gitea.io/en-us/issue-pull-request-templates/ .

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Skip deprecated warning for now, adopting new neo-go API will be done in
another task.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Remember the last synchronized height and use it after service restart.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Add make targets to simplify pre-commit setup for individual developers.

Signed-off-by: Stanislav Bogatyrev <realloc@realloc.spb.ru>
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
Extract assemble logic to assembler

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Add unit tests to cover all assemble statements

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Should return an error in case of a broken LOB reference chain.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Original shellcheck action requires Docker to run and it's not always
available, especially inside Docker containers. Replacing it with python
wrapper to simplify usage with Docker-based CI systems like Drone and
WoodpeckerCI.

Signed-off-by: Stanislav Bogatyrev <s.bogatyrev@yadro.com>
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
Previously a token could've expired in the middle of an object.PUT
stream, leading to upload being interrupted. This is bad, because user
doesn't always now what is the right values for the session token
lifetime. More than that, setting it to a very high value will
eventually blow up the session token database.

In this commit we read the session token once and reuse it for the whole
stream duration.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
It will allow reusing that method in expiration checks.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
Allow replication of any (expired too) locked object. Information about
object locking is considered to be presented on the _container nodes_.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
May make docker builds not so clean.

Signed-off-by: Stanislav Bogatyrev <s.bogatyrev@yadro.com>
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
This reverts commit 2567f8020e. It assumes
that assembling logic could break some failover scenarios if request
forwarding is done. However, it also breaks requesting big objects via a
non-container node with TTL=2. Failover has been rechecked without that
commit and no problems were found. Any (if found) other bugs related to
the forwarding and object assembling must be solved more carefully.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
In our policy we mark commits not having a PR/Issue yet with a `[#XX]`
reference to be replaced after PR creation.

Signed-off-by: Stanislav Bogatyrev <s.bogatyrev@yadro.com>
Minor changes to see what fails first

Signed-off-by: Stanislav Bogatyrev <s.bogatyrev@yadro.com>
Until #139 is fixed, we can't use root inside Docker container running
CI, but Woodpecker CI can't run non-root containers until they fix
https://github.com/woodpecker-ci/woodpecker/issues/1077, hence we use
temporary kludges with custom image and manual permissions in pipelines.

Signed-off-by: Stanislav Bogatyrev <s.bogatyrev@yadro.com>
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Context has to be passed as an argument: https://pkg.go.dev/context

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Long functions are hard to understand and source of errors

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Code with high cognitive complexity is hard intuitively to understand

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#148] linter: Add contextcheck linter
Some checks failed
ci/woodpecker/push/full-pre-commit Pipeline failed
ci/woodpecker/push/pre-commit Pipeline was successful
44b86bac5a
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#152] IR: Process empty basic incomes
Some checks failed
ci/woodpecker/push/full-pre-commit Pipeline failed
ci/woodpecker/push/pre-commit Pipeline was successful
9cd8f7cea0
If network is not configured for basic income earnings, do not distribute
GAS by the Alphabet nodes.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
Added unit test that verifies that GC deletes expired
locked objects in one epoch.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
GC deletes expired locks and objects sequentially. Expired locks and
objects are now being deleted concurrently in batches. Added a config
parameter that controls the number of concurrent workers and batch size.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#145] docs: Add expired object collector params
Some checks failed
ci/woodpecker/pr/full-pre-commit Pipeline failed
ci/woodpecker/pr/pre-commit Pipeline was successful
ci/woodpecker/push/full-pre-commit Pipeline failed
ci/woodpecker/push/pre-commit Pipeline was successful
7a31988a36
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#128] IR: Do not try to emit GAS to nobody
Some checks failed
ci/woodpecker/push/full-pre-commit Pipeline failed
ci/woodpecker/push/pre-commit Pipeline was successful
9a4f40626c
Fix sending GAS to an empty extra wallets receivers list. Also, send GAS to
extra wallets even if netmap is empty.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
[#153] ci: Minor pipeline fixes
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
9aeea0b974
- We can skip full pre-commit run
- On a very slow agent golangci run may take up to 10 minutes

Signed-off-by: Stanislav Bogatyrev <s.bogatyrev@yadro.com>
Signed-off-by: Liza <e.chichindaeva@yadro.com>
Add to Possible Solutions
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
ci/woodpecker/push/pre-commit Pipeline was successful
ec2c5d45b4
Signed-off-by: Liza <e.chichindaeva@yadro.com>
Initially it was there to check whether an update is being initiated by
a proper node. It is now obsolete for 2 reasons:
1. Background synchronization fetches all operations from a single node.
2. There are a lot more problems with trust in the tree service, it is
   only used in controlled environments.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
[#156] object/get: Make toHeadPrm() pass gocognit
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
cbc2efb1d6
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
In case of session token (ST) with object IDs search should
return only objects allowed in static session

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#155] search-service: Fix search with ST
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
49234b915e
Search should return only objects allowed in static session

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#98] fstree: Do not fail iteration over just removed files
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
da8da1c63a
A directory is read and files are saved to a local variable. The iteration
over such files may lead to a non-existing files reading due to a normal SN
operation cycle and, therefore, may lead to a returning the OS error to a
caller. Skip just removed (or lost) files as the golang std library does in
similar situations:
5f1a0320b9/src/os/dir_unix.go (L128-L133).

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
[#158] Fix gitlint run in pre-commit under CI
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
f111704ceb
Wrong stage was used.
It still may run gitlint against only latest commit in PR.

Signed-off-by: Stanislav Bogatyrev <s.bogatyrev@yadro.com>
[#159] Add handle __SYSTEM__ sys attributes
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
ci/woodpecker/push/pre-commit Pipeline was successful
342e571d89
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
[#86] node: Move testing utils to one package
Some checks failed
ci/woodpecker/push/pre-commit Pipeline failed
9808dec591
Move testing utils from tests in local_object_storage package to
unified testutil package

Signed-off-by: Airat Arifullin <aarifullin@yadro.com>
[#86] node: Fix unit test and linter errors
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
ci/woodpecker/push/pre-commit Pipeline was successful
34329d67ff
Signed-off-by: Airat Arifullin <aarifullin@yadro.com>
This change allows to use `[#xx]` placeholders for issue number.
This change adds go unit test run if .go files were changed.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
[#171] Remove unit tests from pipeline
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
9ffa0d8fea
Unit tests are included as pre-commit hook. They are
triggered by the change of .go files.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
It does not use deprecated methods anymore but also adds more code that
removes. Future refactor that will affect more components will optimize
usage of the updated API.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
[#73] morph: Rename vars that collide with package names
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
f3ff9fd251
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
[#174] go.mod: Update dependencies
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
a5ece7889d
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
[#141] Add big object's parts to a lock object's body
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
3c7ed21f74
That will prevent part/link object from being removed by both an external
`DELETE` call and the object expiration procedure.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
Resolve funlen linter for getObject method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for putObject method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#165] ir: Refactor config default init
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
f2e880465e
Resolve funlen linter for defaultConfiguration function

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for restoreContainers function

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for deployContractCmd function

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for updateContracts method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for newInitializeContext function

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for depositNotary function

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for init function

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for manageSubnetAdmins function

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for init method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#161] adm: Refactor storage-config command
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
5bf1ec348f
Resolve funlen linter for storageConfig function

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
[#172] Use ContainersOf() for container list fetching
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
382ecae96a
Previously we were limited by ~2048 containers because of
neo-go VM limits.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
[#176] morph: Resolve funlen linter
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
ci/woodpecker/push/pre-commit Pipeline was successful
91717d4b98
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#180] node: Refactor panics in unit test
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
221203beeb
* Replace panics in unit tests by require.NoError and t.Fatalf

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
If `ctx.Done()`, return `ctx.Err()` in every function that returns an error.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
We have already had and solved plenty of deposit issues and notary balance
is a really important thing. Deserves to be INFO even before the huge logs
severity refactor, happens on an app start only.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
[#184] ir: Delete unused config reader param
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
e843e7f090
Not used since notary environments do endless and non-configurable
deposits.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
[#139] test: Add test storage implementation
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
341fe1688f
This aims to reduce the usage of chmod hackery to induce or simulate
OS-related failures.

Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#183] gc: Fix drop expired locked complex objects
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
ab32067152
Do not delete bucket keys during iteration

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#189] node: Fix bug with sync/atomic package
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
7be5a0fd79
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
Resolve funlen linter for GlagoliticLetter.String method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for Server.Start method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for New function

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for processEmit method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for processStartAudit method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for processAlphabetSync method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for verifySignature method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#185] ir: Resolve containedctx linter
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
8426d25f4b
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve containedctx linter for cfg

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for readConfig method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for initCfg function

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for shardOpts method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for initContainerService function

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve containedctx for remoteLoadAnnounceWriter struct

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for initObjectService function

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for initNetmapService function

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#168] node: Refactor reputation service
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
ci/woodpecker/push/pre-commit Pipeline was successful
ed28ce24cd
Resolve funlen linter for initReputationService function

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
[#181] Update CHANGELOG
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
ci/woodpecker/push/pre-commit Pipeline was successful
2bdf7126b8
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
[#2272] morph: Do not subscribe to events without listening
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
a69c6d1ec9
It led to a neo-go dead-lock in the `subscriber` component. Subscribing to
notifications is the same RPC as any others, so it could also be blocked
forever if no async listening (reading the notification channel) routine
exists. If a number of subscriptions is big enough (or a caller is lucky
enough) subscribing loop might have not finished subscribing before the
first notification is received and then: subscribing RPC is blocked by
received notification (non)handling and listening notifications routine is
blocked by not finished subscription loop.
That commit starts listening notification channel _before_ any subscription
actions.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
Resolve funlen linter for iteratePlacement method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve staticcheck linter for putBytesPool

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve containedctx linter for streamer and remote target

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#199] putsvc: Refactor streamer initialization
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
ece6c820e7
Resolve funlen linter for initTarget method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#195] morph: use blocking unlimited pool for notifications
Some checks failed
ci/woodpecker/push/pre-commit Pipeline failed
0948a280fa
With non-blocking pool restricted by 10 in capacity, the probability of
dropping events is unexpectedly big. Notifications are an essential part of the FrostFS,
we should not drop anything, especially new epochs.
```
Mar 31 07:07:03 vedi neofs-ir[19164]: 2023-03-31T07:07:03.901Z        debug        subscriber/subscriber.go:154        new notification event from sidechain        {"name": "NewEpoch"}
Mar 31 07:07:03 vedi neofs-ir[19164]: 2023-03-31T07:07:03.901Z        warn        event/listener.go:248        listener worker pool drained        {"chain": "morph", "capacity": 10}
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
[#204] gc: Fix GC handlers start
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
8e5a0dcf27
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
[#175] adm: pipeline container iteration
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
49cc23e03c
Do not accumulate everything in memory.
Also, CLI should be responsive.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
[#205] innerring: Provide alphabetState param to epochTimer
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
ee7468daa7
Fix NPE, introduced in f09ee27a.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Resolve funlen linter for Blobovniczas.Put method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for TestIterate function

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for db.put method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for StorageEngine.get method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for StorageEngine.getRange method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen and gocognit linter for StorageEngine.Evacuate method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#188] metabase: Refactor object inhume
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
1f1aed87be
Resolve funlen linter for db.Inhume method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for CheckEACL method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for readObjectHeaders method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for putStreamBasicChecker.Send method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#207] aclsvc: Drop outdated tag
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
4941926c9d
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve containedctx and contextcheck linters.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#208] searchsvc: Refactor request forwarding
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
d85703a963
Resolve funlen & gocognit linters for toPrm method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
We already do this for file writing, however directory creation can also
fail.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
[#211] engine: Unify shard mode checks for tree operations
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
9098d0eec0
All operations must ensure the shard is not in a degraded mode.
Write operations must also ensure the shard is not in a read-only mode.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve containedctx and contextcheck linters.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#210] policier: Refactor nodes processing
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
ci/woodpecker/push/pre-commit Pipeline was successful
d6486d172e
Resolve funlen linter for processNodes method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#116] node: Improve shard/engine construction in tests
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
6f7b6a8813
* Introduce testEngineWrapper that can be constructed with different options

Signed-off-by: Airat Arifullin a.arifullin@yadro.com
Resolve containedctx for routeCtx.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#217] containersvc: Resolve containedctx linter
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
206458c841
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve containedctx and contextcheck linters.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for toHeadPrm method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for toPrm function.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for toRangePrm function.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Pass context to forwarder direct, without closure.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#193] getsvc: Reduce private key requests
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
c58ab0c369
Get private key only once for request forwaring.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#116] node: Fix bug with extra generated files in TestReload
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
ci/woodpecker/push/pre-commit Pipeline was successful
ab891517de
* Create testNewEngine in engineWithShards without default opts

Signed-off-by: Airat Arifullin a.arifullin@yadro.com
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Skip staticcheck for depracated methods. Will be fixed soon.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Do not pass 0 as channel capacity.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Fix shard compare loop.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#203] node: Add staticcheck target and pre-commit
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
9027695371
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#166] node: Parallelize background tree service sync
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
56282edf02
* Run sync task for nodes in parallel within errgroup worker pool

Signed-off-by: Airat Arifullin a.arifullin@yadro.com
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for Client.notaryInvoke method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen and gocognit linters for listener.listenLoop method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#219] morph: Refactor notary preparator
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
fe87735073
Resolve funlen linter for Preparator.Prepare method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve containedctx linter for commonCommunicatorPrm type.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve containedctx linter. Cancel task by listen cancel.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#222] auditsvc: Refactor PoR audit
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
e2f13d03d7
Resolve funlen linter for Context.checkStorageGroupPoR method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolved containedctx linters.
Renamed context structs and interfaces to more understandble names.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#212] reputationsvc: Resolve funlen linter
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
c236b54a65
Resolve funlen linter for Calculator.iterateDaughter method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#228] node: Use uber atomic package instead standard
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
cb172e73a6
Signed-off-by: Airat Arifullin a.arifullin@yadro.com
RemoveDuplicates() removes all duplicate object copies stored on
multiple shards. All shards are processed and the command tries to leave
a copy on the best shard according to HRW.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Doctor RPC performs complex operations on the storage engine.
Currently only duplicate removal is supported.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
[#191] cli: Add control shards doctor command
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
b689027d57
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Resolve funlen linter for newEngineMetrics function.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#213] metrics: Refactor object metrics
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
ci/woodpecker/push/pre-commit Pipeline was successful
02831d427b
Resolve funlen linter for newObjectServiceMetrics function.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for FormatValidator.ValidateContent method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve containedctx linter for SearchSGPrm and GetSGPrm  structs.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve containedctx linter for commonPrm.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen & gocognit linters for traverseSplitChain method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#223] node: Refactor cache usage
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
ci/woodpecker/push/pre-commit Pipeline was successful
2c07f831c7
Drop excess type args.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#230] CHANGELOG.md: Remove older entries
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
6bf11f7cca
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
I tried to add 4 more tests and suddenly, it became harder to navigate in
code. Move directory creation in a common function.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Before this commit it was like this:
```
BenchmarkSubstorageWritePerf/memstore-rand10-8            227425              4859 ns/op
BenchmarkSubstorageWritePerf/fstree_nosync-rand10-8     --- FAIL: BenchmarkSubstorageWritePerf/fstree_nosync-rand10-8
    perf_test.go:165: writing entry: file exists
    perf_test.go:165: writing entry: file exists
    perf_test.go:165: writing entry: file exists
BenchmarkSubstorageWritePerf/fstree-rand10-8            --- FAIL: BenchmarkSubstorageWritePerf/fstree-rand10-8
    perf_test.go:165: writing entry: file exists
    perf_test.go:165: writing entry: file exists
    perf_test.go:165: writing entry: file exists
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
[#236] blobstor/test: Prefill storage in parallel in read benchmark
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
ci/woodpecker/push/pre-commit Pipeline was successful
c4865783fc
`blobovniczatree` takes a really long time to prefill, because each
batch takes at least 10ms, so for 10k iterations we have at least 100s of
prefill.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#135] signature: Add tracing
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
5778980252
Add tracing to verify request and sign response.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
This reverts commit b2ca730547.
This reverts commit 5778980252.
Release v0.36.0
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
ci/woodpecker/push/pre-commit Pipeline was successful
7d39fecc6a
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
[#113] cli: add "name" option for "get container" command
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
01c0c90a86
* Make get container command filter out the container by attribute name

Signed-off-by: Airat Arifullin a.arifullin@yadro.com
`VerifyBlocks` is now `SkipBlockVerification` and is false by default.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Fixes #2230, fixes #2263. CustomGroups are nice while we're only calling NeoFS
contracts, but it doesn't work at all for standard ones like GAS or Notary.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
One signer in the cfg is enough.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
That's the reason #2230 and #2263 were not detected earlier, we actually had
Global scope being used before reconnection to RPC node.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
[#239] morph/client: Simplify code interacting with magic numbers
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
d29b13454f
It can't be uint64 in fact, but this error is buried deeply in the NetworkInfo
API structure, so we're not touching MagicNumber() for now.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Drop duplicate entities.
Format entities.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
[#240] logs: Factor out common service log messages
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
cffcc7745e
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
[#100] adm: Use netmap constants from pkg
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
299b6a6938
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
[#246] .gitattributes: Do not show diff for go.sum
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
adcfce39cf
When we update dependencies it can be rather big. However it is
generated automatically with `go mod tidy`, no need to review.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
[#139] Refactor blobovnicza exist test to not use chmod
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
ci/woodpecker/push/pre-commit Pipeline was successful
41eb3129ae
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
[#238] node: Read cfg from dir even if cfg file not set
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
995db117d0
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Add tracing spans for PUT requests.
Add tracing spans for DELETE requests.
Add tracing spans for SELECT requests.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#242] treesvc: Add tracing spans
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
6121b541b5
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
[#247] logs: Drop reputation log messages
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
ci/woodpecker/pr/pre-commit Pipeline was successful
b453bb754c
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
Drop the code that was expected to work with global reputation network
parameters.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
[#247] client: Drop reputation related RPCs
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
ci/woodpecker/push/pre-commit Pipeline was successful
070154d506
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
[#250] control: remove DumpShard and RestoreShard RPC
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
8466894fdf
We have `Evacuate` with a cleaner interface.
Also, remove them from CLI and engine.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
[#254] innerring: Remove unused TimersHandlers() method from processors
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
ci/woodpecker/push/pre-commit Pipeline was successful
0b42a00a60
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
[#256] blobovniczaTree: Make Exists test stable
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
ci/woodpecker/push/pre-commit Pipeline was successful
262c9c2b93
Corrupt and request _the same_ file.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#249] logs: Drop unused consts
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
ci/woodpecker/push/pre-commit Pipeline was successful
7b981bfe97
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
dkirillov force-pushed KirillovDenis/poc/impersonate from 6218ca502b to 36bc368a3d 2023-04-18 14:51:19 +00:00 Compare
dkirillov force-pushed KirillovDenis/poc/impersonate from 36bc368a3d to de6f94ef9f 2023-04-18 14:56:57 +00:00 Compare
dkirillov changed title from WIP: Poc/impersonate to Poc/impersonate 2023-04-18 14:58:06 +00:00
dkirillov requested review from storage-core-committers 2023-04-18 14:58:20 +00:00
dkirillov requested review from storage-core-developers 2023-04-18 14:58:21 +00:00
dkirillov changed title from Poc/impersonate to Support impersonation flag in bearer token 2023-04-18 14:59:57 +00:00
carpawell reviewed 2023-04-18 15:50:12 +00:00
carpawell left a comment
Contributor

Just in case: should the PR have so variable committers list?

Just in case: should the PR have so variable committers list?
@ -61,0 +65,4 @@
if err = bt.Unmarshal(rawBearer); err != nil {
return eACLErr(eaclOp, fmt.Errorf("invalid bearer token: %w", err))
}
if !bt.VerifySignature() {
Contributor

why check order has been changed? signature verification is more complex thing, i think

why check order has been changed? signature verification is more complex thing, i think
@ -179,2 +196,4 @@
}
if bt != nil && bt.Impersonate() {
pub, err = keys.NewPublicKeyFromBytes(bt.SigningKeyBytes(), elliptic.P256())
Contributor

can we not duplicate NewPublicKeyFromBytes and elliptic.P256() calls in that func?

can we not duplicate `NewPublicKeyFromBytes` and `elliptic.P256()` calls in that func?
dkirillov force-pushed KirillovDenis/poc/impersonate from de6f94ef9f to cd0003cb90 2023-04-19 06:20:36 +00:00 Compare
fyrchik reviewed 2023-04-20 15:27:33 +00:00
@ -115,1 +116,4 @@
if r.bearer != nil && r.bearer.Impersonate() {
issuer := bearer.ResolveIssuer(*r.bearer)
pubKey, err := keys.NewPublicKeyFromBytes(r.bearer.SigningKeyBytes(), elliptic.P256())
Owner

We already unmarshal key inside ResolveIssuer and also do it here. Maybe it makes sense to inline ResolveIssuer?
This may be an API issue for SDK as well.

We already unmarshal key inside `ResolveIssuer` and also do it here. Maybe it makes sense to inline `ResolveIssuer`? This may be an API issue for SDK as well.
@ -104,0 +106,4 @@
if bt.Impersonate() {
tbCore, err := s.eaclSource.GetEACL(cid)
if err != nil {
if client.IsErrEACLNotFound(err) {
Owner

Hm, is gocognit linter happy with this if tower?

Hm, is gocognit linter happy with this if tower?
dkirillov force-pushed KirillovDenis/poc/impersonate from cd0003cb90 to c197335b0f 2023-04-21 08:41:54 +00:00 Compare
dkirillov force-pushed KirillovDenis/poc/impersonate from c197335b0f to 0608545f6b 2023-04-21 13:58:30 +00:00 Compare
acid-ant approved these changes 2023-04-21 14:01:34 +00:00
dkirillov force-pushed KirillovDenis/poc/impersonate from 0608545f6b to 700f39c3f8 2023-04-26 07:24:00 +00:00 Compare
fyrchik approved these changes 2023-04-26 07:26:23 +00:00
fyrchik merged commit 700f39c3f8 into master 2023-04-26 07:26:48 +00:00
fyrchik deleted branch KirillovDenis/poc/impersonate 2023-04-26 07:26:49 +00:00
Sign in to join this conversation.
No reviewers
TrueCloudLab/storage-core-developers
No milestone
No project
No assignees
5 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: TrueCloudLab/frostfs-node#68
No description provided.