Commit Graph

4532 Commits (7cc368e188dc03e761cad0801e7f269a81b1ed83)

Author SHA1 Message Date
Airat Arifullin 7cc368e188 [#986] object: Introduce soft ape checks
* Soft APE check means that APE should allow request even
  it gets status NoRuleFound for a request. Otherwise,
  it is interpreted as Deny.
* Soft APE check is performed if basic ACL mask is not set.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-02-28 19:05:57 +00:00
Anton Nikiforov bc9dbb26ec [#932] adm: Add custom `Actor` to sign tx by all committee accounts
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-28 18:57:16 +00:00
Anton Nikiforov 61c58e2f92 [#932] adm: Add commands to manipulate with `NNS` contract
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-28 18:57:16 +00:00
Anton Nikiforov 9801d08438 [#932] adm: Move defaults for NNS to package `constants`
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-28 18:57:16 +00:00
Dmitrii Stepanov 918613546f [#1008] metabase: Do not update storageID on put
DCO action / DCO (pull_request) Successful in 4m56s Details
Build / Build Components (1.21) (pull_request) Successful in 7m52s Details
Build / Build Components (1.20) (pull_request) Successful in 8m11s Details
Vulncheck / Vulncheck (pull_request) Successful in 8m9s Details
Tests and linters / Staticcheck (pull_request) Successful in 9m58s Details
Tests and linters / Lint (pull_request) Successful in 10m42s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 12m14s Details
Tests and linters / Tests with -race (pull_request) Successful in 12m17s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 12m34s Details
There may be a race condition between put an object and
flushing the writecache:
1. Put object to the writecache
2. Writecache flushes object to the blobstore and sets blobstore's
storageID
3. Put object to the metabase, set writecache's storageID

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-28 11:01:50 +03:00
Dmitrii Stepanov 2ad433dbcb [#1005] engine: Drop shards weights
DCO action / DCO (pull_request) Successful in 4m7s Details
Vulncheck / Vulncheck (pull_request) Successful in 4m53s Details
Build / Build Components (1.21) (pull_request) Successful in 5m46s Details
Build / Build Components (1.20) (pull_request) Successful in 6m21s Details
Tests and linters / Staticcheck (pull_request) Successful in 7m45s Details
Tests and linters / Lint (pull_request) Successful in 8m44s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 13m1s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 15m42s Details
Tests and linters / Tests with -race (pull_request) Successful in 16m10s Details
Unused.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-26 17:25:05 +03:00
Evgenii Stratonikov abea258b65 [#1000] adm: Use default batch size for TraverseIterator()
Nothing is broken now, but will easily become if we change nnsMaxTokens,
thus this change.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-23 06:24:40 +00:00
Evgenii Stratonikov 4b13b85173 [#1000] morph: Fix batch size in TraverseIterator()
Initial prefetch size can be arbitrary an restricted only by VM/RPC
limits. For TraverseIterator() there is an explicit check on the
server-side, though.
Introduced in df055fead5.
Refs #931.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-23 06:24:40 +00:00
Evgenii Stratonikov e18f0f5178 [#999] morph: Use Global scope for proxy contract
DCO action / DCO (pull_request) Successful in 2m28s Details
Vulncheck / Vulncheck (pull_request) Successful in 2m52s Details
Build / Build Components (1.21) (pull_request) Successful in 4m5s Details
Build / Build Components (1.20) (pull_request) Successful in 4m12s Details
Tests and linters / Lint (pull_request) Successful in 6m10s Details
Tests and linters / Staticcheck (pull_request) Successful in 5m59s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 8m24s Details
Tests and linters / Tests with -race (pull_request) Successful in 8m38s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 8m58s Details
Proxy contract can now be used as an owner of NNS domains, thus we need
it not only to pay for the transaction but also to check domain
ownership. CalledByEntry is not enough, because we may register NNS
domains owned by proxy indirectly from the container contract.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-21 14:28:42 +03:00
Dmitrii Stepanov 7470c383dd [#997] metabase: Drop toMoveIt bucket
DCO action / DCO (pull_request) Successful in 2m9s Details
Vulncheck / Vulncheck (pull_request) Successful in 3m3s Details
Build / Build Components (1.20) (pull_request) Successful in 3m56s Details
Build / Build Components (1.21) (pull_request) Successful in 3m54s Details
Tests and linters / Staticcheck (pull_request) Successful in 5m26s Details
Tests and linters / Lint (pull_request) Successful in 6m19s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 8m6s Details
Tests and linters / Tests with -race (pull_request) Successful in 8m20s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 8m39s Details
It is not used.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-21 10:06:05 +03:00
Evgenii Stratonikov adf7ebab5b [#996] metabase: Speed up bucket creation
Most of the time it exits, e.g. when it is per-container and use on each
object PUT. Bbolt implementation first tries to create bucket and then
returns it if it exists. Create operation uses cursor and thus is not
very lightweight, we can avoid it.

```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/frostfs-node/pkg/local_object_storage/metabase
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
                 │     old     │                new                 │
                 │   sec/op    │   sec/op     vs base               │
Put/parallel-8     174.4µ ± 3%   163.3µ ± 3%  -6.39% (p=0.000 n=10)
Put/sequential-8   263.3µ ± 2%   259.0µ ± 1%  -1.64% (p=0.000 n=10)
geomean            214.3µ        205.6µ       -4.05%

                 │     old      │                 new                 │
                 │     B/op     │     B/op      vs base               │
Put/parallel-8     275.3Ki ± 3%   281.1Ki ± 4%       ~ (p=0.063 n=10)
Put/sequential-8   413.0Ki ± 2%   426.6Ki ± 2%  +3.29% (p=0.003 n=10)
geomean            337.2Ki        346.3Ki       +2.70%

                 │     old     │                 new                 │
                 │  allocs/op  │  allocs/op   vs base                │
Put/parallel-8      678.0 ± 1%    524.5 ± 2%  -22.64% (p=0.000 n=10)
Put/sequential-8   1.329k ± 0%   1.183k ± 0%  -10.91% (p=0.000 n=10)
geomean             949.1         787.9       -16.98%
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-20 15:42:58 +00:00
Airat Arifullin 47d9ce71be [#986] cli: Allow add-rule command to parse new actions
* Introduce Object.* and Container.* actions that
  span all methods for services.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-02-20 07:42:29 +00:00
Airat Arifullin 0f064b7962 [#989] util: Introduce any and all statements for ape rule parsing
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-02-20 07:42:29 +00:00
Airat Arifullin 613e11c4d2 [#989] adm: Read and parse chains from file
* Slightly fix the approach to read encoded chain from file
  in frostfs-adm.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-02-20 07:42:29 +00:00
Airat Arifullin 9611710e19 [#989] cli: Read and parse chains from file
* Introduce path flag to make add-rule command read and parse
  chain from file. File is binary/JSON-encoded chain.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-02-20 07:42:29 +00:00
Evgenii Stratonikov 9adcb253be Revert "[#972] go.mod: Bump go version to go1.21"
This reverts commit e39a714c25.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-19 15:36:01 +00:00
Evgenii Stratonikov 9f68305c2e Revert "[#972] pilorama: Remove removeDuplicatesInPlace()"
This reverts commit 45fd4e4ff1.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-19 15:36:01 +00:00
Evgenii Stratonikov dad56d2e98 Revert "[#972] Use min/max builtins"
This reverts commit 89784b2e0a.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-19 15:36:01 +00:00
Evgenii Stratonikov 4bfc6d29b9 Revert "[#972] Adopt slices.BinarySearch()"
This reverts commit d2f13a29de.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-19 15:36:01 +00:00
Evgenii Stratonikov 7627d08914 Revert "[#972] Use require.ElementsMatch() where possible"
This reverts commit 6d9707ff1f.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-19 15:36:01 +00:00
Evgenii Stratonikov 3359349acb Revert "[#972] Use slices.Sort* when useful"
This reverts commit b871d7a5e8.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-19 15:36:01 +00:00
Evgenii Stratonikov 946f2ec2bf Revert "[#972] Drop x/exp/slices dependency"
This reverts commit f3e50772fd.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-19 15:36:01 +00:00
Evgenii Stratonikov 9e55836da5 [#994] cli: Output pilorama path in `shards list`
Do it for JSON too, not only for human output.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-19 15:19:47 +00:00
Dmitrii Stepanov 13d5cd3e21 [#991] logger: Fix journald logger
DCO action / DCO (pull_request) Successful in 1m7s Details
Vulncheck / Vulncheck (pull_request) Successful in 3m32s Details
Build / Build Components (1.22) (pull_request) Successful in 3m52s Details
Build / Build Components (1.21) (pull_request) Successful in 3m58s Details
Tests and linters / Lint (pull_request) Successful in 5m13s Details
Tests and linters / Tests (1.22) (pull_request) Successful in 7m22s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 7m34s Details
Tests and linters / Tests with -race (pull_request) Successful in 7m32s Details
Tests and linters / Staticcheck (pull_request) Successful in 2m22s Details
Allow to change logger level.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-19 16:18:13 +03:00
Evgenii Stratonikov f3e50772fd [#972] Drop x/exp/slices dependency
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-19 13:13:09 +00:00
Evgenii Stratonikov b871d7a5e8 [#972] Use slices.Sort* when useful
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-19 13:13:09 +00:00
Evgenii Stratonikov 6d9707ff1f [#972] Use require.ElementsMatch() where possible
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-19 13:13:09 +00:00
Evgenii Stratonikov d2f13a29de [#972] Adopt slices.BinarySearch()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-19 13:13:09 +00:00
Evgenii Stratonikov 89784b2e0a [#972] Use min/max builtins
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-19 13:13:09 +00:00
Evgenii Stratonikov 45fd4e4ff1 [#972] pilorama: Remove removeDuplicatesInPlace()
Also, check that slices.CompareFunc() indeed passes all the tests before
removal.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-19 13:13:09 +00:00
Evgenii Stratonikov e39a714c25 [#972] go.mod: Bump go version to go1.21
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-19 13:13:09 +00:00
Evgenii Stratonikov 15fc5bac26 [#972] keyer: Use UncompressedBytes() for marshaling
elliptic.Marshal() becomes deprecated in go1.21

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-19 13:13:09 +00:00
Dmitrii Stepanov 2680192ba0 [#988] objectSvc: Fix `SetMarshalData` for PutSingle
DCO action / DCO (pull_request) Successful in 5m10s Details
Vulncheck / Vulncheck (pull_request) Successful in 5m24s Details
Build / Build Components (1.21) (pull_request) Successful in 7m40s Details
Build / Build Components (1.20) (pull_request) Successful in 7m50s Details
Tests and linters / Staticcheck (pull_request) Successful in 7m52s Details
Tests and linters / Lint (pull_request) Successful in 9m8s Details
Tests and linters / Tests with -race (pull_request) Successful in 10m54s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 11m12s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 11m7s Details
After api-go update it is required to pass marshal data
to `SetMarshalData`.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-15 17:21:08 +03:00
Dmitrii Stepanov 6bafdab004 [#988] go.mod: Update api-go and sdk-go versions
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-15 17:20:59 +03:00
Anton Nikiforov 05b5f5ca85 [#959] writecache: Fix panic on `Get` when it is not initialized
Vulncheck / Vulncheck (pull_request) Successful in 2m41s Details
DCO action / DCO (pull_request) Successful in 2m43s Details
Tests and linters / Staticcheck (pull_request) Successful in 3m55s Details
Build / Build Components (1.21) (pull_request) Successful in 3m45s Details
Build / Build Components (1.20) (pull_request) Successful in 4m19s Details
Tests and linters / Lint (pull_request) Successful in 5m30s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 7m57s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 8m22s Details
Tests and linters / Tests with -race (pull_request) Successful in 8m39s Details
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-14 16:10:33 +03:00
Anton Nikiforov 2429508ac5 [#959] shard: Skip rebuild in `DEGRADED` mode
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-14 15:39:28 +03:00
Anton Nikiforov 5cbf57081f [#980] adm: Introduce flag `chain-name` for APE managing commands
DCO action / DCO (pull_request) Successful in 3m14s Details
Vulncheck / Vulncheck (pull_request) Successful in 3m10s Details
Build / Build Components (1.21) (pull_request) Successful in 4m5s Details
Build / Build Components (1.20) (pull_request) Successful in 4m10s Details
Tests and linters / Staticcheck (pull_request) Successful in 4m30s Details
Tests and linters / Lint (pull_request) Successful in 5m38s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 7m32s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 8m7s Details
Tests and linters / Tests with -race (pull_request) Successful in 8m50s Details
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-14 09:41:56 +03:00
Anton Nikiforov 35370283ba [#948] adm: Move `TestNextPollInterval` to package `helper`
Build / Build Components (1.20) (pull_request) Successful in 3m18s Details
DCO action / DCO (pull_request) Successful in 3m26s Details
Vulncheck / Vulncheck (pull_request) Successful in 3m23s Details
Build / Build Components (1.21) (pull_request) Successful in 3m58s Details
Tests and linters / Staticcheck (pull_request) Successful in 5m14s Details
Tests and linters / Lint (pull_request) Successful in 5m54s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 9m24s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 9m33s Details
Tests and linters / Tests with -race (pull_request) Successful in 10m36s Details
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-13 10:08:49 +03:00
Anton Nikiforov 802192cfef [#932] adm: Rename `util` to `helper`
Vulncheck / Vulncheck (pull_request) Successful in 1m57s Details
DCO action / DCO (pull_request) Successful in 3m3s Details
Build / Build Components (1.21) (pull_request) Successful in 5m0s Details
Build / Build Components (1.20) (pull_request) Successful in 5m6s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 7m1s Details
Tests and linters / Staticcheck (pull_request) Successful in 6m55s Details
Tests and linters / Lint (pull_request) Successful in 7m30s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 7m53s Details
Tests and linters / Tests with -race (pull_request) Successful in 8m21s Details
To avoid conflicts with `util` packages in other imports.

Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-13 09:59:27 +03:00
Anton Nikiforov e2cee4cf09 [#932] adm: Move `const` to package `constants`
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-13 09:59:27 +03:00
Anton Nikiforov 814c411f4a [#932] adm: Move flags to package `commonflags`
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-13 09:59:27 +03:00
Anton Nikiforov 7b0e3f5010 [#932] adm: Remove unnecessary import alias usage
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-13 09:59:27 +03:00
Anton Nikiforov 63c34ea707 [#932] adm: Move command `init` to package `initialize`
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-13 09:59:27 +03:00
Anton Nikiforov 86b2515744 [#932] adm: Move `generate.go` to package `generate`
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-13 09:59:27 +03:00
Anton Nikiforov b8cf0a6b88 [#932] adm: Move `deploy` to package `contract`
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-13 09:59:27 +03:00
Anton Nikiforov 76343f19e5 [#932] adm: Move `update-contracts` to package `contract`
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-13 09:59:27 +03:00
Anton Nikiforov e2557b2f0b [#932] adm: Move `dump-hashes` to package `contract`
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-13 09:59:27 +03:00
Anton Nikiforov 9b65f1595a [#932] adm: Move `force-new-epoch` to package `netmap`
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-13 09:59:27 +03:00
Anton Nikiforov ce42547980 [#932] adm: Move `remove-nodes` to package `node`
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-13 09:59:27 +03:00
Anton Nikiforov 9690bd02aa [#932] adm: Move `netmap-candidates` to package `netmap`
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-13 09:59:27 +03:00