Commit graph

167 commits

Author SHA1 Message Date
becacd4251 [#153] selector: Add agent synchronization in Oneshot mode
Some checks failed
DCO action / DCO (pull_request) Failing after 54s
Tests and linters / Tests (1.22) (pull_request) Successful in 1m45s
Tests and linters / Lint (pull_request) Successful in 1m55s
Tests and linters / Tests with -race (pull_request) Successful in 2m17s
Tests and linters / Tests (1.21) (pull_request) Successful in 2m38s
2024-07-29 17:00:49 +03:00
9b9db46a07 [#152] Allow to set mix of policies for containers and buckets
All checks were successful
DCO action / DCO (pull_request) Successful in 43s
Tests and linters / Tests (1.21) (pull_request) Successful in 3m10s
Tests and linters / Tests (1.22) (pull_request) Successful in 3m21s
Tests and linters / Tests with -race (pull_request) Successful in 3m34s
Tests and linters / Lint (pull_request) Successful in 2m56s
Signed-off-by: a.berezin <a.berezin@yadro.com>
2024-07-02 20:45:31 +03:00
335c45c578 [#149] selector: Add read timeout
All checks were successful
DCO action / DCO (pull_request) Successful in 1m15s
Tests and linters / Tests (1.22) (pull_request) Successful in 3m12s
Tests and linters / Tests with -race (pull_request) Successful in 3m26s
Tests and linters / Tests (1.21) (pull_request) Successful in 3m39s
Tests and linters / Lint (pull_request) Successful in 4m21s
If there are no objects for 10 second, then return nil.
It is required to prevent VU iteration hang if there
are no objects pushed to registry.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-06-24 13:59:12 +03:00
e7d4dd404a [#150] scenarios: Use SelectorAwaiting for read and delete load in s3_dar.js, make delete_age optional
All checks were successful
DCO action / DCO (pull_request) Successful in 1m28s
Tests and linters / Tests (1.22) (pull_request) Successful in 3m14s
Tests and linters / Tests with -race (pull_request) Successful in 3m27s
Tests and linters / Tests (1.21) (pull_request) Successful in 3m40s
Tests and linters / Lint (pull_request) Successful in 4m29s
Signed-off-by: m.malygina <m.malygina@yadro.com>
2024-06-21 15:52:57 +03:00
0a9aeab47c [#150] In case we are running both read and delete load SelectorAwaiting
All checks were successful
DCO action / DCO (pull_request) Successful in 2m2s
Tests and linters / Tests (1.21) (pull_request) Successful in 4m31s
Tests and linters / Tests (1.22) (pull_request) Successful in 4m39s
Tests and linters / Tests with -race (pull_request) Successful in 4m43s
Tests and linters / Lint (pull_request) Successful in 5m33s
Signed-off-by: m.malygina <m.malygina@yadro.com>
2024-06-21 10:55:18 +03:00
3bc1229062 [#146] native: Add NetworkInfo cache
All checks were successful
DCO action / DCO (pull_request) Successful in 1m18s
Tests and linters / Tests (1.22) (pull_request) Successful in 3m24s
Tests and linters / Tests with -race (pull_request) Successful in 3m36s
Tests and linters / Tests (1.21) (pull_request) Successful in 3m46s
Tests and linters / Lint (pull_request) Successful in 4m25s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-06-17 09:32:20 +03:00
e92ce668a8 [#145] scenarios: Format js files with clang
All checks were successful
DCO action / DCO (pull_request) Successful in 1m18s
Tests and linters / Tests (1.22) (pull_request) Successful in 3m10s
Tests and linters / Tests (1.21) (pull_request) Successful in 3m27s
Tests and linters / Tests with -race (pull_request) Successful in 3m28s
Tests and linters / Lint (pull_request) Successful in 4m29s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-06-14 11:26:01 +03:00
6d1e7eb49e [#145] native: Allow to specify max_obj_size
All checks were successful
DCO action / DCO (pull_request) Successful in 1m11s
Tests and linters / Tests (1.22) (pull_request) Successful in 3m12s
Tests and linters / Tests with -race (pull_request) Successful in 3m26s
Tests and linters / Tests (1.21) (pull_request) Successful in 3m39s
Tests and linters / Lint (pull_request) Successful in 4m13s
For locally prepared objects it is possible now to
specify cut size.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-06-14 10:57:30 +03:00
f90a645594 [#144] registry: Add tests for obj registry exporter
All checks were successful
DCO action / DCO (pull_request) Successful in 57s
Tests and linters / Tests (1.21) (pull_request) Successful in 3m14s
Tests and linters / Tests (1.22) (pull_request) Successful in 3m15s
Tests and linters / Tests with -race (pull_request) Successful in 3m20s
Tests and linters / Lint (pull_request) Successful in 4m14s
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-06-11 18:09:20 +03:00
3f67606f02 [#144] registry: Fix string generator func in obj info test
`rune` is alias for `int32`. `randString()` used `rand.Int()`
which returns system-specific non-negative integer number. For
64-bit systems it will be int64. An attempt to cast `int64` to
`int32` (`rune`) leads the latter to be a negative number in
case of overflow. This caused the resulting string containing
unexpected symbols.

Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-06-11 17:43:15 +03:00
bdf4c192e1 [#143] Add new s3 scenario where delete only already read objects
Some checks reported warnings
DCO action / DCO (pull_request) Has been cancelled
Tests and linters / Lint (pull_request) Has been cancelled
Tests and linters / Tests (1.21) (pull_request) Has been cancelled
Tests and linters / Tests (1.22) (pull_request) Has been cancelled
Tests and linters / Tests with -race (pull_request) Has been cancelled
Signed-off-by: m.malygina <m.malygina@yadro.com>
2024-06-06 18:37:10 +03:00
3dd559a7b1 [#139] Allow registry to export gRPC objects
All checks were successful
DCO action / DCO (pull_request) Successful in 1m0s
Tests and linters / Tests (1.22) (pull_request) Successful in 2m26s
Tests and linters / Tests with -race (pull_request) Successful in 2m42s
Tests and linters / Tests (1.21) (pull_request) Successful in 2m57s
Tests and linters / Lint (pull_request) Successful in 3m15s
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-06-04 15:05:52 +03:00
4aaa50c8ed [#141] registry: Use Batch() for all methods
All checks were successful
DCO action / DCO (pull_request) Successful in 1m34s
Tests and linters / Tests (1.21) (pull_request) Successful in 2m42s
Tests and linters / Tests (1.22) (pull_request) Successful in 2m47s
Tests and linters / Lint (pull_request) Successful in 4m11s
Tests and linters / Tests with -race (pull_request) Successful in 4m41s
Refs #136

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-05-30 15:33:32 +03:00
de61aef66e [#140] native,s3: Do not set Success flag on hash mismatch
All checks were successful
DCO action / DCO (pull_request) Successful in 1m4s
Tests and linters / Tests (1.21) (pull_request) Successful in 2m0s
Tests and linters / Tests (1.22) (pull_request) Successful in 2m19s
Tests and linters / Lint (pull_request) Successful in 3m32s
Tests and linters / Tests with -race (pull_request) Successful in 3m52s
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-05-30 14:00:17 +03:00
31fac75743 [#138] scenarios: Bump version for node, s3-gw, sdk-go, neo-go
All checks were successful
DCO action / DCO (pull_request) Successful in 58s
Tests and linters / Tests (1.22) (pull_request) Successful in 2m4s
Tests and linters / Tests (1.21) (pull_request) Successful in 2m33s
Tests and linters / Tests with -race (pull_request) Successful in 3m31s
Tests and linters / Lint (pull_request) Successful in 7m33s
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-05-06 10:12:28 +03:00
b5c7c01a11 [#138] go.mod: Bump go version to 1.21
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-05-05 21:29:00 +03:00
e5af4112f9 [#136] registry: Apply updates in batch
All checks were successful
DCO action / DCO (pull_request) Successful in 1m7s
Tests and linters / Tests (1.21) (pull_request) Successful in 2m27s
Tests and linters / Tests (1.20) (pull_request) Successful in 3m0s
Tests and linters / Tests with -race (pull_request) Successful in 3m19s
Tests and linters / Lint (pull_request) Successful in 4m33s
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-05-03 13:56:59 +03:00
7f139734b1 [#133] scenarios: Support one-shot deletion scenario
All checks were successful
DCO action / DCO (pull_request) Successful in 1m56s
Tests and linters / Tests (1.20) (pull_request) Successful in 3m50s
Tests and linters / Tests (1.21) (pull_request) Successful in 4m14s
Tests and linters / Lint (pull_request) Successful in 4m49s
Tests and linters / Tests with -race (pull_request) Successful in 4m40s
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-05 13:43:01 +03:00
86ed8add10 [#133] scenarios: Format with clang-format
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-05 13:43:01 +03:00
87ffb551b6 [#133] registry: Implement oneshot selector
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-05 13:43:01 +03:00
d1ec9e4bf0 [#108] preset: Allow to skip preset
All checks were successful
DCO action / DCO (pull_request) Successful in 37s
Tests and linters / Tests (1.21) (pull_request) Successful in 3m0s
Tests and linters / Tests (1.20) (pull_request) Successful in 3m16s
Tests and linters / Tests with -race (pull_request) Successful in 3m40s
Tests and linters / Lint (pull_request) Successful in 2m38s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-04-01 16:10:04 +03:00
0a6e51ccc9 [#131] registry: Gofumpt
All checks were successful
DCO action / DCO (pull_request) Successful in 1m11s
Tests and linters / Tests (1.21) (pull_request) Successful in 2m20s
Tests and linters / Tests (1.20) (pull_request) Successful in 2m33s
Tests and linters / Lint (pull_request) Successful in 2m55s
Tests and linters / Tests with -race (pull_request) Successful in 3m18s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-03-20 11:41:03 +03:00
93aaec4e0d [#131] scenarios: Format js files with clang formatter
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-03-20 11:40:49 +03:00
0c4e2665ba [#131] registry: Allow to create cycled/forward selectors
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-03-20 11:40:49 +03:00
2d26ac766f [#130] Update documentation
All checks were successful
DCO action / DCO (pull_request) Successful in 1m41s
Tests and linters / Tests (1.20) (pull_request) Successful in 2m10s
Tests and linters / Tests (1.21) (pull_request) Successful in 2m12s
Tests and linters / Lint (pull_request) Successful in 2m34s
Tests and linters / Tests with -race (pull_request) Successful in 3m5s
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-22 16:31:29 +03:00
adacef19bb [#125] Fix recursion issue
Some checks reported warnings
DCO action / DCO (pull_request) Has been cancelled
Tests and linters / Lint (pull_request) Has been cancelled
Tests and linters / Tests (1.20) (pull_request) Has been cancelled
Tests and linters / Tests (1.21) (pull_request) Has been cancelled
Tests and linters / Tests with -race (pull_request) Has been cancelled
Signed-off-by: Andrey Berezin <a.berezin@yadro.com>
2024-02-15 16:40:18 +03:00
d1578a728f [#121] scenarios: Fix tags doc
All checks were successful
DCO action / DCO (pull_request) Successful in 3m49s
Tests and linters / Tests (1.21) (pull_request) Successful in 6m18s
Tests and linters / Tests with -race (pull_request) Successful in 6m56s
Tests and linters / Tests (1.20) (pull_request) Successful in 2m53s
Tests and linters / Lint (pull_request) Successful in 2m25s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-08 10:10:09 +03:00
5cfb958a18 [#121] scenarios: Add info about Grafana annotations
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-08 10:10:09 +03:00
47fc031028 [#125] Adding acl to container and bucket creation
Some checks reported warnings
DCO action / DCO (pull_request) Has been cancelled
Tests and linters / Lint (pull_request) Has been cancelled
Tests and linters / Tests (1.20) (pull_request) Has been cancelled
Tests and linters / Tests (1.21) (pull_request) Has been cancelled
Tests and linters / Tests with -race (pull_request) Has been cancelled
Signed-off-by: Andrey Berezin <a.berezin@yadro.com>
2024-02-05 18:56:36 +03:00
965dcdcbe7 [#119] scenarios: Add Metrics export section to docs
All checks were successful
DCO action / DCO (pull_request) Successful in 52s
Tests and linters / Lint (pull_request) Successful in 2m14s
Tests and linters / Tests (1.21) (pull_request) Successful in 2m7s
Tests and linters / Tests with -race (pull_request) Successful in 9m56s
Tests and linters / Tests (1.20) (pull_request) Successful in 11m43s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-26 10:39:26 +03:00
e9edca3e79 [#119] metrics: Refactor custom metrics
Add `data` metrics to measure payload rate.
Rename `total` metrics to `success`, because these metrics count
success operations count but not total operations count.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-26 10:39:26 +03:00
604982de3e [#119] metrics: Allow to add custom tags
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-26 10:31:54 +03:00
029af2a865 [#114] local: Fix payload type after recent refactoring
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-01-24 16:15:24 +00:00
4ff87f9bf6 [#122] preset_grpc: Allow specify --local without value
All checks were successful
DCO action / DCO (pull_request) Successful in 1m17s
Tests and linters / Lint (pull_request) Successful in 3m40s
Tests and linters / Tests (1.20) (pull_request) Successful in 3m45s
Tests and linters / Tests (1.21) (pull_request) Successful in 3m44s
Tests and linters / Tests with -race (pull_request) Successful in 4m7s
Now `--local` is flag, so it is possible to specify `--local`
instead of `--local True` or '--local False'.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-24 16:28:44 +03:00
339e4e52ec [#114] .forgejo: Add golanci-lint workflow
All checks were successful
DCO action / DCO (pull_request) Successful in 1m7s
Tests and linters / Tests (1.21) (pull_request) Successful in 2m13s
Tests and linters / Tests (1.20) (pull_request) Successful in 2m28s
Tests and linters / Tests with -race (pull_request) Successful in 3m16s
Tests and linters / Lint (pull_request) Successful in 4m25s
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-01-12 19:35:26 +03:00
636a1e9290 [#114] internal: Resolve linter issues
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-01-12 19:29:16 +03:00
d8af19cc83 [#114] datagen: Remove calcHash parameter in GenPayload()
Hash calculation is now done on-demand with a method call.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-01-12 19:29:16 +03:00
4544ec616b [#114] datagen: Allow to generate streaming payloads
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-01-12 19:29:16 +03:00
74121bb387 [#114] datagen: Refactor payload generation
Return an interface which can be converted to `io.Reader` as well as
report payload size and hash.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-01-12 19:29:16 +03:00
17ace8a73d [#117] .gitignore: Add __pycache__
All checks were successful
DCO action / DCO (pull_request) Successful in 1m14s
Tests and linters / Tests (1.20) (pull_request) Successful in 2m24s
Tests and linters / Tests (1.21) (pull_request) Successful in 2m33s
Tests and linters / Tests with -race (pull_request) Successful in 3m57s
It is created after preset script execution.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-01-12 13:19:03 +03:00
14a5eac5b1 [#117] scenarios: Refactor data generator construction
1. Make it easier to change new parameters.
2. Fix a bug where the generator was created even for read-only
   scenarios.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-01-12 13:19:03 +03:00
278b234753 [#117] scenarios: Provide all parameters to datagen.generator()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-01-12 13:18:59 +03:00
0e06020118 [#107] preset_grpc: Allow to create local containers
All checks were successful
DCO action / DCO (pull_request) Successful in 1m2s
Tests and linters / Tests (1.21) (pull_request) Successful in 1m53s
Tests and linters / Tests (1.20) (pull_request) Successful in 2m2s
Tests and linters / Tests with -race (pull_request) Successful in 3m0s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-15 16:57:05 +03:00
bc47d66316 [#106] xk6: Allow to set max total size in local scenarios
All checks were successful
DCO action / DCO (pull_request) Successful in 1m8s
Tests and linters / Tests (1.21) (pull_request) Successful in 1m49s
Tests and linters / Tests (1.20) (pull_request) Successful in 2m1s
Tests and linters / Tests with -race (pull_request) Successful in 3m7s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-15 14:07:35 +03:00
eeededfc18 [#106] go.mod: Update frostfs-node version
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-14 11:35:09 +03:00
3574361f2e [#104] s3local: Use default HTTP client instead of requests
All checks were successful
DCO action / DCO (pull_request) Successful in 1m51s
Tests and linters / Tests (1.21) (pull_request) Successful in 2m55s
Tests and linters / Tests with -race (pull_request) Successful in 3m20s
Tests and linters / Tests (1.20) (pull_request) Successful in 3m33s
`requests` lib is not default, so it can be unavailable.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-17 14:33:32 +03:00
48a95bc50b remove http from s3 multipart upload load scenario, protocol would be set in endpoint parameter
Some checks reported warnings
DCO action / DCO (pull_request) Has been cancelled
Tests and linters / Tests (1.20) (pull_request) Has been cancelled
Tests and linters / Tests (1.21) (pull_request) Has been cancelled
Tests and linters / Tests with -race (pull_request) Has been cancelled
Signed-off-by: m.malygina <m.malygina@yadro.com>
2023-10-27 13:22:38 +03:00
26f5262b3d [#90] Support config folder together with config file
All checks were successful
Tests and linters / Tests (1.20) (pull_request) Successful in 1m46s
Tests and linters / Tests with -race (pull_request) Successful in 2m9s
DCO action / DCO (pull_request) Successful in 3m26s
Tests and linters / Tests (1.21) (pull_request) Successful in 8m10s
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-10-25 16:48:29 +03:00
95ce6f1162 [#96] .forgejo: Copy tests workflow from node
All checks were successful
Tests and linters / Tests (1.21) (pull_request) Successful in 2m28s
Tests and linters / Tests (1.20) (pull_request) Successful in 2m42s
Tests and linters / Tests with -race (pull_request) Successful in 2m51s
DCO action / DCO (pull_request) Successful in 3m18s
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-10-19 11:45:10 +03:00
27db0ac943 [#96] .forgejo: Fix DCO action
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-10-19 11:43:52 +03:00