Commit graph

131 commits

Author SHA1 Message Date
d8af19cc83 [#114] datagen: Remove calcHash parameter in GenPayload()
Hash calculation is now done on-demand with a method call.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-01-12 19:29:16 +03:00
4544ec616b [#114] datagen: Allow to generate streaming payloads
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-01-12 19:29:16 +03:00
74121bb387 [#114] datagen: Refactor payload generation
Return an interface which can be converted to `io.Reader` as well as
report payload size and hash.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-01-12 19:29:16 +03:00
17ace8a73d [#117] .gitignore: Add __pycache__
All checks were successful
DCO action / DCO (pull_request) Successful in 1m14s
Tests and linters / Tests (1.20) (pull_request) Successful in 2m24s
Tests and linters / Tests (1.21) (pull_request) Successful in 2m33s
Tests and linters / Tests with -race (pull_request) Successful in 3m57s
It is created after preset script execution.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-01-12 13:19:03 +03:00
14a5eac5b1 [#117] scenarios: Refactor data generator construction
1. Make it easier to change new parameters.
2. Fix a bug where the generator was created even for read-only
   scenarios.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-01-12 13:19:03 +03:00
278b234753 [#117] scenarios: Provide all parameters to datagen.generator()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-01-12 13:18:59 +03:00
0e06020118 [#107] preset_grpc: Allow to create local containers
All checks were successful
DCO action / DCO (pull_request) Successful in 1m2s
Tests and linters / Tests (1.21) (pull_request) Successful in 1m53s
Tests and linters / Tests (1.20) (pull_request) Successful in 2m2s
Tests and linters / Tests with -race (pull_request) Successful in 3m0s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-15 16:57:05 +03:00
bc47d66316 [#106] xk6: Allow to set max total size in local scenarios
All checks were successful
DCO action / DCO (pull_request) Successful in 1m8s
Tests and linters / Tests (1.21) (pull_request) Successful in 1m49s
Tests and linters / Tests (1.20) (pull_request) Successful in 2m1s
Tests and linters / Tests with -race (pull_request) Successful in 3m7s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-15 14:07:35 +03:00
eeededfc18 [#106] go.mod: Update frostfs-node version
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-14 11:35:09 +03:00
3574361f2e [#104] s3local: Use default HTTP client instead of requests
All checks were successful
DCO action / DCO (pull_request) Successful in 1m51s
Tests and linters / Tests (1.21) (pull_request) Successful in 2m55s
Tests and linters / Tests with -race (pull_request) Successful in 3m20s
Tests and linters / Tests (1.20) (pull_request) Successful in 3m33s
`requests` lib is not default, so it can be unavailable.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-17 14:33:32 +03:00
48a95bc50b remove http from s3 multipart upload load scenario, protocol would be set in endpoint parameter
Some checks reported warnings
DCO action / DCO (pull_request) Has been cancelled
Tests and linters / Tests (1.20) (pull_request) Has been cancelled
Tests and linters / Tests (1.21) (pull_request) Has been cancelled
Tests and linters / Tests with -race (pull_request) Has been cancelled
Signed-off-by: m.malygina <m.malygina@yadro.com>
2023-10-27 13:22:38 +03:00
26f5262b3d [#90] Support config folder together with config file
All checks were successful
Tests and linters / Tests (1.20) (pull_request) Successful in 1m46s
Tests and linters / Tests with -race (pull_request) Successful in 2m9s
DCO action / DCO (pull_request) Successful in 3m26s
Tests and linters / Tests (1.21) (pull_request) Successful in 8m10s
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-10-25 16:48:29 +03:00
95ce6f1162 [#96] .forgejo: Copy tests workflow from node
All checks were successful
Tests and linters / Tests (1.21) (pull_request) Successful in 2m28s
Tests and linters / Tests (1.20) (pull_request) Successful in 2m42s
Tests and linters / Tests with -race (pull_request) Successful in 2m51s
DCO action / DCO (pull_request) Successful in 3m18s
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-10-19 11:45:10 +03:00
27db0ac943 [#96] .forgejo: Fix DCO action
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-10-19 11:43:52 +03:00
e970e52eea [#96] .forgejo: Move workflows folder from .github
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-10-19 11:43:52 +03:00
1311051f60 [#99] Adding read age param to improve k6 runs stability
Signed-off-by: Andrey Berezin <a.berezin@yadro.com>
2023-10-02 20:08:43 +03:00
7db7751334 [#95] Allow to use wallet from config file for frostfs-cli
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-08-23 15:01:53 +03:00
bf884936a7 [#91] Improve logging for preset
Signed-off-by: Andrey Berezin <a.berezin@yadro.com>
2023-08-16 08:25:03 +00:00
108e761639 [#93] go.mod: Update go.k6.io/k6 package to patched version
* The update fixes bug with k6 build

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-08-11 18:52:14 +03:00
5b1793f248 [#30] report: log start and end time of load scenario
Signed-off-by: Airat Arifullin a.arifullin@yadro.com
2023-07-26 14:54:59 +00:00
4ef3795e04 [#84] preset: fix typo
Signed-off-by: Andrey Berezin <a.berezin@yadro.com>
2023-07-21 06:55:40 +00:00
704c0f06bc [#25] selector: Remove next object timeout
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-20 15:01:57 +03:00
0dc0ba1704 [#25] xk6: Read objects from registry for S3 tests
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-20 10:01:41 +03:00
3c26e7c917 [#25] xk6: Read objects from registry for gRPC tests
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-20 10:01:34 +03:00
50e2f55362 [#80] Add dump registry util
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-19 15:57:39 +03:00
77d3dd8d6e [#80] Support parallel multipart
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-19 10:44:44 +03:00
6182d47b43 [#81] remove schema from preset_s3 and k6 load s3 scenarios 2023-07-14 11:36:10 +00:00
ff6814e15d [#72] Add option --prepare-locally
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-07-07 13:16:54 +03:00
56235f5e90 [#72] Update dependencies
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-07-06 12:14:52 +03:00
f633f9a64a [#79] client: Remove bufSize field
Use constant value instead.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-06 11:27:33 +03:00
42f1881580 [#79] object put: Add chunk size parameter
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-06 11:27:33 +03:00
4972bb928e [#79] xk6: Update node and SDK-Go
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-05 15:37:06 +03:00
a1f5738d2f [#77] Use writecache in local scenarios
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-06-30 12:50:42 +00:00
8e99d08aa4 [#12] Allow using multiple endpoints for presets
Signed-off-by: Andrey Berezin <a.berezin@yadro.com>
2023-06-28 20:21:43 +03:00
ba04c682cb [#13] Allow to use english text in the payload
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-27 11:14:05 +00:00
3525d5b4e3 [#15] go.mod: Tidy
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-27 11:14:05 +00:00
62d7b78131 [#73] preset: Allow to sleep before putting objects
For large networks block propagation may take some time.
If we do not wait enough, putting objects can fail for some containers.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-25 13:14:15 +03:00
153390cedb [#65] go.mod: Move to go1.19
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-24 11:41:02 +03:00
1025e80f11 [#65] go.mod: Update dependencies
See https://github.com/grafana/k6/pull/3075/, it is not yet in any
release.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-24 11:41:02 +03:00
6151005b4d [#67] Fail k6 if preset fails
Signed-off-by: Andrey Berezin <a.berezin@yadro.com>
2023-05-24 08:39:37 +00:00
925fe3ec83 [#66] scenarios: Exit if there is nothing to verify
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-23 16:02:12 +03:00
4aa9a359b5 [#64] registry: Delete object from the old bucket
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-23 16:02:03 +03:00
52ed0d6d88 [#63] scenarios: Unify logs in verify script
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-23 16:02:03 +03:00
4c2678077b [#57] preset: Use temporary file for payload
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-22 09:06:10 +00:00
5a1191a1ab [#20] Add pprof extension with support for cpu and mem
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-05-18 15:08:33 +00:00
5c26b4bad4 [#61] scenarios: Fix setObjectStatus parameters
We were providing new status instead of old and expected it to exist in
DB.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-18 09:43:41 +03:00
e71ac35c36 [#42] registry: Do not create selectors without status filters
Should allow to catch bugs in scenarios earlier.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-15 16:08:26 +03:00
b66b5a2f37 [#42] registry: Optimize Count() for ObjSelector
It is the heaviest function executing on setup stage.
The culprit is the linear dependency between its execution
time and the amount of objects in registry. The solution is to store
object by status. While the optimization doesn't work for objects with
no status, it is currently provided by all scenarios.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-15 16:08:26 +03:00
22d7996f79 [#42] registry: Optimize ObjectInfo marshaling
1. Get rid of JSON in the database.
2. Store `CreatedAt` as int64. It decreases JSON marshaling time by
   about ~25% with no changes for native scheme.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-15 16:08:26 +03:00
da01f4bc2a [#59] Revert scheme to http for s3 cli script
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-05-11 15:09:56 +03:00