Commit Graph

41 Commits (dbfadc83d0f0966008c2b746e4e9d704d64f58fb)

Author SHA1 Message Date
Pavel Karpy dbfadc83d0 [#30] preset: Print container progress
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-11-03 13:11:50 +03:00
Pavel Karpy 70d4b1db9d [#30] grpc: Do not start deleters without registry
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-11-03 13:11:50 +03:00
Pavel Karpy 0e34bf64bd [#30] presets: Pretty JSON files
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-11-03 13:11:50 +03:00
Pavel Karpy 3fbaf0753d [#30] gitignore: Add dir for presets
Could be useful for local runs

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-11-03 13:11:50 +03:00
Pavel Karpy af840ed376 [#30] preset: Create missing files
Also, stop if none of the container creation commands succeeded.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-11-03 13:11:50 +03:00
Pavel Karpy 0dda9f17fa [#30] preset: Stop execution if no new containers were created
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-11-03 13:11:50 +03:00
Pavel Karpy 0088fbd7d6 [#30] preset: PEP 8 refactor
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-11-03 13:11:50 +03:00
Vladimir Domnich eaff41c055 Fix misprint in docs
Signed-off-by: Vladimir Domnich <v.domnich@yadro.com>
2022-10-21 08:54:56 +03:00
Vladimir Domnich 258e448dbf Fix grpc preset
Signed-off-by: Vladimir Domnich <v.domnich@yadro.com>
2022-10-17 14:45:35 +03:00
anatoly@nspcc.ru ea604ee8ee Use separate sleep intervals for VUs
To fine-tune read/write/delete load we need to have separate sleep
intervales for readers/writers/deleters.

The changes were originally authored by anatoly@nspcc.ru <anatoly@nspcc.ru>

Signed-off-by: Vladimir Domnich <v.domnich@yadro.com>
2022-10-17 14:30:24 +03:00
Vladimir Domnich 7875aa9c02 [#23] Remove invalid variable from http scenario
Signed-off-by: Vladimir Domnich <v.domnich@yadro.com>
2022-10-13 15:43:53 +03:00
Vladimir Domnich 37e27f6791 [#23] Implement deletion of objects
1. Added simple lock mechanism to reset obj selector. This prevents
   most of concurrency issues when multiple VUs try to reset selector.
2. Added logic to delete objects to grpc and s3 scenarios.
3. Added registry support to http scenario.
4. Deletion logic was not implemented for http scenario, because
   http gateway does not provide web-method to delete objects.

Signed-off-by: Vladimir Domnich <v.domnich@yadro.com>
2022-10-03 17:34:20 +03:00
Vladimir Domnich 89faf927fb [#21] Improve iteration logic in obj selector
1. Implement reset method that allows to start iteration from beginning of
   the registry. This allows to revisit objects in scenarios like object
   deletion.
2. Add filter structure that allows to select objects based on age.

Signed-off-by: Vladimir Domnich <v.domnich@yadro.com>
2022-09-30 15:18:45 +03:00
Vladimir Domnich b1ec6d562c [#19] Stop object iteration after all objects were processed
At the moment we don't need logic that swings back to beginning of registry when
all objects have been processed. So, for now we can stop iterating and return an
error when selector reaches the end of registry.

Signed-off-by: Vladimir Domnich <v.domnich@yadro.com>
2022-09-23 13:36:27 +03:00
Vladimir Domnich bf4a11474e [#19] Add counters to verify scenario
Signed-off-by: Vladimir Domnich <v.domnich@yadro.com>
2022-09-23 13:36:27 +03:00
Vladimir Domnich 2d4e619992 [#19] Implement configurable database name for registry
Signed-off-by: Vladimir Domnich <v.domnich@yadro.com>
2022-09-23 13:36:27 +03:00
Vladimir Domnich 1cf53545f2 [#19] Implement objects registry module
Registry module stores information about uploaded objects in bolt database and
allows to verify their validity after a load test.
Also, implemented logic to verify objects uploaded via gRPC and S3 protocols.

Signed-off-by: Vladimir Domnich <v.domnich@yadro.com>
2022-09-23 13:36:27 +03:00
Vladimir Domnich 9a78d83f01 [#17] Add job to run unit-tests
Signed-off-by: Vladimir Domnich <v.domnich@yadro.com>
2022-09-19 12:57:57 +03:00
Vladimir Domnich 6b50c8dcf2 [#17] Add unit tests for the generator
Signed-off-by: Vladimir Domnich <v.domnich@yadro.com>
2022-09-19 12:57:57 +03:00
Vladimir Domnich 962da644af [#17] Implement k6 extension for random data generation
It improves payload generation in our scenarios. Current implementation
of scenarios generates single random payload at the start and then sends this
same payload on every request. More realistic test is to generate unique payload
for each request. However, this is an expensive operation that can easily cause
a bottleneck on K6 side when we run multiple writing VUs. So instead we generate
a random buffer with some extra bytes and then take slices of this buffer thus
producing a random payload for each request.

Signed-off-by: Vladimir Domnich <v.domnich@yadro.com>
2022-09-19 12:57:57 +03:00
anatoly@nspcc.ru 402d5feb8f add sleep option for all operations
Signed-off-by: anatoly@nspcc.ru <anatoly@nspcc.ru>
2022-08-10 16:46:58 +03:00
anatoly@nspcc.ru 3d7d902da4 fix policy rules
Signed-off-by: anatoly@nspcc.ru <anatoly@nspcc.ru>
2022-08-02 13:57:50 +03:00
anatoly@nspcc.ru f85c5d31db add scenarios with pregen
Signed-off-by: anatoly@nspcc.ru <anatoly@nspcc.ru>
2022-08-02 13:57:50 +03:00
Denis Kirillov 5d77a526d0 [#11] Add container creation
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-06-18 22:00:52 +03:00
Denis Kirillov 5671b5c73f [#11] Add bucket creation
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-06-18 22:00:52 +03:00
Stanislav Bogatyrev 32916a2b25 Add CONTRIBUTING guidelines
Signed-off-by: Stanislav Bogatyrev <stanislav@nspcc.ru>
2022-06-01 17:54:48 +03:00
Denis Kirillov 265545a584 [#8] Update README.md
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-06-01 17:09:34 +03:00
Denis Kirillov f0285b244d [#8] Add templates
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-06-01 17:09:34 +03:00
Denis Kirillov 7660e3c934 [#8] Add workflows
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-06-01 17:09:34 +03:00
Denis Kirillov a13822d566 [#8] Add logo
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-06-01 17:09:34 +03:00
Denis Kirillov 9953c91cc5 [#8] Add LICENSE
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-06-01 17:09:34 +03:00
Alex Vanin faba352e36 [#3] native: Configure buffer size
Default value is increased up to 64 KiB.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-05-25 18:29:44 +03:00
Alex Vanin 41dc2c8a54 [#6] Fix README
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-05-23 15:08:17 +03:00
Alex Vanin 0d1a5e4187 [#6] native: Factor put object upload code
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-05-23 15:02:47 +03:00
Alex Vanin 2da51e4aa2 [#6] native: Support onsite object preparation and uploading
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-05-23 15:02:47 +03:00
Alex Vanin 1c0abb2479 [#5] Avoid payload copy in put methods
With goja.ArrayBuffer type VU won't perform
copy operation during every put invocation.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-05-20 14:33:56 +03:00
Alex Vanin b17261cc3c [#4] Fix example of native protocol bench
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-05-20 14:10:53 +03:00
Alex Vanin a983980951 [#4] Update README
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-05-20 14:10:53 +03:00
Alex Vanin 0cafcd2375 [#4] s3: Include error message in responses
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-05-20 14:10:53 +03:00
Alex Vanin 822c674c92 [#4] native: Include error message in responses
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-05-20 14:10:53 +03:00
Alex Vanin 68999e49d4 Initial commit
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-05-12 23:27:06 +03:00