It is the heaviest function executing on setup stage.
The culprit is the linear dependency between its execution
time and the amount of objects in registry. The solution is to store
object by status. While the optimization doesn't work for objects with
no status, it is currently provided by all scenarios.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
1. Get rid of JSON in the database.
2. Store `CreatedAt` as int64. It decreases JSON marshaling time by
about ~25% with no changes for native scheme.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Previous iteration was bad, because `Logger` instance was shared and
endpoints in different VUs were overriding each other.
New interface is much better, it supports logger extension in any
context with arbitrary fields.
```
const l = logging.new().withFields({endpoint: "my endpoint"});
...
l.withField("cid", container).info("not found");
```
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
correct statistics - total counts of request contains only successful requests.
add endpoint to grpc connection error message
added ability to change workers count during preset
Signed-off-by: Andrey Berezin <a.berezin@yadro.com>
Includes:
1. Logic simplification: no need to call `ObjSelector.Reset` from JS code,
everything could be done inside the Go code. Remove unused mutexes.
2. Do not handle object twice ever: Once handled, any error is expected to be
logged on the JS side and never be handled again. It solves "already removed"
error.
3. Object caching: no need to call bolt's `View` on every object removal: it
blocks other calls and slows down the execution. Read 100 objects (or less if
not available yet), cache them and send to buffered channel.
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
1. Added simple lock mechanism to reset obj selector. This prevents
most of concurrency issues when multiple VUs try to reset selector.
2. Added logic to delete objects to grpc and s3 scenarios.
3. Added registry support to http scenario.
4. Deletion logic was not implemented for http scenario, because
http gateway does not provide web-method to delete objects.
Signed-off-by: Vladimir Domnich <v.domnich@yadro.com>
1. Implement reset method that allows to start iteration from beginning of
the registry. This allows to revisit objects in scenarios like object
deletion.
2. Add filter structure that allows to select objects based on age.
Signed-off-by: Vladimir Domnich <v.domnich@yadro.com>
At the moment we don't need logic that swings back to beginning of registry when
all objects have been processed. So, for now we can stop iterating and return an
error when selector reaches the end of registry.
Signed-off-by: Vladimir Domnich <v.domnich@yadro.com>
Registry module stores information about uploaded objects in bolt database and
allows to verify their validity after a load test.
Also, implemented logic to verify objects uploaded via gRPC and S3 protocols.
Signed-off-by: Vladimir Domnich <v.domnich@yadro.com>
It improves payload generation in our scenarios. Current implementation
of scenarios generates single random payload at the start and then sends this
same payload on every request. More realistic test is to generate unique payload
for each request. However, this is an expensive operation that can easily cause
a bottleneck on K6 side when we run multiple writing VUs. So instead we generate
a random buffer with some extra bytes and then take slices of this buffer thus
producing a random payload for each request.
Signed-off-by: Vladimir Domnich <v.domnich@yadro.com>
