Commit graph

26 commits

Author SHA1 Message Date
bede693470 [#153] selector: Add VU synchronization in 'Oneshot' mode
All checks were successful
DCO action / DCO (pull_request) Successful in 1m46s
Tests and linters / Lint (pull_request) Successful in 2m56s
Tests and linters / Tests with -race (pull_request) Successful in 3m40s
Tests and linters / Tests (pull_request) Successful in 3m59s
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2024-09-10 11:56:40 +03:00
335c45c578 [#149] selector: Add read timeout
All checks were successful
DCO action / DCO (pull_request) Successful in 1m15s
Tests and linters / Tests (1.22) (pull_request) Successful in 3m12s
Tests and linters / Tests with -race (pull_request) Successful in 3m26s
Tests and linters / Tests (1.21) (pull_request) Successful in 3m39s
Tests and linters / Lint (pull_request) Successful in 4m21s
If there are no objects for 10 second, then return nil.
It is required to prevent VU iteration hang if there
are no objects pushed to registry.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-06-24 13:59:12 +03:00
f90a645594 [#144] registry: Add tests for obj registry exporter
All checks were successful
DCO action / DCO (pull_request) Successful in 57s
Tests and linters / Tests (1.21) (pull_request) Successful in 3m14s
Tests and linters / Tests (1.22) (pull_request) Successful in 3m15s
Tests and linters / Tests with -race (pull_request) Successful in 3m20s
Tests and linters / Lint (pull_request) Successful in 4m14s
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-06-11 18:09:20 +03:00
3f67606f02 [#144] registry: Fix string generator func in obj info test
`rune` is alias for `int32`. `randString()` used `rand.Int()`
which returns system-specific non-negative integer number. For
64-bit systems it will be int64. An attempt to cast `int64` to
`int32` (`rune`) leads the latter to be a negative number in
case of overflow. This caused the resulting string containing
unexpected symbols.

Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-06-11 17:43:15 +03:00
3dd559a7b1 [#139] Allow registry to export gRPC objects
All checks were successful
DCO action / DCO (pull_request) Successful in 1m0s
Tests and linters / Tests (1.22) (pull_request) Successful in 2m26s
Tests and linters / Tests with -race (pull_request) Successful in 2m42s
Tests and linters / Tests (1.21) (pull_request) Successful in 2m57s
Tests and linters / Lint (pull_request) Successful in 3m15s
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-06-04 15:05:52 +03:00
4aaa50c8ed [#141] registry: Use Batch() for all methods
All checks were successful
DCO action / DCO (pull_request) Successful in 1m34s
Tests and linters / Tests (1.21) (pull_request) Successful in 2m42s
Tests and linters / Tests (1.22) (pull_request) Successful in 2m47s
Tests and linters / Lint (pull_request) Successful in 4m11s
Tests and linters / Tests with -race (pull_request) Successful in 4m41s
Refs #136

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-05-30 15:33:32 +03:00
e5af4112f9 [#136] registry: Apply updates in batch
All checks were successful
DCO action / DCO (pull_request) Successful in 1m7s
Tests and linters / Tests (1.21) (pull_request) Successful in 2m27s
Tests and linters / Tests (1.20) (pull_request) Successful in 3m0s
Tests and linters / Tests with -race (pull_request) Successful in 3m19s
Tests and linters / Lint (pull_request) Successful in 4m33s
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-05-03 13:56:59 +03:00
87ffb551b6 [#133] registry: Implement oneshot selector
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-05 13:43:01 +03:00
0a6e51ccc9 [#131] registry: Gofumpt
All checks were successful
DCO action / DCO (pull_request) Successful in 1m11s
Tests and linters / Tests (1.21) (pull_request) Successful in 2m20s
Tests and linters / Tests (1.20) (pull_request) Successful in 2m33s
Tests and linters / Lint (pull_request) Successful in 2m55s
Tests and linters / Tests with -race (pull_request) Successful in 3m18s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-03-20 11:41:03 +03:00
0c4e2665ba [#131] registry: Allow to create cycled/forward selectors
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-03-20 11:40:49 +03:00
704c0f06bc [#25] selector: Remove next object timeout
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-20 15:01:57 +03:00
3c26e7c917 [#25] xk6: Read objects from registry for gRPC tests
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-20 10:01:34 +03:00
50e2f55362 [#80] Add dump registry util
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-19 15:57:39 +03:00
4aa9a359b5 [#64] registry: Delete object from the old bucket
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-23 16:02:03 +03:00
e71ac35c36 [#42] registry: Do not create selectors without status filters
Should allow to catch bugs in scenarios earlier.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-15 16:08:26 +03:00
b66b5a2f37 [#42] registry: Optimize Count() for ObjSelector
It is the heaviest function executing on setup stage.
The culprit is the linear dependency between its execution
time and the amount of objects in registry. The solution is to store
object by status. While the optimization doesn't work for objects with
no status, it is currently provided by all scenarios.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-15 16:08:26 +03:00
22d7996f79 [#42] registry: Optimize ObjectInfo marshaling
1. Get rid of JSON in the database.
2. Store `CreatedAt` as int64. It decreases JSON marshaling time by
   about ~25% with no changes for native scheme.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-15 16:08:26 +03:00
171327cdda [#2] Deep rebranding
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2022-12-30 11:05:20 +03:00
883c3c259a [#32] registry: Allow to customize cache size for DELETE
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2022-11-11 10:26:27 +03:00
Pavel Karpy
c43f73704e [#30] grpc: Cache object getters
Includes:
1. Logic simplification: no need to call `ObjSelector.Reset` from JS code,
everything could be done inside the Go code. Remove unused mutexes.
2. Do not handle object twice ever: Once handled, any error is expected to be
logged on the JS side and never be handled again. It solves "already removed"
error.
3. Object caching: no need to call bolt's `View` on every object removal: it
blocks other calls and slows down the execution. Read 100 objects (or less if
not available yet), cache them and send to buffered channel.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-11-03 13:11:50 +03:00
37e27f6791 [#23] Implement deletion of objects
1. Added simple lock mechanism to reset obj selector. This prevents
   most of concurrency issues when multiple VUs try to reset selector.
2. Added logic to delete objects to grpc and s3 scenarios.
3. Added registry support to http scenario.
4. Deletion logic was not implemented for http scenario, because
   http gateway does not provide web-method to delete objects.

Signed-off-by: Vladimir Domnich <v.domnich@yadro.com>
2022-10-03 17:34:20 +03:00
89faf927fb [#21] Improve iteration logic in obj selector
1. Implement reset method that allows to start iteration from beginning of
   the registry. This allows to revisit objects in scenarios like object
   deletion.
2. Add filter structure that allows to select objects based on age.

Signed-off-by: Vladimir Domnich <v.domnich@yadro.com>
2022-09-30 15:18:45 +03:00
b1ec6d562c [#19] Stop object iteration after all objects were processed
At the moment we don't need logic that swings back to beginning of registry when
all objects have been processed. So, for now we can stop iterating and return an
error when selector reaches the end of registry.

Signed-off-by: Vladimir Domnich <v.domnich@yadro.com>
2022-09-23 13:36:27 +03:00
bf4a11474e [#19] Add counters to verify scenario
Signed-off-by: Vladimir Domnich <v.domnich@yadro.com>
2022-09-23 13:36:27 +03:00
2d4e619992 [#19] Implement configurable database name for registry
Signed-off-by: Vladimir Domnich <v.domnich@yadro.com>
2022-09-23 13:36:27 +03:00
1cf53545f2 [#19] Implement objects registry module
Registry module stores information about uploaded objects in bolt database and
allows to verify their validity after a load test.
Also, implemented logic to verify objects uploaded via gRPC and S3 protocols.

Signed-off-by: Vladimir Domnich <v.domnich@yadro.com>
2022-09-23 13:36:27 +03:00