cb813e27a8
[ #293 ] object: Fix payload size limiter
...
* Sending empty chunks by `writeChunk` should not release new
objects as this doesn't change `payloadSizeLimiter` internal
state.
* This also fixes the bug with patcher when an offset of a patch
equals to `MaxSize` - `payloadSizeLimiter` releases object again
although state is the same. This led to error because EC-encoder
receieved empty payload and couldn't not process it.
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-11-07 12:11:19 +00:00
6ce73790ea
[ #276 ] Merge repo with frostfs-api-go
...
DCO / DCO (pull_request) Successful in 38s
Tests and linters / Tests (pull_request) Successful in 1m13s
Tests and linters / Lint (pull_request) Successful in 2m36s
Signed-off-by: Pavel Pogodaev <p.pogodaev@yadro.com>
2024-10-22 14:05:12 +00:00
7e94a6adf2
[ #237 ] pool: Return creation epoch from object put
...
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2024-07-22 06:15:23 +00:00
1af9b6d18b
[ #155 ] sdk-go: Add buffer support for payloadSizeLimiter
...
DCO / DCO (pull_request) Successful in 58s
Tests and linters / Tests (1.21) (pull_request) Successful in 1m18s
Tests and linters / Tests (1.20) (pull_request) Successful in 1m29s
Tests and linters / Lint (pull_request) Successful in 2m12s
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2024-03-25 10:47:04 +03:00
665e5807bc
[ #188 ] transformer: Allow to provide size hint
...
For big objects with known size we can optimize allocation patterns
by providing size hint. As with any hint, it does not affect transformer
functionality: slices with capacity > MaxSize are never allocated.
```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/transformer
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
│ out │
│ sec/op │
Transformer/small/no_size_hint-8 65.44µ ± 3%
Transformer/small/no_size_hint,_with_buffer-8 64.24µ ± 5%
Transformer/small/with_size_hint,_with_buffer-8 58.70µ ± 5%
Transformer/big/no_size_hint-8 367.8m ± 3%
Transformer/big/no_size_hint,_with_buffer-8 562.7m ± 0%
Transformer/big/with_size_hint,_with_buffer-8 385.6m ± 7%
geomean 5.197m
│ out │
│ B/op │
Transformer/small/no_size_hint-8 13.40Ki ± 0%
Transformer/small/no_size_hint,_with_buffer-8 13.40Ki ± 0%
Transformer/small/with_size_hint,_with_buffer-8 13.39Ki ± 0%
Transformer/big/no_size_hint-8 288.0Mi ± 0%
Transformer/big/no_size_hint,_with_buffer-8 1.390Gi ± 0%
Transformer/big/with_size_hint,_with_buffer-8 288.0Mi ± 0%
geomean 2.533Mi
│ out │
│ allocs/op │
Transformer/small/no_size_hint-8 92.00 ± 0%
Transformer/small/no_size_hint,_with_buffer-8 92.00 ± 0%
Transformer/small/with_size_hint,_with_buffer-8 92.00 ± 0%
Transformer/big/no_size_hint-8 546.5 ± 0%
Transformer/big/no_size_hint,_with_buffer-8 607.5 ± 0%
Transformer/big/with_size_hint,_with_buffer-8 545.5 ± 0%
geomean 228.1
```
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-10-30 12:13:04 +00:00
c359a7465a
[ #64 ] transformer: Simplify interface
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-11 07:33:12 +00:00
aa8ffebc63
[ #95 ] transformer: Set parent version
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-23 13:48:02 +03:00
d4fe9a193d
[ #66 ] transformer: Accept constructor in NextTarget
...
The code of frostfs-node is not yet ready to reuse egress target for
multiple objects, let's postpone until #64 .
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-03 11:22:09 +03:00
c42a6119ff
[ #66 ] transformer: Extend tests
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-03 11:22:09 +03:00
38b03ff28b
[ #63 ] transformer: Resolve funlen linter
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-25 13:29:42 +03:00
0fa23a9b14
[ #63 ] transformer: Add context to methods
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-25 13:29:42 +03:00
708d933fe3
[ #47 ] .golangci.yml: Unify with other FrostFS repos
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-07 08:59:19 +03:00
d589d51509
[ #19 ] transformer: Fix dependencies
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-03-16 09:51:03 +03:00
25588ee3be
[ #19 ] transformer: Do not allocate intermeate slice for hashers
...
```
name old time/op new time/op delta
Transformer/small-8 73.7µs ±15% 72.4µs ±16% ~ (p=0.604 n=10+9)
Transformer/big-8 1.36s ± 4% 1.36s ± 8% ~ (p=0.579 n=10+10)
name old alloc/op new alloc/op delta
Transformer/small-8 7.67kB ± 0% 7.57kB ± 0% -1.36% (p=0.000 n=10+10)
Transformer/big-8 49.0kB ± 0% 48.3kB ± 0% -1.48% (p=0.000 n=10+10)
name old allocs/op new allocs/op delta
Transformer/small-8 101 ± 0% 98 ± 0% -2.97% (p=0.000 n=10+10)
Transformer/big-8 609 ± 0% 591 ± 1% -3.00% (p=0.000 n=10+9)
```
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-15 05:39:09 +00:00
9407f30248
[ #19 ] transformer: Optimize payload hashers a bit
...
```
name old time/op new time/op delta
Transformer/small-8 74.8µs ±11% 73.7µs ±15% ~ (p=0.529 n=10+10)
Transformer/big-8 1.38s ±11% 1.36s ± 4% ~ (p=0.796 n=10+10)
name old alloc/op new alloc/op delta
Transformer/small-8 7.69kB ± 0% 7.67kB ± 0% -0.21% (p=0.000 n=10+10)
Transformer/big-8 49.2kB ± 0% 49.0kB ± 0% -0.48% (p=0.004 n=10+10)
name old allocs/op new allocs/op delta
Transformer/small-8 102 ± 0% 101 ± 0% -0.98% (p=0.000 n=9+10)
Transformer/big-8 620 ± 1% 609 ± 0% -1.66% (p=0.000 n=10+10)
```
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-15 05:39:09 +00:00
e45647de3c
[ #19 ] transformer: Do not reuse memory of sent objects
...
Slower, but more correct.
```
name old time/op new time/op delta
Transformer/small-8 72.4µs ± 8% 74.8µs ±11% ~ (p=0.278 n=9+10)
Transformer/big-8 1.31s ± 8% 1.38s ±11% +5.50% (p=0.035 n=10+10)
name old alloc/op new alloc/op delta
Transformer/small-8 7.39kB ± 0% 7.69kB ± 0% +4.04% (p=0.000 n=10+10)
Transformer/big-8 46.9kB ± 0% 49.2kB ± 0% +4.87% (p=0.000 n=10+10)
name old allocs/op new allocs/op delta
Transformer/small-8 94.6 ± 1% 102.0 ± 0% +7.82% (p=0.000 n=10+9)
Transformer/big-8 560 ± 0% 620 ± 1% +10.66% (p=0.000 n=10+10)
```
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-15 05:39:09 +00:00
cc0fef2c55
[ #19 ] transformer: Merge formatter and payload splitter
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-15 05:39:09 +00:00
b696d3c70e
[ #19 ] transformer: Make writeChunk non-recursive
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-15 05:39:09 +00:00
1c94309d7a
[ #19 ] transformer: Simplify AccessIdentifiers
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-15 05:39:09 +00:00
ac8442bf99
[ #19 ] object: Move transformer implementation from node
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-15 05:39:09 +00:00
