cb813e27a8
[ #293 ] object: Fix payload size limiter
...
* Sending empty chunks by `writeChunk` should not release new
objects as this doesn't change `payloadSizeLimiter` internal
state.
* This also fixes the bug with patcher when an offset of a patch
equals to `MaxSize` - `payloadSizeLimiter` releases object again
although state is the same. This led to error because EC-encoder
receieved empty payload and couldn't not process it.
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-11-07 12:11:19 +00:00
6ce73790ea
[ #276 ] Merge repo with frostfs-api-go
...
Signed-off-by: Pavel Pogodaev <p.pogodaev@yadro.com>
2024-10-22 14:05:12 +00:00
46ee543899
[ #265 ] go.mod: Use range over int
...
Since Go 1.22 a `for` statement with a `range` clause is able
to iterate through integer values from zero to an upper limit.
gopatch script:
@@
var i, e expression
@@
-for i := 0; i <= e - 1; i++ {
+for i := range e {
...
}
@@
var i, e expression
@@
-for i := 0; i <= e; i++ {
+for i := range e + 1 {
...
}
@@
var i, e expression
@@
-for i := 0; i < e; i++ {
+for i := range e {
...
}
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-09-04 12:37:46 +03:00
7e94a6adf2
[ #237 ] pool: Return creation epoch from object put
...
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2024-07-22 06:15:23 +00:00
1af9b6d18b
[ #155 ] sdk-go: Add buffer support for payloadSizeLimiter
...
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2024-03-25 10:47:04 +03:00
56debcfa56
[ #190 ] sdk-go: Gofumpt fixes
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-22 19:21:20 +03:00
157a9930e8
[ #190 ] sdk-go: Pass user.ID by value
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-22 19:21:20 +03:00
665e5807bc
[ #188 ] transformer: Allow to provide size hint
...
For big objects with known size we can optimize allocation patterns
by providing size hint. As with any hint, it does not affect transformer
functionality: slices with capacity > MaxSize are never allocated.
```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/transformer
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
│ out │
│ sec/op │
Transformer/small/no_size_hint-8 65.44µ ± 3%
Transformer/small/no_size_hint,_with_buffer-8 64.24µ ± 5%
Transformer/small/with_size_hint,_with_buffer-8 58.70µ ± 5%
Transformer/big/no_size_hint-8 367.8m ± 3%
Transformer/big/no_size_hint,_with_buffer-8 562.7m ± 0%
Transformer/big/with_size_hint,_with_buffer-8 385.6m ± 7%
geomean 5.197m
│ out │
│ B/op │
Transformer/small/no_size_hint-8 13.40Ki ± 0%
Transformer/small/no_size_hint,_with_buffer-8 13.40Ki ± 0%
Transformer/small/with_size_hint,_with_buffer-8 13.39Ki ± 0%
Transformer/big/no_size_hint-8 288.0Mi ± 0%
Transformer/big/no_size_hint,_with_buffer-8 1.390Gi ± 0%
Transformer/big/with_size_hint,_with_buffer-8 288.0Mi ± 0%
geomean 2.533Mi
│ out │
│ allocs/op │
Transformer/small/no_size_hint-8 92.00 ± 0%
Transformer/small/no_size_hint,_with_buffer-8 92.00 ± 0%
Transformer/small/with_size_hint,_with_buffer-8 92.00 ± 0%
Transformer/big/no_size_hint-8 546.5 ± 0%
Transformer/big/no_size_hint,_with_buffer-8 607.5 ± 0%
Transformer/big/with_size_hint,_with_buffer-8 545.5 ± 0%
geomean 228.1
```
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-10-30 12:13:04 +00:00
c359a7465a
[ #64 ] transformer: Simplify interface
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-11 07:33:12 +00:00
aa8ffebc63
[ #95 ] transformer: Set parent version
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-23 13:48:02 +03:00
d4fe9a193d
[ #66 ] transformer: Accept constructor in NextTarget
...
The code of frostfs-node is not yet ready to reuse egress target for
multiple objects, let's postpone until #64 .
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-03 11:22:09 +03:00
c42a6119ff
[ #66 ] transformer: Extend tests
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-03 11:22:09 +03:00
38b03ff28b
[ #63 ] transformer: Resolve funlen linter
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-25 13:29:42 +03:00
0fa23a9b14
[ #63 ] transformer: Add context to methods
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-25 13:29:42 +03:00
708d933fe3
[ #47 ] .golangci.yml: Unify with other FrostFS repos
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-07 08:59:19 +03:00
d589d51509
[ #19 ] transformer: Fix dependencies
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-03-16 09:51:03 +03:00
25588ee3be
[ #19 ] transformer: Do not allocate intermeate slice for hashers
...
```
name old time/op new time/op delta
Transformer/small-8 73.7µs ±15% 72.4µs ±16% ~ (p=0.604 n=10+9)
Transformer/big-8 1.36s ± 4% 1.36s ± 8% ~ (p=0.579 n=10+10)
name old alloc/op new alloc/op delta
Transformer/small-8 7.67kB ± 0% 7.57kB ± 0% -1.36% (p=0.000 n=10+10)
Transformer/big-8 49.0kB ± 0% 48.3kB ± 0% -1.48% (p=0.000 n=10+10)
name old allocs/op new allocs/op delta
Transformer/small-8 101 ± 0% 98 ± 0% -2.97% (p=0.000 n=10+10)
Transformer/big-8 609 ± 0% 591 ± 1% -3.00% (p=0.000 n=10+9)
```
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-15 05:39:09 +00:00
9407f30248
[ #19 ] transformer: Optimize payload hashers a bit
...
```
name old time/op new time/op delta
Transformer/small-8 74.8µs ±11% 73.7µs ±15% ~ (p=0.529 n=10+10)
Transformer/big-8 1.38s ±11% 1.36s ± 4% ~ (p=0.796 n=10+10)
name old alloc/op new alloc/op delta
Transformer/small-8 7.69kB ± 0% 7.67kB ± 0% -0.21% (p=0.000 n=10+10)
Transformer/big-8 49.2kB ± 0% 49.0kB ± 0% -0.48% (p=0.004 n=10+10)
name old allocs/op new allocs/op delta
Transformer/small-8 102 ± 0% 101 ± 0% -0.98% (p=0.000 n=9+10)
Transformer/big-8 620 ± 1% 609 ± 0% -1.66% (p=0.000 n=10+10)
```
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-15 05:39:09 +00:00
94c0a607b5
[ #19 ] transformer: Add a target which sends parts to a channel
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-15 05:39:09 +00:00
e45647de3c
[ #19 ] transformer: Do not reuse memory of sent objects
...
Slower, but more correct.
```
name old time/op new time/op delta
Transformer/small-8 72.4µs ± 8% 74.8µs ±11% ~ (p=0.278 n=9+10)
Transformer/big-8 1.31s ± 8% 1.38s ±11% +5.50% (p=0.035 n=10+10)
name old alloc/op new alloc/op delta
Transformer/small-8 7.39kB ± 0% 7.69kB ± 0% +4.04% (p=0.000 n=10+10)
Transformer/big-8 46.9kB ± 0% 49.2kB ± 0% +4.87% (p=0.000 n=10+10)
name old allocs/op new allocs/op delta
Transformer/small-8 94.6 ± 1% 102.0 ± 0% +7.82% (p=0.000 n=10+9)
Transformer/big-8 560 ± 0% 620 ± 1% +10.66% (p=0.000 n=10+10)
```
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-15 05:39:09 +00:00
611e20587b
[ #19 ] transformer/test: Check owner ID and payload hash for parts
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-15 05:39:09 +00:00
eba6831125
[ #19 ] transformer/test: Add helper functions
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-15 05:39:09 +00:00
7e3810d654
[ #19 ] transformer: Move EpochSource to other types
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-15 05:39:09 +00:00
cc0fef2c55
[ #19 ] transformer: Merge formatter and payload splitter
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-15 05:39:09 +00:00
b696d3c70e
[ #19 ] transformer: Make writeChunk non-recursive
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-15 05:39:09 +00:00
1c94309d7a
[ #19 ] transformer: Simplify AccessIdentifiers
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-15 05:39:09 +00:00
f43f18ecda
[ #19 ] transformer: Cover with unit-tests
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-15 05:39:09 +00:00
ac8442bf99
[ #19 ] object: Move transformer implementation from node
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-15 05:39:09 +00:00
