cb813e27a8
[ #293 ] object: Fix payload size limiter
...
* Sending empty chunks by `writeChunk` should not release new
objects as this doesn't change `payloadSizeLimiter` internal
state.
* This also fixes the bug with patcher when an offset of a patch
equals to `MaxSize` - `payloadSizeLimiter` releases object again
although state is the same. This led to error because EC-encoder
receieved empty payload and couldn't not process it.
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-11-07 12:11:19 +00:00
6ce73790ea
[ #276 ] Merge repo with frostfs-api-go
...
DCO / DCO (pull_request) Successful in 38s
Tests and linters / Tests (pull_request) Successful in 1m13s
Tests and linters / Lint (pull_request) Successful in 2m36s
Signed-off-by: Pavel Pogodaev <p.pogodaev@yadro.com>
2024-10-22 14:05:12 +00:00
e580ee991d
[ #271 ] Drop handling of system attributes with NeoFS prefix
...
DCO / DCO (pull_request) Successful in 43s
Tests and linters / Tests (pull_request) Successful in 55s
Tests and linters / Lint (pull_request) Successful in 1m25s
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-09-18 09:59:38 +00:00
6821fe6fb2
[ #271 ] object: Add UserAttributes method
...
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-09-18 09:59:38 +00:00
46ee543899
[ #265 ] go.mod: Use range over int
...
DCO / DCO (pull_request) Successful in 47s
Tests and linters / Tests (1.22) (pull_request) Successful in 1m7s
Tests and linters / Tests (1.23) (pull_request) Successful in 1m7s
Tests and linters / Lint (pull_request) Successful in 1m42s
Since Go 1.22 a `for` statement with a `range` clause is able
to iterate through integer values from zero to an upper limit.
gopatch script:
@@
var i, e expression
@@
-for i := 0; i <= e - 1; i++ {
+for i := range e {
...
}
@@
var i, e expression
@@
-for i := 0; i <= e; i++ {
+for i := range e + 1 {
...
}
@@
var i, e expression
@@
-for i := 0; i < e; i++ {
+for i := range e {
...
}
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-09-04 12:37:46 +03:00
98aabc45a7
[ #252 ] patcher: Fix applying patch from the same offset
...
DCO / DCO (pull_request) Successful in 38s
Tests and linters / Tests (1.22) (pull_request) Successful in 50s
Tests and linters / Tests (1.21) (pull_request) Successful in 56s
Tests and linters / Lint (pull_request) Successful in 1m54s
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-08-13 18:58:21 +03:00
361739e860
[ #247 ] object: Introduce patcher package
...
* Introduce `patcher` package that contains such interfaces to be
implemented:
- `PatchApplier` - the main patching engine that merges the stream
of patches and the stream of original object payload divided by
ranges. The merged streams result is output to `ChunkedObjectWriter`;
- `RangeProvider` - provides the original object payload by ranges;
- `HeaderProvider` - provides the original object header.
* Introduce `patcher` that implements `PatchApplier`;
* Cover all possible cases with unit-tests.
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-08-05 12:32:14 +00:00
6dd500def9
[ #247 ] object: Introduce Patch type
...
* Make ToV2, FromV2 converters for Patch and PayloadPatch;
* Add unit-tests.
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-08-05 12:32:14 +00:00
7e94a6adf2
[ #237 ] pool: Return creation epoch from object put
...
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2024-07-22 06:15:23 +00:00
ebd8fcd168
[ #224 ] object: Introduce parent attributes in EC-header
...
DCO / DCO (pull_request) Successful in 1m1s
Tests and linters / Tests (1.21) (pull_request) Successful in 1m20s
Tests and linters / Tests (1.20) (pull_request) Successful in 1m32s
Tests and linters / Lint (pull_request) Successful in 2m18s
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-31 16:20:48 +03:00
3de256d05e
[ #223 ] object: Introduce new fields for ECHeader
...
DCO / DCO (pull_request) Successful in 1m7s
Tests and linters / Tests (1.21) (pull_request) Successful in 1m23s
Tests and linters / Tests (1.20) (pull_request) Successful in 1m31s
Tests and linters / Lint (pull_request) Successful in 2m44s
* Introduce `parentSplitID`, `parentSplitParentID` fields
for `ECHeader`;
* Fix ECHeader's constructor;
* Fix `Split` and `Reconstruct`;
* Add unit-tests.
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-17 15:16:28 +03:00
09b79d13f3
[ #223 ] object: Introduce ec_parent search filter
...
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-17 15:03:40 +03:00
20ab57bf7e
[ #214 ] object: Implement Get\Head requests for EC object
...
DCO / DCO (pull_request) Successful in 1m3s
Tests and linters / Tests (1.21) (pull_request) Successful in 1m15s
Tests and linters / Tests (1.20) (pull_request) Successful in 1m27s
Tests and linters / Lint (pull_request) Successful in 2m13s
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-04-24 11:07:26 +03:00
ec0cb2169f
[ #211 ] object: Fix setIDWithSignature
...
DCO / DCO (pull_request) Successful in 1m0s
Tests and linters / Tests (1.21) (pull_request) Successful in 1m7s
Tests and linters / Tests (1.20) (pull_request) Successful in 1m32s
Tests and linters / Lint (pull_request) Successful in 2m12s
* Calculate and set checksum before ID is calculated.
* Add header verification for parts in unit-test.
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-03-29 13:48:04 +03:00
1af9b6d18b
[ #155 ] sdk-go: Add buffer support for payloadSizeLimiter
...
DCO / DCO (pull_request) Successful in 58s
Tests and linters / Tests (1.21) (pull_request) Successful in 1m18s
Tests and linters / Tests (1.20) (pull_request) Successful in 1m29s
Tests and linters / Lint (pull_request) Successful in 2m12s
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2024-03-25 10:47:04 +03:00
bd2d350b09
[ #205 ] object: Initial EC implementation
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-03-22 10:14:12 +00:00
a86170f53a
[ #202 ] object: Reset marshal data on CutPayload
...
DCO / DCO (pull_request) Successful in 1m1s
Tests and linters / Tests (1.21) (pull_request) Successful in 1m20s
Tests and linters / Tests (1.20) (pull_request) Successful in 1m26s
Tests and linters / Lint (pull_request) Successful in 2m13s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-15 15:23:47 +03:00
56debcfa56
[ #190 ] sdk-go: Gofumpt fixes
...
DCO / DCO (pull_request) Successful in 56s
Tests and linters / Tests (1.21) (pull_request) Successful in 1m16s
Tests and linters / Tests (1.20) (pull_request) Successful in 1m13s
Tests and linters / Lint (pull_request) Successful in 2m20s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-22 19:21:20 +03:00
157a9930e8
[ #190 ] sdk-go: Pass user.ID by value
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-22 19:21:20 +03:00
665e5807bc
[ #188 ] transformer: Allow to provide size hint
...
For big objects with known size we can optimize allocation patterns
by providing size hint. As with any hint, it does not affect transformer
functionality: slices with capacity > MaxSize are never allocated.
```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/transformer
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
│ out │
│ sec/op │
Transformer/small/no_size_hint-8 65.44µ ± 3%
Transformer/small/no_size_hint,_with_buffer-8 64.24µ ± 5%
Transformer/small/with_size_hint,_with_buffer-8 58.70µ ± 5%
Transformer/big/no_size_hint-8 367.8m ± 3%
Transformer/big/no_size_hint,_with_buffer-8 562.7m ± 0%
Transformer/big/with_size_hint,_with_buffer-8 385.6m ± 7%
geomean 5.197m
│ out │
│ B/op │
Transformer/small/no_size_hint-8 13.40Ki ± 0%
Transformer/small/no_size_hint,_with_buffer-8 13.40Ki ± 0%
Transformer/small/with_size_hint,_with_buffer-8 13.39Ki ± 0%
Transformer/big/no_size_hint-8 288.0Mi ± 0%
Transformer/big/no_size_hint,_with_buffer-8 1.390Gi ± 0%
Transformer/big/with_size_hint,_with_buffer-8 288.0Mi ± 0%
geomean 2.533Mi
│ out │
│ allocs/op │
Transformer/small/no_size_hint-8 92.00 ± 0%
Transformer/small/no_size_hint,_with_buffer-8 92.00 ± 0%
Transformer/small/with_size_hint,_with_buffer-8 92.00 ± 0%
Transformer/big/no_size_hint-8 546.5 ± 0%
Transformer/big/no_size_hint,_with_buffer-8 607.5 ± 0%
Transformer/big/with_size_hint,_with_buffer-8 545.5 ± 0%
geomean 228.1
```
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-10-30 12:13:04 +00:00
49ad985cad
[ #161 ] *: Do not use math/rand.Read()
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-08 17:17:02 +03:00
c359a7465a
[ #64 ] transformer: Simplify interface
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-11 07:33:12 +00:00
ac95b87e7c
[ #101 ] Add Equals for Address
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-07-10 09:05:01 +00:00
aa8ffebc63
[ #95 ] transformer: Set parent version
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-23 13:48:02 +03:00
d4fe9a193d
[ #66 ] transformer: Accept constructor in NextTarget
...
The code of frostfs-node is not yet ready to reuse egress target for
multiple objects, let's postpone until #64 .
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-03 11:22:09 +03:00
c42a6119ff
[ #66 ] transformer: Extend tests
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-03 11:22:09 +03:00
29b188db57
[ #52 ] Remove storage groups and audit
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-03 08:20:37 +00:00
38b03ff28b
[ #63 ] transformer: Resolve funlen linter
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-25 13:29:42 +03:00
0fa23a9b14
[ #63 ] transformer: Add context to methods
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-25 13:29:42 +03:00
708d933fe3
[ #47 ] .golangci.yml: Unify with other FrostFS repos
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-07 08:59:19 +03:00
d589d51509
[ #19 ] transformer: Fix dependencies
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-03-16 09:51:03 +03:00
25588ee3be
[ #19 ] transformer: Do not allocate intermeate slice for hashers
...
```
name old time/op new time/op delta
Transformer/small-8 73.7µs ±15% 72.4µs ±16% ~ (p=0.604 n=10+9)
Transformer/big-8 1.36s ± 4% 1.36s ± 8% ~ (p=0.579 n=10+10)
name old alloc/op new alloc/op delta
Transformer/small-8 7.67kB ± 0% 7.57kB ± 0% -1.36% (p=0.000 n=10+10)
Transformer/big-8 49.0kB ± 0% 48.3kB ± 0% -1.48% (p=0.000 n=10+10)
name old allocs/op new allocs/op delta
Transformer/small-8 101 ± 0% 98 ± 0% -2.97% (p=0.000 n=10+10)
Transformer/big-8 609 ± 0% 591 ± 1% -3.00% (p=0.000 n=10+9)
```
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-15 05:39:09 +00:00
9407f30248
[ #19 ] transformer: Optimize payload hashers a bit
...
```
name old time/op new time/op delta
Transformer/small-8 74.8µs ±11% 73.7µs ±15% ~ (p=0.529 n=10+10)
Transformer/big-8 1.38s ±11% 1.36s ± 4% ~ (p=0.796 n=10+10)
name old alloc/op new alloc/op delta
Transformer/small-8 7.69kB ± 0% 7.67kB ± 0% -0.21% (p=0.000 n=10+10)
Transformer/big-8 49.2kB ± 0% 49.0kB ± 0% -0.48% (p=0.004 n=10+10)
name old allocs/op new allocs/op delta
Transformer/small-8 102 ± 0% 101 ± 0% -0.98% (p=0.000 n=9+10)
Transformer/big-8 620 ± 1% 609 ± 0% -1.66% (p=0.000 n=10+10)
```
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-15 05:39:09 +00:00
94c0a607b5
[ #19 ] transformer: Add a target which sends parts to a channel
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-15 05:39:09 +00:00
e45647de3c
[ #19 ] transformer: Do not reuse memory of sent objects
...
Slower, but more correct.
```
name old time/op new time/op delta
Transformer/small-8 72.4µs ± 8% 74.8µs ±11% ~ (p=0.278 n=9+10)
Transformer/big-8 1.31s ± 8% 1.38s ±11% +5.50% (p=0.035 n=10+10)
name old alloc/op new alloc/op delta
Transformer/small-8 7.39kB ± 0% 7.69kB ± 0% +4.04% (p=0.000 n=10+10)
Transformer/big-8 46.9kB ± 0% 49.2kB ± 0% +4.87% (p=0.000 n=10+10)
name old allocs/op new allocs/op delta
Transformer/small-8 94.6 ± 1% 102.0 ± 0% +7.82% (p=0.000 n=10+9)
Transformer/big-8 560 ± 0% 620 ± 1% +10.66% (p=0.000 n=10+10)
```
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-15 05:39:09 +00:00
611e20587b
[ #19 ] transformer/test: Check owner ID and payload hash for parts
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-15 05:39:09 +00:00
eba6831125
[ #19 ] transformer/test: Add helper functions
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-15 05:39:09 +00:00
7e3810d654
[ #19 ] transformer: Move EpochSource to other types
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-15 05:39:09 +00:00
cc0fef2c55
[ #19 ] transformer: Merge formatter and payload splitter
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-15 05:39:09 +00:00
b696d3c70e
[ #19 ] transformer: Make writeChunk non-recursive
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-15 05:39:09 +00:00
1c94309d7a
[ #19 ] transformer: Simplify AccessIdentifiers
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-15 05:39:09 +00:00
f43f18ecda
[ #19 ] transformer: Cover with unit-tests
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-15 05:39:09 +00:00
ac8442bf99
[ #19 ] object: Move transformer implementation from node
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-15 05:39:09 +00:00
b461aa64b8
[ #23 ] pre-commit: Add golangci-lint hook
...
Also, fix minor issues. Skip deprecated warning for now.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-09 22:53:14 +03:00
94476f9055
Rename package name
...
Due to source code relocation from GitHub.
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-03-07 15:47:21 +03:00
4ff9c00de3
[ #4 ] Rename NeoFS mentions in comments and method names
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2022-12-30 13:40:50 +03:00
Pavel Karpy
a1748ae0e7
[ #370 ] bearer, session: Clarify expiration epoch
...
The expiration epoch is the _last_ valid epoch for an entity. Also, clarify
the expiration epoch meaning for tombstones and regular objects.
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2022-12-29 16:34:13 +03:00
Pavel Karpy
4c779423f5
Move to frostfs-sdk-go
...
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2022-12-14 09:59:29 +03:00
Denis Kirillov
d047289182
[ #360 ] object: Add new package relations
...
Package relations provides feature to process inner object structure.
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-11-14 17:46:17 +03:00
Denis Kirillov
74234623b2
[ #215 ] object: Add well-known FilePath attribute
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-09-06 17:21:37 +04:00
