TrueCloudLab/frostfs-sdk-go
33
0
Fork 21
Code Issues 34 Pull requests 1 Releases Activity Actions
735 commits 5 branches 0 tags 10 MiB
Commit graph

83 commits

Author SHA1 Message Date
Airat Arifullin
48f4d6933c [#247] object: Introduce patcher package
All checks were successful
DCO / DCO (pull_request) Successful in 41s
Details
Tests and linters / Tests (1.22) (pull_request) Successful in 52s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 56s
Details
Tests and linters / Lint (pull_request) Successful in 1m20s
Details 
* Introduce `patcher` package that contains such interfaces to be
  implemented:
  - `PatchApplier` - the main patching engine that merges the stream
    of patches and the stream of original object payload divided by
    ranges. The merged streams result is output to `ChunkedObjectWriter`;
  - `RangeProvider` - provides the original object payload by ranges;
  - `HeaderProvider` - provides the original object header.
* Introduce `patcher` that implements `PatchApplier`;
* Cover all possible cases with unit-tests.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-08-01 18:39:25 +03:00
Airat Arifullin
07312d053e [#247] object: Introduce Patch type 
* Make ToV2, FromV2 converters for Patch and PayloadPatch;
* Add unit-tests.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-08-01 18:39:19 +03:00
Marina Biryukova
7e94a6adf2 [#237] pool: Return creation epoch from object put 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2024-07-22 06:15:23 +00:00
Airat Arifullin
ebd8fcd168 [#224] object: Introduce parent attributes in EC-header
All checks were successful
DCO / DCO (pull_request) Successful in 1m1s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 1m20s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 1m32s
Details
Tests and linters / Lint (pull_request) Successful in 2m18s
Details 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-05-31 16:20:48 +03:00
Airat Arifullin
3de256d05e [#223] object: Introduce new fields for ECHeader
All checks were successful
DCO / DCO (pull_request) Successful in 1m7s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 1m23s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 1m31s
Details
Tests and linters / Lint (pull_request) Successful in 2m44s
Details 
* Introduce `parentSplitID`, `parentSplitParentID` fields
  for `ECHeader`;
* Fix ECHeader's constructor;
* Fix `Split` and `Reconstruct`;
* Add unit-tests.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-05-17 15:16:28 +03:00
Airat Arifullin
09b79d13f3 [#223] object: Introduce ec_parent search filter 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-05-17 15:03:40 +03:00
Anton Nikiforov
20ab57bf7e [#214] object: Implement Get\Head requests for EC object
All checks were successful
DCO / DCO (pull_request) Successful in 1m3s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 1m15s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 1m27s
Details
Tests and linters / Lint (pull_request) Successful in 2m13s
Details 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-04-24 11:07:26 +03:00
aarifullin
ec0cb2169f [#211] object: Fix setIDWithSignature
All checks were successful
DCO / DCO (pull_request) Successful in 1m0s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 1m7s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 1m32s
Details
Tests and linters / Lint (pull_request) Successful in 2m12s
Details 
* Calculate and set checksum before ID is calculated.
* Add header verification for parts in unit-test.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-03-29 13:48:04 +03:00
Alexander Chuprov
1af9b6d18b [#155] sdk-go: Add buffer support for payloadSizeLimiter
All checks were successful
DCO / DCO (pull_request) Successful in 58s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 1m18s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 1m29s
Details
Tests and linters / Lint (pull_request) Successful in 2m12s
Details 
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2024-03-25 10:47:04 +03:00
Evgenii Stratonikov
bd2d350b09 [#205] object: Initial EC implementation 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-03-22 10:14:12 +00:00
Dmitrii Stepanov
a86170f53a [#202] object: Reset marshal data on CutPayload
All checks were successful
DCO / DCO (pull_request) Successful in 1m1s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 1m20s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 1m26s
Details
Tests and linters / Lint (pull_request) Successful in 2m13s
Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-02-15 15:23:47 +03:00
Dmitrii Stepanov
56debcfa56 [#190] sdk-go: Gofumpt fixes
All checks were successful
DCO / DCO (pull_request) Successful in 56s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 1m16s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 1m13s
Details
Tests and linters / Lint (pull_request) Successful in 2m20s
Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-11-22 19:21:20 +03:00
Dmitrii Stepanov
157a9930e8 [#190] sdk-go: Pass user.ID by value 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-11-22 19:21:20 +03:00
Evgenii Stratonikov
665e5807bc [#188] transformer: Allow to provide size hint 
For big objects with known size we can optimize allocation patterns
by providing size hint. As with any hint, it does not affect transformer
functionality: slices with capacity > MaxSize are never allocated.

```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/transformer
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
                                                │     out     │
                                                │   sec/op    │
Transformer/small/no_size_hint-8                  65.44µ ± 3%
Transformer/small/no_size_hint,_with_buffer-8     64.24µ ± 5%
Transformer/small/with_size_hint,_with_buffer-8   58.70µ ± 5%
Transformer/big/no_size_hint-8                    367.8m ± 3%
Transformer/big/no_size_hint,_with_buffer-8       562.7m ± 0%
Transformer/big/with_size_hint,_with_buffer-8     385.6m ± 7%
geomean                                           5.197m

                                                │     out      │
                                                │     B/op     │
Transformer/small/no_size_hint-8                  13.40Ki ± 0%
Transformer/small/no_size_hint,_with_buffer-8     13.40Ki ± 0%
Transformer/small/with_size_hint,_with_buffer-8   13.39Ki ± 0%
Transformer/big/no_size_hint-8                    288.0Mi ± 0%
Transformer/big/no_size_hint,_with_buffer-8       1.390Gi ± 0%
Transformer/big/with_size_hint,_with_buffer-8     288.0Mi ± 0%
geomean                                           2.533Mi

                                                │    out     │
                                                │ allocs/op  │
Transformer/small/no_size_hint-8                  92.00 ± 0%
Transformer/small/no_size_hint,_with_buffer-8     92.00 ± 0%
Transformer/small/with_size_hint,_with_buffer-8   92.00 ± 0%
Transformer/big/no_size_hint-8                    546.5 ± 0%
Transformer/big/no_size_hint,_with_buffer-8       607.5 ± 0%
Transformer/big/with_size_hint,_with_buffer-8     545.5 ± 0%
geomean                                           228.1
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-10-30 12:13:04 +00:00
Evgenii Stratonikov
49ad985cad [#161] *: Do not use math/rand.Read() 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-09-08 17:17:02 +03:00
Dmitrii Stepanov
c359a7465a [#64] transformer: Simplify interface 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-07-11 07:33:12 +00:00
Anton Nikiforov
ac95b87e7c [#101] Add Equals for Address 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2023-07-10 09:05:01 +00:00
Dmitrii Stepanov
aa8ffebc63 [#95] transformer: Set parent version 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-06-23 13:48:02 +03:00
Evgenii Stratonikov
d4fe9a193d [#66] transformer: Accept constructor in NextTarget 
The code of frostfs-node is not yet ready to reuse egress target for
multiple objects, let's postpone until #64.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-05-03 11:22:09 +03:00
Evgenii Stratonikov
c42a6119ff [#66] transformer: Extend tests 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-05-03 11:22:09 +03:00
Evgenii Stratonikov
29b188db57 [#52] Remove storage groups and audit 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-05-03 08:20:37 +00:00
Evgenii Stratonikov
38b03ff28b [#63] transformer: Resolve funlen linter 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-04-25 13:29:42 +03:00
Evgenii Stratonikov
0fa23a9b14 [#63] transformer: Add context to methods 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-04-25 13:29:42 +03:00
Evgenii Stratonikov
708d933fe3 [#47] .golangci.yml: Unify with other FrostFS repos 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-04-07 08:59:19 +03:00
Denis Kirillov
d589d51509 [#19] transformer: Fix dependencies 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-03-16 09:51:03 +03:00
Evgenii Stratonikov
25588ee3be [#19] transformer: Do not allocate intermeate slice for hashers 
```
name                 old time/op    new time/op    delta
Transformer/small-8    73.7µs ±15%    72.4µs ±16%    ~     (p=0.604 n=10+9)
Transformer/big-8       1.36s ± 4%     1.36s ± 8%    ~     (p=0.579 n=10+10)

name                 old alloc/op   new alloc/op   delta
Transformer/small-8    7.67kB ± 0%    7.57kB ± 0%  -1.36%  (p=0.000 n=10+10)
Transformer/big-8      49.0kB ± 0%    48.3kB ± 0%  -1.48%  (p=0.000 n=10+10)

name                 old allocs/op  new allocs/op  delta
Transformer/small-8       101 ± 0%        98 ± 0%  -2.97%  (p=0.000 n=10+10)
Transformer/big-8         609 ± 0%       591 ± 1%  -3.00%  (p=0.000 n=10+9)
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-03-15 05:39:09 +00:00
Evgenii Stratonikov
9407f30248 [#19] transformer: Optimize payload hashers a bit 
```
name                 old time/op    new time/op    delta
Transformer/small-8    74.8µs ±11%    73.7µs ±15%    ~     (p=0.529 n=10+10)
Transformer/big-8       1.38s ±11%     1.36s ± 4%    ~     (p=0.796 n=10+10)

name                 old alloc/op   new alloc/op   delta
Transformer/small-8    7.69kB ± 0%    7.67kB ± 0%  -0.21%  (p=0.000 n=10+10)
Transformer/big-8      49.2kB ± 0%    49.0kB ± 0%  -0.48%  (p=0.004 n=10+10)

name                 old allocs/op  new allocs/op  delta
Transformer/small-8       102 ± 0%       101 ± 0%  -0.98%  (p=0.000 n=9+10)
Transformer/big-8         620 ± 1%       609 ± 0%  -1.66%  (p=0.000 n=10+10)
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-03-15 05:39:09 +00:00
Evgenii Stratonikov
94c0a607b5 [#19] transformer: Add a target which sends parts to a channel 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-03-15 05:39:09 +00:00
Evgenii Stratonikov
e45647de3c [#19] transformer: Do not reuse memory of sent objects 
Slower, but more correct.
```
name                 old time/op    new time/op    delta
Transformer/small-8    72.4µs ± 8%    74.8µs ±11%     ~     (p=0.278 n=9+10)
Transformer/big-8       1.31s ± 8%     1.38s ±11%   +5.50%  (p=0.035 n=10+10)

name                 old alloc/op   new alloc/op   delta
Transformer/small-8    7.39kB ± 0%    7.69kB ± 0%   +4.04%  (p=0.000 n=10+10)
Transformer/big-8      46.9kB ± 0%    49.2kB ± 0%   +4.87%  (p=0.000 n=10+10)

name                 old allocs/op  new allocs/op  delta
Transformer/small-8      94.6 ± 1%     102.0 ± 0%   +7.82%  (p=0.000 n=10+9)
Transformer/big-8         560 ± 0%       620 ± 1%  +10.66%  (p=0.000 n=10+10)
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-03-15 05:39:09 +00:00
Evgenii Stratonikov
611e20587b [#19] transformer/test: Check owner ID and payload hash for parts 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-03-15 05:39:09 +00:00
Evgenii Stratonikov
eba6831125 [#19] transformer/test: Add helper functions 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-03-15 05:39:09 +00:00
Evgenii Stratonikov
7e3810d654 [#19] transformer: Move EpochSource to other types 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-03-15 05:39:09 +00:00
Evgenii Stratonikov
cc0fef2c55 [#19] transformer: Merge formatter and payload splitter 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-03-15 05:39:09 +00:00
Evgenii Stratonikov
b696d3c70e [#19] transformer: Make writeChunk non-recursive 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-03-15 05:39:09 +00:00
Evgenii Stratonikov
1c94309d7a [#19] transformer: Simplify AccessIdentifiers 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-03-15 05:39:09 +00:00
Evgenii Stratonikov
f43f18ecda [#19] transformer: Cover with unit-tests 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-03-15 05:39:09 +00:00
Evgenii Stratonikov
ac8442bf99 [#19] object: Move transformer implementation from node 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-03-15 05:39:09 +00:00
Evgenii Stratonikov
b461aa64b8 [#23] pre-commit: Add golangci-lint hook 
Also, fix minor issues. Skip deprecated warning for now.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-03-09 22:53:14 +03:00
Alex Vanin
94476f9055 Rename package name 
Due to source code relocation from GitHub.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-03-07 15:47:21 +03:00
Evgenii Stratonikov
4ff9c00de3 [#4] Rename NeoFS mentions in comments and method names 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2022-12-30 13:40:50 +03:00
Pavel Karpy
a1748ae0e7 [#370] bearer, session: Clarify expiration epoch 
The expiration epoch is the _last_ valid epoch for an entity. Also, clarify
the expiration epoch meaning for tombstones and regular objects.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2022-12-29 16:34:13 +03:00
Pavel Karpy
4c779423f5 Move to frostfs-sdk-go 
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2022-12-14 09:59:29 +03:00
Denis Kirillov
d047289182 [#360] object: Add new package relations 
Package relations provides feature to process inner object structure.

Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-11-14 17:46:17 +03:00
Denis Kirillov
74234623b2 [#215] object: Add well-known FilePath attribute 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-09-06 17:21:37 +04:00
Evgenii Stratonikov
84888854ab [#322] *: Go fmt -s 
go1.19 rewrites comments to proper render them in docs.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-08-24 18:58:59 +03:00
Leonard Lyubich
7d10b432d1 [#284] *: Return error from all ReadFromV2 methods 
Return `error` from all `ReadFromV2` methods in order to support
backward compatibility if message will be extended with some formatted
field.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-07-07 14:20:18 +03:00
Leonard Lyubich
30d27c3050 [#289] oid: Add JSON encoding of Address type 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-07-04 14:58:00 +03:00
Leonard Lyubich
82d762f536 [#258] Fix string encoding of identifiers 
Use `EncodeToString` method to get protocol string according to type
docs.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-05-31 12:14:04 +03:00
Leonard Lyubich
6cb513c976 [#257] Upgrade NeoFS API Go module 
New version contains fix for `object.GetRangeResponse` message type.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-05-31 09:47:55 +03:00
Leonard Lyubich
3bbf7ee15d [#252] object/id: Fix typos in docs 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-05-27 12:41:35 +03:00