dstepanov-yadro/frostfs-api-go
1
0
Fork 0
forked from TrueCloudLab/frostfs-api-go
Code Issues Pull requests Projects Releases Packages Wiki Activity
1344 commits 13 branches 61 tags 7.1 MiB
Commit graph

66 commits

Author SHA1 Message Date
Evgenii Stratonikov
29f2157563
[#123] protogen: Treat bytes field as non-nullable 
In protobuf 3.12 they have added an support for `optional` keyword,
which has made it into the main branch in 3.15.
https://github.com/protocolbuffers/protobuf/blob/main/docs/implementing_proto3_presence.md
https://github.com/protocolbuffers/protobuf/blob/v3.12.0/docs/field_presence.md#presence-in-proto3-apis

This means that without an explicit `optional` keyword field presence
for scalars is not tracked, thus empty string in JSON should be
unmarshaled to a nil byte slice. Relevant decoding code and tests from
protojson:
fb995f184a/internal/impl/message_reflect_field.go (L327)
fb995f184a/encoding/protojson/decode_test.go (L134)
fb995f184a/encoding/protojson/decode_test.go (L156)

We do not support `optional` keyword and the generator will fail if it sees on.
So only implement the default behaviour.

Refs #122

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-10-11 14:40:54 +03:00
Evgenii Stratonikov
29c522d5d8 [#122] protogen: Always marshal empty fields 
This is how it was done previously:
a0a9b765f3/rpc/message/encoding.go (L31)

The tricky part is `[]byte` which is marshaled as `null` by easyjson
helper, but as `""` by protojson.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-10-07 15:05:43 +03:00
Evgenii Stratonikov
d94b9c6d0d [#120] protogen: Unmarshal stringified integers from JSON 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-10-01 14:18:45 +03:00
Evgenii Stratonikov
eeb754c327 [#120] protogen: Omit empty fields from JSON output 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-10-01 14:11:43 +03:00
Evgenii Stratonikov
805da79319 [#120] protogen: Marshal enum as string 
Be compatible with protojson.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-10-01 14:11:43 +03:00
Ekaterina Lebedeva
bd588fa2e5 [#113] go.mod: Use range over int 
Since Go 1.22 a `for` statement with a `range` clause is able
to iterate through integer values from zero to an upper limit.

gopatch script:
@@
var i, e expression
@@
-for i := 0; i <= e - 1; i++ {
+for i := range e {
    ...
}

@@
var i, e expression
@@
-for i := 0; i <= e; i++ {
+for i := range e + 1 {
    ...
}

@@
var i, e expression
@@
-for i := 0; i < e; i++ {
+for i := range e {
    ...
}

Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
 2024-09-04 15:44:59 +03:00
Aleksey Savchuk
9c5e32a183
[#106] test: Generate correct data for tests 
Some tests are using invalid data that do not meet the requirements of the
FrostFS protocol, e.g.
- Container/Object IDs aren't 32 bytes long
- Signature key and sign have invalid length
- Split IDs aren't valid UUIDv4
and etc.

Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
 2024-08-30 13:37:29 +03:00
Evgenii Stratonikov
5e1c6a908f [#111] protogen: Emit slice of messages without a pointer 
```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
                                              │      old      │                 new                  │
                                              │    sec/op     │    sec/op     vs base                │
ObjectIDSlice/0_elements/to_grpc_message-8       3.193n ±  2%   3.242n ±  0%   +1.50% (p=0.034 n=10)
ObjectIDSlice/0_elements/from_grpc_message-8     3.197n ±  2%   3.343n ±  1%   +4.57% (p=0.000 n=10)
ObjectIDSlice/0_elements/marshal-8               5.666n ±  3%   5.642n ±  0%   -0.42% (p=0.000 n=10)
ObjectIDSlice/1_elements/to_grpc_message-8       53.10n ±  6%   29.78n ± 12%  -43.92% (p=0.000 n=10)
ObjectIDSlice/1_elements/from_grpc_message-8     28.99n ±  5%   29.77n ±  7%        ~ (p=0.165 n=10)
ObjectIDSlice/1_elements/marshal-8               49.08n ±  7%   50.72n ±  6%        ~ (p=0.218 n=10)
ObjectIDSlice/50_elements/to_grpc_message-8     1652.5n ±  7%   277.2n ±  1%  -83.22% (p=0.000 n=10)
ObjectIDSlice/50_elements/from_grpc_message-8    261.2n ± 11%   226.7n ± 15%  -13.19% (p=0.003 n=10)
ObjectIDSlice/50_elements/marshal-8              1.512µ ±  6%   1.514µ ±  6%        ~ (p=0.955 n=10)
geomean                                          52.15n         39.99n        -23.31%

                                              │      old       │                  new                   │
                                              │      B/op      │     B/op      vs base                  │
ObjectIDSlice/0_elements/to_grpc_message-8        0.000 ± 0%       0.000 ± 0%        ~ (p=1.000 n=10) ¹
ObjectIDSlice/0_elements/from_grpc_message-8      0.000 ± 0%       0.000 ± 0%        ~ (p=1.000 n=10) ¹
ObjectIDSlice/0_elements/marshal-8                0.000 ± 0%       0.000 ± 0%        ~ (p=1.000 n=10) ¹
ObjectIDSlice/1_elements/to_grpc_message-8        32.00 ± 0%       24.00 ± 0%  -25.00% (p=0.000 n=10)
ObjectIDSlice/1_elements/from_grpc_message-8      24.00 ± 0%       24.00 ± 0%        ~ (p=1.000 n=10) ¹
ObjectIDSlice/1_elements/marshal-8                48.00 ± 0%       48.00 ± 0%        ~ (p=1.000 n=10) ¹
ObjectIDSlice/50_elements/to_grpc_message-8     1.578Ki ± 0%     1.250Ki ± 0%  -20.79% (p=0.000 n=10)
ObjectIDSlice/50_elements/from_grpc_message-8   1.250Ki ± 0%     1.250Ki ± 0%        ~ (p=1.000 n=10) ¹
ObjectIDSlice/50_elements/marshal-8             2.000Ki ± 0%     2.000Ki ± 0%        ~ (p=1.000 n=10) ¹
geomean                                                      ²                  -5.62%                ²
¹ all samples are equal
² summaries must be >0 to compute geomean

                                              │      old      │                 new                  │
                                              │   allocs/op   │ allocs/op   vs base                  │
ObjectIDSlice/0_elements/to_grpc_message-8       0.000 ± 0%     0.000 ± 0%        ~ (p=1.000 n=10) ¹
ObjectIDSlice/0_elements/from_grpc_message-8     0.000 ± 0%     0.000 ± 0%        ~ (p=1.000 n=10) ¹
ObjectIDSlice/0_elements/marshal-8               0.000 ± 0%     0.000 ± 0%        ~ (p=1.000 n=10) ¹
ObjectIDSlice/1_elements/to_grpc_message-8       2.000 ± 0%     1.000 ± 0%  -50.00% (p=0.000 n=10)
ObjectIDSlice/1_elements/from_grpc_message-8     1.000 ± 0%     1.000 ± 0%        ~ (p=1.000 n=10) ¹
ObjectIDSlice/1_elements/marshal-8               1.000 ± 0%     1.000 ± 0%        ~ (p=1.000 n=10) ¹
ObjectIDSlice/50_elements/to_grpc_message-8     51.000 ± 0%     1.000 ± 0%  -98.04% (p=0.000 n=10)
ObjectIDSlice/50_elements/from_grpc_message-8    1.000 ± 0%     1.000 ± 0%        ~ (p=1.000 n=10) ¹
ObjectIDSlice/50_elements/marshal-8              1.000 ± 0%     1.000 ± 0%        ~ (p=1.000 n=10) ¹
geomean                                                     ²               -40.18%                ²
¹ all samples are equal
² summaries must be >0 to compute geomean
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-08-28 11:53:08 +03:00
Evgenii Stratonikov
5fece80b42 [#108] protogen: Distinguish between empty and nil messages 
Refs #59

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-08-27 11:02:18 +03:00
Evgenii Stratonikov
adb7c602d7 [#77] protogen: Initial implementation 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-08-19 10:47:00 +03:00
Evgenii Stratonikov
3dfa2f4fd6 [#95] *: Regenerate proto files 
Remove SetExtendedEACL and AnnounceUsedSpace methods from the container
package.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-07-26 10:24:25 +03:00
Evgenii Stratonikov
0fe6b9adbb [#76] Make setter names more consistent 
This will make the transition to a new codegen more seamless.
1. Setter should be named as the field itself.
2. Setter should accept 1 parameter.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-05-03 08:52:04 +03:00
Evgenii Stratonikov
1772b92182 [#69] object: Initial EC implementation 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-03-19 15:23:01 +03:00
Evgenii Stratonikov
17bed735a1 [#20] Update api version 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-08-17 11:10:12 +03:00
Airat Arifullin
9a7b47d769 [#50] util: Use protowire for util/proto helpers 
* Use methods to calculate size

Signed-off-by: Airat Arifullin a.arifullin@yadro.com
 2023-08-10 16:40:53 +00:00
Dmitrii Stepanov
ff6d8db741 [#19] subnet: Drop related types and fields 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-18 13:29:50 +03:00
Pavel Karpy
d9347a05f0 [#21] *: Drop reputation system 
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2023-04-17 15:46:52 +03:00
Alex Vanin
f69d2ad83c Rename package name 
Due to source code relocation from GitHub.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-03-07 13:42:36 +03:00
Evgenii Stratonikov
1351b6656d Move to frostfs-api 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2022-12-12 17:40:48 +03:00
Leonard Lyubich
cf868188ef [#418] netmap: Support NetmapService.NetmapSnapshot RPC 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-09-19 16:44:34 +04:00
Evgenii Stratonikov
ea7a3811bf [#415] *: Add maintenance status and node state 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-09-16 18:50:53 +04:00
Evgenii Stratonikov
1853349f22 [#415] *: Go fmt -s 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-09-16 18:50:53 +04:00
Pavel Karpy
1cd3a43d5b [#407] *: Regenerate v2.13.0 API 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-06-30 08:19:57 +03:00
Evgenii Stratonikov
dd233c3fbc [#386] util/signature: Add WalletConnect API support 
To avoid introducing new dependency (neo-go), crypto routines are used
as in other code.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-06-21 20:09:33 +03:00
Pavel Karpy
f9a91e5f33 [#404] *: Regenerate code after language fixes 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-06-21 14:08:31 +03:00
Pavel Karpy
870c1ffc0a [#397] refs: Fix OIDs marshalling 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-04-27 09:45:03 +03:00
Evgenii Stratonikov
2aacaa6aba refs: optimize marshalin of []ObjectID 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-04-21 15:07:57 +03:00
Evgenii Stratonikov
94f068e462 *: remove error from StableMarshal return values 
We marshal only in-memory structures, no error is expected.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-04-21 15:07:57 +03:00
Evgenii Stratonikov
84d981e88e util/proto: do not return error from primitive marshalers 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-04-21 15:07:57 +03:00
Evgenii Stratonikov
732dd51b1b [#388] *: Remove nil check from setters 
I knew one day `sed` would save me an hour of manual work:
```
sed -i -n -e '
s/) Set/) Set/
p
t setter
b end
:setter
    n
    s/nil/nil/
    t hasif
    p
    b end
    :hasif
        n
        :loop
        p
        n
        s/}/}/
        t end
        b loop
        :end
' $@
goimports -w $@
```

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-03-31 11:32:04 +03:00
Evgenii Stratonikov
30c530e83d [#376] container: Remove pointers from slices 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-03-15 12:19:49 +03:00
Evgenii Stratonikov
84daaf59ef [#376] refs: Replace []*ObjectID with []ObjectID 
```
ObjectIDSlice/0_elements/to_grpc_message-8       5.64ns ± 2%    5.89ns ± 4%   +4.56%  (p=0.000 n=10+10)
ObjectIDSlice/0_elements/from_grpc_message-8     6.68ns ± 3%    6.81ns ± 8%     ~     (p=0.143 n=10+10)
ObjectIDSlice/0_elements/marshal-8               7.41ns ± 3%    7.91ns ± 4%   +6.63%  (p=0.000 n=10+10)
ObjectIDSlice/1_elements/to_grpc_message-8       69.8ns ± 3%    80.5ns ± 7%  +15.39%  (p=0.000 n=10+10)
ObjectIDSlice/1_elements/from_grpc_message-8     56.4ns ± 6%    34.7ns ± 5%  -38.55%  (p=0.000 n=10+9)
ObjectIDSlice/1_elements/marshal-8               68.4ns ± 4%    67.6ns ± 4%     ~     (p=0.404 n=10+10)
ObjectIDSlice/50_elements/to_grpc_message-8      2.52µs ± 7%    2.56µs ± 4%     ~     (p=0.315 n=10+9)
ObjectIDSlice/50_elements/from_grpc_message-8    1.83µs ± 8%    0.44µs ± 1%  -75.73%  (p=0.000 n=10+8)
ObjectIDSlice/50_elements/marshal-8              2.32µs ±17%    2.22µs ± 3%     ~     (p=0.247 n=10+10)

name                                           old alloc/op   new alloc/op   delta
ObjectIDSlice/0_elements/to_grpc_message-8        0.00B          0.00B          ~     (all equal)
ObjectIDSlice/0_elements/from_grpc_message-8      0.00B          0.00B          ~     (all equal)
ObjectIDSlice/0_elements/marshal-8                0.00B          0.00B          ~     (all equal)
ObjectIDSlice/1_elements/to_grpc_message-8        72.0B ± 0%     72.0B ± 0%     ~     (all equal)
ObjectIDSlice/1_elements/from_grpc_message-8      32.0B ± 0%     24.0B ± 0%  -25.00%  (p=0.000 n=10+10)
ObjectIDSlice/1_elements/marshal-8                48.0B ± 0%     48.0B ± 0%     ~     (all equal)
ObjectIDSlice/50_elements/to_grpc_message-8      3.62kB ± 0%    3.62kB ± 0%     ~     (all equal)
ObjectIDSlice/50_elements/from_grpc_message-8    1.62kB ± 0%    1.28kB ± 0%  -20.79%  (p=0.000 n=10+10)
ObjectIDSlice/50_elements/marshal-8              2.05kB ± 0%    2.05kB ± 0%     ~     (all equal)

name                                           old allocs/op  new allocs/op  delta
ObjectIDSlice/0_elements/to_grpc_message-8         0.00           0.00          ~     (all equal)
ObjectIDSlice/0_elements/from_grpc_message-8       0.00           0.00          ~     (all equal)
ObjectIDSlice/0_elements/marshal-8                 0.00           0.00          ~     (all equal)
ObjectIDSlice/1_elements/to_grpc_message-8         2.00 ± 0%      2.00 ± 0%     ~     (all equal)
ObjectIDSlice/1_elements/from_grpc_message-8       2.00 ± 0%      1.00 ± 0%  -50.00%  (p=0.000 n=10+10)
ObjectIDSlice/1_elements/marshal-8                 1.00 ± 0%      1.00 ± 0%     ~     (all equal)
ObjectIDSlice/50_elements/to_grpc_message-8        51.0 ± 0%      51.0 ± 0%     ~     (all equal)
ObjectIDSlice/50_elements/from_grpc_message-8      51.0 ± 0%       1.0 ± 0%  -98.04%  (p=0.000 n=10+10)
ObjectIDSlice/50_elements/marshal-8                1.00 ± 0%      1.00 ± 0%     ~     (all equal)
```

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-03-15 12:19:49 +03:00
Evgenii Stratonikov
529d636ede [#376] refs: Add benchmarks for ObjectID slice conversions 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-03-15 12:19:49 +03:00
Leonard Lyubich
d065453bd0 [#380] Support changes in signature schemes 
Support new `SignatureRFC6979` message. Make `refs.ECDSA_SHA512` to be
default scheme.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-03-02 16:49:50 +03:00
Leonard Lyubich
fd3bf4a56b Fix linter remarks 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-02-25 11:12:57 +03:00
Evgenii Stratonikov
a4349f6692 [#55] refs: Add Scheme field to Signature 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-02-25 10:15:28 +03:00
Evgenii Stratonikov
66e1fb8c53 [#55] refs: Fix linter warnings 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-02-25 10:15:28 +03:00
Leonard Lyubich
e9a8451fb7 [#367] *: Re-compile proto files of NeoFS API protocol 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-01-14 13:17:21 +03:00
Leonard Lyubich
5ce4b22e40 [#356] refs: Implement Unmarshal method on SubnetID 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2021-11-24 17:13:18 +03:00
Leonard Lyubich
5d99e697aa [#356] refs: Test text marshaling of nil SubnetID 
Add `(*refs.SubnetID)(nil).MarshalText()` unit test.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2021-11-24 17:13:18 +03:00
Leonard Lyubich
dc292864aa [#356] refs: Handle uint32 overflow in SubnetID text format 
Clarify the bit size limit in `SubnetID.UnmarshalText` method. Cover
overflow case in unit test.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2021-11-24 17:13:18 +03:00
Leonard Lyubich
0e6e0e4678 [#356] refs: Implement text encoding of SubnetID 
Implement `encoding.TextMarshaler` / `encoding.TextUnmarshaler` interfaces
on `SubnetID` according to NeoFS API V2 protocol.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2021-11-24 17:13:18 +03:00
Leonard Lyubich
f0af5ce759 [#356] refs: Add functions to work with zero subnet 
Add helper functions which provide ease of use with subnet zero IDs.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2021-11-24 17:13:18 +03:00
Leonard Lyubich
580f6c5554 [#355] subnet: Implement types for the new messages 
Define `subnet.Info` structure of corresponding message from NeoFS API
protocol. Provide field getters and setters. Implement
`StableMarshal` / `StableSize` methods of binary encoding. Implement
`ToGRPCMessage` / `FromGRPCMessage` method for testing and
potential transport.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2021-11-18 17:40:33 +03:00
Evgenii Stratonikov
68f28a1918 [#355] v2/refs: add subnetwork ID 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2021-11-18 15:04:15 +03:00
Evgenii Stratonikov
ad6a7999cd [#355] *: regenerate proto files 
Add subnetwork ID to placement policy.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2021-11-18 15:04:15 +03:00
Leonard Lyubich
0f09f0dfc6 Re-compile NeoFS API proto files using updated script 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2021-11-17 15:29:33 +03:00
Leonard Lyubich
25da5d2e13 Add v2 version to go module name 
Replace all elements from `v2` to root directory.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2021-11-17 15:29:33 +03:00
Alex Vanin
1f143e54bd Move api-v2 files into v2 subdir 
This subdir contains generated proto files
and small wrappers.
 2020-09-18 10:40:17 +03:00
Leonard Lyubich
0ee1c3653d Implement field setters on all protobuf messages 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2020-09-18 10:40:17 +03:00