Commit graph

86 commits

Author SHA1 Message Date
b06dad731c [#120] protogen: Marshal 64-bit integers as strings
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-10-01 14:18:46 +03:00
d94b9c6d0d [#120] protogen: Unmarshal stringified integers from JSON
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-10-01 14:18:45 +03:00
eeb754c327 [#120] protogen: Omit empty fields from JSON output
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-10-01 14:11:43 +03:00
805da79319 [#120] protogen: Marshal enum as string
Be compatible with protojson.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-10-01 14:11:43 +03:00
9c5e32a183
[#106] test: Generate correct data for tests
Some tests are using invalid data that do not meet the requirements of the
FrostFS protocol, e.g.
- Container/Object IDs aren't 32 bytes long
- Signature key and sign have invalid length
- Split IDs aren't valid UUIDv4
and etc.

Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-08-30 13:37:29 +03:00
5e1c6a908f [#111] protogen: Emit slice of messages without a pointer
```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
                                              │      old      │                 new                  │
                                              │    sec/op     │    sec/op     vs base                │
ObjectIDSlice/0_elements/to_grpc_message-8       3.193n ±  2%   3.242n ±  0%   +1.50% (p=0.034 n=10)
ObjectIDSlice/0_elements/from_grpc_message-8     3.197n ±  2%   3.343n ±  1%   +4.57% (p=0.000 n=10)
ObjectIDSlice/0_elements/marshal-8               5.666n ±  3%   5.642n ±  0%   -0.42% (p=0.000 n=10)
ObjectIDSlice/1_elements/to_grpc_message-8       53.10n ±  6%   29.78n ± 12%  -43.92% (p=0.000 n=10)
ObjectIDSlice/1_elements/from_grpc_message-8     28.99n ±  5%   29.77n ±  7%        ~ (p=0.165 n=10)
ObjectIDSlice/1_elements/marshal-8               49.08n ±  7%   50.72n ±  6%        ~ (p=0.218 n=10)
ObjectIDSlice/50_elements/to_grpc_message-8     1652.5n ±  7%   277.2n ±  1%  -83.22% (p=0.000 n=10)
ObjectIDSlice/50_elements/from_grpc_message-8    261.2n ± 11%   226.7n ± 15%  -13.19% (p=0.003 n=10)
ObjectIDSlice/50_elements/marshal-8              1.512µ ±  6%   1.514µ ±  6%        ~ (p=0.955 n=10)
geomean                                          52.15n         39.99n        -23.31%

                                              │      old       │                  new                   │
                                              │      B/op      │     B/op      vs base                  │
ObjectIDSlice/0_elements/to_grpc_message-8        0.000 ± 0%       0.000 ± 0%        ~ (p=1.000 n=10) ¹
ObjectIDSlice/0_elements/from_grpc_message-8      0.000 ± 0%       0.000 ± 0%        ~ (p=1.000 n=10) ¹
ObjectIDSlice/0_elements/marshal-8                0.000 ± 0%       0.000 ± 0%        ~ (p=1.000 n=10) ¹
ObjectIDSlice/1_elements/to_grpc_message-8        32.00 ± 0%       24.00 ± 0%  -25.00% (p=0.000 n=10)
ObjectIDSlice/1_elements/from_grpc_message-8      24.00 ± 0%       24.00 ± 0%        ~ (p=1.000 n=10) ¹
ObjectIDSlice/1_elements/marshal-8                48.00 ± 0%       48.00 ± 0%        ~ (p=1.000 n=10) ¹
ObjectIDSlice/50_elements/to_grpc_message-8     1.578Ki ± 0%     1.250Ki ± 0%  -20.79% (p=0.000 n=10)
ObjectIDSlice/50_elements/from_grpc_message-8   1.250Ki ± 0%     1.250Ki ± 0%        ~ (p=1.000 n=10) ¹
ObjectIDSlice/50_elements/marshal-8             2.000Ki ± 0%     2.000Ki ± 0%        ~ (p=1.000 n=10) ¹
geomean                                                      ²                  -5.62%                ²
¹ all samples are equal
² summaries must be >0 to compute geomean

                                              │      old      │                 new                  │
                                              │   allocs/op   │ allocs/op   vs base                  │
ObjectIDSlice/0_elements/to_grpc_message-8       0.000 ± 0%     0.000 ± 0%        ~ (p=1.000 n=10) ¹
ObjectIDSlice/0_elements/from_grpc_message-8     0.000 ± 0%     0.000 ± 0%        ~ (p=1.000 n=10) ¹
ObjectIDSlice/0_elements/marshal-8               0.000 ± 0%     0.000 ± 0%        ~ (p=1.000 n=10) ¹
ObjectIDSlice/1_elements/to_grpc_message-8       2.000 ± 0%     1.000 ± 0%  -50.00% (p=0.000 n=10)
ObjectIDSlice/1_elements/from_grpc_message-8     1.000 ± 0%     1.000 ± 0%        ~ (p=1.000 n=10) ¹
ObjectIDSlice/1_elements/marshal-8               1.000 ± 0%     1.000 ± 0%        ~ (p=1.000 n=10) ¹
ObjectIDSlice/50_elements/to_grpc_message-8     51.000 ± 0%     1.000 ± 0%  -98.04% (p=0.000 n=10)
ObjectIDSlice/50_elements/from_grpc_message-8    1.000 ± 0%     1.000 ± 0%        ~ (p=1.000 n=10) ¹
ObjectIDSlice/50_elements/marshal-8              1.000 ± 0%     1.000 ± 0%        ~ (p=1.000 n=10) ¹
geomean                                                     ²               -40.18%                ²
¹ all samples are equal
² summaries must be >0 to compute geomean
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-08-28 11:53:08 +03:00
5fece80b42 [#108] protogen: Distinguish between empty and nil messages
Refs #59

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-08-27 11:02:18 +03:00
9e82a5a31a [#107] Regenerate proto files
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-08-26 14:36:19 +03:00
adb7c602d7 [#77] protogen: Initial implementation
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-08-19 10:47:00 +03:00
ebaf78c8fa [#100] session: Introduce ObjectPatch verb
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-08-06 12:31:11 +03:00
3dfa2f4fd6 [#95] *: Regenerate proto files
Remove SetExtendedEACL and AnnounceUsedSpace methods from the container
package.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-26 10:24:25 +03:00
0fe6b9adbb [#76] Make setter names more consistent
This will make the transition to a new codegen more seamless.
1. Setter should be named as the field itself.
2. Setter should accept 1 parameter.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-05-03 08:52:04 +03:00
4a330a5706 [#72] session: Unify unmarshaling code
gopatch:
```
@@
var err, msg, data, t, protojson identifier
var obj expression
@@
+import "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/message"
-import protojson "google.golang.org/protobuf/encoding/protojson"

-msg := obj
-if err := protojson.Unmarshal(data, msg); err != nil {
-return err
-}
-return t.FromGRPCMessage(msg)
+return message.UnmarshalJSON(t, data, obj)

@@
var err, msg, data, t, goproto identifier
var obj expression
@@
+import "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/message"
-import goproto "google.golang.org/protobuf/proto"

-msg := obj
-if err := goproto.Unmarshal(data, msg); err != nil {
-return err
-}
-return t.FromGRPCMessage(msg)
+return message.Unmarshal(t, data, obj)
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-20 14:19:19 +03:00
1772b92182 [#69] object: Initial EC implementation
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-03-19 15:23:01 +03:00
a85146250b [#68] Makefile: Use gofumpt for formatting
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-03-14 10:31:58 +03:00
9cc2095446 [#59] util: Restore backwards compatibility in NestedStructureMarshal()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-11-21 15:27:57 +03:00
17bed735a1 [#20] Update api version
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-17 11:10:12 +03:00
d989c8d2a3 [#54] *: Fix linter warnings
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-11 15:21:27 +03:00
43ad0f114c [#49] session: Make StableSize() zero-alloc
For this to work, it is necessary that `NestedStructureSize` is a generic function.
Otherwise, we would allocate to put it in the interface.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-07-26 18:31:31 +03:00
63915bb7a5 [#27] *: Resolve godot linter warnings
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-03 14:15:56 +03:00
Pavel Karpy
d9347a05f0 [#21] *: Drop reputation system
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2023-04-17 15:46:52 +03:00
9dc3753467 [#13] *: Rename __FROSTFS__ prefix to __SYSTEM__
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-03-15 12:52:36 +03:00
8009022a20 [#13] *: Regenerate api
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-03-15 12:52:28 +03:00
cd2e46a17c [#10] Add __FROSTFS__ system attributes
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-03-13 09:59:22 +03:00
c46cd37f71 [#10] Generate api
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-03-13 09:59:22 +03:00
f69d2ad83c Rename package name
Due to source code relocation from GitHub.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-03-07 13:42:36 +03:00
1351b6656d Move to frostfs-api
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2022-12-12 17:40:48 +03:00
Leonard Lyubich
5fc2644c68 [#416] session: Support new format of object session context
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-10-04 18:29:57 +04:00
Leonard Lyubich
cf868188ef [#418] netmap: Support NetmapService.NetmapSnapshot RPC
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-09-19 16:44:34 +04:00
Evgenii Stratonikov
ea7a3811bf [#415] *: Add maintenance status and node state
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-09-16 18:50:53 +04:00
Evgenii Stratonikov
1853349f22 [#415] *: Go fmt -s
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-09-16 18:50:53 +04:00
Leonard Lyubich
cdce7edc39 Release v2.13.0 - Yeonpyeongdo (연평도, 延坪島)
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-07-04 17:49:02 +03:00
Pavel Karpy
f9a91e5f33 [#404] *: Regenerate code after language fixes
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-06-21 14:08:31 +03:00
Leonard Lyubich
c82dcf7e16 [#401] internal: Place randomization code in random package
Also replace seeding into `init` function.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-05-30 22:02:58 +03:00
Leonard Lyubich
d35e935a9d [#401] test: Randomize value for oneof field
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-05-30 22:02:58 +03:00
Evgenii Stratonikov
94f068e462 *: remove error from StableMarshal return values
We marshal only in-memory structures, no error is expected.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-04-21 15:07:57 +03:00
Evgenii Stratonikov
84d981e88e util/proto: do not return error from primitive marshalers
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-04-21 15:07:57 +03:00
Evgenii Stratonikov
732dd51b1b [#388] *: Remove nil check from setters
I knew one day `sed` would save me an hour of manual work:
```
sed -i -n -e '
s/) Set/) Set/
p
t setter
b end
:setter
    n
    s/nil/nil/
    t hasif
    p
    b end
    :hasif
        n
        :loop
        p
        n
        s/}/}/
        t end
        b loop
        :end
' $@
goimports -w $@
```

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-03-31 11:32:04 +03:00
Evgenii Stratonikov
e21a7823c5 [#376] session: Make XHeader slice non-pointer
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-03-15 12:19:49 +03:00
Leonard Lyubich
fd3bf4a56b Fix linter remarks
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-02-25 11:12:57 +03:00
Leonard Lyubich
99370889d1 [#371] Support recent changes in NeoFS API protocol
Support:
  * new status codes (object, container, session);
  * object `Lock` message;
  * different signature schemes.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-02-25 10:24:07 +03:00
Leonard Lyubich
227a871a04 [#367] session: Support magic_number field of RequestMetaHeader
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-01-14 13:17:21 +03:00
Leonard Lyubich
e9a8451fb7 [#367] *: Re-compile proto files of NeoFS API protocol
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-01-14 13:17:21 +03:00
Leonard Lyubich
0f09f0dfc6 Re-compile NeoFS API proto files using updated script
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-11-17 15:29:33 +03:00
Leonard Lyubich
25da5d2e13 Add v2 version to go module name
Replace all elements from `v2` to root directory.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-11-17 15:29:33 +03:00
Alex Vanin
1f143e54bd Move api-v2 files into v2 subdir
This subdir contains generated proto files
and small wrappers.
2020-09-18 10:40:17 +03:00
Leonard Lyubich
0ee1c3653d Implement field setters on all protobuf messages
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-09-18 10:40:17 +03:00
Alex Vanin
fabdd78d63 Add pre-release jindo branch 2020-09-18 10:39:50 +03:00
Alex Vanin
0a5d0ff1a2 Remove v1 code 2020-09-18 10:39:45 +03:00
Leonard Lyubich
74e917810a service: support broken apart signable payload of the requests
In previous implementation service package provided types and functions
that wrapped signing/verification of data with session token.
This allowed us to use these functions for signing / verification of
service requests of other packages. To support the expansion of messages
with additional parts that need to be signed, you must be able to easily
expand the signed data with new parts.

To achieve the described goal, this commit makes the following changes:

  * adds GroupSignedPayloads and GroupVerifyPayloads functions;

  * renames SignedDataWithToken to RequestData, DataWithTokenSignAccumulator
    to RequestSignedData, DataWithTokenSignSource to RequestVerifyData;

  * renames SignDataWithSessionToken/VerifyAccumulatedSignaturesWithToken
    function to SignRequestData/VerifyRequestData and makes it to use
    GroupSignedPayloads/GroupVerifyPayloads internally.
2020-06-10 20:37:10 +03:00