Commit graph

1343 commits

Author SHA1 Message Date
29f2157563
[#123] protogen: Treat bytes field as non-nullable
In protobuf 3.12 they have added an support for `optional` keyword,
which has made it into the main branch in 3.15.
https://github.com/protocolbuffers/protobuf/blob/main/docs/implementing_proto3_presence.md
https://github.com/protocolbuffers/protobuf/blob/v3.12.0/docs/field_presence.md#presence-in-proto3-apis

This means that without an explicit `optional` keyword field presence
for scalars is not tracked, thus empty string in JSON should be
unmarshaled to a nil byte slice. Relevant decoding code and tests from
protojson:
fb995f184a/internal/impl/message_reflect_field.go (L327)
fb995f184a/encoding/protojson/decode_test.go (L134)
fb995f184a/encoding/protojson/decode_test.go (L156)

We do not support `optional` keyword and the generator will fail if it sees on.
So only implement the default behaviour.

Refs #122

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-10-11 14:40:54 +03:00
29c522d5d8 [#122] protogen: Always marshal empty fields
All checks were successful
DCO action / DCO (pull_request) Successful in 1m2s
Tests and linters / Lint (pull_request) Successful in 1m2s
Tests and linters / Tests (pull_request) Successful in 1m16s
Tests and linters / Tests with -race (pull_request) Successful in 1m16s
This is how it was done previously:
a0a9b765f3/rpc/message/encoding.go (L31)

The tricky part is `[]byte` which is marshaled as `null` by easyjson
helper, but as `""` by protojson.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-10-07 15:05:43 +03:00
3e705a3cbe [#121] .golangci.yml: Replace exportloopref with copyloopvar
All checks were successful
Tests and linters / Lint (pull_request) Successful in 52s
Tests and linters / Tests with -race (pull_request) Successful in 1m7s
DCO action / DCO (pull_request) Successful in 1m22s
Tests and linters / Tests (pull_request) Successful in 1m36s
Fix linter warning:
```
WARN The linter 'exportloopref' is deprecated (since v1.60.2) due to: Since Go1.22 (loopvar) this linter is no longer relevant. Replaced by copyloopvar.
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-10-02 09:48:11 +03:00
d9a604fbc1 [#120] proto/test: Unskip protojson compatibility test
All checks were successful
Tests and linters / Lint (pull_request) Successful in 44s
DCO action / DCO (pull_request) Successful in 1m6s
Tests and linters / Tests (pull_request) Successful in 1m12s
Tests and linters / Tests with -race (pull_request) Successful in 1m14s
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-10-01 14:18:52 +03:00
b06dad731c [#120] protogen: Marshal 64-bit integers as strings
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-10-01 14:18:46 +03:00
d94b9c6d0d [#120] protogen: Unmarshal stringified integers from JSON
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-10-01 14:18:45 +03:00
eeb754c327 [#120] protogen: Omit empty fields from JSON output
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-10-01 14:11:43 +03:00
805da79319 [#120] protogen: Marshal enum as string
Be compatible with protojson.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-10-01 14:11:43 +03:00
f812b1ae5b [#120] Regenerate proto files
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-10-01 14:11:43 +03:00
287e98ad67 [#120] proto/test: Add protojson compatibility test
It is failing, thus is skipped.
But implement it now to make it easier to see it failing.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-10-01 14:11:43 +03:00
c3dbbc5eab
[#118] status: Support INVALID_ARGUMENT status
All checks were successful
DCO action / DCO (pull_request) Successful in 1m13s
Tests and linters / Lint (pull_request) Successful in 1m26s
Tests and linters / Tests (pull_request) Successful in 1m30s
Tests and linters / Tests with -race (pull_request) Successful in 1m36s
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-09-24 14:38:14 +03:00
13fa0da374 [#117] rpc: Allow to specify custom gRPC dialer
All checks were successful
Tests and linters / Lint (pull_request) Successful in 41s
DCO action / DCO (pull_request) Successful in 45s
Tests and linters / Tests (pull_request) Successful in 46s
Tests and linters / Tests with -race (pull_request) Successful in 56s
After grpc upgrade there is no DialContext call.
So connection is not actually established after created.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-16 12:35:37 +03:00
c9782cf3ef [#117] go.mod: Upgrade grpc to v1.66.2
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-13 17:22:52 +03:00
c49c482ba6 [#116] Update obsolete URLs
All checks were successful
Tests and linters / Lint (pull_request) Successful in 50s
DCO action / DCO (pull_request) Successful in 1m8s
Tests and linters / Tests (pull_request) Successful in 1m46s
Tests and linters / Tests with -race (pull_request) Successful in 2m3s
Signed-off-by: Vitaliy Potyarkin <v.potyarkin@yadro.com>
2024-09-11 14:09:12 +03:00
0484647aae [#115] netmap: Fix type getters
* Add type instance check for nil to avoid panic by
  accessing fields.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-09-06 12:54:07 +00:00
9c0007fb1d [#115] apemanager: Fix type getters
* Add type instance check for nil to avoid panic by
  accessing fields.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-09-06 12:54:07 +00:00
bd588fa2e5 [#113] go.mod: Use range over int
All checks were successful
DCO action / DCO (pull_request) Successful in 41s
Tests and linters / Lint (pull_request) Successful in 1m11s
Tests and linters / Tests (1.22) (pull_request) Successful in 1m11s
Tests and linters / Tests (1.23) (pull_request) Successful in 1m10s
Tests and linters / Tests with -race (pull_request) Successful in 1m23s
Since Go 1.22 a `for` statement with a `range` clause is able
to iterate through integer values from zero to an upper limit.

gopatch script:
@@
var i, e expression
@@
-for i := 0; i <= e - 1; i++ {
+for i := range e {
    ...
}

@@
var i, e expression
@@
-for i := 0; i <= e; i++ {
+for i := range e + 1 {
    ...
}

@@
var i, e expression
@@
-for i := 0; i < e; i++ {
+for i := range e {
    ...
}

Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-09-04 15:44:59 +03:00
7d71556eee [#113] pre-commit: Update golangci-lint to v1.60.3
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-09-04 15:43:24 +03:00
c11f50efec
[#112] container: Remove GetExtendedACL
All checks were successful
DCO action / DCO (pull_request) Successful in 45s
Tests and linters / Tests (1.22) (pull_request) Successful in 1m5s
Tests and linters / Tests (1.23) (pull_request) Successful in 1m6s
Tests and linters / Lint (pull_request) Successful in 1m12s
Tests and linters / Tests with -race (pull_request) Successful in 1m20s
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-09-02 14:10:49 +03:00
9c5e32a183
[#106] test: Generate correct data for tests
All checks were successful
DCO action / DCO (pull_request) Successful in 42s
Tests and linters / Tests (1.22) (pull_request) Successful in 1m2s
Tests and linters / Tests (1.23) (pull_request) Successful in 1m1s
Tests and linters / Lint (pull_request) Successful in 1m6s
Tests and linters / Tests with -race (pull_request) Successful in 1m14s
Some tests are using invalid data that do not meet the requirements of the
FrostFS protocol, e.g.
- Container/Object IDs aren't 32 bytes long
- Signature key and sign have invalid length
- Split IDs aren't valid UUIDv4
and etc.

Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-08-30 13:37:29 +03:00
5e1c6a908f [#111] protogen: Emit slice of messages without a pointer
All checks were successful
DCO action / DCO (pull_request) Successful in 41s
Tests and linters / Tests (1.22) (pull_request) Successful in 55s
Tests and linters / Tests (1.23) (pull_request) Successful in 57s
Tests and linters / Tests with -race (pull_request) Successful in 1m8s
Tests and linters / Lint (pull_request) Successful in 2m12s
```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
                                              │      old      │                 new                  │
                                              │    sec/op     │    sec/op     vs base                │
ObjectIDSlice/0_elements/to_grpc_message-8       3.193n ±  2%   3.242n ±  0%   +1.50% (p=0.034 n=10)
ObjectIDSlice/0_elements/from_grpc_message-8     3.197n ±  2%   3.343n ±  1%   +4.57% (p=0.000 n=10)
ObjectIDSlice/0_elements/marshal-8               5.666n ±  3%   5.642n ±  0%   -0.42% (p=0.000 n=10)
ObjectIDSlice/1_elements/to_grpc_message-8       53.10n ±  6%   29.78n ± 12%  -43.92% (p=0.000 n=10)
ObjectIDSlice/1_elements/from_grpc_message-8     28.99n ±  5%   29.77n ±  7%        ~ (p=0.165 n=10)
ObjectIDSlice/1_elements/marshal-8               49.08n ±  7%   50.72n ±  6%        ~ (p=0.218 n=10)
ObjectIDSlice/50_elements/to_grpc_message-8     1652.5n ±  7%   277.2n ±  1%  -83.22% (p=0.000 n=10)
ObjectIDSlice/50_elements/from_grpc_message-8    261.2n ± 11%   226.7n ± 15%  -13.19% (p=0.003 n=10)
ObjectIDSlice/50_elements/marshal-8              1.512µ ±  6%   1.514µ ±  6%        ~ (p=0.955 n=10)
geomean                                          52.15n         39.99n        -23.31%

                                              │      old       │                  new                   │
                                              │      B/op      │     B/op      vs base                  │
ObjectIDSlice/0_elements/to_grpc_message-8        0.000 ± 0%       0.000 ± 0%        ~ (p=1.000 n=10) ¹
ObjectIDSlice/0_elements/from_grpc_message-8      0.000 ± 0%       0.000 ± 0%        ~ (p=1.000 n=10) ¹
ObjectIDSlice/0_elements/marshal-8                0.000 ± 0%       0.000 ± 0%        ~ (p=1.000 n=10) ¹
ObjectIDSlice/1_elements/to_grpc_message-8        32.00 ± 0%       24.00 ± 0%  -25.00% (p=0.000 n=10)
ObjectIDSlice/1_elements/from_grpc_message-8      24.00 ± 0%       24.00 ± 0%        ~ (p=1.000 n=10) ¹
ObjectIDSlice/1_elements/marshal-8                48.00 ± 0%       48.00 ± 0%        ~ (p=1.000 n=10) ¹
ObjectIDSlice/50_elements/to_grpc_message-8     1.578Ki ± 0%     1.250Ki ± 0%  -20.79% (p=0.000 n=10)
ObjectIDSlice/50_elements/from_grpc_message-8   1.250Ki ± 0%     1.250Ki ± 0%        ~ (p=1.000 n=10) ¹
ObjectIDSlice/50_elements/marshal-8             2.000Ki ± 0%     2.000Ki ± 0%        ~ (p=1.000 n=10) ¹
geomean                                                      ²                  -5.62%                ²
¹ all samples are equal
² summaries must be >0 to compute geomean

                                              │      old      │                 new                  │
                                              │   allocs/op   │ allocs/op   vs base                  │
ObjectIDSlice/0_elements/to_grpc_message-8       0.000 ± 0%     0.000 ± 0%        ~ (p=1.000 n=10) ¹
ObjectIDSlice/0_elements/from_grpc_message-8     0.000 ± 0%     0.000 ± 0%        ~ (p=1.000 n=10) ¹
ObjectIDSlice/0_elements/marshal-8               0.000 ± 0%     0.000 ± 0%        ~ (p=1.000 n=10) ¹
ObjectIDSlice/1_elements/to_grpc_message-8       2.000 ± 0%     1.000 ± 0%  -50.00% (p=0.000 n=10)
ObjectIDSlice/1_elements/from_grpc_message-8     1.000 ± 0%     1.000 ± 0%        ~ (p=1.000 n=10) ¹
ObjectIDSlice/1_elements/marshal-8               1.000 ± 0%     1.000 ± 0%        ~ (p=1.000 n=10) ¹
ObjectIDSlice/50_elements/to_grpc_message-8     51.000 ± 0%     1.000 ± 0%  -98.04% (p=0.000 n=10)
ObjectIDSlice/50_elements/from_grpc_message-8    1.000 ± 0%     1.000 ± 0%        ~ (p=1.000 n=10) ¹
ObjectIDSlice/50_elements/marshal-8              1.000 ± 0%     1.000 ± 0%        ~ (p=1.000 n=10) ¹
geomean                                                     ²               -40.18%                ²
¹ all samples are equal
² summaries must be >0 to compute geomean
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-08-28 11:53:08 +03:00
a2025376fc [#111] proto/test: Add repeated message test
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-08-28 11:53:08 +03:00
eba18f6e67 [#110] object: Add getter and setter for PatchResponseBody
All checks were successful
DCO action / DCO (pull_request) Successful in 1m19s
Tests and linters / Tests (1.22) (pull_request) Successful in 1m21s
Tests and linters / Tests (1.23) (pull_request) Successful in 1m28s
Tests and linters / Tests with -race (pull_request) Successful in 1m48s
Tests and linters / Lint (pull_request) Successful in 2m34s
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-08-27 13:46:00 +03:00
ca33fc4adb [#109] rpc/message: Remove incorrect copypaste
All checks were successful
DCO action / DCO (pull_request) Successful in 1m22s
Tests and linters / Tests (1.23) (pull_request) Successful in 1m18s
Tests and linters / Tests (1.22) (pull_request) Successful in 1m21s
Tests and linters / Tests with -race (pull_request) Successful in 1m36s
Tests and linters / Lint (pull_request) Successful in 2m33s
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-08-27 11:36:08 +03:00
5fece80b42 [#108] protogen: Distinguish between empty and nil messages
All checks were successful
DCO action / DCO (pull_request) Successful in 39s
Tests and linters / Tests (1.22) (pull_request) Successful in 59s
Tests and linters / Tests (1.23) (pull_request) Successful in 57s
Tests and linters / Tests with -race (pull_request) Successful in 1m9s
Tests and linters / Lint (pull_request) Successful in 2m12s
Refs #59

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-08-27 11:02:18 +03:00
11e194d274 [#108] rpc/message: Add compatibility tests for marshaling
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-08-27 11:02:18 +03:00
9e82a5a31a [#107] Regenerate proto files
All checks were successful
Tests and linters / Tests (1.23) (pull_request) Successful in 1m1s
Tests and linters / Tests with -race (pull_request) Successful in 1m32s
Tests and linters / Tests (1.22) (pull_request) Successful in 45s
Tests and linters / Lint (pull_request) Successful in 1m56s
DCO action / DCO (pull_request) Successful in 28s
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-08-26 14:36:19 +03:00
981dc785f3 [#107] proto/test: Add oneof test
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-08-26 14:36:19 +03:00
866db105ed [#107] protogen: Unify oneof getters with default protoc plugin
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-08-26 14:36:19 +03:00
937a53683a [#107] protogen: Fix oneof JSON marshaling
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-08-26 14:36:19 +03:00
4a00ef946f [#107] protogen: Fix oneof getter scalar default value
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-08-26 14:36:19 +03:00
f484ce6b0b [#107] Makefile: Generate standard protobuf bindings for tests
It was removed accidentally during transition.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-08-26 14:36:19 +03:00
61f6f0f4a2 [#107] proto/test: Regenerate *.pb.go
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-08-26 14:36:19 +03:00
c1b5f46f98 [#107] protogen: Remove unused parameter
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-08-26 14:36:19 +03:00
33653962b7 [#1316] go.mod: Bump go version to 1.22
All checks were successful
DCO action / DCO (pull_request) Successful in 42s
Tests and linters / Tests (1.22) (pull_request) Successful in 49s
Tests and linters / Tests (1.23) (pull_request) Successful in 1m6s
Tests and linters / Tests with -race (pull_request) Successful in 1m8s
Tests and linters / Lint (pull_request) Successful in 3m6s
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-08-21 17:22:56 +03:00
a43110e363 [#77] Makefile: Mark protogen as .PHONY
All checks were successful
Tests and linters / Tests (1.19) (pull_request) Successful in 48s
Tests and linters / Tests (1.20) (pull_request) Successful in 49s
Tests and linters / Tests with -race (pull_request) Successful in 1m5s
DCO action / DCO (pull_request) Successful in 33s
Tests and linters / Lint (pull_request) Successful in 50s
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-08-19 10:47:00 +03:00
7be31eb847 [#77] protogen: Add tests for JSON format
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-08-19 10:47:00 +03:00
adb7c602d7 [#77] protogen: Initial implementation
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-08-19 10:47:00 +03:00
a28ceb251a [#77] util/proto: Optimize int32 marshaling
This is the approach used in easyproto
52d3ac4744/writer.go (L203)

It allows to occupy slightly less space for negative numbers.
The format is still protobuf, although, technically, this is a breaking
change for our stable marshaling format.
However, we don't use int32 at all and all enums have positive values,
so nothing is broken.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-08-16 17:11:13 +03:00
d112a28d38 [#104] object: Add getters for PatchRequestBodyPatch
All checks were successful
DCO action / DCO (pull_request) Successful in 1m8s
Tests and linters / Tests (1.19) (pull_request) Successful in 1m12s
Tests and linters / Tests (1.20) (pull_request) Successful in 1m17s
Tests and linters / Lint (pull_request) Successful in 1m28s
Tests and linters / Tests with -race (pull_request) Successful in 1m35s
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-08-13 18:51:51 +03:00
47a48969b0 [#103] proto: Test end-to-end scenario
All checks were successful
DCO action / DCO (pull_request) Successful in 1m27s
Tests and linters / Tests (1.19) (pull_request) Successful in 1m36s
Tests and linters / Tests (1.20) (pull_request) Successful in 1m43s
Tests and linters / Lint (pull_request) Successful in 1m51s
Tests and linters / Tests with -race (pull_request) Successful in 2m6s
Test the generated code, do not write yet another marshaling routine in
tests.

Before:
```
ok      git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/util/proto      0.003s  coverage: 55.6% of statements
```

After:
```
ok      git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/util/proto      0.003s  coverage: 80.0% of statements
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-08-09 11:18:17 +03:00
19247e8941 [#103] protogen: Handle uint32 type
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-08-09 11:18:17 +03:00
ff4f31b6f3 [#103] protogen: Handle files in all packages
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-08-09 11:18:17 +03:00
a0a9b765f3 [#101] Fix make test
All checks were successful
DCO action / DCO (pull_request) Successful in 43s
Tests and linters / Tests (1.19) (pull_request) Successful in 34s
Tests and linters / Tests (1.20) (pull_request) Successful in 38s
Tests and linters / Lint (pull_request) Successful in 57s
Tests and linters / Tests with -race (pull_request) Successful in 1m5s
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-08-09 09:35:25 +03:00
280d052cef [#101] Remove usage of folder vendor
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-08-08 17:38:49 +03:00
35e7397d48 [#87] netmap: Extend enum Operation
All checks were successful
DCO action / DCO (pull_request) Successful in 39s
Tests and linters / Tests (1.19) (pull_request) Successful in 44s
Tests and linters / Tests (1.20) (pull_request) Successful in 51s
Tests and linters / Tests with -race (pull_request) Successful in 1m17s
Tests and linters / Lint (pull_request) Successful in 1m59s
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-08-07 16:21:50 +03:00
174773454e [#87] netmap: Regenerate to add LIKE operation for filter
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-08-06 17:20:29 +03:00
b72aa14bab [#87] proto: Process files with protoc version 27.2
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-08-06 17:20:29 +03:00
42e50c9633 [#87] Makefile: Add target protoc-install
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-08-06 17:20:21 +03:00
611355510c [#87] go.mod: Update google.golang.org/grpc to v1.63.2
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-08-06 16:57:57 +03:00