Commit graph

62 commits

Author SHA1 Message Date
29f2157563
[#123] protogen: Treat bytes field as non-nullable
In protobuf 3.12 they have added an support for `optional` keyword,
which has made it into the main branch in 3.15.
https://github.com/protocolbuffers/protobuf/blob/main/docs/implementing_proto3_presence.md
https://github.com/protocolbuffers/protobuf/blob/v3.12.0/docs/field_presence.md#presence-in-proto3-apis

This means that without an explicit `optional` keyword field presence
for scalars is not tracked, thus empty string in JSON should be
unmarshaled to a nil byte slice. Relevant decoding code and tests from
protojson:
fb995f184a/internal/impl/message_reflect_field.go (L327)
fb995f184a/encoding/protojson/decode_test.go (L134)
fb995f184a/encoding/protojson/decode_test.go (L156)

We do not support `optional` keyword and the generator will fail if it sees on.
So only implement the default behaviour.

Refs #122

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-10-11 14:40:54 +03:00
29c522d5d8 [#122] protogen: Always marshal empty fields
All checks were successful
DCO action / DCO (pull_request) Successful in 1m2s
Tests and linters / Lint (pull_request) Successful in 1m2s
Tests and linters / Tests (pull_request) Successful in 1m16s
Tests and linters / Tests with -race (pull_request) Successful in 1m16s
This is how it was done previously:
a0a9b765f3/rpc/message/encoding.go (L31)

The tricky part is `[]byte` which is marshaled as `null` by easyjson
helper, but as `""` by protojson.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-10-07 15:05:43 +03:00
b06dad731c [#120] protogen: Marshal 64-bit integers as strings
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-10-01 14:18:46 +03:00
d94b9c6d0d [#120] protogen: Unmarshal stringified integers from JSON
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-10-01 14:18:45 +03:00
eeb754c327 [#120] protogen: Omit empty fields from JSON output
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-10-01 14:11:43 +03:00
805da79319 [#120] protogen: Marshal enum as string
Be compatible with protojson.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-10-01 14:11:43 +03:00
f812b1ae5b [#120] Regenerate proto files
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-10-01 14:11:43 +03:00
0484647aae [#115] netmap: Fix type getters
* Add type instance check for nil to avoid panic by
  accessing fields.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-09-06 12:54:07 +00:00
bd588fa2e5 [#113] go.mod: Use range over int
All checks were successful
DCO action / DCO (pull_request) Successful in 41s
Tests and linters / Lint (pull_request) Successful in 1m11s
Tests and linters / Tests (1.22) (pull_request) Successful in 1m11s
Tests and linters / Tests (1.23) (pull_request) Successful in 1m10s
Tests and linters / Tests with -race (pull_request) Successful in 1m23s
Since Go 1.22 a `for` statement with a `range` clause is able
to iterate through integer values from zero to an upper limit.

gopatch script:
@@
var i, e expression
@@
-for i := 0; i <= e - 1; i++ {
+for i := range e {
    ...
}

@@
var i, e expression
@@
-for i := 0; i <= e; i++ {
+for i := range e + 1 {
    ...
}

@@
var i, e expression
@@
-for i := 0; i < e; i++ {
+for i := range e {
    ...
}

Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-09-04 15:44:59 +03:00
5e1c6a908f [#111] protogen: Emit slice of messages without a pointer
All checks were successful
DCO action / DCO (pull_request) Successful in 41s
Tests and linters / Tests (1.22) (pull_request) Successful in 55s
Tests and linters / Tests (1.23) (pull_request) Successful in 57s
Tests and linters / Tests with -race (pull_request) Successful in 1m8s
Tests and linters / Lint (pull_request) Successful in 2m12s
```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
                                              │      old      │                 new                  │
                                              │    sec/op     │    sec/op     vs base                │
ObjectIDSlice/0_elements/to_grpc_message-8       3.193n ±  2%   3.242n ±  0%   +1.50% (p=0.034 n=10)
ObjectIDSlice/0_elements/from_grpc_message-8     3.197n ±  2%   3.343n ±  1%   +4.57% (p=0.000 n=10)
ObjectIDSlice/0_elements/marshal-8               5.666n ±  3%   5.642n ±  0%   -0.42% (p=0.000 n=10)
ObjectIDSlice/1_elements/to_grpc_message-8       53.10n ±  6%   29.78n ± 12%  -43.92% (p=0.000 n=10)
ObjectIDSlice/1_elements/from_grpc_message-8     28.99n ±  5%   29.77n ±  7%        ~ (p=0.165 n=10)
ObjectIDSlice/1_elements/marshal-8               49.08n ±  7%   50.72n ±  6%        ~ (p=0.218 n=10)
ObjectIDSlice/50_elements/to_grpc_message-8     1652.5n ±  7%   277.2n ±  1%  -83.22% (p=0.000 n=10)
ObjectIDSlice/50_elements/from_grpc_message-8    261.2n ± 11%   226.7n ± 15%  -13.19% (p=0.003 n=10)
ObjectIDSlice/50_elements/marshal-8              1.512µ ±  6%   1.514µ ±  6%        ~ (p=0.955 n=10)
geomean                                          52.15n         39.99n        -23.31%

                                              │      old       │                  new                   │
                                              │      B/op      │     B/op      vs base                  │
ObjectIDSlice/0_elements/to_grpc_message-8        0.000 ± 0%       0.000 ± 0%        ~ (p=1.000 n=10) ¹
ObjectIDSlice/0_elements/from_grpc_message-8      0.000 ± 0%       0.000 ± 0%        ~ (p=1.000 n=10) ¹
ObjectIDSlice/0_elements/marshal-8                0.000 ± 0%       0.000 ± 0%        ~ (p=1.000 n=10) ¹
ObjectIDSlice/1_elements/to_grpc_message-8        32.00 ± 0%       24.00 ± 0%  -25.00% (p=0.000 n=10)
ObjectIDSlice/1_elements/from_grpc_message-8      24.00 ± 0%       24.00 ± 0%        ~ (p=1.000 n=10) ¹
ObjectIDSlice/1_elements/marshal-8                48.00 ± 0%       48.00 ± 0%        ~ (p=1.000 n=10) ¹
ObjectIDSlice/50_elements/to_grpc_message-8     1.578Ki ± 0%     1.250Ki ± 0%  -20.79% (p=0.000 n=10)
ObjectIDSlice/50_elements/from_grpc_message-8   1.250Ki ± 0%     1.250Ki ± 0%        ~ (p=1.000 n=10) ¹
ObjectIDSlice/50_elements/marshal-8             2.000Ki ± 0%     2.000Ki ± 0%        ~ (p=1.000 n=10) ¹
geomean                                                      ²                  -5.62%                ²
¹ all samples are equal
² summaries must be >0 to compute geomean

                                              │      old      │                 new                  │
                                              │   allocs/op   │ allocs/op   vs base                  │
ObjectIDSlice/0_elements/to_grpc_message-8       0.000 ± 0%     0.000 ± 0%        ~ (p=1.000 n=10) ¹
ObjectIDSlice/0_elements/from_grpc_message-8     0.000 ± 0%     0.000 ± 0%        ~ (p=1.000 n=10) ¹
ObjectIDSlice/0_elements/marshal-8               0.000 ± 0%     0.000 ± 0%        ~ (p=1.000 n=10) ¹
ObjectIDSlice/1_elements/to_grpc_message-8       2.000 ± 0%     1.000 ± 0%  -50.00% (p=0.000 n=10)
ObjectIDSlice/1_elements/from_grpc_message-8     1.000 ± 0%     1.000 ± 0%        ~ (p=1.000 n=10) ¹
ObjectIDSlice/1_elements/marshal-8               1.000 ± 0%     1.000 ± 0%        ~ (p=1.000 n=10) ¹
ObjectIDSlice/50_elements/to_grpc_message-8     51.000 ± 0%     1.000 ± 0%  -98.04% (p=0.000 n=10)
ObjectIDSlice/50_elements/from_grpc_message-8    1.000 ± 0%     1.000 ± 0%        ~ (p=1.000 n=10) ¹
ObjectIDSlice/50_elements/marshal-8              1.000 ± 0%     1.000 ± 0%        ~ (p=1.000 n=10) ¹
geomean                                                     ²               -40.18%                ²
¹ all samples are equal
² summaries must be >0 to compute geomean
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-08-28 11:53:08 +03:00
5fece80b42 [#108] protogen: Distinguish between empty and nil messages
All checks were successful
DCO action / DCO (pull_request) Successful in 39s
Tests and linters / Tests (1.22) (pull_request) Successful in 59s
Tests and linters / Tests (1.23) (pull_request) Successful in 57s
Tests and linters / Tests with -race (pull_request) Successful in 1m9s
Tests and linters / Lint (pull_request) Successful in 2m12s
Refs #59

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-08-27 11:02:18 +03:00
adb7c602d7 [#77] protogen: Initial implementation
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-08-19 10:47:00 +03:00
35e7397d48 [#87] netmap: Extend enum Operation
All checks were successful
DCO action / DCO (pull_request) Successful in 39s
Tests and linters / Tests (1.19) (pull_request) Successful in 44s
Tests and linters / Tests (1.20) (pull_request) Successful in 51s
Tests and linters / Tests with -race (pull_request) Successful in 1m17s
Tests and linters / Lint (pull_request) Successful in 1m59s
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-08-07 16:21:50 +03:00
174773454e [#87] netmap: Regenerate to add LIKE operation for filter
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-08-06 17:20:29 +03:00
3dfa2f4fd6 [#95] *: Regenerate proto files
All checks were successful
DCO action / DCO (pull_request) Successful in 1m0s
Tests and linters / Tests (1.19) (pull_request) Successful in 1m22s
Tests and linters / Tests (1.20) (pull_request) Successful in 1m25s
Tests and linters / Lint (pull_request) Successful in 1m32s
Tests and linters / Tests with -race (pull_request) Successful in 2m20s
Remove SetExtendedEACL and AnnounceUsedSpace methods from the container
package.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-26 10:24:25 +03:00
491a47e7fe [#70] netmap: Regenerate protobuf for Replica
All checks were successful
DCO action / DCO (pull_request) Successful in 1m3s
Tests and linters / Tests (1.19) (pull_request) Successful in 1m32s
Tests and linters / Tests (1.20) (pull_request) Successful in 1m28s
Tests and linters / Lint (pull_request) Successful in 1m38s
Tests and linters / Tests with -race (pull_request) Successful in 2m0s
* Since EC field tags are renumerated, it is required
  to regenerate netmap grpc.
* Add a generator with EC replica to check it with
  unit-test.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-03-27 12:56:03 +03:00
1772b92182 [#69] object: Initial EC implementation
All checks were successful
DCO action / DCO (pull_request) Successful in 1m7s
Tests and linters / Tests (1.19) (pull_request) Successful in 1m37s
Tests and linters / Tests (1.20) (pull_request) Successful in 1m33s
Tests and linters / Lint (pull_request) Successful in 1m46s
Tests and linters / Tests with -race (pull_request) Successful in 1m45s
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-03-19 15:23:01 +03:00
17bed735a1 [#20] Update api version
All checks were successful
DCO action / DCO (pull_request) Successful in 1m14s
Tests and linters / Tests (1.19) (pull_request) Successful in 1m39s
Tests and linters / Tests (1.20) (pull_request) Successful in 1m42s
Tests and linters / Lint (pull_request) Successful in 1m57s
Tests and linters / Tests with -race (pull_request) Successful in 2m5s
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-17 11:10:12 +03:00
964c3edb3f [#51] *: Do not panic in StableSize()
All checks were successful
Tests and linters / Lint (pull_request) Successful in 2m0s
Tests and linters / Tests (1.19) (pull_request) Successful in 3m27s
Tests and linters / Tests with -race (pull_request) Successful in 1m44s
Tests and linters / Tests (1.20) (pull_request) Successful in 3m29s
After #49 it can be called on nil structures.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-02 10:55:10 +03:00
582d94c81c [#47] types: Fix unique flag conversion
All checks were successful
Tests and linters / Tests (1.19) (pull_request) Successful in 1m26s
Tests and linters / Tests (1.20) (pull_request) Successful in 2m6s
Tests and linters / Tests with -race (pull_request) Successful in 2m23s
Tests and linters / Lint (pull_request) Successful in 9m12s
Signed-off-by: Airat Arifullin a.arifullin@yadro.com
2023-07-19 13:03:35 +03:00
3add88d435 [#44] netmap: Fix ineffassign linter warning
All checks were successful
Tests and linters / Tests (1.20) (pull_request) Successful in 50s
Tests and linters / Lint (pull_request) Successful in 3m1s
Tests and linters / Tests (1.19) (pull_request) Successful in 4m58s
Tests and linters / Tests with -race (pull_request) Successful in 5m56s
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-29 11:26:49 +03:00
e9d67aa1b2 [#31] netmap: Add unique field marshaling for PlacementPolicy
Signed-off-by: Airat Arifullin a.arifullin@yadro.com
2023-06-14 13:02:50 +00:00
8266b31092 [#26] netmap: Add NOT and UNIQUE keywords
* Regenerate types.pb.go for netmap grpc
* Add unique flag to PlacementPolicy struct

Signed-off-by: Airat Arifullin a.arifullin@yadro.com
2023-05-31 12:06:25 +03:00
e297616a48 [#27] *: Resolve unused linter warnings
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-03 14:15:56 +03:00
ff6d8db741 [#19] subnet: Drop related types and fields
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-18 13:29:50 +03:00
Pavel Karpy
d9347a05f0 [#21] *: Drop reputation system
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2023-04-17 15:46:52 +03:00
9dc3753467 [#13] *: Rename __FROSTFS__ prefix to __SYSTEM__
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-03-15 12:52:36 +03:00
8009022a20 [#13] *: Regenerate api
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-03-15 12:52:28 +03:00
cd2e46a17c [#10] Add __FROSTFS__ system attributes
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-03-13 09:59:22 +03:00
c46cd37f71 [#10] Generate api
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-03-13 09:59:22 +03:00
f69d2ad83c Rename package name
Due to source code relocation from GitHub.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-03-07 13:42:36 +03:00
1351b6656d Move to frostfs-api
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2022-12-12 17:40:48 +03:00
Leonard Lyubich
5fc2644c68 [#416] session: Support new format of object session context
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-10-04 18:29:57 +04:00
Leonard Lyubich
cf868188ef [#418] netmap: Support NetmapService.NetmapSnapshot RPC
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-09-19 16:44:34 +04:00
Evgenii Stratonikov
ea7a3811bf [#415] *: Add maintenance status and node state
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-09-16 18:50:53 +04:00
Evgenii Stratonikov
1853349f22 [#415] *: Go fmt -s
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-09-16 18:50:53 +04:00
Leonard Lyubich
cdce7edc39 Release v2.13.0 - Yeonpyeongdo (연평도, 延坪島)
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-07-04 17:49:02 +03:00
Pavel Karpy
f9a91e5f33 [#404] *: Regenerate code after language fixes
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-06-21 14:08:31 +03:00
Evgenii Stratonikov
94f068e462 *: remove error from StableMarshal return values
We marshal only in-memory structures, no error is expected.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-04-21 15:07:57 +03:00
Evgenii Stratonikov
84d981e88e util/proto: do not return error from primitive marshalers
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-04-21 15:07:57 +03:00
Evgenii Stratonikov
732dd51b1b [#388] *: Remove nil check from setters
I knew one day `sed` would save me an hour of manual work:
```
sed -i -n -e '
s/) Set/) Set/
p
t setter
b end
:setter
    n
    s/nil/nil/
    t hasif
    p
    b end
    :hasif
        n
        :loop
        p
        n
        s/}/}/
        t end
        b loop
        :end
' $@
goimports -w $@
```

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-03-31 11:32:04 +03:00
Evgenii Stratonikov
6f4908edc2 [#376] netmap: Make attributes a non-pointer slice
The speed of copying (which is done regulary for e.g. subnet changes)
is less, however it isn't on the hot path and the absolute time
difference is insignificant.
```
name              old time/op    new time/op    delta
NodeAttributes-8    96.2ns ± 1%   158.3ns ± 1%  +64.61%  (p=0.000 n=10+10)

name              old alloc/op   new alloc/op   delta
NodeAttributes-8     32.0B ± 0%     32.0B ± 0%     ~     (all equal)

name              old allocs/op  new allocs/op  delta
NodeAttributes-8      2.00 ± 0%      2.00 ± 0%     ~     (all equal)
```

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-03-15 12:19:49 +03:00
Evgenii Stratonikov
2f0eee96fc [#376] netmap: Replace slices of pointers with structs
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-03-15 12:19:49 +03:00
Leonard Lyubich
99370889d1 [#371] Support recent changes in NeoFS API protocol
Support:
  * new status codes (object, container, session);
  * object `Lock` message;
  * different signature schemes.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-02-25 10:24:07 +03:00
Leonard Lyubich
e9a8451fb7 [#367] *: Re-compile proto files of NeoFS API protocol
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-01-14 13:17:21 +03:00
Pavel Karpy
3604d96f3f [#362] netmap: Add marshaling of the subnetID field of container policy
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-12-01 16:45:23 +03:00
Leonard Lyubich
d93828f465 [#356] netmap: Fix post-loop handling of zero subnet in IterateSubnets
Return any error other than `ErrRemoveSubnet` from `IterateSubnets`
during post-loop processing of zero subnet.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-11-24 17:13:18 +03:00
Leonard Lyubich
50ab22c19c [#356] netmap: Improve docs and slightly optimize the code
API changes:
  * Rename `Enters` method of the `NodeSubnetInfo` to `Enabled`.
  * Declare undefined behavior of `WriteSubnetInfo` with non-unique
  attributes.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-11-24 17:13:18 +03:00
Leonard Lyubich
abc494b2bc [#356] netmap: Simplify zero subnet processing in IterateSubnets
For now we don't need to distinguish cases of missing zero subnet's
attribute and the one with `True` value.

Remove local enum `zeroStatus` of `IterateSubnets` since we need to
distinguish between two statuses and use `bool` variable for this.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-11-24 17:13:18 +03:00
Leonard Lyubich
896cee4bb0 [#356] netmap: Increase test coverage of IterateSubnets
Add both `False` and `True` subnet attributes.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-11-24 17:13:18 +03:00