Commit Graph

1277 Commits (master)

Author SHA1 Message Date
Airat Arifullin 0803bc6ded [#83] object: Regenerate protobufs for `ECHeader`
Tests and linters / Tests (1.19) (pull_request) Successful in 11m59s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 12m5s Details
DCO action / DCO (pull_request) Successful in 12m27s Details
Tests and linters / Tests with -race (pull_request) Successful in 12m25s Details
Tests and linters / Lint (pull_request) Successful in 12m52s Details
* Fix marshalers and converters;
* Fix unit-tests.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-16 16:31:03 +03:00
Airat Arifullin 4fe42ac4ad [#82] object: Add `FilterHeaderECParent` filter for v2
DCO action / DCO (pull_request) Successful in 1m0s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 1m20s Details
Tests and linters / Tests (1.19) (pull_request) Successful in 1m30s Details
Tests and linters / Lint (pull_request) Successful in 1m53s Details
Tests and linters / Tests with -race (pull_request) Successful in 1m49s Details
* `FilterHeaderECParent` is used by `Object.Search`

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-13 17:03:55 +03:00
Airat Arifullin 063ce11c24 [#82] object: Erase field `Signature` from `ECHeader` v2 type
DCO action / DCO (pull_request) Successful in 1m17s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 1m45s Details
Tests and linters / Tests (1.19) (pull_request) Successful in 1m58s Details
Tests and linters / Lint (pull_request) Successful in 2m6s Details
Tests and linters / Tests with -race (pull_request) Successful in 2m9s Details
* The field `Signature` is not used and should be factored out.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-08 15:09:16 +03:00
Airat Arifullin bc7b49eed2 [#82] object: Introduce new fields for EC header
* Regenerate protobufs as frostfs-api introduced `parent_split_id` and
  `parent_split_index` fields.
* Fix marshaller and converter for EC header.
* Extend v2 type `ECHeader` with `ParentSplitID` and `ParentSplitIndex` fields.
* Fix message_test for `Object`. Also generate EC headers and check it.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-08 15:09:12 +03:00
Airat Arifullin b171364079 [#81] rpc: Fix apemanager rpc service name
DCO action / DCO (pull_request) Successful in 1m4s Details
Tests and linters / Tests (1.19) (pull_request) Successful in 1m10s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 1m27s Details
Tests and linters / Lint (pull_request) Successful in 1m44s Details
Tests and linters / Tests with -race (pull_request) Successful in 1m48s Details
* Introduce `frostfsServiceNamePrefix` for rpc interface
  since `apemanager` uses `frostfs` root instead `neo.fs`.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-06 14:46:54 +03:00
Evgenii Stratonikov 0fe6b9adbb [#76] Make setter names more consistent
DCO action / DCO (pull_request) Successful in 1m2s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 1m33s Details
Tests and linters / Tests (1.19) (pull_request) Successful in 1m38s Details
Tests and linters / Lint (pull_request) Successful in 1m46s Details
Tests and linters / Tests with -race (pull_request) Successful in 1m44s Details
This will make the transition to a new codegen more seamless.
1. Setter should be named as the field itself.
2. Setter should accept 1 parameter.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-05-03 08:52:04 +03:00
Evgenii Stratonikov fa8f92d662 [#75] audit: Remove deprecated service
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-05-02 11:54:35 +00:00
Airat Arifullin 387b850e5e [#74] apemanager: Generate protobufs for apemanager service
* Generate protobufs.
* Make marshallers, unmarshallers, json-encoders etc.
* Create message encoding/decoding unit-tests.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-02 11:53:26 +00:00
Anton Nikiforov 67c6f305b2 [#79] object: Implement `Range\RangeHash` requests for EC object
DCO action / DCO (pull_request) Successful in 1m16s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 1m37s Details
Tests and linters / Lint (pull_request) Successful in 1m56s Details
Tests and linters / Tests (1.19) (pull_request) Successful in 2m9s Details
Tests and linters / Tests with -race (pull_request) Successful in 2m2s Details
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-04-27 23:04:46 +03:00
Anton Nikiforov df9b65324a [#73] object: Implement `Get\Head` requests for EC object
DCO action / DCO (pull_request) Successful in 45s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 1m13s Details
Tests and linters / Tests (1.19) (pull_request) Successful in 1m27s Details
Tests and linters / Lint (pull_request) Successful in 1m35s Details
Tests and linters / Tests with -race (pull_request) Successful in 1m58s Details
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-04-22 18:14:50 +03:00
Evgenii Stratonikov 4a330a5706 [#72] session: Unify unmarshaling code
DCO action / DCO (pull_request) Successful in 1m1s Details
Tests and linters / Tests (1.19) (pull_request) Successful in 1m24s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 1m32s Details
Tests and linters / Lint (pull_request) Successful in 1m43s Details
Tests and linters / Tests with -race (pull_request) Successful in 2m1s Details
gopatch:
```
@@
var err, msg, data, t, protojson identifier
var obj expression
@@
+import "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/message"
-import protojson "google.golang.org/protobuf/encoding/protojson"

-msg := obj
-if err := protojson.Unmarshal(data, msg); err != nil {
-return err
-}
-return t.FromGRPCMessage(msg)
+return message.UnmarshalJSON(t, data, obj)

@@
var err, msg, data, t, goproto identifier
var obj expression
@@
+import "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/message"
-import goproto "google.golang.org/protobuf/proto"

-msg := obj
-if err := goproto.Unmarshal(data, msg); err != nil {
-return err
-}
-return t.FromGRPCMessage(msg)
+return message.Unmarshal(t, data, obj)
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-20 14:19:19 +03:00
Airat Arifullin 491a47e7fe [#70] netmap: Regenerate protobuf for Replica
DCO action / DCO (pull_request) Successful in 1m3s Details
Tests and linters / Tests (1.19) (pull_request) Successful in 1m32s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 1m28s Details
Tests and linters / Lint (pull_request) Successful in 1m38s Details
Tests and linters / Tests with -race (pull_request) Successful in 2m0s Details
* Since EC field tags are renumerated, it is required
  to regenerate netmap grpc.
* Add a generator with EC replica to check it with
  unit-test.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-03-27 12:56:03 +03:00
Airat Arifullin 6e9d385f3c [#71] container: Introduce CONTAINER_ACCESS_DENIED status
DCO action / DCO (pull_request) Successful in 1m19s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 1m56s Details
Tests and linters / Tests (1.19) (pull_request) Successful in 2m2s Details
Tests and linters / Lint (pull_request) Successful in 2m12s Details
Tests and linters / Tests with -race (pull_request) Successful in 2m28s Details
* Regenerate protobufs for object and container service to
  use a new frostfs-api status CONTAINER_ACCESS_DENIED.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-03-26 18:55:05 +03:00
Evgenii Stratonikov 1772b92182 [#69] object: Initial EC implementation
DCO action / DCO (pull_request) Successful in 1m7s Details
Tests and linters / Tests (1.19) (pull_request) Successful in 1m37s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 1m33s Details
Tests and linters / Lint (pull_request) Successful in 1m46s Details
Tests and linters / Tests with -race (pull_request) Successful in 1m45s Details
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-03-19 15:23:01 +03:00
Evgenii Stratonikov a85146250b [#68] Makefile: Use gofumpt for formatting
DCO action / DCO (pull_request) Successful in 1m17s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 2m22s Details
Tests and linters / Lint (pull_request) Successful in 2m51s Details
Tests and linters / Tests (1.19) (pull_request) Successful in 2m48s Details
Tests and linters / Tests with -race (pull_request) Successful in 2m40s Details
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-03-14 10:31:58 +03:00
Evgenii Stratonikov ea6e390c7b [#68] pre-commit: Add unit test hook
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-03-14 10:31:58 +03:00
Evgenii Stratonikov 4d35da7c77 [#68] Makefile: Allow to override testflags
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-03-14 10:31:58 +03:00
Evgenii Stratonikov a7bcad6aa9 [#68] Makefile: Drop GO111MODULE usages
It is ignored since go1.17.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-03-14 10:31:58 +03:00
Evgenii Stratonikov 6fb9cae479 [#68] pre-commit: Update golangci-lint to 1.56.2
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-03-14 10:31:58 +03:00
Evgenii Stratonikov f69ad7ade0 [#68] pre-commit: Remove gitlint
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-03-14 10:31:58 +03:00
Dmitrii Stepanov c1c7b344b9 [#67] go.mod: Bump protobuf version
DCO action / DCO (pull_request) Successful in 6m8s Details
Tests and linters / Tests (1.19) (pull_request) Successful in 6m16s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 6m27s Details
Tests and linters / Tests with -race (pull_request) Successful in 6m40s Details
Tests and linters / Lint (pull_request) Successful in 7m0s Details
Found by vulncheck:
Vulnerability #1: GO-2024-2611
    Infinite loop in JSON unmarshaling in google.golang.org/protobuf
  More info: https://pkg.go.dev/vuln/GO-2024-2611
  Module: google.golang.org/protobuf
    Found in: google.golang.org/protobuf@v1.32.0
    Fixed in: google.golang.org/protobuf@v1.33.0

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-03-06 13:18:14 +03:00
Dmitrii Stepanov 634e24aba7 [#65] object: Fix SetMarshalData for PutSingle request
DCO action / DCO (pull_request) Successful in 1m11s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 1m25s Details
Tests and linters / Tests (1.19) (pull_request) Successful in 1m33s Details
Tests and linters / Tests with -race (pull_request) Successful in 1m56s Details
Tests and linters / Lint (pull_request) Successful in 1m14s Details
Allow to reset marshal data

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-15 15:44:01 +03:00
Dmitrii Stepanov 2a124b95bc [#64] object: Allow to reset marshal data
DCO action / DCO (pull_request) Successful in 58s Details
Tests and linters / Tests (1.19) (pull_request) Successful in 1m24s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 1m32s Details
Tests and linters / Lint (pull_request) Successful in 1m43s Details
Tests and linters / Tests with -race (pull_request) Successful in 2m5s Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-15 14:47:28 +03:00
Evgenii Stratonikov d60ce83e42 [#63] .forgejo: Update dco-go to v3
DCO action / DCO (pull_request) Successful in 5m9s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 7m17s Details
Tests and linters / Tests with -race (pull_request) Successful in 8m39s Details
Tests and linters / Tests (1.19) (pull_request) Successful in 10m2s Details
Tests and linters / Lint (pull_request) Successful in 12m37s Details
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-01-26 12:19:53 +03:00
Alexander Chuprov 72885aae83 [#62] signature: Refactor BufferPool
Tests and linters / Tests (1.19) (pull_request) Successful in 7m46s Details
DCO action / DCO (pull_request) Successful in 8m16s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 9m24s Details
Tests and linters / Tests with -race (pull_request) Successful in 9m25s Details
Tests and linters / Lint (pull_request) Successful in 9m49s Details
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2024-01-12 18:09:28 +03:00
Evgenii Stratonikov b46e8cfbda [#59] util: Rename stableMarshaler
DCO action / DCO (pull_request) Successful in 1m23s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 1m46s Details
Tests and linters / Lint (pull_request) Successful in 1m59s Details
Tests and linters / Tests (1.19) (pull_request) Successful in 2m7s Details
Tests and linters / Tests with -race (pull_request) Successful in 2m22s Details
It should be with a single `l`, see `json.Marshaler`.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-11-21 15:28:03 +03:00
Evgenii Stratonikov 9cc2095446 [#59] util: Restore backwards compatibility in NestedStructureMarshal()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-11-21 15:27:57 +03:00
Dmitrii Stepanov 498877e378 [#58] makefile: Disable test results caching
DCO action / DCO (pull_request) Successful in 59s Details
Tests and linters / Tests (1.19) (pull_request) Successful in 1m38s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 1m37s Details
Tests and linters / Tests with -race (pull_request) Successful in 1m52s Details
Tests and linters / Lint (pull_request) Successful in 2m13s Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-10-31 13:47:48 +03:00
Dmitrii Stepanov f50872f1bc [#58] object: Allow to set marshal data
Now it is possible set marshaled data to reduce memory
allocations.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-10-31 13:47:48 +03:00
Evgenii Stratonikov 309aa4ac78 Release v2.16.0
DCO action / DCO (pull_request) Successful in 52s Details
Tests and linters / Tests (1.19) (pull_request) Successful in 1m24s Details
Tests and linters / Tests with -race (pull_request) Successful in 1m28s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 6m25s Details
Tests and linters / Lint (pull_request) Successful in 7m11s Details
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-14 13:20:13 +03:00
Evgenii Stratonikov 17bed735a1 [#20] Update api version
DCO action / DCO (pull_request) Successful in 1m14s Details
Tests and linters / Tests (1.19) (pull_request) Successful in 1m39s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 1m42s Details
Tests and linters / Lint (pull_request) Successful in 1m57s Details
Tests and linters / Tests with -race (pull_request) Successful in 2m5s Details
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-17 11:10:12 +03:00
Evgenii Stratonikov 022f818735 [#20] Remove storage groups
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-17 11:10:08 +03:00
Evgenii Stratonikov d989c8d2a3 [#54] *: Fix linter warnings
DCO action / DCO (pull_request) Successful in 2m25s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 3m19s Details
Tests and linters / Tests with -race (pull_request) Successful in 4m42s Details
Tests and linters / Tests (1.19) (pull_request) Successful in 5m29s Details
Tests and linters / Lint (pull_request) Successful in 5m53s Details
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-11 15:21:27 +03:00
Evgenii Stratonikov 0671f42ee1 [#54] .forgejo: Add DCO action
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-11 15:21:27 +03:00
Airat Arifullin 9a7b47d769 [#50] util: Use protowire for util/proto helpers
* Use methods to calculate size

Signed-off-by: Airat Arifullin a.arifullin@yadro.com
2023-08-10 16:40:53 +00:00
Evgenii Stratonikov 67b18d3550 [#53] Bump min go version to go1.20
Tests and linters / Lint (pull_request) Failing after 4s Details
Tests and linters / Tests (1.19) (pull_request) Failing after 3s Details
Tests and linters / Tests (1.20) (pull_request) Failing after 2s Details
Tests and linters / Tests with -race (pull_request) Failing after 2s Details
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-09 09:55:51 +03:00
Evgenii Stratonikov 964c3edb3f [#51] *: Do not panic in StableSize()
Tests and linters / Lint (pull_request) Successful in 2m0s Details
Tests and linters / Tests (1.19) (pull_request) Successful in 3m27s Details
Tests and linters / Tests with -race (pull_request) Successful in 1m44s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 3m29s Details
After #49 it can be called on nil structures.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-02 10:55:10 +03:00
Evgenii Stratonikov 3072090c77 [#51] protogen: Do not panic in StableSize()
Tests and linters / Lint (pull_request) Successful in 1m56s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 1m7s Details
Tests and linters / Tests (1.19) (pull_request) Successful in 2m37s Details
Tests and linters / Tests with -race (pull_request) Successful in 1m44s Details
Refs TrueCloudLab/frostfs-node#554 .

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-02 10:47:14 +03:00
Evgenii Stratonikov 7a5ee927c8 [#49] util/proto: Do not allocate in StringSize()
Tests and linters / Tests (1.19) (pull_request) Successful in 1m1s Details
Tests and linters / Lint (pull_request) Successful in 2m10s Details
Tests and linters / Tests with -race (pull_request) Successful in 2m18s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 2m52s Details
It was not catched by the test because most of the time the function is
inlined. However, I've seen it allocating with pprof in one of the
earlier builds.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-07-26 18:52:59 +03:00
Evgenii Stratonikov 7133a01ccf [#49] message/test: Add test for zero-alloc StableSize()
Tests and linters / Tests (1.19) (pull_request) Successful in 54s Details
Tests and linters / Lint (pull_request) Successful in 1m44s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 2m28s Details
Tests and linters / Tests with -race (pull_request) Successful in 4m46s Details
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-07-26 18:32:17 +03:00
Evgenii Stratonikov 43ad0f114c [#49] session: Make StableSize() zero-alloc
For this to work, it is necessary that `NestedStructureSize` is a generic function.
Otherwise, we would allocate to put it in the interface.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-07-26 18:31:31 +03:00
Evgenii Stratonikov 6e92d7d5de [#49] util/proto: Make `NestedStructure*` generic
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-07-26 18:31:23 +03:00
Evgenii Stratonikov 849de02bc3 [#49] util/proto: Calculate repeated field size without allocations
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-07-26 18:31:23 +03:00
Evgenii Stratonikov 59c8421597 [#49] util/proto: Use StableSize() to determine if the struct is empty
`reflect` is not necessary here, and checking `StableSize` is what we
_want_ anyway.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-07-26 18:31:00 +03:00
Airat Arifullin 582d94c81c [#47] types: Fix unique flag conversion
Tests and linters / Tests (1.19) (pull_request) Successful in 1m26s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 2m6s Details
Tests and linters / Tests with -race (pull_request) Successful in 2m23s Details
Tests and linters / Lint (pull_request) Successful in 9m12s Details
Signed-off-by: Airat Arifullin a.arifullin@yadro.com
2023-07-19 13:03:35 +03:00
Dmitrii Stepanov 285516a94e [#45] api-go: Add PutSingle RPC call
Tests and linters / Tests (1.19) (pull_request) Successful in 52s Details
Tests and linters / Lint (pull_request) Successful in 1m56s Details
Tests and linters / Tests with -race (pull_request) Successful in 3m4s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 4m35s Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-04 12:27:42 +03:00
Dmitrii Stepanov d5b55d0926 [#45] api-go: Add PutSingle response wrappers
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-04 12:27:42 +03:00
Dmitrii Stepanov 2cb57a8835 [#45] api-go: Add PutSingle request wrappers
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-04 12:27:42 +03:00
Dmitrii Stepanov b17995a238 [#45] api-go: Add Object.PutSingle implementation
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-04 12:27:42 +03:00
Evgenii Stratonikov 3add88d435 [#44] netmap: Fix ineffassign linter warning
Tests and linters / Tests (1.20) (pull_request) Successful in 50s Details
Tests and linters / Lint (pull_request) Successful in 3m1s Details
Tests and linters / Tests (1.19) (pull_request) Successful in 4m58s Details
Tests and linters / Tests with -race (pull_request) Successful in 5m56s Details
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-29 11:26:49 +03:00