90 commits

Author	SHA1	Message	Date
Evgenii Stratonikov	f0fc40e116	[#123] Resolve funlen linter issue ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-10-11 14:40:54 +03:00
Evgenii Stratonikov	29f2157563	[#123] protogen: Treat bytes field as non-nullable ... In protobuf 3.12 they have added an support for `optional` keyword, which has made it into the main branch in 3.15. https://github.com/protocolbuffers/protobuf/blob/main/docs/implementing_proto3_presence.md https://github.com/protocolbuffers/protobuf/blob/v3.12.0/docs/field_presence.md#presence-in-proto3-apis This means that without an explicit `optional` keyword field presence for scalars is not tracked, thus empty string in JSON should be unmarshaled to a nil byte slice. Relevant decoding code and tests from protojson: fb995f184a/internal/impl/message_reflect_field.go (L327) fb995f184a/encoding/protojson/decode_test.go (L134) fb995f184a/encoding/protojson/decode_test.go (L156) We do not support `optional` keyword and the generator will fail if it sees on. So only implement the default behaviour. Refs #122 Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-10-11 14:40:54 +03:00
Evgenii Stratonikov	29c522d5d8	[#122] protogen: Always marshal empty fields ... This is how it was done previously: a0a9b765f3/rpc/message/encoding.go (L31) The tricky part is `[]byte` which is marshaled as `null` by easyjson helper, but as `""` by protojson. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-10-07 15:05:43 +03:00
Evgenii Stratonikov	d9a604fbc1	[#120] proto/test: Unskip protojson compatibility test ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-10-01 14:18:52 +03:00
Evgenii Stratonikov	b06dad731c	[#120] protogen: Marshal 64-bit integers as strings ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-10-01 14:18:46 +03:00
Evgenii Stratonikov	d94b9c6d0d	[#120] protogen: Unmarshal stringified integers from JSON ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-10-01 14:18:45 +03:00
Evgenii Stratonikov	eeb754c327	[#120] protogen: Omit empty fields from JSON output ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-10-01 14:11:43 +03:00
Evgenii Stratonikov	805da79319	[#120] protogen: Marshal enum as string ... Be compatible with protojson. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-10-01 14:11:43 +03:00
Evgenii Stratonikov	287e98ad67	[#120] proto/test: Add protojson compatibility test ... It is failing, thus is skipped. But implement it now to make it easier to see it failing. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-10-01 14:11:43 +03:00
Evgenii Stratonikov	5e1c6a908f	[#111] protogen: Emit slice of messages without a pointer ... ``` goos: linux goarch: amd64 pkg: git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz │ old │ new │ │ sec/op │ sec/op vs base │ ObjectIDSlice/0_elements/to_grpc_message-8 3.193n ± 2% 3.242n ± 0% +1.50% (p=0.034 n=10) ObjectIDSlice/0_elements/from_grpc_message-8 3.197n ± 2% 3.343n ± 1% +4.57% (p=0.000 n=10) ObjectIDSlice/0_elements/marshal-8 5.666n ± 3% 5.642n ± 0% -0.42% (p=0.000 n=10) ObjectIDSlice/1_elements/to_grpc_message-8 53.10n ± 6% 29.78n ± 12% -43.92% (p=0.000 n=10) ObjectIDSlice/1_elements/from_grpc_message-8 28.99n ± 5% 29.77n ± 7% ~ (p=0.165 n=10) ObjectIDSlice/1_elements/marshal-8 49.08n ± 7% 50.72n ± 6% ~ (p=0.218 n=10) ObjectIDSlice/50_elements/to_grpc_message-8 1652.5n ± 7% 277.2n ± 1% -83.22% (p=0.000 n=10) ObjectIDSlice/50_elements/from_grpc_message-8 261.2n ± 11% 226.7n ± 15% -13.19% (p=0.003 n=10) ObjectIDSlice/50_elements/marshal-8 1.512µ ± 6% 1.514µ ± 6% ~ (p=0.955 n=10) geomean 52.15n 39.99n -23.31% │ old │ new │ │ B/op │ B/op vs base │ ObjectIDSlice/0_elements/to_grpc_message-8 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹ ObjectIDSlice/0_elements/from_grpc_message-8 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹ ObjectIDSlice/0_elements/marshal-8 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹ ObjectIDSlice/1_elements/to_grpc_message-8 32.00 ± 0% 24.00 ± 0% -25.00% (p=0.000 n=10) ObjectIDSlice/1_elements/from_grpc_message-8 24.00 ± 0% 24.00 ± 0% ~ (p=1.000 n=10) ¹ ObjectIDSlice/1_elements/marshal-8 48.00 ± 0% 48.00 ± 0% ~ (p=1.000 n=10) ¹ ObjectIDSlice/50_elements/to_grpc_message-8 1.578Ki ± 0% 1.250Ki ± 0% -20.79% (p=0.000 n=10) ObjectIDSlice/50_elements/from_grpc_message-8 1.250Ki ± 0% 1.250Ki ± 0% ~ (p=1.000 n=10) ¹ ObjectIDSlice/50_elements/marshal-8 2.000Ki ± 0% 2.000Ki ± 0% ~ (p=1.000 n=10) ¹ geomean ² -5.62% ² ¹ all samples are equal ² summaries must be >0 to compute geomean │ old │ new │ │ allocs/op │ allocs/op vs base │ ObjectIDSlice/0_elements/to_grpc_message-8 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹ ObjectIDSlice/0_elements/from_grpc_message-8 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹ ObjectIDSlice/0_elements/marshal-8 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹ ObjectIDSlice/1_elements/to_grpc_message-8 2.000 ± 0% 1.000 ± 0% -50.00% (p=0.000 n=10) ObjectIDSlice/1_elements/from_grpc_message-8 1.000 ± 0% 1.000 ± 0% ~ (p=1.000 n=10) ¹ ObjectIDSlice/1_elements/marshal-8 1.000 ± 0% 1.000 ± 0% ~ (p=1.000 n=10) ¹ ObjectIDSlice/50_elements/to_grpc_message-8 51.000 ± 0% 1.000 ± 0% -98.04% (p=0.000 n=10) ObjectIDSlice/50_elements/from_grpc_message-8 1.000 ± 0% 1.000 ± 0% ~ (p=1.000 n=10) ¹ ObjectIDSlice/50_elements/marshal-8 1.000 ± 0% 1.000 ± 0% ~ (p=1.000 n=10) ¹ geomean ² -40.18% ² ¹ all samples are equal ² summaries must be >0 to compute geomean ``` Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-08-28 11:53:08 +03:00
Evgenii Stratonikov	a2025376fc	[#111] proto/test: Add repeated message test ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-08-28 11:53:08 +03:00
Evgenii Stratonikov	5fece80b42	[#108] protogen: Distinguish between empty and nil messages ... Refs #59 Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-08-27 11:02:18 +03:00
Evgenii Stratonikov	981dc785f3	[#107] proto/test: Add oneof test ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-08-26 14:36:19 +03:00
Evgenii Stratonikov	866db105ed	[#107] protogen: Unify oneof getters with default protoc plugin ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-08-26 14:36:19 +03:00
Evgenii Stratonikov	937a53683a	[#107] protogen: Fix oneof JSON marshaling ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-08-26 14:36:19 +03:00
Evgenii Stratonikov	4a00ef946f	[#107] protogen: Fix oneof getter scalar default value ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-08-26 14:36:19 +03:00
Evgenii Stratonikov	61f6f0f4a2	[#107] proto/test: Regenerate *.pb.go ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-08-26 14:36:19 +03:00
Evgenii Stratonikov	c1b5f46f98	[#107] protogen: Remove unused parameter ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-08-26 14:36:19 +03:00
Evgenii Stratonikov	7be31eb847	[#77] protogen: Add tests for JSON format ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-08-19 10:47:00 +03:00
Evgenii Stratonikov	adb7c602d7	[#77] protogen: Initial implementation ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-08-19 10:47:00 +03:00
Evgenii Stratonikov	a28ceb251a	[#77] util/proto: Optimize int32 marshaling ... This is the approach used in easyproto 52d3ac4744/writer.go (L203) It allows to occupy slightly less space for negative numbers. The format is still protobuf, although, technically, this is a breaking change for our stable marshaling format. However, we don't use int32 at all and all enums have positive values, so nothing is broken. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-08-16 17:11:13 +03:00
Evgenii Stratonikov	47a48969b0	[#103] proto: Test end-to-end scenario ... Test the generated code, do not write yet another marshaling routine in tests. Before: ``` ok git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/util/proto 0.003s coverage: 55.6% of statements ``` After: ``` ok git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/util/proto 0.003s coverage: 80.0% of statements ``` Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-08-09 11:18:17 +03:00
Evgenii Stratonikov	19247e8941	[#103] protogen: Handle uint32 type ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-08-09 11:18:17 +03:00
Evgenii Stratonikov	ff4f31b6f3	[#103] protogen: Handle files in all packages ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-08-09 11:18:17 +03:00
Anton Nikiforov	b72aa14bab	[#87] proto: Process files with protoc version 27.2 ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-08-06 17:20:29 +03:00
Airat Arifullin	9b90d139c5	[#94] object: Generate protobufs for Patch method ... * Generate protobufs for patch method; * Create marshalers, unmarshalers, converters for gererated types; * Add unit-tests. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-07-29 08:55:14 +00:00
Evgenii Stratonikov	f517e39491	[#91] protogen: Support unpacked repeated uint64 fields ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-07-16 14:39:20 +03:00
Evgenii Stratonikov	610c450a65	[#90] proto/test: Fix go vet warnings ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-07-15 16:40:21 +03:00
Evgenii Stratonikov	3f92d7bfb0	[#90] proto/test: Fix proto file formatting ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-07-15 16:40:21 +03:00
Evgenii Stratonikov	1772b92182	[#69] object: Initial EC implementation ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-03-19 15:23:01 +03:00
Evgenii Stratonikov	a85146250b	[#68] Makefile: Use gofumpt for formatting ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-03-14 10:31:58 +03:00
Dmitrii Stepanov	634e24aba7	[#65] object: Fix SetMarshalData for PutSingle request ... Allow to reset marshal data Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-02-15 15:44:01 +03:00
Alexander Chuprov	72885aae83	[#62] signature: Refactor BufferPool ... Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>	2024-01-12 18:09:28 +03:00
Evgenii Stratonikov	b46e8cfbda	[#59] util: Rename stableMarshaler ... It should be with a single `l`, see `json.Marshaler`. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-11-21 15:28:03 +03:00
Evgenii Stratonikov	9cc2095446	[#59] util: Restore backwards compatibility in NestedStructureMarshal() ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-11-21 15:27:57 +03:00
Dmitrii Stepanov	f50872f1bc	[#58] object: Allow to set marshal data ... Now it is possible set marshaled data to reduce memory allocations. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-10-31 13:47:48 +03:00
Evgenii Stratonikov	17bed735a1	[#20] Update api version ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-08-17 11:10:12 +03:00
Airat Arifullin	9a7b47d769	[#50] util: Use protowire for util/proto helpers ... * Use methods to calculate size Signed-off-by: Airat Arifullin a.arifullin@yadro.com	2023-08-10 16:40:53 +00:00
Evgenii Stratonikov	3072090c77	[#51] protogen: Do not panic in StableSize() ... Refs TrueCloudLab/frostfs-node#554 . Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-08-02 10:47:14 +03:00
Evgenii Stratonikov	7a5ee927c8	[#49] util/proto: Do not allocate in StringSize() ... It was not catched by the test because most of the time the function is inlined. However, I've seen it allocating with pprof in one of the earlier builds. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-07-26 18:52:59 +03:00
Evgenii Stratonikov	6e92d7d5de	[#49] util/proto: Make NestedStructure* generic ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-07-26 18:31:23 +03:00
Evgenii Stratonikov	849de02bc3	[#49] util/proto: Calculate repeated field size without allocations ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-07-26 18:31:23 +03:00
Evgenii Stratonikov	59c8421597	[#49] util/proto: Use StableSize() to determine if the struct is empty ... `reflect` is not necessary here, and checking `StableSize` is what we _want_ anyway. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-07-26 18:31:00 +03:00
Dmitrii Stepanov	68021b910a	[#38] signature: Increase pool max object size ... According to the results of profiling, objects with a size of 72KB are mainly allocated. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-06-02 17:27:16 +03:00
Evgenii Stratonikov	e9e7901f67	[#27] util/proto: Fix staticcheck warning ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-05-03 14:15:56 +03:00
Evgenii Stratonikov	2aa3ee46e7	[#27] util/signature: Fix staticcheck warning ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-05-03 14:15:56 +03:00
Pavel Karpy	d9347a05f0	[#21] *: Drop reputation system ... Signed-off-by: Pavel Karpy <p.karpy@yadro.com>	2023-04-17 15:46:52 +03:00
Dmitrii Stepanov	ec0d0274fa	[#3] signature: Add buffer pool ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-03-10 13:36:26 +03:00
Dmitrii Stepanov	73fde0e37c	[#3] signature: Verify parts in parallel ... Verify request/response parts in parallel Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-03-10 13:36:26 +03:00
Dmitrii Stepanov	d005bf0393	[#3] signature: Refactor sign and verify ... Split methods to separate files, drop redundant intefaces Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-03-10 13:36:26 +03:00