3222 commits

Author	SHA1	Message	Date
Ekaterina Lebedeva	5b100699d7	[#566] policer: Move isClientErrMaintenance to frostfs-sdk-go ... Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2024-06-18 10:20:45 +03:00
Dmitrii Stepanov	76cf7a051b	[#1178] shard: Check metabase existence before read shard ID ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-06-17 09:59:15 +03:00
Evgenii Stratonikov	96fe271bab	[#1170] innerring: Support morph mTLS ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-06-14 09:28:32 +03:00
Evgenii Stratonikov	42ecc2f2b9	[#1170] morph: Support mTLS ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-06-14 09:26:27 +03:00
Anton Nikiforov	68ac490729	[#1174] shard: Update metric mode_info on Init ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-06-13 08:32:59 +00:00
Dmitrii Stepanov	6a39c3d15e	[#1086] engine: Do not use metabase if shard looks bad ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-06-13 07:35:22 +00:00
Evgenii Stratonikov	9d73f9c2c6	Reapply "[#446] engine: Move to read-only on blobstor errors" ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-06-13 07:35:22 +00:00
Anton Nikiforov	b9fcaad21f	[#1168] shard: Set Disabled as default mode for components ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-06-11 15:13:38 +00:00
Dmitrii Stepanov	6cf512e574	[#1166] blobovniczatree: Handle blobovnicza's NoSpaceLeft error ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-06-07 17:15:43 +03:00
Dmitrii Stepanov	e7d479f4c2	[#1166] blobovnicza: Return NoSpaceLeft error instead of syscall.ENOSPC ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-06-07 17:15:43 +03:00
Airat Arifullin	239323eeef	[#1157] tree: Make tree service use Bearer token's APE overrides ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-06-07 12:11:11 +00:00
Airat Arifullin	04a3f891fd	[#1157] object: Make APE checker use Bearer-token's APE overrides ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-06-07 12:11:11 +00:00
Anton Nikiforov	4edff5eea6	[#1157] ape: Introduce single-run chain router ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-06-07 12:11:11 +00:00
Anton Nikiforov	a90310335d	[#1156] ape: Return not found when removing local overrides ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-06-07 12:10:57 +00:00
Evgenii Stratonikov	a849236b68	[#1161] node: Remove notification functionality ... It is unused and will be reworked in future. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-06-07 12:10:51 +00:00
Evgenii Stratonikov	3f1961157e	[#1163] metabase: Handle multiple splitInfos for EC ... For REP updating split info is handled explicitly by a high-level PUT logic. For EC it is trickier, because the address of an object we put is only distantly related to a split info. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-06-06 16:26:29 +03:00
Evgenii Stratonikov	2e074d3846	[#1163] metabase: Properly save EC parent split ID ... Search by SplitID should return all parts of a complex object. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-06-05 12:40:16 +03:00
Alexander Chuprov	806236da78	[#1121] node: Change mode of shard components ... Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>	2024-06-05 05:55:24 +00:00
Alexander Chuprov	6f2187a420	[#1121] node: Refactor mods of shard ... Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>	2024-06-05 05:55:24 +00:00
Dmitrii Stepanov	cc2449beaf	[#1158] metabase: Fix EC storage schema ... Do not store EC info twice. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-06-04 17:24:40 +03:00
Airat Arifullin	5aacb8fc86	[#1144] metabase: Save parent attributes for ec-chunks ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-05-31 19:55:32 +03:00
Airat Arifullin	f8e33f8e3a	[#1144] metabase: Proprely choose root OID for EC-splitted objects ... * If EC-parent is a part of Split itself, then save to root bucket its parent; * If EC-parent is not a part of Split itself, then save to root bucket OID of this EC-parent. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-05-31 19:53:32 +03:00
Airat Arifullin	f0edebea18	[#1144] metabase: Support ec parent filter for Search ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-05-31 19:53:32 +03:00
Airat Arifullin	0b367007fc	[#1152] go.mod: Update api-go and sdk versions ... * Resolve conflicts for apemanager since api-go contains ape and apemanager packages and SDK only ape package. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-05-31 15:39:09 +03:00
Anton Nikiforov	92e19feb57	[#1147] node: Use public fields for shard.ExistsPrm ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-05-30 08:13:04 +00:00
Anton Nikiforov	c1af13b47e	[#1147] node: Fix issue from gopls ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-05-30 08:13:04 +00:00
Anton Nikiforov	6130650bb6	[#1147] node: Implement Lock\Delete requests for EC object ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-05-30 08:13:04 +00:00
Anton Nikiforov	a82c8cc5b8	[#1147] gc: Execute callback for expired tombstones when they exists ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-05-30 08:13:04 +00:00
Anton Nikiforov	d355274cd0	[#1147] object: Use methods on pointer for searchsvc.execCtx ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-05-30 08:13:04 +00:00
Anton Nikiforov	3555c73225	[#1147] object: Use methods on pointer for deletesvc.execCtx ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-05-30 08:13:04 +00:00
Anton Nikiforov	e43e7bec3a	[#1147] log: Remove redundant address field from log ... Filled when logger created for `request` object from package `getsvc`. Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-05-30 08:13:04 +00:00
Anton Nikiforov	50923ed81c	[#1147] Fix gofumpt issue ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-05-30 08:13:04 +00:00
Alex Vanin	4a34d0d40e	[#1149] go.mod: Bump neo-go up to v0.106.0 ... Required to work with neo-go v0.106.0 node with default hardfork configuration. Without neo-go client version bump, it throws error. failed to get network magic: unexpected hardfork: Cockatrice Signed-off-by: Alex Vanin <a.vanin@yadro.com>	2024-05-30 08:11:58 +00:00
Airat Arifullin	3627b44e92	[#1142] tree: Fill APE-request with source IP property ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-05-27 10:17:17 +00:00
Airat Arifullin	482c5129ac	[#1142] object: Fill APE-request with source IP property ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-05-27 10:17:17 +00:00
Airat Arifullin	43625e7536	[#1142] container: Fill APE-request property with source IP ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-05-27 10:17:17 +00:00
Airat Arifullin	542d3adcb2	[#1105] apemanager: Implement apemanager service ... * Introduce grpc server for apemanager service and its implementation in `pkg/services/apemanager`. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-05-27 09:34:21 +00:00
Airat Arifullin	51ade979e8	[#1105] ape: Introduce contract storage with proxy contract verification ... * `ProxyVerificationContractStorage` uses Proxy contract as a cosigner. * `ProxyVerificationContractStorage` recreates a contract storage for each handler invocation because of an issue: rpc-actor from morph client may be expired. This way won't create a bottlenecks because it is expected that this contract storage implementation will be used not so often. * Make morph client return `RPCActor` (that is websocket client in fact). * Make `SwitchRPCGuardedActor` return `RPCActor` as it will be used for `ProxyVerificationContractStorage`. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-05-27 09:34:21 +00:00
Airat Arifullin	40b04c00ef	[#1141] metabase: Fix IsUserObject method ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-05-20 13:24:15 +03:00
Airat Arifullin	89a80e9a0f	[#1141] metabase: Fix putUniqueIndexItem ... * `GetECHeader` is not correct way to determine if an object's got EC-header: `ECHeader` must be used for that. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-05-20 13:24:08 +03:00
Airat Arifullin	8fd678e269	[#1141] go.mod: Update frostfs-sdk-go and frostfs-api-go/v2 ... * Also fix unit-test. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-05-20 13:22:48 +03:00
Dmitrii Stepanov	436c9f5558	[#1129] policer: Restore EC object ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-05-17 14:36:18 +03:00
Dmitrii Stepanov	44f2e8f27f	[#1129] putSvc: Allow to put single unprepared object to EC container ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-05-16 16:28:49 +03:00
Dmitrii Stepanov	0e42126ddc	[#1129] object: Fix check owner for EC part ... Do not validate EC part owner if request from container node. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-05-16 16:28:49 +03:00
Dmitrii Stepanov	1cd8562db8	[#1129] policer: Refactor shortage ... Drop override inside method. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-05-16 16:28:49 +03:00
Dmitrii Stepanov	4ab6c404f7	[#1129] policer: Drop unused ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-05-16 16:28:49 +03:00
Dmitrii Stepanov	cbe9757490	[#1129] policer: Pull required EC chunks ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-05-16 16:28:49 +03:00
Dmitrii Stepanov	d45d086acd	[#1129] policer: Add EC chunk replication ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-05-16 16:28:48 +03:00
Alexander Chuprov	b078fe5ba1	[#1092] control: Move SignMessage to separate package ... Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>	2024-05-16 12:14:01 +03:00
Evgenii Stratonikov	f3e09cb09b	[#1135] sdnotify: Send MONOTONIC_USEC on reload ... Fixes #1135 Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-05-15 12:52:52 +00:00
Anton Nikiforov	5c582e96fd	[#1136] metabase: Fix creation of ECInfoError ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-05-15 11:04:27 +00:00
Evgenii Stratonikov	b3eaa8a9bc	[#1083] objsvc/v2: Check response status in RANGE_HASH forwarder ... Fixes #1083 Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-05-15 12:04:21 +03:00
Evgenii Stratonikov	0924b62a95	[#1083] objsvc/v2: Unify response verification after forwarding ... 1. Use the same routine for HEAD/GET_RANGE methods. 2. Make error message similar. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-05-15 12:04:06 +03:00
Evgenii Stratonikov	300654b045	[#1083] objsvc/v2: Properly check response status after forwarding ... Previously we had cryptic error: ``` debug get/remote.go:38 remote call failed {"component": "Object.Get service", "request": "HEAD", "address": "9sTxoVrhJ7WBtXQfK2NJ7zDV5yCF7BPLKK1XTxYPdGsP/BbHV4KZZ8y2BPqAT5kyjdHRLkfbtY2xf5uYoMVqxACn1", "raw": false, "local": false, "with session": false, "with bearer": false, "error": "unexpected header type <nil>"} ``` Now we have and expected error: ``` debug get/remote.go:38 remote call failed {"component": "Object.Get service", "request": "HEAD", "address": "D2rqaMG4D2VHdv3HKky8UYSYmwQFH2v9oXXqtyRZPTMy/BbHV4KZZ8y2BPqAT5kyjdHRLkfbtY2xf5uYoMVqxACn1", "raw": false, "local": false, "with session": false, "with bearer": false, "error": "status: code = 2049 message = object not found"} ``` Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-05-15 12:04:06 +03:00
Evgenii Stratonikov	6e71ae3bda	[#1130] fstree: Remove useless Stat() call ... ``` goos: linux goarch: amd64 pkg: git.frostfs.info/TrueCloudLab/frostfs-node/pkg/local_object_storage/blobstor cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz │ old │ new │ │ sec/op │ sec/op vs base │ SubstorageReadPerf/fstree_nosync-seq100-8 2.689µ ± 2% 2.428µ ± 4% -9.72% (p=0.000 n=10) SubstorageReadPerf/fstree_nosync-rand100-8 2.727µ ± 1% 2.497µ ± 2% -8.42% (p=0.000 n=10) geomean 2.708µ 2.462µ -9.07% ``` Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-05-14 16:05:45 +03:00
Evgenii Stratonikov	bf9bdde8ea	[#1128] util/test: Remove unused package ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-05-14 12:48:13 +00:00
Airat Arifullin	952d13cd2b	[#1124] cli: Improve APE rule parsing ... * Make APE rule parser to read condition's kind in unambiguous using lexemes `ResourceCondition`, `RequestCondition` instead confusing `Object.Request`, `Object.Resource`. * Fix unit-tests. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-05-14 12:23:26 +03:00
aarifullin	20baf6e112	[#1108] ape: Update policy-engine version for listing by iteration ... * Update go.mod with a new version of policy-engine pacakge. * Adapt SwitchRPCGuardedActor to ContractStorage interface. * Fix `frostfs-adm` util. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-05-14 12:17:56 +03:00
Dmitrii Stepanov	0144117cc9	[#1125] objectSvc: Add EC header APE check ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-05-08 16:25:55 +03:00
Dmitrii Stepanov	ada1b9f737	[#1120] objectSvc: Fix EC put placement ... Use parent object ID to compute placement. Fix too many copies saving. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-05-08 15:23:57 +03:00
Anton Nikiforov	fe2c1c926f	[#1112] node: Fix race warning for GetObjectAndWritePayload ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-05-07 14:47:21 +03:00
Anton Nikiforov	3e782527b8	[#1112] node: Add test for Range request for EC object ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-05-07 14:47:21 +03:00
Anton Nikiforov	21a490da8f	[#1112] Fix issue from gofumpt ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-05-07 14:47:21 +03:00
Evgenii Stratonikov	93c0ccad4f	[#1077] objectsvc: Fix possible panic in GetRange() ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-05-07 14:47:21 +03:00
Anton Nikiforov	00b2b77b26	[#1112] node: Implement Range\RangeHash requests for EC object ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-05-07 14:47:21 +03:00
aarifullin	b60a51b862	[#1117] ape: Introduce FormFrostfsIDRequestProperties method ... * `FormFrostfsIDRequestProperties` gets user claim tags and group id and sets them as ape request properties. * Make tree, container and object service use the method. * Fix unit-tests. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-05-07 10:01:21 +00:00
aarifullin	6c76c9b457	[#1117] core: Introduce SubjectProvider interface for FrostfsID ... * Make tree, object and container services use SubjectProvider interface. * Fix unit-tests. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-05-07 10:01:21 +00:00
aarifullin	45f4e6939d	[#1117] morph: Make frostfsid client provide GetSubjectExtended method ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-05-07 10:01:21 +00:00
Ekaterina Lebedeva	e07869a8cf	[#1100] Remove unused fields ... Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2024-05-06 10:14:36 +03:00
Alexander Chuprov	ec2873caa7	[#1116] node: Fix writecache metrics ... Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>	2024-05-02 20:03:33 +03:00
Evgenii Stratonikov	71789676d5	[#1114] aclsvc: Add tests for request ownership ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-05-02 11:57:39 +03:00
Alexander Chuprov	c9efaa5819	[#966] node: Add path of the write_cache to metric labels ... Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>	2024-05-02 06:46:46 +00:00
Alexander Chuprov	4730ecfdb8	[#966] node: Refactor WriteCacheMetrics interface ... Grouping common fields of methods will enhance the readability of the interface. Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>	2024-05-02 06:46:46 +00:00
Anton Nikiforov	411a8d0245	[#1004] blobovnicza: Use TTL for blobovnicza tree cache ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-04-26 19:54:29 +03:00
Anton Nikiforov	112a7c690f	[#1103] node: Implement Get\Head requests for EC object ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-04-24 18:15:53 +03:00
Anton Nikiforov	167c52a1a9	[#1103] node: Reduce amount of lines for method StorageEngine.head ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-04-24 16:31:04 +03:00
Anton Nikiforov	700e891b85	[#1103] Fix end of file and trim trailing whitespace ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-04-24 16:31:04 +03:00
Airat Arifullin	10ee865e98	[#1096] tree: Make verifyClient fill ape request with user claim tags ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-04-16 15:12:46 +03:00
Airat Arifullin	c21d72ac23	[#1096] object: Make ape middleware fill request with user claim tags ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-04-16 15:12:44 +03:00
Airat Arifullin	6772976657	[#1096] container: Make ape middleware fill request with user claim tags ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-04-16 15:10:20 +03:00
Evgenii Stratonikov	3ea1d7b729	[#1089] control: Add USER and GROUP targets for local override storage ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-04-16 11:03:50 +00:00
Evgenii Stratonikov	0094186299	[#1089] control: Format proto files with clang-format ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-04-16 11:03:50 +00:00
Evgenii Stratonikov	91e79c98ba	[#1089] ape: Provide request actor as an additional target ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-04-16 11:03:50 +00:00
Evgenii Stratonikov	e5e0542482	[#1085] log: Move storage log message to constants package ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-04-15 07:57:00 +00:00
Evgenii Stratonikov	5be36924e3	[#41] log: Log storage operations in only in Debug ... They are mostly useless unless we need to _debug_ a specific issue. The amount of logs we produce is too big. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-04-15 07:57:00 +00:00
aarifullin	6a46c6d229	[#1090] tree: Make workaround for APE checks ... * Make `verifyClient` method perform APE check if a container was created with zero-filled basic ACL. * Object verbs are used in APE, until tree verbs are introduced. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-04-15 07:45:45 +00:00
aarifullin	f4dcb418f2	[#1090] ape: Move ape request and resource implementations to common package ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-04-15 07:45:45 +00:00
Dmitrii Stepanov	40781b3a20	[#1086] engine: Change mode in case of errors async ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-04-10 12:29:43 +00:00
Evgenii Stratonikov	5ef5734c4e	Reapply "[#972] Drop x/exp/slices dependency" ... This reverts commit 946f2ec2bf. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-04-10 12:09:34 +00:00
Evgenii Stratonikov	669103a33e	Reapply "[#972] Use slices.Sort* when useful" ... This reverts commit 3359349acb. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-04-10 12:09:34 +00:00
Evgenii Stratonikov	2c4b50a71e	Reapply "[#972] Use require.ElementsMatch() where possible" ... This reverts commit 7627d08914. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-04-10 12:09:34 +00:00
Evgenii Stratonikov	3dc81cb4fc	Reapply "[#972] Use min/max builtins" ... This reverts commit dad56d2e98. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-04-10 12:09:34 +00:00
Evgenii Stratonikov	d864945961	Reapply "[#972] pilorama: Remove removeDuplicatesInPlace()" ... This reverts commit 9f68305c2e. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-04-10 12:09:34 +00:00
Dmitrii Stepanov	1b17258c04	[#1029] metabase: Add refill metrics ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-04-10 13:05:44 +03:00
Dmitrii Stepanov	e3d9dd6ee8	[#1024] blobovnicza: Copy data on iterate ... DB value is only valid while the tx is alive. But handler may to run something in other goroutine. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-04-10 10:21:11 +03:00
Dmitrii Stepanov	57466594fb	[#1024] shard: Resync metabase concurrently ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-04-10 10:21:10 +03:00
Dmitrii Stepanov	1005bf4f56	[#1024] shard: Add refill metabase benchmark ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-04-10 10:21:10 +03:00
Dmitrii Stepanov	76398c06b0	[#1080] metabase: Add StorageID metric ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-04-10 10:00:08 +03:00
Dmitrii Stepanov	7b1adfed3e	[#1080] metabase: Open bucket for container counter once ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-04-10 10:00:08 +03:00
Dmitrii Stepanov	e74bdaa5d5	[#1080] ape: Use value for APE request ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-04-09 18:42:03 +03:00
Dmitrii Stepanov	338d8cbebd	[#1080] ape: Do not read object headers before Head/Get ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-04-09 15:27:40 +03:00
Dmitrii Stepanov	5b8200de88	[#984] blobovnicza: Do not fail rebuild on big objects ... If blobovnicza contains objects larger than object size parameter value, then rebuild fails with an error, because there is no such bucket in database. This commit forces to create bucket on rebuild. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-04-09 11:51:18 +00:00
Anton Nikiforov	2b88361849	[#1062] object: Fix buffer allocation for PayloadRange ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-04-09 11:59:07 +03:00
Dmitrii Stepanov	f5b67c6735	[#1064] policer: Disable EC processing ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-04-09 07:08:53 +00:00
Dmitrii Stepanov	1c5e0f90aa	[#1064] putsvc: Add EC put ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-04-09 07:08:53 +00:00
Dmitrii Stepanov	39da643354	[#1064] putsvc: Refactor distributed target ... Extract object builder. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-04-09 07:08:53 +00:00
Alexander Chuprov	17f7adb640	[#1065] adm: Add support EC parameters ... Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>	2024-04-08 12:27:30 +03:00
Anton Nikiforov	9aa533e59a	[#1072] node, ir: Add new config option kludge_compatibility_mode ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-04-04 11:05:25 +03:00
Anton Nikiforov	d614f04a0a	[#1072] Fix gofumpt issues ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-04-03 22:21:14 +03:00
aarifullin	4738508ce2	[#1063] go.mod: Update SDK version ... * Update frostfs-sdk and frostfs-api-go versions. * Refactor depreacted method ReplicaNumberByIndex. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-04-03 10:33:51 +00:00
Evgenii Stratonikov	ff4c23f59a	[#1070] services/tree: Fix fast listing depth processing ... For unsorted `GetSubTree()` we return a single node for depth=1. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-04-02 14:41:31 +00:00
Evgenii Stratonikov	17af91619a	[#1070] pilorama: Fix cycling behaviour for sorted listing ... In case there are no items left, return empty slice. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-04-02 14:41:31 +00:00
Ekaterina Lebedeva	d5194ab2a6	[#949] metabase: fix shard.UpdateID() ... metabase.Open() now reports metabase mode metric. shard.UpdateID() needs to read shard ID from metabase => needs to open metabase. It caused reporting 'shard undefined' metrics. To avoid reporting wrong metrics metabase.GetShardID() was added which also opens metabase and does not report metrics. Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2024-04-01 17:27:34 +03:00
Ekaterina Lebedeva	81a0346a96	[#949] metabase: fix metabase mode metric ... It used to always show CLOSED regardless of actual mode. Now metric represents actual metabase mode of operations. Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2024-04-01 17:27:34 +03:00
Evgenii Stratonikov	e12fcc041d	[#1059] services/tree: Fast sorted listing ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-04-01 12:37:34 +00:00
Evgenii Stratonikov	f23e38c285	Revert "[#446] engine: Move to read-only on blobstor errors" ... This reverts commit 69df0d21c2. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-04-01 12:48:30 +03:00
Dmitrii Stepanov	942d83611b	[#874] engine: Revert Check object existance concurrently ... This reverts commit f526f49995. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-04-01 08:42:34 +00:00
Evgenii Stratonikov	fd8cdb9671	[#1057] netmap: Do not iterate over external addresses in Node ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-03-29 20:46:24 +00:00
Ekaterina Lebedeva	0990a9b0bd	[#1055] blobstor: fix mode metric ... It used to always show CLOSED after setting shard mode to read-only regardless of actual mode. Now metric represents actual blobstor mode of operations. Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2024-03-29 20:44:47 +00:00
Evgenii Stratonikov	c7a12ca3d8	[#1054] network: Optimize IsTLSEnabled() ... No big deal, but it is called multiple times in sorting routine, this easily results in 20 allocations per group traversal. ``` goos: linux goarch: amd64 pkg: git.frostfs.info/TrueCloudLab/frostfs-node/pkg/network cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz │ old │ new │ │ sec/op │ sec/op vs base │ AddressTLSEnabled-8 184.6n ± 1% 103.3n ± 6% -44.04% (p=0.000 n=10) │ old │ new │ │ B/op │ B/op vs base │ AddressTLSEnabled-8 704.0 ± 0% 0.0 ± 0% -100.00% (p=0.000 n=10) │ old │ new │ │ allocs/op │ allocs/op vs base │ AddressTLSEnabled-8 1.000 ± 0% 0.000 ± 0% -100.00% (p=0.000 n=10) ``` Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-03-21 18:48:35 +03:00
Dmitrii Stepanov	c09c701613	[#1048] metabase: Fix drop buckets during resync ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-03-19 14:28:31 +03:00
aarifullin	6959e617c4	[#1047] object: Set container owner ID property to ape request ... * Introduce ContainerOwner field in RequestContext. * Set ContainerOwner in aclv2 middleware. * Set PropertyKeyContainerOwnerID for object ape request. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-03-18 15:39:50 +00:00
aarifullin	d7be70e93f	[#1040] object: Wrap CheckAPE errors to status errors ... * All methods should wrap CheckAPE error, if it occurs, to status error. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-03-14 07:34:03 +00:00
Anton Nikiforov	5ee5f1df42	[#976] control: Introduce new method RemoveChainLocalOverridesByTarget ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-03-13 15:33:19 +03:00
Anton Nikiforov	2d595ec15f	[#976] ape: Update dependency ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-03-13 14:54:19 +03:00
Anton Nikiforov	7ed07d2dfd	[#976] morph: Implement missing methods for SwitchRPCGuardedActor ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-03-13 14:54:19 +03:00
Dmitrii Stepanov	17f5463389	[#1043] cli: Add reset evacuation status command ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-03-13 10:29:45 +00:00
Dmitrii Stepanov	31e2396a5f	[#1043] control: Add ResetEvacuationStatus implementation ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-03-13 10:29:45 +00:00
Dmitrii Stepanov	926cdeb072	[#1043] services: Regenerate proto ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-03-13 10:29:45 +00:00
Airat Arifullin	5c252c9193	[#1039] object: Skip APE check for certain request roles ... * Skip APE check if a role is Container. * Skip APE check if a role is IR and methods are get-like. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-03-12 16:15:20 +03:00
Anton Nikiforov	3195142d67	[#959] writecache: Avoid manipulation with cache in DEGRADED mode ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-03-11 18:35:41 +00:00
Dmitrii Stepanov	d433b49265	[#973] node: Resolve perfsprint linter ... `fmt.Errorf can be replaced with errors.New` and `fmt.Sprintf can be replaced with string addition` Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-03-11 17:55:50 +03:00
Dmitrii Stepanov	66a26b7775	[#973] node: Resolve revive: unused-parameter linter ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-03-11 17:11:49 +03:00
Dmitrii Stepanov	0882840bf5	[#634] shard: Add writecache inhume tests ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-03-06 13:12:34 +03:00
Anton Nikiforov	ae5bb87e70	Revert "[#866] Use TTL for blobovnicza tree cache" ... This reverts commit d9cbb16bd3. Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-03-01 19:29:33 +03:00
Dmitrii Stepanov	d6534fd755	[#1016] frostfs-node: Fix gopls issues ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-03-01 12:13:43 +03:00
Airat Arifullin	93bf9acbc2	[#898] control: Remove removed flag from RemoveChainLocalOverrideResponse ... * Remove removed flag in service.proto for RemoveChainLocalOverrideResponse. * Regenerate control API. * Return error only if RemoveOverride returns non-NotFound code. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-02-28 19:07:07 +00:00
Airat Arifullin	75a1a95c2c	[#986] tree: Skip ACL checks if basicACL mask is unset ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-02-28 19:05:57 +00:00
Airat Arifullin	b1d171c261	[#986] container: Interpret APE NoRuleFound as request deny ... * If APE check returns NoRuleFound, then it is taken for request deny. * Add more unit-test for ape container middleware. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-02-28 19:05:57 +00:00
Airat Arifullin	7cc368e188	[#986] object: Introduce soft ape checks ... * Soft APE check means that APE should allow request even it gets status NoRuleFound for a request. Otherwise, it is interpreted as Deny. * Soft APE check is performed if basic ACL mask is not set. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-02-28 19:05:57 +00:00
Dmitrii Stepanov	918613546f	[#1008] metabase: Do not update storageID on put ... There may be a race condition between put an object and flushing the writecache: 1. Put object to the writecache 2. Writecache flushes object to the blobstore and sets blobstore's storageID 3. Put object to the metabase, set writecache's storageID Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-02-28 11:01:50 +03:00
Dmitrii Stepanov	2ad433dbcb	[#1005] engine: Drop shards weights ... Unused. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-02-26 17:25:05 +03:00
Evgenii Stratonikov	4b13b85173	[#1000] morph: Fix batch size in TraverseIterator() ... Initial prefetch size can be arbitrary an restricted only by VM/RPC limits. For TraverseIterator() there is an explicit check on the server-side, though. Introduced in df055fead5. Refs #931. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-23 06:24:40 +00:00
Evgenii Stratonikov	e18f0f5178	[#999] morph: Use Global scope for proxy contract ... Proxy contract can now be used as an owner of NNS domains, thus we need it not only to pay for the transaction but also to check domain ownership. CalledByEntry is not enough, because we may register NNS domains owned by proxy indirectly from the container contract. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-21 14:28:42 +03:00
Dmitrii Stepanov	7470c383dd	[#997] metabase: Drop toMoveIt bucket ... It is not used. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-02-21 10:06:05 +03:00
Evgenii Stratonikov	adf7ebab5b	[#996] metabase: Speed up bucket creation ... Most of the time it exits, e.g. when it is per-container and use on each object PUT. Bbolt implementation first tries to create bucket and then returns it if it exists. Create operation uses cursor and thus is not very lightweight, we can avoid it. ``` goos: linux goarch: amd64 pkg: git.frostfs.info/TrueCloudLab/frostfs-node/pkg/local_object_storage/metabase cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz │ old │ new │ │ sec/op │ sec/op vs base │ Put/parallel-8 174.4µ ± 3% 163.3µ ± 3% -6.39% (p=0.000 n=10) Put/sequential-8 263.3µ ± 2% 259.0µ ± 1% -1.64% (p=0.000 n=10) geomean 214.3µ 205.6µ -4.05% │ old │ new │ │ B/op │ B/op vs base │ Put/parallel-8 275.3Ki ± 3% 281.1Ki ± 4% ~ (p=0.063 n=10) Put/sequential-8 413.0Ki ± 2% 426.6Ki ± 2% +3.29% (p=0.003 n=10) geomean 337.2Ki 346.3Ki +2.70% │ old │ new │ │ allocs/op │ allocs/op vs base │ Put/parallel-8 678.0 ± 1% 524.5 ± 2% -22.64% (p=0.000 n=10) Put/sequential-8 1.329k ± 0% 1.183k ± 0% -10.91% (p=0.000 n=10) geomean 949.1 787.9 -16.98% ``` Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-20 15:42:58 +00:00
Evgenii Stratonikov	9f68305c2e	Revert "[#972] pilorama: Remove removeDuplicatesInPlace()" ... This reverts commit 45fd4e4ff1. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-19 15:36:01 +00:00
Evgenii Stratonikov	dad56d2e98	Revert "[#972] Use min/max builtins" ... This reverts commit 89784b2e0a. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-19 15:36:01 +00:00
Evgenii Stratonikov	7627d08914	Revert "[#972] Use require.ElementsMatch() where possible" ... This reverts commit 6d9707ff1f. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-19 15:36:01 +00:00
Evgenii Stratonikov	3359349acb	Revert "[#972] Use slices.Sort* when useful" ... This reverts commit b871d7a5e8. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-19 15:36:01 +00:00