3297 commits

Author	SHA1	Message	Date
Dmitrii Stepanov	68029d756e	[#1302] writecache: Allow to specify custom page size ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-08-09 12:12:07 +03:00
Dmitrii Stepanov	fa82854af4	[#1302] writecache: Add put->flush->put benchmark ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-08-09 12:12:07 +03:00
Dmitrii Stepanov	c985b1198f	[#1302] putSvc: Override SuccessAfter for non-regular objects in EC containers ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-08-09 10:42:55 +03:00
Dmitrii Stepanov	08b1f18bca	[#1296] writecache: Add count limit ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-08-09 06:30:32 +00:00
Dmitrii Stepanov	36efccd862	[#1298] writecache: Add shrink flag for Seal command ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-08-08 16:32:29 +03:00
Dmitrii Stepanov	5c01bd5be8	[#1298] writecache: Add restore-mode flag for Seal command ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-08-07 11:20:14 +03:00
Dmitrii Stepanov	8e51d7849a	[#1295] getSvc: Assemble complex EC object headers without linking object ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-08-06 16:48:12 +03:00
Dmitrii Stepanov	10602b55b1	[#1295] engine: Resolve funlen linter ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-08-06 09:45:47 +03:00
Dmitrii Stepanov	eeca796d2e	[#1295] engine: Log object address in case of error ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-08-06 09:45:43 +03:00
Ekaterina Lebedeva	327d364f34	[#1262] sdnotify: Get rid of go:linkname for nanotime ... Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2024-08-05 12:37:16 +00:00
Dmitrii Stepanov	dc3dcabadc	[#1291] morph: Reconnect to the highest priority endpoint ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-08-02 17:54:22 +03:00
Dmitrii Stepanov	8021bacc43	[#1288] putSvc: Respect TTL for EC put ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-08-02 13:01:42 +00:00
Dmitrii Stepanov	7e04083c27	[#1278] containerSvc: Validate FrostFSID subject exitence on Put ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-07-29 16:01:37 +03:00
Dmitrii Stepanov	a12c39667d	[#1278] ir: Do not allow to create container without FrostFSID record ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-07-29 16:01:37 +03:00
Evgenii Stratonikov	8377372a40	[#1276] go.mod: Update api-go ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-07-26 16:44:19 +03:00
Dmitrii Stepanov	7fd7961dfa	[#1271] getSvc: Fix local EC chunk get ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-07-24 08:52:16 +03:00
Dmitrii Stepanov	8398a8b4b3	[#1271] getSvc: Fix head --raw assemble for EC ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-07-24 08:52:08 +03:00
George Bartolomey	9c2c76ca32	[#1259] Move pkg/util/locode to frostfs-locode-db ... Removed pkg/util/locode package, added git.frostfs.info/TrueCloudLab/frostfs-locode-db/pkg/locode dependency. Signed-off-by: George Bartolomey <george@bh4.ru>	2024-07-23 14:59:14 +03:00
Aleksey Savchuk	9ee1bd3669	[#1265] *: Run gofumpt ... Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2024-07-23 10:36:32 +03:00
Dmitrii Stepanov	18182e578e	[#1261] shard: Fix delete objects from FSTree ... Replace nil storageID with empty like by shard.Get. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-07-22 14:38:09 +03:00
Dmitrii Stepanov	3119f2fbc3	[#1257] metabase: Delete EC gc marks and split info ... EC parent and split gc marks should be deleted after last EC chunk delete. Also should delete split info for EC parent and split parent. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-07-19 17:14:14 +03:00
Airat Arifullin	eadcea8df0	[#1249] object: Remove all APE pre-checks in handlers ... * Methods `Head`, `Get`, `GetRangeHash` should no longer use APE pre-checks as that leads only to incorrect rule chain processing for requests: 1. Immediate return with `NoRuleFound` may be unexpected as some `Allow` rule is actually defined but can't be matched yet as it gets no object attributes; 2. Immdediate return with `Allow` may be incorrect as some `Deny` rule is actually defined but can't bet matched yet as it gets no object attirbutes; 3. Pre-check breaks compatibility for converted EACL-tables. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-07-18 13:52:43 +00:00
Dmitrii Stepanov	5e5ee545b6	[#1254] policer: Fix svacer warning ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-07-18 11:30:39 +00:00
Dmitrii Stepanov	27caa8a72f	[#1256] metabase: Put split parent object ID for EC chunks ... It is required to save split parent ID too, not only split ID. Otherwise inhume operation works incorrect: shard with last part may be skipped and parent object will be available. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-07-17 18:08:07 +03:00
Dmitrii Stepanov	e83d39e33f	[#1253] deleteSvc: Use copy of common parameters ... getSvc may change the values of some fields, so Head will affect Delete or Put. In this case, the change is necessary so that the session token is stored in the tombstone object (EC assemble calls `ForgetTokens`). Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-07-17 14:24:27 +03:00
Dmitrii Stepanov	fc383ea6ae	[#1253] getSvc: Fix EC objects get ... Now EC objects assembling is performed concurrently. Also fixed issue with an error in case of getting EC object via non-container node. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-07-17 14:24:27 +03:00
Evgenii Stratonikov	00a88b9936	[#1251] *: Run gofumpt ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-07-17 11:25:07 +03:00
Evgenii Stratonikov	3940bc17c1	[#1251] pilorama: Allow traversing multiple branches in parallel ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-07-17 11:25:07 +03:00
Evgenii Stratonikov	e5767c9002	[#1250] *: Reformat proto filets with clang-format ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-07-16 15:13:08 +00:00
Evgenii Stratonikov	286df198c9	[#1248] placement: Decouple ContainerNodes() cache from the placement builder. ... Also, write tests. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-07-16 12:21:30 +00:00
Evgenii Stratonikov	3a48b282b6	[#1248] placement: Use epoch to track netmap versions ... Previously we used pointer, this could have worked, because most of the time, the netmap is cached. This didn't work, however, because `lastNm` field was always nil. Rework the mechanism completely: 1. Use epoch to track netmap versions, as it it simpler and is unrelated to the TTL of an underlying cache. 2. Fix a bug where the epoch could change while mutex was unlocked. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-07-16 12:21:30 +00:00
Evgenii Stratonikov	21431f22c0	[#1248] placement: Use cid.ID as key in the cache ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-07-16 12:21:30 +00:00
Airat Arifullin	d5dc14c639	[#1243] object: Make APE checker set x-headers to request properties ... * Update go.mod, go.sum; * Add x-headers to request properties; * Add a unit-test. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-07-16 07:28:42 +00:00
Ekaterina Lebedeva	39866a957c	[#1196] morph/client: Fix ArrayFromStackItem() description ... Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2024-07-15 16:24:58 +03:00
Ekaterina Lebedeva	d02a7c2d38	[#1196] morph/client: Remove duplicate utility functions ... * We used several utility functions to parse frostfsid client subject and extended subject. However, following the changes in TrueCloudLab/frostfs-contract#97, these utility functions have become public. So there is no more need to have them here. * There was a mismatch of slice parameter required length between frostfs-node's and frostfs-contract's utility functions, `checkStackItem()` solves this problem. Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2024-07-15 16:24:48 +03:00
Anton Nikiforov	3bf6e6dde6	[#1238] engine: Add non-blocking send in the shard's notification channel ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-07-10 11:37:11 +03:00
Evgenii Stratonikov	b027a7f91e	[#1234] pilorama: Fix GetByPath() on duplicate directories ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-07-10 06:11:38 +00:00
Evgenii Stratonikov	6ace2f597e	[#1234] pilorama: Add test for duplicate directory behaviour ... When AddByPath() is called concurrently on 2 different nodes, internal path components may be created twice. This violates some of our assumptions in GetByPath() and, indirectly, in S3 handling of GetSubTree() results. Add a test for the correct behaviour, fixes will follow. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-07-10 06:11:38 +00:00
Dmitrii Stepanov	84ecd61dfd	[#1233] putSvc: Try to put EC chunk to any node ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-07-09 07:54:29 +00:00
Airat Arifullin	d90aab5454	[#1229] util: Fix session token expiration check ... * Make session token expired at `current_epoch + 1` but not at `current_epoch` when it's still valid. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-07-08 08:15:56 +00:00
Airat Arifullin	0c2b6f3dac	[#1216] ape: Make services use bearer chains fed router ... * Refactor object and tree service - they should instantiate chain router cheking the bearer token. If there are no bearer token rules, then defaul chain router is used. * Fix unit-tests. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-07-05 18:26:48 +00:00
Airat Arifullin	47bcd346d3	[#1216] ape: Introduce BearerChainFeedRouter ... * Unlike default chain router, `BearerChainFedRouter` performs checks for overrides defined in the bearer token; * Add unit-test for the introduced router. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-07-05 18:26:48 +00:00
Dmitrii Stepanov	8eb591d668	[#1231] policer: Add EC node-off unit test ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-07-05 12:11:36 +03:00
Denis Kirillov	cfd5e3d403	[#1227] morph/event: Release ants.Pool on listener stopping ... Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>	2024-07-04 10:55:05 +00:00
Dmitrii Stepanov	62cbb72a5e	[#1226] blobovniczatree: Delete fix db extensions in Init() ... Since several releases have been released, this code is not relevant. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-07-04 12:22:06 +03:00
Dmitrii Stepanov	78b1d9b18d	[#1226] blobovniczatree: Drop leaf width limitation ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-07-04 12:22:06 +03:00
Dmitrii Stepanov	40c9ddb6ba	[#1226] blobovniczatree: Drop init in advance option ... To make blobovniczatree unlimited. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-07-04 12:22:06 +03:00
Dmitrii Stepanov	3a797e4682	[#1222] engine: Fix tree evacuation ... Do not fail if it is unable to evacuate tree to other node. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-07-04 10:38:10 +03:00
Dmitrii Stepanov	2bac82cd6f	[#1222] engine: Fix object evacuation ... Do not fail evacuation if it unable to evacuate object to other node. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-07-04 10:38:10 +03:00
Anton Nikiforov	bbe95dac8b	[#1225] engine: Log the error when check object existence ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-07-04 07:22:54 +00:00
Evgenii Stratonikov	80d7459560	Revert "[#1196] morph/client: Remove duplicate utility functions" ... This reverts commit 259007540f. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-07-03 15:44:35 +03:00
Airat Arifullin	4bd4667791	[#1218] tree: Fix bearer token validation ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-07-03 07:22:11 +00:00
Airat Arifullin	f3a861806e	[#1218] object: Fix bearer token validation ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-07-03 07:22:11 +00:00
Airat Arifullin	a378ff9cf6	[#1218] object: Pass container owner for backward get method check ... * `getStreamBasicChecker` must define `containerOwner` for backward checks, otherwise bearer token cannot be validated for the token issuer. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-07-03 07:22:11 +00:00
Evgenii Stratonikov	91bed3b0ba	[#1219] Remove Container.SetEACL method ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-07-02 13:05:40 +00:00
Ekaterina Lebedeva	259007540f	[#1196] morph/client: Remove duplicate utility functions ... We used several utility functions to parse frostfsid client subject and extended subject. However, following the changes in TrueCloudLab/frostfs-contract#97, these utility functions have become public. So there is no more need to have them here. Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2024-07-02 15:19:59 +03:00
Ekaterina Lebedeva	56eeb630b6	[#1217] Fix grammar mistakes and misspelling ... Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2024-07-01 19:14:25 +03:00
Dmitrii Stepanov	7a8ac4907a	[#1213] engine: Drop unused ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-07-01 06:49:35 +00:00
Dmitrii Stepanov	dc2867682f	[#1213] deleteSvc: Do not allow to delete EC chunks ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-07-01 06:49:35 +00:00
Dmitrii Stepanov	dc6778f385	[#1213] fmt: Drop unused interfaces ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-07-01 06:49:35 +00:00
Evgenii Stratonikov	7085723c6b	[#1074] pilorama: Allow empty filenames in SortedByFilename() ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-06-28 17:46:24 +03:00
Dmitrii Stepanov	4c7ff159ec	[#1201] writecache: Cancel background flush without lock ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-06-28 09:02:36 +03:00
Dmitrii Stepanov	4f7d76c9ef	[#1206] audit: Drop not required events ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-06-27 10:54:31 +00:00
Dmitrii Stepanov	4951babd5f	[#1208] blobstor: Fix delete without storage id ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-06-27 11:59:08 +03:00
Dmitrii Stepanov	81ea91de52	[#451] metrics: Move to internal ... `metrics` don't look like something others want to import. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-06-25 08:52:37 +00:00
Airat Arifullin	11a38a0a84	[#1190] tree: GroupIDs must also be target of APE checks ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-06-25 08:49:20 +00:00
Airat Arifullin	0b87388c18	[#1190] object: GroupIDs must also be target of APE checks ... * Also add new test case for ape middleware in container service. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-06-25 08:49:20 +00:00
Airat Arifullin	621dbf58ab	[#1190] container: GroupIDs must also be target of APE checks ... * Also add new test case for ape middleware in container service. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-06-25 08:49:20 +00:00
Airat Arifullin	a1f7615b7e	[#1190] ape: Introduce Groups util function to retrieve actor's groupIDs ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-06-25 08:49:20 +00:00
Dmitrii Stepanov	a83eeddb1d	[#60] control: Add GetNetmapStatus method ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-06-20 16:28:42 +03:00
Anton Nikiforov	9ac74efc41	[#1173] shard: Use mode from config on reload ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-06-20 11:29:10 +00:00
Anton Nikiforov	40b68bcb6c	[#1109] object: Validate attribute EXPIRATION_EPOCH on put ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-06-19 17:19:27 +00:00
Dmitrii Stepanov	fd28461def	[#1184] ir: Add grpc middleware for control service ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-06-19 16:05:53 +03:00
Dmitrii Stepanov	ecd1ed7a5e	[#1184] node: Add audit middleware for grpc services ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-06-19 16:05:53 +03:00
Evgenii Stratonikov	75eedf71f3	[#1187] pilorama/test: Remove debug print ... Introduced in e12fcc041d. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-06-18 15:09:44 +03:00
Ekaterina Lebedeva	5b100699d7	[#566] policer: Move isClientErrMaintenance to frostfs-sdk-go ... Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2024-06-18 10:20:45 +03:00
Dmitrii Stepanov	76cf7a051b	[#1178] shard: Check metabase existence before read shard ID ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-06-17 09:59:15 +03:00
Evgenii Stratonikov	96fe271bab	[#1170] innerring: Support morph mTLS ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-06-14 09:28:32 +03:00
Evgenii Stratonikov	42ecc2f2b9	[#1170] morph: Support mTLS ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-06-14 09:26:27 +03:00
Anton Nikiforov	68ac490729	[#1174] shard: Update metric mode_info on Init ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-06-13 08:32:59 +00:00
Dmitrii Stepanov	6a39c3d15e	[#1086] engine: Do not use metabase if shard looks bad ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-06-13 07:35:22 +00:00
Evgenii Stratonikov	9d73f9c2c6	Reapply "[#446] engine: Move to read-only on blobstor errors" ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-06-13 07:35:22 +00:00
Anton Nikiforov	b9fcaad21f	[#1168] shard: Set Disabled as default mode for components ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-06-11 15:13:38 +00:00
Dmitrii Stepanov	6cf512e574	[#1166] blobovniczatree: Handle blobovnicza's NoSpaceLeft error ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-06-07 17:15:43 +03:00
Dmitrii Stepanov	e7d479f4c2	[#1166] blobovnicza: Return NoSpaceLeft error instead of syscall.ENOSPC ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-06-07 17:15:43 +03:00
Airat Arifullin	239323eeef	[#1157] tree: Make tree service use Bearer token's APE overrides ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-06-07 12:11:11 +00:00
Airat Arifullin	04a3f891fd	[#1157] object: Make APE checker use Bearer-token's APE overrides ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-06-07 12:11:11 +00:00
Anton Nikiforov	4edff5eea6	[#1157] ape: Introduce single-run chain router ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-06-07 12:11:11 +00:00
Anton Nikiforov	a90310335d	[#1156] ape: Return not found when removing local overrides ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-06-07 12:10:57 +00:00
Evgenii Stratonikov	a849236b68	[#1161] node: Remove notification functionality ... It is unused and will be reworked in future. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-06-07 12:10:51 +00:00
Evgenii Stratonikov	3f1961157e	[#1163] metabase: Handle multiple splitInfos for EC ... For REP updating split info is handled explicitly by a high-level PUT logic. For EC it is trickier, because the address of an object we put is only distantly related to a split info. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-06-06 16:26:29 +03:00
Evgenii Stratonikov	2e074d3846	[#1163] metabase: Properly save EC parent split ID ... Search by SplitID should return all parts of a complex object. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-06-05 12:40:16 +03:00
Alexander Chuprov	806236da78	[#1121] node: Change mode of shard components ... Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>	2024-06-05 05:55:24 +00:00
Alexander Chuprov	6f2187a420	[#1121] node: Refactor mods of shard ... Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>	2024-06-05 05:55:24 +00:00
Dmitrii Stepanov	cc2449beaf	[#1158] metabase: Fix EC storage schema ... Do not store EC info twice. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-06-04 17:24:40 +03:00
Airat Arifullin	5aacb8fc86	[#1144] metabase: Save parent attributes for ec-chunks ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-05-31 19:55:32 +03:00
Airat Arifullin	f8e33f8e3a	[#1144] metabase: Proprely choose root OID for EC-splitted objects ... * If EC-parent is a part of Split itself, then save to root bucket its parent; * If EC-parent is not a part of Split itself, then save to root bucket OID of this EC-parent. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-05-31 19:53:32 +03:00
Airat Arifullin	f0edebea18	[#1144] metabase: Support ec parent filter for Search ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-05-31 19:53:32 +03:00
Airat Arifullin	0b367007fc	[#1152] go.mod: Update api-go and sdk versions ... * Resolve conflicts for apemanager since api-go contains ape and apemanager packages and SDK only ape package. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-05-31 15:39:09 +03:00
Anton Nikiforov	92e19feb57	[#1147] node: Use public fields for shard.ExistsPrm ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-05-30 08:13:04 +00:00
Anton Nikiforov	c1af13b47e	[#1147] node: Fix issue from gopls ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-05-30 08:13:04 +00:00
Anton Nikiforov	6130650bb6	[#1147] node: Implement Lock\Delete requests for EC object ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-05-30 08:13:04 +00:00
Anton Nikiforov	a82c8cc5b8	[#1147] gc: Execute callback for expired tombstones when they exists ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-05-30 08:13:04 +00:00
Anton Nikiforov	d355274cd0	[#1147] object: Use methods on pointer for searchsvc.execCtx ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-05-30 08:13:04 +00:00
Anton Nikiforov	3555c73225	[#1147] object: Use methods on pointer for deletesvc.execCtx ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-05-30 08:13:04 +00:00
Anton Nikiforov	e43e7bec3a	[#1147] log: Remove redundant address field from log ... Filled when logger created for `request` object from package `getsvc`. Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-05-30 08:13:04 +00:00
Anton Nikiforov	50923ed81c	[#1147] Fix gofumpt issue ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-05-30 08:13:04 +00:00
Alex Vanin	4a34d0d40e	[#1149] go.mod: Bump neo-go up to v0.106.0 ... Required to work with neo-go v0.106.0 node with default hardfork configuration. Without neo-go client version bump, it throws error. failed to get network magic: unexpected hardfork: Cockatrice Signed-off-by: Alex Vanin <a.vanin@yadro.com>	2024-05-30 08:11:58 +00:00
Airat Arifullin	3627b44e92	[#1142] tree: Fill APE-request with source IP property ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-05-27 10:17:17 +00:00
Airat Arifullin	482c5129ac	[#1142] object: Fill APE-request with source IP property ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-05-27 10:17:17 +00:00
Airat Arifullin	43625e7536	[#1142] container: Fill APE-request property with source IP ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-05-27 10:17:17 +00:00
Airat Arifullin	542d3adcb2	[#1105] apemanager: Implement apemanager service ... * Introduce grpc server for apemanager service and its implementation in `pkg/services/apemanager`. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-05-27 09:34:21 +00:00
Airat Arifullin	51ade979e8	[#1105] ape: Introduce contract storage with proxy contract verification ... * `ProxyVerificationContractStorage` uses Proxy contract as a cosigner. * `ProxyVerificationContractStorage` recreates a contract storage for each handler invocation because of an issue: rpc-actor from morph client may be expired. This way won't create a bottlenecks because it is expected that this contract storage implementation will be used not so often. * Make morph client return `RPCActor` (that is websocket client in fact). * Make `SwitchRPCGuardedActor` return `RPCActor` as it will be used for `ProxyVerificationContractStorage`. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-05-27 09:34:21 +00:00
Airat Arifullin	40b04c00ef	[#1141] metabase: Fix IsUserObject method ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-05-20 13:24:15 +03:00
Airat Arifullin	89a80e9a0f	[#1141] metabase: Fix putUniqueIndexItem ... * `GetECHeader` is not correct way to determine if an object's got EC-header: `ECHeader` must be used for that. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-05-20 13:24:08 +03:00
Airat Arifullin	8fd678e269	[#1141] go.mod: Update frostfs-sdk-go and frostfs-api-go/v2 ... * Also fix unit-test. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-05-20 13:22:48 +03:00
Dmitrii Stepanov	436c9f5558	[#1129] policer: Restore EC object ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-05-17 14:36:18 +03:00
Dmitrii Stepanov	44f2e8f27f	[#1129] putSvc: Allow to put single unprepared object to EC container ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-05-16 16:28:49 +03:00
Dmitrii Stepanov	0e42126ddc	[#1129] object: Fix check owner for EC part ... Do not validate EC part owner if request from container node. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-05-16 16:28:49 +03:00
Dmitrii Stepanov	1cd8562db8	[#1129] policer: Refactor shortage ... Drop override inside method. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-05-16 16:28:49 +03:00
Dmitrii Stepanov	4ab6c404f7	[#1129] policer: Drop unused ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-05-16 16:28:49 +03:00
Dmitrii Stepanov	cbe9757490	[#1129] policer: Pull required EC chunks ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-05-16 16:28:49 +03:00
Dmitrii Stepanov	d45d086acd	[#1129] policer: Add EC chunk replication ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-05-16 16:28:48 +03:00
Alexander Chuprov	b078fe5ba1	[#1092] control: Move SignMessage to separate package ... Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>	2024-05-16 12:14:01 +03:00
Evgenii Stratonikov	f3e09cb09b	[#1135] sdnotify: Send MONOTONIC_USEC on reload ... Fixes #1135 Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-05-15 12:52:52 +00:00
Anton Nikiforov	5c582e96fd	[#1136] metabase: Fix creation of ECInfoError ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-05-15 11:04:27 +00:00
Evgenii Stratonikov	b3eaa8a9bc	[#1083] objsvc/v2: Check response status in RANGE_HASH forwarder ... Fixes #1083 Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-05-15 12:04:21 +03:00
Evgenii Stratonikov	0924b62a95	[#1083] objsvc/v2: Unify response verification after forwarding ... 1. Use the same routine for HEAD/GET_RANGE methods. 2. Make error message similar. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-05-15 12:04:06 +03:00
Evgenii Stratonikov	300654b045	[#1083] objsvc/v2: Properly check response status after forwarding ... Previously we had cryptic error: ``` debug get/remote.go:38 remote call failed {"component": "Object.Get service", "request": "HEAD", "address": "9sTxoVrhJ7WBtXQfK2NJ7zDV5yCF7BPLKK1XTxYPdGsP/BbHV4KZZ8y2BPqAT5kyjdHRLkfbtY2xf5uYoMVqxACn1", "raw": false, "local": false, "with session": false, "with bearer": false, "error": "unexpected header type <nil>"} ``` Now we have and expected error: ``` debug get/remote.go:38 remote call failed {"component": "Object.Get service", "request": "HEAD", "address": "D2rqaMG4D2VHdv3HKky8UYSYmwQFH2v9oXXqtyRZPTMy/BbHV4KZZ8y2BPqAT5kyjdHRLkfbtY2xf5uYoMVqxACn1", "raw": false, "local": false, "with session": false, "with bearer": false, "error": "status: code = 2049 message = object not found"} ``` Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-05-15 12:04:06 +03:00
Evgenii Stratonikov	6e71ae3bda	[#1130] fstree: Remove useless Stat() call ... ``` goos: linux goarch: amd64 pkg: git.frostfs.info/TrueCloudLab/frostfs-node/pkg/local_object_storage/blobstor cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz │ old │ new │ │ sec/op │ sec/op vs base │ SubstorageReadPerf/fstree_nosync-seq100-8 2.689µ ± 2% 2.428µ ± 4% -9.72% (p=0.000 n=10) SubstorageReadPerf/fstree_nosync-rand100-8 2.727µ ± 1% 2.497µ ± 2% -8.42% (p=0.000 n=10) geomean 2.708µ 2.462µ -9.07% ``` Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-05-14 16:05:45 +03:00
Evgenii Stratonikov	bf9bdde8ea	[#1128] util/test: Remove unused package ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-05-14 12:48:13 +00:00
Airat Arifullin	952d13cd2b	[#1124] cli: Improve APE rule parsing ... * Make APE rule parser to read condition's kind in unambiguous using lexemes `ResourceCondition`, `RequestCondition` instead confusing `Object.Request`, `Object.Resource`. * Fix unit-tests. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-05-14 12:23:26 +03:00
aarifullin	20baf6e112	[#1108] ape: Update policy-engine version for listing by iteration ... * Update go.mod with a new version of policy-engine pacakge. * Adapt SwitchRPCGuardedActor to ContractStorage interface. * Fix `frostfs-adm` util. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-05-14 12:17:56 +03:00
Dmitrii Stepanov	0144117cc9	[#1125] objectSvc: Add EC header APE check ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-05-08 16:25:55 +03:00
Dmitrii Stepanov	ada1b9f737	[#1120] objectSvc: Fix EC put placement ... Use parent object ID to compute placement. Fix too many copies saving. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-05-08 15:23:57 +03:00
Anton Nikiforov	fe2c1c926f	[#1112] node: Fix race warning for GetObjectAndWritePayload ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-05-07 14:47:21 +03:00
Anton Nikiforov	3e782527b8	[#1112] node: Add test for Range request for EC object ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-05-07 14:47:21 +03:00
Anton Nikiforov	21a490da8f	[#1112] Fix issue from gofumpt ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-05-07 14:47:21 +03:00
Evgenii Stratonikov	93c0ccad4f	[#1077] objectsvc: Fix possible panic in GetRange() ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-05-07 14:47:21 +03:00
Anton Nikiforov	00b2b77b26	[#1112] node: Implement Range\RangeHash requests for EC object ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-05-07 14:47:21 +03:00
aarifullin	b60a51b862	[#1117] ape: Introduce FormFrostfsIDRequestProperties method ... * `FormFrostfsIDRequestProperties` gets user claim tags and group id and sets them as ape request properties. * Make tree, container and object service use the method. * Fix unit-tests. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-05-07 10:01:21 +00:00
aarifullin	6c76c9b457	[#1117] core: Introduce SubjectProvider interface for FrostfsID ... * Make tree, object and container services use SubjectProvider interface. * Fix unit-tests. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-05-07 10:01:21 +00:00
aarifullin	45f4e6939d	[#1117] morph: Make frostfsid client provide GetSubjectExtended method ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-05-07 10:01:21 +00:00
Ekaterina Lebedeva	e07869a8cf	[#1100] Remove unused fields ... Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2024-05-06 10:14:36 +03:00
Alexander Chuprov	ec2873caa7	[#1116] node: Fix writecache metrics ... Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>	2024-05-02 20:03:33 +03:00
Evgenii Stratonikov	71789676d5	[#1114] aclsvc: Add tests for request ownership ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-05-02 11:57:39 +03:00
Alexander Chuprov	c9efaa5819	[#966] node: Add path of the write_cache to metric labels ... Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>	2024-05-02 06:46:46 +00:00
Alexander Chuprov	4730ecfdb8	[#966] node: Refactor WriteCacheMetrics interface ... Grouping common fields of methods will enhance the readability of the interface. Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>	2024-05-02 06:46:46 +00:00
Anton Nikiforov	411a8d0245	[#1004] blobovnicza: Use TTL for blobovnicza tree cache ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-04-26 19:54:29 +03:00
Anton Nikiforov	112a7c690f	[#1103] node: Implement Get\Head requests for EC object ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-04-24 18:15:53 +03:00