3048 commits

Author	SHA1	Message	Date
Dmitrii Stepanov	15d853ea22	[#947] controlSvc: Return tree evacuation stat ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-02-09 11:20:39 +03:00
Dmitrii Stepanov	b3f3505ada	[#947] cli: Allow to specify evacuation scope ... It may be required to evacuate only objects or only tree or all, so now it spossible to specify. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-02-09 11:20:38 +03:00
Dmitrii Stepanov	a6eb66bf9c	[#947] evacuate: Refactor evacuate parameters ... Drop methods to make it easier to extend. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-02-09 11:20:38 +03:00
Dmitrii Stepanov	8e2a0611f4	[#947] tree: Add method to list all trees ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-02-09 11:20:38 +03:00
Evgenii Stratonikov	edbe06e07e	[#956] policer/test: Reuse testPool helper ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-08 15:49:43 +00:00
Evgenii Stratonikov	cbfeb72466	[#956] policer: Remove WithMaxCapacity option ... We already provide the pool and this argument is used only for preallocation. No functional changes. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-08 15:49:43 +00:00
Dmitrii Stepanov	cfc5ce7853	[#964] metabase: Drop GC marks if object not found ... GC inhumes expired locks and tombstones on all the shards. So it could be GC mark without object. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-02-08 07:54:39 +00:00
Evgenii Stratonikov	c3fa902780	[#969] policer: Restrict the number of remembered errors ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-08 10:10:41 +03:00
Evgenii Stratonikov	6010dfdf3d	[#969] policer: Make error skip thread-safe ... Introduces in afd2ba9a66. Refs #914 Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-08 10:10:41 +03:00
Evgenii Stratonikov	a6c9a337cd	[#965] morph: Get rid of container.List invocations ... ContainersOf() is better in almost every aspect, besides creating a session when the containers number is between 1024 and 2048 (prefetch script does limited unwrapping). Making List() private helps to ensure it is no longer used and can be safely removed in future. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-07 08:56:27 +00:00
Dmitrii Stepanov	9ba48c582d	[#917] engine: Allow to detach shards ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-02-06 14:49:47 +03:00
Dmitrii Stepanov	4358d3c423	[#917] controlSvc: Add DetachShards handler ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-02-06 14:47:52 +03:00
Ekaterina Lebedeva	afd2ba9a66	[#110] Add check for repeated error log in policer ... processObject() returns 3 types of errors: container not found errors, could not get container error and placement vector building error. Every error will occur for all objects in container simultaneously, so we can log each error once and safely ignore the rest. Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2024-02-06 00:56:41 +03:00
Evgenii Stratonikov	602ee11123	[#934] containersvc: Marhal public key in short format for APE ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-02 17:51:38 +00:00
Airat Arifullin	c1a5b831b6	[#955] chainbase: Fix rule chain unmarshalling ... * Use correct way DecodeBytes instead unmarshalling by json. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-02-02 20:28:04 +03:00
Anton Nikiforov	9916598dfb	[#922] control: Extend api with ListOverrideDefinedTargets ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-02-02 12:09:51 +00:00
Anton Nikiforov	95e15f499f	[#922] Update files generated by protoc ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-02-02 12:09:51 +00:00
Anton Nikiforov	2cb04379a4	[#922] go.mod: Update APE ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-02-02 12:09:51 +00:00
Airat Arifullin	a5446bc17d	[#952] object: Pass namespace within context in ACL service ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-02-02 14:48:11 +03:00
Anton Nikiforov	d0eadf7ea2	[#799] engine: Skip put when object removed from shard ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-02-01 17:49:22 +00:00
Anton Nikiforov	6534252c22	[#799] policer: Refactor method processNodes ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-02-01 17:49:22 +00:00
Airat Arifullin	5be2af881a	[#934] container: Make container APE middleware read namespaces ... * Those methods that can access already existing containers and thus can get container properties should read namespace from Zone property. If Zone is not set, take a namespace for root. * Otherwise, define namespaces by owner ID via frostfs-id contract. * Improve unit-tests, consider more cases. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-02-01 17:38:24 +00:00
Airat Arifullin	4352bd0e8e	[#934] ape: Transform empty namespace within chainbase ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-02-01 17:38:24 +00:00
Anton Nikiforov	483a67b170	[#937] ape: Validate chain resource name ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-01-31 11:34:35 +03:00
Anton Nikiforov	e3573de6db	[#930] gc: Stop internal activity by context ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-01-31 08:30:34 +00:00
Anton Nikiforov	c441296592	[#930] policer: Release task pool when context cancelled ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-01-31 08:30:34 +00:00
Dmitrii Stepanov	675eec91f3	[#938] shard: Update only changed counters ... If metric value hasn't changed, but we update metric, then non existed metric will apear with zero value. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-30 12:37:48 +03:00
Dmitrii Stepanov	c681354afd	[#938] engine: Fix container count removal ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-30 12:37:48 +03:00
Evgenii Stratonikov	df055fead5	[#931] morph: Provide batch size for container listing explicitly ... Besides VM stack item limit we also have restrictions on the size of JSON for a stackitem, which is 128k. This limit is much harder to calculate, because JSON representation includes type and the encoding is different for different items. Thus is makes no sense to invent our own default, so use the one provided by neo-go. But for container listing we know exactly what we process, so use big enough value, which is tested. Introduced in be8607a1f6. Refs #902 Refs https://github.com/nspcc-dev/neo-go/blob/v0.105.0/pkg/vm/stackitem/json.go#L353 Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-01-29 14:00:11 +00:00
Evgenii Stratonikov	6e2cc32768	[#681] objsvc: Validate session token owner for local sessions ... Previously, the check was in place only when session token was missing. Format validator checks are applied only to fully-prepared object, so this lead to the following situation: 1. Object is put locally with malformed token, because there are no checks. 2. Object cannot be replicated, because the token is malformed. This is now fixed and token check is done before any payload receival. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-01-26 08:52:29 +00:00
Anton Nikiforov	b6fc3321c5	[#876] Fix linters ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-01-25 20:26:13 +03:00
Airat Arifullin	1fe7736d92	[#925] morph: Introduce switch rpc guarded rpc actor ... * Introduce switch rpc guarded rpc actor in morph client to prevent using invalidated rpc actor when RPC switch happens. * Initialize NewContractStorage with SwitchRPCGuardedActor. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-01-25 13:24:13 +00:00
Dmitrii Stepanov	5ed330e436	[#927] metabase: Delete GC marks ... `key` is changed inside `db.get`, so encode address again after get. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-24 18:51:16 +03:00
Dmitrii Stepanov	931a5e9aaf	[#918] engine: Move shard to degraded mode if metabase open failed ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-23 11:16:40 +03:00
Airat Arifullin	f2f3294fc3	[#919] ape: Improve error messages in ape service ... * Wrap all APE middleware errors in apeErr that makes errors more explicit with status AccessDenied. * Use denyingRuleErr for denying status from chain router. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-01-23 08:11:24 +00:00
Dmitrii Stepanov	f526f49995	[#874] engine: Check object existance concurrently ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-23 09:28:29 +03:00
Dmitrii Stepanov	f5160b27fc	[#920] tests: Fix data races ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-19 14:06:05 +03:00
Airat Arifullin	be33070550	[#910] control: Take empty namespace for 'root' ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-01-19 06:22:10 +00:00
Dmitrii Stepanov	63d3ed1ad8	[#904] tests: Close test engine after test ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-17 19:04:39 +03:00
Dmitrii Stepanov	57171907e3	[#904] metabase: Return if object was actuall inserted ... This requires to count metrics properly. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-17 19:04:39 +03:00
Dmitrii Stepanov	c1a80235db	[#904] metabase: Log Inhume operation ... It will be very useful for troubleshooting. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-17 18:42:52 +03:00
Airat Arifullin	96b020626f	[#915] ape: Fix method name in getStreamBasicChecker ... * Replace incorrect MethodGetContainer by MethodGetObject constant. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-01-16 23:52:37 +03:00
Airat Arifullin	c8baf76fae	[#872] object: Introduce APE middlewar for object service ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-01-12 18:41:35 +03:00
Dmitrii Stepanov	52ffa9f164	[#891] getSvc: Refactor Get service V2 creation ... Use arguments for mandatory fields. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-12 13:35:38 +03:00
Dmitrii Stepanov	394f086fe2	[#891] getSvc: Fix get range hash implementation ... Get range can perform GET request, so this request must be done from container node to not to get access denied error. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-12 13:35:38 +03:00
Evgenii Stratonikov	be8607a1f6	[#902] morph: Avoid creating session in TestInvokeIterator ... When the number of items to iterate over is less than 2048, there is no need to create a session and consume resources. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-01-12 10:25:56 +00:00
Dmitrii Stepanov	a2ab373a0a	[#895] metabase: Do not delete GC mark for virtual objects ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-11 12:32:09 +00:00
Dmitrii Stepanov	7166e77c2b	[#895] test: Add logger to test shard ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-11 12:32:09 +00:00
Dmitrii Stepanov	47dcfa20f3	[#895] test: Use t.Cleanup only for external resources ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-11 12:32:09 +00:00
Dmitrii Stepanov	836818fb75	[#895] test: Use zaptest logger ... Previous implementation mixes logs from different tests, now logs are separeted by test. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-11 12:32:09 +00:00
Dmitrii Stepanov	f1b2b8bffa	[#895] test: Fix NewLogger arguments list ... `debug` is always true. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-11 12:32:09 +00:00
Airat Arifullin	a8e52ef7aa	[#898] control: Fix codes for returning APE errors ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-01-11 12:31:31 +00:00
Airat Arifullin	5d982976fd	[#898] ape: Fix bug with creating type bucket in chainbase ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-01-11 12:31:31 +00:00
Denis Kirillov	c19396d203	[#885] control: Make chain id bytes in grpc ... Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>	2024-01-11 07:24:22 +00:00
Dmitrii Stepanov	5c0a736a25	[#899] containerSvc: Fix invalid session token type ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-10 18:37:54 +03:00
Alexander Chuprov	79bebe4a68	[#884] cli: Fix error message for undefined endpoint ... Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>	2024-01-10 10:02:16 +00:00
Dmitrii Stepanov	4b8b4da681	[#864] engine: Drop container count metric if container removed ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-10 10:45:32 +03:00
Dmitrii Stepanov	d75e7e9a21	[#864] engine: Drop container size metric if container deleted ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-10 10:44:54 +03:00
Dmitrii Stepanov	dfd62ca6b1	[#864] metabase: Refactor delete/inhume ... Available -> Logic, Raw -> Phy for delete/inhume results. Use single counter instead of vectors. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-09 09:59:42 +03:00
Evgenii Stratonikov	225fe2d4d5	[#894] blobovniczatree: Speedup rebuild test ... Down from 3s to 300ms. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-12-29 16:28:54 +00:00
Evgenii Stratonikov	530249e3bd	[#893] go.mod: Update neo-go ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-12-29 15:03:47 +00:00
Dmitrii Stepanov	581887148a	[#569] cli: Add control shards writecache seal command ... It does the same as `control shards flush-writecache --seal`, but has better name. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-29 16:05:37 +03:00
Dmitrii Stepanov	7a9db5bcdd	[#569] writecache: Do not wait modeMtx if mode changes ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-29 16:05:37 +03:00
Dmitrii Stepanov	32c282ca10	[#569] writecache: Refactor flush ... Make single RUnlock call instead of two. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-29 16:05:37 +03:00
Dmitrii Stepanov	0cb0fc1735	[#569] writecache: Allow to seal writecache after flush ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-29 16:05:37 +03:00
Dmitrii Stepanov	b118734909	[#890] getsvc: Log node PK ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-29 14:24:13 +03:00
Dmitrii Stepanov	764f70634d	[#881] containerSvc: Add APE validation ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-27 11:05:34 +03:00
Dmitrii Stepanov	8180a0664f	[#887] node: Drop badger writecache implementation ... Badger implementation isn't tested and works not well, but requires human resources to maintain. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-22 13:00:54 +03:00
Anton Nikiforov	eab981bf1a	[#876] Fix linter error ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2023-12-21 18:21:37 +03:00
Anton Nikiforov	d9cbb16bd3	[#866] Use TTL for blobovnicza tree cache ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2023-12-19 16:36:28 +00:00
Airat Arifullin	7d7cf05575	[#851] ape: Initialize and use policy contract interface ... * Replace inmemory policy contract mock by initialized policy contract interface. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2023-12-18 11:06:35 +00:00
Airat Arifullin	bdd43f6211	[#869] object: Pass just CID to chain router ... * Do not convert CID from request to native-schema resource format - this step is unneccessary for APE. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2023-12-14 11:01:20 +00:00
Ekaterina Lebedeva	2d4c0a0f4a	[#552] Add systemd notifications to ir service ... Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2023-12-13 17:51:41 +03:00
Ekaterina Lebedeva	eca7ac9f0d	[#552] Add sdnotify package ... To avoid using third-party dependencies. Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2023-12-13 17:49:26 +03:00
Ekaterina Lebedeva	05f8f49289	[#552] gofumpt changes ... Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2023-12-13 15:02:25 +03:00
Dmitrii Stepanov	7eb46404a1	[#863] blobovnicza: Fix counters ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-13 13:34:29 +03:00
Dmitrii Stepanov	11add38e87	[#857] golangci: Add protogetter linter ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-12 16:27:02 +03:00
Dmitrii Stepanov	94ffe8bb45	[#857] golangci: Add testifylint linter ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-12 16:27:02 +03:00
Dmitrii Stepanov	5d7833c89b	[#857] golangci: Add perfsprint linter ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-12 16:27:02 +03:00
Dmitrii Stepanov	3b7c0362a8	[#861] shard: Fix Delete object ... It is possible that object doesn't exist in metabase. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-12 14:25:40 +03:00
Dmitrii Stepanov	681b2c5fd4	[#825] policer: Do not drop required linking objects ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-12 11:04:03 +00:00
Dmitrii Stepanov	a3ef7b58b4	[#755] innerring: Check container owner namespace ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-12 12:36:34 +03:00
Dmitrii Stepanov	1cd2bfe51a	[#755] morph: Drop FrostFSID contract usage ... Unused. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-12 12:36:34 +03:00
Anton Nikiforov	70e0c1e082	[#841] ir: Execute netmap.addPeerIR only for state online ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2023-12-11 13:14:41 +00:00
aarifullin	0f45e3d344	[#804] ape: Implement boltdb storage for local overrides ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2023-12-07 19:08:41 +03:00
aarifullin	e361e017f3	[#842] control: Pass target instead resource name ... * Update policy-engine package version in go.mod, go.sum. * Refactor CheckIfRequestPermitted: pass container target instead container ID. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2023-12-07 14:21:55 +00:00
aarifullin	39060382a1	[#842] control: Recieve target in gRPC methods for APE managing ... * Introduce Target type and pass it to all gRPC methods for APE chain managing instead CID. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2023-12-07 14:21:55 +00:00
Dmitrii Stepanov	db49ad16cc	[#826] blobovniczatree: Do not create DB's on init ... Blobovniczas will be created on write requests. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:37:33 +03:00
Dmitrii Stepanov	ad0697adc4	[#661] blobovnicza: Compute size with record size ... To get more accurate size of blobovnicza use record size (lenght of key + lenght of data). Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:37:33 +03:00
Dmitrii Stepanov	e54dc3dc7c	[#698] blobovnicza: Store counter values ... Blobovnicza initialization take a long time because of bucket Stat() call. So now blobovnicza stores counters in META bucket. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:37:33 +03:00
Dmitrii Stepanov	5e8c08da3e	[#661] blobstore: Add address to error logs ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:37:33 +03:00
Dmitrii Stepanov	8911656b1a	[#661] metrcis: Add rebuild percent metric ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:37:33 +03:00
Dmitrii Stepanov	2407e5f5ff	[#661] blobovniczatree: Do not sort DB's and indicies ... Put stores object to next active DB, so there is no need to sort DBs. In addition, it adds unnecessary DB openings. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:37:33 +03:00
Dmitrii Stepanov	c6a739e746	[#661] blobovniczatree: Make Rebuild concurrent for objects ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:37:33 +03:00
Dmitrii Stepanov	f1c7905263	[#661] blobovniczatree: Make Rebuild concurrent ... Different DBs can be rebuild concurrently. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:37:33 +03:00
Dmitrii Stepanov	d4d905ecc6	[#661] metrics: Add blobovniczatree rebuild metrics ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:37:33 +03:00
Dmitrii Stepanov	b2769ca3de	[#661] blobovniczatree: Make Rebuild failover safe ... Now move info stores in blobovnicza, so in case of failover rebuild completes previous operation first. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:37:33 +03:00
Dmitrii Stepanov	da4fee2d0b	[#698] blobovniczatree: Init blobovniczas concurrently ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:37:33 +03:00
Dmitrii Stepanov	422226da18	[#661] blobovniczatree: Add Rebuild implementation ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:37:32 +03:00
Dmitrii Stepanov	a531eaf8bc	[#661] blobstor: Add Rebuild implementation ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:37:32 +03:00
Dmitrii Stepanov	c1667a11d2	[#661] blobovniczatree: Allow to change depth or width ... Now it is possible to change depth or with of blobovniczatree. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:37:32 +03:00
Dmitrii Stepanov	484eb59893	[#661] blobovniczatree: Use .db extension for db files ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:37:32 +03:00
Dmitrii Stepanov	44552a849b	[#661] shard: Add blobstor rebuilder ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:37:32 +03:00
Dmitrii Stepanov	a478050639	[#838] metabase: Resolve funlen linter ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-06 15:44:21 +03:00
Dmitrii Stepanov	d30ab5f29e	[#838] metabase: Count user objects ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-06 15:44:21 +03:00
Dmitrii Stepanov	f314da4af3	[#838] metabase: Add user object type counter ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-06 15:44:21 +03:00
Dmitrii Stepanov	29550fe600	[#838] shard: Refactor updateMetrics method ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-06 15:44:21 +03:00
Alexander Chuprov	b2c63e57ba	[#651] engine/test: Speedup StorageEngine_Inhume ... Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>	2023-11-30 13:19:43 +00:00
Alexander Chuprov	445ebcc0e7	[#651] shard/test: Speedup Shard_Delete ... Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>	2023-11-30 13:19:43 +00:00
Alexander Chuprov	2302e5d342	[#651] shard/test: Refactor Shard_Delete ... Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>	2023-11-30 13:19:43 +00:00
Dmitrii Stepanov	26e4f7005c	[#741] treesvc: Refactor tree sync ... Fix linter issues. Add error logging. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-30 12:45:02 +00:00
Dmitrii Stepanov	b21be1abdd	[#741] treesvc: Do not update sync height if some node is unavailable ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-30 12:45:02 +00:00
Dmitrii Stepanov	b215817e14	[#741] treesvc: Remove unused height variables ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-30 12:45:02 +00:00
Dmitrii Stepanov	c516c7c5f4	[#821] node: Pass user.ID by value ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-23 10:21:07 +03:00
Dmitrii Stepanov	07390ad4e3	[#715] node: Unify config parameter names ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-22 17:13:50 +03:00
Dmitrii Stepanov	8d18fa159e	[#667] writecache: Fix flush test ... Allow to disable background flusher for testing purposes. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-17 17:45:43 +03:00
aarifullin	02454df14a	[#809] client: Refactor PrmInit, PrmDial usage ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2023-11-17 13:37:03 +00:00
Dmitrii Stepanov	76ff26039c	[#96] node: Drop neo-go's slices package ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-17 13:24:04 +03:00
aarifullin	47286ebf32	[#805] pilorama: Fix TreeDrop ... * If treeID is empty then deleting buckets for cursor may get invalidated. So, buckets should be gathered before deleting. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2023-11-17 10:21:35 +00:00
Dmitrii Stepanov	29fe8c41f3	[#655] storage: Drop ErrorHandler ... The only one usage was for logging. Now logging performed by storage anyway. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-16 17:27:38 +03:00
Dmitrii Stepanov	137e987a4e	[#655] storage: Drop LazyHandler ... LazyHandler is implemented and used incorrectly. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-16 17:27:38 +03:00
aarifullin	4d5be5ccb5	[#811] ape: Update policy-engine module version and rebase ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2023-11-16 11:31:37 +03:00
Evgenii Stratonikov	a952a406a2	[#506] container: Use uint64 for epoch type ... It is `uint64` in netmap source interfaces and other code. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-11-14 19:00:11 +00:00
Evgenii Stratonikov	f04806ccd3	[#506] container: Use user.ID in DeletionInfo response ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-11-14 19:00:11 +00:00
Dmitrii Stepanov	8088063195	[#787] netmap: Refactor NewEpoch method ... Split for user and control methods. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-13 17:22:31 +03:00
Dmitrii Stepanov	c8a62ffedd	[#787] morph: Calculate VUB and nonce when hash is nil ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-13 17:13:03 +03:00
Dmitrii Stepanov	2393d13e4d	[#787] morph: Return VUB for IR service calls ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-13 17:13:03 +03:00
Dmitrii Stepanov	518f3baf41	[#787] morph: Return VUB from Invoke method ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-13 17:13:03 +03:00
Dmitrii Stepanov	bdfa523487	[#787] proto: Add VUB field for IR service ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-13 17:13:03 +03:00
Dmitrii Stepanov	0f75e48138	[#796] policer: Fix tombstone objects replication ... Tombstone objects must be replicated to all container nodes. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-09 13:39:33 +03:00
Dmitrii Stepanov	7cdae4f660	[#792] proto: Regenerate with fixed version ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-09 10:09:13 +00:00
Dmitrii Stepanov	9133b4389e	[#788] objectsvc: Fix formatting (gofumpt) ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-09 10:27:32 +03:00
Dmitrii Stepanov	1b22801eed	[#788] engine: Fix flaky tests ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-09 10:25:46 +03:00
aarifullin	3534d6d05b	[#794] objectsvc: Return accidentally removed acl checks for Head ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2023-11-08 17:13:58 +03:00
Airat Arifullin	66848d3288	[#770] cli: Add methods to work with APE rules via control svc ... * Add methods to frostfs-cli * Implement rpc in control service Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2023-11-08 13:34:03 +00:00
Airat Arifullin	8e11ef46b8	[#770] object: Introduce ape chain checker for object svc ... * Introduce Request type converted from RequestInfo type to implement policy-engine's Request interface * Implement basic ape checker to check if a request is permitted to be performed * Make put handlers use APE checker instead EACL Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2023-11-08 13:34:03 +00:00
Airat Arifullin	5ec73fe8a0	[#770] node: Introduce ape chain source ... * Provide methods to access rule chains with access policy engine (APE) chain source * Initialize apeChainSource within object service initialization * Share apeChainSource with control service * Implement dummy apeChainSource instance based on in-memory implementation Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2023-11-08 13:34:03 +00:00
Airat Arifullin	3a2c319b87	[#770] control: Generate gRPC methods to manipulate APE chains ... * Define new types and gRPC methods to manipulate APE chains in control service. * Stub gRPC handlers for the generated methods. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2023-11-08 13:34:03 +00:00
Dmitrii Stepanov	70ab1ebd54	[#763] metrics: Add container_objects_total metric ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-08 12:30:57 +03:00
Dmitrii Stepanov	9c98fa6152	[#763] metabase: Add container objects counter ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-08 12:30:57 +03:00
Dmitrii Stepanov	226e84d782	[#684] node: Add skipped objects count to evacuation result ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-07 12:17:11 +00:00
Dmitrii Stepanov	523fb3ca51	[#684] proto: Add skipped count to evacuation status response ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-07 12:17:11 +00:00
aarifullin	74c91eeef5	[#777] client: Refactor PrmContainerList, PrmObjectSearch usage ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2023-11-06 06:50:11 +00:00
Dmitrii Stepanov	20d6132f31	[#531] signSvc: Add SetMarshaledData method call ... To reduce memory allocations add `SetMarshaledData` method call to return already marshalled data in next `StableMarshal` calls. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-02 17:34:33 +03:00
Anton Nikiforov	a26483fc30	[#749] morph: Fix panic when closing morph client ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2023-11-01 10:48:10 +00:00
Dmitrii Stepanov	c80b46fad3	[#754] blobstor: Estimate compressability ... Now it is possible to enable compressability estimation. If data is likely uncompressable, it should reduce CPU time and memory. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-01 11:24:32 +03:00
Dmitrii Stepanov	05b508f79a	[#772] proto: Fix file ending ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-10-31 17:03:04 +03:00
Dmitrii Stepanov	79088baa06	[#772] node: Apply gofumpt ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-10-31 17:03:03 +03:00
Evgenii Stratonikov	00aa6d9749	[#633] shard/test: Fix TestCounters() ... Introduced in 362f24953a, forgotten to be changed because test generator didn't provide payload size. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-10-31 12:53:28 +00:00
Evgenii Stratonikov	b8f79f4227	[#633] shard/test: Fix race conditions in TestCounters() ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-10-31 12:53:28 +00:00