14 commits

Author	SHA1	Message	Date
Anna Shaleva	4b7b2ef701	docs: take care of upload-bin documentation ... Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>	2024-12-17 10:18:34 +03:00
Ekaterina Pavlova	247ee831e5	cli: change block timestamp attribute of objects in upload-bin ... Close #3654 Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>	2024-12-13 16:35:14 +03:00
Ekaterina Pavlova	62615f8c7e	cli: update upload-bin to create index files during block uploading ... Close #3655 Close #3652 Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>	2024-12-12 12:08:49 +03:00
Ekaterina Pavlova	119ca27994	blockfetcher: fix block enqueue logic ... Previously, the `blockQueuer` routine, which enqueues blocks into `bQueue`, could be blocked on enqueing newer blocks if older blocks downloading is delayed by NeoFS. The `blocksCh` channel, acting as a queue ordered by download speed, conflicted with the BQueue requirement for strict sequential enqueuing (expecting an exact range of blocks), resulting in a deadlock that stalled the process. Before with default config settings: ``` 2024-11-27T17:12:19.348+0300 INFO persisted to disk {"blocks": 0, "keys": 116, "headerHeight": 0, "blockHeight": 0, "took": "15 .509083ms"} 2024-11-27T17:19:39.574+0300 INFO persisted to disk {"blocks": 16, "keys": 11107, "headerHeight": 216768, "blockHeight": 216768, "took": "62.762041ms"} ``` Average block persistence speed: 492.40 block/s Average blocks number for each persist log: 584.28 After: ``` 2024-11-27T17:29:03.362+0300 INFO persisted to disk {"blocks": 0, "keys": 116, "headerHeight": 0, "blockHeight": 0, "took": "19 .485084ms"} 2024-11-27T17:34:58.527+0300 INFO persisted to disk {"blocks": 16, "keys": 11109, "headerHeight": 216770, "blockHeight": 216769, "took": "52.43925ms"} ``` Average block persistence speed: 610.33 block/s Average blocks number for each persist log: 752.61 Close #3699 Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>	2024-11-29 13:37:50 +03:00
Ekaterina Pavlova	43609dd984	*: migrate to SEARCH with strict equality comparator ... Close #3670 Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>	2024-11-25 13:19:35 +03:00
Ekaterina Pavlova	58ed448f8d	cli: add debug mode to upload-bin command ... Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>	2024-11-15 17:18:44 +03:00
Ekaterina Pavlova	29e2d712f4	cli: add flag max-retries for upload-bin ... Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>	2024-11-06 11:18:41 +03:00
Ekaterina Pavlova	6a93f70728	cli: add flag for skipping block uploading in upload-bin ... Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>	2024-10-31 18:39:40 +03:00
Ekaterina Pavlova	7b3eeb9061	cli: fix usage of upload-bin ... Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>	2024-10-24 12:19:12 +03:00
Ekaterina Pavlova	36e855609d	cli: add workers and searchers flags to upload-bin command ... Number of workers to fetch, upload and search blocks concurrently is now configurable. Close #3641 Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>	2024-10-24 12:19:10 +03:00
Ekaterina Pavlova	c47d4e6c5b	BlockFetcher/BlockUploader: fix and add additional attributes ... Capitalize block objects and index file objects attributes. Add network magic attribute to container. Close #3631 Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>	2024-10-23 13:52:06 +03:00
Ekaterina Pavlova	6199240598	cli: add upload-bin ... This command is used for keeping container with blocks for blockfetcher updated. Close #3578 Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>	2024-10-17 11:56:07 +03:00
Ekaterina Pavlova	852dcb0f64	config, docs: adjust NeoFSBlockFetcher index file and time attributes ... Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>	2024-10-16 19:23:56 +03:00
Ekaterina Pavlova	0b31a29f39	services: add new service for fetching blocks from NeoFS ... Close #3496 Co-authored-by: Anna Shaleva <shaleva.ann@nspcc.ru> Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>	2024-09-11 08:50:31 +04:00