Consider using tree service #3

New issue

Closed

opened 2023-10-04 13:04:14 +00:00 by dkirillov · 0 comments

dkirillov commented

2023-10-04 13:04:14 +00:00

Member

Currently to find any uploaded object or resume upload we do search. As the number of objects increases, it becomes slower. So we can consider using tree service almost the same way as it's done in s3-gw.

At least we can add to the tree mapping of object name to its OID for fully uploaded objects.
Besides we can also avoid searching phy objects if we will temporarily save intermediate objects oid during long upload (after each object release)

Currently to find any uploaded object or resume upload we do [search](https://git.frostfs.info/TrueCloudLab/distribution/src/commit/5c26d0c9c17e4149415a020579478cd0b97e438b/registry/storage/driver/frostfs/frostfs.go#L853). As the number of objects increases, it becomes slower. So we can consider using tree service almost the same way as it's done in [s3-gw](https://git.frostfs.info/TrueCloudLab/frostfs-s3-gw/src/commit/c4af1dc4ad1ebb494d6dc64278bb3087202db4d5/pkg/service/tree/tree.go). At least we can add to the tree mapping of object name to its OID for fully uploaded objects. Besides we can also avoid searching [phy objects](https://git.frostfs.info/TrueCloudLab/distribution/src/commit/5c26d0c9c17e4149415a020579478cd0b97e438b/registry/storage/driver/frostfs/frostfs.go#L761) if we will temporarily save intermediate objects oid during long upload (after each [object release](https://git.frostfs.info/TrueCloudLab/distribution/src/commit/5c26d0c9c17e4149415a020579478cd0b97e438b/registry/storage/driver/frostfs/frostfs.go#L961))