Implement asyncrhonous expiration rule executor #4

New issue

Closed

opened 2024-06-27 12:19:08 +00:00 by alexvanin · 0 comments

alexvanin commented

2024-06-27 12:19:08 +00:00

Owner

When lifecycle structure is prepared (see #3), all its rules must be sent into queue of expiration rule executor.

Expiration rule executor contains several workers which perform expiration action: add delete-marker, remove object, abort multipart-upload. Worker must receive container, credentials and filter to find all objects that applied to the rule.

Note! Here we can introduce some optimizations to avoid processing irrelevant rules (e.g. if container was created a day ago, and lifecycle rule asks to delete objects after 30 days).

Requirements:

Reuse bearer token that has been used during job creation and tree traversal
Provide a way to clean queue immediately when new epoch appears

When lifecycle structure is prepared (see https://git.frostfs.info/TrueCloudLab/frostfs-s3-lifecycler/issues/3), all its rules must be sent into **queue** of expiration rule executor. Expiration rule executor contains **several workers** which perform expiration action: add delete-marker, remove object, abort multipart-upload. Worker must receive container, credentials and filter to find all objects that applied to the rule. > Note! Here we can introduce some optimizations to avoid processing irrelevant rules (e.g. if container was created a day ago, and lifecycle rule asks to delete objects after 30 days). Requirements: * Reuse bearer token that has been used during job creation and tree traversal * Provide a way **to clean queue immediately** when new epoch appears