Release v0.1.2 #24

Merged
dkirillov merged 1 commit from dkirillov/frostfs-s3-lifecycler:release/v0.1.2 into master 2024-09-25 11:11:59 +00:00
3 changed files with 157 additions and 1 deletions

View file

@ -3,3 +3,27 @@
This document outlines major changes between releases.
## [Unreleased]
## [0.1.2] - 2024-09-25
### Added
- More debug logs (#20)
## [0.1.1] - 2024-09-17
### Added
- Allow to configure several sources (#16)
### Fixed
- Don't create redundant delete markers (#17)
- Don't user btoken for separate lifecycle container (#19)
## [0.1.0] - 2024-07-26
### Added
- Basic lifecycle logic (#1, #2, #3, #4)
[0.1.0]: https://git.frostfs.info/TrueCloudLab/frostfs-s3-lifecycler/compare/27189a38bb...v0.1.0
[0.1.1]: https://git.frostfs.info/TrueCloudLab/frostfs-s3-lifecycler/compare/v0.1.0...v0.1.1
[0.1.2]: https://git.frostfs.info/TrueCloudLab/frostfs-s3-lifecycler/compare/v0.1.1...v0.1.2
[Unreleased]: https://git.frostfs.info/TrueCloudLab/frostfs-s3-lifecycler/compare/v0.1.2...master

132
README.md
View file

@ -1 +1,133 @@
# FrostFS S3 Lifecycler
The purpose of this service is to provide lifecycle management of objects
(https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-lifecycle-mgmt.html).
This service works with objects and lifecycle configurations uploaded to FrostFS mainly by using
[s3-gw](https://git.frostfs.info/TrueCloudLab/frostfs-s3-gw).
## Build and run
To build service use the following command:
```shell
$ make
```
To run service use the following command:
```shell
$ ./bin/frostfs-s3-lifecycler --config config.yaml
```
Minimal config example to run with [dev-env](https://git.frostfs.info/TrueCloudLab/frostfs-dev-env) can be:

typo dev-env

typo dev-env
```yaml
wallet:
path: ./frostfs-dev-env/wallets/wallet.json
address: NbUgTSFvPmsRxmGeWpuuGeJUoRoi6PErcM
passphrase: ""
morph:
rpc_endpoint:
- address: ws://morph-chain.frostfs.devenv:30333/ws
credential:
use: wallets
source:
wallets:
- path: ./frostfs-dev-env/wallets/wallet.json
address: NbUgTSFvPmsRxmGeWpuuGeJUoRoi6PErcM
passphrase: ""
frostfs:
peers:
0:
priority: 1
weight: 1
address: s01.frostfs.devenv:8080
```
More detailed description of each parameter can be found [here](docs/configuration.md).
## Description
Once running service listens new epoch notification from FrostFS network.
On each epoch service lists all users from `frostfsid`
[contract](https://git.frostfs.info/TrueCloudLab/frostfs-contract/src/branch/master/frostfsid) and check each
user bucket lifecycle configuration to validate if objects or multipart uploads should be deleted according to this
configuration.
> **NOTE**
>
> Lifecycler handles only expiration actions (transition isn't supported).
If object should be deleted or multipart aborted lifecycler service perform delete action. It must have user credential
for this operation, so service must be provided with private key of each user. It can be done by specifying wallets
in config:
```yaml
credential:
use: wallets
source:
wallets:
- path: ./frostfs-dev-env/wallets/wallet.json
address: NbUgTSFvPmsRxmGeWpuuGeJUoRoi6PErcM
passphrase: ""
```
Currently only wallet source is supported.
### S3
To apply lifecycle configuration to bucket we must invoke
[PutBucketLifecycleConfiguration](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutBucketLifecycleConfiguration.html)
method on [s3-gw](https://git.frostfs.info/TrueCloudLab/frostfs-s3-gw). It can be done using
[AWS CLI](https://docs.aws.amazon.com/cli/latest/reference/s3api/put-bucket-lifecycle-configuration.html).
Lifecycle configuration example:
```json
{
"Rules": [
{
"Status": "Enabled",
"ID": "Abort 1 day old multiparts",
"AbortIncompleteMultipartUpload": {
"DaysAfterInitiation": 1
}
},
{
"Status": "Enabled",
"ID": "Expire objects with prefix after specific date",
"Expiration": {
"Date": "2024-07-31T06:32:00Z"
},
"Filter": {
"Prefix": "prefix"
}
},
{
"Status": "Enabled",
"ID": "Expire objects by tags after specific date",
"Expiration": {
"Date": "2024-07-31T06:32:00Z"
},
"Filter": {
"And": {
"Prefix": "tags",
"Tags": [{
"Key":"tag-key",
"Value":"tag-val"
}]
}
}
}
]
}
```
Details can be found in AWS documentaion
* https://docs.aws.amazon.com/AmazonS3/latest/userguide/intro-lifecycle-rules.html
* https://docs.aws.amazon.com/AmazonS3/latest/userguide/lifecycle-configuration-examples.html

View file

@ -1 +1 @@
v0.1.0
v0.1.2