Commit graph

1072 commits

Author SHA1 Message Date
b445f7bbf9 [#125] api/auth: DisableURIPathEscaping for presign
Don't use escaping when presign url.
Escape manually before.

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-02 14:16:19 +00:00
868edfdb31 [#78] Add test of bucket removal with object not found error
All checks were successful
DCO
Builds (1.20)
Tests (1.19)
Tests (1.20)
Builds (1.19)
Lint
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-06-01 16:13:28 +03:00
34bbbcf1ed [#78] Do not show objects missing from object service
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-06-01 16:13:28 +03:00
fae03c2b50 [#78] Process 'not found' error when object exists in tree
When object exists in tree but missing in storage, we can't remove
bucket. While storage node does not sync tree service and object
service, the only way to delete such broken bucket is to ignore
'object not found' error, clear cache and do not include missing
objects in the listing result.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-06-01 15:31:22 +03:00
81e860481d [#122] Fix linter warnings
All checks were successful
Builds (1.19)
DCO
Builds (1.20)
Lint
Tests (1.19)
Tests (1.20)
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-05-31 19:43:03 +03:00
e97fea30ea [#122] Update actions for forgejo ci
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-05-31 19:43:00 +03:00
cfc94e39ef [#122] Move files from .github to .forgejo
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-05-31 19:42:58 +03:00
ce9294685c [#101] Update docs for frostfs.set_copies_number
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-05-25 09:11:09 +00:00
a0f0d792b8 [#78] layer: Clean up already removed object from tree
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2023-05-24 17:42:46 +03:00
43e336e155 [#118] go.mod: Update min go version to 1.19
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-05-23 17:32:05 +03:00
9f186d9aba [#104] app: Reload copies numbers on SIGHUP
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2023-05-23 13:19:58 +03:00
11f30a037b [#114] tree: Fix retry tests
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-05-23 11:35:44 +03:00
136a186c14 [#114] tree: Don't ignore unhealthy endpoints
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-05-19 17:22:39 +03:00
24390fdec8 [#110] tree: Add more logs for switching tree endpoints
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-05-19 17:20:38 +03:00
1fdbfb0dab [#110] tree: Update errors to switch endpoint
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-05-19 17:20:34 +03:00
1406f57bba [#1] Update comment lines
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-05-19 07:30:04 +00:00
3fbad97ae4 [#1] Rename files with mentions of previous project
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-05-19 07:30:04 +00:00
fa5889a0f5 [#94] Update prometheus to v1.15.0
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2023-05-18 17:36:32 +03:00
e24bc3f2ce [#101] app: Refactor the default copies number setting
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2023-05-17 11:36:28 +03:00
bf47978cfe [#105] Update SDK to fix impersonated token
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-05-05 18:11:33 +03:00
Denis Kirillov
b366e75366 [#81] Use impersonate bearer token
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-05-03 17:22:52 +03:00
e487ee5b7d [#70] Add arrays of copies numbers for location constraints
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2023-05-03 13:48:26 +03:00
69d8779daf [#74] tree: Simplify retry
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-04-26 16:39:57 +03:00
9da77667f3 [#74] Add round tree retry
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-04-26 16:39:57 +03:00
f200dd310e [#74] Update docs
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-04-26 16:39:57 +03:00
ee8cce662b [#74] service/tree: Add logger
Log error instead of failing when multiple unversioned nodes are found

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-04-26 16:39:57 +03:00
ca8791a5fd [#74] Support multiple tree endpoints
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-04-26 16:39:57 +03:00
2ab6f004f1 Makefile: Add syncTree folder clean up to 'make clean'
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2023-04-21 17:09:53 +03:00
3da2d40fa8 syncTree: Update file filter to 'frostfs'
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2023-04-21 17:08:37 +03:00
1cbc7f323f [#80] metrics: Use map for constant labels
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-04-20 11:14:52 +03:00
c154f934e4 [#80] Add type to metrics description
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-04-20 11:14:52 +03:00
644524e8a5 [#80] metrics: Make global description unexported
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-04-20 11:14:52 +03:00
ee6118c3d7 [#80] Update CHANGELOG.md
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-04-20 11:14:52 +03:00
9e72fe1662 [#80] Refactor metrics, support dump descriptions
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-04-20 11:14:52 +03:00
70ec5a0a5b [#83] Don't create extra delete marker
We shouldn't create delete marker if:
1. object doesn't exist at all
2. last version is already a delete marker

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-04-19 17:56:11 +03:00
4a8c382491 [#91] Update values for health metric
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-04-18 10:25:11 +03:00
a5c89b78bc [#79] Return X-Owner-Id in head-bucket response
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2023-04-17 10:12:25 +03:00
fb99b26209 [#57] api: Clean up List and Name caches when object is missing in Tree service 2023-04-04 17:15:04 +03:00
01afa1cae4 [#75] Make grpc tree client implementation internal
Since we have pkg 'internal/frostfs/services/tree' that is downloading
during build we cannot export any package that is depended on it.

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-04-03 14:04:35 +03:00
6c68e21777 [#69] Update SDK to fix handle request canceling
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-03-30 15:56:57 +03:00
a025f2e9c5 [#59] tree: Make interface for tree service client
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-03-29 09:34:10 +03:00
bd3164c57f [#68] Fix pre-commit issues
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-03-24 16:22:06 +03:00
fb90c0f52c [#65] Enable pre-commit
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-03-24 07:28:04 +00:00
f2f90e260e [#66] Add Issue Template
Add bug report and feature request templates

Signed-off-by: Liza <e.chichindaeva@yadro.com>
2023-03-23 12:25:16 +03:00
a0937126cb [#64] Support new system attributes
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-03-22 11:46:58 +03:00
655889a1a2 [#60] Update docs
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-03-22 11:22:11 +03:00
ef556bd8ac [#60] Use session token to set eACL during Complete Multipart Upload
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-03-22 11:22:11 +03:00
5104683f68 [#60] Refactor start of periodic XML writer
Reduce code duplication for error handling

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-03-22 11:02:16 +03:00
8151753eeb [#60] Use periodic white space XML writer in Complete Multipart Upload
This mechanism is used by Amazon S3 to keep client's
connection alive while object is being constructed from
the upload parts.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-03-20 13:34:22 +03:00
2282c32822 [#60] Add *NoHeader functions
Such functions should be used together with periodic white space
XML writer.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-03-20 12:07:36 +03:00