be03c5178f
[ #143 ] Fix NoSuchKey error on get/head
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-30 12:04:52 +03:00
9df8695463
[ #143 ] Fix transformToS3Error function
...
Unwrap error before checking for s3 error
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-21 17:16:40 +03:00
614d703726
[ #106 ] Add chunk uploading
...
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2023-06-21 10:23:57 +03:00
23593eee3d
[ #111 ] Use request scope logger
...
Store child zap logger with request scope fields into context.
Request scoped fields: request_id, api/method, bucket, object
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-19 13:54:51 +03:00
84358f6742
[ #135 ] authmate: Support CRDT GSet for credentials
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-16 09:42:49 +03:00
7a380fa46c
[ #135 ] frostfs: Add SEARCH operation
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-16 09:42:41 +03:00
462589fc0c
[ #103 ] Return 504 http code on timeout errors
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-14 09:16:21 +03:00
9dabaf6ecd
[ #133 ] Use tree pool from SDK
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-09 09:34:04 +03:00
4e1fd9589b
[ #84 ] add tracing support
...
Signed-off-by: Pavel Pogodaev <p.pogodaev@yadro.com>
2023-06-06 10:25:27 +03:00
4a6e3a19ce
[ #125 ] Handle negative Content-Length on put
...
Add computing actual object size during calculating hash on put.
Use this actual value to save in tree and cache
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-02 14:16:19 +00:00
b445f7bbf9
[ #125 ] api/auth: DisableURIPathEscaping for presign
...
Don't use escaping when presign url.
Escape manually before.
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-02 14:16:19 +00:00
868edfdb31
[ #78 ] Add test of bucket removal with object not found error
...
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-06-01 16:13:28 +03:00
34bbbcf1ed
[ #78 ] Do not show objects missing from object service
...
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-06-01 16:13:28 +03:00
fae03c2b50
[ #78 ] Process 'not found' error when object exists in tree
...
When object exists in tree but missing in storage, we can't remove
bucket. While storage node does not sync tree service and object
service, the only way to delete such broken bucket is to ignore
'object not found' error, clear cache and do not include missing
objects in the listing result.
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-06-01 15:31:22 +03:00
81e860481d
[ #122 ] Fix linter warnings
...
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-05-31 19:43:03 +03:00
a0f0d792b8
[ #78 ] layer: Clean up already removed object from tree
...
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2023-05-24 17:42:46 +03:00
9f186d9aba
[ #104 ] app: Reload copies numbers on SIGHUP
...
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2023-05-23 13:19:58 +03:00
3fbad97ae4
[ #1 ] Rename files with mentions of previous project
...
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-05-19 07:30:04 +00:00
e24bc3f2ce
[ #101 ] app: Refactor the default copies number setting
...
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2023-05-17 11:36:28 +03:00
Denis Kirillov
b366e75366
[ #81 ] Use impersonate bearer token
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-05-03 17:22:52 +03:00
e487ee5b7d
[ #70 ] Add arrays of copies numbers for location constraints
...
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2023-05-03 13:48:26 +03:00
ee8cce662b
[ #74 ] service/tree: Add logger
...
Log error instead of failing when multiple unversioned nodes are found
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-04-26 16:39:57 +03:00
9e72fe1662
[ #80 ] Refactor metrics, support dump descriptions
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-04-20 11:14:52 +03:00
70ec5a0a5b
[ #83 ] Don't create extra delete marker
...
We shouldn't create delete marker if:
1. object doesn't exist at all
2. last version is already a delete marker
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-04-19 17:56:11 +03:00
a5c89b78bc
[ #79 ] Return X-Owner-Id in head-bucket response
...
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2023-04-17 10:12:25 +03:00
fb99b26209
[ #57 ] api: Clean up List and Name caches when object is missing in Tree service
2023-04-04 17:15:04 +03:00
a025f2e9c5
[ #59 ] tree: Make interface for tree service client
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-03-29 09:34:10 +03:00
bd3164c57f
[ #68 ] Fix pre-commit issues
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-03-24 16:22:06 +03:00
a0937126cb
[ #64 ] Support new system attributes
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-03-22 11:46:58 +03:00
ef556bd8ac
[ #60 ] Use session token to set eACL during Complete Multipart Upload
...
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-03-22 11:22:11 +03:00
5104683f68
[ #60 ] Refactor start of periodic XML writer
...
Reduce code duplication for error handling
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-03-22 11:02:16 +03:00
8151753eeb
[ #60 ] Use periodic white space XML writer in Complete Multipart Upload
...
This mechanism is used by Amazon S3 to keep client's
connection alive while object is being constructed from
the upload parts.
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-03-20 13:34:22 +03:00
2282c32822
[ #60 ] Add *NoHeader functions
...
Such functions should be used together with periodic white space
XML writer.
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-03-20 12:07:36 +03:00
43685e03d9
[ #60 ] Implement flusher on all http.ResponseWriters
...
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-03-20 12:04:34 +03:00
cf18158da4
[ #60 ] Implement periodic white space XML writer
...
Periodic white space XML writer sends XML header
and white spaces to the io.Writer.
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-03-20 12:04:34 +03:00
0af06c3bd9
[TrueCloudLab#40] Add param to configure xml decoder
...
This parameter enables parsing xml body without
xmlns="http://s3.amazonaws.com/doc/2006-03-01/ " attribute
for CompleteMultipartUpload requests
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-03-09 15:44:13 +03:00
64e7356acc
[TrueCloudLab#32] Add custom policy unmarshaler
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-03-09 11:19:01 +00:00
32bf915502
[TrueCloudLab#37] Limit number of objects to delete
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-03-09 11:16:22 +00:00
813aa2f173
Rename package name
...
Due to source code relocation from GitHub.
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-03-07 17:38:08 +03:00
740acadd37
[TrueCloudLab#36] Fix cors object payload
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-02-20 19:22:48 +03:00
aadefd98b6
[TrueCloudLab#25] Process allow and deny lists of zones in bucket head requests
...
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-02-17 13:47:48 +03:00
8ac630ee71
[TrueCloudLab#25] Add zone data to BucketInfo
...
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-02-17 13:47:48 +03:00
9f823bd65a
[TrueCloudLab#26] Add billing metrics to separate registry
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-02-10 10:55:38 +03:00
9dcacc230e
[TrueCloudLab#23] Return error on unknown LocationConstraint
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-02-08 15:04:12 +03:00
5ee4bf80ae
[ #18 ] Return container name in head-bucket response
...
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2023-02-01 14:25:21 +03:00
e278ab9362
[TrueCloudLab#5] Refactor middlewares
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-01-25 11:02:23 +03:00
c5570e661d
[TrueCloudLab#5] Add traffic metrics per user
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-01-25 11:02:23 +03:00
fc5c09c084
[TrueCloudLab#5] Request metrics per user
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-01-25 11:02:23 +03:00
8e14ccad9f
[ #754 ] Remove duplicating CID/OID logging calls
...
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
(cherry picked from commit 8c178fc3c7265fecc897a2526419bb748847c668)
2022-12-28 17:06:32 +03:00
af7fbd6f32
[ #754 ] Add CID and OID to logs
...
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2022-12-28 17:06:32 +03:00
