399a6d6d65
[ #370 ] Fix removing combined object
...
Port TrueCloudLab/frostfs-s3-gw#364
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-04-17 14:35:12 +03:00
486b8a4284
[ #321 ] Use correct owner id in billing metrics
...
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2024-02-27 11:23:55 +03:00
94980059b7
[ #280 ] Add put requests to duration metric
...
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2024-01-29 06:28:04 +00:00
809bc2eac8
[ #262 ] Set tree request id
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-12-07 12:20:53 +00:00
aa2c016f83
[ #205 ] Add md5 checksum in ETag by config param
...
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-12-07 12:20:14 +00:00
2f29229383
[ #249 ] Skip flaky test
...
Test manually passes but it breaks CI/CD sometimes.
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-10-26 10:48:43 +03:00
e87c3715c5
[ #233 ] Clean tag node in the tree service instead of removal
...
With new retry policy of tree service pool, gateway should avoid
deletion of system nodes from tree. Absence of node in the tree
will trigger retry. Other storage in the network may return already
deleted node while tree is not completely synced, and client will
get unexpected result.
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-10-18 11:52:48 +03:00
5f9555afad
[ #237 ] Fix list object versions marker param
...
According to https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjectVersions.html
we have to use `key-marker`
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-10-12 11:36:39 +03:00
51e591877b
[ #207 ] Fix list parts with empty list
...
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-09-21 11:27:20 +03:00
a4c612614a
[ #210 ] Fix multipart object reader
...
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-09-19 16:30:08 +03:00
12cf29aed2
[ #207 ] Fix part-number-marker handling
...
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-09-19 12:43:07 +03:00
066b9a0250
[ #142 ] Add trace ID into log when tracing is enabled
...
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-09-07 14:19:37 +03:00
9120e97ac5
[ #203 ] Add go1.21 to CI
...
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-08-31 15:26:07 +03:00
b5fce5c8d2
[ #168 ] Skip only invalid policies and copies instead of ignoring all of them
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-25 12:05:30 +03:00
6617adc22b
[ #185 ] Use correct object size when object is combined or encrypted
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-24 18:03:08 +03:00
8898c2ec08
[ #185 ] Add tests for list multipart uploads
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-24 18:01:08 +03:00
8efcc957ea
[ #96 ] Move log messages to constants
...
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
2023-08-23 18:32:31 +03:00
6b728fef87
[ #192 ] Add tests to make sure client_cut flag is passed to sdk
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-23 06:26:55 +00:00
6b1f365e65
[ #192 ] Support client cut
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-23 06:26:55 +00:00
fcf1c45ad2
[ #188 ] Fix url escaping
...
Url escaping has already been done in `net/http/request.go`
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-22 11:27:39 +03:00
94a42fa273
[ #51 ] Update frostfs-sdk-go
...
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2023-08-17 14:26:25 +03:00
40d7f844e3
[ #137 ] Refactor context data retrievers
...
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
2023-08-16 14:05:38 +00:00
6b109eee92
[ #182 ] Fix parsing signed headers in presigned urls
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-08 13:04:53 +03:00
18878b66d3
[ #175 ] Use gate owner as object owner
...
This is required because node check session token owner
TrueCloudLab/frostfs-node#528
For client cut TrueCloudLab/frostfs-sdk-go#114
such owner will be gate owner
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-08 12:22:33 +03:00
fe897ec588
[ #178 ] wrapReader: Fix goroutine leak
...
In case of error in FrostFS.CreateObject wrapped reader
can be blocked because of synchronous pipe. We have to read out all payload in such case.
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-01 17:59:34 +03:00
52931663e1
[ #176 ] multipart: Replace part on re-upload
...
We want to have exactly one object and tree node for each part number
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-21 16:20:04 +03:00
1a09041cd1
[ #63 ] Simplify multiObjectReader and add tests
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-20 14:00:15 +03:00
631b7f67b4
[ #63 ] multipart: Log upload id for every failed request
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-20 14:00:15 +03:00
bcf5a85aab
[ #63 ] multipart: Fix copying
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-20 14:00:15 +03:00
ad81b599dd
[ #63 ] Add fast multipart upload
...
Add new flag to object tree meta `isCombined` that means
the object payload is list of parts that forms real payload.
Set this attribute when complete multipart upload not to do unnecessary copying.
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-20 14:00:15 +03:00
361d10cc78
[ #174 ] Fix query for listing multipart uploads
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-20 12:30:17 +03:00
62e6b49254
[ #174 ] Log unmatched requests
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-20 12:30:17 +03:00
80c4982bd4
[ #174 ] Add router tests
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-20 12:30:17 +03:00
73ed3f7782
[ #174 ] Fix router filter query matching
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-20 12:30:17 +03:00
6e3595e35b
[ #174 ] Fix object keys with slashes in chi
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-20 12:30:17 +03:00
b59aa06637
[ #146 ] Add kludge.bypass_content_encoding_check_in_chunks flag
...
Flag allows to skip checking `Content-Encoding` for `aws-chunked` value
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-18 14:49:52 +03:00
d62aa7b979
[ #146 ] Fix preconditions: trim quotes in etags
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-18 14:49:03 +03:00
751a9be7cc
[ #146 ] Move getting chunk payload reader to separate function
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-18 14:49:03 +03:00
14ef9ff091
[ #158 ] Separate init object reader from read itself
...
To be able to handle cases and return appropriate http status code
when object missed in storage but gate cache contains its metadata
we need write code after init object reader.
So we separate init reader from actual reading.
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-11 17:32:05 +03:00
fc90981c03
[ #149 ] Update inner imports after moving middlewares
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-11 17:25:09 +03:00
83cdfbee78
[ #149 ] Move middlewares to separate package
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-11 17:25:09 +03:00
37f2f468fe
[ #149 ] Add host bucket router
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-11 17:25:09 +03:00
e30a452cdb
[ #149 ] Use chi instead of gorilla mux
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-11 17:25:09 +03:00
7f708b3a2d
[ #111 ] auth: Get log from real request context
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-06 11:57:04 +00:00
d531b13866
[ #143 ] Add more context to some s3 errors
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-30 12:08:33 +03:00
f921bc8af5
[ #143 ] Fix typo
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-30 12:04:52 +03:00
be03c5178f
[ #143 ] Fix NoSuchKey error on get/head
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-30 12:04:52 +03:00
9df8695463
[ #143 ] Fix transformToS3Error function
...
Unwrap error before checking for s3 error
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-21 17:16:40 +03:00
614d703726
[ #106 ] Add chunk uploading
...
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2023-06-21 10:23:57 +03:00
23593eee3d
[ #111 ] Use request scope logger
...
Store child zap logger with request scope fields into context.
Request scoped fields: request_id, api/method, bucket, object
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-19 13:54:51 +03:00
