473239bf36
[ #257 ] Add policy checker
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-12-06 17:47:51 +03:00
93cf7c462b
[ #271 ] Add namespace label to billing metrics
...
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-12-04 14:54:40 +03:00
28c6bb4cb8
[ #266 ] Support per namespace placement policies configuration
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-11-28 16:12:42 +03:00
ff1ec56d24
[ #260 ] Use namespace as domain when create bucket
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-11-27 15:30:12 +03:00
055cc6a22a
[ #260 ] Use namespace as domain when resolve bucket
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-11-23 11:00:11 +03:00
6304d7bfda
[ #260 ] Support frostfsid validation
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-11-23 11:00:11 +03:00
cf7254f8cd
[ #260 ] Refactor api/auth/center.go
...
Move the Center interface to middleware package where it's used
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-11-23 11:00:09 +03:00
861454e499
[ #218 ] Add check content sha256 header
...
The X-Amz-Content-Sha256 header check is done only for unencrypted payload.
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
2023-11-22 11:33:52 +00:00
b28ecef43b
[ #219 ] Return ETag in quotes
...
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-11-22 11:12:32 +00:00
fe796ba538
[ #217 ] Consider Copy-Source-SSE-* headers during copy
...
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-11-13 13:22:58 +00:00
5ee73fad6a
[ #248 ] Correct NextVersionIDMarker in listing versions
...
Despite the spec https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjectVersions.html#API_ListObjectVersions_ResponseElements
says that
"When the number of responses exceeds the value of MaxKeys,
NextVersionIdMarker specifies the first object version not returned
that satisfies the search criteria. Use this value for the
version-id-marker request parameter in a subsequent request."
the actual behavior of AWS S3 is returning NextVersionIdMarker as the last returned object version
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-10-31 17:36:24 +03:00
890a8ed237
[ #227 ] Add versionID header after complete multipart
...
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-10-31 14:07:08 +00:00
0bed25816c
[ #224 ] Add conditional escaping for object name
...
Chi gives inconsistent results in terms of whether
the strings returned are URL coded or not
See:
* https://github.com/go-chi/chi/issues/641
* https://github.com/go-chi/chi/issues/642
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-10-31 13:58:51 +00:00
b169c5e6c3
[ #239 ] Update test for check goroutines leak
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-10-31 13:51:23 +00:00
cf13aae342
[ #225 ] Add default storage class to responses
...
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-10-31 13:37:07 +00:00
0938d7ee82
[ #226 ] Fix status code in GET/HEAD delete marker
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-10-27 10:58:57 +03:00
4f5f5fb5c8
[ #222 ] Fix marshaling errors in DeleteObjects method
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-10-25 14:54:02 +00:00
25bb581fee
[ #205 ] Add md5 checksum in header
...
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-10-25 11:04:19 +03:00
8d6aa0d40a
[ #243 ] Fix list object versions marker param
...
According to https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjectVersions.html
we have to use `key-marker`
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-10-18 10:35:47 +03:00
7e91f62c28
[ #223 ] Add store content language
...
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
2023-10-17 14:42:02 +00:00
01323ca8e0
[ #216 ] Add check tag key uniqueness
...
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
2023-10-17 14:40:29 +00:00
298662df9d
[ #221 ] Expand xmlns field ignore
...
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-10-13 16:21:13 +03:00
65412ce1d3
[ #197 ] Configure buffer max size for PUT
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-10-11 12:32:48 +00:00
7de73f6b73
[ #197 ] Disable homomorphic hash for PUT
...
Disable TZ hash for PUT if it's disabled for container itself
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-10-11 12:32:48 +00:00
e3f2d59565
[ #154 ] Rename access key to secret key
...
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
2023-10-06 09:20:39 +00:00
c4af1dc4ad
[ #171 ] Update message error auth header malformed
...
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
2023-10-04 11:13:12 +00:00
b8c93ed391
[ #172 ] Convert handler config to interface
...
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-10-04 11:01:27 +00:00
51e591877b
[ #207 ] Fix list parts with empty list
...
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-09-21 11:27:20 +03:00
a4c612614a
[ #210 ] Fix multipart object reader
...
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-09-19 16:30:08 +03:00
12cf29aed2
[ #207 ] Fix part-number-marker handling
...
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-09-19 12:43:07 +03:00
066b9a0250
[ #142 ] Add trace ID into log when tracing is enabled
...
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-09-07 14:19:37 +03:00
9120e97ac5
[ #203 ] Add go1.21 to CI
...
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-08-31 15:26:07 +03:00
b5fce5c8d2
[ #168 ] Skip only invalid policies and copies instead of ignoring all of them
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-25 12:05:30 +03:00
6617adc22b
[ #185 ] Use correct object size when object is combined or encrypted
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-24 18:03:08 +03:00
8898c2ec08
[ #185 ] Add tests for list multipart uploads
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-24 18:01:08 +03:00
8efcc957ea
[ #96 ] Move log messages to constants
...
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
2023-08-23 18:32:31 +03:00
6b728fef87
[ #192 ] Add tests to make sure client_cut flag is passed to sdk
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-23 06:26:55 +00:00
6b1f365e65
[ #192 ] Support client cut
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-23 06:26:55 +00:00
fcf1c45ad2
[ #188 ] Fix url escaping
...
Url escaping has already been done in `net/http/request.go`
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-22 11:27:39 +03:00
94a42fa273
[ #51 ] Update frostfs-sdk-go
...
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2023-08-17 14:26:25 +03:00
40d7f844e3
[ #137 ] Refactor context data retrievers
...
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
2023-08-16 14:05:38 +00:00
6b109eee92
[ #182 ] Fix parsing signed headers in presigned urls
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-08 13:04:53 +03:00
18878b66d3
[ #175 ] Use gate owner as object owner
...
This is required because node check session token owner
TrueCloudLab/frostfs-node#528
For client cut TrueCloudLab/frostfs-sdk-go#114
such owner will be gate owner
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-08 12:22:33 +03:00
fe897ec588
[ #178 ] wrapReader: Fix goroutine leak
...
In case of error in FrostFS.CreateObject wrapped reader
can be blocked because of synchronous pipe. We have to read out all payload in such case.
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-01 17:59:34 +03:00
52931663e1
[ #176 ] multipart: Replace part on re-upload
...
We want to have exactly one object and tree node for each part number
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-21 16:20:04 +03:00
1a09041cd1
[ #63 ] Simplify multiObjectReader and add tests
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-20 14:00:15 +03:00
631b7f67b4
[ #63 ] multipart: Log upload id for every failed request
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-20 14:00:15 +03:00
bcf5a85aab
[ #63 ] multipart: Fix copying
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-20 14:00:15 +03:00
ad81b599dd
[ #63 ] Add fast multipart upload
...
Add new flag to object tree meta `isCombined` that means
the object payload is list of parts that forms real payload.
Set this attribute when complete multipart upload not to do unnecessary copying.
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-20 14:00:15 +03:00
361d10cc78
[ #174 ] Fix query for listing multipart uploads
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-20 12:30:17 +03:00
62e6b49254
[ #174 ] Log unmatched requests
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-20 12:30:17 +03:00
80c4982bd4
[ #174 ] Add router tests
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-20 12:30:17 +03:00
73ed3f7782
[ #174 ] Fix router filter query matching
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-20 12:30:17 +03:00
6e3595e35b
[ #174 ] Fix object keys with slashes in chi
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-20 12:30:17 +03:00
b59aa06637
[ #146 ] Add kludge.bypass_content_encoding_check_in_chunks flag
...
Flag allows to skip checking `Content-Encoding` for `aws-chunked` value
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-18 14:49:52 +03:00
d62aa7b979
[ #146 ] Fix preconditions: trim quotes in etags
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-18 14:49:03 +03:00
751a9be7cc
[ #146 ] Move getting chunk payload reader to separate function
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-18 14:49:03 +03:00
14ef9ff091
[ #158 ] Separate init object reader from read itself
...
To be able to handle cases and return appropriate http status code
when object missed in storage but gate cache contains its metadata
we need write code after init object reader.
So we separate init reader from actual reading.
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-11 17:32:05 +03:00
fc90981c03
[ #149 ] Update inner imports after moving middlewares
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-11 17:25:09 +03:00
83cdfbee78
[ #149 ] Move middlewares to separate package
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-11 17:25:09 +03:00
37f2f468fe
[ #149 ] Add host bucket router
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-11 17:25:09 +03:00
e30a452cdb
[ #149 ] Use chi instead of gorilla mux
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-11 17:25:09 +03:00
7f708b3a2d
[ #111 ] auth: Get log from real request context
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-06 11:57:04 +00:00
d531b13866
[ #143 ] Add more context to some s3 errors
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-30 12:08:33 +03:00
f921bc8af5
[ #143 ] Fix typo
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-30 12:04:52 +03:00
be03c5178f
[ #143 ] Fix NoSuchKey error on get/head
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-30 12:04:52 +03:00
9df8695463
[ #143 ] Fix transformToS3Error function
...
Unwrap error before checking for s3 error
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-21 17:16:40 +03:00
614d703726
[ #106 ] Add chunk uploading
...
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2023-06-21 10:23:57 +03:00
23593eee3d
[ #111 ] Use request scope logger
...
Store child zap logger with request scope fields into context.
Request scoped fields: request_id, api/method, bucket, object
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-19 13:54:51 +03:00
84358f6742
[ #135 ] authmate: Support CRDT GSet for credentials
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-16 09:42:49 +03:00
7a380fa46c
[ #135 ] frostfs: Add SEARCH operation
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-16 09:42:41 +03:00
462589fc0c
[ #103 ] Return 504 http code on timeout errors
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-14 09:16:21 +03:00
9dabaf6ecd
[ #133 ] Use tree pool from SDK
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-09 09:34:04 +03:00
4e1fd9589b
[ #84 ] add tracing support
...
Signed-off-by: Pavel Pogodaev <p.pogodaev@yadro.com>
2023-06-06 10:25:27 +03:00
4a6e3a19ce
[ #125 ] Handle negative Content-Length on put
...
Add computing actual object size during calculating hash on put.
Use this actual value to save in tree and cache
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-02 14:16:19 +00:00
b445f7bbf9
[ #125 ] api/auth: DisableURIPathEscaping for presign
...
Don't use escaping when presign url.
Escape manually before.
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-02 14:16:19 +00:00
868edfdb31
[ #78 ] Add test of bucket removal with object not found error
...
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-06-01 16:13:28 +03:00
34bbbcf1ed
[ #78 ] Do not show objects missing from object service
...
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-06-01 16:13:28 +03:00
fae03c2b50
[ #78 ] Process 'not found' error when object exists in tree
...
When object exists in tree but missing in storage, we can't remove
bucket. While storage node does not sync tree service and object
service, the only way to delete such broken bucket is to ignore
'object not found' error, clear cache and do not include missing
objects in the listing result.
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-06-01 15:31:22 +03:00
81e860481d
[ #122 ] Fix linter warnings
...
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-05-31 19:43:03 +03:00
a0f0d792b8
[ #78 ] layer: Clean up already removed object from tree
...
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2023-05-24 17:42:46 +03:00
9f186d9aba
[ #104 ] app: Reload copies numbers on SIGHUP
...
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2023-05-23 13:19:58 +03:00
3fbad97ae4
[ #1 ] Rename files with mentions of previous project
...
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-05-19 07:30:04 +00:00
e24bc3f2ce
[ #101 ] app: Refactor the default copies number setting
...
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2023-05-17 11:36:28 +03:00
Denis Kirillov
b366e75366
[ #81 ] Use impersonate bearer token
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-05-03 17:22:52 +03:00
e487ee5b7d
[ #70 ] Add arrays of copies numbers for location constraints
...
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2023-05-03 13:48:26 +03:00
ee8cce662b
[ #74 ] service/tree: Add logger
...
Log error instead of failing when multiple unversioned nodes are found
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-04-26 16:39:57 +03:00
9e72fe1662
[ #80 ] Refactor metrics, support dump descriptions
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-04-20 11:14:52 +03:00
70ec5a0a5b
[ #83 ] Don't create extra delete marker
...
We shouldn't create delete marker if:
1. object doesn't exist at all
2. last version is already a delete marker
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-04-19 17:56:11 +03:00
a5c89b78bc
[ #79 ] Return X-Owner-Id in head-bucket response
...
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2023-04-17 10:12:25 +03:00
fb99b26209
[ #57 ] api: Clean up List and Name caches when object is missing in Tree service
2023-04-04 17:15:04 +03:00
a025f2e9c5
[ #59 ] tree: Make interface for tree service client
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-03-29 09:34:10 +03:00
bd3164c57f
[ #68 ] Fix pre-commit issues
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-03-24 16:22:06 +03:00
a0937126cb
[ #64 ] Support new system attributes
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-03-22 11:46:58 +03:00
ef556bd8ac
[ #60 ] Use session token to set eACL during Complete Multipart Upload
...
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-03-22 11:22:11 +03:00
5104683f68
[ #60 ] Refactor start of periodic XML writer
...
Reduce code duplication for error handling
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-03-22 11:02:16 +03:00
8151753eeb
[ #60 ] Use periodic white space XML writer in Complete Multipart Upload
...
This mechanism is used by Amazon S3 to keep client's
connection alive while object is being constructed from
the upload parts.
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-03-20 13:34:22 +03:00
2282c32822
[ #60 ] Add *NoHeader functions
...
Such functions should be used together with periodic white space
XML writer.
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-03-20 12:07:36 +03:00
43685e03d9
[ #60 ] Implement flusher on all http.ResponseWriters
...
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-03-20 12:04:34 +03:00
cf18158da4
[ #60 ] Implement periodic white space XML writer
...
Periodic white space XML writer sends XML header
and white spaces to the io.Writer.
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-03-20 12:04:34 +03:00
0af06c3bd9
[TrueCloudLab#40] Add param to configure xml decoder
...
This parameter enables parsing xml body without
xmlns="http://s3.amazonaws.com/doc/2006-03-01/ " attribute
for CompleteMultipartUpload requests
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-03-09 15:44:13 +03:00
64e7356acc
[TrueCloudLab#32] Add custom policy unmarshaler
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-03-09 11:19:01 +00:00
32bf915502
[TrueCloudLab#37] Limit number of objects to delete
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-03-09 11:16:22 +00:00
813aa2f173
Rename package name
...
Due to source code relocation from GitHub.
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-03-07 17:38:08 +03:00
740acadd37
[TrueCloudLab#36] Fix cors object payload
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-02-20 19:22:48 +03:00
aadefd98b6
[TrueCloudLab#25] Process allow and deny lists of zones in bucket head requests
...
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-02-17 13:47:48 +03:00
8ac630ee71
[TrueCloudLab#25] Add zone data to BucketInfo
...
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-02-17 13:47:48 +03:00
9f823bd65a
[TrueCloudLab#26] Add billing metrics to separate registry
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-02-10 10:55:38 +03:00
9dcacc230e
[TrueCloudLab#23] Return error on unknown LocationConstraint
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-02-08 15:04:12 +03:00
5ee4bf80ae
[ #18 ] Return container name in head-bucket response
...
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2023-02-01 14:25:21 +03:00
e278ab9362
[TrueCloudLab#5] Refactor middlewares
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-01-25 11:02:23 +03:00
c5570e661d
[TrueCloudLab#5] Add traffic metrics per user
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-01-25 11:02:23 +03:00
fc5c09c084
[TrueCloudLab#5] Request metrics per user
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-01-25 11:02:23 +03:00
8e14ccad9f
[ #754 ] Remove duplicating CID/OID logging calls
...
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
(cherry picked from commit 8c178fc3c7265fecc897a2526419bb748847c668)
2022-12-28 17:06:32 +03:00
af7fbd6f32
[ #754 ] Add CID and OID to logs
...
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2022-12-28 17:06:32 +03:00
000d9ed42a
[ #2 ] Update issue links
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2022-12-21 11:17:45 +03:00
388482e230
[ #2 ] Rename internals
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2022-12-21 11:17:45 +03:00
765fd28a49
[ #2 ] Update build
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2022-12-20 18:24:30 +03:00
96dff367db
[ #1 ] Build S3 Gateway with FrostFS dependencies
...
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2022-12-15 12:43:52 +03:00
Denis Kirillov
dd4f66712c
[ #742 ] Add multiple listeners
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-11-24 17:48:26 +03:00
Denis Kirillov
d2587b21af
[ #747 ] Reload policies on SIGHUP
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-11-18 13:14:25 +03:00
Angira Kekteeva
f5fe9a9b4b
[ #741 ] Fix data race
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-11-16 18:19:31 +03:00
Denis Kirillov
8f1bbb7dc1
[ #749 ] Update SDK
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-11-15 10:16:41 +03:00
Denis Kirillov
094eb12578
[ #726 ] Use client time on regular requests
...
Use `X-Amz-Date` header as `now` when
* compute expiration epoch
* set Timestamp for object and container
* forming locks
* send notifications
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-11-14 14:53:55 +03:00
Denis Kirillov
d47840f137
[ #568 ] Add configuration for region to policy map
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-11-11 13:12:49 +03:00
Denis Kirillov
2886ac161c
[ #740 ] Fix forming policy by ast
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-11-10 15:33:03 +03:00
Denis Kirillov
16ad0cadcb
[ #733 ] Fix container name
...
User system name instead of regular.
Set name as CID if name attribute is missed
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-10-26 17:52:02 +03:00
Denis Kirillov
18a6aca4b4
[ #617 ] api/handler: Simplify tests
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-10-25 19:04:37 +03:00
Angira Kekteeva
712740b668
[ #728 ] Remove redundant log message
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-10-25 19:04:11 +03:00
Angira Kekteeva
bbd7ba670a
[ #728 ] Add bucket and object to method logs
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-10-25 19:04:11 +03:00
Angira Kekteeva
da61fd8bcb
[ #728 ] Add error message to AbortMultipartUpload
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-10-25 19:04:11 +03:00
Denis Kirillov
0d0ecf5b10
[ #650 ] Drop unused object cache methods
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-10-25 16:00:21 +03:00
Denis Kirillov
821df3d648
[ #669 ] Optimize getNodeVersion for tags operation
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-10-20 17:56:09 +03:00
Denis Kirillov
90eb4f0188
[ #713 ] Add check access control for system caches
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-10-11 15:26:42 +03:00
Denis Kirillov
cb55d36063
[ #713 ] Update tests
...
Add bearer token to test context
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-10-11 15:26:42 +03:00
Denis Kirillov
3d31c2ab4a
[ #713 ] Fix cache access
...
User can access only keys that he puts into caches
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-10-11 15:26:42 +03:00
Denis Kirillov
c6098e53f6
[ #719 ] Add tests
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-10-11 11:13:20 +03:00
Denis Kirillov
b3d96e6460
[ #719 ] Improve lock error handling
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-10-11 11:13:20 +03:00
Denis Kirillov
2a41929be3
[ #702 ] Reload resolvers and TLS certs on SIGHUP
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-09-28 15:07:31 +03:00
Denis Kirillov
be4600988e
[ #709 ] Add expiration attribute for legal hold
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-09-26 16:51:53 +03:00
Denis Kirillov
3d08562843
[ #505 ] Handle access denied from tree service
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-09-13 18:05:16 +03:00
Denis Kirillov
80beedf13e
[ #504 ] Use bktInfo in tree service to check owner
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-09-13 18:05:16 +03:00
Denis Kirillov
4082cd6b54
[ #606 ] Keep eacl records order on conflict
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-09-08 12:21:47 +03:00
Denis Kirillov
ffd259671a
[ #704 ] Fix nil pointer dereference on decReader
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-09-07 15:51:44 +03:00
Denis Kirillov
9cd4ef1ac4
[ #657 ] Replace FileName with FilePath attribute
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-09-07 15:50:43 +03:00
Denis Kirillov
6108f8d929
[ #624 ] Update tree service revision
...
Remove extra request to tree service to know parentID
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-09-07 12:17:51 +03:00
Angira Kekteeva
3e4fc52428
[ #674 ] Add AccessKeyID filter
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-09-06 13:47:20 +03:00
272c485706
[ #669 ] Use object cache in PutLockInfo optimization
...
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2022-08-31 13:19:00 +03:00
5ffe28a42f
[ #669 ] Optimize PutLockInfo
...
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2022-08-31 13:19:00 +03:00
Denis Kirillov
96c7b79d1c
[ #683 ] Forbid copy to itself in unversioned bucket
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-08-30 13:31:48 +03:00
Denis Kirillov
44d5878a80
[ #656 ] Read full body on upload error
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-08-30 09:39:20 +03:00
9590b39333
[ #682 ] Return added node id in AddVersion
...
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2022-08-29 18:24:00 +03:00
Denis Kirillov
5c7a509cc9
[ #684 ] Fix continuation token for directories
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-08-29 16:10:01 +03:00
Denis Kirillov
e38bdae07a
[ #676 ] Fix object acl
...
Put object acl always add rules to specific version of object.
Get object acl consider READ rights as FULL_CONTROL
because WRITE cannot be applied to object
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-08-29 13:20:30 +03:00
Denis Kirillov
163038b37d
[ #672 ] Fix handling X-Amz-Copy-Source header
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-08-25 17:40:15 +03:00
Denis Kirillov
fdc926e70b
[ #661 ] Fix updating cache when request fails
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-08-25 12:55:49 +03:00
Denis Kirillov
1a580b6fa4
[ #666 ] Support tagging directive for CopyObject
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-08-25 12:55:33 +03:00
Denis Kirillov
f013282290
[ #660 ] Fix panic on invalid versioning status
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-08-25 11:50:44 +03:00
Denis Kirillov
24e1f7e1dd
[ #545 ] Drop object search
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-08-25 11:49:08 +03:00
Denis Kirillov
c3ad6d2faf
[ #637 ] Add header to override CopiesNumber
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-08-19 10:31:26 +03:00
Angira Kekteeva
3b343d1bd5
[ #634 ] Fix tests
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-08-16 17:49:15 +03:00
Angira Kekteeva
5307211398
[ #634 ] Add CopiesNumber in NeoFS requests
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-08-16 17:49:15 +03:00
Denis Kirillov
6aab3936d9
[ #625 ] Limit listing generator with maxKeys
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-08-16 13:47:37 +03:00
Denis Kirillov
3824151699
[ #616 ] Reduce number of requests during listing
...
Check if object is directory before request to NeoFS
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-08-16 13:47:37 +03:00
Denis Kirillov
107d8a9033
[ #595 ] Fix typos and rewording
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-08-13 10:26:00 +03:00
Denis Kirillov
7ab473a688
[ #595 ] Simplify encryption.Params struct
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-08-13 10:26:00 +03:00
Denis Kirillov
94a6a55919
[ #595 ] Move decrypter to separate package
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-08-13 10:26:00 +03:00
Denis Kirillov
d824db7f69
[ #595 ] Allow SSE-C only with TLS
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-08-13 10:26:00 +03:00
Denis Kirillov
87c05eb514
[ #595 ] Add SSE with customer key
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-08-13 10:26:00 +03:00
Angira Kekteeva
1987f271f9
[ #577 ] Remove usages of deprecated io/ioutil
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-08-09 17:20:40 +04:00
Angira Kekteeva
f66e4a77cd
[ #577 ] Add IsDeleteMarker() to NodeVersion
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-08-09 17:20:40 +04:00
Angira Kekteeva
2a933f48be
[ #577 ] Remove redundant set of isLatest
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-08-09 17:20:40 +04:00
Angira Kekteeva
3188fd8eb7
[ #577 ] Fix delete from tagging cache
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-08-09 17:20:40 +04:00
Angira Kekteeva
8031baddb6
[ #577 ] Fix cache tests
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-08-09 17:20:40 +04:00
Angira Kekteeva
245e64900d
[ #577 ] Replace ObjInfo with ExtObjInfo in cache
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-08-09 17:20:40 +04:00
Angira Kekteeva
dc77ad4863
[ #577 ] Fix tests
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-08-09 17:20:40 +04:00
Angira Kekteeva
93e0cae743
[ #577 ] Fix version in GetObjectAttributes
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-08-09 17:20:40 +04:00
Angira Kekteeva
659e7bfc97
[ #577 ] Fix version in GetObject and HeadObject
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-08-09 17:20:40 +04:00
Angira Kekteeva
dfd734b9ec
[ #577 ] Separate GetObjectInfo and GetExtendedObjectInfo
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-08-09 17:20:40 +04:00
Angira Kekteeva
37c46e926e
[ #577 ] Add Version() to ExtendedObjectInfo
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-08-09 17:20:40 +04:00
Angira Kekteeva
3ac3f1cc9d
[ #577 ] Rename objectInfo.Version() to VersionID()
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-08-09 17:20:40 +04:00
Angira Kekteeva
eaf76afe50
[ #577 ] Remove ObjectVersionInfo
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-08-09 17:20:40 +04:00
Angira Kekteeva
2d58b25ea0
[ #643 ] Fix linter issues
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-08-05 15:32:46 +03:00
Angira Kekteeva
1ee1b8bb79
[ #643 ] Fix missing const api.QueryVersionID
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-08-05 15:32:46 +03:00
Angira Kekteeva
072e694540
[ #643 ] Optimize GetObjectTagging
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-08-05 15:32:46 +03:00
Angira Kekteeva
9372534345
[ #643 ] Fix tagging cache
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-08-05 15:32:46 +03:00
Alex Vanin
5219965105
[ #498 ] Remove log string sanitizer
...
`zap` lib does all the job anyway.
Suppress CWE-117 from CodeQL.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-07-29 09:32:37 +03:00
Denis Kirillov
1fd943ee88
[ #610 ] Fix deleted object removal
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-29 09:31:54 +03:00
Denis Kirillov
5dd658302d
[ #612 ] Fix comment
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-26 17:36:21 +03:00
Denis Kirillov
b2e8b1cfb3
[ #612 ] Make listing more robust
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-26 17:36:21 +03:00
Angira Kekteeva
4483c6f57a
[ #619 ] Filter system metadata
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-07-26 15:37:13 +03:00
Angira Kekteeva
4380406602
[ #619 ] Remove unused PutSystemObjectParams
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-07-26 15:37:13 +03:00
Alex Vanin
e0136feb73
[ #611 ] Split name processing from object info creation procedure
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-07-25 14:23:30 +03:00
Alex Vanin
97f81d3270
[ #611 ] Process delimiters on the object info cache hit
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-07-25 14:23:30 +03:00
Alex Vanin
757600608c
[ #539 ] Fix imports
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
85eacdb970
[ #602 ] Fix removal non-empty bucket
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
0057f6b7db
[ #546 ] Add size and etag in nodeVersionInfo
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Angira Kekteeva
56eb2dc3dc
[ #569 ] Rename versioning consts
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-07-22 15:19:16 +03:00
Angira Kekteeva
77f466b1d5
[ #569 ] Fix tests
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-07-22 15:19:16 +03:00
Angira Kekteeva
72013e08ec
[ #569 ] Refactor versioning in tree service
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-07-22 15:19:16 +03:00
