Commit graph

318 commits

Author SHA1 Message Date
Denis Kirillov
458f9cf17b [#242] Fix versions sorting
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-09-07 17:06:22 +03:00
Denis Kirillov
42ed6a16ea [#190] Add POST object
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-09-07 11:10:48 +03:00
Angira Kekteeva
239742f413 [#236] api: Create info.go for basic structs
Moved BucketInfo and ObjectInfo from layer and handler to api

Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-09-07 11:08:48 +03:00
Denis Kirillov
a9be642eaf [#213] Add object acl versioning
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-08-31 12:38:23 +03:00
Denis Kirillov
5502fb97c3 [#196] Add tagging caching
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-08-31 10:46:42 +03:00
Denis Kirillov
987185b9e1 [#196] Add bucket tagging
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-08-31 10:46:42 +03:00
Denis Kirillov
16da1aba64 [#196] Add DeleteObjectTagging
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-08-31 10:46:42 +03:00
Denis Kirillov
4a2575fbf3 [#196] Add GetObjectTagging
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-08-31 10:46:42 +03:00
Denis Kirillov
8b5ebe2ec2 [#196] Add PutObjectTagging
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-08-31 10:46:42 +03:00
Kirillov Denis
04b7958ab3
Merge pull request #207 from KirillovDenis/feature/122-poc_versioning
[#122] Add enabling versioning
2021-08-25 15:01:33 +03:00
Denis Kirillov
4bb885d526 [#122] Refactor
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-08-25 14:49:21 +03:00
Angira Kekteeva
2299db4e81 [#218] handler,s3-gw: Make policy configurable
Now default policy of placing containers can be set via config/env
variable.

Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-08-25 12:18:23 +03:00
Angira Kekteeva
6d4fe34f3d [#218] handler: Remove unused struct
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-08-25 12:18:23 +03:00
Denis Kirillov
11558124cd [#122] Add versioning cache
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-08-25 09:15:00 +03:00
Denis Kirillov
9c058a70fd [#122] Add tests
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-08-25 09:15:00 +03:00
Denis Kirillov
43185de52a [#122] Add list object versions
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-08-25 09:15:00 +03:00
Denis Kirillov
d5aef7566f [#122] Add delete versioned object
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-08-25 09:15:00 +03:00
Denis Kirillov
3130784ee6 [#122] Add getting specific object version
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-08-25 09:13:12 +03:00
Denis Kirillov
f463522f34 [#122] Add versioning put object
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-08-25 09:10:46 +03:00
Denis Kirillov
c50a16a5e3 [#122] Add enabling versioning
New handlers:
PutBucketVersioning, GetBucketVersioning

Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-08-25 09:10:46 +03:00
Denis Kirillov
d81a3d7b45 [#216] Add bucket owner check
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-08-25 09:07:38 +03:00
Alex Vanin
68e4e1bbc3
Merge pull request #224 from KirillovDenis/feature/220-add_cid_to_head_bucket_response
[#220] Add cid to HeadBucket response
2021-08-24 18:10:13 +03:00
Alex Vanin
036662feab
Merge pull request #222 from KirillovDenis/feature/221-unify_source_filenames
[#221] Unify source file naming
2021-08-24 12:44:23 +03:00
Denis Kirillov
84508a1706 [#220] Add cid to HeadBucket response
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-08-23 11:37:08 +03:00
Denis Kirillov
76fc241bc1 [#214] Add condition to update eacl
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-08-20 16:32:51 +03:00
Denis Kirillov
bf3d81f928 [#221] Unify source file naming
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-08-20 15:19:49 +03:00
Denis Kirillov
efe11c271f [#49] Add basic ACL translation
Implement functions:
GetBucketACL, PutBucketACL, GetObjectACL,
PutObjectACL, GetBucketPolicy, PutBucketPolicy

Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-08-18 17:20:17 +03:00
Denis Kirillov
d332096598 [#199] Refactor
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-08-11 13:17:37 +03:00
Denis Kirillov
f3a6636efd [#191] Improve copy object compatibility
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-08-11 12:37:26 +03:00
Denis Kirillov
7eb9713a67 [#189] Add bucket name checking
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-08-11 12:04:00 +03:00
Denis Kirillov
a89add8ff3 [#198] Fix object metadata
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-08-06 15:06:40 +03:00
Denis Kirillov
e5b1dae750 [#194] Fix range header handling
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-08-06 13:14:50 +03:00
Denis Kirillov
f4c29cd300 [#184] Unify error handling
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-08-05 12:18:52 +03:00
Angira Kekteeva
eaec271283 [#180] api: Add unittests
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-08-03 12:40:28 +03:00
Angira Kekteeva
88aaad361a [#180] api: Add continuation-token check
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-08-03 12:40:02 +03:00
Angira Kekteeva
7291b81040 [#180] api: Format object_list.go
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-07-29 17:31:48 +03:00
Angira Kekteeva
afff616675 [#180] api: Remove extra parses of url requests
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-07-29 17:18:51 +03:00
Angira Kekteeva
c24fe5cc21 [#112] Move getBoxData from handler to layer
And made it exported

Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-07-28 16:27:06 +03:00
Denis Kirillov
2479650739 [#118] Correct NotFound status
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-07-26 14:03:26 +03:00
Alex Vanin
f4cd1e4c38
Merge pull request #170 from KirillovDenis/feature/89-placement_policy
[#89] Add placement policy
2021-07-26 13:43:38 +03:00
Denis Kirillov
8aac61652f [#89] Move BoxData to CreateBucketParams
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-07-26 13:35:52 +03:00
Alex Vanin
3f7941fdf3
Merge pull request #174 from nspcc-dev/feature/159-optional_owner
[#159] Add fetch-owner param
2021-07-26 13:32:04 +03:00
Denis Kirillov
28974474f2 [#149] Refactoring
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-07-22 09:55:35 +03:00
Denis Kirillov
63652d213c [#149] Removed grpc errors
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-07-22 09:54:12 +03:00
Denis Kirillov
b695e6a3b4 [#89] Refactor error logging
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-07-22 09:45:24 +03:00
Denis Kirillov
65a61a9f7d [#159] Add fetch-owner param
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-07-20 15:40:38 +03:00
Denis Kirillov
c05e2d6757 [#160] Fix head dir object
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-07-20 12:36:41 +03:00
Angira Kekteeva
b1cda2a714 [#154] api: refactor EncodeV1 and EncodeV2
Move common parts of Encode to separate functions.

Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-07-19 16:38:51 +03:00
Angira Kekteeva
ee84062154 [#154] api: refactor ListObjectsV1 and V2
Separate ListObject for different versions.
Remove useless grouping of keys on V2.

Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-07-19 10:12:47 +03:00
Denis Kirillov
1fbd192bd7 [#89] Add placement policy
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-07-16 16:01:35 +03:00
Alex Vanin
22faaadc32 Merge pull request #161 from nspcc-dev/feature/158-s3_tests_conditional_headers
[#158] Handled s3 errors on conditional headers
2021-07-16 09:39:17 +03:00
Denis Kirillov
352d5345fc [#158] Handled s3 errors on conditional headers
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-07-16 09:30:08 +03:00
Alex Vanin
3a7876e292 Merge pull request #157 from KirillovDenis/feature/155-s3_ceph_listObject_compatibility
[#155] Improved s3 listObjects compatibility
2021-07-15 18:48:26 +03:00
Denis Kirillov
f2eeed0b85 [#155] Fix error handling
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-07-14 19:06:58 +03:00
Denis Kirillov
1ecf32c302 [#155] Fixed invalid max-keys handling
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-07-14 12:21:09 +03:00
Denis Kirillov
9f57756351 [#155] Added s3 url encoder
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-07-14 11:34:49 +03:00
Denis Kirillov
288f6edce8 [#155] Fixed default maxKey value
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-07-13 17:57:48 +03:00
Denis Kirillov
5fb4c4fad6 [#155] Fixed empty delimiter response
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-07-13 17:46:32 +03:00
Denis Kirillov
b004996d51 [#155] Fix KeyCount
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-07-13 17:45:38 +03:00
Denis Kirillov
e1c2cee365 [#152] Removed redundant GetBucketInfo call
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-07-13 14:22:12 +03:00
Alex Vanin
7b1058a9bd Merge pull request #143 from KirillovDenis/bugfix/125-check_bucket_unique
[#125] Fixed bucket creation
2021-07-09 17:44:28 +03:00
Angira Kekteeva
d5dfc0006a [#131] Remove metadata from Object in ListObjects
Because ListObjects v1 and v2 shouldn't show metadata. Also aws cli
doesn't show metadata even we return it.

Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-07-09 13:44:24 +03:00
Angira Kekteeva
da9a6e4b33 [#131] Fix metadata in HeadObject and GetObject
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-07-09 13:44:24 +03:00
Angira Kekteeva
e8bf46a2fa [#131] Add meta to PutObject
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-07-09 13:44:24 +03:00
Denis Kirillov
e78543adf3 [#125] Updated error handling
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-07-09 11:57:44 +03:00
Denis Kirillov
6ba5167f5d [#125] Fixed acl rule
Container must be public (basic acl) to enable bearer token.

Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-07-08 12:22:02 +03:00
Denis Kirillov
0aae8c595a [#125] Fixed bucket creation
Bucket should has unique name.

Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-07-08 12:22:02 +03:00
Alex Vanin
f873be8019 Merge pull request #136 from masterSplinter01/bugfix/135-max-keys-check
[#135] Add max-keys check to ListObjects
2021-07-06 15:03:43 +03:00
Alex Vanin
2956f34fb7 Merge pull request #139 from KirillovDenis/bugfix/133-wrong_time_format
[#133] Fix response time format
2021-07-06 15:03:15 +03:00
Denis Kirillov
bdc7f9acba [#133] Fix response time format
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-07-06 14:07:09 +03:00
Angira Kekteeva
fd41c325f7 [#135] Add max-keys check to ListObjects
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-07-06 12:23:46 +03:00
Angira Kekteeva
aac1ac2868 [#119] Add null versioning and handler
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-07-06 12:11:03 +03:00
Denis Kirillov
95476b1c9c [#95] Supported copy match headers
Supported x-amz-copy-source-if-match
and x-amz-copy-source-if-none-match.

Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-07-02 19:21:53 +03:00
Denis Kirillov
568d7ac810 [#95] Supported get match headers
Supported If-Match and If-None-Match.

Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-07-02 19:05:43 +03:00
Roman Khimov
daed0978a6 Merge pull request #126 from KirillovDenis/feature/94-time_based_conditional
[#94] CopyObject/GetObject support conditional headers
2021-07-02 17:19:16 +03:00
Denis Kirillov
42384730a0 [#121] Fixed directory listing
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-07-01 14:41:20 +03:00
Denis Kirillov
116ffbb438 [#94] Fix time format
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-07-01 14:25:16 +03:00
Denis Kirillov
c4631e5806 [#94] Refactoring
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-07-01 14:24:00 +03:00
Denis Kirillov
ab8dd4201c [#94] CopyObject support conditional headers
Supported X-Amz-Copy-Source-If-Modified-Since and
X-Amz-Copy-Source-If-Unmodified-Since.

Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-07-01 10:45:55 +03:00
Denis Kirillov
da4eca5da5 [#94] GetObject support conditional headers
Supported If-Modified-Since and If-Unmodified-Since.

Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-07-01 08:39:19 +03:00
Denis Kirillov
2af1b16b59 [#93] Removed unnecessary
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-06-30 17:25:46 +03:00
Denis Kirillov
a6ec27b40d [#93] Object ETag support
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-06-30 12:55:31 +03:00
Denis Kirillov
b565045649 [#93] Fixed order headers set
Writing headers had no effect early.

Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-06-30 12:55:31 +03:00
Denis Kirillov
65be38831c [#98] Supported listObjects delimiter
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-06-30 10:50:47 +03:00
Denis Kirillov
23dfa0c46e [#97] List object paging
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-06-28 17:05:14 +03:00
Denis Kirillov
c4fef3d948 [#96] Support ranges for GetObject
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-06-25 12:18:52 +03:00
Angira Kekteeva
3aa9aceda5 [#47] handler, layer: Add DeleteBucket
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-06-23 23:25:00 +03:00
Angira Kekteeva
c12abf6243 [#47] handler, layer: Add CreateBucket
Renamed PutBucket to CreateBucket

Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-06-23 23:21:15 +03:00
Angira Kekteeva
a399590204 Replace s3-gate by s3-gw
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-05-18 14:12:24 +03:00
Roman Khimov
5eb863dc22 *: fix golint warnings about comments to exported things. 2021-05-13 23:25:31 +03:00
Roman Khimov
70a70bfa2c *: fix all godot errors 2021-05-13 22:25:32 +03:00
Roman Khimov
a0ecb8ff52 handler: suppress exhaustive linter 2021-05-13 22:15:21 +03:00
Roman Khimov
b851889934 *: fix whitespace errors 2021-05-13 22:13:09 +03:00
Evgeniy Kulikov
2a93a216f8 Migrate to SDK 0.3.0 and fixes
- fix displaying list objects
- simplify `ListObjects`
- simplify `GetObjectHandler`
- simplify `HeadObjectHandler`
- add method for `GetBucketVersioningHandler`
- add method for `ListMultipartUploadsHandler`
- improvements for `HeadObjectHandler`, to display folders meta
- update dependencies
  - github.com/aws/aws-sdk-go v1.36.26
  - github.com/google/uuid v1.1.4
  - github.com/gorilla/mux v1.8.0
  - github.com/nspcc-dev/cdn-sdk v0.3.0
  - github.com/nspcc-dev/neofs-api-go v1.22.0
  - github.com/prometheus/client_golang v1.9.0
  - github.com/stretchr/testify v1.7.0
  - google.golang.org/grpc v1.35.0

Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2021-01-14 20:39:48 +03:00
Evgeniy Kulikov
776eed76fb Add writer detector
WriterDetector allows to fetch ContentType and write status response, when streaming content

Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-12-10 18:14:32 +03:00
Evgeniy Kulikov
1868034723 Refactoring api/handler
- don't require token.BearerToken
- don't call `GetObjectInfo`

Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-11-27 15:32:52 +03:00
Evgeniy Kulikov
29cb755d49 Fixes of usage auth package
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-11-24 10:02:10 +03:00
Evgeniy Kulikov
256850b8fe [#25] Refactoring and make fixes
closes #25
closes #33

Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-10-24 16:09:22 +03:00
Evgeniy Kulikov
4d605d1113 [#25] Refactoring and make fixes
closes #25
closes #32

Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-10-23 03:21:14 +03:00
Evgeniy Kulikov
b72a957341 [#25] Migrate handler to NeoFS API v2
Naive migration, without any tries to start application

closes #25

Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-10-19 04:05:08 +03:00
Evgeniy Kulikov
8c93dbf867 [#29] Implement ListObjectsV2
ListObjectsV1 and ListObjectsV2 now simplified and return different results

closes #29

Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-08-22 05:49:00 +03:00
Evgeniy Kulikov
079e7a9827 [#28] Implement HeadBucket
closes #28

Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-08-22 05:45:48 +03:00
Evgeniy Kulikov
fd2c6e372b [#26] Implement DeleteMultipleObjects
closes #26

Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-08-22 05:45:15 +03:00
Evgeniy Kulikov
34f6eb7d55 Add note about multiple delete of objects
Could not find how to use it.
I propose open issue and describe how to use it and then implement method.

Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-08-20 22:35:39 +03:00
Evgeniy Kulikov
5bb2c34052 Cleanup unimplemented handlers
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-08-20 02:37:06 +03:00
Evgeniy Kulikov
1d98c4ecc2 Implement GetObject handler
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-08-20 02:36:46 +03:00
Evgeniy Kulikov
bfc4b8786f Implement HeadObject handler
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-08-20 02:36:34 +03:00
Evgeniy Kulikov
9493f3d281 Implement PutObject handler
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-08-20 02:36:17 +03:00
Evgeniy Kulikov
9746978b1d Implement DeleteObject handler
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-08-20 02:36:00 +03:00
Evgeniy Kulikov
9ab0571d29 Implement CopyObject handler
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-08-20 02:35:51 +03:00
Evgeniy Kulikov
c6d4b6d731 Implement MarshalXML for StringMap type
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-08-20 02:34:47 +03:00
Evgeniy Kulikov
fb13d1fcb3 Move ListObjects and ListBuckets in same file
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-08-20 02:33:43 +03:00
Evgeniy Kulikov
cea8f9d4ca Add bucket info handler
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-08-19 14:53:17 +03:00
Evgeniy Kulikov
92b039fa5e Add unsupported handlers
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-08-19 14:52:44 +03:00
Evgeniy Kulikov
b14e2d54c8 Prepare list-objects handler
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-08-11 17:37:20 +03:00
Evgeniy Kulikov
ccd481e4e2 Prepare list-buckets handler
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-08-11 17:36:52 +03:00
Evgeniy Kulikov
d9f691a59a Refactoring API handlers
- add responses for list-objects and list-buckets
- set `implement me` errors more useful

Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-08-11 17:35:54 +03:00
Evgeniy Kulikov
0161d2fbd3 [#13] Rename go module name according to NSPCC standards
- refactoring s3 gate structure
- cleanup unused code
- rename go module to `github.com/nspcc-dev/neofs-s3-gate`

closes #13

Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-08-06 15:02:13 +03:00