11558124cd
[ #122 ] Add versioning cache
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-08-25 09:15:00 +03:00
f6c51cc9ee
[ #122 ] Update listObjectVerions
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-08-25 09:15:00 +03:00
9c058a70fd
[ #122 ] Add tests
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-08-25 09:15:00 +03:00
43185de52a
[ #122 ] Add list object versions
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-08-25 09:15:00 +03:00
d5aef7566f
[ #122 ] Add delete versioned object
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-08-25 09:15:00 +03:00
3130784ee6
[ #122 ] Add getting specific object version
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-08-25 09:13:12 +03:00
f463522f34
[ #122 ] Add versioning put object
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-08-25 09:10:46 +03:00
feb45d0633
[ #122 ] Add replacing objects
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-08-25 09:10:46 +03:00
c50a16a5e3
[ #122 ] Add enabling versioning
...
New handlers:
PutBucketVersioning, GetBucketVersioning
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-08-25 09:10:46 +03:00
efe11c271f
[ #49 ] Add basic ACL translation
...
Implement functions:
GetBucketACL, PutBucketACL, GetObjectACL,
PutObjectACL, GetBucketPolicy, PutBucketPolicy
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-08-18 17:20:17 +03:00
b1c6629b10
[ #206 ] Make caches' options configurable
...
Now caches' options can be configured in .yanl file.
Export caches' variables (listobjects and objects).
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-08-18 10:16:27 +03:00
295be71f00
[ #206 ] api: Add objects cache
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-08-18 10:16:27 +03:00
ae19eb4ad4
[ #206 ] Refactor ListObjectsCache
...
Replaced conflicting names
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-08-18 10:16:27 +03:00
d332096598
[ #199 ] Refactor
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-08-11 13:17:37 +03:00
3cdcbf6e21
[ #179 ] Add unittests for cache
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-08-09 13:52:11 +03:00
8a69c7cca0
[ #179 ] api: Add cache for ListObjectsV1/V2
...
Refactored cache for ListObjects:
made cache common for all versions,
simplified: remove dependendence on token/startafter
add mitable cachelifetime.
Refactored listobjects
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-08-09 13:52:11 +03:00
0ceea95e11
[ #112 ] Add cache to ListObjects and layer
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-07-28 17:05:11 +03:00
c24fe5cc21
[ #112 ] Move getBoxData from handler to layer
...
And made it exported
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-07-28 16:27:06 +03:00
97a7d16f68
[ #112 ] api: Refactor allObjectsParams struct
...
Replaced Bucket (means name) by BucketInfo struct
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-07-26 23:22:05 +03:00
f4cd1e4c38
Merge pull request #170 from KirillovDenis/feature/89-placement_policy
...
[#89 ] Add placement policy
2021-07-26 13:43:38 +03:00
8aac61652f
[ #89 ] Move BoxData to CreateBucketParams
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-07-26 13:35:52 +03:00
3f7941fdf3
Merge pull request #174 from nspcc-dev/feature/159-optional_owner
...
[#159 ] Add fetch-owner param
2021-07-26 13:32:04 +03:00
28974474f2
[ #149 ] Refactoring
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-07-22 09:55:35 +03:00
63652d213c
[ #149 ] Removed grpc errors
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-07-22 09:54:12 +03:00
65a61a9f7d
[ #159 ] Add fetch-owner param
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-07-20 15:40:38 +03:00
c05e2d6757
[ #160 ] Fix head dir object
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-07-20 12:36:41 +03:00
ee84062154
[ #154 ] api: refactor ListObjectsV1 and V2
...
Separate ListObject for different versions.
Remove useless grouping of keys on V2.
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-07-19 10:12:47 +03:00
1fbd192bd7
[ #89 ] Add placement policy
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-07-16 16:01:35 +03:00
1ecf32c302
[ #155 ] Fixed invalid max-keys handling
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-07-14 12:21:09 +03:00
7b1058a9bd
Merge pull request #143 from KirillovDenis/bugfix/125-check_bucket_unique
...
[#125 ] Fixed bucket creation
2021-07-09 17:44:28 +03:00
acc2cdd96c
[ #125 ] Added exact eacl checking
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-07-09 17:06:35 +03:00
e78543adf3
[ #125 ] Updated error handling
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-07-09 11:57:44 +03:00
d09a4e4e6c
[ #145 ] Fix util tests
...
Fixed contentType attribute difference in expected and actual results
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-07-09 00:36:19 +03:00
e11b1b76ba
[ #125 ] Added eacl to newly created container
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-07-08 13:10:46 +03:00
0aae8c595a
[ #125 ] Fixed bucket creation
...
Bucket should has unique name.
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-07-08 12:22:02 +03:00
aac1ac2868
[ #119 ] Add null versioning and handler
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-07-06 12:11:03 +03:00
e145effd17
[ #129 ] Fix root objects listing
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-07-04 23:25:53 +03:00
daed0978a6
Merge pull request #126 from KirillovDenis/feature/94-time_based_conditional
...
[#94 ] CopyObject/GetObject support conditional headers
2021-07-02 17:19:16 +03:00
42384730a0
[ #121 ] Fixed directory listing
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-07-01 14:41:20 +03:00
ab8dd4201c
[ #94 ] CopyObject support conditional headers
...
Supported X-Amz-Copy-Source-If-Modified-Since and
X-Amz-Copy-Source-If-Unmodified-Since.
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-07-01 10:45:55 +03:00
2af1b16b59
[ #93 ] Removed unnecessary
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-06-30 17:25:46 +03:00
a6ec27b40d
[ #93 ] Object ETag support
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-06-30 12:55:31 +03:00
65be38831c
[ #98 ] Supported listObjects delimiter
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-06-30 10:50:47 +03:00
23dfa0c46e
[ #97 ] List object paging
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-06-28 17:05:14 +03:00
c4fef3d948
[ #96 ] Support ranges for GetObject
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-06-25 12:18:52 +03:00
26d6fd95c7
[ #47 ] authmate, layer: use sdk-go Container impl
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-06-23 23:36:39 +03:00
3aa9aceda5
[ #47 ] handler, layer: Add DeleteBucket
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-06-23 23:25:00 +03:00
c12abf6243
[ #47 ] handler, layer: Add CreateBucket
...
Renamed PutBucket to CreateBucket
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-06-23 23:21:15 +03:00
95f75ec880
[ #47 ] Add session token to context
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-06-21 13:56:26 +03:00
6a135a6bb4
[ #82 ] Using bearer token
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-06-15 17:16:26 +03:00
fba29a9a66
[ #78 ] Using client.Object from sdk
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-06-15 17:06:19 +03:00
47fc10a869
[ #65 ] Using containerList falls back
...
Looking container up using the owner list if its name is not cid
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-06-13 14:32:13 +03:00
8185b71462
[ #65 ] Allow no sign requests
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-06-11 14:52:03 +03:00
b3b190adc0
[ #68 ] *: Replace deprecated
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-06-04 16:01:42 +03:00
d15687f37c
*: drop creds/neofs
...
Signed-off-by: Roman Khimov <roman@nspcc.ru>
2021-05-28 23:48:39 +03:00
35ffc1fbc1
*: use sdk-go instead of http-gw
...
Now that the packages have moved there we can do it.
Signed-off-by: Roman Khimov <roman@nspcc.ru>
2021-05-28 14:05:46 +03:00
d19ce03072
*: drop old sdk dependecies, bump neofs-api-go version
...
I'm not sure it works, but it's enough code-wise for now. We're reusing some
http-gw components here that are to be moved into sdk-go in future.
Signed-off-by: Roman Khimov <roman@nspcc.ru>
2021-05-26 21:01:46 +03:00
dbe65ae602
creds: move credential management into s3 gate
...
Mostly taken from old SDK (abe47687cd11266f946cad57f07572cc10c67226), but
error handling adapted to eliminate pkg/errors and internal packages.
Signed-off-by: Roman Khimov <roman@nspcc.ru>
2021-05-25 23:00:19 +03:00
9d496d70a7
[ #53 ] *: Remove external pkg/errors dependency
...
Replaced functions from pkg/errors by functions from errors, fixed not fully correct comment
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-05-20 15:50:14 +03:00
a399590204
Replace s3-gate by s3-gw
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-05-18 14:12:24 +03:00
5eb863dc22
*: fix golint warnings about comments to exported things.
2021-05-13 23:25:31 +03:00
70a70bfa2c
*: fix all godot errors
2021-05-13 22:25:32 +03:00
2a93a216f8
Migrate to SDK 0.3.0 and fixes
...
- fix displaying list objects
- simplify `ListObjects`
- simplify `GetObjectHandler`
- simplify `HeadObjectHandler`
- add method for `GetBucketVersioningHandler`
- add method for `ListMultipartUploadsHandler`
- improvements for `HeadObjectHandler`, to display folders meta
- update dependencies
- github.com/aws/aws-sdk-go v1.36.26
- github.com/google/uuid v1.1.4
- github.com/gorilla/mux v1.8.0
- github.com/nspcc-dev/cdn-sdk v0.3.0
- github.com/nspcc-dev/neofs-api-go v1.22.0
- github.com/prometheus/client_golang v1.9.0
- github.com/stretchr/testify v1.7.0
- google.golang.org/grpc v1.35.0
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2021-01-14 20:39:48 +03:00
8ff289ea4f
Simplify OffsetWriter
...
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-12-10 18:15:43 +03:00
8168181658
Remove bufio writer and double creation of OffsetWriter
...
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-12-10 18:15:18 +03:00
f4263d50a7
Refactoring api/layer
...
- don't require token.BearerToken
- simplify method layer.objectPut
- add method to check that object exists
- add method to fetch Owner (from context or sdk client)
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-11-27 15:36:15 +03:00
31d3e55489
Add content detector
...
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-11-27 15:33:31 +03:00
0284bf136b
Migrate to Credentials client
...
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-11-24 12:17:36 +03:00
0ee3a5f9ba
Migrate api/layer to SDK
...
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-11-24 10:01:38 +03:00
256850b8fe
[ #25 ] Refactoring and make fixes
...
closes #25
closes #33
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-10-24 16:09:22 +03:00
4d605d1113
[ #25 ] Refactoring and make fixes
...
closes #25
closes #32
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-10-23 03:21:14 +03:00
fbd4a83602
[ #25 ] Refactoring over api/layer
...
closes #25
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-10-22 03:19:16 +03:00
7deeb68d47
[ #25 ] Migrate layer to NeoFS API v2
...
Naive migration, without any tries to start application
closes #25
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-10-19 04:04:37 +03:00
cfe2811efc
Add owner field for ListObjects method
...
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-08-22 05:43:53 +03:00
3244f54022
Change returned error for GetBucketInfo method
...
method should returns gRPC status errors
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-08-22 05:43:06 +03:00
8eff7d4dd3
Refactoring DeleteObject/DeleteObjects methods
...
- methods should return api.DeleteErrors
- DeleteObjects should return only slice of errors
- don't add nil errors
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-08-22 05:42:27 +03:00
960aa02bd3
Refactoring DeleteObject/DeleteObjects methods
...
- methods should return api.DeleteErrors
- DeleteObjects should return only slice of errors
- don't add nil errors
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-08-22 05:41:58 +03:00
5eae2aac78
Change NotFound error for objectFindID
...
method should return gRPC status error
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-08-22 05:39:12 +03:00
5258342881
Add Owner field into layer.ObjectInfo
...
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-08-22 05:37:39 +03:00
0bea843d90
Fix DeleteObject handler
...
DeleteObject should remove all objects with passed nice name
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-08-20 22:41:52 +03:00
21591ebe65
Add method to find all ID's for the nice names
...
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-08-20 22:33:59 +03:00
edd44ea981
Check that aws filename wasn't set
...
we should check that filename header wasn't set, otherwise we should not change custom filename
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-08-20 02:32:42 +03:00
00a17d8316
CopyObject should contains custom headers
...
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-08-20 02:31:04 +03:00
fb00af23fb
Fixed possible NPE in PutObject
...
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-08-20 02:30:18 +03:00
64429d736d
Refactoring API layer
...
- logging RequestID
- should return error, when headers already received
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-08-11 14:32:04 +03:00
0161d2fbd3
[ #13 ] Rename go module name according to NSPCC standards
...
- refactoring s3 gate structure
- cleanup unused code
- rename go module to `github.com/nspcc-dev/neofs-s3-gate`
closes #13
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-08-06 15:02:13 +03:00
Denis Kirillov
Angira Kekteeva
Alex Vanin
Roman Khimov
Evgeniy Kulikov