4f43aad495
[ #485 ] Upgrade SDK with latest `bearer` package API
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-03 09:38:49 +03:00
12d9eb62cb
[ #489 ] Sanitize log records that may contain user input
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-06-02 19:42:31 +04:00
933ef2bc71
[ #441 ] Remove unused
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-05-31 14:48:58 +03:00
2bca4755f9
[ #441 ] Optimize put objects
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-05-31 14:48:58 +03:00
faa3c65290
[ #462 ] Fix None versioning status
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-05-30 14:44:21 +03:00
0e37242b65
[ #463 ] Move DefaultLocationConstraint constant into api
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-05-30 14:41:32 +03:00
80d4d071d8
[ #463 ] Restrict overriding default location constraint in authmate
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-05-30 14:41:32 +03:00
89ff89a32b
[ #463 ] Define default location constraint
...
Fixes `test_bucket_get_location` from s3-tests.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-05-30 14:41:32 +03:00
087d500c5f
[ #458 ] *: Refactor working with NeoFS identities
...
Pull latest changes from NeoFS SDK Go library. Decrease redundant and
unsafe usage of ID pointers. Use `EncodeToString` method in order to
calculate protocol strings.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-05-27 17:22:37 +04:00
385437aace
[ #441 ] Don't produce extra object.Head request at regular object upload
...
Hash can be calculated locally in S3 gateway.
Creation epoch used for versioning and will be
fetched during get and list requests. To avoid
conflicts, put method do not update cache anymore.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-05-20 10:56:36 +03:00
2575462daa
[ #439 ] Less cache misses for empty bucket setting
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-05-20 10:56:16 +03:00
7d69f9f74b
[ #432 ] Add get-object-attributes
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-05-13 11:42:43 +03:00
b778c2e072
[ #431 ] Fix ETag value
...
ETag is a string with SHA256 of NeoFS object payload.
Hash is taken from object header by neofs-sdk-go getter.
Checksum type in neofs-sdk-go has changes `String()`
output from `<hash>` to `SHA256:<hash>` in latest update.
S3 gateway should not be relied on unstable string format
implementations and use raw value.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-05-12 21:19:59 +04:00
1c33f06bfe
[ #428 ] Update SDK
...
Includes:
- container removal fix
- new session token structure: authmate does not
parse session context anymore, instead it is
application defined flexible structure with
container ID encoded in human-readable format
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-05-06 14:47:06 +04:00
94caa2247e
[ #391 ] Refactor notifications
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-05-06 10:59:31 +03:00
e3c16a32dd
[ #409 ] Update SDK
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-04-26 12:51:52 +04:00
6e91074b50
[ #367 ] Check errors using status
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-04-22 15:30:42 +04:00
bf38007692
[ #405 ] English Check
...
Signed-off-by: Elizaveta Chichindaeva <elizaveta@nspcc.ru>
2022-04-22 14:01:40 +03:00
f274747e83
[ #400 ] Make multipart-upload parts system objects
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-04-19 12:51:04 +03:00
3c5c2f20d8
[ #399 ] Use bearer token for system objects
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-04-13 16:44:32 +04:00
7710de39ec
[ #365 ] Update SDK
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-04-08 17:12:52 +03:00
01c721ee53
[ #380 ] cmd/s3-gw: Refactor logger
...
Do not import `logger` package from NeoFS SDK Go. Discard unusable
`name` and `version` configuration values from `app` section. Discard
all unusable onfiguration values from `logger` section except `level`.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-04-06 17:31:01 +03:00
ed47bc1596
Fixed deletes for failed tests
...
test_multi_object_delete
test_multi_objectv2_delete
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-04-06 09:55:00 +03:00
e319f2422e
[ #357 ] Remove checks of bucket settings is not nil
...
Via #389
Also fixed test
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-04-01 14:57:19 +03:00
a718b92652
[ #357 ] Add ObjectDelete notifications
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-04-01 14:57:19 +03:00
021f5d4dd0
[ #357 ] Refactor delete objects
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-04-01 14:57:19 +03:00
67c1ba2c61
[ #357 ] Add tests
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-04-01 14:57:19 +03:00
371eb2feda
[ #357 ] Add events and sending of events
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-04-01 14:57:19 +03:00
40e7dbf768
[ #357 ] Add test events and check of bucket notif conf
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-04-01 14:57:19 +03:00
2b6843f8fa
[ #389 ] Fix checks of bucket settings
...
In Put/CopyObject and PutBucketVersioning
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-04-01 14:22:10 +04:00
c399cfbdda
[ #382 ] Refactor neofs related types
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-03-31 11:47:24 +03:00
83bb6fc020
[ #384 ] Add check of Notificator interface value
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-03-30 10:29:28 +03:00
f0914b8a43
[ #377 ] Reuse BucketInfo in layer
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-03-29 02:43:28 +04:00
9d19acadcd
[ #195 ] Use exp epoch attribute instead of tick
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-03-23 14:32:13 +04:00
e0b8bc6cd6
[ #195 ] Fix object deletion
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-03-23 14:32:13 +04:00
4a67e4b311
[ #195 ] Set tick attribute to lock objects
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-03-23 14:32:13 +04:00
4c3c4b6bee
[ #195 ] Add Lock tick listening
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-03-23 14:32:13 +04:00
32e83db064
[ #195 ] Set Lock type to system lock objects
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-03-23 14:32:13 +04:00
13080d6d96
[ #195 ] Refactor TestNeoFS
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-03-23 14:32:13 +04:00
5c530123ab
[ #195 ] Fix lint errors
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-03-23 14:32:13 +04:00
6cf01bed14
[ #195 ] Add tests
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-03-23 14:32:13 +04:00
7d6271be8a
[ #195 ] Implement PUT, GET locks to certain object
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-03-23 14:32:13 +04:00
8553158b81
[ #195 ] Add handling lock headers for PUT and COPY
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-03-23 14:32:13 +04:00
b96c3c5a33
[ #195 ] Refactor
...
Using object settings to save bucket versioning
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-03-23 14:32:13 +04:00
e98c663bd6
[ #195 ] Support enabling object locking for bucket
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-03-23 14:32:13 +04:00
94406b08f3
[ #376 ] Update NeoFS SDK to v1.0.0-rc.3
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-03-16 15:44:19 +03:00
02f4524d67
[ #368 ] Unify cache value deletion
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-03-16 10:59:42 +03:00
e7b742df4d
[ #366 ] Add wait for container removed
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-03-04 10:41:15 +03:00
20b8e3d249
[ #346 ] Upgrade NeoFS SDK Go to 2nd release candidate v1.0.0
...
Avoid using the deprecated elements.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-03-04 00:14:30 +03:00
8fb3835250
[ #346 ] api: Do not use `io.Pipe` in `CompleteMultipartUpload`
...
Replace `layer.objectWritePayload` method with `initObjectPayloadReader`
which returns `io.Reader` of the object payload. Copy payload data to
the parameterized `io.Writer` in `layer.GetObject`. Remove `io.Pipe`
from `CompleteMultipartUpload` implementation and build analogue of
`io.MultiReader` for the part list.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-03-04 00:14:30 +03:00
eac4c4d849
[ #346 ] api: Remove unused and no longer needed `layer.Get` method
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-03-04 00:14:30 +03:00
1517b5c638
[ #346 ] neofs: Fix belated `updateCRDT2PSetHeaders` call
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-03-04 00:14:30 +03:00
cd64f41ce8
[ #346 ] *: Refactor communication with NeoFS at the protocol level
...
Make `tokens`, `authmate` and `layer` packages to depend from locally
defined `NeoFS` interface of the virtual connection to NeoFS network.
Create internal `neofs` package and implement these interfaces through
`pool.Pool` there. Implement mediators between `NeoFS` interfaces and
`neofs.NeoFS` implementation.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-03-04 00:14:30 +03:00
34a221c5c9
[ #346 ] Upgrade NeoFS SDK Go library
...
Core changes:
- `object.ID` moved to new package `oid`;
- `object.Address` moved to new package `address`;
- `pool.Object` interface changes.
Additionally:
- Set container owner in `Agent.IssueSecret`.
- Remove no longer needed fields from `GetObjectParams`
- `Length` and `Offset` are never assigned. These values
are set in `Range` field.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-03-04 00:14:30 +03:00
4454821285
[ #340 ] Add notification configuration handlers
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-02-18 16:59:42 +03:00
e0c6544567
[ #340 ] Move notification controller to layer
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-02-18 16:59:42 +03:00
f04815c32f
[ #354 ] Check container owner if bucket exists
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-02-15 17:08:46 +03:00
c383f332d5
[ #329 ] Use suitable tokens for container requests
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-01-31 11:05:35 +03:00
8fc19b11db
[ #333 ] Update sdk
...
Update sdk to fix "invalid session token owner" error
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-01-28 12:42:40 +03:00
1502857dba
[ #313 ] Remove extra QueryUnescapes
...
Removed extra QueryUnescapes in search and objectPut
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-01-27 10:25:11 +03:00
4f5b316211
[ #319 ] Fix tests
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-01-26 16:22:58 +03:00
8985681493
[ #319 ] Delete null versions in different modes
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-01-26 16:22:58 +03:00
f5d365af1d
[ #319 ] Update CRDT headers
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-01-26 16:22:58 +03:00
58df410111
[ #319 ] Head and delete null versions
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-01-26 16:22:58 +03:00
a46726a545
[ #331 ] Use UTC Last-Modified time
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-01-26 16:04:14 +03:00
962d136e73
[ #308 ] Correct access denied status code
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-01-24 18:47:03 +03:00
2b4ed8487b
[ #310 ] Support response headers overriding
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-01-21 16:02:06 +03:00
c202893563
[ #307 ] Filter system metadata
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-01-21 10:28:51 +03:00
8872b6f196
[ #285 ] Add resolving order
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-01-17 18:14:19 +03:00
b9f77b3d96
[ #301 ] Implement GetBucketLocation
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-01-13 20:24:21 +03:00
9b14340816
[ #299 ] Fix creating named containers
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-01-13 13:12:20 +03:00
12c15504ef
[ #297 ] Update neofs-sdk-go and neo-go
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-12-30 13:00:23 +03:00
8127d68128
[ #186 ] Fix errors
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-11-30 11:32:59 +03:00
8c80b16360
[ #186 ] Add unittests for MultipartUpload
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-11-30 11:32:59 +03:00
d6f0ab8ea4
[ #186 ] Fix versioning issues in UploadPart
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-11-30 11:32:59 +03:00
873622d4d5
[ #186 ] Add MultipartUpload support
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-11-30 11:32:59 +03:00
284a560ea6
[ #186 ] Refactor objectSearch, fix objectPut
...
Fixed panic in objectPut() if input reader is nil
Made objectSearch searching by any attribute
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-11-30 11:32:59 +03:00
befe084900
[ #219 ] Add container name resolving
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-11-24 13:24:39 +03:00
d36dfe8c61
[ #271 ] Update neo-sdk-go to the latest version
...
Refactoring invoking pool methods for anonymous requests.
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-11-16 14:42:08 +03:00
ae87effb28
[ #271 ] Add random key for no sign requests
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-11-16 14:42:08 +03:00
91bed76010
[ #274 ] Refactor system cache and cors
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-10-15 17:44:40 +03:00
7d0bc1e992
[ #217 ] Add CORS support
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-10-12 09:03:09 +03:00
ccf5db95a5
[ #217 ] Refactor system objects
...
Moved into a separate file
getSystemObject renamed to headSystemObject, implemented getSystemObject
for system objects with payload
Refactor putSystemObjects
Moved systemCacheKey from data system_object
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-10-12 09:03:09 +03:00
0460225869
[ #272 ] Fix linter warning
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-10-01 12:25:54 +03:00
3231ecab03
[ #263 ] Add LWW to system objects
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-09-23 20:37:55 +03:00
19b917e3b5
[ #253 ] Caches refactoring
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-09-14 10:46:18 +03:00
3c2e25f977
[ #248 ] Support delete marker
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-09-14 09:44:39 +03:00
458f9cf17b
[ #242 ] Fix versions sorting
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-09-07 17:06:22 +03:00
be08596c22
[ #231 ] cache: Add invalidation of ListObjectsCache
...
In put operations ListObjectsCache remove entries which can contain
put object
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-09-07 11:08:48 +03:00
1ece42b23f
[ #236 ] cache: Refactor ListObjectsCache
...
Replaced map in ListObjectsCache by gcache.
Now ListObjectsCache keeps only objectIDs and
requests ObjectInfo from cache or NeoFS.
Refactored ListObjectsCache keys: removed delimiter and method fields.
Now ListObjectsCache keeps cache with all objects versions.
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-09-07 11:08:48 +03:00
1bc2e51cbc
[ #236 ] api: Refactor caches: ObjectsList, Objects
...
Move ObjectsList from layer to cache package
Rename object_cache.go to objects.go
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-09-07 11:08:48 +03:00
239742f413
[ #236 ] api: Create info.go for basic structs
...
Moved BucketInfo and ObjectInfo from layer and handler to api
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-09-07 11:08:48 +03:00
5502fb97c3
[ #196 ] Add tagging caching
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-08-31 10:46:42 +03:00
987185b9e1
[ #196 ] Add bucket tagging
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-08-31 10:46:42 +03:00
16da1aba64
[ #196 ] Add DeleteObjectTagging
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-08-31 10:46:42 +03:00
4a2575fbf3
[ #196 ] Add GetObjectTagging
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-08-31 10:46:42 +03:00
8b5ebe2ec2
[ #196 ] Add PutObjectTagging
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-08-31 10:46:42 +03:00
44a2f1b471
[ #234 ] *: Update go.mod
...
Remove neofs-node dep
Update versions neofs reps
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-08-30 17:01:41 +03:00
dcac1aa3e3
[ #230 ] layer: Change objects_list_cache's key
...
Remove access_key from objectsListCache key
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-08-26 12:54:58 +03:00
4bb885d526
[ #122 ] Refactor
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-08-25 14:49:21 +03:00
11558124cd
[ #122 ] Add versioning cache
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-08-25 09:15:00 +03:00
f6c51cc9ee
[ #122 ] Update listObjectVerions
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-08-25 09:15:00 +03:00
9c058a70fd
[ #122 ] Add tests
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-08-25 09:15:00 +03:00
43185de52a
[ #122 ] Add list object versions
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-08-25 09:15:00 +03:00
d5aef7566f
[ #122 ] Add delete versioned object
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-08-25 09:15:00 +03:00
3130784ee6
[ #122 ] Add getting specific object version
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-08-25 09:13:12 +03:00
f463522f34
[ #122 ] Add versioning put object
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-08-25 09:10:46 +03:00
feb45d0633
[ #122 ] Add replacing objects
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-08-25 09:10:46 +03:00
c50a16a5e3
[ #122 ] Add enabling versioning
...
New handlers:
PutBucketVersioning, GetBucketVersioning
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-08-25 09:10:46 +03:00
efe11c271f
[ #49 ] Add basic ACL translation
...
Implement functions:
GetBucketACL, PutBucketACL, GetObjectACL,
PutObjectACL, GetBucketPolicy, PutBucketPolicy
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-08-18 17:20:17 +03:00
b1c6629b10
[ #206 ] Make caches' options configurable
...
Now caches' options can be configured in .yanl file.
Export caches' variables (listobjects and objects).
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-08-18 10:16:27 +03:00
295be71f00
[ #206 ] api: Add objects cache
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-08-18 10:16:27 +03:00
ae19eb4ad4
[ #206 ] Refactor ListObjectsCache
...
Replaced conflicting names
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-08-18 10:16:27 +03:00
d332096598
[ #199 ] Refactor
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-08-11 13:17:37 +03:00
3cdcbf6e21
[ #179 ] Add unittests for cache
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-08-09 13:52:11 +03:00
8a69c7cca0
[ #179 ] api: Add cache for ListObjectsV1/V2
...
Refactored cache for ListObjects:
made cache common for all versions,
simplified: remove dependendence on token/startafter
add mitable cachelifetime.
Refactored listobjects
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-08-09 13:52:11 +03:00
0ceea95e11
[ #112 ] Add cache to ListObjects and layer
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-07-28 17:05:11 +03:00
c24fe5cc21
[ #112 ] Move getBoxData from handler to layer
...
And made it exported
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-07-28 16:27:06 +03:00
97a7d16f68
[ #112 ] api: Refactor allObjectsParams struct
...
Replaced Bucket (means name) by BucketInfo struct
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-07-26 23:22:05 +03:00
f4cd1e4c38
Merge pull request #170 from KirillovDenis/feature/89-placement_policy
...
[#89 ] Add placement policy
2021-07-26 13:43:38 +03:00
8aac61652f
[ #89 ] Move BoxData to CreateBucketParams
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-07-26 13:35:52 +03:00
3f7941fdf3
Merge pull request #174 from nspcc-dev/feature/159-optional_owner
...
[#159 ] Add fetch-owner param
2021-07-26 13:32:04 +03:00
28974474f2
[ #149 ] Refactoring
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-07-22 09:55:35 +03:00
63652d213c
[ #149 ] Removed grpc errors
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-07-22 09:54:12 +03:00
65a61a9f7d
[ #159 ] Add fetch-owner param
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-07-20 15:40:38 +03:00
c05e2d6757
[ #160 ] Fix head dir object
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-07-20 12:36:41 +03:00
ee84062154
[ #154 ] api: refactor ListObjectsV1 and V2
...
Separate ListObject for different versions.
Remove useless grouping of keys on V2.
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-07-19 10:12:47 +03:00
1fbd192bd7
[ #89 ] Add placement policy
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-07-16 16:01:35 +03:00
1ecf32c302
[ #155 ] Fixed invalid max-keys handling
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-07-14 12:21:09 +03:00
7b1058a9bd
Merge pull request #143 from KirillovDenis/bugfix/125-check_bucket_unique
...
[#125 ] Fixed bucket creation
2021-07-09 17:44:28 +03:00
acc2cdd96c
[ #125 ] Added exact eacl checking
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-07-09 17:06:35 +03:00
e78543adf3
[ #125 ] Updated error handling
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-07-09 11:57:44 +03:00
d09a4e4e6c
[ #145 ] Fix util tests
...
Fixed contentType attribute difference in expected and actual results
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-07-09 00:36:19 +03:00
e11b1b76ba
[ #125 ] Added eacl to newly created container
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-07-08 13:10:46 +03:00
0aae8c595a
[ #125 ] Fixed bucket creation
...
Bucket should has unique name.
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-07-08 12:22:02 +03:00
aac1ac2868
[ #119 ] Add null versioning and handler
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-07-06 12:11:03 +03:00
e145effd17
[ #129 ] Fix root objects listing
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-07-04 23:25:53 +03:00
daed0978a6
Merge pull request #126 from KirillovDenis/feature/94-time_based_conditional
...
[#94 ] CopyObject/GetObject support conditional headers
2021-07-02 17:19:16 +03:00
42384730a0
[ #121 ] Fixed directory listing
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-07-01 14:41:20 +03:00
ab8dd4201c
[ #94 ] CopyObject support conditional headers
...
Supported X-Amz-Copy-Source-If-Modified-Since and
X-Amz-Copy-Source-If-Unmodified-Since.
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-07-01 10:45:55 +03:00
2af1b16b59
[ #93 ] Removed unnecessary
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-06-30 17:25:46 +03:00
a6ec27b40d
[ #93 ] Object ETag support
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-06-30 12:55:31 +03:00
65be38831c
[ #98 ] Supported listObjects delimiter
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-06-30 10:50:47 +03:00
23dfa0c46e
[ #97 ] List object paging
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-06-28 17:05:14 +03:00
c4fef3d948
[ #96 ] Support ranges for GetObject
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-06-25 12:18:52 +03:00
26d6fd95c7
[ #47 ] authmate, layer: use sdk-go Container impl
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-06-23 23:36:39 +03:00
3aa9aceda5
[ #47 ] handler, layer: Add DeleteBucket
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-06-23 23:25:00 +03:00
c12abf6243
[ #47 ] handler, layer: Add CreateBucket
...
Renamed PutBucket to CreateBucket
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-06-23 23:21:15 +03:00
95f75ec880
[ #47 ] Add session token to context
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-06-21 13:56:26 +03:00
6a135a6bb4
[ #82 ] Using bearer token
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-06-15 17:16:26 +03:00
fba29a9a66
[ #78 ] Using client.Object from sdk
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-06-15 17:06:19 +03:00
47fc10a869
[ #65 ] Using containerList falls back
...
Looking container up using the owner list if its name is not cid
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-06-13 14:32:13 +03:00
8185b71462
[ #65 ] Allow no sign requests
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-06-11 14:52:03 +03:00
b3b190adc0
[ #68 ] *: Replace deprecated
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-06-04 16:01:42 +03:00
d15687f37c
*: drop creds/neofs
...
Signed-off-by: Roman Khimov <roman@nspcc.ru>
2021-05-28 23:48:39 +03:00
35ffc1fbc1
*: use sdk-go instead of http-gw
...
Now that the packages have moved there we can do it.
Signed-off-by: Roman Khimov <roman@nspcc.ru>
2021-05-28 14:05:46 +03:00
d19ce03072
*: drop old sdk dependecies, bump neofs-api-go version
...
I'm not sure it works, but it's enough code-wise for now. We're reusing some
http-gw components here that are to be moved into sdk-go in future.
Signed-off-by: Roman Khimov <roman@nspcc.ru>
2021-05-26 21:01:46 +03:00
dbe65ae602
creds: move credential management into s3 gate
...
Mostly taken from old SDK (abe47687cd11266f946cad57f07572cc10c67226), but
error handling adapted to eliminate pkg/errors and internal packages.
Signed-off-by: Roman Khimov <roman@nspcc.ru>
2021-05-25 23:00:19 +03:00
9d496d70a7
[ #53 ] *: Remove external pkg/errors dependency
...
Replaced functions from pkg/errors by functions from errors, fixed not fully correct comment
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-05-20 15:50:14 +03:00
a399590204
Replace s3-gate by s3-gw
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-05-18 14:12:24 +03:00
5eb863dc22
*: fix golint warnings about comments to exported things.
2021-05-13 23:25:31 +03:00
70a70bfa2c
*: fix all godot errors
2021-05-13 22:25:32 +03:00
2a93a216f8
Migrate to SDK 0.3.0 and fixes
...
- fix displaying list objects
- simplify `ListObjects`
- simplify `GetObjectHandler`
- simplify `HeadObjectHandler`
- add method for `GetBucketVersioningHandler`
- add method for `ListMultipartUploadsHandler`
- improvements for `HeadObjectHandler`, to display folders meta
- update dependencies
- github.com/aws/aws-sdk-go v1.36.26
- github.com/google/uuid v1.1.4
- github.com/gorilla/mux v1.8.0
- github.com/nspcc-dev/cdn-sdk v0.3.0
- github.com/nspcc-dev/neofs-api-go v1.22.0
- github.com/prometheus/client_golang v1.9.0
- github.com/stretchr/testify v1.7.0
- google.golang.org/grpc v1.35.0
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2021-01-14 20:39:48 +03:00
8ff289ea4f
Simplify OffsetWriter
...
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-12-10 18:15:43 +03:00
8168181658
Remove bufio writer and double creation of OffsetWriter
...
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-12-10 18:15:18 +03:00
f4263d50a7
Refactoring api/layer
...
- don't require token.BearerToken
- simplify method layer.objectPut
- add method to check that object exists
- add method to fetch Owner (from context or sdk client)
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-11-27 15:36:15 +03:00
31d3e55489
Add content detector
...
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-11-27 15:33:31 +03:00
0284bf136b
Migrate to Credentials client
...
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-11-24 12:17:36 +03:00
0ee3a5f9ba
Migrate api/layer to SDK
...
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-11-24 10:01:38 +03:00
256850b8fe
[ #25 ] Refactoring and make fixes
...
closes #25
closes #33
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-10-24 16:09:22 +03:00
4d605d1113
[ #25 ] Refactoring and make fixes
...
closes #25
closes #32
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-10-23 03:21:14 +03:00
fbd4a83602
[ #25 ] Refactoring over api/layer
...
closes #25
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-10-22 03:19:16 +03:00
7deeb68d47
[ #25 ] Migrate layer to NeoFS API v2
...
Naive migration, without any tries to start application
closes #25
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-10-19 04:04:37 +03:00
cfe2811efc
Add owner field for ListObjects method
...
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-08-22 05:43:53 +03:00
3244f54022
Change returned error for GetBucketInfo method
...
method should returns gRPC status errors
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-08-22 05:43:06 +03:00
8eff7d4dd3
Refactoring DeleteObject/DeleteObjects methods
...
- methods should return api.DeleteErrors
- DeleteObjects should return only slice of errors
- don't add nil errors
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-08-22 05:42:27 +03:00
960aa02bd3
Refactoring DeleteObject/DeleteObjects methods
...
- methods should return api.DeleteErrors
- DeleteObjects should return only slice of errors
- don't add nil errors
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-08-22 05:41:58 +03:00
5eae2aac78
Change NotFound error for objectFindID
...
method should return gRPC status error
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-08-22 05:39:12 +03:00
5258342881
Add Owner field into layer.ObjectInfo
...
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-08-22 05:37:39 +03:00
0bea843d90
Fix DeleteObject handler
...
DeleteObject should remove all objects with passed nice name
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-08-20 22:41:52 +03:00
21591ebe65
Add method to find all ID's for the nice names
...
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-08-20 22:33:59 +03:00
edd44ea981
Check that aws filename wasn't set
...
we should check that filename header wasn't set, otherwise we should not change custom filename
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-08-20 02:32:42 +03:00
00a17d8316
CopyObject should contains custom headers
...
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-08-20 02:31:04 +03:00
fb00af23fb
Fixed possible NPE in PutObject
...
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-08-20 02:30:18 +03:00
64429d736d
Refactoring API layer
...
- logging RequestID
- should return error, when headers already received
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-08-11 14:32:04 +03:00
0161d2fbd3
[ #13 ] Rename go module name according to NSPCC standards
...
- refactoring s3 gate structure
- cleanup unused code
- rename go module to `github.com/nspcc-dev/neofs-s3-gate`
closes #13
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-08-06 15:02:13 +03:00
Leonard Lyubich
Alex Vanin
Denis Kirillov
Angira Kekteeva
Elizaveta Chichindaeva
Roman Khimov
Evgeniy Kulikov