Commit graph

354 commits

Author SHA1 Message Date
Denis Kirillov
9f740b9683 [#289] Add detect mimetype by extension
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-06-29 19:13:16 +03:00
Denis Kirillov
f72bc538b9 [#551] Refactor notifications logs
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-06-27 17:31:17 +03:00
Denis Kirillov
35f55c5af5 [#539] Fix tests
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-06-27 02:23:19 +04:00
Denis Kirillov
7ca519cb32 [#539] Add context to errors
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-06-27 02:23:19 +04:00
Leonard Lyubich
e1f1e6d960 [#541] Upgrade NeoFS SDK Go with changed basic ACL API
SDK now provides dedicated type for basic ACL with convenient interface.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-24 16:43:50 +03:00
Leonard Lyubich
a8fc313ff5 [#544] layer: Remove no longer needed deleteContainer method
Call `DeleteContainer` on `NeoFS` component directly.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-24 13:00:26 +03:00
Leonard Lyubich
f596c8be06 [#544] layer: Add session token parameter to DeleteBucketParams
Add `DeleteBucketParams.SessionToken` field in order to unify the
parameters with `CreateBucketParams` and `PutBucketACLParams`.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-24 13:00:26 +03:00
Leonard Lyubich
028a152e04 [#544] Upgrade NeoFS SDK Go with another approach of container sessions
After recent changes in NeoFS SDK Go library session tokens aren't
embedded into `container.Container` and `eacl.Table` structures.
Instead, the operations of storing given values in NeoFS are
parameterized by elements of the corresponding type.

Add dedicated session parameters to operations of bucket and eACL
setting.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-24 13:00:26 +03:00
Denis Kirillov
818176e7e1 [#528] Adopt aws v4signer
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-06-20 09:44:25 +03:00
Leonard Lyubich
f0749fd23e [#537] Upgrade NeoFS SDK Go with changed netmap package
`PlacementPolicy` type now provides methods to work with QL-encoded
policies. System network parameters can be read using dedicated method
without iterating. Applications can work with `PlacementPolicy`
variables directly so there is no need to use pointers.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-17 08:28:59 +03:00
Angira Kekteeva
cfe7591cf7 [#523] Add putObjectACL notification
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-06-17 01:43:46 +04:00
Angira Kekteeva
dd0d21b690 [#523] Fix typo
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-06-17 01:43:46 +04:00
Angira Kekteeva
bd5fd041b7 [#523] Add object tagging notifications
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-06-17 01:43:46 +04:00
Denis Kirillov
d521af2065 [#529] Add presign URLs support
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-06-16 17:17:45 +03:00
Leonard Lyubich
880ffe7108 [#492] layer: Don't parameterize basic ACL of created containers
`CreateBucket` handler always creates containers with extended public
ACL, so there is no need to configure it in `NeoFS.CreateContainer`.

Make internal `NeoFS` implementation to create containers with
`eacl-public-read-write` basic ACL if corresponding parameter is unset.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-16 10:46:50 +03:00
Angira Kekteeva
a8bff13801 [#495] Add acl to copy-object
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-06-07 17:36:12 +03:00
Denis Kirillov
de7281ac58 [#465] Unify log messages
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-06-07 17:20:26 +03:00
Denis Kirillov
300d4359d8 [#465] Handle cache cast failure
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-06-07 17:20:26 +03:00
Denis Kirillov
f00ca1b6c4 [#438] Drop layer/neofs package
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-06-07 13:59:45 +03:00
Alex Vanin
4ed939773b [#487] Use updated SDK method to get bearer issuer
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-06-03 11:20:47 +03:00
Angira Kekteeva
dbfac29171 [#487] Fix tests
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-06-03 10:57:56 +03:00
Angira Kekteeva
e904ed51c7 [#487] Optimize bucketInfo in initObjectPayloadReader
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-06-03 10:57:56 +03:00
Angira Kekteeva
4767eeed8c [#487] Remove attach of bearer token
When bucket owner is not an issuer of the bearer token

Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-06-03 10:57:56 +03:00
Leonard Lyubich
4f43aad495 [#485] Upgrade SDK with latest bearer package API
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-03 09:38:49 +03:00
Denis Kirillov
ea252421f5 [#484] Add tests
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-06-03 09:35:06 +03:00
Denis Kirillov
f282e877e2 [#484] Handle conditional headers
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-06-03 09:35:06 +03:00
Alex Vanin
ee0f3fb196 [#489] Avoid sensitive data logging
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-06-02 19:42:31 +04:00
Alex Vanin
12d9eb62cb [#489] Sanitize log records that may contain user input
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-06-02 19:42:31 +04:00
Denis Kirillov
933ef2bc71 [#441] Remove unused
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-05-31 14:48:58 +03:00
Denis Kirillov
2bca4755f9 [#441] Optimize put objects
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-05-31 14:48:58 +03:00
Denis Kirillov
faa3c65290 [#462] Fix None versioning status
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-05-30 14:44:21 +03:00
Alex Vanin
0e37242b65 [#463] Move DefaultLocationConstraint constant into api
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-05-30 14:41:32 +03:00
Alex Vanin
80d4d071d8 [#463] Restrict overriding default location constraint in authmate
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-05-30 14:41:32 +03:00
Alex Vanin
89ff89a32b [#463] Define default location constraint
Fixes `test_bucket_get_location` from s3-tests.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-05-30 14:41:32 +03:00
Leonard Lyubich
087d500c5f [#458] *: Refactor working with NeoFS identities
Pull latest changes from NeoFS SDK Go library. Decrease redundant and
unsafe usage of ID pointers. Use `EncodeToString` method in order to
calculate protocol strings.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-05-27 17:22:37 +04:00
Alex Vanin
80c6122f55 [#450] Do not pass nil error to logAndSendError
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-05-25 00:36:42 +04:00
Alex Vanin
385437aace [#441] Don't produce extra object.Head request at regular object upload
Hash can be calculated locally in S3 gateway.
Creation epoch used for versioning and will be
fetched during get and list requests. To avoid
conflicts, put method do not update cache anymore.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-05-20 10:56:36 +03:00
Alex Vanin
2575462daa [#439] Less cache misses for empty bucket setting
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-05-20 10:56:16 +03:00
Angira Kekteeva
7d69f9f74b [#432] Add get-object-attributes
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-05-13 11:42:43 +03:00
Alex Vanin
b778c2e072 [#431] Fix ETag value
ETag is a string with SHA256 of NeoFS object payload.
Hash is taken from object header by neofs-sdk-go getter.
Checksum type in neofs-sdk-go has changes `String()`
output from `<hash>` to `SHA256:<hash>` in latest update.

S3 gateway should not be relied on unstable string format
implementations and use raw value.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-05-12 21:19:59 +04:00
Alex Vanin
1c33f06bfe [#428] Update SDK
Includes:
- container removal fix
- new session token structure: authmate does not
  parse session context anymore, instead it is
  application defined flexible structure with
  container ID encoded in human-readable format

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-05-06 14:47:06 +04:00
Denis Kirillov
94caa2247e [#391] Refactor notifications
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-05-06 10:59:31 +03:00
Denis Kirillov
e3c16a32dd [#409] Update SDK
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-04-26 12:51:52 +04:00
Denis Kirillov
6e91074b50 [#367] Check errors using status
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-04-22 15:30:42 +04:00
Elizaveta Chichindaeva
bf38007692 [#405] English Check
Signed-off-by: Elizaveta Chichindaeva <elizaveta@nspcc.ru>
2022-04-22 14:01:40 +03:00
Angira Kekteeva
a0a04a73bd [#347] Add setEACL session token checks
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-04-20 12:42:33 +03:00
Angira Kekteeva
f274747e83 [#400] Make multipart-upload parts system objects
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-04-19 12:51:04 +03:00
Denis Kirillov
3c5c2f20d8 [#399] Use bearer token for system objects
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-04-13 16:44:32 +04:00
Denis Kirillov
7710de39ec [#365] Update SDK
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-04-08 17:12:52 +03:00
Leonard Lyubich
01c721ee53 [#380] cmd/s3-gw: Refactor logger
Do not import `logger` package from NeoFS SDK Go. Discard unusable
`name` and `version` configuration values from `app` section. Discard
all unusable onfiguration values from `logger` section except `level`.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-04-06 17:31:01 +03:00