Commit Graph

89 Commits (v0.28.1)

Author SHA1 Message Date
Marina Biryukova aa2c016f83 [#205] Add md5 checksum in ETag by config param
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-12-07 12:20:14 +00:00
Roman Loginov 8efcc957ea [#96] Move log messages to constants
/ DCO (pull_request) Successful in 1m35s Details
/ Builds (1.19) (pull_request) Successful in 2m14s Details
/ Builds (1.20) (pull_request) Successful in 2m9s Details
/ Vulncheck (pull_request) Successful in 5m39s Details
/ Lint (pull_request) Successful in 2m49s Details
/ Tests (1.19) (pull_request) Successful in 7m34s Details
/ Tests (1.20) (pull_request) Successful in 1m44s Details
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
2023-08-23 18:32:31 +03:00
Roman Loginov 40d7f844e3 [#137] Refactor context data retrievers
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
2023-08-16 14:05:38 +00:00
Denis Kirillov b59aa06637 [#146] Add kludge.bypass_content_encoding_check_in_chunks flag
/ Builds (1.19) (pull_request) Successful in 3m0s Details
/ Builds (1.20) (pull_request) Successful in 2m50s Details
/ DCO (pull_request) Failing after 1m9s Details
/ Vulncheck (pull_request) Successful in 1m20s Details
/ Lint (pull_request) Successful in 7m50s Details
/ Tests (1.19) (pull_request) Successful in 3m1s Details
/ Tests (1.20) (pull_request) Successful in 3m21s Details
Flag allows to skip checking `Content-Encoding` for `aws-chunked` value

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-18 14:49:52 +03:00
Denis Kirillov 751a9be7cc [#146] Move getting chunk payload reader to separate function
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-18 14:49:03 +03:00
Denis Kirillov fc90981c03 [#149] Update inner imports after moving middlewares
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-11 17:25:09 +03:00
Artem Tataurov 614d703726 [#106] Add chunk uploading
/ DCO (pull_request) Successful in 1m4s Details
/ Vulncheck (pull_request) Successful in 1m50s Details
/ Builds (1.19) (pull_request) Successful in 9m1s Details
/ Builds (1.20) (pull_request) Successful in 2m20s Details
/ Lint (pull_request) Successful in 10m19s Details
/ Tests (1.19) (pull_request) Successful in 2m45s Details
/ Tests (1.20) (pull_request) Successful in 3m19s Details
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2023-06-21 10:23:57 +03:00
Denis Kirillov 23593eee3d [#111] Use request scope logger
/ Lint (pull_request) Failing after 32s Details
/ Tests (1.19) (pull_request) Failing after 32s Details
/ Tests (1.20) (pull_request) Failing after 32s Details
/ Builds (1.19) (pull_request) Failing after 32s Details
/ Builds (1.20) (pull_request) Failing after 32s Details
/ Vulncheck (pull_request) Failing after 57s Details
/ DCO (pull_request) Failing after 1m0s Details
Store child zap logger with request scope fields into context.
Request scoped fields: request_id, api/method, bucket, object

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-19 13:54:51 +03:00
Denis Kirillov 4a6e3a19ce [#125] Handle negative Content-Length on put
Add computing actual object size during calculating hash on put.
Use this actual value to save in tree and cache

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-02 14:16:19 +00:00
Artem Tataurov 9f186d9aba [#104] app: Reload copies numbers on SIGHUP
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2023-05-23 13:19:58 +03:00
Artem Tataurov e487ee5b7d [#70] Add arrays of copies numbers for location constraints
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2023-05-03 13:48:26 +03:00
Alexey Vanin 813aa2f173 Rename package name
Due to source code relocation from GitHub.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-03-07 17:38:08 +03:00
Denis Kirillov 9dcacc230e [TrueCloudLab#23] Return error on unknown LocationConstraint
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-02-08 15:04:12 +03:00
Artem Tataurov 8e14ccad9f [#754] Remove duplicating CID/OID logging calls
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
(cherry picked from commit 8c178fc3c7)
2022-12-28 17:06:32 +03:00
Artem Tataurov af7fbd6f32 [#754] Add CID and OID to logs
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2022-12-28 17:06:32 +03:00
Denis Kirillov 388482e230 [#2] Rename internals
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2022-12-21 11:17:45 +03:00
Alexey Vanin 96dff367db [#1] Build S3 Gateway with FrostFS dependencies
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2022-12-15 12:43:52 +03:00
Denis Kirillov dd4f66712c [#742] Add multiple listeners
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-11-24 17:48:26 +03:00
Denis Kirillov d2587b21af [#747] Reload policies on SIGHUP
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-11-18 13:14:25 +03:00
Denis Kirillov 094eb12578 [#726] Use client time on regular requests
Use `X-Amz-Date` header as `now` when
* compute expiration epoch
* set Timestamp for object and container
* forming locks
* send notifications

Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-11-14 14:53:55 +03:00
Denis Kirillov d47840f137 [#568] Add configuration for region to policy map
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-11-11 13:12:49 +03:00
Denis Kirillov 821df3d648 [#669] Optimize getNodeVersion for tags operation
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-10-20 17:56:09 +03:00
Denis Kirillov 44d5878a80 [#656] Read full body on upload error
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-08-30 09:39:20 +03:00
Denis Kirillov c3ad6d2faf [#637] Add header to override CopiesNumber
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-08-19 10:31:26 +03:00
Angira Kekteeva 5307211398 [#634] Add CopiesNumber in NeoFS requests
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-08-16 17:49:15 +03:00
Denis Kirillov 7ab473a688 [#595] Simplify encryption.Params struct
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-08-13 10:26:00 +03:00
Denis Kirillov 94a6a55919 [#595] Move decrypter to separate package
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-08-13 10:26:00 +03:00
Denis Kirillov d824db7f69 [#595] Allow SSE-C only with TLS
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-08-13 10:26:00 +03:00
Denis Kirillov 87c05eb514 [#595] Add SSE with customer key
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-08-13 10:26:00 +03:00
Angira Kekteeva 3ac3f1cc9d [#577] Rename objectInfo.Version() to VersionID()
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-08-09 17:20:40 +04:00
Denis Kirillov 0057f6b7db [#546] Add size and etag in nodeVersionInfo
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Angira Kekteeva 56eb2dc3dc [#569] Rename versioning consts
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-07-22 15:19:16 +03:00
Angira Kekteeva 72013e08ec [#569] Refactor versioning in tree service
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov 85c203e157 [#488] Sync using cid.ID
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov dd534e8738 [#451] Handle lock objects using tree service
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Angira Kekteeva 99feb1d936 [#449] Add tree service for object tagging
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov 7ca519cb32 [#539] Add context to errors
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-06-27 02:23:19 +04:00
Leonard Lyubich 028a152e04 [#544] Upgrade NeoFS SDK Go with another approach of container sessions
After recent changes in NeoFS SDK Go library session tokens aren't
embedded into `container.Container` and `eacl.Table` structures.
Instead, the operations of storing given values in NeoFS are
parameterized by elements of the corresponding type.

Add dedicated session parameters to operations of bucket and eACL
setting.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-24 13:00:26 +03:00
Leonard Lyubich f0749fd23e [#537] Upgrade NeoFS SDK Go with changed `netmap` package
`PlacementPolicy` type now provides methods to work with QL-encoded
policies. System network parameters can be read using dedicated method
without iterating. Applications can work with `PlacementPolicy`
variables directly so there is no need to use pointers.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-17 08:28:59 +03:00
Leonard Lyubich 880ffe7108 [#492] layer: Don't parameterize basic ACL of created containers
`CreateBucket` handler always creates containers with extended public
ACL, so there is no need to configure it in `NeoFS.CreateContainer`.

Make internal `NeoFS` implementation to create containers with
`eacl-public-read-write` basic ACL if corresponding parameter is unset.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-16 10:46:50 +03:00
Leonard Lyubich 087d500c5f [#458] *: Refactor working with NeoFS identities
Pull latest changes from NeoFS SDK Go library. Decrease redundant and
unsafe usage of ID pointers. Use `EncodeToString` method in order to
calculate protocol strings.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-05-27 17:22:37 +04:00
Alex Vanin 1c33f06bfe [#428] Update SDK
Includes:
- container removal fix
- new session token structure: authmate does not
  parse session context anymore, instead it is
  application defined flexible structure with
  container ID encoded in human-readable format

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-05-06 14:47:06 +04:00
Denis Kirillov 94caa2247e [#391] Refactor notifications
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-05-06 10:59:31 +03:00
Angira Kekteeva a0a04a73bd [#347] Add setEACL session token checks
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-04-20 12:42:33 +03:00
Angira Kekteeva f3df5ff633 [#395] Fix grantee in ACL
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-04-06 15:30:17 +03:00
Angira Kekteeva b7aac223df [#357] Add ObjectCreated notifications
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-04-01 14:57:19 +03:00
Angira Kekteeva 2b6843f8fa [#389] Fix checks of bucket settings
In Put/CopyObject and PutBucketVersioning

Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-04-01 14:22:10 +04:00
Denis Kirillov f0914b8a43 [#377] Reuse BucketInfo in layer
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-03-29 02:43:28 +04:00
Denis Kirillov 6cf01bed14 [#195] Add tests
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-03-23 14:32:13 +04:00
Denis Kirillov 8553158b81 [#195] Add handling lock headers for PUT and COPY
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-03-23 14:32:13 +04:00