Commit graph

1180 commits

Author SHA1 Message Date
1a09041cd1 [#63] Simplify multiObjectReader and add tests
All checks were successful
/ DCO (pull_request) Successful in 1m19s
/ Vulncheck (pull_request) Successful in 2m31s
/ Builds (1.19) (pull_request) Successful in 2m51s
/ Builds (1.20) (pull_request) Successful in 2m34s
/ Lint (pull_request) Successful in 5m51s
/ Tests (1.19) (pull_request) Successful in 9m36s
/ Tests (1.20) (pull_request) Successful in 10m20s
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-20 14:00:15 +03:00
631b7f67b4 [#63] multipart: Log upload id for every failed request
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-20 14:00:15 +03:00
8ca2998297 [#63] Update CHANGELOG.md
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-20 14:00:15 +03:00
bcf5a85aab [#63] multipart: Fix copying
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-20 14:00:15 +03:00
ad81b599dd [#63] Add fast multipart upload
Add new flag to object tree meta `isCombined` that means
the object payload is list of parts that forms real payload.
Set this attribute when complete multipart upload not to do unnecessary copying.

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-20 14:00:15 +03:00
361d10cc78 [#174] Fix query for listing multipart uploads
All checks were successful
/ Builds (1.19) (pull_request) Successful in 3m2s
/ Builds (1.20) (pull_request) Successful in 2m50s
/ DCO (pull_request) Successful in 3m59s
/ Vulncheck (pull_request) Successful in 2m39s
/ Lint (pull_request) Successful in 3m56s
/ Tests (1.19) (pull_request) Successful in 3m5s
/ Tests (1.20) (pull_request) Successful in 3m12s
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-20 12:30:17 +03:00
62e6b49254 [#174] Log unmatched requests
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-20 12:30:17 +03:00
80c4982bd4 [#174] Add router tests
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-20 12:30:17 +03:00
73ed3f7782 [#174] Fix router filter query matching
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-20 12:30:17 +03:00
6e3595e35b [#174] Fix object keys with slashes in chi
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-20 12:30:17 +03:00
57add29643 [#173] Use forked actions in workflow
All checks were successful
/ Builds (1.19) (pull_request) Successful in 2m37s
/ Builds (1.20) (pull_request) Successful in 2m39s
/ DCO (pull_request) Successful in 3m38s
/ Vulncheck (pull_request) Successful in 5m10s
/ Lint (pull_request) Successful in 3m30s
/ Tests (1.19) (pull_request) Successful in 2m50s
/ Tests (1.20) (pull_request) Successful in 8m11s
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-07-18 15:18:34 +03:00
b59aa06637 [#146] Add kludge.bypass_content_encoding_check_in_chunks flag
Some checks failed
/ Builds (1.19) (pull_request) Successful in 3m0s
/ Builds (1.20) (pull_request) Successful in 2m50s
/ DCO (pull_request) Failing after 1m9s
/ Vulncheck (pull_request) Successful in 1m20s
/ Lint (pull_request) Successful in 7m50s
/ Tests (1.19) (pull_request) Successful in 3m1s
/ Tests (1.20) (pull_request) Successful in 3m21s
Flag allows to skip checking `Content-Encoding` for `aws-chunked` value

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-18 14:49:52 +03:00
d62aa7b979 [#146] Fix preconditions: trim quotes in etags
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-18 14:49:03 +03:00
751a9be7cc [#146] Move getting chunk payload reader to separate function
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-18 14:49:03 +03:00
e58ea40463 Release v0.27.0
Some checks failed
/ DCO (pull_request) Failing after 1m14s
/ Vulncheck (pull_request) Successful in 1m34s
/ Builds (1.19) (pull_request) Successful in 2m59s
/ Builds (1.20) (pull_request) Successful in 8m35s
/ Lint (pull_request) Successful in 3m22s
/ Tests (1.19) (pull_request) Successful in 2m56s
/ Tests (1.20) (pull_request) Successful in 2m47s
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-07-14 10:53:28 +03:00
14ef9ff091 [#158] Separate init object reader from read itself
All checks were successful
/ Vulncheck (pull_request) Successful in 1m40s
/ Builds (1.19) (pull_request) Successful in 2m54s
/ Builds (1.20) (pull_request) Successful in 2m39s
/ DCO (pull_request) Successful in 3m48s
/ Lint (pull_request) Successful in 3m32s
/ Tests (1.19) (pull_request) Successful in 2m38s
/ Tests (1.20) (pull_request) Successful in 2m55s
To be able to handle cases and return appropriate http status code
when object missed in storage but gate cache contains its metadata
we need write code after init object reader.
So we separate init reader from actual reading.

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-11 17:32:05 +03:00
fc90981c03 [#149] Update inner imports after moving middlewares
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-11 17:25:09 +03:00
83cdfbee78 [#149] Move middlewares to separate package
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-11 17:25:09 +03:00
37f2f468fe [#149] Add host bucket router
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-11 17:25:09 +03:00
e30a452cdb [#149] Use chi instead of gorilla mux
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-11 17:25:09 +03:00
7be70243f7 [#166] Update sdk to support grpc schemes in tree pool
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-11 17:25:03 +03:00
7f708b3a2d [#111] auth: Get log from real request context
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-06 11:57:04 +00:00
d531b13866 [#143] Add more context to some s3 errors
All checks were successful
/ DCO (pull_request) Successful in 1m18s
/ Vulncheck (pull_request) Successful in 1m50s
/ Lint (pull_request) Successful in 3m1s
/ Tests (1.19) (pull_request) Successful in 2m47s
/ Tests (1.20) (pull_request) Successful in 3m2s
/ Builds (1.19) (pull_request) Successful in 2m54s
/ Builds (1.20) (pull_request) Successful in 7m58s
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-30 12:08:33 +03:00
f921bc8af5 [#143] Fix typo
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-30 12:04:52 +03:00
be03c5178f [#143] Fix NoSuchKey error on get/head
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-30 12:04:52 +03:00
499f4c6495 [#155] metrics: Use default registerer for app metrics
All checks were successful
/ DCO (pull_request) Successful in 3m45s
/ Lint (pull_request) Successful in 3m29s
/ Tests (1.19) (pull_request) Successful in 2m44s
/ Tests (1.20) (pull_request) Successful in 3m19s
/ Vulncheck (pull_request) Successful in 4m40s
/ Builds (1.19) (pull_request) Successful in 2m58s
/ Builds (1.20) (pull_request) Successful in 1m47s
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-28 12:00:47 +03:00
2cbe3b9a27 [#131] Update docs
All checks were successful
/ Builds (1.19) (pull_request) Successful in 3m3s
/ Builds (1.20) (pull_request) Successful in 2m44s
/ DCO (pull_request) Successful in 3m59s
/ Vulncheck (pull_request) Successful in 1m34s
/ Lint (pull_request) Successful in 3m41s
/ Tests (1.19) (pull_request) Successful in 3m9s
/ Tests (1.20) (pull_request) Successful in 2m36s
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-27 14:18:16 +03:00
c588d485fa [#131] authmate: Add update-secret cobra cmd
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-27 14:18:16 +03:00
3927223bb0 [#131] authmate: Add generate-presigned-url cobra cmd
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-27 14:18:16 +03:00
eba85b50b6 [#131] authmate: Add obtain-secret corba cmd
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-27 14:18:16 +03:00
043447600e [#131] authmate: Add issue-secret cobra cmd
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-27 14:18:16 +03:00
0cd353707a [#131] authmate: Make authmate use cobra
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-27 14:18:16 +03:00
f74ab12f91 [#131] authmate: Add agent.UpdateSecret
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-27 14:18:16 +03:00
dea7b39805 [#131] Fix session token limit by container
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-27 14:18:16 +03:00
9df8695463 [#143] Fix transformToS3Error function
All checks were successful
/ Builds (1.19) (pull_request) Successful in 3m11s
/ Builds (1.20) (pull_request) Successful in 2m57s
/ DCO (pull_request) Successful in 4m7s
/ Lint (pull_request) Successful in 2m25s
/ Tests (1.19) (pull_request) Successful in 3m9s
/ Tests (1.20) (pull_request) Successful in 3m18s
/ Vulncheck (pull_request) Successful in 1m15s
Unwrap error before checking for s3 error

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-21 17:16:40 +03:00
614d703726 [#106] Add chunk uploading
All checks were successful
/ DCO (pull_request) Successful in 1m4s
/ Vulncheck (pull_request) Successful in 1m50s
/ Builds (1.19) (pull_request) Successful in 9m1s
/ Builds (1.20) (pull_request) Successful in 2m20s
/ Lint (pull_request) Successful in 10m19s
/ Tests (1.19) (pull_request) Successful in 2m45s
/ Tests (1.20) (pull_request) Successful in 3m19s
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2023-06-21 10:23:57 +03:00
23593eee3d [#111] Use request scope logger
Some checks failed
/ Lint (pull_request) Failing after 32s
/ Tests (1.19) (pull_request) Failing after 32s
/ Tests (1.20) (pull_request) Failing after 32s
/ Builds (1.19) (pull_request) Failing after 32s
/ Builds (1.20) (pull_request) Failing after 32s
/ Vulncheck (pull_request) Failing after 57s
/ DCO (pull_request) Failing after 1m0s
Store child zap logger with request scope fields into context.
Request scoped fields: request_id, api/method, bucket, object

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-19 13:54:51 +03:00
dfc4476afd [#135] authmate: Update docs
All checks were successful
/ Vulncheck (pull_request) Successful in 1m30s
/ Builds (1.19) (pull_request) Successful in 2m37s
/ Builds (1.20) (pull_request) Successful in 2m31s
/ DCO (pull_request) Successful in 3m6s
/ Lint (pull_request) Successful in 2m46s
/ Tests (1.19) (pull_request) Successful in 5m26s
/ Tests (1.20) (pull_request) Successful in 2m36s
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-16 09:42:49 +03:00
84358f6742 [#135] authmate: Support CRDT GSet for credentials
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-16 09:42:49 +03:00
7a380fa46c [#135] frostfs: Add SEARCH operation
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-16 09:42:41 +03:00
0590f84d68 [#135] crdt: Add GSet
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-16 09:39:43 +03:00
751d66bde0 [#140] Use gRPC interceptor from observability package
Previous SDK implementation had implicit gRPC interceptor
for tracing. Now pool constructors allow any dial options,
so gateway should manually pass tracing interceptors from
observability package.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-06-14 15:46:43 +00:00
462589fc0c [#103] Return 504 http code on timeout errors
All checks were successful
/ Vulncheck (pull_request) Successful in 1m42s
/ Lint (pull_request) Successful in 3m31s
/ Tests (1.19) (pull_request) Successful in 2m57s
/ Tests (1.20) (pull_request) Successful in 3m4s
/ Builds (1.19) (pull_request) Successful in 2m35s
/ Builds (1.20) (pull_request) Successful in 2m39s
/ DCO (pull_request) Successful in 1m7s
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-14 09:16:21 +03:00
8fcaf76f41 [#132] authmate: Add bearer token to obtain-secret result
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-13 15:14:19 +00:00
19c89b38e6 [#133] Update docs
All checks were successful
Builds (1.20)
Builds (1.19)
DCO
Tests (1.19)
Tests (1.20)
Vulncheck
Lint
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-09 09:34:49 +03:00
0bcda6ea37 [#133] Drop sync-tree
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-09 09:34:36 +03:00
9dabaf6ecd [#133] Use tree pool from SDK
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-09 09:34:04 +03:00
840d457cb9 [#2] Keep issue templates in .github
The only thing supported in .forgejo dir is a workflows.
It might be useful to keep .github dir for repo mirrors.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-06-07 15:35:57 +00:00
c2fae4c199 [#2] Update CONTRIBUTING
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-06-07 15:35:57 +00:00
9e0c39dcc2 [#2] Return shields
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-06-07 15:35:57 +00:00