Commit graph

487 commits

Author SHA1 Message Date
11c1a86404 [#339] Fix stream sigV4a chunk-encoding
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-12-10 15:04:56 +03:00
c4c757eea6 [#339] Drop aws-sdk-go v1
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-12-10 15:04:56 +03:00
cc9a68401f [#339] Add aws-sdk-go-v2
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-12-10 15:04:56 +03:00
f215d200e8 [#559] Remove multipart objects using tombstones
All checks were successful
/ DCO (pull_request) Successful in 1m33s
/ Vulncheck (pull_request) Successful in 2m38s
/ Builds (pull_request) Successful in 2m48s
/ Lint (pull_request) Successful in 4m6s
/ Tests (pull_request) Successful in 2m48s
/ Vulncheck (push) Successful in 1m19s
/ Builds (push) Successful in 2m14s
/ Lint (push) Successful in 3m40s
/ Tests (push) Successful in 2m21s
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2024-12-04 11:03:01 +03:00
e71ba5e22a [#543] Add md5 sse-c S3Tests compatability
All checks were successful
/ Vulncheck (push) Successful in 1m28s
/ Builds (push) Successful in 2m13s
/ Lint (push) Successful in 3m23s
/ Tests (push) Successful in 2m13s
Signed-off-by: Pavel Pogodaev <p.pogodaev@yadro.com>
2024-11-28 06:06:55 +00:00
e3141fc8e3 [#563] Ignore precondition headers with invalid date format
Some checks failed
/ DCO (pull_request) Successful in 1m43s
/ Vulncheck (pull_request) Successful in 1m51s
/ Builds (pull_request) Successful in 2m6s
/ Lint (pull_request) Successful in 3m14s
/ Tests (pull_request) Successful in 2m8s
/ Builds (push) Has been cancelled
/ Lint (push) Has been cancelled
/ Tests (push) Has been cancelled
/ Vulncheck (push) Has been cancelled
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2024-11-27 12:07:04 +03:00
9875307c9b [#556] Check bucket name not only during creation
All checks were successful
/ Vulncheck (push) Successful in 2m3s
/ Builds (push) Successful in 1m36s
/ Lint (push) Successful in 2m47s
/ Tests (push) Successful in 2m56s
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-11-20 08:13:27 +00:00
eff0de43d5 [#538] Return headers with 304 Not Modified
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2024-11-13 13:49:09 +00:00
fb00dff83b [#540] Add md5 S3Tests compatability
All checks were successful
/ DCO (pull_request) Successful in 2m14s
/ Vulncheck (pull_request) Successful in 2m20s
/ Builds (pull_request) Successful in 2m46s
/ Lint (pull_request) Successful in 4m4s
/ Tests (pull_request) Successful in 3m31s
Signed-off-by: Pavel Pogodaev <p.pogodaev@yadro.com>
2024-11-13 14:50:16 +03:00
d8f126b339 [#539] Fix listing v1 bookmark marker
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-11-12 12:58:09 +00:00
7ab902d8d2 [#536] Add rule ID generation
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2024-11-12 12:51:02 +00:00
0792fcf456 [#536] Fix error codes in lifecycle configuration check
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2024-11-12 12:51:02 +00:00
c46ffa8146 [#536] Add prefix to lifecycle rule
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2024-11-12 12:51:02 +00:00
3260308cc0 [#528] Check owner ID before deleting bucket
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2024-11-12 12:47:43 +00:00
539dab8680 [#501] Add the trace id to the logs of middlware
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
2024-11-02 08:51:48 +00:00
76008d4ba1 [#501] Consider using request logger in logAndSendError
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
2024-11-02 08:51:48 +00:00
09c11262c6 [#516] Check Content-Md5 of lifecycle configuration
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2024-10-22 14:21:49 +00:00
f120715a37 [#516] Convert expiration date to epoch
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2024-10-22 14:21:49 +00:00
e35b582fe2 [#506] Deny bucket names with dot
All checks were successful
/ DCO (pull_request) Successful in 1m14s
/ Vulncheck (pull_request) Successful in 1m49s
/ Builds (pull_request) Successful in 1m37s
/ Lint (pull_request) Successful in 3m3s
/ Tests (pull_request) Successful in 1m34s
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2024-10-08 12:50:22 +03:00
39fc7aa3ee [#404] Fix using encoding type in multipart upload listing
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2024-10-08 09:44:05 +00:00
da41f47826 [#404] Add check of encoding type in object listing
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2024-10-08 09:44:05 +00:00
346243b159 [#450] Fix getPutPayloadSize
All checks were successful
/ DCO (pull_request) Successful in 1m17s
/ Vulncheck (pull_request) Successful in 1m20s
/ Builds (pull_request) Successful in 1m24s
/ Lint (pull_request) Successful in 2m23s
/ Tests (pull_request) Successful in 1m25s
If X-Amz-Decoded-Content-Length explicitly set use it even if value is 0

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-10-03 14:23:06 +03:00
c2adbd758a [#488] middleware/auth: Add frostfs-to-s3 error transformation
All checks were successful
/ Vulncheck (pull_request) Successful in 1m3s
/ DCO (pull_request) Successful in 1m27s
/ Builds (pull_request) Successful in 1m27s
/ Lint (pull_request) Successful in 3m5s
/ Tests (pull_request) Successful in 1m56s
Signed-off-by: Nikita Zinkevich <n.zinkevich@yadro.com>
2024-10-02 13:18:25 +03:00
bc17ab5e47 [#488] middleware/policy: Add frostfs-to-s3 error transformation
Signed-off-by: Nikita Zinkevich <n.zinkevich@yadro.com>
2024-10-02 12:35:04 +03:00
9fadfbbc2f [#488] Renamed api/errors, layer/frostfs and layer/tree package names
Signed-off-by: Nikita Zinkevich <n.zinkevich@yadro.com>
2024-10-02 12:35:04 +03:00
827ea1a41e [#488] Move layer/frostfs.go to layer/frostfs/frostfs.go
Signed-off-by: Nikita Zinkevich <n.zinkevich@yadro.com>
2024-10-02 12:35:04 +03:00
968f10a72f [#488] Move layer/tree_service.go to layer/tree/tree_service.go
Signed-off-by: Nikita Zinkevich <n.zinkevich@yadro.com>
2024-10-02 12:35:04 +03:00
738ce14f50 [#434] Remove container on failed bucket creation
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-09-25 07:15:24 +00:00
26baf8a94e [#492] Add panic catchers to fuzzing tests
Signed-off-by: Roman Ognev <r.ognev@yadro.com>
2024-09-16 10:36:30 +00:00
f187141ae5 [#486] Fix PUT object with negative Content-Length
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2024-09-16 08:45:46 +00:00
d0e4d55772 [#460] Add network info cache
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2024-09-13 09:56:24 +00:00
42e72889a5 [#450] Add test for chunk-encoded object size
All checks were successful
/ DCO (pull_request) Successful in 1m9s
/ Vulncheck (pull_request) Successful in 1m20s
/ Builds (pull_request) Successful in 1m52s
/ Lint (pull_request) Successful in 2m47s
/ Tests (pull_request) Successful in 1m49s
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2024-09-13 12:00:24 +03:00
98815d5473 [#450] Fix aws-chunked header workflow
Signed-off-by: Pavel Pogodaev <p.pogodaev@yadro.com>
2024-09-13 11:59:07 +03:00
d919e6cce2 [#482] Fix containers resolving
All checks were successful
/ DCO (pull_request) Successful in 1m8s
/ Builds (pull_request) Successful in 1m7s
/ Vulncheck (pull_request) Successful in 1m12s
/ Lint (pull_request) Successful in 2m24s
/ Tests (pull_request) Successful in 1m24s
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2024-09-05 12:33:14 +03:00
056f168d77 [#448] multipart: Support removing duplicated parts
Previously after tree split we can have duplicated parts
(several objects and tree node referred to the same part number).
Some of them couldn't be deleted after abort or compete action.

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-09-03 13:20:38 +00:00
d6b506f6d9 [#466] Implement PATCH for multipart objects
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2024-09-03 11:57:59 +00:00
b08f476ea7 [#462] Implement PATCH for regular objects
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2024-09-03 11:57:59 +00:00
664f83b2b7 [#480] Add fuzzing tests
All checks were successful
/ DCO (pull_request) Successful in 1m34s
/ Vulncheck (pull_request) Successful in 1m45s
/ Builds (1.22) (pull_request) Successful in 1m51s
/ Builds (1.23) (pull_request) Successful in 1m56s
/ Lint (pull_request) Successful in 3m3s
/ Tests (1.22) (pull_request) Successful in 1m52s
/ Tests (1.23) (pull_request) Successful in 1m51s
Signed-off-by: Roman Ognev <r.ognev@yadro.com>
2024-09-02 15:59:07 +03:00
d76c4fe2a2 [#472] tree: Don't use sorted GetSubTree for nodes without FileName
Sorted GetSubTree doesn't return nodes without FileName attribute
if there are more than 1000 nodes in subtree

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-08-27 09:43:29 +00:00
534ae7f0f1 [#446] Add support virtual-hosted-style
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
2024-08-23 08:35:05 +00:00
e73f11c251 [#452] tree: Fix logging
All checks were successful
/ DCO (pull_request) Successful in 1m6s
/ Vulncheck (pull_request) Successful in 1m10s
/ Builds (1.21) (pull_request) Successful in 1m32s
/ Builds (1.22) (pull_request) Successful in 1m31s
/ Lint (pull_request) Successful in 1m46s
/ Tests (1.21) (pull_request) Successful in 1m51s
/ Tests (1.22) (pull_request) Successful in 1m45s
Don't log parsing tags error in listing

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-08-21 10:38:36 +03:00
5cb77018f8 [#42] Change valid time format for lifecycle configuration
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2024-08-21 10:38:36 +03:00
fa68a4ce40 [#412] Store creation epoch of delete markers
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2024-08-21 10:38:35 +03:00
481520705a [#42] Support expiration lifecycle
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2024-08-21 10:38:35 +03:00
20719bd85c [#456] PostObject: check object key for emptiness
All checks were successful
/ DCO (pull_request) Successful in 1m1s
/ Vulncheck (pull_request) Successful in 1m12s
/ Builds (1.21) (pull_request) Successful in 1m29s
/ Builds (1.22) (pull_request) Successful in 1m33s
/ Lint (pull_request) Successful in 2m20s
/ Tests (1.21) (pull_request) Successful in 1m41s
/ Tests (1.22) (pull_request) Successful in 1m39s
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-08-13 09:27:26 +03:00
3dc989d7fe [#451] Support Location in CompleteMultipart response
All checks were successful
/ Vulncheck (pull_request) Successful in 1m15s
/ DCO (pull_request) Successful in 1m13s
/ Builds (1.21) (pull_request) Successful in 1m31s
/ Builds (1.22) (pull_request) Successful in 1m25s
/ Lint (pull_request) Successful in 2m26s
/ Tests (1.21) (pull_request) Successful in 1m40s
/ Tests (1.22) (pull_request) Successful in 1m43s
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-08-06 15:45:09 +03:00
69e77aecc9 [#451] Don't skip [Next]PartNumberMarker in response
In ListParts response we should always keep PartNumberMarker (even if it's zero)
and NextPartNumberMarker (even if response isn't truncated)

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-08-06 15:44:56 +03:00
51c5c227c2 [#31] Add force bucket delete flag
All checks were successful
/ DCO (pull_request) Successful in 1m11s
/ Vulncheck (pull_request) Successful in 1m22s
/ Builds (1.21) (pull_request) Successful in 1m26s
/ Builds (1.22) (pull_request) Successful in 2m23s
/ Lint (pull_request) Successful in 2m22s
/ Tests (1.21) (pull_request) Successful in 1m23s
/ Tests (1.22) (pull_request) Successful in 1m25s
Signed-off-by: Pavel Pogodaev <p.pogodaev@yadro.com>
2024-07-25 14:04:54 +03:00
6cb0026007 [#427] layer: Split FrostFS ReadObject to separate methods
All checks were successful
/ Builds (1.21) (pull_request) Successful in 1m20s
/ Builds (1.22) (pull_request) Successful in 1m15s
/ Vulncheck (pull_request) Successful in 1m6s
/ DCO (pull_request) Successful in 57s
/ Lint (pull_request) Successful in 2m59s
/ Tests (1.21) (pull_request) Successful in 1m9s
/ Tests (1.22) (pull_request) Successful in 1m23s
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-07-23 16:53:59 +03:00
3213dd7236 [#404] Add processing of encoding-type in versions listing
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2024-07-23 12:10:35 +00:00