Commit graph

707 commits

Author SHA1 Message Date
7d86b816a1 [#399] Add OPTIONS method for object operations
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>

(cherry picked from commit e25dc90c20)
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2024-06-19 08:55:45 +03:00
7ae4323116 [#354] Do not proceed on bucket remove error
Some checks failed
/ DCO (pull_request) Successful in 1m59s
/ Builds (1.20) (pull_request) Successful in 2m18s
/ Builds (1.21) (pull_request) Successful in 2m12s
/ Vulncheck (pull_request) Failing after 3m3s
/ Lint (pull_request) Successful in 4m56s
/ Tests (1.20) (pull_request) Successful in 3m18s
/ Tests (1.21) (pull_request) Successful in 3m15s
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2024-05-17 20:35:57 +03:00
c0f959ece7 [#383] Fix request type determination
Some checks failed
/ Vulncheck (pull_request) Failing after 1m38s
/ DCO (pull_request) Successful in 1m47s
/ Builds (1.20) (pull_request) Successful in 2m18s
/ Builds (1.21) (pull_request) Successful in 2m10s
/ Lint (pull_request) Successful in 2m52s
/ Tests (1.20) (pull_request) Successful in 2m24s
/ Tests (1.21) (pull_request) Successful in 2m14s
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2024-05-07 15:21:19 +03:00
0cf19d24ee [#269] Create frostfsid wrapper with cache
Some checks failed
/ DCO (pull_request) Successful in 1m55s
/ Builds (1.20) (pull_request) Successful in 2m36s
/ Builds (1.21) (pull_request) Successful in 2m28s
/ Vulncheck (pull_request) Failing after 2m17s
/ Lint (pull_request) Successful in 4m21s
/ Tests (1.20) (pull_request) Successful in 2m42s
/ Tests (1.21) (pull_request) Successful in 2m37s
(cherry picked from commit 5315f7b733)

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-04-26 12:00:37 +03:00
c6d0e5fc44 [#269] Add frostfsid cache configuration
(cherry picked from commit fec3b3f31e)

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-04-26 11:59:45 +03:00
f02bad65a8 [#362] Check user and groups during policy check
Some checks failed
/ DCO (pull_request) Successful in 1m56s
/ Builds (1.20) (pull_request) Successful in 2m11s
/ Builds (1.21) (pull_request) Successful in 1m52s
/ Vulncheck (pull_request) Failing after 2m7s
/ Lint (pull_request) Successful in 4m21s
/ Tests (1.20) (pull_request) Successful in 2m37s
/ Tests (1.21) (pull_request) Successful in 2m31s
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2024-04-12 16:52:08 +03:00
272e3a0f03 [#363] Fix removing combined object
Some checks failed
/ DCO (pull_request) Successful in 1m18s
/ Builds (1.20) (pull_request) Successful in 2m20s
/ Builds (1.21) (pull_request) Successful in 1m33s
/ Vulncheck (pull_request) Failing after 4m46s
/ Lint (pull_request) Successful in 6m10s
/ Tests (1.20) (pull_request) Successful in 5m12s
/ Tests (1.21) (pull_request) Successful in 3m40s
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-04-12 14:56:06 +03:00
65a8e2dadc [#360] Reuse single target during policy check
Some checks failed
/ DCO (pull_request) Successful in 1m40s
/ Vulncheck (pull_request) Failing after 1m51s
/ Builds (1.20) (pull_request) Successful in 2m29s
/ Builds (1.21) (pull_request) Successful in 1m44s
/ Lint (pull_request) Successful in 3m57s
/ Tests (1.20) (pull_request) Successful in 2m26s
/ Tests (1.21) (pull_request) Successful in 2m18s
Policy engine library is able to manage multiple
targets and resolve different status results.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2024-04-10 17:40:25 +03:00
b7e15402a1 [#360] Use 'c' prefix for bucket policies instead of 'n'
With 'c' prefix, acl chains become shorter, thus gateway
receives shorter results and avoids sessions to neo-go.

There is still issue with many IAM rules.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2024-04-10 17:40:25 +03:00
51be9d9778 [#354] Remove policies when delete bucket
Some checks failed
/ DCO (pull_request) Successful in 2m10s
/ Builds (1.20) (pull_request) Successful in 3m30s
/ Builds (1.21) (pull_request) Successful in 2m29s
/ Vulncheck (pull_request) Failing after 3m10s
/ Lint (pull_request) Successful in 4m59s
/ Tests (1.20) (pull_request) Successful in 3m12s
/ Tests (1.21) (pull_request) Successful in 1m34s
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-04-09 17:30:20 +03:00
bcfbcdc82f [#345] acl: Update APE and fix using
Some checks failed
/ Lint (pull_request) Successful in 2m53s
/ Tests (1.20) (pull_request) Successful in 2m43s
/ Tests (1.21) (pull_request) Successful in 2m44s
/ DCO (pull_request) Successful in 2m40s
/ Vulncheck (pull_request) Failing after 3m10s
/ Builds (1.20) (pull_request) Successful in 3m18s
/ Builds (1.21) (pull_request) Successful in 3m33s
* Remove native policy when remove bucket policy
* Allow policies that contain only s3 compatible statements
(now deny rules cannot be converted to native rules)

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-04-02 12:42:43 +00:00
1f5f2bd3d5 [#306] In APE buckets forbid canned acl except private
Some checks failed
/ DCO (pull_request) Successful in 1m35s
/ Builds (1.20) (pull_request) Successful in 2m9s
/ Builds (1.21) (pull_request) Successful in 2m3s
/ Vulncheck (pull_request) Failing after 3m30s
/ Lint (pull_request) Successful in 6m1s
/ Tests (1.20) (pull_request) Successful in 3m13s
/ Tests (1.21) (pull_request) Successful in 2m52s
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-03-19 16:56:32 +03:00
a32b41716f [#328] Log error on failed response writing
Some checks failed
/ DCO (pull_request) Successful in 3m48s
/ Builds (1.20) (pull_request) Successful in 3m58s
/ Builds (1.21) (pull_request) Successful in 3m59s
/ Vulncheck (pull_request) Failing after 3m44s
/ Lint (pull_request) Successful in 2m43s
/ Tests (1.20) (pull_request) Successful in 3m58s
/ Tests (1.21) (pull_request) Successful in 3m53s
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-03-15 11:04:05 +03:00
4ee3648183 [#328] Log invalid lock enabled header
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-03-04 15:09:51 +03:00
ee48d1dc85 [#325] Log error on failed request id generation
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-03-04 09:49:41 +00:00
f958eef2b3 [#325] Use default empty data.LockInfo in get/head in case of error
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-03-04 09:49:41 +00:00
81b44ab3d3 [#325] Fix mutex usage in controller
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-03-04 09:49:41 +00:00
8050ca2d51 [#306] Use session token for container read operations
All checks were successful
/ DCO (pull_request) Successful in 1m54s
/ Vulncheck (pull_request) Successful in 1m55s
/ Builds (1.20) (pull_request) Successful in 2m49s
/ Builds (1.21) (pull_request) Successful in 1m56s
/ Lint (pull_request) Successful in 3m59s
/ Tests (1.20) (pull_request) Successful in 2m30s
/ Tests (1.21) (pull_request) Successful in 2m19s
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-03-01 18:14:33 +03:00
c12e264697 [#306] Simplify cid resolver for metrics
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-03-01 17:46:16 +03:00
e9f38a49e4 [#306] Fix forming key for bucket cache
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-03-01 16:09:40 +03:00
fabb4134bc [#318] Use log msg from constants
All checks were successful
/ DCO (pull_request) Successful in 1m44s
/ Builds (1.20) (pull_request) Successful in 2m24s
/ Builds (1.21) (pull_request) Successful in 2m18s
/ Vulncheck (pull_request) Successful in 2m17s
/ Lint (pull_request) Successful in 2m36s
/ Tests (1.20) (pull_request) Successful in 1m42s
/ Tests (1.21) (pull_request) Successful in 1m32s
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-29 17:30:28 +03:00
e1ee36b979 [#318] Fix tests
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-29 17:30:28 +03:00
937367caaf [#318] Fix panic on invalid multipart form
Previously, simple 'curl -X POST http://localhost:8084/test' leads to panic because of wrong handle matching

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-29 17:30:28 +03:00
7b86bac6ee [#318] Log unmatched requests
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-29 17:30:28 +03:00
529ec7e0b9 [#318] Don't log empty bucket/name
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-29 17:30:28 +03:00
4741e74210 [#318] Log successfully authenticated accessKeyIDs
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-29 17:30:28 +03:00
f1470bab4a [#318] auth: Add context for logged errors
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-29 17:30:28 +03:00
6e5bcaef97 [#318] Log policy request checking
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-29 17:30:28 +03:00
1522db05c5 [#318] Log namespace for requests
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-29 17:30:28 +03:00
31da31862a [#300] Update error logging in DeleteMultipleObjects
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2024-02-29 14:24:32 +00:00
7de1ffdbe9 [#306] Fix billing tests
All checks were successful
/ DCO (pull_request) Successful in 1m43s
/ Vulncheck (pull_request) Successful in 1m42s
/ Builds (1.20) (pull_request) Successful in 2m34s
/ Builds (1.21) (pull_request) Successful in 1m56s
/ Lint (pull_request) Successful in 3m50s
/ Tests (1.20) (pull_request) Successful in 2m20s
/ Tests (1.21) (pull_request) Successful in 2m9s
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-28 18:00:27 +03:00
3285a2e105 [#306] policy: Change default access strategy
Use access strategy based on bucket type and/or config flags.

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-28 17:53:13 +03:00
56b50f2075 [#306] Remove flag to disable policy contract
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-28 17:50:08 +03:00
8f89f275bd [#306] Save bucket policy as native chain
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-28 17:50:08 +03:00
ff15f9f28a [#306] Fix update settings for buckets without owner key in tree
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-28 17:50:08 +03:00
c868af8a62 [#306] Add flag to enable old ACL bucket creation
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-28 17:50:08 +03:00
bac1b3fb2d [#306] Use zero basic acl to mark APE containers
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-28 17:50:08 +03:00
c452d58ce2 [#306] Reduce number of policy contract invocations
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-28 17:50:08 +03:00
d9d12debc3 [#306] Add tests
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-28 17:50:08 +03:00
3d0d2032c6 [#306] acl: Handle put/get acl for APE buckets
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-28 17:50:08 +03:00
1f2cf0ed67 [#306] Use APE instead of eACL on bucket creation
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-28 17:50:08 +03:00
37be8851b3 [#306] Simplify namespaces configuration
Resolve ns alias at the beginning of the request just once.
Keep in ns map only one default ns key.

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-28 17:50:08 +03:00
c4c199defe [#306] Use s3 as chain id prefix to be consistent
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-28 17:50:08 +03:00
2981a47e99 [#321] Use correct owner id in billing metrics
All checks were successful
/ DCO (pull_request) Successful in 1m20s
/ Vulncheck (pull_request) Successful in 1m54s
/ Builds (1.20) (pull_request) Successful in 2m22s
/ Builds (1.21) (pull_request) Successful in 2m8s
/ Lint (pull_request) Successful in 4m32s
/ Tests (1.20) (pull_request) Successful in 2m27s
/ Tests (1.21) (pull_request) Successful in 2m13s
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2024-02-28 14:52:44 +03:00
563c1d9bd7 [#308] Fix linter issues
All checks were successful
/ DCO (pull_request) Successful in 1m31s
/ Builds (1.20) (pull_request) Successful in 1m59s
/ Builds (1.21) (pull_request) Successful in 1m37s
/ Vulncheck (pull_request) Successful in 2m14s
/ Lint (pull_request) Successful in 4m7s
/ Tests (1.20) (pull_request) Successful in 2m28s
/ Tests (1.21) (pull_request) Successful in 1m58s
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2024-02-16 18:25:06 +03:00
924e87face [#305] Support checking if accessbox was removed
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-09 09:41:48 +03:00
c334adeb6d [#165] Sort nodes in ServiceClientMemory
Some checks failed
/ DCO (pull_request) Successful in 1m22s
/ Builds (1.20) (pull_request) Successful in 1m52s
/ Builds (1.21) (pull_request) Successful in 1m17s
/ Vulncheck (pull_request) Successful in 1m51s
/ Lint (pull_request) Failing after 1m55s
/ Tests (1.20) (pull_request) Successful in 2m24s
/ Tests (1.21) (pull_request) Successful in 2m24s
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-02 16:53:12 +03:00
a74d498df2 [#165] Return sort after HEAD in listing
Some checks failed
/ DCO (pull_request) Successful in 1m16s
/ Builds (1.20) (pull_request) Successful in 2m9s
/ Builds (1.21) (pull_request) Successful in 1m39s
/ Vulncheck (pull_request) Successful in 1m52s
/ Lint (pull_request) Failing after 1m47s
/ Tests (1.20) (pull_request) Successful in 2m27s
/ Tests (1.21) (pull_request) Successful in 2m13s
We have to sort object after HEAD because we make request in different goroutines,
so the order is not deterministic.

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-02 16:15:08 +03:00
69297a4a38 [#165] Delete object from tree in case of storage error
Extend storage node errors in case of which we continue deleting from tree
with 'object not found' error

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-02 16:15:08 +03:00
71d82d1cc8 [#165] Fix lint issues
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-02-02 16:15:08 +03:00