473239bf36
[ #257 ] Add policy checker
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-12-06 17:47:51 +03:00
42862fd69e
[ #258 ] Support policy management in control svc
...
Add PutPolicies, RemovePolicies, GetPolicy, ListPolicies methods
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-12-01 15:47:12 +03:00
c7a65bd075
[ #258 ] Add control service
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-12-01 14:17:06 +03:00
28c6bb4cb8
[ #266 ] Support per namespace placement policies configuration
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-11-28 16:12:42 +03:00
ff1ec56d24
[ #260 ] Use namespace as domain when create bucket
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-11-27 15:30:12 +03:00
a61ff3b8cb
[ #260 ] authmate: Support key registration in frostfsid contract
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-11-23 11:00:11 +03:00
6304d7bfda
[ #260 ] Support frostfsid validation
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-11-23 11:00:11 +03:00
0bed25816c
[ #224 ] Add conditional escaping for object name
...
Chi gives inconsistent results in terms of whether
the strings returned are URL coded or not
See:
* https://github.com/go-chi/chi/issues/641
* https://github.com/go-chi/chi/issues/642
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-10-31 13:58:51 +00:00
25bb581fee
[ #205 ] Add md5 checksum in header
...
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-10-25 11:04:19 +03:00
7de73f6b73
[ #197 ] Disable homomorphic hash for PUT
...
Disable TZ hash for PUT if it's disabled for container itself
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-10-11 12:32:48 +00:00
b8c93ed391
[ #172 ] Convert handler config to interface
...
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-10-04 11:01:27 +00:00
c66c09765d
[ #196 ] Support soft memory limit setting
...
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
2023-09-05 13:13:56 +00:00
2fc328a6d2
[ #195 ] Add log constants linter
...
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-08-28 12:58:44 +03:00
b5fce5c8d2
[ #168 ] Skip only invalid policies and copies instead of ignoring all of them
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-25 12:05:30 +03:00
631d9d83b6
[ #185 ] Fix payload reader
...
When we use io.CopyBuffer it check for exact io.EOF matching,
so we need keep original EOF error otherwise io.CopyBuffer returns error
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-24 18:01:08 +03:00
8efcc957ea
[ #96 ] Move log messages to constants
...
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
2023-08-23 18:32:31 +03:00
6b1f365e65
[ #192 ] Support client cut
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-23 06:26:55 +00:00
012ece40bb
[ #180 ] Fix linter issues
...
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-08-21 17:23:24 +03:00
94a42fa273
[ #51 ] Update frostfs-sdk-go
...
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2023-08-17 14:26:25 +03:00
40d7f844e3
[ #137 ] Refactor context data retrievers
...
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
2023-08-16 14:05:38 +00:00
18878b66d3
[ #175 ] Use gate owner as object owner
...
This is required because node check session token owner
TrueCloudLab/frostfs-node#528
For client cut TrueCloudLab/frostfs-sdk-go#114
such owner will be gate owner
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-08 12:22:33 +03:00
fc90981c03
[ #149 ] Update inner imports after moving middlewares
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-11 17:25:09 +03:00
84358f6742
[ #135 ] authmate: Support CRDT GSet for credentials
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-16 09:42:49 +03:00
7a380fa46c
[ #135 ] frostfs: Add SEARCH operation
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-16 09:42:41 +03:00
0590f84d68
[ #135 ] crdt: Add GSet
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-16 09:39:43 +03:00
462589fc0c
[ #103 ] Return 504 http code on timeout errors
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-14 09:16:21 +03:00
0bcda6ea37
[ #133 ] Drop sync-tree
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-09 09:34:36 +03:00
9dabaf6ecd
[ #133 ] Use tree pool from SDK
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-09 09:34:04 +03:00
11f30a037b
[ #114 ] tree: Fix retry tests
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-05-23 11:35:44 +03:00
136a186c14
[ #114 ] tree: Don't ignore unhealthy endpoints
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-05-19 17:22:39 +03:00
24390fdec8
[ #110 ] tree: Add more logs for switching tree endpoints
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-05-19 17:20:38 +03:00
1fdbfb0dab
[ #110 ] tree: Update errors to switch endpoint
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-05-19 17:20:34 +03:00
1406f57bba
[ #1 ] Update comment lines
...
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-05-19 07:30:04 +00:00
Denis Kirillov
b366e75366
[ #81 ] Use impersonate bearer token
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-05-03 17:22:52 +03:00
e487ee5b7d
[ #70 ] Add arrays of copies numbers for location constraints
...
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2023-05-03 13:48:26 +03:00
69d8779daf
[ #74 ] tree: Simplify retry
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-04-26 16:39:57 +03:00
9da77667f3
[ #74 ] Add round tree retry
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-04-26 16:39:57 +03:00
ca8791a5fd
[ #74 ] Support multiple tree endpoints
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-04-26 16:39:57 +03:00
01afa1cae4
[ #75 ] Make grpc tree client implementation internal
...
Since we have pkg 'internal/frostfs/services/tree' that is downloading
during build we cannot export any package that is depended on it.
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-04-03 14:04:35 +03:00
a025f2e9c5
[ #59 ] tree: Make interface for tree service client
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-03-29 09:34:10 +03:00
a0937126cb
[ #64 ] Support new system attributes
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-03-22 11:46:58 +03:00
0af06c3bd9
[TrueCloudLab#40] Add param to configure xml decoder
...
This parameter enables parsing xml body without
xmlns="http://s3.amazonaws.com/doc/2006-03-01/ " attribute
for CompleteMultipartUpload requests
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-03-09 15:44:13 +03:00
813aa2f173
Rename package name
...
Due to source code relocation from GitHub.
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-03-07 17:38:08 +03:00
388482e230
[ #2 ] Rename internals
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2022-12-21 11:17:45 +03:00
765fd28a49
[ #2 ] Update build
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2022-12-20 18:24:30 +03:00
96dff367db
[ #1 ] Build S3 Gateway with FrostFS dependencies
...
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2022-12-15 12:43:52 +03:00
Denis Kirillov
094eb12578
[ #726 ] Use client time on regular requests
...
Use `X-Amz-Date` header as `now` when
* compute expiration epoch
* set Timestamp for object and container
* forming locks
* send notifications
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-11-14 14:53:55 +03:00
Denis Kirillov
f59bf91a67
[ #699 ] neofs/tree: Add healthcheck
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-10-15 13:59:54 +03:00
Denis Kirillov
3d08562843
[ #505 ] Handle access denied from tree service
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-09-13 18:05:16 +03:00
Denis Kirillov
80beedf13e
[ #504 ] Use bktInfo in tree service to check owner
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-09-13 18:05:16 +03:00
Denis Kirillov
5184ed66c6
[ #657 ] Don't use <empty> to handle trailing slash
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-09-07 15:50:43 +03:00
Denis Kirillov
9cd4ef1ac4
[ #657 ] Replace FileName with FilePath attribute
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-09-07 15:50:43 +03:00
Denis Kirillov
6108f8d929
[ #624 ] Update tree service revision
...
Remove extra request to tree service to know parentID
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-09-07 12:17:51 +03:00
Denis Kirillov
e33d6fceda
[ #624 ] Get full subTree for listing
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-09-07 12:17:51 +03:00
9590b39333
[ #682 ] Return added node id in AddVersion
...
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2022-08-29 18:24:00 +03:00
a28d6b64b9
[ #682 ] Return added node id in addNodeByPath
...
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2022-08-29 18:24:00 +03:00
Denis Kirillov
7b165d3f18
[ #671 ] Fix using nested name in multipart upload
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-08-25 12:01:49 +03:00
Denis Kirillov
24e1f7e1dd
[ #545 ] Drop object search
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-08-25 11:49:08 +03:00
Denis Kirillov
e4ec983213
[ #678 ] Update SDK to fix panic on go1.19
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-08-25 11:33:17 +03:00
Angira Kekteeva
5307211398
[ #634 ] Add CopiesNumber in NeoFS requests
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-08-16 17:49:15 +03:00
Angira Kekteeva
8eb1afbcb8
[ #645 ] Replace tagging when PutObject
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-08-10 09:21:27 +03:00
Angira Kekteeva
f66e4a77cd
[ #577 ] Add IsDeleteMarker() to NodeVersion
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-08-09 17:20:40 +04:00
Denis Kirillov
08898f4fb4
[ #615 ] Expose pool metrics
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-08-02 16:04:31 +03:00
Alex Vanin
5219965105
[ #498 ] Remove log string sanitizer
...
`zap` lib does all the job anyway.
Suppress CWE-117 from CodeQL.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-07-29 09:32:37 +03:00
Denis Kirillov
ec6039c53b
[ #627 ] Correct listing object same as directory
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-29 09:32:23 +03:00
Alex Vanin
685a5f0ce8
[ #540 ] Do not set sticky bit in authmate container
...
In public containers sticky bit allows to limit
ownership of the objects by request authors.
In private or public-read containers it doesn't
make any sense so sticky bit is redundant there.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-07-28 09:09:24 +03:00
Denis Kirillov
75afc0e4a6
[ #601 ] Fix GetLatestVersionsByPrefix
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
0057f6b7db
[ #546 ] Add size and etag in nodeVersionInfo
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Angira Kekteeva
56eb2dc3dc
[ #569 ] Rename versioning consts
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-07-22 15:19:16 +03:00
Angira Kekteeva
72013e08ec
[ #569 ] Refactor versioning in tree service
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
e46763e429
[ #533 ] Fix typos
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
c88a9842db
[ #488 ] Sync using oid.ID
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
85c203e157
[ #488 ] Sync using cid.ID
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
12a2060dd0
[ #524 ] Optimize listing
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
d8ab1b4799
[ #520 ] Fix saving two lock types simultaneously
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Angira Kekteeva
87d19f09f7
[ #519 ] Fix transform of AcessDenied error
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
cb5237fb82
[ #510 ] Fix listing with prefix
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
64931c77f4
[ #503 ] Handle leading slash in names
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
41c815441b
[ #502 ] Fix list empty bucket
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Alex Vanin
f8496973b8
[ #477 ] Aggregate fetching tags and lock in layer.TreeService
...
New method aggregates GetObjectTagging and GetLock
methods in single RPC to the tree service.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-07-22 15:19:16 +03:00
Alex Vanin
5304e68b9a
[ #477 ] Add getTreeNodes method
...
Find multiple treeNodes from
subtree in single request.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
9b1ccd39be
[ #475 ] Add extra attributes to partInfo
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
5b9a5cadef
[ #452 ] Fix lint issues
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
9fb3fb1274
[ #452 ] Fix versions tests
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
dd534e8738
[ #451 ] Handle lock objects using tree service
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
6fe46a5944
[ #453 ] Use only FileName as path attribute
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
70957d75fd
[ #417 ] Complete multipart upload using tree service
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
6b2ddcadd0
[ #417 ] Abort multipart upload using tree service
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
58f2bf44b1
[ #417 ] List parts using tree service
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
bc0bdc7767
[ #417 ] Upload part using tree service
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
e1b9a4432a
[ #417 ] List multipart uploads using tree service
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
13e01164d7
[ #417 ] Create multipart upload using tree service
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-07-22 15:19:16 +03:00
Angira Kekteeva
24bad60048
[ #449 ] Add prefix for user tags in tree service
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-07-22 15:19:16 +03:00
Angira Kekteeva
385a2102f0
[ #449 ] Add tree service for bucket tagging
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-07-22 15:19:16 +03:00
Angira Kekteeva
99feb1d936
[ #449 ] Add tree service for object tagging
...
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
9c74cca9af
[ #420 ] Using tree service to list object versions
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
55c38e73e6
[ #444 ] Handle trailing slash in names
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
cfa6098919
[ #416 ] Make emptyOID a local variable
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
25477cdaf8
[ #416 ] Use tree service to list objects
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
7e8b57605a
[ #413 ] Fix delete object without version
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
