Commit graph

487 commits

Author SHA1 Message Date
Denis Kirillov
5dd658302d [] Fix comment
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-26 17:36:21 +03:00
Denis Kirillov
b2e8b1cfb3 [] Make listing more robust
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-26 17:36:21 +03:00
Angira Kekteeva
4483c6f57a [] Filter system metadata
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-07-26 15:37:13 +03:00
Angira Kekteeva
4380406602 [] Remove unused PutSystemObjectParams
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-07-26 15:37:13 +03:00
Alex Vanin
e0136feb73 [] Split name processing from object info creation procedure
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-07-25 14:23:30 +03:00
Alex Vanin
97f81d3270 [] Process delimiters on the object info cache hit
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-07-25 14:23:30 +03:00
Alex Vanin
757600608c [] Fix imports
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
85eacdb970 [] Fix removal non-empty bucket
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
0057f6b7db [] Add size and etag in nodeVersionInfo
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Angira Kekteeva
56eb2dc3dc [] Rename versioning consts
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-07-22 15:19:16 +03:00
Angira Kekteeva
77f466b1d5 [] Fix tests
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-07-22 15:19:16 +03:00
Angira Kekteeva
72013e08ec [] Refactor versioning in tree service
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
9f019c7470 [] Add test to check delete marker
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
b380d4ab53 [] Clear list cache on object removal
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
f1a0288e3a [] Handle unversioned mode
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
fdf0974679 [] Fix object removal
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
88c392d024 [] Optimize GetObjectTaggingAndLock
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
206a7aa395 [] Change cache key types
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
e104855633 [] Sync using oid.Address
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
c88a9842db [] Sync using oid.ID
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
85c203e157 [] Sync using cid.ID
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
dca7b5bc36 [] Fix tests
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
42a761c1f2 [] Cache ObjectInfo instead of Object
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
127b91a97f [] Cache empty lists without error
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
5984f2c2bc [] Add comment to clarify anonymous func
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
ad95d1745c [] Parallelize listing
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
6ad7c988e6 [] Fix object removal
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
12a2060dd0 [] Optimize listing
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Angira Kekteeva
3f5afabd4a [] Fix nil reference when a part is missing
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
93a7c2eaeb [] Missed version id in tags
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
59570ac075 [] Don't remove delete mark object
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Alex Vanin
4f3d206422 [] Don't fetch epoch at object upload
Creation epoch was used for versioning. With
tree service, versioning is done on the NeoFS
side in the tree.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
9dfc7e043f [] Save full parts info to multipart object
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Alex Vanin
8731dcfd3c [] Use tags and lock aggregation in handler
Reduce amount of tree service RPCs
by using new method in layer package.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-07-22 15:19:16 +03:00
Alex Vanin
7fd27e04fe [] Provide aggregation of tags and locks in layer.Client
Adopts GetObjectTaggingAndLock method from
TreeClient in layer.Client with caching.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-07-22 15:19:16 +03:00
Alex Vanin
f8496973b8 [] Aggregate fetching tags and lock in layer.TreeService
New method aggregates GetObjectTagging and GetLock
methods in single RPC to the tree service.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
a02900a4f7 [] Use appropriate null version during listing
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
c8e8ba9f6a [] Fix parts info for GetObjectAttributes
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
9b1ccd39be [] Add extra attributes to partInfo
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
5b9a5cadef [] Fix lint issues
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
1a456eaa8b [] Remove unused
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
9fb3fb1274 [] Fix versions tests
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
5529fb914e [] Fix lock tests
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
dd534e8738 [] Handle lock objects using tree service
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
70957d75fd [] Complete multipart upload using tree service
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
6b2ddcadd0 [] Abort multipart upload using tree service
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
58f2bf44b1 [] List parts using tree service
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
bc0bdc7767 [] Upload part using tree service
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
e1b9a4432a [] List multipart uploads using tree service
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
13e01164d7 [] Create multipart upload using tree service
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-07-22 15:19:16 +03:00
Angira Kekteeva
b6a43c6f4c [] Replace error checks and literals
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-07-22 15:19:16 +03:00
Angira Kekteeva
385a2102f0 [] Add tree service for bucket tagging
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-07-22 15:19:16 +03:00
Angira Kekteeva
99feb1d936 [] Add tree service for object tagging
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
9c74cca9af [] Using tree service to list object versions
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
25477cdaf8 [] Use tree service to list objects
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
7e8b57605a [] Fix delete object without version
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
37b1baed41 [] Refactor
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
977f176713 [] Use tree service to delete object
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
49bd77d9cf [] Use tree service to head objects
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
36f3c43af5 [] Use tree service to put object
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Angira Kekteeva
ab5c44ac14 [] Refactor TreeService for notif,cors,settings
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-07-22 15:19:16 +03:00
Angira Kekteeva
8eff857e41 [] Add tree service for cors
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-07-22 15:19:16 +03:00
Angira Kekteeva
7520952792 [] Add tree service for notifications
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-07-22 15:19:16 +03:00
Angira Kekteeva
b03ae827fb [] Refactor tree service
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-07-22 15:19:16 +03:00
Denis Kirillov
9ed081db3e [] Add service tree client prototype
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:19:16 +03:00
Alex Vanin
d7f77ce874 [] Produce deny records for private objects in put-object-acl
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-07-21 17:55:15 +03:00
Alex Vanin
66fe3fee7b [] Produce deny records for private objects
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-07-21 17:55:15 +03:00
Denis Kirillov
7ba7e7dc4d [] Make service records valid
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-20 19:10:23 +03:00
Denis Kirillov
1e26cf1541 [] Use service records to save resource info
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-20 19:10:23 +03:00
Angira Kekteeva
b144e50f7f [] Refactor formRecords func
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-07-19 11:54:27 +03:00
Angira Kekteeva
3f4a55f39e [] Fix order in astToTable
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-07-19 11:54:27 +03:00
Angira Kekteeva
260fb95677 [] Fix order in tableToAst
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-07-19 11:54:27 +03:00
Angira Kekteeva
74300a75a9 [] Add tests
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-07-19 11:54:27 +03:00
Alex Vanin
aad4862c59 [] Don't parse lock attribute if it is omitted in container
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-07-15 01:33:48 +04:00
Denis Kirillov
1575da65a4 [] Fix object acl filters
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-14 13:33:11 +03:00
Alex Vanin
a57b8d34d3 [] Add more comments about eacl.RoleUnknown
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-07-07 12:05:25 +03:00
Alex Vanin
06d043e1eb [] Optimize target formation with multiple keys
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-07-07 12:05:25 +03:00
Alex Vanin
d6065c64c4 [] Check group grantee based on stored list of users
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-07-07 12:05:25 +03:00
Alex Vanin
c7de7d2928 [] Do not use user role with public keys in eacl target
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-07-07 12:05:25 +03:00
Alex Vanin
36029ca864 [] Fix user removal in astOperation
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-07-06 17:40:41 +03:00
Leonard Lyubich
5bfc549746 [] neofs: Merge if with same condition in CreateContainer
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-07-04 23:42:50 +04:00
Leonard Lyubich
4a8a248f34 [] Upgrade NeoFS SDK Go with changed container API
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-07-04 23:42:50 +04:00
Denis Kirillov
6e1a1f3839 [] Suppress CodeQL error
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-04 11:03:55 +03:00
Denis Kirillov
9f740b9683 [] Add detect mimetype by extension
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-06-29 19:13:16 +03:00
Denis Kirillov
f72bc538b9 [] Refactor notifications logs
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-06-27 17:31:17 +03:00
Denis Kirillov
35f55c5af5 [] Fix tests
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-06-27 02:23:19 +04:00
Denis Kirillov
7ca519cb32 [] Add context to errors
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-06-27 02:23:19 +04:00
Leonard Lyubich
e1f1e6d960 [] Upgrade NeoFS SDK Go with changed basic ACL API
SDK now provides dedicated type for basic ACL with convenient interface.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-24 16:43:50 +03:00
Leonard Lyubich
a8fc313ff5 [] layer: Remove no longer needed deleteContainer method
Call `DeleteContainer` on `NeoFS` component directly.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-24 13:00:26 +03:00
Leonard Lyubich
f596c8be06 [] layer: Add session token parameter to DeleteBucketParams
Add `DeleteBucketParams.SessionToken` field in order to unify the
parameters with `CreateBucketParams` and `PutBucketACLParams`.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-24 13:00:26 +03:00
Leonard Lyubich
028a152e04 [] Upgrade NeoFS SDK Go with another approach of container sessions
After recent changes in NeoFS SDK Go library session tokens aren't
embedded into `container.Container` and `eacl.Table` structures.
Instead, the operations of storing given values in NeoFS are
parameterized by elements of the corresponding type.

Add dedicated session parameters to operations of bucket and eACL
setting.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-24 13:00:26 +03:00
Denis Kirillov
818176e7e1 [] Adopt aws v4signer
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-06-20 09:44:25 +03:00
Leonard Lyubich
f0749fd23e [] Upgrade NeoFS SDK Go with changed netmap package
`PlacementPolicy` type now provides methods to work with QL-encoded
policies. System network parameters can be read using dedicated method
without iterating. Applications can work with `PlacementPolicy`
variables directly so there is no need to use pointers.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-17 08:28:59 +03:00
Angira Kekteeva
cfe7591cf7 [] Add putObjectACL notification
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-06-17 01:43:46 +04:00
Angira Kekteeva
dd0d21b690 [] Fix typo
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-06-17 01:43:46 +04:00
Angira Kekteeva
bd5fd041b7 [] Add object tagging notifications
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-06-17 01:43:46 +04:00
Denis Kirillov
d521af2065 [] Add presign URLs support
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-06-16 17:17:45 +03:00
Leonard Lyubich
880ffe7108 [] layer: Don't parameterize basic ACL of created containers
`CreateBucket` handler always creates containers with extended public
ACL, so there is no need to configure it in `NeoFS.CreateContainer`.

Make internal `NeoFS` implementation to create containers with
`eacl-public-read-write` basic ACL if corresponding parameter is unset.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-16 10:46:50 +03:00
Angira Kekteeva
a8bff13801 [] Add acl to copy-object
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-06-07 17:36:12 +03:00
Denis Kirillov
de7281ac58 [] Unify log messages
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-06-07 17:20:26 +03:00