nzinkevich/frostfs-s3-gw
1
0
Fork 0
forked from TrueCloudLab/frostfs-s3-gw
Code Pull requests Activity
1277 commits 11 branches 26 tags 19 MiB
Commit graph

678 commits

Author SHA1 Message Date
Alex Vanin
2282c32822 [#60] Add *NoHeader functions 
Such functions should be used together with periodic white space
XML writer.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-03-20 12:07:36 +03:00
Alex Vanin
43685e03d9 [#60] Implement flusher on all http.ResponseWriters 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-03-20 12:04:34 +03:00
Alex Vanin
cf18158da4 [#60] Implement periodic white space XML writer 
Periodic white space XML writer sends XML header
and white spaces to the io.Writer.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-03-20 12:04:34 +03:00
Denis Kirillov
0af06c3bd9 [TrueCloudLab#40] Add param to configure xml decoder 
This parameter enables parsing xml body without
xmlns="http://s3.amazonaws.com/doc/2006-03-01/" attribute
for CompleteMultipartUpload requests

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-03-09 15:44:13 +03:00
Denis Kirillov
64e7356acc [TrueCloudLab#32] Add custom policy unmarshaler 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-03-09 11:19:01 +00:00
Denis Kirillov
32bf915502 [TrueCloudLab#37] Limit number of objects to delete 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-03-09 11:16:22 +00:00
Alex Vanin
813aa2f173 Rename package name 
Due to source code relocation from GitHub.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-03-07 17:38:08 +03:00
Denis Kirillov
740acadd37 [TrueCloudLab#36] Fix cors object payload 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-02-20 19:22:48 +03:00
Alex Vanin
aadefd98b6 [TrueCloudLab#25] Process allow and deny lists of zones in bucket head requests 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-02-17 13:47:48 +03:00
Alex Vanin
8ac630ee71 [TrueCloudLab#25] Add zone data to BucketInfo 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-02-17 13:47:48 +03:00
Denis Kirillov
9f823bd65a [TrueCloudLab#26] Add billing metrics to separate registry 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-02-10 10:55:38 +03:00
Denis Kirillov
9dcacc230e [TrueCloudLab#23] Return error on unknown LocationConstraint 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-02-08 15:04:12 +03:00
Artem Tataurov
5ee4bf80ae [#18] Return container name in head-bucket response 
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
 2023-02-01 14:25:21 +03:00
Denis Kirillov
e278ab9362 [TrueCloudLab#5] Refactor middlewares 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-01-25 11:02:23 +03:00
Denis Kirillov
c5570e661d [TrueCloudLab#5] Add traffic metrics per user 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-01-25 11:02:23 +03:00
Denis Kirillov
fc5c09c084 [TrueCloudLab#5] Request metrics per user 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-01-25 11:02:23 +03:00
Artem Tataurov
8e14ccad9f [#754] Remove duplicating CID/OID logging calls 
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
(cherry picked from commit 8c178fc3c7)
 2022-12-28 17:06:32 +03:00
Artem Tataurov
af7fbd6f32 [#754] Add CID and OID to logs 
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
 2022-12-28 17:06:32 +03:00
Denis Kirillov
000d9ed42a [#2] Update issue links 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2022-12-21 11:17:45 +03:00
Denis Kirillov
388482e230 [#2] Rename internals 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2022-12-21 11:17:45 +03:00
Denis Kirillov
765fd28a49 [#2] Update build 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2022-12-20 18:24:30 +03:00
Alex Vanin
96dff367db [#1] Build S3 Gateway with FrostFS dependencies 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2022-12-15 12:43:52 +03:00
Denis Kirillov
dd4f66712c [#742] Add multiple listeners 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-11-24 17:48:26 +03:00
Denis Kirillov
d2587b21af [#747] Reload policies on SIGHUP 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-11-18 13:14:25 +03:00
Angira Kekteeva
f5fe9a9b4b [#741] Fix data race 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-11-16 18:19:31 +03:00
Denis Kirillov
8f1bbb7dc1 [#749] Update SDK 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-11-15 10:16:41 +03:00
Denis Kirillov
094eb12578 [#726] Use client time on regular requests 
Use `X-Amz-Date` header as `now` when
* compute expiration epoch
* set Timestamp for object and container
* forming locks
* send notifications

Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-11-14 14:53:55 +03:00
Denis Kirillov
d47840f137 [#568] Add configuration for region to policy map 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-11-11 13:12:49 +03:00
Denis Kirillov
2886ac161c [#740] Fix forming policy by ast 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-11-10 15:33:03 +03:00
Denis Kirillov
16ad0cadcb [#733] Fix container name 
User system name instead of regular.
Set name as CID if name attribute is missed

Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-10-26 17:52:02 +03:00
Denis Kirillov
18a6aca4b4 [#617] api/handler: Simplify tests 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-10-25 19:04:37 +03:00
Angira Kekteeva
712740b668 [#728] Remove redundant log message 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-10-25 19:04:11 +03:00
Angira Kekteeva
bbd7ba670a [#728] Add bucket and object to method logs 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-10-25 19:04:11 +03:00
Angira Kekteeva
da61fd8bcb [#728] Add error message to AbortMultipartUpload 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-10-25 19:04:11 +03:00
Denis Kirillov
0d0ecf5b10 [#650] Drop unused object cache methods 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-10-25 16:00:21 +03:00
Denis Kirillov
821df3d648 [#669] Optimize getNodeVersion for tags operation 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-10-20 17:56:09 +03:00
Denis Kirillov
90eb4f0188 [#713] Add check access control for system caches 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-10-11 15:26:42 +03:00
Denis Kirillov
cb55d36063 [#713] Update tests 
Add bearer token to test context

Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-10-11 15:26:42 +03:00
Denis Kirillov
3d31c2ab4a [#713] Fix cache access 
User can access only keys that he puts into caches

Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-10-11 15:26:42 +03:00
Denis Kirillov
c6098e53f6 [#719] Add tests 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-10-11 11:13:20 +03:00
Denis Kirillov
b3d96e6460 [#719] Improve lock error handling 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-10-11 11:13:20 +03:00
Denis Kirillov
2a41929be3 [#702] Reload resolvers and TLS certs on SIGHUP 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-09-28 15:07:31 +03:00
Denis Kirillov
be4600988e [#709] Add expiration attribute for legal hold 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-09-26 16:51:53 +03:00
Denis Kirillov
3d08562843 [#505] Handle access denied from tree service 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-09-13 18:05:16 +03:00
Denis Kirillov
80beedf13e [#504] Use bktInfo in tree service to check owner 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-09-13 18:05:16 +03:00
Denis Kirillov
4082cd6b54 [#606] Keep eacl records order on conflict 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-09-08 12:21:47 +03:00
Denis Kirillov
ffd259671a [#704] Fix nil pointer dereference on decReader 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-09-07 15:51:44 +03:00
Denis Kirillov
9cd4ef1ac4 [#657] Replace FileName with FilePath attribute 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-09-07 15:50:43 +03:00
Denis Kirillov
6108f8d929 [#624] Update tree service revision 
Remove extra request to tree service to know parentID

Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-09-07 12:17:51 +03:00
Angira Kekteeva
3e4fc52428 [#674] Add AccessKeyID filter 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-09-06 13:47:20 +03:00
Alex Vanin
272c485706 [#669] Use object cache in PutLockInfo optimization 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2022-08-31 13:19:00 +03:00
Alex Vanin
5ffe28a42f [#669] Optimize PutLockInfo 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2022-08-31 13:19:00 +03:00
Denis Kirillov
96c7b79d1c [#683] Forbid copy to itself in unversioned bucket 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-08-30 13:31:48 +03:00
Denis Kirillov
44d5878a80 [#656] Read full body on upload error 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-08-30 09:39:20 +03:00
Alex Vanin
9590b39333 [#682] Return added node id in AddVersion 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2022-08-29 18:24:00 +03:00
Denis Kirillov
5c7a509cc9 [#684] Fix continuation token for directories 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-08-29 16:10:01 +03:00
Denis Kirillov
e38bdae07a [#676] Fix object acl 
Put object acl always add rules to specific version of object.
Get object acl consider READ rights as FULL_CONTROL
because WRITE cannot be applied to object

Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-08-29 13:20:30 +03:00
Denis Kirillov
163038b37d [#672] Fix handling X-Amz-Copy-Source header 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-08-25 17:40:15 +03:00
Denis Kirillov
fdc926e70b [#661] Fix updating cache when request fails 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-08-25 12:55:49 +03:00
Denis Kirillov
1a580b6fa4 [#666] Support tagging directive for CopyObject 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-08-25 12:55:33 +03:00
Denis Kirillov
f013282290 [#660] Fix panic on invalid versioning status 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-08-25 11:50:44 +03:00
Denis Kirillov
24e1f7e1dd [#545] Drop object search 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-08-25 11:49:08 +03:00
Denis Kirillov
c3ad6d2faf [#637] Add header to override CopiesNumber 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-08-19 10:31:26 +03:00
Angira Kekteeva
3b343d1bd5 [#634] Fix tests 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-08-16 17:49:15 +03:00
Angira Kekteeva
5307211398 [#634] Add CopiesNumber in NeoFS requests 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-08-16 17:49:15 +03:00
Denis Kirillov
6aab3936d9 [#625] Limit listing generator with maxKeys 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-08-16 13:47:37 +03:00
Denis Kirillov
3824151699 [#616] Reduce number of requests during listing 
Check if object is directory before request to NeoFS

Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-08-16 13:47:37 +03:00
Denis Kirillov
107d8a9033 [#595] Fix typos and rewording 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-08-13 10:26:00 +03:00
Denis Kirillov
7ab473a688 [#595] Simplify encryption.Params struct 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-08-13 10:26:00 +03:00
Denis Kirillov
94a6a55919 [#595] Move decrypter to separate package 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-08-13 10:26:00 +03:00
Denis Kirillov
d824db7f69 [#595] Allow SSE-C only with TLS 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-08-13 10:26:00 +03:00
Denis Kirillov
87c05eb514 [#595] Add SSE with customer key 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-08-13 10:26:00 +03:00
Angira Kekteeva
1987f271f9 [#577] Remove usages of deprecated io/ioutil 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-08-09 17:20:40 +04:00
Angira Kekteeva
f66e4a77cd [#577] Add IsDeleteMarker() to NodeVersion 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-08-09 17:20:40 +04:00
Angira Kekteeva
2a933f48be [#577] Remove redundant set of isLatest 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-08-09 17:20:40 +04:00
Angira Kekteeva
3188fd8eb7 [#577] Fix delete from tagging cache 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-08-09 17:20:40 +04:00
Angira Kekteeva
8031baddb6 [#577] Fix cache tests 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-08-09 17:20:40 +04:00
Angira Kekteeva
245e64900d [#577] Replace ObjInfo with ExtObjInfo in cache 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-08-09 17:20:40 +04:00
Angira Kekteeva
dc77ad4863 [#577] Fix tests 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-08-09 17:20:40 +04:00
Angira Kekteeva
93e0cae743 [#577] Fix version in GetObjectAttributes 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-08-09 17:20:40 +04:00
Angira Kekteeva
659e7bfc97 [#577] Fix version in GetObject and HeadObject 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-08-09 17:20:40 +04:00
Angira Kekteeva
dfd734b9ec [#577] Separate GetObjectInfo and GetExtendedObjectInfo 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-08-09 17:20:40 +04:00
Angira Kekteeva
37c46e926e [#577] Add Version() to ExtendedObjectInfo 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-08-09 17:20:40 +04:00
Angira Kekteeva
3ac3f1cc9d [#577] Rename objectInfo.Version() to VersionID() 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-08-09 17:20:40 +04:00
Angira Kekteeva
eaf76afe50 [#577] Remove ObjectVersionInfo 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-08-09 17:20:40 +04:00
Angira Kekteeva
2d58b25ea0 [#643] Fix linter issues 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-08-05 15:32:46 +03:00
Angira Kekteeva
1ee1b8bb79 [#643] Fix missing const api.QueryVersionID 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-08-05 15:32:46 +03:00
Angira Kekteeva
072e694540 [#643] Optimize GetObjectTagging 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-08-05 15:32:46 +03:00
Angira Kekteeva
9372534345 [#643] Fix tagging cache 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-08-05 15:32:46 +03:00
Alex Vanin
5219965105 [#498] Remove log string sanitizer 
`zap` lib does all the job anyway.
Suppress CWE-117 from CodeQL.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2022-07-29 09:32:37 +03:00
Denis Kirillov
1fd943ee88 [#610] Fix deleted object removal 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-29 09:31:54 +03:00
Denis Kirillov
5dd658302d [#612] Fix comment 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-26 17:36:21 +03:00
Denis Kirillov
b2e8b1cfb3 [#612] Make listing more robust 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-26 17:36:21 +03:00
Angira Kekteeva
4483c6f57a [#619] Filter system metadata 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-07-26 15:37:13 +03:00
Angira Kekteeva
4380406602 [#619] Remove unused PutSystemObjectParams 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-07-26 15:37:13 +03:00
Alex Vanin
e0136feb73 [#611] Split name processing from object info creation procedure 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2022-07-25 14:23:30 +03:00
Alex Vanin
97f81d3270 [#611] Process delimiters on the object info cache hit 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2022-07-25 14:23:30 +03:00
Alex Vanin
757600608c [#539] Fix imports 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
85eacdb970 [#602] Fix removal non-empty bucket 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
0057f6b7db [#546] Add size and etag in nodeVersionInfo 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Angira Kekteeva
56eb2dc3dc [#569] Rename versioning consts 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Angira Kekteeva
77f466b1d5 [#569] Fix tests 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Angira Kekteeva
72013e08ec [#569] Refactor versioning in tree service 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
9f019c7470 [#542] Add test to check delete marker 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
b380d4ab53 [#542] Clear list cache on object removal 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
f1a0288e3a [#542] Handle unversioned mode 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
fdf0974679 [#542] Fix object removal 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
88c392d024 [#490] Optimize GetObjectTaggingAndLock 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
206a7aa395 [#488] Change cache key types 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
e104855633 [#488] Sync using oid.Address 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
c88a9842db [#488] Sync using oid.ID 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
85c203e157 [#488] Sync using cid.ID 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
dca7b5bc36 [#547] Fix tests 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
42a761c1f2 [#547] Cache ObjectInfo instead of Object 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
127b91a97f [#547] Cache empty lists without error 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
5984f2c2bc [#525] Add comment to clarify anonymous func 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
ad95d1745c [#525] Parallelize listing 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
6ad7c988e6 [#532] Fix object removal 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
12a2060dd0 [#524] Optimize listing 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Angira Kekteeva
3f5afabd4a [#515] Fix nil reference when a part is missing 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
93a7c2eaeb [#514] Missed version id in tags 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
59570ac075 [#513] Don't remove delete mark object 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Alex Vanin
4f3d206422 [#478] Don't fetch epoch at object upload 
Creation epoch was used for versioning. With
tree service, versioning is done on the NeoFS
side in the tree.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
9dfc7e043f [#476] Save full parts info to multipart object 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Alex Vanin
8731dcfd3c [#477] Use tags and lock aggregation in handler 
Reduce amount of tree service RPCs
by using new method in layer package.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Alex Vanin
7fd27e04fe [#477] Provide aggregation of tags and locks in layer.Client 
Adopts GetObjectTaggingAndLock method from
TreeClient in layer.Client with caching.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Alex Vanin
f8496973b8 [#477] Aggregate fetching tags and lock in layer.TreeService 
New method aggregates GetObjectTagging and GetLock
methods in single RPC to the tree service.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
a02900a4f7 [#474] Use appropriate null version during listing 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
c8e8ba9f6a [#476] Fix parts info for GetObjectAttributes 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
9b1ccd39be [#475] Add extra attributes to partInfo 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
5b9a5cadef [#452] Fix lint issues 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
1a456eaa8b [#452] Remove unused 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
9fb3fb1274 [#452] Fix versions tests 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
5529fb914e [#452] Fix lock tests 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
dd534e8738 [#451] Handle lock objects using tree service 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
70957d75fd [#417] Complete multipart upload using tree service 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
6b2ddcadd0 [#417] Abort multipart upload using tree service 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
58f2bf44b1 [#417] List parts using tree service 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
bc0bdc7767 [#417] Upload part using tree service 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
e1b9a4432a [#417] List multipart uploads using tree service 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
13e01164d7 [#417] Create multipart upload using tree service 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Angira Kekteeva
b6a43c6f4c [#449] Replace error checks and literals 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Angira Kekteeva
385a2102f0 [#449] Add tree service for bucket tagging 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Angira Kekteeva
99feb1d936 [#449] Add tree service for object tagging 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
9c74cca9af [#420] Using tree service to list object versions 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
25477cdaf8 [#416] Use tree service to list objects 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
7e8b57605a [#413] Fix delete object without version 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
37b1baed41 [#413] Refactor 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
977f176713 [#413] Use tree service to delete object 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
49bd77d9cf [#413] Use tree service to head objects 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
36f3c43af5 [#413] Use tree service to put object 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Angira Kekteeva
ab5c44ac14 [#429] Refactor TreeService for notif,cors,settings 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Angira Kekteeva
8eff857e41 [#429] Add tree service for cors 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Angira Kekteeva
7520952792 [#429] Add tree service for notifications 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Angira Kekteeva
b03ae827fb [#429] Refactor tree service 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
9ed081db3e [#407] Add service tree client prototype 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Alex Vanin
d7f77ce874 [#574] Produce deny records for private objects in put-object-acl 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2022-07-21 17:55:15 +03:00
Alex Vanin
66fe3fee7b [#574] Produce deny records for private objects 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2022-07-21 17:55:15 +03:00
Denis Kirillov
7ba7e7dc4d [#590] Make service records valid 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-20 19:10:23 +03:00
Denis Kirillov
1e26cf1541 [#590] Use service records to save resource info 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-20 19:10:23 +03:00
Angira Kekteeva
b144e50f7f [#584] Refactor formRecords func 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-07-19 11:54:27 +03:00
Angira Kekteeva
3f4a55f39e [#584] Fix order in astToTable 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-07-19 11:54:27 +03:00
Angira Kekteeva
260fb95677 [#584] Fix order in tableToAst 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-07-19 11:54:27 +03:00
Angira Kekteeva
74300a75a9 [#584] Add tests 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-07-19 11:54:27 +03:00
Alex Vanin
aad4862c59 [#564] Don't parse lock attribute if it is omitted in container 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2022-07-15 01:33:48 +04:00
Denis Kirillov
1575da65a4 [#573] Fix object acl filters 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-14 13:33:11 +03:00
Alex Vanin
a57b8d34d3 [#553] Add more comments about eacl.RoleUnknown 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2022-07-07 12:05:25 +03:00
Alex Vanin
06d043e1eb [#553] Optimize target formation with multiple keys 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2022-07-07 12:05:25 +03:00
Alex Vanin
d6065c64c4 [#553] Check group grantee based on stored list of users 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2022-07-07 12:05:25 +03:00
Alex Vanin
c7de7d2928 [#553] Do not use user role with public keys in eacl target 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2022-07-07 12:05:25 +03:00
Alex Vanin
36029ca864 [#580] Fix user removal in astOperation 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2022-07-06 17:40:41 +03:00
Leonard Lyubich
5bfc549746 [#564] neofs: Merge if with same condition in CreateContainer 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-07-04 23:42:50 +04:00
Leonard Lyubich
4a8a248f34 [#564] Upgrade NeoFS SDK Go with changed container API 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-07-04 23:42:50 +04:00
Denis Kirillov
6e1a1f3839 [#522] Suppress CodeQL error 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-04 11:03:55 +03:00
Denis Kirillov
9f740b9683 [#289] Add detect mimetype by extension 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-06-29 19:13:16 +03:00
Denis Kirillov
f72bc538b9 [#551] Refactor notifications logs 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-06-27 17:31:17 +03:00
Denis Kirillov
35f55c5af5 [#539] Fix tests 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-06-27 02:23:19 +04:00
Denis Kirillov
7ca519cb32 [#539] Add context to errors 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-06-27 02:23:19 +04:00
Leonard Lyubich
e1f1e6d960 [#541] Upgrade NeoFS SDK Go with changed basic ACL API 
SDK now provides dedicated type for basic ACL with convenient interface.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-06-24 16:43:50 +03:00
Leonard Lyubich
a8fc313ff5 [#544] layer: Remove no longer needed deleteContainer method 
Call `DeleteContainer` on `NeoFS` component directly.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-06-24 13:00:26 +03:00
Leonard Lyubich
f596c8be06 [#544] layer: Add session token parameter to DeleteBucketParams 
Add `DeleteBucketParams.SessionToken` field in order to unify the
parameters with `CreateBucketParams` and `PutBucketACLParams`.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-06-24 13:00:26 +03:00
Leonard Lyubich
028a152e04 [#544] Upgrade NeoFS SDK Go with another approach of container sessions 
After recent changes in NeoFS SDK Go library session tokens aren't
embedded into `container.Container` and `eacl.Table` structures.
Instead, the operations of storing given values in NeoFS are
parameterized by elements of the corresponding type.

Add dedicated session parameters to operations of bucket and eACL
setting.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-06-24 13:00:26 +03:00
Denis Kirillov
818176e7e1 [#528] Adopt aws v4signer 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-06-20 09:44:25 +03:00
Leonard Lyubich
f0749fd23e [#537] Upgrade NeoFS SDK Go with changed netmap package 
`PlacementPolicy` type now provides methods to work with QL-encoded
policies. System network parameters can be read using dedicated method
without iterating. Applications can work with `PlacementPolicy`
variables directly so there is no need to use pointers.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-06-17 08:28:59 +03:00
Angira Kekteeva
cfe7591cf7 [#523] Add putObjectACL notification 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-06-17 01:43:46 +04:00
Angira Kekteeva
dd0d21b690 [#523] Fix typo 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-06-17 01:43:46 +04:00
Angira Kekteeva
bd5fd041b7 [#523] Add object tagging notifications 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-06-17 01:43:46 +04:00
Denis Kirillov
d521af2065 [#529] Add presign URLs support 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-06-16 17:17:45 +03:00
Leonard Lyubich
880ffe7108 [#492] layer: Don't parameterize basic ACL of created containers 
`CreateBucket` handler always creates containers with extended public
ACL, so there is no need to configure it in `NeoFS.CreateContainer`.

Make internal `NeoFS` implementation to create containers with
`eacl-public-read-write` basic ACL if corresponding parameter is unset.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-06-16 10:46:50 +03:00
Angira Kekteeva
a8bff13801 [#495] Add acl to copy-object 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-06-07 17:36:12 +03:00
Denis Kirillov
de7281ac58 [#465] Unify log messages 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-06-07 17:20:26 +03:00
Denis Kirillov
300d4359d8 [#465] Handle cache cast failure 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-06-07 17:20:26 +03:00
Denis Kirillov
f00ca1b6c4 [#438] Drop layer/neofs package 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-06-07 13:59:45 +03:00
Alex Vanin
4ed939773b [#487] Use updated SDK method to get bearer issuer 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2022-06-03 11:20:47 +03:00
Angira Kekteeva
dbfac29171 [#487] Fix tests 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-06-03 10:57:56 +03:00
Angira Kekteeva
e904ed51c7 [#487] Optimize bucketInfo in initObjectPayloadReader 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-06-03 10:57:56 +03:00
Angira Kekteeva
4767eeed8c [#487] Remove attach of bearer token 
When bucket owner is not an issuer of the bearer token

Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-06-03 10:57:56 +03:00
Leonard Lyubich
4f43aad495 [#485] Upgrade SDK with latest bearer package API 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-06-03 09:38:49 +03:00
Denis Kirillov
ea252421f5 [#484] Add tests 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-06-03 09:35:06 +03:00
Denis Kirillov
f282e877e2 [#484] Handle conditional headers 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-06-03 09:35:06 +03:00