Commit graph

153 commits

Author SHA1 Message Date
Angira Kekteeva
ccf5db95a5 [#217] Refactor system objects
Moved into a separate file
getSystemObject renamed to headSystemObject, implemented getSystemObject
for system objects with payload
Refactor putSystemObjects
Moved systemCacheKey from data system_object

Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-10-12 09:03:09 +03:00
Denis Kirillov
3231ecab03 [#263] Add LWW to system objects
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-09-23 20:37:55 +03:00
Denis Kirillov
19b917e3b5 [#253] Caches refactoring
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-09-14 10:46:18 +03:00
Denis Kirillov
458f9cf17b [#242] Fix versions sorting
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-09-07 17:06:22 +03:00
Angira Kekteeva
be08596c22 [#231] cache: Add invalidation of ListObjectsCache
In put operations ListObjectsCache remove entries which can contain
put object

Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-09-07 11:08:48 +03:00
Angira Kekteeva
1ece42b23f [#236] cache: Refactor ListObjectsCache
Replaced map in ListObjectsCache by gcache.
Now ListObjectsCache keeps only objectIDs and
requests ObjectInfo from cache or NeoFS.
Refactored ListObjectsCache keys: removed delimiter and method fields.
Now ListObjectsCache keeps cache with all objects versions.

Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-09-07 11:08:48 +03:00
Angira Kekteeva
1bc2e51cbc [#236] api: Refactor caches: ObjectsList, Objects
Move ObjectsList from layer to cache package
Rename object_cache.go to objects.go

Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-09-07 11:08:48 +03:00
Angira Kekteeva
239742f413 [#236] api: Create info.go for basic structs
Moved BucketInfo and ObjectInfo from layer and handler to api

Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-09-07 11:08:48 +03:00
Denis Kirillov
8b5ebe2ec2 [#196] Add PutObjectTagging
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-08-31 10:46:42 +03:00
Angira Kekteeva
dcac1aa3e3 [#230] layer: Change objects_list_cache's key
Remove access_key from objectsListCache key

Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-08-26 12:54:58 +03:00
Denis Kirillov
4bb885d526 [#122] Refactor
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-08-25 14:49:21 +03:00
Denis Kirillov
11558124cd [#122] Add versioning cache
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-08-25 09:15:00 +03:00
Denis Kirillov
f6c51cc9ee [#122] Update listObjectVerions
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-08-25 09:15:00 +03:00
Denis Kirillov
9c058a70fd [#122] Add tests
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-08-25 09:15:00 +03:00
Denis Kirillov
43185de52a [#122] Add list object versions
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-08-25 09:15:00 +03:00
Denis Kirillov
d5aef7566f [#122] Add delete versioned object
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-08-25 09:15:00 +03:00
Denis Kirillov
3130784ee6 [#122] Add getting specific object version
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-08-25 09:13:12 +03:00
Denis Kirillov
f463522f34 [#122] Add versioning put object
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-08-25 09:10:46 +03:00
Denis Kirillov
feb45d0633 [#122] Add replacing objects
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-08-25 09:10:46 +03:00
Angira Kekteeva
295be71f00 [#206] api: Add objects cache
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-08-18 10:16:27 +03:00
Angira Kekteeva
ae19eb4ad4 [#206] Refactor ListObjectsCache
Replaced conflicting names

Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-08-18 10:16:27 +03:00
Denis Kirillov
d332096598 [#199] Refactor
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-08-11 13:17:37 +03:00
Angira Kekteeva
8a69c7cca0 [#179] api: Add cache for ListObjectsV1/V2
Refactored cache for ListObjects:
made cache common for all versions,
simplified: remove dependendence on token/startafter
add mitable cachelifetime.

Refactored listobjects

Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-08-09 13:52:11 +03:00
Angira Kekteeva
0ceea95e11 [#112] Add cache to ListObjects and layer
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-07-28 17:05:11 +03:00
Angira Kekteeva
97a7d16f68 [#112] api: Refactor allObjectsParams struct
Replaced Bucket (means name) by BucketInfo struct

Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-07-26 23:22:05 +03:00
Alex Vanin
3f7941fdf3
Merge pull request #174 from nspcc-dev/feature/159-optional_owner
[#159] Add fetch-owner param
2021-07-26 13:32:04 +03:00
Denis Kirillov
63652d213c [#149] Removed grpc errors
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-07-22 09:54:12 +03:00
Denis Kirillov
65a61a9f7d [#159] Add fetch-owner param
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-07-20 15:40:38 +03:00
Denis Kirillov
c05e2d6757 [#160] Fix head dir object
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-07-20 12:36:41 +03:00
Angira Kekteeva
ee84062154 [#154] api: refactor ListObjectsV1 and V2
Separate ListObject for different versions.
Remove useless grouping of keys on V2.

Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-07-19 10:12:47 +03:00
Denis Kirillov
a6ec27b40d [#93] Object ETag support
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-06-30 12:55:31 +03:00
Denis Kirillov
c4fef3d948 [#96] Support ranges for GetObject
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-06-25 12:18:52 +03:00
Denis Kirillov
6a135a6bb4 [#82] Using bearer token
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-06-15 17:16:26 +03:00
Denis Kirillov
fba29a9a66 [#78] Using client.Object from sdk
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-06-15 17:06:19 +03:00
Angira Kekteeva
b3b190adc0 [#68] *: Replace deprecated
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-06-04 16:01:42 +03:00
Roman Khimov
d15687f37c *: drop creds/neofs
Signed-off-by: Roman Khimov <roman@nspcc.ru>
2021-05-28 23:48:39 +03:00
Roman Khimov
d19ce03072 *: drop old sdk dependecies, bump neofs-api-go version
I'm not sure it works, but it's enough code-wise for now. We're reusing some
http-gw components here that are to be moved into sdk-go in future.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2021-05-26 21:01:46 +03:00
Angira Kekteeva
9d496d70a7 [#53] *: Remove external pkg/errors dependency
Replaced functions from pkg/errors by functions from errors, fixed not fully correct comment

Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-05-20 15:50:14 +03:00
Angira Kekteeva
a399590204 Replace s3-gate by s3-gw
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-05-18 14:12:24 +03:00
Evgeniy Kulikov
2a93a216f8 Migrate to SDK 0.3.0 and fixes
- fix displaying list objects
- simplify `ListObjects`
- simplify `GetObjectHandler`
- simplify `HeadObjectHandler`
- add method for `GetBucketVersioningHandler`
- add method for `ListMultipartUploadsHandler`
- improvements for `HeadObjectHandler`, to display folders meta
- update dependencies
  - github.com/aws/aws-sdk-go v1.36.26
  - github.com/google/uuid v1.1.4
  - github.com/gorilla/mux v1.8.0
  - github.com/nspcc-dev/cdn-sdk v0.3.0
  - github.com/nspcc-dev/neofs-api-go v1.22.0
  - github.com/prometheus/client_golang v1.9.0
  - github.com/stretchr/testify v1.7.0
  - google.golang.org/grpc v1.35.0

Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2021-01-14 20:39:48 +03:00
Evgeniy Kulikov
8168181658 Remove bufio writer and double creation of OffsetWriter
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-12-10 18:15:18 +03:00
Evgeniy Kulikov
f4263d50a7 Refactoring api/layer
- don't require token.BearerToken
- simplify method layer.objectPut
- add method to check that object exists
- add method to fetch Owner (from context or sdk client)

Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-11-27 15:36:15 +03:00
Evgeniy Kulikov
0284bf136b Migrate to Credentials client
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-11-24 12:17:36 +03:00
Evgeniy Kulikov
0ee3a5f9ba Migrate api/layer to SDK
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-11-24 10:01:38 +03:00
Evgeniy Kulikov
256850b8fe [#25] Refactoring and make fixes
closes #25
closes #33

Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-10-24 16:09:22 +03:00
Evgeniy Kulikov
4d605d1113 [#25] Refactoring and make fixes
closes #25
closes #32

Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-10-23 03:21:14 +03:00
Evgeniy Kulikov
fbd4a83602 [#25] Refactoring over api/layer
closes #25

Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-10-22 03:19:16 +03:00
Evgeniy Kulikov
7deeb68d47 [#25] Migrate layer to NeoFS API v2
Naive migration, without any tries to start application

closes #25

Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-10-19 04:04:37 +03:00
Evgeniy Kulikov
5eae2aac78 Change NotFound error for objectFindID
method should return gRPC status error

Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-08-22 05:39:12 +03:00
Evgeniy Kulikov
21591ebe65 Add method to find all ID's for the nice names
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-08-20 22:33:59 +03:00
Evgeniy Kulikov
edd44ea981 Check that aws filename wasn't set
we should check that filename header wasn't set, otherwise we should not change custom filename

Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-08-20 02:32:42 +03:00
Evgeniy Kulikov
64429d736d Refactoring API layer
- logging RequestID
- should return error, when headers already received

Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-08-11 14:32:04 +03:00
Evgeniy Kulikov
0161d2fbd3 [#13] Rename go module name according to NSPCC standards
- refactoring s3 gate structure
- cleanup unused code
- rename go module to `github.com/nspcc-dev/neofs-s3-gate`

closes #13

Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
2020-08-06 15:02:13 +03:00
Renamed from neofs/layer/object.go (Browse further)