EliChin/frostfs-s3-gw
1
0
Fork 0
forked from TrueCloudLab/frostfs-s3-gw
Code Issues Pull requests Projects Releases Packages Wiki Activity
1365 commits 6 branches 19 tags 19 MiB
Commit graph

142 commits

Author SHA1 Message Date
Denis Kirillov
b35f146cec [TrueCloudLab#20] Renew tokens beforehand 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-02-03 16:59:49 +03:00
Denis Kirillov
361d1d3881 [TrueCloudLab#16] Update go version to 1.18 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-01-25 10:39:44 +03:00
Artem Tataurov
9ad7982807 [#14] Update neo-go and viper 
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
 2023-01-25 10:23:16 +03:00
Denis Kirillov
cafe079072 [TrueCloudLab#13] Update frostfs SDK 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-01-23 13:59:32 +03:00
Alex Vanin
9473335234 [#11] Update FrostFS SDK 
Contains debug logs for switching
connections in pool.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-01-18 12:22:44 +03:00
Denis Kirillov
19d8f8fcfe [#7] Update SDK 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-01-10 11:24:43 +03:00
Alex Vanin
96dff367db [#1] Build S3 Gateway with FrostFS dependencies 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2022-12-15 12:43:52 +03:00
Denis Kirillov
d1825f97fb [#750] Update SDK to support timeout for stream 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-11-15 18:10:28 +03:00
Denis Kirillov
8f1bbb7dc1 [#749] Update SDK 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-11-15 10:16:41 +03:00
Denis Kirillov
a3cae5ff56 [#712] Update SDK to support interrupt pool dial 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-10-11 16:55:26 +03:00
Denis Kirillov
c051d21420 [#657] Update SDK 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-09-07 15:50:43 +03:00
Denis Kirillov
ece40d5972 [#687] Update SDK 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-08-29 15:03:04 +03:00
Denis Kirillov
e4ec983213 [#678] Update SDK to fix panic on go1.19 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-08-25 11:33:17 +03:00
Denis Kirillov
70ed9d79d6 [#665] Fix go mod 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-08-23 10:26:55 +03:00
Angira Kekteeva
7a6d562c69 [#634] Update SDK 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-08-16 17:49:15 +03:00
Denis Kirillov
87c05eb514 [#595] Add SSE with customer key 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-08-13 10:26:00 +03:00
Denis Kirillov
5d3eb836cf [#636] Update SDK to have EACLNotFound status 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-08-01 20:47:31 +03:00
Denis Kirillov
f685b0e68a [#614] Update SDK with pool that provides stats 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-25 14:24:35 +03:00
Denis Kirillov
ad95d1745c [#525] Parallelize listing 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Denis Kirillov
9ed081db3e [#407] Add service tree client prototype 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 15:19:16 +03:00
Leonard Lyubich
edc6aa2f88 [#572] Upgrade NeoFS SDK Go to v1.0.0-rc.5 
Use `pool.SyncContainerWithNetwork` function to correctly compose new
container in `neofs.NeoFS.CreateContainer`.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-07-06 17:36:27 +03:00
Leonard Lyubich
4a8a248f34 [#564] Upgrade NeoFS SDK Go with changed container API 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-07-04 23:42:50 +04:00
Leonard Lyubich
ef4550ba45 [#557] Upgrade NeoFS SDK Go with fixed policy parser 
Now policy parser doesn't print failures into stdout but returns the
first one in `error` value.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-06-28 17:50:24 +03:00
Leonard Lyubich
e1f1e6d960 [#541] Upgrade NeoFS SDK Go with changed basic ACL API 
SDK now provides dedicated type for basic ACL with convenient interface.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-06-24 16:43:50 +03:00
Leonard Lyubich
028a152e04 [#544] Upgrade NeoFS SDK Go with another approach of container sessions 
After recent changes in NeoFS SDK Go library session tokens aren't
embedded into `container.Container` and `eacl.Table` structures.
Instead, the operations of storing given values in NeoFS are
parameterized by elements of the corresponding type.

Add dedicated session parameters to operations of bucket and eACL
setting.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-06-24 13:00:26 +03:00
Leonard Lyubich
f0749fd23e [#537] Upgrade NeoFS SDK Go with changed netmap package 
`PlacementPolicy` type now provides methods to work with QL-encoded
policies. System network parameters can be read using dedicated method
without iterating. Applications can work with `PlacementPolicy`
variables directly so there is no need to use pointers.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-06-17 08:28:59 +03:00
Alex Vanin
5485adc477 [#422] Update NeoFS SDK 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2022-06-09 10:30:57 +03:00
Leonard Lyubich
4f43aad495 [#485] Upgrade SDK with latest bearer package API 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-06-03 09:38:49 +03:00
Denis Kirillov
6cf7dc4010 [#460] Update sdk 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-05-31 11:20:41 +03:00
Leonard Lyubich
087d500c5f [#458] *: Refactor working with NeoFS identities 
Pull latest changes from NeoFS SDK Go library. Decrease redundant and
unsafe usage of ID pointers. Use `EncodeToString` method in order to
calculate protocol strings.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-05-27 17:22:37 +04:00
Angira Kekteeva
01fd43c1b0 [#434] Update go version to 1.17 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-05-16 10:49:51 +03:00
Alex Vanin
1c33f06bfe [#428] Update SDK 
Includes:
- container removal fix
- new session token structure: authmate does not
  parse session context anymore, instead it is
  application defined flexible structure with
  container ID encoded in human-readable format

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2022-05-06 14:47:06 +04:00
Denis Kirillov
e3c16a32dd [#409] Update SDK 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-04-26 12:51:52 +04:00
Denis Kirillov
7710de39ec [#365] Update SDK 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-04-08 17:12:52 +03:00
Denis Kirillov
94406b08f3 [#376] Update NeoFS SDK to v1.0.0-rc.3 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-03-16 15:44:19 +03:00
Leonard Lyubich
20b8e3d249 [#346] Upgrade NeoFS SDK Go to 2nd release candidate v1.0.0 
Avoid using the deprecated elements.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-03-04 00:14:30 +03:00
Leonard Lyubich
dc628aef22 [#346] Upgrade NeoFS SDK Go module 
Use `pool.ResObjectSearch.Iterate` method in `NeoFS.SelectObjects`
implementation.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-03-04 00:14:30 +03:00
Leonard Lyubich
34a221c5c9 [#346] Upgrade NeoFS SDK Go library 
Core changes:
  - `object.ID` moved to new package `oid`;
  - `object.Address` moved to new package `address`;
  - `pool.Object` interface changes.

Additionally:
  - Set container owner in `Agent.IssueSecret`.
  - Remove no longer needed fields from `GetObjectParams`
  - `Length` and `Offset` are never assigned. These values
  are set in `Range` field.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-03-04 00:14:30 +03:00
Angira Kekteeva
3277293bb3 [#341] Add notifications controller 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-02-04 14:41:22 +03:00
Denis Kirillov
8fc19b11db [#333] Update sdk 
Update sdk to fix "invalid session token owner" error

Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-01-28 12:42:40 +03:00
Denis Kirillov
12c15504ef [#297] Update neofs-sdk-go and neo-go 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2021-12-30 13:00:23 +03:00
Denis Kirillov
befe084900 [#219] Add container name resolving 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2021-11-24 13:24:39 +03:00
Denis Kirillov
d36dfe8c61 [#271] Update neo-sdk-go to the latest version 
Refactoring invoking pool methods for anonymous requests.

Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2021-11-16 14:42:08 +03:00
Denis Kirillov
214c55431b [#258] Treat lifetime as time.Duration 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2021-11-08 12:41:22 +03:00
Angira Kekteeva
44a2f1b471 [#234] *: Update go.mod 
Remove neofs-node dep
Update versions neofs reps

Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2021-08-30 17:01:41 +03:00
Angira Kekteeva
295be71f00 [#206] api: Add objects cache 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2021-08-18 10:16:27 +03:00
Angira Kekteeva
d33463a734 [#141] Update api-go and sdk-go versions 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2021-07-07 18:03:25 +03:00
Angira Kekteeva
4d05a68f89 [#138] Update sdk-go and api-go versions 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2021-07-06 17:44:55 +03:00
Angira Kekteeva
aac1ac2868 [#119] Add null versioning and handler 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2021-07-06 12:11:03 +03:00
Angira Kekteeva
172eaa2ed6 [#88] Remove tov2, update api-go version 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2021-06-30 13:44:25 +03:00
Denis Kirillov
3853111293 [#47] Fixed dependencies 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2021-06-24 11:44:17 +03:00
Denis Kirillov
fd8130a42f [#92] Remove keys generation 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2021-06-23 14:57:33 +03:00
Denis Kirillov
b5c8befa25 [#92] Support NEP-6 wallet 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2021-06-22 17:10:14 +03:00
Angira Kekteeva
44da6cf70c [#72] authmate: Add session token 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2021-06-17 14:09:58 +03:00
Denis Kirillov
fba29a9a66 [#78] Using client.Object from sdk 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2021-06-15 17:06:19 +03:00
Denis Kirillov
2e96ce6dcd [#73] Dropped grpc keepalive options 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2021-06-11 09:49:32 +03:00
Angira Kekteeva
abfaa0bd48 [#68] go.mod: Update api-go and sdk-go versions 
Updated version of api-go from 1.26.1 to 1.27.0.
Removed replace in go.mod.
Updated version of sdk-go.

Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2021-06-04 16:00:58 +03:00
Roman Khimov
d15687f37c *: drop creds/neofs 
Signed-off-by: Roman Khimov <roman@nspcc.ru>
 2021-05-28 23:48:39 +03:00
Roman Khimov
35ffc1fbc1 *: use sdk-go instead of http-gw 
Now that the packages have moved there we can do it.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
 2021-05-28 14:05:46 +03:00
Roman Khimov
d19ce03072 *: drop old sdk dependecies, bump neofs-api-go version 
I'm not sure it works, but it's enough code-wise for now. We're reusing some
http-gw components here that are to be moved into sdk-go in future.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
 2021-05-26 21:01:46 +03:00
Angira Kekteeva
acc84a6464 [#46] authmate: Add authmate'dependencies to go.mod 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2021-05-19 12:27:41 +03:00
Evgeniy Kulikov
26636235d6 Update dependencies 
- update dependencies
  - github.com/aws/aws-sdk-go v1.37.9
  - github.com/nspcc-dev/cdn-sdk v0.3.4
  - github.com/nspcc-dev/neofs-api-go v1.23.0

Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
 2021-02-12 12:27:22 +03:00
Evgeniy Kulikov
cdc21422ff Refactoring Makefile 
- add `help.mk`
- add target to build and publish image
- update README

Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
 2021-02-08 12:45:18 +03:00
Evgeniy Kulikov
8467a07c9b Update dependencies and fixes 
- Closes #39
- Cleanup code
- Make code more readable
- Update dependencies
	- github.com/aws/aws-sdk-go v1.37.1
	- github.com/google/uuid v1.2.0
	- github.com/nspcc-dev/cdn-sdk v0.3.3
	- github.com/nspcc-dev/neofs-api-go v1.22.2

Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
 2021-02-01 19:59:42 +03:00
Evgeniy Kulikov
c26045007b Release v0.14.1 with CDN SDK v0.3.1 and NeoFS API Go v1.22.1 
- github.com/aws/aws-sdk-go v1.36.26 => v1.36.28
- github.com/google/uuid v1.1.4 => v1.1.5
- github.com/nspcc-dev/cdn-sdk v0.3.0 => v0.3.1
- github.com/nspcc-dev/neofs-api-go v1.22.0 => v1.22.1

Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
 2021-01-18 11:04:54 +03:00
Evgeniy Kulikov
2a93a216f8 Migrate to SDK 0.3.0 and fixes 
- fix displaying list objects
- simplify `ListObjects`
- simplify `GetObjectHandler`
- simplify `HeadObjectHandler`
- add method for `GetBucketVersioningHandler`
- add method for `ListMultipartUploadsHandler`
- improvements for `HeadObjectHandler`, to display folders meta
- update dependencies
  - github.com/aws/aws-sdk-go v1.36.26
  - github.com/google/uuid v1.1.4
  - github.com/gorilla/mux v1.8.0
  - github.com/nspcc-dev/cdn-sdk v0.3.0
  - github.com/nspcc-dev/neofs-api-go v1.22.0
  - github.com/prometheus/client_golang v1.9.0
  - github.com/stretchr/testify v1.7.0
  - google.golang.org/grpc v1.35.0

Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
 2021-01-14 20:39:48 +03:00
Evgeniy Kulikov
d7a36f2fb4 Migrate to CDN SDK release 
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
 2020-12-02 12:36:18 +03:00
Evgeniy Kulikov
5752f2f89f Add method to fetch SecretAccessKey from BearerToken 
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
 2020-11-27 16:33:56 +03:00
Evgeniy Kulikov
67f2239e6b Migrate to NeoFS API Go v1.20.3 
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
 2020-11-27 15:37:25 +03:00
Evgeniy Kulikov
d8d6dc7593 Upgrade dependencies 
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
 2020-11-24 09:57:40 +03:00
Evgeniy Kulikov
258993551c Prepare go modules to build 
Need `GOPRIVATE=github.com/nspcc-dev` while repos in private:
- github.com/nspcc-dev/neofs-api-go
- github.com/nspcc-dev/neofs-authmate

Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
 2020-11-09 15:28:42 +03:00
Evgeniy Kulikov
4d605d1113 [#25] Refactoring and make fixes 
closes #25
closes #32

Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
 2020-10-23 03:21:14 +03:00
Evgeniy Kulikov
fbd4a83602 [#25] Refactoring over api/layer 
closes #25

Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
 2020-10-22 03:19:16 +03:00
Evgeniy Kulikov
ec56a1818e [#25] Upgrade dependencies 
closes #25

Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
 2020-10-19 04:03:32 +03:00
Evgeniy Kulikov
339c3d934d [#25] Upgrade dependencies 
closes #25

Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
 2020-10-13 12:30:12 +03:00
Pavel Korotkov
f6cb0a6ecc [#8] Update deps 
Signed-off-by: Pavel Korotkov <pkorotkov@gmail.com>
 2020-08-07 14:14:52 +03:00
Evgeniy Kulikov
0161d2fbd3 [#13] Rename go module name according to NSPCC standards 
- refactoring s3 gate structure
- cleanup unused code
- rename go module to `github.com/nspcc-dev/neofs-s3-gate`

closes #13

Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
 2020-08-06 15:02:13 +03:00
Evgeniy Kulikov
4c6572141b Update dependencies (cleanup from legacy code usage) 2020-08-06 13:51:03 +03:00
Evgeniy Kulikov
2731246239 Refactoring ObjectLayer 
- simplify layer.Client
- refactoring neofs pool client

Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
 2020-08-03 14:51:28 +03:00
Evgeniy Kulikov
b12b5008e3 NFSSVC-27 Update dependencies 2020-07-28 01:55:08 +03:00
Pavel Korotkov
f6aeb06ee9 Fast hot fix before rework of auth scheme 2020-07-24 17:03:02 +03:00
Pavel Korotkov
5d0505001c Update deps 2020-07-21 02:42:48 +03:00
Evgeniy Kulikov
170f9b4b08 Update scripts for building docker image 2020-07-14 14:33:52 +03:00
Pavel Korotkov
fc0a329f0f Update deps 2020-07-08 14:16:10 +03:00
Pavel Korotkov
0f14addfa3 Remove the cbrotli dep; set neofs-api-go a specific commit 2020-07-08 10:39:32 +03:00
Pavel Korotkov
2a1a8aa379 Move auth file to layer; add RSA keys 2020-07-08 02:37:27 +03:00
Pavel Korotkov
ac500c8d09 Update deps 2020-07-07 10:10:05 +03:00
Evgeniy Kulikov
f117e2207d Remove minio/gateways 2020-07-06 12:22:09 +03:00
Evgeniy Kulikov
f36a911914 Prepare base application 2020-07-06 12:18:16 +03:00
Evgeniy Kulikov
8d97dcbf10 Cleanup + Makefile + Dockerfile 2020-07-03 18:08:57 +03:00
Evgeniy Kulikov
be43700e8f Add NeoFS ObjectLayer and connection pool 2020-07-03 16:58:49 +03:00
Evgeniy Kulikov
9bf57615b0 Initial commit based on https://github.com/minio/minio/releases/tag/RELEASE.2020-07-02T00-15-09Z 2020-07-03 15:03:06 +03:00