potyarkin/frostfs-http-gw
1
0
Fork 0
forked from TrueCloudLab/frostfs-http-gw
Code Pull requests Activity
378 commits 1 branch 22 tags 3.2 MiB
Commit graph

52 commits

Author SHA1 Message Date
Alex Vanin
2d9eee81c2 [#1] Build HTTP Gateway with FrostFS dependencies 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2022-12-15 13:26:08 +03:00
Denis Kirillov
b2db6300c4 [#222] Fix zip streaming 
Skip objects with invalid FilePath

Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-11-08 14:34:33 +03:00
Denis Kirillov
b567a08a85 [#206] Refactor neofs 'not found' errors 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-09-30 10:06:05 +03:00
Denis Kirillov
3fa0e3caef [#200] Update docs 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-09-16 17:08:20 +03:00
Denis Kirillov
82eba97505 [#200] Unregister metrics on shutdown to fix test 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-09-16 17:08:20 +03:00
Denis Kirillov
ce84dc7068 [#200] Reload upload/download settings on SIGHUP 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-09-16 17:08:20 +03:00
Denis Kirillov
9ebcb6b964 [#188] Update SDK to avoid panic on go1.19 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-08-24 17:57:40 +03:00
Denis Kirillov
0fb75a1cde [#178] Update SDK to v1.0.0-rc.5 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-25 14:27:24 +03:00
Denis Kirillov
85ee0c44a2 [#163] Fix zip streaming 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-06-20 09:42:06 +03:00
Denis Kirillov
a42606742a [#142] Support resolving container nicename 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-04-28 17:18:13 +03:00
Denis Kirillov
2b780c1772 [#142] Update SDK 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-04-28 17:18:13 +03:00
Elizaveta Chichindaeva
11283c1c79 [#153] English Check 
Signed-off-by: Elizaveta Chichindaeva <elizaveta@nspcc.ru>
 2022-04-22 18:05:51 +03:00
Alex Vanin
1e3df95eed [#145] Use application context in NeoFS API requests 
It is meaningless to use RequestCtx as a context.Context
for NeoFS operation, because context won't be closed
until application shutdown. Moreover, it also triggers
data race detection, because server's done channel, which
is accessible for reading from RequestCtx, is set to `nil`.

Using application context doesn't change gateway behavior,
but it suppresses data race trigger at shutdown. It also
allows possibility to set configurable timeouts for NeoFS
networking if we will ever need them.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2022-04-22 15:33:11 +04:00
Denis Kirillov
4f2b21a14b [#144] Add more verbose error output 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-04-22 15:14:52 +04:00
Denis Kirillov
1c2fec8182 [#147] Download zip by FilePath attribute 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-04-19 20:10:00 +04:00
Denis Kirillov
baf425453d [#139] fix lint 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-04-08 00:23:12 +04:00
Denis Kirillov
5080b43a04 [#139] Update SDK 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-04-08 00:23:12 +04:00
Denis Kirillov
6d362894ad [#132] Update NeoFS SDK to v1.0.0-rc.3 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-03-16 17:15:21 +03:00
Leonard Lyubich
f6ab94027c [#128] downloader: Avoid var naming collision 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-03-04 13:43:34 +03:00
Leonard Lyubich
271451dc32 [#128] downloader: Simplify detecting the Content-Type from payload 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-03-04 13:43:34 +03:00
Leonard Lyubich
c482bbd25a [#126] downloader: Replace Read with Iterate on ObjectListReader 
Make `Downloader.DownloadZipped` to call `Iterate` method instead of
`Read` one during processing the `ObjectListReader`.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-03-03 18:25:53 +03:00
Leonard Lyubich
2b7e4a36fb [#126] Upgrade NeoFS SDK Go library 
Core changes:
  - `object.ID` moved to new package `oid`;
  - `object.Address` moved to new package `address`;
  - `pool.Object` interface changes.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-03-03 18:25:53 +03:00
Denis Kirillov
39c29c6d6a [#125] Set object ids in a separate function 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-02-15 12:21:42 +03:00
Denis Kirillov
2764fabf04 [#125] Use the same HEAD, GET headers formation 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-02-15 12:21:42 +03:00
Angira Kekteeva
9d085740e0 [#123] Add url-encoded queries for prefix values 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-01-27 16:51:11 +03:00
Angira Kekteeva
c11b2332f9 [#123] Add url-encoded queries for attributes 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-01-27 16:51:11 +03:00
Angira Kekteeva
1bb6c5f384 [#111] Fix X- headers 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2021-12-09 16:12:13 +03:00
Angira Kekteeva
d6dd244756 [#111] Fix expiration epoch headers 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2021-12-09 16:12:13 +03:00
Angira Kekteeva
dbbc9e05cf [#111] Move attributes to a separate file 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2021-12-09 16:12:13 +03:00
Alex Vanin
79765fb7cc [#106] Update neofs-sdk-go to the latest version 
Includes:
 - fix for unavailable endpoints of client pool,
 - session token cache with request retry in case
   of missing session token,
 - migrated neofs client.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2021-11-15 14:46:50 +03:00
Denis Kirillov
6265fcb26a [#105] Add newline to errors 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2021-11-12 15:32:18 +03:00
Denis Kirillov
9b92c0146c [#96] Add zip compression 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2021-11-01 11:27:22 +03:00
Denis Kirillov
0b364504a7 [#92] Support zip download 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2021-10-22 18:36:09 +03:00
Denis Kirillov
03a3f986ff [#30] Fix not found responses 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2021-07-20 13:40:39 +03:00
Denis Kirillov
87918483c6 [#76] Refactor HeadObject logic 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2021-07-15 10:57:22 +03:00
Denis Kirillov
d856fdf4c4 [#76] Added HEAD methods 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2021-07-14 18:56:27 +03:00
Roman Khimov
dee182a0c1 downloader: fix Last-Modified header 
It should never ever be sent in local time, instead of
  Last-Modified: Tue, 29 Jun 2021 12:02:57 MSK
one should receive
  Last-Modified: Tue, 29 Jun 2021 09:02:57 GMT

Signed-off-by: Roman Khimov <roman@nspcc.ru>
 2021-06-29 12:25:14 +03:00
Denis Kirillov
5eed0fd22d [#67] Added streaming to download 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2021-06-24 13:06:13 +03:00
Denis Kirillov
9a49e2c85e [#65] Supported object Content-Type 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2021-06-21 16:56:51 +03:00
Denis Kirillov
fffb0b2f12 [#63] Using client.Object from sdk 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2021-06-15 11:28:30 +03:00
Roman Khimov
e177e1c9ae *: use client.Object where appropriate 
It's enough to do the job, we don't really need full client.Client interface
here.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
 2021-06-11 11:54:30 +03:00
Angira Kekteeva
2f8ab7cb8d [#57] downloader, uploader: Replace deprecated 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2021-06-04 15:55:56 +03:00
Roman Khimov
e8b94553c3 *: update to use unneofsed sdk-go 
Signed-off-by: Roman Khimov <roman@nspcc.ru>
 2021-05-31 23:07:01 +03:00
Angira Kekteeva
82b2126bfd [#46] *: Remove moved to sdk packages, refactoring 
Removed connections, logger, neofs because they were moved to sdk repo.
Made changes in downloader, uploader, main.go and app.go via
refactoring of neofs.
Replaced dependencies to removed packages by sdk packages.

Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2021-05-28 14:45:46 +03:00
Angira Kekteeva
33b1a28bf1 Replace http-gate by http-gw in files 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2021-05-18 15:14:36 +03:00
Roman Khimov
df3c87af79 *: fix all comment-related golint warnings 
Some of this code is going to be moved to SDK library, so it's important.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
 2021-05-13 18:57:42 +03:00
Roman Khimov
293debe6db downloader: ensure downloader/uploader header symmetry 
Use X-Attribute-* everywhere.
 2021-04-30 14:31:14 +03:00
Roman Khimov
7a46917781 downloader: limit headers sent to the client to some valid subset 
fasthttp doesn't do complete filtering/escaping for us, thus filter here.
 2021-04-30 14:31:14 +03:00
Roman Khimov
91c894cb8c downloader: set object/container/owner x-attributes after other ones 
Avoid name clashes with regular attributes.
 2021-04-30 14:31:14 +03:00
Roman Khimov
afbb9d51f1 *: drop github.com/pkg/errors dependency 
Use standard error wrapping/unwrapping instead. The conversion is mostly
straightforward, but see grpc/grpc-go#2934 for GRPC `status.FromError`, it
doesn't currently support unwrapping/errors.As(), so we're unwrapping manually
here.
 2021-04-30 14:31:14 +03:00