TrueCloudLab/frostfs-http-gw
15
1
Fork 17
Code Issues 8 Pull requests 2 Projects Releases 8 Activity Actions
349 commits 6 branches 23 tags 3.6 MiB
Commit graph

65 commits

Author SHA1 Message Date
Denis Kirillov
ad2c7ca671 [#200] Reload certs on SIGHUP 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-09-16 17:08:20 +03:00
Denis Kirillov
82eba97505 [#200] Unregister metrics on shutdown to fix test 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-09-16 17:08:20 +03:00
Denis Kirillov
ce84dc7068 [#200] Reload upload/download settings on SIGHUP 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-09-16 17:08:20 +03:00
Denis Kirillov
1e05d8a935 [#200] Reload resolvers on SIGHUP 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-09-16 17:08:20 +03:00
Denis Kirillov
939f5f0c65 [#200] Reload config level and metrics on SIGHUP 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-09-16 17:08:20 +03:00
Angira Kekteeva
d8b04223d4 [#186] Improve wallet params 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-08-03 17:57:48 +03:00
Denis Kirillov
ee49355bb7 [#179] Expose pool metrics 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-08-02 17:26:11 +03:00
Denis Kirillov
0f7737088d [#184] Add config param for pool error threshold 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-29 11:23:44 +03:00
Denis Kirillov
af732d294c [#171] Sync metrics and pprof configuration 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-29 09:49:57 +03:00
Denis Kirillov
0fb75a1cde [#178] Update SDK to v1.0.0-rc.5 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-25 14:27:24 +03:00
Denis Kirillov
cf018c2fab [#175] Add health metric 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 17:17:36 +03:00
Denis Kirillov
a42606742a [#142] Support resolving container nicename 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-04-28 17:18:13 +03:00
Alex Vanin
1e3df95eed [#145] Use application context in NeoFS API requests 
It is meaningless to use RequestCtx as a context.Context
for NeoFS operation, because context won't be closed
until application shutdown. Moreover, it also triggers
data race detection, because server's done channel, which
is accessible for reading from RequestCtx, is set to `nil`.

Using application context doesn't change gateway behavior,
but it suppresses data race trigger at shutdown. It also
allows possibility to set configurable timeouts for NeoFS
networking if we will ever need them.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2022-04-22 15:33:11 +04:00
Denis Kirillov
eacd7be33e [#140] Improve docs 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-04-15 03:26:02 +04:00
Denis Kirillov
baf425453d [#139] fix lint 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-04-08 00:23:12 +04:00
Denis Kirillov
5080b43a04 [#139] Update SDK 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-04-08 00:23:12 +04:00
Denis Kirillov
d891c13cb3 [#137] Drop sdk logger 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-03-29 10:15:59 +03:00
Denis Kirillov
6d362894ad [#132] Update NeoFS SDK to v1.0.0-rc.3 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-03-16 17:15:21 +03:00
Leonard Lyubich
2b7e4a36fb [#126] Upgrade NeoFS SDK Go library 
Core changes:
  - `object.ID` moved to new package `oid`;
  - `object.Address` moved to new package `address`;
  - `pool.Object` interface changes.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-03-03 18:25:53 +03:00
Denis Kirillov
71e846a4fc [#115] Update sdk version to support priority 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2021-12-10 10:59:43 +03:00
Angira Kekteeva
06be798111 [#109] Add version to logs in start 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2021-11-29 13:56:36 +03:00
Alex Vanin
79765fb7cc [#106] Update neofs-sdk-go to the latest version 
Includes:
 - fix for unavailable endpoints of client pool,
 - session token cache with request retry in case
   of missing session token,
 - migrated neofs client.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2021-11-15 14:46:50 +03:00
Denis Kirillov
6265fcb26a [#105] Add newline to errors 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2021-11-12 15:32:18 +03:00
Denis Kirillov
9b92c0146c [#96] Add zip compression 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2021-11-01 11:27:22 +03:00
Denis Kirillov
0b364504a7 [#92] Support zip download 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2021-10-22 18:36:09 +03:00
Denis Kirillov
d856fdf4c4 [#76] Added HEAD methods 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2021-07-14 18:56:27 +03:00
Denis Kirillov
2b8aa4914e [#77] Added requests logging 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2021-07-09 13:48:50 +03:00
Denis Kirillov
2d41fbc95e [#68] Support NEP-6 wallets 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2021-06-23 14:53:04 +03:00
Denis Kirillov
7734be609c [#60] Dropped grpc keepalive options 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2021-06-10 18:48:55 +03:00
Roman Khimov
e8b94553c3 *: update to use unneofsed sdk-go 
Signed-off-by: Roman Khimov <roman@nspcc.ru>
 2021-05-31 23:07:01 +03:00
Angira Kekteeva
82b2126bfd [#46] *: Remove moved to sdk packages, refactoring 
Removed connections, logger, neofs because they were moved to sdk repo.
Made changes in downloader, uploader, main.go and app.go via
refactoring of neofs.
Replaced dependencies to removed packages by sdk packages.

Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2021-05-28 14:45:46 +03:00
Angira Kekteeva
33b1a28bf1 Replace http-gate by http-gw in files 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2021-05-18 15:14:36 +03:00
Roman Khimov
df3c87af79 *: fix all comment-related golint warnings 
Some of this code is going to be moved to SDK library, so it's important.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
 2021-05-13 18:57:42 +03:00
Roman Khimov
f99f9e88a7 connections: make use of keepalive options 2021-04-30 14:31:14 +03:00
Roman Khimov
b94bfb3b76 app: drop FIXME comment 
We've made long way since a428a0b1b3 and it's
just no longer relevant, the fix is there in fasthttp version we use and
streaming works fine.
 2021-04-30 14:31:14 +03:00
Roman Khimov
ed27e28a30 app: drop unused Worker/jobDone 2021-04-30 14:31:14 +03:00
Roman Khimov
491ae13190 app: add support for TLS key/cert options 
Run in TLS mode if anything is specified.
 2021-04-15 20:24:49 +03:00
Roman Khimov
a6f63c2bac app: use 1 for weight if not specified or wrong 
If we have a number of nodes with unspecified weights they'd all be treated
equal which seems to be fair.
 2021-04-15 09:45:43 +03:00
Roman Khimov
c06a3137e5 app: add support for ephemeral keys 2021-04-14 23:52:59 +03:00
Pavel Korotkov
fad05b76d4 Put artifacts into pool solely 
Signed-off-by: Pavel Korotkov <pavel@nspcc.ru>
 2021-04-08 12:39:29 +03:00
Pavel Korotkov
62a03251ce Add connection pool implementation (part 1) 
Signed-off-by: Pavel Korotkov <pavel@nspcc.ru>
 2021-04-06 12:19:21 +03:00
Pavel Korotkov
aedd468c18 [#19] Move connection list sorting to a proper place 
Signed-off-by: Pavel Korotkov <pavel@nspcc.ru>
 2021-04-06 12:19:21 +03:00
Pavel Korotkov
0b6e138270 [#19] Prepare using connection pool 
Signed-off-by: Pavel Korotkov <pavel@nspcc.ru>
 2021-04-06 12:19:21 +03:00
Pavel Korotkov
3a5d9fe94c [#19] Extract downloading logic into a separate package 
Signed-off-by: Pavel Korotkov <pavel@nspcc.ru>
 2021-04-06 12:19:21 +03:00
Pavel Korotkov
eb92219e14 [#19] Extract uploading logic into a separate package 
Signed-off-by: Pavel Korotkov <pavel@nspcc.ru>
 2021-04-06 12:19:21 +03:00
Pavel Korotkov
4c96885a42 [#19] Add a version with no cdn-sdk deps 
Signed-off-by: Pavel Korotkov <pavel@nspcc.ru>
 2021-04-06 12:19:21 +03:00
Evgeniy Kulikov
3f635a018a
Refactoring uploading 
- DisablePreParseMultipartForm = true
- used `fetchMultipartFile` method instead of `MultipartForm`
- don't store temporary files, only streaming

Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
 2021-02-13 19:25:12 +03:00
Evgeniy Kulikov
3b8bf3017d
Migrate to fasthttp v1.20.0 
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
 2021-02-12 18:48:52 +03:00
Evgeniy Kulikov
a428a0b1b3
fix conflicts 2021-02-12 15:24:52 +03:00
Evgeniy Kulikov
cbaf9e6142
Fixes after review 
After discussion, we decided to simplify attribute translation for now. Full translation requires support for UTF-8 values encoded according to RFC 8187/7230, this can be clarified and implemented in further PRs.

1. Remove translation tables
2. Use simple translation rules only, for now ignoring UTF-8 keys and values

- `X-Attribute-NEOFS-` prefixed headers considered well-known system attributes
- System Attribute key is uppercased
- System Attribute key's `-` translates to `_`
- `X-Attribute-` prefixed headers considered regular object attributes
- Normal attribute key is a result of http header prefix trimming
- Value string should be left as is, for now

```
HTTP:
X-Attribute-NEOFS-Expiration-Epoch: 123
X-Attribute-FileName: cat.jpg

NeoFS:
__NEOFS__EXPIRATION_EPOCH: "123"
FileName: "cat.jpg"
```

Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
 2021-02-12 15:21:53 +03:00