TrueCloudLab/frostfs-http-gw
16
1
Fork 17
Code Issues 9 Pull requests 2 Projects Releases 8 Activity Actions
451 commits 6 branches 23 tags 3.6 MiB
Commit graph

59 commits

Author SHA1 Message Date
Marina Biryukova
d219943542 [#73] Uploader, downloader structures refactoring
All checks were successful
/ Vulncheck (pull_request) Successful in 1m32s
Details
/ Lint (pull_request) Successful in 2m28s
Details
/ Tests (1.20) (pull_request) Successful in 1m52s
Details
/ Tests (1.21) (pull_request) Successful in 1m21s
Details
/ DCO (pull_request) Successful in 3m25s
Details
/ Builds (1.20) (pull_request) Successful in 5m38s
Details
/ Builds (1.21) (pull_request) Successful in 1m23s
Details 
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
 2023-09-05 18:18:04 +03:00
Roman Loginov
40568590c7 [#72] Support soft memory limit setting 
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
 2023-09-05 13:14:30 +00:00
Roman Loginov
fa28f1ff82 [#36] Move log messages to constants
All checks were successful
/ Vulncheck (pull_request) Successful in 1m34s
Details
/ Builds (1.19) (pull_request) Successful in 2m11s
Details
/ Builds (1.20) (pull_request) Successful in 1m32s
Details
/ DCO (pull_request) Successful in 4m14s
Details
/ Lint (pull_request) Successful in 2m14s
Details
/ Tests (1.19) (pull_request) Successful in 6m30s
Details
/ Tests (1.20) (pull_request) Successful in 1m42s
Details 
Signed-off-by: Roman Loginov <r.loginov@yadro.com>
 2023-08-27 18:09:02 +03:00
Denis Kirillov
d9122e2093 [#62] Update sdk to support grpc schemes in tree pool
All checks were successful
/ Builds (1.19) (pull_request) Successful in 3m15s
Details
/ Builds (1.20) (pull_request) Successful in 3m3s
Details
/ DCO (pull_request) Successful in 1m14s
Details
/ Vulncheck (pull_request) Successful in 1m35s
Details
/ Lint (pull_request) Successful in 2m57s
Details
/ Tests (1.19) (pull_request) Successful in 2m49s
Details
/ Tests (1.20) (pull_request) Successful in 12m44s
Details 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-07-07 16:43:13 +03:00
Alex Vanin
6f64557a4b [#60] Use gRPC interceptor from observability package
All checks were successful
/ Vulncheck (pull_request) Successful in 1m25s
Details
/ Lint (pull_request) Successful in 3m11s
Details
/ Tests (1.19) (pull_request) Successful in 3m0s
Details
/ Tests (1.20) (pull_request) Successful in 2m46s
Details
/ DCO (pull_request) Successful in 1m19s
Details
/ Builds (1.19) (pull_request) Successful in 2m48s
Details
/ Builds (1.20) (pull_request) Successful in 4m33s
Details 
Previous SDK implementation had implicit gRPC interceptor
for tracing. Now pool constructors allow any dial options,
so gateway should manually pass tracing interceptors from
observability package.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-06-14 12:22:44 +03:00
Denis Kirillov
1dfbe36eca [#59] Use tree pool from SDK 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-06-09 09:50:04 +03:00
Alex Vanin
f17f6747c4 [#54] Fix linter warnings
All checks were successful
Builds (1.19)
Details
Builds (1.20)
Details
DCO
Details
Lint
Details
Tests (1.19)
Details
Tests (1.20)
Details 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-05-31 20:18:00 +03:00
Pavel Pogodaev
8c3c3782f5 [#30] add object name resolving 
Signed-off-by: Pavel Pogodaev <p.pogodaev@yadro.com>
 2023-05-12 12:52:57 +03:00
Dmitrii Stepanov
a945cdd42c [#20] get/head: Add tracing support 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-05-04 16:03:30 +03:00
Alex Vanin
e02ee50d7b Rename package name 
Due to source code relocation from GitHub.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-03-07 17:08:53 +03:00
Denis Kirillov
6be8d47d92 [TrueCloudLab#12] Support multiple configs 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-02-06 13:56:55 +03:00
Denis Kirillov
31d396a125 [#2] Update building 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2022-12-20 13:34:42 +03:00
Alex Vanin
2d9eee81c2 [#1] Build HTTP Gateway with FrostFS dependencies 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2022-12-15 13:26:08 +03:00
Pavel Pogodaev
f3d58e4ef0 [#228] add support of multiple sockets 
Signed-off-by: Pavel Pogodaev <p.pogodaev@yadro.com>
 2022-11-24 18:07:04 +03:00
Denis Kirillov
408d914347 [#234] Update SDK to support timeout for stream 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-11-15 18:06:53 +03:00
Denis Kirillov
939f5f0c65 [#200] Reload config level and metrics on SIGHUP 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-09-16 17:08:20 +03:00
Angira Kekteeva
d8b04223d4 [#186] Improve wallet params 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-08-03 17:57:48 +03:00
Angira Kekteeva
9d4049cfe3 [#186] Don't print env vars with empty values 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-08-03 17:57:48 +03:00
Denis Kirillov
0f7737088d [#184] Add config param for pool error threshold 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-29 11:23:44 +03:00
Denis Kirillov
af732d294c [#171] Sync metrics and pprof configuration 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-29 09:49:57 +03:00
Denis Kirillov
3c64a33b2f [#172] Change pool defaults params 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 17:16:44 +03:00
Denis Kirillov
4a2f9d6d6c [#169] Unify version output 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-18 13:35:05 +03:00
Denis Kirillov
a42606742a [#142] Support resolving container nicename 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-04-28 17:18:13 +03:00
Alex Vanin
d906732ef4 [#146] Update default values for HTTP_GW_READ/WRITE_TIMEOUT 
10 minute upload and 5 minute download timeouts are long enough
to handle average size objects in the real networks. For big
data streams these timeouts should be disabled.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2022-04-22 15:21:42 +04:00
Denis Kirillov
eacd7be33e [#140] Improve docs 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-04-15 03:26:02 +04:00
Denis Kirillov
d891c13cb3 [#137] Drop sdk logger 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-03-29 10:15:59 +03:00
Denis Kirillov
71e846a4fc [#115] Update sdk version to support priority 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2021-12-10 10:59:43 +03:00
Denis Kirillov
9b92c0146c [#96] Add zip compression 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2021-11-01 11:27:22 +03:00
Denis Kirillov
5b6cc990c0 [#71] Fix yaml config support 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2021-06-28 16:17:08 +03:00
Denis Kirillov
2d41fbc95e [#68] Support NEP-6 wallets 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2021-06-23 14:53:04 +03:00
Denis Kirillov
7734be609c [#60] Dropped grpc keepalive options 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2021-06-10 18:48:55 +03:00
Angira Kekteeva
b402f5009e *: add short name for key parameter 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2021-05-28 16:11:19 +03:00
Roman Khimov
6c73296012 misc: drop Build variable 
Makes builds reproducible.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
 2021-05-25 13:20:01 +03:00
Angira Kekteeva
33b1a28bf1 Replace http-gate by http-gw in files 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2021-05-18 15:14:36 +03:00
Roman Khimov
3173c70eb6 settings: fix godot warnings 
Signed-off-by: Roman Khimov <roman@nspcc.ru>
 2021-05-13 18:57:42 +03:00
Roman Khimov
491ae13190 app: add support for TLS key/cert options 
Run in TLS mode if anything is specified.
 2021-04-15 20:24:49 +03:00
Roman Khimov
f019c97ce8 connections: normalize weights, make config a bit more human-oriented 
Current scheme requires people to calculate exact percentages for their
nodes. People are bad with pecentage calculations and it's really inconvenient
if you have more than 3-4 nodes. What I'd like to have is ability to specify
weights like weights, not percentages:
  .._0_WEIGHT=3
  .._1_WEIGHT=1
  .._2_WEIGHT=1

and let the gateway calculate things for me.
 2021-04-15 09:45:43 +03:00
Roman Khimov
c06a3137e5 app: add support for ephemeral keys 2021-04-14 23:52:59 +03:00
Pavel Korotkov
62a03251ce Add connection pool implementation (part 1) 
Signed-off-by: Pavel Korotkov <pavel@nspcc.ru>
 2021-04-06 12:19:21 +03:00
Pavel Korotkov
4c96885a42 [#19] Add a version with no cdn-sdk deps 
Signed-off-by: Pavel Korotkov <pavel@nspcc.ru>
 2021-04-06 12:19:21 +03:00
Evgeniy Kulikov
3b8bf3017d
Migrate to fasthttp v1.20.0 
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
 2021-02-12 18:48:52 +03:00
Evgeniy Kulikov
a428a0b1b3
fix conflicts 2021-02-12 15:24:52 +03:00
Evgeniy Kulikov
cbaf9e6142
Fixes after review 
After discussion, we decided to simplify attribute translation for now. Full translation requires support for UTF-8 values encoded according to RFC 8187/7230, this can be clarified and implemented in further PRs.

1. Remove translation tables
2. Use simple translation rules only, for now ignoring UTF-8 keys and values

- `X-Attribute-NEOFS-` prefixed headers considered well-known system attributes
- System Attribute key is uppercased
- System Attribute key's `-` translates to `_`
- `X-Attribute-` prefixed headers considered regular object attributes
- Normal attribute key is a result of http header prefix trimming
- Value string should be left as is, for now

```
HTTP:
X-Attribute-NEOFS-Expiration-Epoch: 123
X-Attribute-FileName: cat.jpg

NeoFS:
__NEOFS__EXPIRATION_EPOCH: "123"
FileName: "cat.jpg"
```

Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
 2021-02-12 15:21:53 +03:00
Evgeniy Kulikov
3cbd4dbd09
Fixes after review 
- Maybe it could be rephrased to "attribute name prefixed with X-Attribute-"
- Maybe it worth renaming to userAttributeHeaderPrefix
- Requirement for having only one file uploaded at a time has to be reflected in documentation.
- Maybe this should be reflected in doc
- I'm not sure if missing timestamp attribute should be forced on gateway level. Normally it should be set by app and there should be a way not to set it.

Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
 2021-02-12 15:21:53 +03:00
Evgeniy Kulikov
462de45e12
Added POST method to upload files into NeoFS through HTTP Gate 
- Updated README
- Added method to upload files into NeoFS
- HTTP Upload Header Filter loaded from settings
- Added `HeaderFilter` that filters headers (object attributes)

Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
 2021-02-12 15:21:53 +03:00
Evgeniy Kulikov
65a5a3f9fc
Update README 
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
 2020-12-02 12:25:18 +03:00
Evgeniy Kulikov
8bd210d67e
Use constant settings keys 
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
 2020-12-02 12:25:18 +03:00
Evgeniy Kulikov
01f13ef210
Prepare to release 
- set output for flag set
- change env prefix for all environments
- migrate to CDN SDK release v0.1.0

Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
 2020-12-02 12:25:18 +03:00
Evgeniy Kulikov
31186382e8
Change ENV prefix 
TODO should be replaced with HTTP_GW before release

Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
 2020-11-27 18:18:53 +03:00
Evgeniy Kulikov
6226729e38
Migrate to CDN SDK 2020-11-10 14:46:28 +03:00