realloc/frostfs-http-gw
1
0
Fork 0
forked from TrueCloudLab/frostfs-http-gw
Code Issues Pull requests Projects Releases Packages Wiki Activity
409 commits 2 branches 0 tags 2 MiB
Commit graph

50 commits

Author SHA1 Message Date
Alex Vanin
e02ee50d7b Rename package name 
Due to source code relocation from GitHub.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-03-07 17:08:53 +03:00
Denis Kirillov
6be8d47d92 [TrueCloudLab#12] Support multiple configs 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-02-06 13:56:55 +03:00
Denis Kirillov
31d396a125 [#2] Update building 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2022-12-20 13:34:42 +03:00
Alex Vanin
2d9eee81c2 [#1] Build HTTP Gateway with FrostFS dependencies 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2022-12-15 13:26:08 +03:00
Pavel Pogodaev
f3d58e4ef0 [#228] add support of multiple sockets 
Signed-off-by: Pavel Pogodaev <p.pogodaev@yadro.com>
 2022-11-24 18:07:04 +03:00
Denis Kirillov
408d914347 [#234] Update SDK to support timeout for stream 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-11-15 18:06:53 +03:00
Denis Kirillov
939f5f0c65 [#200] Reload config level and metrics on SIGHUP 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-09-16 17:08:20 +03:00
Angira Kekteeva
d8b04223d4 [#186] Improve wallet params 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-08-03 17:57:48 +03:00
Angira Kekteeva
9d4049cfe3 [#186] Don't print env vars with empty values 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2022-08-03 17:57:48 +03:00
Denis Kirillov
0f7737088d [#184] Add config param for pool error threshold 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-29 11:23:44 +03:00
Denis Kirillov
af732d294c [#171] Sync metrics and pprof configuration 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-29 09:49:57 +03:00
Denis Kirillov
3c64a33b2f [#172] Change pool defaults params 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-22 17:16:44 +03:00
Denis Kirillov
4a2f9d6d6c [#169] Unify version output 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-07-18 13:35:05 +03:00
Denis Kirillov
a42606742a [#142] Support resolving container nicename 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-04-28 17:18:13 +03:00
Alex Vanin
d906732ef4 [#146] Update default values for HTTP_GW_READ/WRITE_TIMEOUT 
10 minute upload and 5 minute download timeouts are long enough
to handle average size objects in the real networks. For big
data streams these timeouts should be disabled.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2022-04-22 15:21:42 +04:00
Denis Kirillov
eacd7be33e [#140] Improve docs 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-04-15 03:26:02 +04:00
Denis Kirillov
d891c13cb3 [#137] Drop sdk logger 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-03-29 10:15:59 +03:00
Denis Kirillov
71e846a4fc [#115] Update sdk version to support priority 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2021-12-10 10:59:43 +03:00
Denis Kirillov
9b92c0146c [#96] Add zip compression 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2021-11-01 11:27:22 +03:00
Denis Kirillov
5b6cc990c0 [#71] Fix yaml config support 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2021-06-28 16:17:08 +03:00
Denis Kirillov
2d41fbc95e [#68] Support NEP-6 wallets 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2021-06-23 14:53:04 +03:00
Denis Kirillov
7734be609c [#60] Dropped grpc keepalive options 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2021-06-10 18:48:55 +03:00
Angira Kekteeva
b402f5009e *: add short name for key parameter 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2021-05-28 16:11:19 +03:00
Roman Khimov
6c73296012 misc: drop Build variable 
Makes builds reproducible.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
 2021-05-25 13:20:01 +03:00
Angira Kekteeva
33b1a28bf1 Replace http-gate by http-gw in files 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2021-05-18 15:14:36 +03:00
Roman Khimov
3173c70eb6 settings: fix godot warnings 
Signed-off-by: Roman Khimov <roman@nspcc.ru>
 2021-05-13 18:57:42 +03:00
Roman Khimov
491ae13190 app: add support for TLS key/cert options 
Run in TLS mode if anything is specified.
 2021-04-15 20:24:49 +03:00
Roman Khimov
f019c97ce8 connections: normalize weights, make config a bit more human-oriented 
Current scheme requires people to calculate exact percentages for their
nodes. People are bad with pecentage calculations and it's really inconvenient
if you have more than 3-4 nodes. What I'd like to have is ability to specify
weights like weights, not percentages:
  .._0_WEIGHT=3
  .._1_WEIGHT=1
  .._2_WEIGHT=1

and let the gateway calculate things for me.
 2021-04-15 09:45:43 +03:00
Roman Khimov
c06a3137e5 app: add support for ephemeral keys 2021-04-14 23:52:59 +03:00
Pavel Korotkov
62a03251ce Add connection pool implementation (part 1) 
Signed-off-by: Pavel Korotkov <pavel@nspcc.ru>
 2021-04-06 12:19:21 +03:00
Pavel Korotkov
4c96885a42 [#19] Add a version with no cdn-sdk deps 
Signed-off-by: Pavel Korotkov <pavel@nspcc.ru>
 2021-04-06 12:19:21 +03:00
Evgeniy Kulikov
3b8bf3017d
Migrate to fasthttp v1.20.0 
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
 2021-02-12 18:48:52 +03:00
Evgeniy Kulikov
a428a0b1b3
fix conflicts 2021-02-12 15:24:52 +03:00
Evgeniy Kulikov
cbaf9e6142
Fixes after review 
After discussion, we decided to simplify attribute translation for now. Full translation requires support for UTF-8 values encoded according to RFC 8187/7230, this can be clarified and implemented in further PRs.

1. Remove translation tables
2. Use simple translation rules only, for now ignoring UTF-8 keys and values

- `X-Attribute-NEOFS-` prefixed headers considered well-known system attributes
- System Attribute key is uppercased
- System Attribute key's `-` translates to `_`
- `X-Attribute-` prefixed headers considered regular object attributes
- Normal attribute key is a result of http header prefix trimming
- Value string should be left as is, for now

```
HTTP:
X-Attribute-NEOFS-Expiration-Epoch: 123
X-Attribute-FileName: cat.jpg

NeoFS:
__NEOFS__EXPIRATION_EPOCH: "123"
FileName: "cat.jpg"
```

Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
 2021-02-12 15:21:53 +03:00
Evgeniy Kulikov
3cbd4dbd09
Fixes after review 
- Maybe it could be rephrased to "attribute name prefixed with X-Attribute-"
- Maybe it worth renaming to userAttributeHeaderPrefix
- Requirement for having only one file uploaded at a time has to be reflected in documentation.
- Maybe this should be reflected in doc
- I'm not sure if missing timestamp attribute should be forced on gateway level. Normally it should be set by app and there should be a way not to set it.

Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
 2021-02-12 15:21:53 +03:00
Evgeniy Kulikov
462de45e12
Added POST method to upload files into NeoFS through HTTP Gate 
- Updated README
- Added method to upload files into NeoFS
- HTTP Upload Header Filter loaded from settings
- Added `HeaderFilter` that filters headers (object attributes)

Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
 2021-02-12 15:21:53 +03:00
Evgeniy Kulikov
65a5a3f9fc
Update README 
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
 2020-12-02 12:25:18 +03:00
Evgeniy Kulikov
8bd210d67e
Use constant settings keys 
Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
 2020-12-02 12:25:18 +03:00
Evgeniy Kulikov
01f13ef210
Prepare to release 
- set output for flag set
- change env prefix for all environments
- migrate to CDN SDK release v0.1.0

Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
 2020-12-02 12:25:18 +03:00
Evgeniy Kulikov
31186382e8
Change ENV prefix 
TODO should be replaced with HTTP_GW before release

Signed-off-by: Evgeniy Kulikov <kim@nspcc.ru>
 2020-11-27 18:18:53 +03:00
Evgeniy Kulikov
6226729e38
Migrate to CDN SDK 2020-11-10 14:46:28 +03:00
Evgeniy Kulikov
ea4710eaba
update to new router 2020-04-15 11:10:42 +03:00
Evgeniy Kulikov
164b0870de
refactoring + moved to neofs-api-go 2020-03-31 11:37:10 +03:00
Evgeniy Kulikov
6e917c1982
move rebalance timer to settings 2020-02-27 13:09:27 +03:00
Evgeniy Kulikov
65a2da6995
Add constants with default values 
- default values constants
- add conn_ttl `GW_CONN_TTL=duration` (gRPC connection time to live)
 2020-02-19 13:47:21 +03:00
Evgeniy Kulikov
fbfab8cae7
Add state.HealthCheck to isActive 2020-02-13 17:33:12 +03:00
Evgeniy Kulikov
590444459d
Migrate to NeoFS API 2020-02-04 13:35:25 +03:00
Evgeniy Kulikov
746946290c
NSPCC-762 Connection pool 
- implement connection pool
- wait until container creates
- refactoring gw service
- add config option to enable prometheus and pprof
- update neofs-proto to v0.2.8
 2019-12-21 13:26:14 +03:00
Evgeniy Kulikov
39279e4c5f
NSPCC-786 Rename gen => generated 2019-12-17 18:34:04 +03:00
Evgeniy Kulikov
97d1a99f58
NSPCC-493 Separate repository for NeoFS GW 
- Update dependencies
- Update README (configuration: flags section)
- Add gRPC logger and flag to enable gRPC connection debugging
- Refactored settings (add flags, use defaults instead of yaml representation)
 2019-12-13 19:09:14 +03:00