nzinkevich/frostfs-http-gw
1
0
Fork 0
forked from TrueCloudLab/frostfs-http-gw
Code Pull requests Activity
323 commits 10 branches 15 tags 3.7 MiB
Commit graph

323 commits

This branch
This branch
All branches
Author SHA1 Message Date
Roman Khimov
37fa2c6757
Merge pull request #34 from roman-khimov/fix-tests 
Fix tests, add workflow and makefile targets
 2021-04-30 19:19:07 +03:00
Roman Khimov
d5cdcb29c3 workflows: use Ubuntu 20.04 instead of 18.04 
18.04 is a bit old.
 2021-04-30 19:14:12 +03:00
Roman Khimov
25d273f88e workflows: move linter to tests from builds 2021-04-30 19:09:41 +03:00
Roman Khimov
f05a6eda7d workflows: add testing workflow 2021-04-30 19:08:57 +03:00
Roman Khimov
7ba5aed9ca tokens: fix tests 
Broken by afbb9d51f1 which changed some
strings.
 2021-04-30 19:02:30 +03:00
Roman Khimov
727ec9b22f
Merge pull request #33 from roman-khimov/add-image-push 
Makefile: add simple image-push target
 2021-04-30 19:00:54 +03:00
Roman Khimov
93fcaffaaa Makefile: add simple image-push target 
And use it in workflow.
 2021-04-30 19:00:05 +03:00
Roman Khimov
ebd3ad8842 Makefile: add test and cover targets 2021-04-30 18:59:21 +03:00
Roman Khimov
12859c2e5d metrics: fix linter warning 
It's useless, but still we want to be green
  Error: missing cases in switch of type promhttp.HandlerErrorHandling: ContinueOnError (exhaustive)
 2021-04-30 17:52:29 +03:00
Roman Khimov
3ee5ad52b6 workflows: use latest Ubuntu for linter 
Try to fix
  Error: NotifyContext not declared by package signal (typecheck)
 2021-04-30 17:52:29 +03:00
Roman Khimov
59f5f2da10 workflow: fix missing space 
The workflow is not valid. .github/workflows/publish_to_dockerhub.yml (Line: 68, Col: 11): Unexpected value 'HUB_TAG:latest'
 2021-04-30 17:52:29 +03:00
Roman Khimov
6301a2c6b8 workflow: add workflow to push images to Docker Hub 2021-04-30 17:28:52 +03:00
Roman Khimov
df43caa951 workflows: add build/lint workflows 2021-04-30 17:28:52 +03:00
Roman Khimov
0995dda595 .gitignore: extend with *~ 2021-04-30 17:28:52 +03:00
Roman Khimov
e47cb5f9c6 workflows: add CodeQL workflow 2021-04-30 17:28:52 +03:00
Roman Khimov
90b0613182 README: add something about bearer tokens 2021-04-30 14:31:14 +03:00
Roman Khimov
293debe6db downloader: ensure downloader/uploader header symmetry 
Use X-Attribute-* everywhere.
 2021-04-30 14:31:14 +03:00
Roman Khimov
e80d147d72 README: rewrite all documentation 
I think we can fit into one page with this, thus how-to-check.md was removed
(and it's a bit bloated to me anyway).
 2021-04-30 14:31:14 +03:00
Roman Khimov
7a46917781 downloader: limit headers sent to the client to some valid subset 
fasthttp doesn't do complete filtering/escaping for us, thus filter here.
 2021-04-30 14:31:14 +03:00
Roman Khimov
91c894cb8c downloader: set object/container/owner x-attributes after other ones 
Avoid name clashes with regular attributes.
 2021-04-30 14:31:14 +03:00
Roman Khimov
f99f9e88a7 connections: make use of keepalive options 2021-04-30 14:31:14 +03:00
Roman Khimov
afbb9d51f1 *: drop github.com/pkg/errors dependency 
Use standard error wrapping/unwrapping instead. The conversion is mostly
straightforward, but see grpc/grpc-go#2934 for GRPC `status.FromError`, it
doesn't currently support unwrapping/errors.As(), so we're unwrapping manually
here.
 2021-04-30 14:31:14 +03:00
Roman Khimov
ba293a3ff9 connections: refuse to build zero-length pool 
Prevent starting a gateway with no nodes configured.
 2021-04-30 14:31:14 +03:00
Roman Khimov
61d61a76cc logger: fix infinite recursive call 
logger/grpc.go:58:66        staticcheck  SA5007: infinite recursive call
 2021-04-30 14:31:14 +03:00
Roman Khimov
d9f9a03625 neofs: fix linter warnings 
neofs/client-plant.go:78:1  golint       receiver name cc should be consistent with previous receiver name cp for neofsClientPlant
neofs/client-plant.go:85:1  golint       receiver name cc should be consistent with previous receiver name cp for neofsClientPlant
 2021-04-30 14:31:14 +03:00
Roman Khimov
4e4f873786 misc: gofmt -s 2021-04-30 14:31:14 +03:00
Roman Khimov
88e0963240 go.mod: temporarily use nspcc-dev fork of valyala/fasthttp 
We need proper chunk processing.
 2021-04-30 14:31:14 +03:00
Roman Khimov
ed3c576b70 uploader: drain body buffer before ending request processing 
Fixes

2021-04-28T00:03:36.514+0300    debug   uploader/upload.go:64   close temporary multipart/form file     {"cid": "Dxhf4PNprrJHWWTG5RGLdfLkJiSQ3AQqit1MSnEPRkDZ", "address": "Dxhf4PNprrJHWWTG5RGLdfLkJiSQ3AQqit1MSnEPRkDZ/2m8PtaoricLouCn5zE8hAFr3gZEBDCZFe9BEgVJTSocX", "filename": "vid.mp4"}
2021/04/28 00:03:36 error when serving connection "127.0.0.1:8082"<->"127.0.0.1:41390": error when reading request headers: cannot find http request method in "0\r\n\r\n". Buffer size=5, contents: "0\r\n\r\n"
 2021-04-30 14:31:14 +03:00
Roman Khimov
b94bfb3b76 app: drop FIXME comment 
We've made long way since a428a0b1b3 and it's
just no longer relevant, the fix is there in fasthttp version we use and
streaming works fine.
 2021-04-30 14:31:14 +03:00
Roman Khimov
464a0134bd neofs: drop PrepareObjectOnsite option 
It's using NeoFS node internal packages which shouldn't ever be used in gate
code, it's not adding any value to the code we have (standard api-go
interfaces work fine).
 2021-04-30 14:31:14 +03:00
Roman Khimov
b020cd6652 uploader: fix passing attributes to object 2021-04-30 14:31:14 +03:00
Roman Khimov
ed27e28a30 app: drop unused Worker/jobDone 2021-04-30 14:31:14 +03:00
Roman Khimov
491ae13190 app: add support for TLS key/cert options 
Run in TLS mode if anything is specified.
 2021-04-15 20:24:49 +03:00
Roman Khimov
a6f63c2bac app: use 1 for weight if not specified or wrong 
If we have a number of nodes with unspecified weights they'd all be treated
equal which seems to be fair.
 2021-04-15 09:45:43 +03:00
Roman Khimov
f019c97ce8 connections: normalize weights, make config a bit more human-oriented 
Current scheme requires people to calculate exact percentages for their
nodes. People are bad with pecentage calculations and it's really inconvenient
if you have more than 3-4 nodes. What I'd like to have is ability to specify
weights like weights, not percentages:
  .._0_WEIGHT=3
  .._1_WEIGHT=1
  .._2_WEIGHT=1

and let the gateway calculate things for me.
 2021-04-15 09:45:43 +03:00
Roman Khimov
0d21ca382f neofs: drop WIF() from Credentials 
It's never used and never needs to be used, the key shouldn't be exported from
the gate.
 2021-04-14 23:52:59 +03:00
Roman Khimov
c06a3137e5 app: add support for ephemeral keys 2021-04-14 23:52:59 +03:00
Roman Khimov
1156223231 go.sum: clean up 
Make this thing build without go.sum changes.
 2021-04-13 12:54:31 +03:00
Stanislav Bogatyrev
b2e7a2cb61 Clean up Makefile and image build 
Signed-off-by: Stanislav Bogatyrev <stanislav@nspcc.ru>
 2021-04-12 17:09:39 +03:00
Pavel Korotkov
e3dbecf844 Rework connection artifacts choice logic 
Signed-off-by: Pavel Korotkov <pavel@nspcc.ru>
 2021-04-08 12:39:29 +03:00
Pavel Korotkov
fad05b76d4 Put artifacts into pool solely 
Signed-off-by: Pavel Korotkov <pavel@nspcc.ru>
 2021-04-08 12:39:29 +03:00
Pavel Korotkov
d7617110b7 Remove test dot-env file 
Signed-off-by: Pavel Korotkov <pavel@nspcc.ru>
 2021-04-06 12:19:21 +03:00
Pavel Korotkov
c21324bf77 Make minor refactoring 
Signed-off-by: Pavel Korotkov <pavel@nspcc.ru>
 2021-04-06 12:19:21 +03:00
Pavel Korotkov
f7007f2085 Add connection pool implementation (part 3) 
Signed-off-by: Pavel Korotkov <pavel@nspcc.ru>
 2021-04-06 12:19:21 +03:00
Pavel Korotkov
a44551d42b Add connection pool implementation (part 2) 
Signed-off-by: Pavel Korotkov <pavel@nspcc.ru>
 2021-04-06 12:19:21 +03:00
Pavel Korotkov
62a03251ce Add connection pool implementation (part 1) 
Signed-off-by: Pavel Korotkov <pavel@nspcc.ru>
 2021-04-06 12:19:21 +03:00
Pavel Korotkov
c909c99f72 [#19] Rename client-oriented entities before adding fake/mock ones 
Signed-off-by: Pavel Korotkov <pavel@nspcc.ru>
 2021-04-06 12:19:21 +03:00
Pavel Korotkov
aedd468c18 [#19] Move connection list sorting to a proper place 
Signed-off-by: Pavel Korotkov <pavel@nspcc.ru>
 2021-04-06 12:19:21 +03:00
Pavel Korotkov
0b6e138270 [#19] Prepare using connection pool 
Signed-off-by: Pavel Korotkov <pavel@nspcc.ru>
 2021-04-06 12:19:21 +03:00
Pavel Korotkov
3a5d9fe94c [#19] Extract downloading logic into a separate package 
Signed-off-by: Pavel Korotkov <pavel@nspcc.ru>
 2021-04-06 12:19:21 +03:00