forked from TrueCloudLab/frostfs-http-gw
[#153] English Check
Signed-off-by: Elizaveta Chichindaeva <elizaveta@nspcc.ru>
This commit is contained in:
Elizaveta Chichindaeva
Alex Vanin
parent
1e3df95eed
commit
11283c1c79
6 changed files with 57 additions and 57 deletions
|
|
@ -9,14 +9,14 @@ everyone. Please follow the guidelines:
|
|||
|
||||
- Open an issue first, to discuss a new feature or enhancement.
|
||||
|
||||
- Write tests, and make sure the test suite passes locally and on CI.
|
||||
- Write tests and make sure the test suite passes locally and on CI.
|
||||
|
||||
- Open a pull request, and reference the relevant issue(s).
|
||||
- Open a pull request and reference the relevant issue(s).
|
||||
|
||||
- Make sure your commits are logically separated and have good comments
|
||||
explaining the details of your change.
|
||||
|
||||
- After receiving feedback, amend your commits or add new ones as
|
||||
- After receiving a feedback, amend your commits or add new ones as
|
||||
appropriate.
|
||||
|
||||
- **Have fun!**
|
||||
|
|
@ -48,7 +48,7 @@ $ git merge upstream/master
|
|||
|
||||
### Create your feature branch
|
||||
Before making code changes, make sure you create a separate branch for these
|
||||
changes. Maybe you will find it convenient to name branch in
|
||||
changes. Maybe you will find it convenient to name a branch in
|
||||
`<type>/<Issue>-<changes_topic>` format.
|
||||
|
||||
```
|
||||
|
|
@ -98,7 +98,7 @@ reviewed and approved, it will be merged.
|
|||
## DCO Sign off
|
||||
|
||||
All authors to the project retain copyright to their work. However, to ensure
|
||||
that they are only submitting work that they have rights to, we are requiring
|
||||
that they are only submitting work that they have rights to, we require
|
||||
everyone to acknowledge this by signing their work.
|
||||
|
||||
Any copyright notices in this repository should specify the authors as "the
|
||||
|
|
@ -110,7 +110,7 @@ To sign your work, just add a line like this at the end of your commit message:
|
|||
Signed-off-by: Samii Sakisaka <samii@nspcc.ru>
|
||||
```
|
||||
|
||||
This can easily be done with the `--signoff` option to `git commit`.
|
||||
This can be easily done with the `--signoff` option to `git commit`.
|
||||
|
||||
By doing this you state that you can certify the following (from [The Developer
|
||||
Certificate of Origin](https://developercertificate.org/)):
|
||||
|
|
|
|||
74
README.md
74
README.md
|
|
@ -13,7 +13,7 @@
|
|||
# NeoFS HTTP Gateway
|
||||
|
||||
NeoFS HTTP Gateway bridges NeoFS internal protocol and HTTP standard.
|
||||
- you can download one file per request from NeoFS Network
|
||||
- you can download one file per request from the NeoFS Network
|
||||
- you can upload one file per request into the NeoFS Network
|
||||
|
||||
## Installation
|
||||
|
|
@ -35,8 +35,8 @@ version Show current version
|
|||
```
|
||||
|
||||
Or you can also use a [Docker
|
||||
image](https://hub.docker.com/r/nspccdev/neofs-http-gw) provided for released
|
||||
(and occasionally unreleased) versions of gateway (`:latest` points to the
|
||||
image](https://hub.docker.com/r/nspccdev/neofs-http-gw) provided for the released
|
||||
(and occasionally unreleased) versions of the gateway (`:latest` points to the
|
||||
latest stable release).
|
||||
|
||||
## Execution
|
||||
|
|
@ -47,8 +47,8 @@ can be done either via `-p` parameter or via `HTTP_GW_PEERS_<N>_ADDRESS` and
|
|||
`HTTP_GW_PEERS_<N>_WEIGHT` environment variables (the gate supports multiple
|
||||
NeoFS nodes with weighted load balancing).
|
||||
|
||||
If you're launching HTTP gateway in bundle with [neofs-dev-env](https://github.com/nspcc-dev/neofs-dev-env),
|
||||
you can get an IP address of the node in output of `make hosts` command
|
||||
If you launch HTTP gateway in bundle with [neofs-dev-env](https://github.com/nspcc-dev/neofs-dev-env),
|
||||
you can get the IP address of the node in the output of `make hosts` command
|
||||
(with s0*.neofs.devenv name).
|
||||
|
||||
These two commands are functionally equivalent, they run the gate with one
|
||||
|
|
@ -89,9 +89,9 @@ This command will make gateway use 192.168.130.71 while it is healthy. Otherwise
|
|||
192.168.130.72 for 90% of requests and 192.168.130.73 for remaining 10%.
|
||||
|
||||
### Keys
|
||||
You can provide wallet via `--wallet` or `-w` flag also you can specify account address using `--address`
|
||||
(if no address provided default one will be used). If wallet is used you need to set `HTTP_GW_WALLET_PASSPHRASE` variable to decrypt wallet.
|
||||
If no wallet provided gateway autogenerates key pair it will use for NeoFS requests.
|
||||
You can provide a wallet via `--wallet` or `-w` flag. You can also specify the account address using `--address`
|
||||
(if no address provided default one will be used). If wallet is used, you need to set `HTTP_GW_WALLET_PASSPHRASE` variable to decrypt the wallet.
|
||||
If no wallet provided, the gateway autogenerates a key pair it will use for NeoFS requests.
|
||||
```
|
||||
$ neofs-http-gw -p $NEOFS_NODE -w $WALLET_PATH --address $ACCOUNT_ADDRESS
|
||||
```
|
||||
|
|
@ -105,10 +105,10 @@ $ neofs-http-gw -p 192.168.130.72:8080 -w wallet.json --address NfgHwwTi3wHAS8aF
|
|||
Gateway binds to `0.0.0.0:8082` by default and you can change that with
|
||||
`--listen_address` option.
|
||||
|
||||
It can also provide TLS interface for its users, just specify paths to key and
|
||||
It can also provide TLS interface for its users, just specify paths to the key and
|
||||
certificate files via `--tls_key` and `--tls_certificate` parameters. Note
|
||||
that using these options makes gateway TLS-only, if you need to serve both TLS
|
||||
and plain text HTTP you either have to run two gateway instances or use some
|
||||
that using these options makes gateway TLS-only. If you need to serve both TLS
|
||||
and plain text HTTP, you either have to run two gateway instances or use some
|
||||
external redirecting solution.
|
||||
|
||||
Example to bind to `192.168.130.130:443` and serve TLS there:
|
||||
|
|
@ -131,7 +131,7 @@ and `HTTP_GW_WEB_WRITE_TIMEOUT=0`. Otherwise, HTTP Gateway will terminate
|
|||
request with data stream after timeout.
|
||||
|
||||
`HTTP_GW_WEB_STREAM_REQUEST_BODY` environment variable can be used to disable
|
||||
request body streaming (effectively it'll make gateway accept file completely
|
||||
request body streaming (effectively it'll make the gateway accept the file completely
|
||||
first and only then try sending it to NeoFS).
|
||||
|
||||
`HTTP_GW_WEB_MAX_REQUEST_BODY_SIZE` controls maximum request body size
|
||||
|
|
@ -145,17 +145,17 @@ variable to control this behavior.
|
|||
|
||||
### Monitoring and metrics
|
||||
|
||||
Pprof and Prometheus are integrated into the gateway, but not enabled by
|
||||
Pprof and Prometheus are integrated into the gateway, but they are not enabled by
|
||||
default. To enable them use `--pprof` and `--metrics` flags or
|
||||
`HTTP_GW_PPROF`/`HTTP_GW_METRICS` environment variables.
|
||||
|
||||
### Timeouts
|
||||
|
||||
You can tune gRPC interface parameters with `--connect_timeout` (for
|
||||
connection to node) and `--request_timeout` (for request processing over
|
||||
connection to a node) and `--request_timeout` (for request processing over
|
||||
established connection) options.
|
||||
|
||||
gRPC-level checks allow gateway to detect dead peers, but it declares them
|
||||
gRPC-level checks allow the gateway to detect dead peers, but it declares them
|
||||
unhealthy at pool level once per `--rebalance_timer` interval, so check for it
|
||||
if needed.
|
||||
|
||||
|
|
@ -194,16 +194,16 @@ supported.
|
|||
Before uploading or downloading a file make sure you have a prepared container.
|
||||
You can create it with instructions below.
|
||||
|
||||
Also in case of downloading you need to have a file inside a container.
|
||||
Also ,in case of downloading, you need to have a file inside a container.
|
||||
|
||||
#### Create a container
|
||||
|
||||
You can create a container via [neofs-cli](https://github.com/nspcc-dev/neofs-node/releases):
|
||||
You can create a container via [neofs-cli](https://github.com/nspcc-dev/neofs-node/releases):
|
||||
```
|
||||
$ neofs-cli -r $NEOFS_NODE -k $KEY container create --policy $POLICY --basic-acl $ACL
|
||||
```
|
||||
where `$KEY` can be a path to private key file (as raw bytes), a hex string or
|
||||
(unencrypted) WIF string,
|
||||
where `$KEY` can be a path to a private key file (as raw bytes), a hex string or
|
||||
a (unencrypted) WIF string,
|
||||
`$ACL` -- hex encoded basic ACL value or keywords 'private, 'public-read', 'public-read-write' and
|
||||
`$POLICY` -- QL-encoded or JSON-encoded placement policy or path to file with it
|
||||
|
||||
|
|
@ -212,7 +212,7 @@ For example:
|
|||
$ neofs-cli -r 192.168.130.72:8080 -k 6PYLKJhiSub5imt6WCVy6Quxtd9xu176omev1vWYovzkAQCTSQabAAQXii container create --policy "REP 3" --basic-acl public --await
|
||||
```
|
||||
|
||||
If you launched nodes via [neofs-dev-env](https://github.com/nspcc-dev/neofs-dev-env)
|
||||
If you have launched nodes via [neofs-dev-env](https://github.com/nspcc-dev/neofs-dev-env),
|
||||
you can get the key value from `wallets/wallet.json` or write the path to
|
||||
the file `wallets/wallet.key`.
|
||||
|
||||
|
|
@ -279,7 +279,7 @@ $ wget http://localhost:8082/get_by_attribute/88GdaZFTcYJn1dqiSECss8kKPmmun6d6Bf
|
|||
$ wget http://localhost:8082/get_by_attribute/88GdaZFTcYJn1dqiSECss8kKPmmun6d6BfvC4zhwfLYM/FileName/cat%25jpeg # means 'cat%jpeg'
|
||||
```
|
||||
|
||||
Some other user-defined attribute:
|
||||
Some other user-defined attributes:
|
||||
|
||||
```
|
||||
$ wget http://localhost:8082/get_by_attribute/Dxhf4PNprrJHWWTG5RGLdfLkJiSQ3AQqit1MSnEPRkDZ/Ololo/100500
|
||||
|
|
@ -332,10 +332,10 @@ set of reply headers generated using the following rules:
|
|||
|
||||
### Uploading
|
||||
|
||||
You can POST files to `/upload/$CID` path where `$CID` is container ID. The
|
||||
You can POST files to `/upload/$CID` path where `$CID` is a container ID. The
|
||||
request must contain multipart form with mandatory `filename` parameter. Only
|
||||
one part in multipart form will be processed, so to upload another file just
|
||||
issue new POST request.
|
||||
issue a new POST request.
|
||||
|
||||
Example request:
|
||||
|
||||
|
|
@ -345,7 +345,7 @@ $ curl -F 'file=@cat.jpeg;filename=cat.jpeg' http://localhost:8082/upload/Dxhf4P
|
|||
|
||||
Chunked encoding is supported by the server (but check for request read
|
||||
timeouts if you're planning some streaming). You can try streaming support
|
||||
with large file piped through named FIFO pipe:
|
||||
with a large file piped through named FIFO pipe:
|
||||
|
||||
```
|
||||
$ mkfifo pipe
|
||||
|
|
@ -383,7 +383,7 @@ which transforms to `X-Attribute-Neofs-Expiration-Epoch`. So you can provide exp
|
|||
|
||||
---
|
||||
|
||||
For successful uploads you get JSON data in reply body with container and
|
||||
For successful uploads you get JSON data in reply body with a container and
|
||||
object ID, like this:
|
||||
```
|
||||
{
|
||||
|
|
@ -396,7 +396,7 @@ object ID, like this:
|
|||
|
||||
You can always upload files to public containers (open for anyone to put
|
||||
objects into), but for restricted containers you need to explicitly allow PUT
|
||||
operations for request signed with your HTTP Gateway keys.
|
||||
operations for a request signed with your HTTP Gateway keys.
|
||||
|
||||
If your don't want to manage gateway's secret keys and adjust eACL rules when
|
||||
gateway configuration changes (new gate, key rotation, etc) or you plan to use
|
||||
|
|
@ -410,23 +410,23 @@ documentation for more details). There are two options to pass them to gateway:
|
|||
credentials field
|
||||
* "Bearer" cookie with base64-encoded token contents
|
||||
|
||||
For example you have a mobile application frontend with a backend part storing
|
||||
data in NeoFS. When user authorizes in mobile app, the backend issues a NeoFS
|
||||
Bearer token and provides it to the frontend. Then the mobile app may generate
|
||||
For example, you have a mobile application frontend with a backend part storing
|
||||
data in NeoFS. When a user authorizes in the mobile app, the backend issues a NeoFS
|
||||
Bearer token and provides it to the frontend. Then, the mobile app may generate
|
||||
some data and upload it via any available NeoFS HTTP Gateway by adding
|
||||
the corresponding header to the upload request. Accessing the ACL protected data
|
||||
works the same way.
|
||||
|
||||
##### Example
|
||||
In order to generate bearer token, you need to know container owner key and
|
||||
address of sender who will be do request to NeoFS (in our case it's gateway wallet address).
|
||||
In order to generate a bearer token, you need to know the container owner key and
|
||||
the address of the sender who will do the request to NeoFS (in our case, it's a gateway wallet address).
|
||||
|
||||
Suppose we have:
|
||||
* **KxDgvEKzgSBPPfuVfw67oPQBSjidEiqTHURKSDL1R7yGaGYAeYnr** (container owner key)
|
||||
* **NhVtreTTCoqsMQV5Wp55fqnriiUCpEaKm3** (token owner address)
|
||||
* **BJeErH9MWmf52VsR1mLWKkgF3pRm3FkubYxM7TZkBP4K** (container id)
|
||||
|
||||
Firstly we need to encode container id and sender address to base64 (now it's base58).
|
||||
Firstly, we need to encode the container id and the sender address to base64 (now it's base58).
|
||||
So use **base58** and **base64** utils.
|
||||
|
||||
1. Encoding container id:
|
||||
|
|
@ -441,7 +441,7 @@ $ echo 'NhVtreTTCoqsMQV5Wp55fqnriiUCpEaKm3' | base58 --decode | base64
|
|||
# output: NezFK4ujidF+X7bB88uzREQzRQeAvdj3Gg==
|
||||
```
|
||||
|
||||
Now we can form Bearer token (10000 is liftetime expiration in epoch) and save it to **bearer.json**:
|
||||
Now, we can form a Bearer token (10000 is liftetime expiration in epoch) and save it to **bearer.json**:
|
||||
```
|
||||
{
|
||||
"body": {
|
||||
|
|
@ -468,17 +468,17 @@ Now we can form Bearer token (10000 is liftetime expiration in epoch) and save i
|
|||
}
|
||||
```
|
||||
|
||||
Then sign it with container owner key:
|
||||
Next, sign it with the container owner key:
|
||||
```
|
||||
$ neofs-cli util sign bearer-token --from bearer.json --to signed.json -k KxDgvEKzgSBPPfuVfw67oPQBSjidEiqTHURKSDL1R7yGaGYAeYnr
|
||||
```
|
||||
Encoding to base64 to use via header:
|
||||
Encoding to base64 to use via the header:
|
||||
```
|
||||
$ base64 -w 0 signed.json
|
||||
# output: Ck4KKgoECAIQBhIiCiCZGdlbN7DPGPMg9rsWqV+p2XdMzUqknRiexewSFp8kmBIbChk17MUri6OJ0X5ftsHzy7NERDNFB4C92PcaGgMIkE4SZgohAxpsb7vfAso1F0X6hrm6WpRS14WsT3/Ct1SMoqRsT89KEkEEGxKi8GjKSf52YqhppgaOTQHbUsL3jn7SHLqS3ndAQ7NtAATnmRHleZw2V2xRRSRBQdjDC05KK83LhdSax72Fsw==
|
||||
```
|
||||
|
||||
After that Bearer token can be used:
|
||||
After that, the Bearer token can be used:
|
||||
|
||||
```
|
||||
$ curl -F 'file=@cat.jpeg;filename=cat.jpeg' -H "Authorization: Bearer Ck4KKgoECAIQBhIiCiCZGdlbN7DPGPMg9rsWqV+p2XdMzUqknRiexewSFp8kmBIbChk17MUri6OJ0X5ftsHzy7NERDNFB4C92PcaGgMIkE4SZgohAxpsb7vfAso1F0X6hrm6WpRS14WsT3/Ct1SMoqRsT89KEkEEGxKi8GjKSf52YqhppgaOTQHbUsL3jn7SHLqS3ndAQ7NtAATnmRHleZw2V2xRRSRBQdjDC05KK83LhdSax72Fsw==" \
|
||||
|
|
@ -500,7 +500,7 @@ For example:
|
|||
$ neofs-cli --key KxDgvEKzgSBPPfuVfw67oPQBSjidEiqTHURKSDL1R7yGaGYAeYnr --basic-acl 0x0FFFCFFF -r 192.168.130.72:8080 container create --policy "REP 3" --await
|
||||
```
|
||||
|
||||
To deny access to the container without a token, set the eACL rules:
|
||||
To deny access to a container without a token, set the eACL rules:
|
||||
```
|
||||
$ neofs-cli --key KxDgvEKzgSBPPfuVfw67oPQBSjidEiqTHURKSDL1R7yGaGYAeYnr -r 192.168.130.72:8080 container set-eacl --table eacl.json --await --cid BJeErH9MWmf52VsR1mLWKkgF3pRm3FkubYxM7TZkBP4K
|
||||
```
|
||||
|
|
|
|||
|
|
@ -66,8 +66,8 @@ type readCloser struct {
|
|||
io.Closer
|
||||
}
|
||||
|
||||
// initializes io.Reader with limited size and detects Content-Type from it.
|
||||
// Returns r's error directly. Also returns processed data.
|
||||
// initializes io.Reader with the limited size and detects Content-Type from it.
|
||||
// Returns r's error directly. Also returns the processed data.
|
||||
func readContentType(maxSize uint64, rInit func(uint64) (io.Reader, error)) (string, []byte, error) {
|
||||
if maxSize > sizeToDetectType {
|
||||
maxSize = sizeToDetectType
|
||||
|
|
@ -167,7 +167,7 @@ func (r request) receiveFile(clnt *pool.Pool, objectAddress *address.Address) {
|
|||
return
|
||||
}
|
||||
|
||||
// reset payload reader since part of the data has been read
|
||||
// reset payload reader since a part of the data has been read
|
||||
var headReader io.Reader = bytes.NewReader(payloadHead)
|
||||
|
||||
if err != io.EOF { // otherwise, we've already read full payload
|
||||
|
|
@ -276,7 +276,7 @@ func (d *Downloader) DownloadByAddress(c *fasthttp.RequestCtx) {
|
|||
d.byAddress(c, request.receiveFile)
|
||||
}
|
||||
|
||||
// byAddress is wrapper for function (e.g. request.headObject, request.receiveFile) that
|
||||
// byAddress is a wrapper for function (e.g. request.headObject, request.receiveFile) that
|
||||
// prepares request and object address to it.
|
||||
func (d *Downloader) byAddress(c *fasthttp.RequestCtx, f func(request, *pool.Pool, *address.Address)) {
|
||||
var (
|
||||
|
|
@ -300,7 +300,7 @@ func (d *Downloader) DownloadByAttribute(c *fasthttp.RequestCtx) {
|
|||
d.byAttribute(c, request.receiveFile)
|
||||
}
|
||||
|
||||
// byAttribute is wrapper similar to byAddress.
|
||||
// byAttribute is a wrapper similar to byAddress.
|
||||
func (d *Downloader) byAttribute(c *fasthttp.RequestCtx, f func(request, *pool.Pool, *address.Address)) {
|
||||
var (
|
||||
httpStatus = fasthttp.StatusBadRequest
|
||||
|
|
|
|||
2
misc.go
2
misc.go
|
|
@ -5,6 +5,6 @@ package main
|
|||
const Prefix = "HTTP_GW"
|
||||
|
||||
var (
|
||||
// Version is gateway version.
|
||||
// Version is the gateway version.
|
||||
Version = "dev"
|
||||
)
|
||||
|
|
|
|||
|
|
@ -26,7 +26,7 @@ const (
|
|||
// return
|
||||
// }
|
||||
|
||||
// BearerTokenFromHeader extracts bearer token from Authorization request header.
|
||||
// BearerTokenFromHeader extracts a bearer token from Authorization request header.
|
||||
func BearerTokenFromHeader(h *fasthttp.RequestHeader) []byte {
|
||||
auth := h.Peek(fasthttp.HeaderAuthorization)
|
||||
if auth == nil || !bytes.HasPrefix(auth, []byte(bearerTokenHdr)) {
|
||||
|
|
@ -38,7 +38,7 @@ func BearerTokenFromHeader(h *fasthttp.RequestHeader) []byte {
|
|||
return auth
|
||||
}
|
||||
|
||||
// BearerTokenFromCookie extracts bearer token from cookies.
|
||||
// BearerTokenFromCookie extracts a bearer token from cookies.
|
||||
func BearerTokenFromCookie(h *fasthttp.RequestHeader) []byte {
|
||||
auth := h.Cookie(bearerTokenHdr)
|
||||
if len(auth) == 0 {
|
||||
|
|
@ -48,7 +48,7 @@ func BearerTokenFromCookie(h *fasthttp.RequestHeader) []byte {
|
|||
return auth
|
||||
}
|
||||
|
||||
// StoreBearerToken extracts bearer token from header or cookie and stores
|
||||
// StoreBearerToken extracts a bearer token from the header or cookie and stores
|
||||
// it in the request context.
|
||||
func StoreBearerToken(ctx *fasthttp.RequestCtx) error {
|
||||
tkn, err := fetchBearerToken(ctx)
|
||||
|
|
@ -60,7 +60,7 @@ func StoreBearerToken(ctx *fasthttp.RequestCtx) error {
|
|||
return nil
|
||||
}
|
||||
|
||||
// LoadBearerToken returns bearer token stored in context given (if it's
|
||||
// LoadBearerToken returns a bearer token stored in the context given (if it's
|
||||
// present there).
|
||||
func LoadBearerToken(ctx context.Context) (*token.BearerToken, error) {
|
||||
if tkn, ok := ctx.Value(bearerTokenKey).(*token.BearerToken); ok && tkn != nil {
|
||||
|
|
|
|||
|
|
@ -15,7 +15,7 @@ import (
|
|||
var neofsAttributeHeaderPrefixes = [...][]byte{[]byte("Neofs-"), []byte("NEOFS-"), []byte("neofs-")}
|
||||
|
||||
func systemTranslator(key, prefix []byte) []byte {
|
||||
// replace specified prefix with `__NEOFS__`
|
||||
// replace the specified prefix with `__NEOFS__`
|
||||
key = bytes.Replace(key, prefix, []byte(utils.SystemAttributePrefix), 1)
|
||||
|
||||
// replace `-` with `_`
|
||||
|
|
@ -30,12 +30,12 @@ func filterHeaders(l *zap.Logger, header *fasthttp.RequestHeader) map[string]str
|
|||
prefix := []byte(utils.UserAttributeHeaderPrefix)
|
||||
|
||||
header.VisitAll(func(key, val []byte) {
|
||||
// checks that key and val not empty
|
||||
// checks that the key and the val not empty
|
||||
if len(key) == 0 || len(val) == 0 {
|
||||
return
|
||||
}
|
||||
|
||||
// checks that key has attribute prefix
|
||||
// checks that the key has attribute prefix
|
||||
if !bytes.HasPrefix(key, prefix) {
|
||||
return
|
||||
}
|
||||
|
|
@ -51,7 +51,7 @@ func filterHeaders(l *zap.Logger, header *fasthttp.RequestHeader) map[string]str
|
|||
}
|
||||
}
|
||||
|
||||
// checks that attribute key not empty
|
||||
// checks that the attribute key is not empty
|
||||
if len(key) == 0 {
|
||||
return
|
||||
}
|
||||
|
|
|
|||
Loading…
Reference in a new issue