forked from TrueCloudLab/frostfs-s3-gw
317 lines
18 KiB
Markdown
317 lines
18 KiB
Markdown
# Authentication and authorization scheme
|
|
|
|
This document describes s3-gw authentication and authorization mechanism.
|
|
|
|
## General overview
|
|
|
|
Basic provisions:
|
|
|
|
* A request to s3-gw can be signed or not (request that isn't signed we will call anonymous or just anon)
|
|
* To manage resources (buckets/objects) using s3-gw you must have appropriate access rights
|
|
|
|
Each request must be authenticated (at least as anonymous) and authorized. The following scheme shows components that
|
|
are involved in this
|
|
process.
|
|
|
|
<a>
|
|
<img src="images/authentication/auth-overview.svg" alt="Auth general overview"/>
|
|
</a>
|
|
|
|
There are several participants in this process:
|
|
|
|
1. User that make a request
|
|
2. S3-GW that accepts a request
|
|
3. FrostFS Storage that stores AccessObjects (objects are needed for authentication)
|
|
4. Blockchain smart contracts (`frostfsid`, `policy`) that store user info and access rules.
|
|
|
|
## Data auth process
|
|
|
|
Let's look at the process in more detail:
|
|
|
|
<a>
|
|
<img src="images/authentication/auth-sequence.svg" alt="Auth sequence diagram"/>
|
|
</a>
|
|
|
|
* First of all, someone makes a request. If request is signed we will check its signature (`Authentication`) after that
|
|
we will check access rights using policies (`Auhorization`). For anonymous requests only authorization is performed.
|
|
|
|
* **Authentication steps**:
|
|
* Each signed request is provided with `AccessKeyId` and signature. So if request is signed we must check its
|
|
signature. To do this we must know the `AccessKeyId`/`SecretAccessKey` pair (For how the signature is calculated
|
|
using this pair, see [signing](#aws-signing). Client and server (s3-gw) use the same credentials and algorithm to
|
|
compute signature). The `AccessKeyId` is a public part of credentials, and it's passed to the gate in request. The
|
|
private part of credentials is `SecretAccessKey` and it's encrypted and stored in [AccessBox](#accessbox). So on
|
|
this step we must find appropriate `AccessBox` in FrostFS storage node (For how to find appropriate `AccessBox`
|
|
knowing `AccessKeyId`, see [search algorithm](#search-algorithm)). On this stage we can get `AccessDenied` from
|
|
FrostFS storage node if the s3-gw doesn't have permission to read this `AccessBox` object.
|
|
|
|
* After successfully retrieving the object we must extract `SecretAccessKey` from it. Since it's encrypted, the s3-gw must
|
|
decrypt (see [encryption](#encryption)) this object using its own private key and `SeedKey` from `AccessBox`
|
|
(see [AccessBox inner structure](#accessbox)). After s3-gw got the `AccessKeyId`/`SecretAccessKey` pair it
|
|
[calculates signature](#aws-signing) and compares this signature with one provided by the request. If signature doesn't
|
|
match the `AccessDenied` is returned.
|
|
|
|
* `AccessBox` also contains `OwnerID` that is related to `AccessKeyId` that was provided. So we have to check if
|
|
such `OwnerID` exists in `frsotfsid` contract (that stores all registered valid users). If user doesn't exist in
|
|
contract the `AccessDenied` is returned.
|
|
|
|
* **Authorization steps**:
|
|
* To know if user has access right to do what he wants to do we must find appropriate access policies. Such policies
|
|
are stored in `policy` contract and locally (can be manged using [control api](#control-auth-process)). So we need
|
|
to get policies from contract and [check them](#policies) along with local to decide if user has access right. If
|
|
he doesn't have such right the `AccessDenied` is returned.
|
|
|
|
* After successful authentication and authorization the request will be processed by s3-gw business logic and finally be
|
|
propagated to FrostFS storage node which also performs some auth checks and can return `AccessDenied`. If this happens
|
|
s3-gw also returns `AccessDenied` as a response.
|
|
|
|
### AWS Signing
|
|
|
|
Every interaction with FrostFS S3 gateway is either authenticated or anonymous. This section explains request
|
|
authentication with the AWS Signature Version 4 algorithm. More info in AWS documentation:
|
|
|
|
* [Authenticating Requests (AWS Signature Version 4)](https://docs.aws.amazon.com/AmazonS3/latest/API/sig-v4-authenticating-requests.html)
|
|
* [Signing AWS API requests](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_aws-signing.html)
|
|
|
|
#### Authentication Methods
|
|
|
|
You can express authentication information by using one of the following methods:
|
|
|
|
* **HTTP Authorization header** - Using the HTTP Authorization header is the most common method of authenticating
|
|
FrostFS S3 request. All the FrostFS S3 REST operations (except for browser-based uploads using POST requests) require
|
|
this header. For more information about the Authorization header value, and how to calculate signature and related
|
|
options,
|
|
see [Authenticating Requests: Using the Authorization Header (AWS Signature Version 4)](https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-auth-using-authorization-header.html).
|
|
* **Query string parameters** - You can use a query string to express a request entirely in a URL. In this case, you use
|
|
query parameters to provide request information, including the authentication information. Because the request
|
|
signature is part of the URL, this type of URL is often referred to as a presigned URL. You can use presigned URLs to
|
|
embed clickable links, which can be valid for up to seven days, in HTML. For more information,
|
|
see [Authenticating Requests: Using Query Parameters (AWS Signature Version 4)](https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-query-string-auth.html).
|
|
|
|
FrostFS S3 also supports browser-based uploads that use HTTP POST requests. With an HTTP POST request, you can upload
|
|
content to FrostFS S3 directly from the browser. For information about authenticating POST requests,
|
|
see [Browser-Based Uploads Using POST (AWS Signature Version 4)](https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-UsingHTTPPOST.html).
|
|
|
|
#### Introduction to Signing Requests
|
|
|
|
Authentication information that you send in a request must include a signature. To calculate a signature, you first
|
|
concatenate select request elements to form a string, referred to as the string to sign. You then use a signing key to
|
|
calculate the hash-based message authentication code (HMAC) of the string to sign.
|
|
|
|
In AWS Signature Version 4, you don't use your secret access key to sign the request. Instead, you first use your secret
|
|
access key to derive a signing key. The derived signing key is specific to the date, service, and Region. For more
|
|
information about how to derive a signing key in different programming languages, see Examples of how to derive a
|
|
signing key for Signature Version 4.
|
|
|
|
The following diagram illustrates the general process of computing a signature.
|
|
|
|
<a>
|
|
<img src="images/authentication/aws-signing.png" alt="AWS Signing"/>
|
|
</a>
|
|
|
|
The string to sign depends on the request type. For example, when you use the HTTP Authorization header or the query
|
|
parameters for authentication, you use a varying combination of request elements to create the string to sign. For an
|
|
HTTP POST request, the POST policy in the request is the string you sign. For more information about computing string to
|
|
sign, follow links provided at the end of this section.
|
|
|
|
For signing key, the diagram shows series of calculations, where the result of each step you feed into the next step. The
|
|
final step is the signing key.
|
|
|
|
Upon receiving an authenticated request, FrostFS S3 servers re-create the signature by using the authentication
|
|
information that is contained in the request. If the signatures match, FrostFS S3 processes your request; otherwise, the
|
|
request is rejected.
|
|
|
|
##### Signature Calculations for the Authorization Header
|
|
|
|
To calculate a signature, you first need a string to sign. You then calculate a HMAC-SHA256 hash of the string to sign
|
|
by using a signing key. The following diagram illustrates the process, including the various components of the string
|
|
that you create for signing.
|
|
|
|
When FrostFS S3 receives an authenticated request, it computes the signature and then compares it with the signature
|
|
that you provided in the request. For that reason, you must compute the signature by using the same method that is used
|
|
by FrostFS S3. The process of putting a request in an agreed-upon form for signing is called canonicalization.
|
|
|
|
<a>
|
|
<img src="images/authentication/auth-header-signing.png" alt="Signature Calculations for the Authorization Header"/>
|
|
</a>
|
|
|
|
See detains in [AWS documentation](https://docs.aws.amazon.com/AmazonS3/latest/API/sig-v4-header-based-auth.html).
|
|
|
|
#### s3-gw
|
|
|
|
s3-gw supports the following ways to provide the singed request:
|
|
|
|
* [HTTP Authorization header](https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-auth-using-authorization-header.html)
|
|
* [Query string parameters](https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-query-string-auth.html)
|
|
* [Browser-Based Uploads Using POST](https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-UsingHTTPPOST.html)
|
|
|
|
All these methods provide `AccessKeyId` and signature. Using `AccessKeyId` s3-gw can get `SecretAccessKey`
|
|
(see [data auth](#data-auth-process)) to compute signature using exactly the same mechanics
|
|
as [client does](#introduction-to-signing-requests). After signature calculation the s3-gw just compares signatures and
|
|
if they don't match the access denied is returned.
|
|
|
|
### AccessBox
|
|
|
|
`AccessBox` is an ordinary object in FrostFS storage. It contains all information that can be used by s3-gw to
|
|
successfully authenticate request. Also, it contains data that is required for successful authentication in FrostFS
|
|
storage node.
|
|
|
|
Object s3 credentials are formed based on:
|
|
|
|
* `AccessKeyId` - is concatenated container id and object id (`<cid>0<oid>`) of `AccessBox` (
|
|
e.g. `2XGRML5EW3LMHdf64W2DkBy1Nkuu4y4wGhUj44QjbXBi05ZNvs8WVwy1XTmSEkcVkydPKzCgtmR7U3zyLYTj3Snxf`).
|
|
Or it can be arbitrary user-provided unique string with min length 4 and max length 128.
|
|
* `SecretAccessKey` - hex-encoded random generated 32 bytes (that is encrypted and stored in object payload).
|
|
Or it can be arbitrary user-provided unique string with min length 4 and max length 128.
|
|
|
|
> **Note**: sensitive info in `AccessBox` is [encrypted](#encryption), so only someone who posses specific private key
|
|
> can decrypt such info.
|
|
|
|
`AccessBox` has the following structure:
|
|
|
|
<a>
|
|
<img src="images/authentication/accessbox-object.svg" alt="AccessBox object structure"/>
|
|
</a>
|
|
|
|
**Headers:**
|
|
|
|
`AccessBox` object has the following attributes (at least them, it also can contain custom ones):
|
|
|
|
* `Timestamp` - unix timestamp indicating when the object was created
|
|
* `__SYSTEM__EXPIRATION_EPOCH` - epoch after which the object isn't available anymore
|
|
* `S3-CRDT-Versions-Add` - comma separated list of previous versions of `AccessBox` (
|
|
see [AccessBox versions](#accessbox-versions))
|
|
* `S3-Access-Box-CRDT-Name` - `AccessKeyId` of credentials to which current `AccessBox` is related (
|
|
see [AccessBox versions](#accessbox-versions))
|
|
* `FilePath` - just object name
|
|
|
|
**Payload:**
|
|
|
|
The `AccessBox` payload is an encoded [AccessBox protobuf type](../creds/accessbox/accessbox.proto) .
|
|
It contains:
|
|
|
|
* Seed key - hex-encoded public seed key to compute shared secret using ECDH (see [encryption](#encryption))
|
|
* List of gate data:
|
|
* Gate public key (so that gate (when it will decrypt data later) know which item from the list it should process)
|
|
* Encrypted tokens:
|
|
* `SecretAccessKey` - hex-encoded random generated 32 bytes (or arbitrary user-provided string)
|
|
* Marshaled bearer token - more detail
|
|
in [spec](https://git.frostfs.info/TrueCloudLab/frostfs-api/src/commit/4c68d92468503b10282c8a92af83a56f170c8a3a/acl/types.proto#L189)
|
|
* Marshaled session token - more detail
|
|
in [spec](https://git.frostfs.info/TrueCloudLab/frostfs-api/src/commit/4c68d92468503b10282c8a92af83a56f170c8a3a/session/types.proto#L89)
|
|
* Container placement policies:
|
|
* `LocationsConstraint` - name of location constraint that can be used to create bucket/container using s3
|
|
credentials related to this `AccessBox`
|
|
* Marshaled placement policy - more detail
|
|
in [spec](https://git.frostfs.info/TrueCloudLab/frostfs-api/src/commit/4c68d92468503b10282c8a92af83a56f170c8a3a/netmap/types.proto#L111)
|
|
|
|
#### AccessBox versions
|
|
|
|
Imagine the following scenario:
|
|
|
|
* There is a system where only one s3-gw exists
|
|
* There is an `AccessBox` that can be used by this s3-gw
|
|
* User has s3 credentials (`AccessKeyId`/`SecretAccessKey`) related to corresponding `AccessBox` and can successfully
|
|
make request to s3-gw
|
|
* The system is expanded and a new s3-gw is added
|
|
* User must be able to use the credentials (that he has already had) to make request to the new s3-gw
|
|
|
|
Since `AccessBox` object is immutable and `SecretAccessKey` is encrypted only for restricted list of keys (can be used
|
|
(decrypted) only by limited number of s3-gw) we have to create a new `AccessBox` that has encrypted secrets for a new list
|
|
of s3-gw and is related to the initial s3 credentials (`AccessKeyId`/`SecretAccessKey`). Such relation is done
|
|
by `S3-Access-Box-CRDT-Name`.
|
|
|
|
##### Search algorithm
|
|
|
|
To support scenario from previous section and find appropriate version of `AccessBox` (that contains more recent and
|
|
relevant data) the following sequence is used:
|
|
|
|
<a>
|
|
<img src="images/authentication/accessbox-search.svg" alt="AccessBox search process"/>
|
|
</a>
|
|
|
|
* Search all object whose attribute `S3-Access-Box-CRDT-Name` is equal to `AccessKeyId` (extract container id
|
|
from `AccessKeyId` that has format: `<cid>0<oid>` if `AccessBox` was created with default parameters, or it can also
|
|
be arbitrary user-defined string).
|
|
* Get metadata for these object using `HEAD` requests (not `Get` to reduce network traffic)
|
|
* Sort all these objects by creation epoch and object id
|
|
* Pick last object id (If no object is found then extract object id from `AccessKeyId` that has format: `<cid>0<oid>`
|
|
(if `AccessBox` was created with default parameters, or it can also be arbitrary user-defined string).
|
|
We need to do this because versions of `AccessBox` can miss the `S3-Access-Box-CRDT-Name` attribute.)
|
|
* Get appropriate object from FrostFS storage
|
|
* Decrypt `AccessBox` (see [encryption](#encryption))
|
|
|
|
#### Encryption
|
|
|
|
Each `AccessBox` contains sensitive information (`AccessSecretKey`, bearer/session tokens etc.) that must be protected
|
|
and available only to trusted parties (in our case it's a s3-gw).
|
|
|
|
To encrypt/decrypt data the authenticated encryption with associated
|
|
data ([AEAD](https://en.wikipedia.org/wiki/Authenticated_encryption)) is used. The encryption algorithm
|
|
is [ChaCha20-Poly1305](https://en.wikipedia.org/wiki/ChaCha20-Poly1305) ([RFC](https://datatracker.ietf.org/doc/html/rfc7905)).
|
|
|
|
Is the following algorithm the ECDSA keys (with curve implements NIST P-256 (FIPS 186-3, section D.2.3) also known as
|
|
secp256r1 or prime256v1) is used (unless otherwise stated).
|
|
|
|
**Encryption:**
|
|
|
|
* Create ephemeral key (`SeedKey`), it's need to generate shared secret
|
|
* Generate random 32-byte (that after hex-encoded be `SecretAccessKey`) or use existing secret access key
|
|
(if `AccessBox` is being updated rather than creating brand new) or use arbitrary user-provided string
|
|
* Generate shared secret as [ECDH](https://en.wikipedia.org/wiki/Elliptic-curve_Diffie%E2%80%93Hellman)
|
|
* Derive 32-byte key using shared secret from previous step with key derivation function based on
|
|
HMAC with SHA256 [HKDF](https://en.wikipedia.org/wiki/HKDF)
|
|
* Encrypt marshaled [Tokens](../creds/accessbox) using derived key
|
|
with [ChaCha20-Poly1305](https://en.wikipedia.org/wiki/ChaCha20-Poly1305) algorithm without additional data.
|
|
|
|
**Decryption:**
|
|
|
|
* Get public part of `SeedKey` from `AccessBox`
|
|
* Generate shared secret as follows:
|
|
* Make scalar curve multiplication of public part of `SeedKey` and private part of s3-gw key
|
|
* Use `X` part of multiplication (with zero padding at the beginning to fit 32-byte)
|
|
* Derive 32-byte key using shared secret from previous step with key derivation function based on
|
|
HMAC with SHA256 [HKDF](https://en.wikipedia.org/wiki/HKDF)
|
|
* Decrypt encrypted marshaled [Tokens](../creds/accessbox) using derived key
|
|
with [ChaCha20-Poly1305](https://en.wikipedia.org/wiki/ChaCha20-Poly1305) algorithm without additional data.
|
|
|
|
### Policies
|
|
|
|
The main repository that contains policy implementation is https://git.frostfs.info/TrueCloudLab/policy-engine.
|
|
|
|
Policies can be stored locally (using [control api](#control-auth-process)) or in `policy` contract. When policies check
|
|
is performed the following algorithm is applied:
|
|
|
|
* Check local policies:
|
|
* If any rule was matched return checking result.
|
|
* Check contract policies:
|
|
* If any rule was matched return checking result.
|
|
* If no rules were matched return `deny` status.
|
|
|
|
To local and contract policies `deny first` scheme is applied. This means that if several rules were matched for
|
|
reqeust (with both statuses `allow` and `deny`) the resulting status is `deny`.
|
|
|
|
Policy rules validate if specified request can be performed on the specific resource. Request and resource can contain
|
|
some properties, and rules can contain conditions on some of these properties.
|
|
|
|
In s3-gw resource is `/bucket/object`, `/bucket` or just `/` (if request is trying to list buckets).
|
|
Currently, request that is checked contains the following properties (so policy rule can contain conditions on them):
|
|
|
|
* `Owner` - address of owner that is performing request (this is taken from bearer token from `AccessBox`)
|
|
* `frostfsid:groupID` - groups to which the owner belongs (this is taken from `frostfsid` contract)
|
|
|
|
## Control auth process
|
|
|
|
There are control path [grpc api](../pkg/service/control/service.proto) in s3-gw that also has their own authentication
|
|
and authorization process.
|
|
|
|
But this process is quite straight forward:
|
|
|
|
* Get grpc request
|
|
* Check if signing key belongs to [allowed key list](configuration.md#control-section) (that is located in config file)
|
|
* Validate signature
|
|
|
|
For signing process the asymmetric encryption based on elliptic curves (`ECDSA_SHA512`) is used.
|
|
For more details see the appropriate code
|
|
in [frostfs-api](https://git.frostfs.info/TrueCloudLab/frostfs-api/src/commit/4c68d92468503b10282c8a92af83a56f170c8a3a/refs/types.proto#L94)
|
|
and [frostfs-api-go](https://git.frostfs.info/TrueCloudLab/frostfs-api-go/src/commit/a85146250b312fcdd6da9a71285527fed544234f/refs/types.go#L38).
|