Check Content-Encoding header in chunked payload based on kludge flag #146

Closed
opened 2023-06-21 08:24:38 +00:00 by alexvanin · 1 comment
Owner

In #126 gateway uses chunk payload reader if request contains X-Amz-Content-Sha256 header with STREAMING-AWS4-HMAC-SHA256-PAYLOAD value. However specification also mentions Content-Encoding header with aws-chunked value. However not all clients set this header, e.g. minio-go client.

Describe the solution you'd like

Add a new flag to kludge config section: ignore_content_encoding_in_chunks. If it is set, then ignore Content-Encoding header in chunk payloads. Otherwise check it and return error if it is missing.

Describe alternatives you've considered

None.

Additional context

None.

## Is your feature request related to a problem? Please describe. In #126 gateway uses chunk payload reader if request contains `X-Amz-Content-Sha256` header with `STREAMING-AWS4-HMAC-SHA256-PAYLOAD` value. However specification also [mentions](https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-streaming.html) `Content-Encoding` header with `aws-chunked` value. However not all clients set this header, e.g. minio-go client. ## Describe the solution you'd like Add a new flag to `kludge` config section: `ignore_content_encoding_in_chunks`. If it is set, then ignore `Content-Encoding` header in chunk payloads. Otherwise check it and return error if it is missing. ## Describe alternatives you've considered None. ## Additional context None.
Member

If it is set, then ignore Content-Encoding

Actually, we should bypass this header without aws-chuncked but still eliminate this value from header, because it can contains several values and we should preserve user-defined values. see https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-streaming.html

> If it is set, then ignore Content-Encoding Actually, we should bypass this header without `aws-chuncked` but still eliminate this value from header, because it can contains several values and we should preserve user-defined values. see https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-streaming.html
alexvanin added this to the v0.28.0 milestone 2023-09-08 12:01:40 +00:00
Sign in to join this conversation.
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: TrueCloudLab/frostfs-s3-gw#146
No description provided.