[#170] Support tar.gz archives downloading #177

Open
nzinkevich wants to merge 2 commits from nzinkevich/frostfs-http-gw:tar_download into master
Member

Add handler DownloadTar for downloading tar.gz archives with objects by prefix (same as zip). Make methods more universal for using in both implementations

Add handler DownloadTar for downloading tar.gz archives with objects by prefix (same as `zip`). Make methods more universal for using in both implementations
nzinkevich self-assigned this 2024-12-06 12:41:27 +00:00
nzinkevich added 2 commits 2024-12-06 12:41:28 +00:00
Split DownloadZipped handler on methods. Add handler DownloadTar for downloading tar.gz archives. Make methods more universal for using in both implementations

Signed-off-by: Nikita Zinkevich <n.zinkevich@yadro.com>
[#170] Updated docs and configuration of archive section
All checks were successful
/ DCO (pull_request) Successful in 3m9s
/ Vulncheck (pull_request) Successful in 3m16s
/ Builds (pull_request) Successful in 1m54s
/ Lint (pull_request) Successful in 2m58s
/ Tests (pull_request) Successful in 1m42s
967ed1d6ae
Signed-off-by: Nikita Zinkevich <n.zinkevich@yadro.com>
nzinkevich requested review from alexvanin 2024-12-06 12:41:28 +00:00
nzinkevich requested review from dkirillov 2024-12-06 12:41:28 +00:00
nzinkevich requested review from storage-services-committers 2024-12-06 12:41:47 +00:00
nzinkevich requested review from storage-services-developers 2024-12-06 12:41:51 +00:00
nzinkevich changed title from [#170] Support downloading tar.gz archive to [#170] Support tar.gz archives downloading 2024-12-06 12:42:50 +00:00
dkirillov reviewed 2024-12-11 11:36:30 +00:00
@ -130,2 +130,2 @@
// Zip compression.
cfgZipCompression = "zip.compression"
// Archive compression.
cfgArchiveCompression = "archive.compression"
Member

I'm not sure if we can remove zip.compression parameter. Probably we should keep it for one release.
cc @alexvanin

I'm not sure if we can remove `zip.compression` parameter. Probably we should keep it for one release. cc @alexvanin
@ -65,0 +61,4 @@
| Common headers | See [bearer token](#bearer-token). |
| `X-Attribute-System-*` | Used to set system FrostFS object attributes <br/> (e.g. use "X-Attribute-System-Expiration-Epoch" to set `__SYSTEM__EXPIRATION_EPOCH` attribute). |
| `X-Attribute-*` | Used to set regular object attributes <br/> (e.g. use "X-Attribute-My-Tag" to set `My-Tag` attribute). |
| `X-Attribute-Explode-Archive` | If this header is set, gate reads files from uploaded `tar.gz` archive and creates object for each file in it. Set FilePath attribute as relative path from archive root. |
Member

This should be X-Explode-Archive

This should be `X-Explode-Archive`
@ -115,3 +140,1 @@
c.Response.Header.Set(fasthttp.HeaderContentType, "application/zip")
c.Response.Header.Set(fasthttp.HeaderContentDisposition, "attachment; filename=\"archive.zip\"")
c.Response.SetStatusCode(http.StatusOK)
c.Response.Header.Set(fasthttp.HeaderContentType, "application/x-gzip")
Member

Shouldn't it be application/x-gtar ?
https://en.wikipedia.org/wiki/List_of_archive_formats

Shouldn't it be `application/x-gtar` ? https://en.wikipedia.org/wiki/List_of_archive_formats
@ -165,1 +197,3 @@
}
func (h *Handler) putObjectToArchive(ctx context.Context, log *zap.Logger, cnrID cid.ID, bufZip *[]byte, createArchiveHeader func(obj *object.Object) (io.Writer, error)) func(id oid.ID) bool {
return func(id oid.ID) bool {
log := log.With(zap.String("oid", id.EncodeToString()))
Member

Should be

log = log.With(zap.String("oid", id.EncodeToString()))
Should be ```golang log = log.With(zap.String("oid", id.EncodeToString())) ```
@ -86,2 +88,4 @@
}
if header := c.Request.Header.Peek(utils.UserAttributeHeaderPrefix + explodeArchiveHeader); header != nil {
h.explodeGzip(c, log, bktInfo, file)
Member

Is this different from #176 ?
If no than we should either:

  • keep only one PR with logically separated commits (upload/download etc)
  • make this PR draft and mention that it requires #176 (and again commits be different)

If yes than why do we ever need #176 ?

Is this different from https://git.frostfs.info/TrueCloudLab/frostfs-http-gw/pulls/176 ? If no than we should either: * keep only one PR with logically separated commits (upload/download etc) * make this PR draft and mention that it requires #176 (and again commits be different) If yes than why do we ever need #176 ?
All checks were successful
/ DCO (pull_request) Successful in 3m9s
Required
Details
/ Vulncheck (pull_request) Successful in 3m16s
Required
Details
/ Builds (pull_request) Successful in 1m54s
Required
Details
/ Lint (pull_request) Successful in 2m58s
Required
Details
/ Tests (pull_request) Successful in 1m42s
Required
Details
This pull request doesn't have enough approvals yet. 0 of 2 approvals granted.
This branch is out-of-date with the base branch
You are not authorized to merge this pull request.
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u tar_download:nzinkevich-tar_download
git checkout nzinkevich-tar_download
Sign in to join this conversation.
No reviewers
TrueCloudLab/storage-services-committers
TrueCloudLab/storage-services-developers
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: TrueCloudLab/frostfs-http-gw#177
No description provided.