Parse FilePath in `/get/<cid>/<oid>` endpoint #214

New issue

Open

opened 2025-02-14 10:07:45 +00:00 by alexvanin · 3 comments

alexvanin commented

2025-02-14 10:07:45 +00:00

Owner

We want to support these cases in HTTP Gateway:

get S3 Object <bucket-name>/<key>
get FrostFS object based on FrostFS identifiers <cid>/<oid>
get FrostFS object based on nice name identifiers (FilePath for and NNS name in )
support combined versions of <cid>/<FilePath> and <cnr-name>/<oid>
return index page for <bucket-name>/*/ + <cnr-name>/*/ and <bucket-name>/ + <cnr-name>/

All these cases should be supported in /get/<cid>/<oid>. Right now we do not support <cnr-name>/<FilePath> and maybe some combinations.

This is useful when objects are stored by some 3rd-paty tools like [rclone] and received from HTTP gateway. These tools may support container nice names.

We want to support these cases in HTTP Gateway: - get S3 Object `<bucket-name>/<key>` - get FrostFS object based on FrostFS identifiers `<cid>/<oid>` - get FrostFS object based on nice name identifiers (FilePath for <oid> and NNS name in <cid>) - support combined versions of `<cid>/<FilePath>` and `<cnr-name>/<oid>` - return index page for `<bucket-name>/*/` + `<cnr-name>/*/` and `<bucket-name>/` + `<cnr-name>/` All these cases should be supported in `/get/<cid>/<oid>`. Right now we do not support `<cnr-name>/<FilePath>` and maybe some combinations. This is useful when objects are stored by some 3rd-paty tools like [rclone] and received from HTTP gateway. These tools may support container nice names.

🚀 1

alexvanin commented

2025-02-14 10:10:42 +00:00

Author

Owner

We can discuss moving index page somewhere different, e.g. /index/<bucket-name>. The only issue I see is that maybe download links will be a bit more convoluted (need to replace /index/ with /get/)

In my opinion, we can keep index page in /get/.

I invite you for discussion: @dkirillov, @realloc, @nzinkevich

We can discuss moving index page somewhere different, e.g. `/index/<bucket-name>`. The only issue I see is that maybe download links will be a bit more convoluted (need to replace `/index/` with `/get/`) In my opinion, we can keep index page in `/get/`. I invite you for discussion: @dkirillov, @realloc, @nzinkevich

dkirillov commented

2025-02-17 14:22:27 +00:00

Member

As I understand, after #207 and

support combined versions of / and /

It's getting harder to tell the difference between <cid>/<invali-FilePath-that-ends-with-slash> and index page <cid>/<prefix>.

I would like to see index on different enpoint, keep /get/... just for returning single object.

For example github use ../tree/mater/dir to list directory and ../blob/master/dir/file.txt to get it.
Forgejo also has different endpoint to get raw files.

As I understand, after https://git.frostfs.info/TrueCloudLab/frostfs-http-gw/issues/207 and > support combined versions of <cid>/<FilePath> and <cnr-name>/<oid> It's getting harder to tell the difference between `<cid>/<invali-FilePath-that-ends-with-slash>` and index page `<cid>/<prefix>`. I would like to see index on different enpoint, keep `/get/...` just for returning single object. For example github use `../tree/mater/dir` to list directory and `../blob/master/dir/file.txt` to get it. Forgejo also has different endpoint to get raw files.

alexvanin added this to the v0.33.0 milestone 2025-02-27 08:23:06 +00:00

alexvanin commented

2025-03-05 06:37:52 +00:00

Author

Owner

I would like to see index on different enpoint, keep /get/... just for returning single object.

I had a discussion with @realloc about this. We pointed out that regular web-servers like nginx or apache usually try multiple options before sending response to the user. So we are not going to simplify gateway, instead it will be more complex, unfortunately.

So expected behavior is:

try to get object with filepath ended in slash
if no object found, then draw index page

> I would like to see index on different enpoint, keep /get/... just for returning single object. I had a discussion with @realloc about this. We pointed out that regular web-servers like nginx or apache usually try multiple options before sending response to the user. So we are not going to simplify gateway, instead it will be more complex, unfortunately. So expected behavior is: 1) try to get object with filepath ended in slash 2) if no object found, then draw index page