TrueCloudLab/frostfs-http-gw
17
1
Fork 18
Code Issues 11 Pull requests 5 Projects Releases 16 Activity Actions

[#197] Add a leading slash to the FilePath attribute #209

Open
r.loginov wants to merge 1 commit from r.loginov/frostfs-http-gw:feature/197-treat_filepath_attribute_with_leading_slash into master
pull from: r.loginov/frostfs-http-gw:feature/197-treat_filepath_attribute_with_leading_slash
merge into: TrueCloudLab:master
Conversation 0 Commits 1 Files changed 2 +51 -9
r.loginov commented 2025-02-10 18:34:36 +00:00
Member
Copy link

close #197

According to the frostfs api specification,
the File Path attribute must start with a
leading slash. More info:
https://git.frostfs.info/TrueCloudLab/frostfs-api

Test scenarios:
CID: 2yDhMPpQpYJEdR1aWqKdBo3SaTLTU4tQm841BWaP2HPZ

Creating an object with FilePath=/global/cats/cat.txt:

frostfs-cli object put --file cat.txt --cid $cid -r localhost:8080 -w /home/roma/dev/forgejo/upstream/frostfs-aio/s3-gw/user-wallet.json --attributes FilePath="/global/ca
ts/cat.txt"
Enter password > 
 12 / 12 [======================================================================================================================================================] 100.00% 0s
[cat.txt] Object successfully stored
  OID: 2vzydzMZCbDLbGeoWnqrBmQ56UKqCLxawptZmKCevkb7
  CID: 2yDhMPpQpYJEdR1aWqKdBo3SaTLTU4tQm841BWaP2HPZ

Let's check the values of its attributes:

frostfs-cli object head --cid 2yDhMPpQpYJEdR1aWqKdBo3SaTLTU4tQm841BWaP2HPZ --oid 2vzydzMZCbDLbGeoWnqrBmQ56UKqCLxawptZmKCevkb7 -r localhost:8080 -w /home/roma/dev/forgejo/
upstream/frostfs-aio/s3-gw/user-wallet.json
...
Attributes:
  FilePath=/global/cats/cat.txt
  FileName=cat.txt
  Timestamp=1739125897 (2025-02-09 21:31:37 +0300 MSK)
...

Executing the query without using the leading slash at the beginning of the `FilePath' attribute

curl -i  http://localhost:8881/get_by_attribute/2yDhMPpQpYJEdR1aWqKdBo3SaTLTU4tQm841BWaP2HPZ/FilePath/global/cats/cat.txt -H "Authorization: Bearer $token"
HTTP/1.1 200 OK
Date: Sun, 09 Feb 2025 18:32:57 GMT
Content-Type: text/plain; charset=utf-8
Content-Length: 12
X-Object-Id: 2vzydzMZCbDLbGeoWnqrBmQ56UKqCLxawptZmKCevkb7
X-Owner-Id: NWeByJPgNC97F83hTUnSbnZSBKaFvk5HNw
X-Container-Id: 2yDhMPpQpYJEdR1aWqKdBo3SaTLTU4tQm841BWaP2HPZ
X-Attribute-FilePath: /global/cats/cat.txt
X-Attribute-FileName: cat.txt
X-Attribute-Timestamp: 1739125897
Last-Modified: Sun, 09 Feb 2025 18:31:37 GMT
Content-Disposition: inline; filename=cat.txt
Access-Control-Max-Age: 600

this is cat

Execute the query using the leading slash at the beginning of the `FilePath' attribute:

curl -i  http://localhost:8881/get_by_attribute/2yDhMPpQpYJEdR1aWqKdBo3SaTLTU4tQm841BWaP2HPZ/FilePath//global/cats/cat.txt -H "Authorization: Bearer $token"
HTTP/1.1 200 OK
Date: Sun, 09 Feb 2025 18:33:42 GMT
Content-Type: text/plain; charset=utf-8
Content-Length: 12
X-Object-Id: 2vzydzMZCbDLbGeoWnqrBmQ56UKqCLxawptZmKCevkb7
X-Owner-Id: NWeByJPgNC97F83hTUnSbnZSBKaFvk5HNw
X-Container-Id: 2yDhMPpQpYJEdR1aWqKdBo3SaTLTU4tQm841BWaP2HPZ
X-Attribute-FilePath: /global/cats/cat.txt
X-Attribute-FileName: cat.txt
X-Attribute-Timestamp: 1739125897
Last-Modified: Sun, 09 Feb 2025 18:31:37 GMT
Content-Disposition: inline; filename=cat.txt
Access-Control-Max-Age: 600

this is cat

Signed-off-by: Roman Loginov r.loginov@yadro.com

close #197 According to the frostfs api specification, the File Path attribute must start with a leading slash. More info: https://git.frostfs.info/TrueCloudLab/frostfs-api Test scenarios: CID: `2yDhMPpQpYJEdR1aWqKdBo3SaTLTU4tQm841BWaP2HPZ` Creating an object with `FilePath=/global/cats/cat.txt`: ``` frostfs-cli object put --file cat.txt --cid $cid -r localhost:8080 -w /home/roma/dev/forgejo/upstream/frostfs-aio/s3-gw/user-wallet.json --attributes FilePath="/global/ca ts/cat.txt" Enter password > 12 / 12 [======================================================================================================================================================] 100.00% 0s [cat.txt] Object successfully stored OID: 2vzydzMZCbDLbGeoWnqrBmQ56UKqCLxawptZmKCevkb7 CID: 2yDhMPpQpYJEdR1aWqKdBo3SaTLTU4tQm841BWaP2HPZ ``` Let's check the values of its attributes: ``` frostfs-cli object head --cid 2yDhMPpQpYJEdR1aWqKdBo3SaTLTU4tQm841BWaP2HPZ --oid 2vzydzMZCbDLbGeoWnqrBmQ56UKqCLxawptZmKCevkb7 -r localhost:8080 -w /home/roma/dev/forgejo/ upstream/frostfs-aio/s3-gw/user-wallet.json ... Attributes: FilePath=/global/cats/cat.txt FileName=cat.txt Timestamp=1739125897 (2025-02-09 21:31:37 +0300 MSK) ... ``` Executing the query without using the leading slash at the beginning of the `FilePath' attribute ``` curl -i http://localhost:8881/get_by_attribute/2yDhMPpQpYJEdR1aWqKdBo3SaTLTU4tQm841BWaP2HPZ/FilePath/global/cats/cat.txt -H "Authorization: Bearer $token" HTTP/1.1 200 OK Date: Sun, 09 Feb 2025 18:32:57 GMT Content-Type: text/plain; charset=utf-8 Content-Length: 12 X-Object-Id: 2vzydzMZCbDLbGeoWnqrBmQ56UKqCLxawptZmKCevkb7 X-Owner-Id: NWeByJPgNC97F83hTUnSbnZSBKaFvk5HNw X-Container-Id: 2yDhMPpQpYJEdR1aWqKdBo3SaTLTU4tQm841BWaP2HPZ X-Attribute-FilePath: /global/cats/cat.txt X-Attribute-FileName: cat.txt X-Attribute-Timestamp: 1739125897 Last-Modified: Sun, 09 Feb 2025 18:31:37 GMT Content-Disposition: inline; filename=cat.txt Access-Control-Max-Age: 600 this is cat ``` Execute the query using the leading slash at the beginning of the `FilePath' attribute: ``` curl -i http://localhost:8881/get_by_attribute/2yDhMPpQpYJEdR1aWqKdBo3SaTLTU4tQm841BWaP2HPZ/FilePath//global/cats/cat.txt -H "Authorization: Bearer $token" HTTP/1.1 200 OK Date: Sun, 09 Feb 2025 18:33:42 GMT Content-Type: text/plain; charset=utf-8 Content-Length: 12 X-Object-Id: 2vzydzMZCbDLbGeoWnqrBmQ56UKqCLxawptZmKCevkb7 X-Owner-Id: NWeByJPgNC97F83hTUnSbnZSBKaFvk5HNw X-Container-Id: 2yDhMPpQpYJEdR1aWqKdBo3SaTLTU4tQm841BWaP2HPZ X-Attribute-FilePath: /global/cats/cat.txt X-Attribute-FileName: cat.txt X-Attribute-Timestamp: 1739125897 Last-Modified: Sun, 09 Feb 2025 18:31:37 GMT Content-Disposition: inline; filename=cat.txt Access-Control-Max-Age: 600 this is cat ``` Signed-off-by: Roman Loginov <r.loginov@yadro.com>
r.loginov self-assigned this 2025-02-10 18:34:36 +00:00
r.loginov added 1 commit 2025-02-10 18:34:37 +00:00
[#197] Add a leading slash to the FilePath attribute
All checks were successful
/ DCO (pull_request) Successful in 34s
Details
/ Vulncheck (pull_request) Successful in 50s
Details
/ Builds (pull_request) Successful in 57s
Details
/ OCI image (pull_request) Successful in 1m18s
Details
/ Lint (pull_request) Successful in 2m15s
Details
/ Tests (pull_request) Successful in 1m0s
Details
/ Integration tests (pull_request) Successful in 5m40s
Details
b07d8fcf81 
According to the frostfs api specification,
the File Path attribute must start with a
leading slash. More info:
https://git.frostfs.info/TrueCloudLab/frostfs-api

Signed-off-by: Roman Loginov <r.loginov@yadro.com>
requested reviews from storage-services-developers, storage-services-committers 2025-02-10 18:34:37 +00:00
r.loginov reviewed 2025-02-10 18:38:37 +00:00
internal/handler/handler.go
@ -319,6 +318,18 @@ func (h *Handler) needSearchByFileName(key, val string) bool {
return strings.HasPrefix(val, "/") && strings.Count(val, "/") == 1 || !strings.Contains(val, "/")
}
func prepareAtribute(attrKey, attrVal string) string {
r.loginov commented 2025-02-10 18:38:33 +00:00
Author
Member
Copy link

It seems in this case it will be very convenient to disable strict compliance with the specification (#207) through a parameter from the configuration. It will only be enough to add a check for the value of the new flag.

It seems in this case it will be very convenient to disable strict compliance with the specification (#207) through a parameter from the configuration. It will only be enough to add a check for the value of the new flag.
👍 1
dkirillov approved these changes 2025-02-11 08:28:31 +00:00
r.loginov reviewed 2025-02-11 13:29:07 +00:00
internal/handler/handler.go
@ -322,1 +330,4 @@
return attrVal
}
func prepareFileName(fileName string) string {
r.loginov commented 2025-02-11 13:29:07 +00:00
Author
Member
Copy link

I have a question here. Don't we want to check the FileName for the absence of slashes in general, and not just at the beginning? After all, according to the specification, FileName should not contain slashes at all.

I have a question here. Don't we want to check the `FileName` for the absence of slashes in general, and not just at the beginning? After all, according to the specification, `FileName` should not contain slashes at all.
r.loginov commented 2025-02-17 08:27:18 +00:00
Author
Member
Copy link

We decided that it still needed to be done. Created a task: #215

We decided that it still needed to be done. Created a task: #215
r.loginov marked this conversation as resolved
alexvanin added this to the v0.33.0 milestone 2025-02-11 15:45:06 +00:00
All checks were successful
/ DCO (pull_request) Successful in 34s
Required
Details
/ Vulncheck (pull_request) Successful in 50s
Required
Details
/ Builds (pull_request) Successful in 57s
Required
Details
/ OCI image (pull_request) Successful in 1m18s
Required
Details
/ Lint (pull_request) Successful in 2m15s
Required
Details
/ Tests (pull_request) Successful in 1m0s
Required
Details
/ Integration tests (pull_request) Successful in 5m40s
Required
Details
This pull request doesn't have enough approvals yet. 1 of 2 approvals granted.
This branch is out-of-date with the base branch
You are not authorized to merge this pull request.
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u feature/197-treat_filepath_attribute_with_leading_slash:r.loginov-feature/197-treat_filepath_attribute_with_leading_slash
git checkout r.loginov-feature/197-treat_filepath_attribute_with_leading_slash
Sign in to join this conversation.
Reviewers
No reviewers
dkirillov
Labels
Clear labels   
P0

Highest

  
P1

High

  
P2

Medium

  
P3

Low

  
good first issue

Good for newcomers

  
Infrastructure

Infrastructure, Automation, CI/CD or DevOps related stuff

  
blocked

The issue or PR is blocked by something

  
bug

Something is not working

  
config

Configuration format update or breaking change

  
discussion

Talking about something in order to reach a decision or to exchange ideas

  
documentation

Documentation related

  
duplicate

This issue or pull request already exists

  
enhancement

New feature

  
go

Language features adoption

  
help wanted

Extra attention is needed

  
internal

   
invalid

Something is wrong

  
kludge

This is a kludge and we know it

  
observability

Related to metrics, tracing and logs

  
perfomance

Perfomance related

  
question

More information is needed

  
refactoring

Refactoring

  
wontfix

This won't be fixed

No labels
P0
P1
P2
P3
good first issue
Infrastructure
blocked
bug
config
discussion
documentation
duplicate
enhancement
go
help wanted
internal
invalid
kludge
observability
perfomance
question
refactoring
wontfix
Milestone
Clear milestone
No items
No milestone
v0.33.0
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
r.loginov
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: TrueCloudLab/frostfs-http-gw#209
No description provided.
Delete branch "r.loginov/frostfs-http-gw:feature/197-treat_filepath_attribute_with_leading_slash"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?