TrueCloudLab/frostfs-http-gw
17
1
Fork 18
Code Issues 11 Pull requests 1 Projects Releases 16 Packages 1 Activity Actions

[#188] Improve content-type detector #189

Merged
alexvanin merged 1 commit from KurlesHS/frostfs-http-gw:feat/improve_content_type_detector into master 2025-01-21 06:52:37 +00:00
Conversation 0 Commits 1 Files changed 3 +87 -20
KurlesHS commented 2024-12-24 16:41:45 +00:00
Member
Copy link

I looked at several libraries that guess the content type from the content, but none of them correctly identified either the text/css or text/javascript content type.
I also investigated how the rclone utility and the MinIO server determine the type of content. Both of these software first pay attention to the content-type header. If it's missing, they set the content type by file extension.

In our case, I decided to combine both methods. First, we will attempt to determine the content type based on the content itself. If the content appears to be text/plain, we will then use the file extension to determine the exact type of content.

Signed-off-by: Aleksey Kravchenko al.kravchenko@yadro.com

I looked at several libraries that guess the content type from the content, but none of them correctly identified either the `text/css` or `text/javascript` content type. I also investigated how the rclone utility and the MinIO server determine the type of content. Both of these software first pay attention to the `content-type` header. If it's missing, they set the content type by file extension. In our case, I decided to combine both methods. First, we will attempt to determine the content type based on the content itself. If the content appears to be `text/plain`, we will then use the file extension to determine the exact type of content. Signed-off-by: Aleksey Kravchenko <al.kravchenko@yadro.com>
KurlesHS added 1 commit 2024-12-24 16:41:46 +00:00
[#188] Improve content-type detector
All checks were successful
/ DCO (pull_request) Successful in 2m58s
Details
/ Vulncheck (pull_request) Successful in 3m14s
Details
/ Builds (pull_request) Successful in 4m1s
Details
/ Lint (pull_request) Successful in 4m8s
Details
/ Tests (pull_request) Successful in 3m36s
Details
a1097b7c59 
Signed-off-by: Aleksey Kravchenko <al.kravchenko@yadro.com>
requested reviews from storage-services-developers, storage-services-committers 2024-12-24 16:41:46 +00:00
KurlesHS self-assigned this 2024-12-24 16:42:37 +00:00
alexvanin added this to the v0.33.0 milestone 2024-12-25 06:33:15 +00:00
dkirillov approved these changes 2024-12-25 08:23:22 +00:00
Dismissed
dkirillov left a comment
Member
Copy link

LGTM

LGTM
mbiryukova approved these changes 2025-01-10 04:39:49 +00:00
Dismissed
internal/handler/reader_test.go Outdated
@ -34,0 +64,4 @@
{
Name: "html content type by file extension",
ExpectedContentType: htmlContentType,
Content: htmlBody,
mbiryukova commented 2025-01-10 04:39:36 +00:00
Member
Copy link

Should sb.String() be used here to determine content type by extension?

Should `sb.String()` be used here to determine content type by extension?
KurlesHS marked this conversation as resolved
KurlesHS force-pushed feat/improve_content_type_detector from a1097b7c59 to d465d90bd4 2025-01-13 06:13:15 +00:00 Compare
KurlesHS dismissed dkirillov's review 2025-01-13 06:13:16 +00:00
Reason:

New commits pushed, approval review dismissed automatically according to repository settings

KurlesHS dismissed mbiryukova's review 2025-01-13 06:13:16 +00:00
Reason:

New commits pushed, approval review dismissed automatically according to repository settings

mbiryukova approved these changes 2025-01-13 06:16:55 +00:00
Dismissed
dkirillov approved these changes 2025-01-14 13:50:17 +00:00
Dismissed
alexvanin approved these changes 2025-01-21 06:40:53 +00:00
Dismissed
alexvanin left a comment
Owner
Copy link

Overall LGTM, see comment about one more test case

Overall LGTM, see comment about one more test case
internal/handler/reader_test.go Outdated
@ -34,0 +65,4 @@
Name: "html content type by file extension",
ExpectedContentType: htmlContentType,
Content: sb.String(),
FileName: "test.html",
alexvanin commented 2025-01-21 06:40:29 +00:00
Owner
Copy link

Can you also add a test case when filename is a filepath to cover this code

if filename == "" {
	filename = filepath
}
Can you also add a test case when filename is a filepath to cover this code ```go if filename == "" { filename = filepath } ```
KurlesHS commented 2025-01-21 06:50:10 +00:00
Author
Member
Copy link

Can you also add a test case when filename is a filepath to cover this code

Done.

> Can you also add a test case when filename is a filepath to cover this code Done.
alexvanin marked this conversation as resolved
KurlesHS force-pushed feat/improve_content_type_detector from d465d90bd4 to 787722df33 2025-01-21 06:48:16 +00:00 Compare
KurlesHS dismissed mbiryukova's review 2025-01-21 06:48:17 +00:00
Reason:

New commits pushed, approval review dismissed automatically according to repository settings

KurlesHS dismissed dkirillov's review 2025-01-21 06:48:17 +00:00
Reason:

New commits pushed, approval review dismissed automatically according to repository settings

KurlesHS dismissed alexvanin's review 2025-01-21 06:48:17 +00:00
Reason:

New commits pushed, approval review dismissed automatically according to repository settings

alexvanin approved these changes 2025-01-21 06:52:32 +00:00
alexvanin merged commit f0c999d9a2 into master 2025-01-21 06:52:37 +00:00
alexvanin deleted branch feat/improve_content_type_detector 2025-01-21 06:52:39 +00:00
Sign in to join this conversation.
Reviewers
No reviewers
alexvanin
Labels
Clear labels   
P0

Highest

  
P1

High

  
P2

Medium

  
P3

Low

  
good first issue

Good for newcomers

  
Infrastructure

Infrastructure, Automation, CI/CD or DevOps related stuff

  
blocked

The issue or PR is blocked by something

  
bug

Something is not working

  
config

Configuration format update or breaking change

  
discussion

Talking about something in order to reach a decision or to exchange ideas

  
documentation

Documentation related

  
duplicate

This issue or pull request already exists

  
enhancement

New feature

  
go

Language features adoption

  
help wanted

Extra attention is needed

  
internal

   
invalid

Something is wrong

  
kludge

This is a kludge and we know it

  
observability

Related to metrics, tracing and logs

  
perfomance

Perfomance related

  
question

More information is needed

  
refactoring

Refactoring

  
wontfix

This won't be fixed

No labels
P0
P1
P2
P3
good first issue
Infrastructure
blocked
bug
config
discussion
documentation
duplicate
enhancement
go
help wanted
internal
invalid
kludge
observability
perfomance
question
refactoring
wontfix
Milestone
Clear milestone
No items
No milestone
v0.33.0
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
KurlesHS
4 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: TrueCloudLab/frostfs-http-gw#189
No description provided.
Delete branch "KurlesHS/frostfs-http-gw:feat/improve_content_type_detector"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?