TrueCloudLab/frostfs-node
20
1
Fork 27
Code Issues 93 Pull requests 4 Projects Releases 1 Activity Actions

adm: Allow to download contracts from Gitea #958

Merged
fyrchik merged 1 commit from okonstantinova/frostfs-node:feature/466-download_contracts_from_gitea into master 2024-02-09 07:50:35 +00:00
Conversation 2 Commits 1 Files changed 6 +108 -7
okonstantinova commented 2024-02-04 20:33:03 +00:00
Contributor
Copy link

Close #466

For the morph init command the contract command line parameter has become optional and the contract-url parameter has been introduced. The contract-url parameter allows to specify an HTTP link to download a contract archive from. If neither of the parameters is used then the default behavior is to take the latest release from the frostfs-contract repository.

Signed-off-by: Olga Konstantinova kola43843@gmail.com

Close #466 For the `morph init` command the `contract` command line parameter has become optional and the `contract-url` parameter has been introduced. The `contract-url` parameter allows to specify an HTTP link to download a contract archive from. If neither of the parameters is used then the default behavior is to take the latest release from the `frostfs-contract` repository. Signed-off-by: Olga Konstantinova <kola43843@gmail.com>
acid-ant changed title from [#466] Allow to download contracts from Gitea to adm: Allow to download contracts from Gitea 2024-02-05 06:19:26 +00:00
acid-ant requested review from storage-core-committers 2024-02-05 06:20:37 +00:00
acid-ant requested review from storage-core-developers 2024-02-05 06:20:38 +00:00
acid-ant commented 2024-02-05 06:23:34 +00:00
Member
Copy link

Could you please reword commit message a bit:
[#466] Allow to download... --> [#466] adm: Allow to download...

Could you please reword commit message a bit: `[#466] Allow to download...` --> `[#466] adm: Allow to download...`
acid-ant requested changes 2024-02-05 06:56:25 +00:00
cmd/frostfs-adm/internal/modules/morph/initialize.go Outdated
@ -242,0 +252,4 @@
}
ctrURL, err := cmd.Flags().GetString(contractsURLFlag)
if err != nil {
acid-ant commented 2024-02-05 06:43:52 +00:00
Member
Copy link

It is ok to skip check for the error here, the function is redundant at all.

It is ok to skip check for the error here, the function is redundant at all.
👍 1
okonstantinova commented 2024-02-06 22:53:50 +00:00
Author
Contributor
Copy link

Thanks, fixed

Thanks, fixed
fyrchik marked this conversation as resolved
cmd/frostfs-adm/internal/modules/morph/root.go Outdated
@ -372,3 +373,2 @@
updateContractsCmd.Flags().StringP(endpointFlag, endpointFlagShort, "", endpointFlagDesc)
updateContractsCmd.Flags().String(contractsInitFlag, "", "Path to archive with compiled FrostFS contracts")
_ = updateContractsCmd.MarkFlagRequired(contractsInitFlag)
updateContractsCmd.Flags().String(contractsInitFlag, "", "Path to archive with compiled FrostFS contracts (the default is to fetch the latest release from the official repository)")
acid-ant commented 2024-02-05 06:53:05 +00:00
Member
Copy link

How about to mark flags contracts and contracts-url as mutually exclusive?

updateContractsCmd.MarkFlagsMutuallyExclusive(contractsInitFlag, contractsURLFlag)
How about to mark flags `contracts` and `contracts-url` as mutually exclusive? ``` updateContractsCmd.MarkFlagsMutuallyExclusive(contractsInitFlag, contractsURLFlag) ```
okonstantinova commented 2024-02-06 22:55:08 +00:00
Author
Contributor
Copy link

Thanks, done.

Thanks, done.
acid-ant marked this conversation as resolved
aarifullin reviewed 2024-02-05 08:50:53 +00:00
cmd/frostfs-adm/internal/modules/morph/download.go Outdated
@ -0,0 +14,4 @@
func downloadContracts(cmd *cobra.Command, url string) (io.ReadCloser, error) {
cmd.Printf("Downloading contracts archive from '%s'\n", url)
resp, err := http.Get(url)
aarifullin commented 2024-02-05 08:50:53 +00:00
Member
Copy link

Using bare http.Get is fine if you use it within a core of well-designed client.
But in this case:

  1. We cannot cancel request performing, if cmd.Context() is cancelled. It means the command may hang up until Get is finished and we, as users, are obliged to wait for its finishing
  2. We do not attempt to retry request, if url is unvailable for some reason
  3. We do not configure timeouts

So, I would like to suggest to import resty client. Because we have not been using it, you need to go get it

go get github.com/go-resty/resty/v2

Then

import (
   resty "github.com/go-resty/resty/v2"
)

/*...*/
        const defaultRetryCount = 3 // can use just as a constant, no flag introducing is needed

        client := resty.New()
	    client.SetRetryCount(defaultRetryCount).R().SetContext(cmd.Context()).Get(url)
        // also it needs to set timeout etc.
/*...*/
Using _bare_ `http.Get` is fine if you use it within a core of well-designed client. But in this case: 1. We cannot cancel request performing, if `cmd.Context()` is cancelled. It means the command may _hang up_ until `Get` is finished and we, as users, are obliged to wait for its finishing 2. We do not attempt to retry request, if `url` is unvailable for some reason 3. We do not configure timeouts So, I would like to suggest to import resty client. Because we have not been using it, you need to `go get` it ``` go get github.com/go-resty/resty/v2 ``` Then ```go import ( resty "github.com/go-resty/resty/v2" ) /*...*/ const defaultRetryCount = 3 // can use just as a constant, no flag introducing is needed client := resty.New() client.SetRetryCount(defaultRetryCount).R().SetContext(cmd.Context()).Get(url) // also it needs to set timeout etc. /*...*/ ```
fyrchik commented 2024-02-05 09:45:02 +00:00
Owner
Copy link

This is quite a simple task, could we avoid importing a whole new dependency for the sake of rarely to be used command?
It would be nice if we could use just the stdlib there.

This is quite a simple task, could we avoid importing a whole new dependency for the sake of rarely to be used command? It would be nice if we could use just the stdlib there.
👍 1
aarifullin commented 2024-02-05 10:42:01 +00:00
Member
Copy link

Alright, I don't mind

Alright, I don't mind
fyrchik commented 2024-02-05 14:24:39 +00:00
Owner
Copy link

Well, we still could try using context: an example https://gist.github.com/superbrothers/dae0030c151d1f3c24311df77405169b

Well, we still could try using context: an example https://gist.github.com/superbrothers/dae0030c151d1f3c24311df77405169b
👍 1
okonstantinova commented 2024-02-06 23:01:07 +00:00
Author
Contributor
Copy link

I have changed the implementation, now it sends an HTTP request with connect timeout and uses a context with timeout. Thank you!

I have changed the implementation, now it sends an HTTP request with connect timeout and uses a context with timeout. Thank you!
fyrchik marked this conversation as resolved
okonstantinova force-pushed feature/466-download_contracts_from_gitea from 9cffb1918d to 82dfcf9a3e 2024-02-06 22:44:54 +00:00 Compare
okonstantinova force-pushed feature/466-download_contracts_from_gitea from 82dfcf9a3e to 9736f8f696 2024-02-06 22:51:14 +00:00 Compare
okonstantinova force-pushed feature/466-download_contracts_from_gitea from 9736f8f696 to 1034e30114 2024-02-06 23:02:31 +00:00 Compare
okonstantinova changed title from adm: Allow to download contracts from Gitea to WIP: adm: Allow to download contracts from Gitea 2024-02-06 23:07:48 +00:00
okonstantinova changed title from WIP: adm: Allow to download contracts from Gitea to adm: Allow to download contracts from Gitea 2024-02-06 23:07:58 +00:00
acid-ant requested changes 2024-02-07 06:48:18 +00:00
cmd/frostfs-adm/internal/modules/morph/initialize.go Outdated
@ -152,6 +153,11 @@ func newInitializeContext(cmd *cobra.Command, v *viper.Viper) (*initializeContex
return nil, err
}
ctrURL, err := getContractsURL(cmd, needContracts)
acid-ant commented 2024-02-07 06:42:00 +00:00
Member
Copy link

How about this:

	var ctrURL string
	if needContracts {
		ctrURL, _ = cmd.Flags().GetString(contractsURLFlag)
	}

It is ok to skip check for the error here, the function getContractsURL is redundant at all.

How about this: ``` var ctrURL string if needContracts { ctrURL, _ = cmd.Flags().GetString(contractsURLFlag) } ``` It is ok to skip check for the error here, the function `getContractsURL` is redundant at all.
okonstantinova commented 2024-02-07 19:41:35 +00:00
Author
Contributor
Copy link

Done as recommended, thanks

Done as recommended, thanks
acid-ant marked this conversation as resolved
cmd/frostfs-adm/internal/modules/morph/root.go Outdated
@ -372,3 +373,2 @@
updateContractsCmd.Flags().StringP(endpointFlag, endpointFlagShort, "", endpointFlagDesc)
updateContractsCmd.Flags().String(contractsInitFlag, "", "Path to archive with compiled FrostFS contracts")
_ = updateContractsCmd.MarkFlagRequired(contractsInitFlag)
updateContractsCmd.Flags().String(contractsInitFlag, "", "Path to archive with compiled FrostFS contracts (the default is to fetch the latest release from the official repository)")
acid-ant commented 2024-02-07 06:45:30 +00:00
Member
Copy link

Please move description to the constant contractsInitFlagDesc and reuse it for initCmd.

Please move description to the constant `contractsInitFlagDesc` and reuse it for `initCmd`.
okonstantinova commented 2024-02-07 19:42:09 +00:00
Author
Contributor
Copy link

Thanks, done

Thanks, done
acid-ant marked this conversation as resolved
cmd/frostfs-adm/internal/modules/morph/root.go Outdated
@ -373,2 +374,2 @@
updateContractsCmd.Flags().String(contractsInitFlag, "", "Path to archive with compiled FrostFS contracts")
_ = updateContractsCmd.MarkFlagRequired(contractsInitFlag)
updateContractsCmd.Flags().String(contractsInitFlag, "", "Path to archive with compiled FrostFS contracts (the default is to fetch the latest release from the official repository)")
updateContractsCmd.Flags().String(contractsURLFlag, "", "URL to archive with compiled FrostFS contracts")
acid-ant commented 2024-02-07 06:46:47 +00:00
Member
Copy link

Please move description to the constant contractsURLFlagDesc and reuse it for initCmd.

Please move description to the constant `contractsURLFlagDesc` and reuse it for `initCmd`.
okonstantinova commented 2024-02-07 19:42:52 +00:00
Author
Contributor
Copy link

Thanks, done

Thanks, done
acid-ant marked this conversation as resolved
acid-ant commented 2024-02-07 06:50:52 +00:00
Member
Copy link

Please squash commits.

Please squash commits.
aarifullin reviewed 2024-02-07 09:45:28 +00:00
cmd/frostfs-adm/internal/modules/morph/download.go Outdated
@ -0,0 +26,4 @@
},
}
ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
aarifullin commented 2024-02-07 09:45:28 +00:00
Member
Copy link

Good :) But instead context.Background() we need to use cmd.Context()

Good :) But instead `context.Background()` we need to use `cmd.Context()`
okonstantinova commented 2024-02-07 19:46:36 +00:00
Author
Contributor
Copy link

I suspected that something like this would be needed, but was not sure. Thank you for help!

I suspected that something like this would be needed, but was not sure. Thank you for help!
👍 1
aarifullin marked this conversation as resolved
okonstantinova force-pushed feature/466-download_contracts_from_gitea from 1034e30114 to bb45fdf01a 2024-02-07 19:39:18 +00:00 Compare
acid-ant approved these changes 2024-02-08 06:35:36 +00:00
aarifullin approved these changes 2024-02-08 08:15:49 +00:00
dstepanov-yadro approved these changes 2024-02-08 08:49:15 +00:00
achuprov approved these changes 2024-02-08 13:24:41 +00:00
okonstantinova force-pushed feature/466-download_contracts_from_gitea from bb45fdf01a to 80b581d499 2024-02-08 21:07:58 +00:00 Compare
fyrchik merged commit 80b581d499 into master 2024-02-09 07:50:35 +00:00
Sign in to join this conversation.
Reviewers
No reviewers
acid-ant
aarifullin
dstepanov-yadro
achuprov
Labels
Clear labels   
P0

Highest

  
P1

High

  
P2

Medium

  
P3

Low

  
badger

Badger DB related issues

  
frostfs-adm

   
frostfs-cli

   
frostfs-ir

   
frostfs-lens

   
frostfs-node

   
good first issue

Good for newcomers

  
triage
  
Infrastructure

Infrastructure, Automation, CI/CD or DevOps related stuff

  
blocked

The issue or PR is blocked by something

  
bug

Something is not working

  
config

Configuration format update or breaking change

  
discussion

Talking about something in order to reach a decision or to exchange ideas

  
documentation

Documentation related

  
duplicate

This issue or pull request already exists

  
enhancement

New feature

  
go

Language features adoption

  
help wanted

Extra attention is needed

  
internal

   
invalid

Something is wrong

  
kludge

This is a kludge and we know it

  
observability

Related to metrics, tracing and logs

  
perfomance

Perfomance related

  
question

More information is needed

  
refactoring

Refactoring

  
wontfix

This won't be fixed

No labels
P0
P1
P2
P3
badger
frostfs-adm
frostfs-cli
frostfs-ir
frostfs-lens
frostfs-node
good first issue
triage
Infrastructure
blocked
bug
config
discussion
documentation
duplicate
enhancement
go
help wanted
internal
invalid
kludge
observability
perfomance
question
refactoring
wontfix
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
6 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: TrueCloudLab/frostfs-node#958
No description provided.
Delete branch "okonstantinova/frostfs-node:feature/466-download_contracts_from_gitea"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?