feature/integration_with_frostfs #1

Merged
alexvanin merged 1 commit from :feature/integration_with_frostfs into tcl/master 2024-09-04 19:51:22 +00:00
Member

Signed-off-by: Roman Loginov r.loginov@yadro.com

Signed-off-by: Roman Loginov r.loginov@yadro.com
r.loginov force-pushed feature/integration_with_frostfs from 82139145ef to 972e99bf1e 2023-08-30 06:45:51 +00:00 Compare
r.loginov requested review from dkirillov 2023-08-30 07:23:51 +00:00
r.loginov force-pushed feature/integration_with_frostfs from 972e99bf1e to 472e2dfac6 2023-08-31 06:45:55 +00:00 Compare
dkirillov reviewed 2023-09-01 13:13:00 +00:00
go.mod Outdated
@ -32,3 +36,4 @@
gopkg.in/yaml.v2 v2.4.0
)
require (
Member

Let's keep just two require section

Let's keep just two `require` section
Member
We have to register `frostfs` driver in https://git.frostfs.info/r.loginov/distribution/src/branch/feature/integration_with_frostfs/cmd/registry/main.go
Member

Did you try to upload image to distribution using frostfs as driver?

I got the following error

ERRO[0048] response completed with error                 environment=development err.code=unknown err.detail="frostfs: couldn't parse split id as upload uuid '': invalid UUID length: 0" err.message="unknown error" go.version=go1.20.4 http.request.host="localhost:5000" http.request.id=c9e7d318-a4c0-4041-8bf8-6633f69efaa6 http.request.method=POST http.request.remoteaddr="[::1]:59248" http.request.uri=/v2/truecloudlab/frostfs-storage/blobs/uploads/ http.request.useragent="docker/20.10.5+dfsg1 go/go1.15.15 git-commit/363e9a8 kernel/6.4.5-x64v1-xanmod1 os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.5+dfsg1 \\(linux\\))" http.response.contenttype=application/json http.response.duration=67.028766ms http.response.status=500 http.response.written=106 instance.id=539b52fc-fe49-4238-8013-159d9da79d46 service=registry vars.name=truecloudlab/frostfs-storage version=v3.0.0+unknown

I ran distribution with the following config

version: 0.1
log:
  level: debug
  fields:
    service: registry
    environment: development
  hooks:
    - type: mail
      disabled: true
      levels:
        - panic
      options:
        smtp:
          addr: mail.example.com:25
          username: mailuser
          password: password
          insecure: true
        from: sender@example.com
        to:
          - errors@example.com
storage:
  delete:
    enabled: true
  cache:
    blobdescriptor: inmemory
  maintenance:
    uploadpurging:
      enabled: false

  frostfs:
    wallet:
      path: /home/denis/github/tcl/frostfs-dev-env/wallets/wallet.json
      password: ""
    peers:
      0:
        address: s01.frostfs.devenv:8080
        weight: 1
        priority: 1
      1:
        address: s02.frostfs.devenv:8080
        weight: 1
        priority: 1
      2:
        address: s03.frostfs.devenv:8080
        weight: 1
        priority: 1
      3:
        address: s04.frostfs.devenv:8080
        weight: 1
        priority: 1
    # container can be nicename (rpc_endpoint is required)
    container: 6horME4kUEUnN5bweqmLSU5YdYfkUw3kb24Y3FfDc76H
    # the following params are optional
    session_expiration_duration: 1000 # in blocks
    connection_timeout: 5s
    request_timeout: 5s
    rebalance_interval: 30s
    rpc_endpoint: http://morph_chain.frostfs.devenv:30333
http:
  addr: :5000
  debug:
    addr: :5001
    prometheus:
      enabled: true
      path: /metrics
  headers:
    X-Content-Type-Options: [ nosniff ]
redis:
  addr: localhost:6379
  pool:
    maxidle: 16
    maxactive: 64
    idletimeout: 300s
  dialtimeout: 10ms
  readtimeout: 10ms
  writetimeout: 10ms
notifications:
  events:
    includereferences: true
  endpoints:
    - name: local-5003
      url: http://localhost:5003/callback
      headers:
        Authorization: [ Bearer <an example token> ]
      timeout: 1s
      threshold: 10
      backoff: 1s
      disabled: true
    - name: local-8083
      url: http://localhost:8083/callback
      timeout: 1s
      threshold: 10
      backoff: 1s
      disabled: true
health:
  storagedriver:
    enabled: true
    interval: 30s
    threshold: 3

And tried to push image

$ docker tag truecloudlab/frostfs-storage:365a7ca0 localhost:5000/truecloudlab/frostfs-storage:365a7ca0
$ docker push localhost:5000/truecloudlab/frostfs-storage:365a7ca0
Did you try to upload image to distribution using frostfs as driver? I got the following error ``` ERRO[0048] response completed with error environment=development err.code=unknown err.detail="frostfs: couldn't parse split id as upload uuid '': invalid UUID length: 0" err.message="unknown error" go.version=go1.20.4 http.request.host="localhost:5000" http.request.id=c9e7d318-a4c0-4041-8bf8-6633f69efaa6 http.request.method=POST http.request.remoteaddr="[::1]:59248" http.request.uri=/v2/truecloudlab/frostfs-storage/blobs/uploads/ http.request.useragent="docker/20.10.5+dfsg1 go/go1.15.15 git-commit/363e9a8 kernel/6.4.5-x64v1-xanmod1 os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.5+dfsg1 \\(linux\\))" http.response.contenttype=application/json http.response.duration=67.028766ms http.response.status=500 http.response.written=106 instance.id=539b52fc-fe49-4238-8013-159d9da79d46 service=registry vars.name=truecloudlab/frostfs-storage version=v3.0.0+unknown ``` I ran distribution with the following config ```yaml version: 0.1 log: level: debug fields: service: registry environment: development hooks: - type: mail disabled: true levels: - panic options: smtp: addr: mail.example.com:25 username: mailuser password: password insecure: true from: sender@example.com to: - errors@example.com storage: delete: enabled: true cache: blobdescriptor: inmemory maintenance: uploadpurging: enabled: false frostfs: wallet: path: /home/denis/github/tcl/frostfs-dev-env/wallets/wallet.json password: "" peers: 0: address: s01.frostfs.devenv:8080 weight: 1 priority: 1 1: address: s02.frostfs.devenv:8080 weight: 1 priority: 1 2: address: s03.frostfs.devenv:8080 weight: 1 priority: 1 3: address: s04.frostfs.devenv:8080 weight: 1 priority: 1 # container can be nicename (rpc_endpoint is required) container: 6horME4kUEUnN5bweqmLSU5YdYfkUw3kb24Y3FfDc76H # the following params are optional session_expiration_duration: 1000 # in blocks connection_timeout: 5s request_timeout: 5s rebalance_interval: 30s rpc_endpoint: http://morph_chain.frostfs.devenv:30333 http: addr: :5000 debug: addr: :5001 prometheus: enabled: true path: /metrics headers: X-Content-Type-Options: [ nosniff ] redis: addr: localhost:6379 pool: maxidle: 16 maxactive: 64 idletimeout: 300s dialtimeout: 10ms readtimeout: 10ms writetimeout: 10ms notifications: events: includereferences: true endpoints: - name: local-5003 url: http://localhost:5003/callback headers: Authorization: [ Bearer <an example token> ] timeout: 1s threshold: 10 backoff: 1s disabled: true - name: local-8083 url: http://localhost:8083/callback timeout: 1s threshold: 10 backoff: 1s disabled: true health: storagedriver: enabled: true interval: 30s threshold: 3 ``` And tried to push image ```shell $ docker tag truecloudlab/frostfs-storage:365a7ca0 localhost:5000/truecloudlab/frostfs-storage:365a7ca0 $ docker push localhost:5000/truecloudlab/frostfs-storage:365a7ca0 ```
Member

I got the following error

Oh, I see. It seems we have to adopt the following fix for linkedblobstore.go 1ba38226ab (diff-1c30d8b7d3e65b3e66719176279abd0dd0bbb8402ac0482fc3cc6523e65520c4)

> I got the following error Oh, I see. It seems we have to adopt the following fix for `linkedblobstore.go` https://github.com/nspcc-dev/distribution/commit/1ba38226abd948e47e237ed13b49451d1990819b#diff-1c30d8b7d3e65b3e66719176279abd0dd0bbb8402ac0482fc3cc6523e65520c4
dkirillov reviewed 2023-09-01 13:32:54 +00:00
@ -0,0 +1,111 @@
package transformer
Member

It seems we can drop this file and use types from here

It seems we can drop this file and use types from [here](https://git.frostfs.info/TrueCloudLab/frostfs-sdk-go/src/commit/84e7e69f98ac8c0d6ef09fb90bb60ba035aade17/object/transformer/types.go#L13)
dkirillov reviewed 2023-09-01 13:47:19 +00:00
@ -0,0 +1,461 @@
package frostfs
Member

Probably, we could try to rewrite this to be more like 84e7e69f98/pool/object_put_pool_transformer.go or do this in follow up issues

Probably, we could try to rewrite this to be more like https://git.frostfs.info/TrueCloudLab/frostfs-sdk-go/src/commit/84e7e69f98ac8c0d6ef09fb90bb60ba035aade17/pool/object_put_pool_transformer.go or do this in follow up issues
dkirillov reviewed 2023-09-01 13:47:52 +00:00
dkirillov left a comment
Member

Overall looks good.

As follow up we could consider using tree service

Overall looks good. As follow up we could consider using tree service
r.loginov force-pushed feature/integration_with_frostfs from 472e2dfac6 to 9c08a6ecdf 2023-09-04 12:39:15 +00:00 Compare
r.loginov force-pushed feature/integration_with_frostfs from 9c08a6ecdf to 5c26d0c9c1 2023-09-05 06:41:12 +00:00 Compare
r.loginov changed title from WIP: feature/integration_with_frostfs to feature/integration_with_frostfs 2023-09-08 07:17:21 +00:00
dkirillov approved these changes 2023-09-20 13:14:09 +00:00
Member

Maybe we should create new branch and merge to that new branch rather than upstream main? @alexvanin

Maybe we should create new branch and merge to that new branch rather than upstream main? @alexvanin
Owner

@dkirillov Sure, like we did for certificates and lego.

@dkirillov Sure, like we did for [certificates](https://git.frostfs.info/TrueCloudLab/certificates) and [lego](https://git.frostfs.info/TrueCloudLab/lego).
alexvanin changed title from feature/integration_with_frostfs to feature/integration_with_frostfs 2023-09-20 14:40:40 +00:00
alexvanin changed title from feature/integration_with_frostfs to feature/integration_with_frostfs 2023-09-20 14:41:05 +00:00
alexvanin changed title from feature/integration_with_frostfs to feature/integration_with_frostfs 2023-09-20 14:42:06 +00:00
alexvanin changed title from feature/integration_with_frostfs to feature/integration_with_frostfs 2023-09-20 14:42:46 +00:00
alexvanin changed title from feature/integration_with_frostfs to feature/integration_with_frostfs 2023-09-20 14:44:18 +00:00
alexvanin changed target branch from main to tcl/master 2023-09-20 14:45:24 +00:00
r.loginov changed title from feature/integration_with_frostfs to feature/integration_with_frostfs 2023-09-20 14:47:22 +00:00
r.loginov changed title from feature/integration_with_frostfs to feature/integration_with_frostfs 2023-09-20 14:47:36 +00:00
alexvanin merged commit 5c26d0c9c1 into tcl/master 2023-09-20 14:48:53 +00:00
alexvanin deleted branch feature/integration_with_frostfs 2023-09-20 14:48:53 +00:00
alexvanin referenced this pull request from a commit 2023-09-20 14:48:55 +00:00
alexvanin referenced this pull request from a commit 2023-09-20 14:48:55 +00:00
r.loginov referenced this pull request from a commit 2024-04-03 13:36:14 +00:00
r.loginov referenced this pull request from a commit 2024-04-03 13:36:14 +00:00
Sign in to join this conversation.
No description provided.