TrueCloudLab/frostfs-node
20
1
Fork 27
Code Issues 100 Pull requests 4 Projects Releases 1 Activity Actions

Source-based routing support #1422

Merged
dstepanov-yadro merged 11 commits from dstepanov-yadro/frostfs-node:feat/multinet into master 2024-10-17 13:15:09 +00:00
Conversation 2 Commits 11 Files changed 32 +724 -105
dstepanov-yadro commented 2024-10-08 12:25:28 +00:00
Member
Copy link

Added source based routing (aka multinet) support for grpc clients, tree service and morph.

before:

# ss -p -t -a | grep frostfs-node | grep -wv "192.168.200.9:http-alt" | grep -wv "192.168.201.9:http-alt" | grep http-alt                                                                  
ESTAB     0      0        192.168.201.9:57488    192.168.200.16:http-alt users:(("frostfs-node",pid=57483,fd=275))                                                                        
ESTAB     0      0        192.168.201.9:43950     192.168.200.8:http-alt users:(("frostfs-node",pid=57483,fd=276))                                         
ESTAB     0      0        192.168.201.9:57462    192.168.200.16:http-alt users:(("frostfs-node",pid=57483,fd=273))                                         
ESTAB     0      0        192.168.201.9:59232    192.168.200.11:http-alt users:(("frostfs-node",pid=57483,fd=278))                                         
ESTAB     0      0        192.168.201.9:59228    192.168.200.11:http-alt users:(("frostfs-node",pid=57483,fd=272))                                         
ESTAB     0      0        192.168.201.9:57510    192.168.200.16:http-alt users:(("frostfs-node",pid=57483,fd=279))                                         
ESTAB     0      0        192.168.201.9:57446    192.168.200.16:http-alt users:(("frostfs-node",pid=57483,fd=271))                                         
ESTAB     0      0        192.168.201.9:57424    192.168.200.16:http-alt users:(("frostfs-node",pid=57483,fd=264))                                         
ESTAB     0      0        192.168.201.9:43916     192.168.200.8:http-alt users:(("frostfs-node",pid=57483,fd=266))                                         
ESTAB     968    0        192.168.201.9:38462     192.168.200.8:http-alt users:(("frostfs-node",pid=57483,fd=62))                                                                          
ESTAB     0      0        192.168.201.9:57526    192.168.200.16:http-alt users:(("frostfs-node",pid=57483,fd=280))                                         
ESTAB     0      0        192.168.201.9:59248    192.168.200.11:http-alt users:(("frostfs-node",pid=57483,fd=283))                                                                             
ESTAB     0      0        192.168.201.9:43918     192.168.200.8:http-alt users:(("frostfs-node",pid=57483,fd=267))                                         
ESTAB     0      0        192.168.201.9:57478    192.168.200.16:http-alt users:(("frostfs-node",pid=57483,fd=274))                                         
ESTAB     0      0        192.168.201.9:57432    192.168.200.16:http-alt users:(("frostfs-node",pid=57483,fd=265))                                         
ESTAB     0      0        192.168.201.9:57440    192.168.200.16:http-alt users:(("frostfs-node",pid=57483,fd=270))                                         
ESTAB     0      0        192.168.201.9:59268    192.168.200.11:http-alt users:(("frostfs-node",pid=57483,fd=286))                                         
ESTAB     0      0        192.168.201.9:59256    192.168.200.11:http-alt users:(("frostfs-node",pid=57483,fd=285))                                         
ESTAB     0      0        192.168.201.9:57540    192.168.200.16:http-alt users:(("frostfs-node",pid=57483,fd=281))                                         
ESTAB     0      0        192.168.201.9:57416    192.168.200.16:http-alt users:(("frostfs-node",pid=57483,fd=263))                                         
ESTAB     0      0        192.168.201.9:57552    192.168.200.16:http-alt users:(("frostfs-node",pid=57483,fd=282))                                         
ESTAB     0      0        192.168.201.9:43944     192.168.200.8:http-alt users:(("frostfs-node",pid=57483,fd=269))                                                                              
ESTAB     0      0        192.168.201.9:57496    192.168.200.16:http-alt users:(("frostfs-node",pid=57483,fd=277))                                                                             
ESTAB     30     0        192.168.201.9:35128    192.168.200.11:http-alt users:(("frostfs-node",pid=57483,fd=63))                                                                               
ESTAB     0      0        192.168.201.9:59254    192.168.200.11:http-alt users:(("frostfs-node",pid=57483,fd=284))                                         
ESTAB     0      0        192.168.201.9:43928     192.168.200.8:http-alt users:(("frostfs-node",pid=57483,fd=268))                                         
ESTAB     0      982      192.168.201.9:42896    192.168.200.16:http-alt users:(("frostfs-node",pid=57483,fd=61))

after:

ESTAB     0      0        192.168.201.9:57741    192.168.200.11:http-alt users:(("frostfs-node",pid=76061,fd=82))                                          
ESTAB     0      0        192.168.200.9:53767     192.168.200.8:http-alt users:(("frostfs-node",pid=76061,fd=71))                                                                             
ESTAB     0      0        192.168.200.9:56567    192.168.200.16:http-alt users:(("frostfs-node",pid=76061,fd=85))                                          
ESTAB     0      0        192.168.200.9:59993    192.168.200.11:http-alt users:(("frostfs-node",pid=76061,fd=77))                                          
ESTAB     1889   0        192.168.201.9:47247     192.168.200.8:http-alt users:(("frostfs-node",pid=76061,fd=62))                                          
ESTAB     0      0        192.168.201.9:44479    192.168.200.16:http-alt users:(("frostfs-node",pid=76061,fd=84))                                          
ESTAB     0      0        192.168.201.9:53915    192.168.200.11:http-alt users:(("frostfs-node",pid=76061,fd=86))                                          
ESTAB     0      0        192.168.201.9:54665     192.168.200.8:http-alt users:(("frostfs-node",pid=76061,fd=90))                                                                           
ESTAB     0      0        192.168.200.9:44509     192.168.200.8:http-alt users:(("frostfs-node",pid=76061,fd=73))                                          
ESTAB     0      0        192.168.200.9:35041     192.168.200.8:http-alt users:(("frostfs-node",pid=76061,fd=89))                                          
ESTAB     0      0        192.168.201.9:35953     192.168.200.8:http-alt users:(("frostfs-node",pid=76061,fd=70))                                          
ESTAB     0      0        192.168.201.9:44189     192.168.200.8:http-alt users:(("frostfs-node",pid=76061,fd=88))                                          
ESTAB     0      0        192.168.200.9:41843     192.168.200.8:http-alt users:(("frostfs-node",pid=76061,fd=87))                                          
ESTAB     0      0        192.168.200.9:37175     192.168.200.8:http-alt users:(("frostfs-node",pid=76061,fd=81))                                          
ESTAB     0      0        192.168.201.9:52001     192.168.200.8:http-alt users:(("frostfs-node",pid=76061,fd=80))                                                                            
ESTAB     0      0        192.168.200.9:54465    192.168.200.11:http-alt users:(("frostfs-node",pid=76061,fd=69))                                          
ESTAB     0      0        192.168.201.9:39721     192.168.200.8:http-alt users:(("frostfs-node",pid=76061,fd=78))                                          
ESTAB     0      0        192.168.201.9:44589    192.168.200.11:http-alt users:(("frostfs-node",pid=76061,fd=76))                                          
ESTAB     0      0        192.168.200.9:33409     192.168.200.8:http-alt users:(("frostfs-node",pid=76061,fd=79))                                          
ESTAB     0      0        192.168.201.9:34509     192.168.200.8:http-alt users:(("frostfs-node",pid=76061,fd=72))                                          
ESTAB     0      446      192.168.200.9:42251    192.168.200.11:http-alt users:(("frostfs-node",pid=76061,fd=61))                                          
ESTAB     0      0        192.168.201.9:35365    192.168.200.11:http-alt users:(("frostfs-node",pid=76061,fd=68))                                          
ESTAB     0      0        192.168.201.9:48325     192.168.200.8:http-alt users:(("frostfs-node",pid=76061,fd=74))                                          
ESTAB     0      0        192.168.200.9:35385    192.168.200.11:http-alt users:(("frostfs-node",pid=76061,fd=75))                                                                     
ESTAB     0      0        192.168.200.9:56843     192.168.200.8:http-alt users:(("frostfs-node",pid=76061,fd=91))                                          
ESTAB     0      0        192.168.201.9:37903    192.168.200.16:http-alt users:(("frostfs-node",pid=76061,fd=53))                                                             
ESTAB     0      0        192.168.200.9:49139    192.168.200.16:http-alt users:(("frostfs-node",pid=76061,fd=67))                                          
ESTAB     0      0        192.168.200.9:45113     192.168.200.8:http-alt users:(("frostfs-node",pid=76061,fd=83))

Perf tests:
master:

     data_received..............: 0 B     0 B/s
     data_sent..................: 31 GB   34 MB/s
     frostfs_obj_put_bytes......: 31 GB   34 MB/s
     frostfs_obj_put_duration...: avg=170.65ms min=13.45ms  med=158.55ms max=2.11s p(90)=266.12ms p(95)=317.88ms
     frostfs_obj_put_success....: 3011045 3344.830896/s
     iteration_duration.........: avg=179.32ms min=288.72µs med=167.44ms max=2.12s p(90)=275.16ms p(95)=327.02ms
     iterations.................: 3011045 3344.830896/s
     vus........................: 168     min=168       max=600

     data_received..............: 28 GB   31 MB/s
     data_sent..................: 0 B     0 B/s
     frostfs_obj_get_bytes......: 28 GB   32 MB/s
     frostfs_obj_get_duration...: avg=194.8ms  min=2.1ms    med=88.65ms max=19.75s p(90)=440.91ms p(95)=686.98ms
     frostfs_obj_get_success....: 2768294 3074.239976/s
     iteration_duration.........: avg=195.05ms min=437.03µs med=88.91ms max=19.75s p(90)=441.17ms p(95)=687.23ms
     iterations.................: 2768294 3074.239976/s
     vus........................: 1       min=1         max=600

branch:

     data_received..............: 0 B     0 B/s
     data_sent..................: 31 GB   34 MB/s
     frostfs_obj_put_bytes......: 31 GB   34 MB/s
     frostfs_obj_put_duration...: avg=170.69ms min=12.65ms  med=159.22ms max=2.9s  p(90)=271.4ms  p(95)=319.77ms
     frostfs_obj_put_success....: 3010469 3344.340253/s
     iteration_duration.........: avg=179.35ms min=305.34µs med=168.11ms max=2.91s p(90)=280.39ms p(95)=328.69ms
     iterations.................: 3010469 3344.340253/s
     vus........................: 600     min=600       max=600

     data_received..............: 29 GB   33 MB/s
     data_sent..................: 0 B     0 B/s
     frostfs_obj_get_bytes......: 29 GB   33 MB/s
     frostfs_obj_get_duration...: avg=187.78ms min=1.86ms   med=77.43ms max=16.83s p(90)=443.79ms p(95)=672.85ms
     frostfs_obj_get_success....: 2871704 3188.083029/s
     iteration_duration.........: avg=188.03ms min=226.28µs med=77.68ms max=16.83s p(90)=444.05ms p(95)=673.14ms
     iterations.................: 2871704 3188.083029/s
     vus........................: 6       min=6         max=600
Added source based routing (aka multinet) support for grpc clients, tree service and morph. before: ``` # ss -p -t -a | grep frostfs-node | grep -wv "192.168.200.9:http-alt" | grep -wv "192.168.201.9:http-alt" | grep http-alt ESTAB 0 0 192.168.201.9:57488 192.168.200.16:http-alt users:(("frostfs-node",pid=57483,fd=275)) ESTAB 0 0 192.168.201.9:43950 192.168.200.8:http-alt users:(("frostfs-node",pid=57483,fd=276)) ESTAB 0 0 192.168.201.9:57462 192.168.200.16:http-alt users:(("frostfs-node",pid=57483,fd=273)) ESTAB 0 0 192.168.201.9:59232 192.168.200.11:http-alt users:(("frostfs-node",pid=57483,fd=278)) ESTAB 0 0 192.168.201.9:59228 192.168.200.11:http-alt users:(("frostfs-node",pid=57483,fd=272)) ESTAB 0 0 192.168.201.9:57510 192.168.200.16:http-alt users:(("frostfs-node",pid=57483,fd=279)) ESTAB 0 0 192.168.201.9:57446 192.168.200.16:http-alt users:(("frostfs-node",pid=57483,fd=271)) ESTAB 0 0 192.168.201.9:57424 192.168.200.16:http-alt users:(("frostfs-node",pid=57483,fd=264)) ESTAB 0 0 192.168.201.9:43916 192.168.200.8:http-alt users:(("frostfs-node",pid=57483,fd=266)) ESTAB 968 0 192.168.201.9:38462 192.168.200.8:http-alt users:(("frostfs-node",pid=57483,fd=62)) ESTAB 0 0 192.168.201.9:57526 192.168.200.16:http-alt users:(("frostfs-node",pid=57483,fd=280)) ESTAB 0 0 192.168.201.9:59248 192.168.200.11:http-alt users:(("frostfs-node",pid=57483,fd=283)) ESTAB 0 0 192.168.201.9:43918 192.168.200.8:http-alt users:(("frostfs-node",pid=57483,fd=267)) ESTAB 0 0 192.168.201.9:57478 192.168.200.16:http-alt users:(("frostfs-node",pid=57483,fd=274)) ESTAB 0 0 192.168.201.9:57432 192.168.200.16:http-alt users:(("frostfs-node",pid=57483,fd=265)) ESTAB 0 0 192.168.201.9:57440 192.168.200.16:http-alt users:(("frostfs-node",pid=57483,fd=270)) ESTAB 0 0 192.168.201.9:59268 192.168.200.11:http-alt users:(("frostfs-node",pid=57483,fd=286)) ESTAB 0 0 192.168.201.9:59256 192.168.200.11:http-alt users:(("frostfs-node",pid=57483,fd=285)) ESTAB 0 0 192.168.201.9:57540 192.168.200.16:http-alt users:(("frostfs-node",pid=57483,fd=281)) ESTAB 0 0 192.168.201.9:57416 192.168.200.16:http-alt users:(("frostfs-node",pid=57483,fd=263)) ESTAB 0 0 192.168.201.9:57552 192.168.200.16:http-alt users:(("frostfs-node",pid=57483,fd=282)) ESTAB 0 0 192.168.201.9:43944 192.168.200.8:http-alt users:(("frostfs-node",pid=57483,fd=269)) ESTAB 0 0 192.168.201.9:57496 192.168.200.16:http-alt users:(("frostfs-node",pid=57483,fd=277)) ESTAB 30 0 192.168.201.9:35128 192.168.200.11:http-alt users:(("frostfs-node",pid=57483,fd=63)) ESTAB 0 0 192.168.201.9:59254 192.168.200.11:http-alt users:(("frostfs-node",pid=57483,fd=284)) ESTAB 0 0 192.168.201.9:43928 192.168.200.8:http-alt users:(("frostfs-node",pid=57483,fd=268)) ESTAB 0 982 192.168.201.9:42896 192.168.200.16:http-alt users:(("frostfs-node",pid=57483,fd=61)) ``` after: ``` ESTAB 0 0 192.168.201.9:57741 192.168.200.11:http-alt users:(("frostfs-node",pid=76061,fd=82)) ESTAB 0 0 192.168.200.9:53767 192.168.200.8:http-alt users:(("frostfs-node",pid=76061,fd=71)) ESTAB 0 0 192.168.200.9:56567 192.168.200.16:http-alt users:(("frostfs-node",pid=76061,fd=85)) ESTAB 0 0 192.168.200.9:59993 192.168.200.11:http-alt users:(("frostfs-node",pid=76061,fd=77)) ESTAB 1889 0 192.168.201.9:47247 192.168.200.8:http-alt users:(("frostfs-node",pid=76061,fd=62)) ESTAB 0 0 192.168.201.9:44479 192.168.200.16:http-alt users:(("frostfs-node",pid=76061,fd=84)) ESTAB 0 0 192.168.201.9:53915 192.168.200.11:http-alt users:(("frostfs-node",pid=76061,fd=86)) ESTAB 0 0 192.168.201.9:54665 192.168.200.8:http-alt users:(("frostfs-node",pid=76061,fd=90)) ESTAB 0 0 192.168.200.9:44509 192.168.200.8:http-alt users:(("frostfs-node",pid=76061,fd=73)) ESTAB 0 0 192.168.200.9:35041 192.168.200.8:http-alt users:(("frostfs-node",pid=76061,fd=89)) ESTAB 0 0 192.168.201.9:35953 192.168.200.8:http-alt users:(("frostfs-node",pid=76061,fd=70)) ESTAB 0 0 192.168.201.9:44189 192.168.200.8:http-alt users:(("frostfs-node",pid=76061,fd=88)) ESTAB 0 0 192.168.200.9:41843 192.168.200.8:http-alt users:(("frostfs-node",pid=76061,fd=87)) ESTAB 0 0 192.168.200.9:37175 192.168.200.8:http-alt users:(("frostfs-node",pid=76061,fd=81)) ESTAB 0 0 192.168.201.9:52001 192.168.200.8:http-alt users:(("frostfs-node",pid=76061,fd=80)) ESTAB 0 0 192.168.200.9:54465 192.168.200.11:http-alt users:(("frostfs-node",pid=76061,fd=69)) ESTAB 0 0 192.168.201.9:39721 192.168.200.8:http-alt users:(("frostfs-node",pid=76061,fd=78)) ESTAB 0 0 192.168.201.9:44589 192.168.200.11:http-alt users:(("frostfs-node",pid=76061,fd=76)) ESTAB 0 0 192.168.200.9:33409 192.168.200.8:http-alt users:(("frostfs-node",pid=76061,fd=79)) ESTAB 0 0 192.168.201.9:34509 192.168.200.8:http-alt users:(("frostfs-node",pid=76061,fd=72)) ESTAB 0 446 192.168.200.9:42251 192.168.200.11:http-alt users:(("frostfs-node",pid=76061,fd=61)) ESTAB 0 0 192.168.201.9:35365 192.168.200.11:http-alt users:(("frostfs-node",pid=76061,fd=68)) ESTAB 0 0 192.168.201.9:48325 192.168.200.8:http-alt users:(("frostfs-node",pid=76061,fd=74)) ESTAB 0 0 192.168.200.9:35385 192.168.200.11:http-alt users:(("frostfs-node",pid=76061,fd=75)) ESTAB 0 0 192.168.200.9:56843 192.168.200.8:http-alt users:(("frostfs-node",pid=76061,fd=91)) ESTAB 0 0 192.168.201.9:37903 192.168.200.16:http-alt users:(("frostfs-node",pid=76061,fd=53)) ESTAB 0 0 192.168.200.9:49139 192.168.200.16:http-alt users:(("frostfs-node",pid=76061,fd=67)) ESTAB 0 0 192.168.200.9:45113 192.168.200.8:http-alt users:(("frostfs-node",pid=76061,fd=83)) ``` Perf tests: master: ``` data_received..............: 0 B 0 B/s data_sent..................: 31 GB 34 MB/s frostfs_obj_put_bytes......: 31 GB 34 MB/s frostfs_obj_put_duration...: avg=170.65ms min=13.45ms med=158.55ms max=2.11s p(90)=266.12ms p(95)=317.88ms frostfs_obj_put_success....: 3011045 3344.830896/s iteration_duration.........: avg=179.32ms min=288.72µs med=167.44ms max=2.12s p(90)=275.16ms p(95)=327.02ms iterations.................: 3011045 3344.830896/s vus........................: 168 min=168 max=600 data_received..............: 28 GB 31 MB/s data_sent..................: 0 B 0 B/s frostfs_obj_get_bytes......: 28 GB 32 MB/s frostfs_obj_get_duration...: avg=194.8ms min=2.1ms med=88.65ms max=19.75s p(90)=440.91ms p(95)=686.98ms frostfs_obj_get_success....: 2768294 3074.239976/s iteration_duration.........: avg=195.05ms min=437.03µs med=88.91ms max=19.75s p(90)=441.17ms p(95)=687.23ms iterations.................: 2768294 3074.239976/s vus........................: 1 min=1 max=600 ``` branch: ``` data_received..............: 0 B 0 B/s data_sent..................: 31 GB 34 MB/s frostfs_obj_put_bytes......: 31 GB 34 MB/s frostfs_obj_put_duration...: avg=170.69ms min=12.65ms med=159.22ms max=2.9s p(90)=271.4ms p(95)=319.77ms frostfs_obj_put_success....: 3010469 3344.340253/s iteration_duration.........: avg=179.35ms min=305.34µs med=168.11ms max=2.91s p(90)=280.39ms p(95)=328.69ms iterations.................: 3010469 3344.340253/s vus........................: 600 min=600 max=600 data_received..............: 29 GB 33 MB/s data_sent..................: 0 B 0 B/s frostfs_obj_get_bytes......: 29 GB 33 MB/s frostfs_obj_get_duration...: avg=187.78ms min=1.86ms med=77.43ms max=16.83s p(90)=443.79ms p(95)=672.85ms frostfs_obj_get_success....: 2871704 3188.083029/s iteration_duration.........: avg=188.03ms min=226.28µs med=77.68ms max=16.83s p(90)=444.05ms p(95)=673.14ms iterations.................: 2871704 3188.083029/s vus........................: 6 min=6 max=600 ```
dstepanov-yadro added 1 commit 2024-10-08 12:25:28 +00:00
[#9999] config: Add multinet config
Some checks failed
DCO action / DCO (pull_request) Successful in 34s
Details
Vulncheck / Vulncheck (pull_request) Successful in 1m47s
Details
Pre-commit hooks / Pre-commit (pull_request) Failing after 2m16s
Details
Build / Build Components (pull_request) Successful in 3m12s
Details
Tests and linters / Lint (pull_request) Successful in 3m14s
Details
Tests and linters / Run gofumpt (pull_request) Successful in 3m10s
Details
Tests and linters / Staticcheck (pull_request) Successful in 3m27s
Details
Tests and linters / gopls check (pull_request) Successful in 3m58s
Details
Tests and linters / Tests with -race (pull_request) Successful in 4m52s
Details
Tests and linters / Tests (pull_request) Successful in 6m35s
Details
e205b686e8 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
dstepanov-yadro force-pushed feat/multinet from e205b686e8 to 3ddc094211 2024-10-08 12:28:29 +00:00 Compare
dstepanov-yadro force-pushed feat/multinet from 3ddc094211 to ec6611b496 2024-10-08 12:34:16 +00:00 Compare
dstepanov-yadro force-pushed feat/multinet from ec6611b496 to 8cd3a83709 2024-10-08 12:37:41 +00:00 Compare
fyrchik requested review from storage-core-committers 2024-10-08 13:10:17 +00:00
fyrchik requested review from storage-core-developers 2024-10-08 13:10:18 +00:00
dstepanov-yadro added 1 commit 2024-10-08 14:26:16 +00:00
[#1422] node: Add dialer source to config
All checks were successful
DCO action / DCO (pull_request) Successful in 1m7s
Details
Tests and linters / Run gofumpt (pull_request) Successful in 1m37s
Details
Vulncheck / Vulncheck (pull_request) Successful in 2m16s
Details
Pre-commit hooks / Pre-commit (pull_request) Successful in 2m26s
Details
Build / Build Components (pull_request) Successful in 2m42s
Details
Tests and linters / Staticcheck (pull_request) Successful in 3m8s
Details
Tests and linters / gopls check (pull_request) Successful in 3m9s
Details
Tests and linters / Lint (pull_request) Successful in 3m25s
Details
Tests and linters / Tests (pull_request) Successful in 4m26s
Details
Tests and linters / Tests with -race (pull_request) Successful in 5m35s
Details
1ebce64242 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
dstepanov-yadro force-pushed feat/multinet from 1ebce64242 to 05c39655b9 2024-10-08 14:33:29 +00:00 Compare
dstepanov-yadro force-pushed feat/multinet from 05c39655b9 to f041c072ac 2024-10-08 15:09:17 +00:00 Compare
dstepanov-yadro force-pushed feat/multinet from f041c072ac to 0d129151f2 2024-10-09 07:30:42 +00:00 Compare
dstepanov-yadro force-pushed feat/multinet from 0d129151f2 to 3989b97c92 2024-10-09 07:37:07 +00:00 Compare
dstepanov-yadro force-pushed feat/multinet from 3989b97c92 to ccb3cc0a1f 2024-10-09 08:52:50 +00:00 Compare
dstepanov-yadro force-pushed feat/multinet from ccb3cc0a1f to 1ff0e20c7b 2024-10-14 11:15:47 +00:00 Compare
dstepanov-yadro added 1 commit 2024-10-14 11:22:59 +00:00
[#1422] morph: Resolve funlen linter
Some checks failed
Tests and linters / Tests with -race (pull_request) Failing after 59s
Details
Build / Build Components (pull_request) Failing after 1m8s
Details
DCO action / DCO (pull_request) Successful in 1m9s
Details
Vulncheck / Vulncheck (pull_request) Failing after 1m15s
Details
Tests and linters / Run gofumpt (pull_request) Successful in 1m23s
Details
Tests and linters / Tests (pull_request) Failing after 1m32s
Details
Pre-commit hooks / Pre-commit (pull_request) Failing after 1m41s
Details
Tests and linters / Staticcheck (pull_request) Failing after 1m53s
Details
Tests and linters / gopls check (pull_request) Failing after 2m31s
Details
Tests and linters / Lint (pull_request) Failing after 3m14s
Details
3598628632 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
dstepanov-yadro added 1 commit 2024-10-15 12:22:20 +00:00
[#1422] multinet: Add metrics
Some checks failed
Tests and linters / Run gofumpt (pull_request) Successful in 46s
Details
Tests and linters / Tests with -race (pull_request) Failing after 51s
Details
DCO action / DCO (pull_request) Successful in 1m11s
Details
Build / Build Components (pull_request) Failing after 1m17s
Details
Tests and linters / Tests (pull_request) Failing after 1m25s
Details
Pre-commit hooks / Pre-commit (pull_request) Failing after 1m32s
Details
Vulncheck / Vulncheck (pull_request) Failing after 1m50s
Details
Tests and linters / Staticcheck (pull_request) Failing after 2m3s
Details
Tests and linters / gopls check (pull_request) Failing after 2m20s
Details
Tests and linters / Lint (pull_request) Failing after 3m19s
Details
d12a90465d 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
dstepanov-yadro force-pushed feat/multinet from d12a90465d to 1591e3fbf5 2024-10-15 12:34:25 +00:00 Compare
dstepanov-yadro force-pushed feat/multinet from 1591e3fbf5 to f143081728 2024-10-15 14:13:31 +00:00 Compare
dstepanov-yadro force-pushed feat/multinet from f143081728 to 4d27e1d266 2024-10-16 10:21:22 +00:00 Compare
dstepanov-yadro changed title from WIP: Source-based routing support to Source-based routing support 2024-10-16 11:52:38 +00:00
fyrchik commented 2024-10-16 15:51:32 +00:00
Owner
Copy link

Perf tests:

So, no difference for PUT but non-negligible improvement for GET. How can we explain it?

>Perf tests: So, no difference for PUT but non-negligible improvement for GET. How can we explain it?
dstepanov-yadro commented 2024-10-17 06:38:47 +00:00
Author
Member
Copy link

Perf tests:

So, no difference for PUT but non-negligible improvement for GET. How can we explain it?

The difference in the results does not look significant. These were short tests (15 minutes), the main purpose of which was to confirm that there was no significant degradation.

> >Perf tests: > > So, no difference for PUT but non-negligible improvement for GET. How can we explain it? The difference in the results does not look significant. These were short tests (15 minutes), the main purpose of which was to confirm that there was no significant degradation.
acid-ant approved these changes 2024-10-17 07:05:42 +00:00
Dismissed
fyrchik requested changes 2024-10-17 07:42:54 +00:00
Dismissed
cmd/frostfs-ir/defaults.go Outdated
@ -134,0 +139,4 @@
cfg.SetDefault("multinet.balancer", "")
cfg.SetDefault("multinet.restrict", false)
cfg.SetDefault("multinet.fallback_delay", "0s")
cfg.SetDefault("multinet.subnets", "")
fyrchik commented 2024-10-17 07:17:31 +00:00
Owner
Copy link

Shouldn't it be an empty list?

Shouldn't it be an empty list?
dstepanov-yadro commented 2024-10-17 08:59:14 +00:00
Author
Member
Copy link

It works so. To specify empty list it is required to add Subnet type to IR config, but this is only required to set defaults.

It works so. To specify empty list it is required to add `Subnet` type to IR config, but this is only required to set defaults.
fyrchik commented 2024-10-17 11:02:20 +00:00
Owner
Copy link

I've meant "[]", but if it already works, ok.

I've meant `"[]"`, but if it already works, ok.
fyrchik marked this conversation as resolved
internal/net/dialer.go Outdated
@ -0,0 +18,4 @@
}
func newDefaulDialer() net.Dialer {
// From `grpc.WithContextDialer` comment:
fyrchik commented 2024-10-17 07:39:55 +00:00
Owner
Copy link

Could you provide a link to github?

Could you provide a link to github?
dstepanov-yadro commented 2024-10-17 08:48:40 +00:00
Author
Member
Copy link

done

done
fyrchik marked this conversation as resolved
internal/net/dialer.go Outdated
@ -0,0 +20,4 @@
func newDefaulDialer() net.Dialer {
// From `grpc.WithContextDialer` comment:
//
// Note: All supported releases of Go (as of December 2023) override the OS
fyrchik commented 2024-10-17 07:22:42 +00:00
Owner
Copy link

Why do we care about it in this PR?

Why do we care about it in this PR?
dstepanov-yadro commented 2024-10-17 08:50:25 +00:00
Author
Member
Copy link

multinet is used primarily for gprs connections, so I think it's important to preserve the behavior of the standard grpc library.

`multinet` is used primarily for gprs connections, so I think it's important to preserve the behavior of the standard grpc library.
dstepanov-yadro commented 2024-10-17 08:51:19 +00:00
Author
Member
Copy link

Also I think it's correct to use OS settings for keep alive.

Also I think it's correct to use OS settings for keep alive.
fyrchik marked this conversation as resolved
internal/net/dialer_source.go Outdated
@ -0,0 +43,4 @@
// Dialer returns configured Dialer and True, if source based routing enabled.
// Otherwise returns nil and False.
func (s *DialerSource) Dialer() (Dialer, bool) {
fyrchik commented 2024-10-17 07:26:38 +00:00
Owner
Copy link

Frankly, this is a rather generic package (it is even called net)
Why don't we return Dialer only?
It seems every client of this function has to check the second return value and use the same default otherwise.

Frankly, this is a rather generic package (it is even called `net`) Why don't we return `Dialer` only? It seems _every_ client of this function has to check the second return value and use _the same_ default otherwise.
dstepanov-yadro commented 2024-10-17 08:45:48 +00:00
Author
Member
Copy link

fixed

fixed
fyrchik marked this conversation as resolved
internal/net/event_handler.go Outdated
@ -0,0 +18,4 @@
if sourceIP != nil {
sourceIPString = sourceIP.Network() + "://" + sourceIP.String()
}
statusString := "OK"
fyrchik commented 2024-10-17 07:41:39 +00:00
Owner
Copy link

We have success bool in many other places, why do we go with status here?

We have `success bool` in many other places, why do we go with `status` here?
dstepanov-yadro commented 2024-10-17 08:53:11 +00:00
Author
Member
Copy link

In the distant bright future, it may be necessary to have separated error types, for example, for failover scenarios.

In the distant bright future, it may be necessary to have separated error types, for example, for failover scenarios.
fyrchik commented 2024-10-17 11:00:37 +00:00
Owner
Copy link

Do you mean for this error or for everything?
I would rather be consistent across the codebase.
Also, what different kinds of errors do you have in mind?

Do you mean for this error or for everything? I would rather be consistent across the codebase. Also, what different kinds of errors do you have in mind?
dstepanov-yadro commented 2024-10-17 11:15:51 +00:00
Author
Member
Copy link

For example, it may be necessary to distinguish between errors of the server to which the connection is being made and errors of the network interface through which the connection is being made.

For example, it may be necessary to distinguish between errors of the server to which the connection is being made and errors of the network interface through which the connection is being made.
fyrchik commented 2024-10-17 11:36:51 +00:00
Owner
Copy link

I doubt we will do sth like this in the near future (remember: it is Dial, not some protocol method, so there is no status)
And if we will or will need to, we can always add a label.

I doubt we will do sth like this in the near future (remember: it is `Dial`, not some protocol method, so there is no status) And if we will or will need to, we can always add a label.
dstepanov-yadro commented 2024-10-17 12:22:39 +00:00
Author
Member
Copy link

fixed

fixed
fyrchik marked this conversation as resolved
pkg/network/cache/multi.go Outdated
@ -63,0 +76,4 @@
d, enabled := x.opts.DialerSource.Dialer()
if enabled {
grpcOpts = append(grpcOpts, grpc.WithContextDialer(func(ctx context.Context, s string) (net.Conn, error) {
return internalNet.DialContextTCP(ctx, s, d)
fyrchik commented 2024-10-17 07:37:02 +00:00
Owner
Copy link

Unix domain sockets can be used:
830135e6c5/internal/transport/http2_client.go (L159)
We should parse the address and return an explicit error if we do not support this.

Unix domain sockets can be used: https://github.com/grpc/grpc-go/blob/830135e6c5a351abf75f0c9cfdf978e5df8daeba/internal/transport/http2_client.go#L159 We should parse the address and return an explicit error if we do not support this.
dstepanov-yadro commented 2024-10-17 08:53:47 +00:00
Author
Member
Copy link

Fixed. unix sockets may be supported if restrict = false.

Fixed. unix sockets may be supported if `restrict = false`.
fyrchik marked this conversation as resolved
pkg/services/tree/cache.go Outdated
@ -104,0 +108,4 @@
d, enabled := c.ds.Dialer()
if enabled {
opts = append(opts, grpc.WithContextDialer(func(ctx context.Context, s string) (net.Conn, error) {
return internalNet.DialContextTCP(ctx, s, d)
fyrchik commented 2024-10-17 07:37:34 +00:00
Owner
Copy link

Same here -- it can be unix://

Same here -- it can be `unix://`
dstepanov-yadro commented 2024-10-17 08:53:56 +00:00
Author
Member
Copy link

fixed

fixed
fyrchik marked this conversation as resolved
dstepanov-yadro force-pushed feat/multinet from 4d27e1d266 to 4f26aba3f2 2024-10-17 08:45:13 +00:00 Compare
dstepanov-yadro dismissed acid-ant's review 2024-10-17 08:45:13 +00:00
Reason:

New commits pushed, approval review dismissed automatically according to repository settings

dstepanov-yadro force-pushed feat/multinet from 4f26aba3f2 to be3fab65ab 2024-10-17 08:48:26 +00:00 Compare
fyrchik reviewed 2024-10-17 11:03:52 +00:00
internal/net/dialer.go Outdated
@ -0,0 +41,4 @@
}
// parseDialTarget returns the network and address to pass to dialer.
// From https://github.com/grpc/grpc-go/blob/830135e6c5a351abf75f0c9cfdf978e5df8daeba/internal/transport/http_util.go#L443
fyrchik commented 2024-10-17 11:03:52 +00:00
Owner
Copy link

Do we need to copy some copyright or license here if we reuse code?

Do we need to copy some copyright or license here if we reuse code?
dstepanov-yadro commented 2024-10-17 12:22:33 +00:00
Author
Member
Copy link

done

done
fyrchik marked this conversation as resolved
dstepanov-yadro force-pushed feat/multinet from be3fab65ab to d9feca670c 2024-10-17 11:59:11 +00:00 Compare
dstepanov-yadro force-pushed feat/multinet from d9feca670c to 5b9aa3fcf3 2024-10-17 12:05:38 +00:00 Compare
fyrchik approved these changes 2024-10-17 12:28:38 +00:00
acid-ant approved these changes 2024-10-17 13:02:47 +00:00
dstepanov-yadro merged commit c0a2f20eee into master 2024-10-17 13:15:09 +00:00
dstepanov-yadro deleted branch feat/multinet 2024-10-17 13:15:10 +00:00
dstepanov-yadro referenced this pull request from a commit 2024-10-17 13:15:11 +00:00
[#1422] ir: Add dialer source
dstepanov-yadro referenced this pull request from a commit 2024-10-17 13:15:11 +00:00
[#1422] multinet: Add metrics
Sign in to join this conversation.
Reviewers
No reviewers
fyrchik
acid-ant
TrueCloudLab/storage-core-developers
Labels
Clear labels   
P0

Highest

  
P1

High

  
P2

Medium

  
P3

Low

  
badger

Badger DB related issues

  
frostfs-adm

   
frostfs-cli

   
frostfs-ir

   
frostfs-lens

   
frostfs-node

   
good first issue

Good for newcomers

  
triage
  
Infrastructure

Infrastructure, Automation, CI/CD or DevOps related stuff

  
blocked

The issue or PR is blocked by something

  
bug

Something is not working

  
config

Configuration format update or breaking change

  
discussion

Talking about something in order to reach a decision or to exchange ideas

  
documentation

Documentation related

  
duplicate

This issue or pull request already exists

  
enhancement

New feature

  
go

Language features adoption

  
help wanted

Extra attention is needed

  
internal

   
invalid

Something is wrong

  
kludge

This is a kludge and we know it

  
observability

Related to metrics, tracing and logs

  
perfomance

Perfomance related

  
question

More information is needed

  
refactoring

Refactoring

  
wontfix

This won't be fixed

No labels
P0
P1
P2
P3
badger
frostfs-adm
frostfs-cli
frostfs-ir
frostfs-lens
frostfs-node
good first issue
triage
Infrastructure
blocked
bug
config
discussion
documentation
duplicate
enhancement
go
help wanted
internal
invalid
kludge
observability
perfomance
question
refactoring
wontfix
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
3 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: TrueCloudLab/frostfs-node#1422
No description provided.
Delete branch "dstepanov-yadro/frostfs-node:feat/multinet"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?