After install on one node epoch froze on first epoch #194

New issue

Closed

opened 2023-03-31 14:42:02 +00:00 by anikeev-yadro · 0 comments

anikeev-yadro commented

2023-03-31 14:42:02 +00:00

Member

After system was installed on one node epoch froze on first epoch:

service@az:~$ frostfs-cli netmap snapshot -r localhost:8080 -g
Epoch: 1
Node 1: 0203cce0980de0994cd3c1c610de6ecb890c7ec229d41c4840a6671a18f7a972d1 ONLINE /ip4/172.26.160.13/tcp/8080
        Continent: Europe
        Country: Russia
        CountryCode: RU
        Deployed: YACZROKH
        Location: Moskva
        Node: 172.26.160.13
        Price: 10
        SubDiv: Moskva
        SubDivCode: MOW
        UN-LOCODE: RU MOW
Node 2: 02cf2b3fb2423c9677f123e3aef6ef2d569b07ffd0bd864b094301c2ac4cb5219c ONLINE /ip4/172.26.160.238/tcp/8080
        Continent: Europe
        Country: Sweden
        CountryCode: SE
        Deployed: YACZROKH
        Location: Stockholm
        Node: 172.26.160.238
        Price: 10
        SubDiv: Stockholms l�n
        SubDivCode: AB
        UN-LOCODE: SE STO
Node 3: 03ac17222b80e3344d2f867d202d1f6842b5a006b0b7c2ffa45620ec748c6f4fa7 ONLINE /ip4/172.26.160.122/tcp/8080
        Continent: Europe
        Country: Russia
        CountryCode: RU
        Deployed: YACZROKH
        Location: Saint Petersburg (ex Leningrad)
        Node: 172.26.160.122
        Price: 10
        SubDiv: Sankt-Peterburg
        SubDivCode: SPE
        UN-LOCODE: RU LED
Node 4: 03bf38ffb46ee0275516b081f0f988c66f3c05e57f88068dbe3f31abfb517a9ef9 ONLINE /ip4/172.26.160.233/tcp/8080
        Continent: Europe
        Country: Finland
        CountryCode: FI
        Deployed: YACZROKH
        Location: Helsinki (Helsingfors)
        Node: 172.26.160.233
        Price: 10
        SubDiv: Uusimaa
        SubDivCode: 18
        UN-LOCODE: FI HEL

On the other nodes epoch successfully ticks:

service@buky:~$ frostfs-cli netmap snapshot -r localhost:8080 -g
Epoch: 33
Node 1: 02cf2b3fb2423c9677f123e3aef6ef2d569b07ffd0bd864b094301c2ac4cb5219c ONLINE /ip4/172.26.160.238/tcp/8080
        Continent: Europe
        Country: Sweden
        CountryCode: SE
        Deployed: YACZROKH
        Location: Stockholm
        Node: 172.26.160.238
        Price: 10
        SubDiv: Stockholms l�n
        SubDivCode: AB
        UN-LOCODE: SE STO
Node 2: 03ac17222b80e3344d2f867d202d1f6842b5a006b0b7c2ffa45620ec748c6f4fa7 ONLINE /ip4/172.26.160.122/tcp/8080
        Continent: Europe
        Country: Russia
        CountryCode: RU
        Deployed: YACZROKH
        Location: Saint Petersburg (ex Leningrad)
        Node: 172.26.160.122
        Price: 10
        SubDiv: Sankt-Peterburg
        SubDivCode: SPE
        UN-LOCODE: RU LED
Node 3: 03bf38ffb46ee0275516b081f0f988c66f3c05e57f88068dbe3f31abfb517a9ef9 ONLINE /ip4/172.26.160.233/tcp/8080
        Continent: Europe
        Country: Finland
        CountryCode: FI
        Deployed: YACZROKH
        Location: Helsinki (Helsingfors)
        Node: 172.26.160.233
        Price: 10
        SubDiv: Uusimaa
        SubDivCode: 18
        UN-LOCODE: FI HEL

At the same time all operations on node AZ was failed:

service@az:~$ frostfs-cli --rpc-endpoint 172.26.160.13:8080 --wallet wallet.json container list  --config empty_pass.yml
Enter password >
rpc error: write request: context deadline exceeded

Version

FrostFS Storage node
Version: v0.0.1-316-ged28ce24
GoVersion: go1.19.7

Your Environment

Virtual

After system was installed on one node epoch froze on first epoch: ``` service@az:~$ frostfs-cli netmap snapshot -r localhost:8080 -g Epoch: 1 Node 1: 0203cce0980de0994cd3c1c610de6ecb890c7ec229d41c4840a6671a18f7a972d1 ONLINE /ip4/172.26.160.13/tcp/8080 Continent: Europe Country: Russia CountryCode: RU Deployed: YACZROKH Location: Moskva Node: 172.26.160.13 Price: 10 SubDiv: Moskva SubDivCode: MOW UN-LOCODE: RU MOW Node 2: 02cf2b3fb2423c9677f123e3aef6ef2d569b07ffd0bd864b094301c2ac4cb5219c ONLINE /ip4/172.26.160.238/tcp/8080 Continent: Europe Country: Sweden CountryCode: SE Deployed: YACZROKH Location: Stockholm Node: 172.26.160.238 Price: 10 SubDiv: Stockholms l�n SubDivCode: AB UN-LOCODE: SE STO Node 3: 03ac17222b80e3344d2f867d202d1f6842b5a006b0b7c2ffa45620ec748c6f4fa7 ONLINE /ip4/172.26.160.122/tcp/8080 Continent: Europe Country: Russia CountryCode: RU Deployed: YACZROKH Location: Saint Petersburg (ex Leningrad) Node: 172.26.160.122 Price: 10 SubDiv: Sankt-Peterburg SubDivCode: SPE UN-LOCODE: RU LED Node 4: 03bf38ffb46ee0275516b081f0f988c66f3c05e57f88068dbe3f31abfb517a9ef9 ONLINE /ip4/172.26.160.233/tcp/8080 Continent: Europe Country: Finland CountryCode: FI Deployed: YACZROKH Location: Helsinki (Helsingfors) Node: 172.26.160.233 Price: 10 SubDiv: Uusimaa SubDivCode: 18 UN-LOCODE: FI HEL ``` On the other nodes epoch successfully ticks: ``` service@buky:~$ frostfs-cli netmap snapshot -r localhost:8080 -g Epoch: 33 Node 1: 02cf2b3fb2423c9677f123e3aef6ef2d569b07ffd0bd864b094301c2ac4cb5219c ONLINE /ip4/172.26.160.238/tcp/8080 Continent: Europe Country: Sweden CountryCode: SE Deployed: YACZROKH Location: Stockholm Node: 172.26.160.238 Price: 10 SubDiv: Stockholms l�n SubDivCode: AB UN-LOCODE: SE STO Node 2: 03ac17222b80e3344d2f867d202d1f6842b5a006b0b7c2ffa45620ec748c6f4fa7 ONLINE /ip4/172.26.160.122/tcp/8080 Continent: Europe Country: Russia CountryCode: RU Deployed: YACZROKH Location: Saint Petersburg (ex Leningrad) Node: 172.26.160.122 Price: 10 SubDiv: Sankt-Peterburg SubDivCode: SPE UN-LOCODE: RU LED Node 3: 03bf38ffb46ee0275516b081f0f988c66f3c05e57f88068dbe3f31abfb517a9ef9 ONLINE /ip4/172.26.160.233/tcp/8080 Continent: Europe Country: Finland CountryCode: FI Deployed: YACZROKH Location: Helsinki (Helsingfors) Node: 172.26.160.233 Price: 10 SubDiv: Uusimaa SubDivCode: 18 UN-LOCODE: FI HEL ``` At the same time all operations on node AZ was failed: ``` service@az:~$ frostfs-cli --rpc-endpoint 172.26.160.13:8080 --wallet wallet.json container list --config empty_pass.yml Enter password > rpc error: write request: context deadline exceeded ``` ## Version ``` FrostFS Storage node Version: v0.0.1-316-ged28ce24 GoVersion: go1.19.7 ``` ## Your Environment Virtual

journal_az.tar.gz

999 KiB