TrueCloudLab
/
frostfs-node
18
1
Fork 20
Code Issues 88 Pull Requests 7 Actions Releases Activity
3,582 Commits
7 Branches
20 Tags
47 MiB
Commit Graph

3582 Commits (c1cbe6ff2d48bd19b0fa375e70e25ac78590ccbc)

Author SHA1 Message Date
Anton Nikiforov 85cf1f47ac [#1465] node: Prevent process from killing by systemd when shutting down 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2023-02-17 12:13:00 +03:00
Artem Tataurov 362f24953a [#47] shard: Switch container size metric from physical to logical capacity 
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
 2023-02-17 12:03:42 +03:00
Pavel Karpy 901d62567d [#57] node: Broadcast link objects 
It boosts object assembling by an _average_ container node.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2023-02-17 11:58:27 +03:00
Aleksey Pastukhov 269a4e9b50 [#53] Fix dirty in version 
Signed-off-by: Aleksey Pastukhov <a.pastukhov@yadro.com>
 2023-02-16 11:10:43 +03:00
Evgenii Stratonikov 3e5bc394b5 [#48] adm: Add initialize test for 1 node 
Single node is used in dev-env, worth testing.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-02-16 09:07:18 +03:00
Evgenii Stratonikov 1d3669232e [#48] adm: Allow using nonzero magic with local client 
neo-go actor API uses `getVersion` call which returned incorrect magic.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-02-16 09:07:18 +03:00
Evgenii Stratonikov 204cd3a11c [#31] fstree: Optimize `treePath` 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-02-10 12:49:31 +03:00
Evgenii Stratonikov dee4498c1e [#31] fstree: Do not check for a file existence twice 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-02-10 12:49:31 +03:00
Evgenii Stratonikov abbecf49d6 [#31] fstree: Speedup string-to-address conversion 
```
name                  old time/op    new time/op    delta
_addressFromString-8    1.25µs ±30%    1.02µs ± 6%  -18.49%  (p=0.000 n=9+9)

name                  old alloc/op   new alloc/op   delta
_addressFromString-8      352B ± 0%      256B ± 0%  -27.27%  (p=0.000 n=9+10)

name                  old allocs/op  new allocs/op  delta
_addressFromString-8      6.00 ± 0%      4.00 ± 0%  -33.33%  (p=0.000
n=10+10)
```

Also, assure compiler that `s` doesn't escape:
Before this commit:
```
./fstree.go:74:24: leaking param: s
./fstree.go:90:6: moved to heap: addr
```

After this commit:
```
./fstree.go:74:24: s does not escape
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-02-10 12:49:31 +03:00
Artem Tataurov ab21d90cfb [#1794] shard: Add increasing case for the payload size metric 
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
 2023-02-09 13:30:23 +03:00
Stanislav Bogatyrev 5ffa826897 [#1] Update Changelog 
Signed-off-by: Stanislav Bogatyrev <s.bogatyrev@yadro.com>
 2023-02-06 17:41:14 +03:00
Stanislav Bogatyrev cb016d53a6 [#1] Fix comments and error messages 
Signed-off-by: Stanislav Bogatyrev <s.bogatyrev@yadro.com>
 2023-02-06 17:41:14 +03:00
Stanislav Bogatyrev c761a95eef [#1] Fix project name in control service 
Signed-off-by: Stanislav Bogatyrev <s.bogatyrev@yadro.com>
 2023-02-06 17:41:14 +03:00
Stanislav Bogatyrev f825cfac78 [#1] Fix project name in comments here and there 
Signed-off-by: Stanislav Bogatyrev <s.bogatyrev@yadro.com>
 2023-02-06 17:41:14 +03:00
Stanislav Bogatyrev c6645ef775 [#1] Documentation rebranding 
Signed-off-by: Stanislav Bogatyrev <s.bogatyrev@yadro.com>
 2023-02-06 17:41:14 +03:00
Stanislav Bogatyrev 488eece25f [#1] Fix Debian package references to old upstream 
Signed-off-by: Stanislav Bogatyrev <s.bogatyrev@yadro.com>
 2023-02-06 17:41:14 +03:00
Stanislav Bogatyrev 1cedd446bb [#1] Fix sample configs 
- Update sample configs to match unit tests
- Remove Docker container for N3 testnet
  Will return with updates when FrostFS enters N3

Signed-off-by: Stanislav Bogatyrev <s.bogatyrev@yadro.com>
 2023-02-06 17:41:14 +03:00
Stanislav Bogatyrev 1858f11146 [#1] Fix viper env prefix in cli tools 
Changing env prefix and corresponding example config files.

Signed-off-by: Stanislav Bogatyrev <s.bogatyrev@yadro.com>
 2023-02-06 17:41:14 +03:00
Pavel Karpy 73bc1b0b68 [#38] node: Fix linter warnings 
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2023-02-06 17:27:54 +03:00
Anton Nikiforov 515c60bdf4 [#1889] adm: Add command `morph netmap-candidates` 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2023-02-06 17:26:34 +03:00
Anton Nikiforov ee24815748 [#1889] Move flag `--config` in `cmd/frostfs-adm/internal/commonflags/flags.go` 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2023-02-06 17:26:34 +03:00
Anton Nikiforov 2b09564355 [#1889] Move netmap.go and exit.go from `cli` to `cmd/internal/common` 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2023-02-06 17:26:34 +03:00
Anton Nikiforov 5a9d6a09d8 [#8] cli: Set flag `mode` required for `control shards set-mode` 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2023-02-06 16:45:03 +03:00
Dmitriy Zabolotskiy 2540598779 [#36] Packaging: allow any version 
Also updated changelog

Signed-off-by: Dmitriy Zabolotskiy <d.zabolotskiy@yadro.com>
 2023-01-31 16:33:32 +03:00
Alexey Vanin 5d64a354cb [#1] Fix naming in FrostFS Adm help output 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-01-31 11:24:50 +03:00
Alexey Vanin d7e9e2ef9e [#1] Fix naming in FrostFS Lens help output 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-01-31 11:24:50 +03:00
Alexey Vanin d31d8c5335 [#1] Fix naming in FrostFS CLI help output 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-01-31 11:24:50 +03:00
Alexey Vanin 406ff1360f [#1] Fix version output for all compiled binaries 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-01-31 11:24:50 +03:00
Pavel Karpy 89a0266f5e [#1794] metrics: Track physical object capacity per shard 
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2023-01-26 20:06:28 +03:00
Evgenii Stratonikov 9513f163aa [#2116] metrics: Track physical object capacity in the container 
Currently we track based on `PayloadSize`, because it is already stored
in the metabase and it is easier to calculate without slowing down the
whole system.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2023-01-26 20:06:28 +03:00
Evgenii Stratonikov d65a95a2c6 [#28] pilorama: Remove `LogMove` struct 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-01-25 15:31:47 +03:00
Evgenii Stratonikov 46c62be7e8 [#28] Fix linter issues 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-01-25 15:31:47 +03:00
Evgenii Stratonikov 072a7d61ab [#2176] neofs-node: Do not cache full container list 
We rarely need to list all containers: as one example
we need it for tree service synchronization once per epoch.
Given that cache TTL has the order of block time it makes no sense
to cache the list of all containers.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-01-25 15:31:47 +03:00
Evgenii Stratonikov 8078af3424 [#2176] neofs-node: Do not invalidate old cache items 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-01-25 15:31:47 +03:00
Evgenii Stratonikov e5a40b90b6 [#2176] neofs-node: `Peek` during cache invalidation 
`Get` needs write mutex and makes our item move to top.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-01-25 15:31:47 +03:00
Evgenii Stratonikov 93eb72ef44 [#2176] neofs-node: Remove unused field from `morphContainerWriter` 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-01-25 15:31:47 +03:00
Anton Nikiforov 8ee590794f [#1962] cli: `common.PrintVerbose` prints via `cobra.Command.Printf` 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-01-25 15:31:47 +03:00
Anton Nikiforov cff4184cd3 [#2128] Add doc for extended headers 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-01-25 15:31:47 +03:00
Evgenii Stratonikov c72576e72f [#2208] engine: Log time-consuming shard operations 
Currently the only way to tell whether `evacuate/set-mode` is finished
is to set a very big timeout and _hope_ that the operation will finish.
In this commit we add INFO logs for such operations which should
simplify the life of an administrator.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-01-25 15:31:47 +03:00
Evgenii Stratonikov 87f0e3ea25 [#2208] fstree: Rename file after write 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-01-25 15:31:47 +03:00
Evgenii Stratonikov 792319a044 [#2208] fstree: Remove file if there was an error during write 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-01-25 15:31:47 +03:00
Evgenii Stratonikov 67c97c6804 [#2210] services/tree: Drop messages not in queue 
Currently, under high load clients are blocked on channel send
and the number of goroutines can increase indefinitely.
In this commit we drop replication messages if send/recv queue is full
and rely on a background synchronization.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-01-25 15:31:47 +03:00
Evgenii Stratonikov 25d5995cef [#2210] pilorama: Allocate bucket name outside of batches 
1. Reduce allocations inside transactions.
2. Do not encode container ID to string: it allocates a lot and takes more
space.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-01-25 15:31:47 +03:00
Evgenii Stratonikov 165a600624 [#2210] pilorama: Reduce the amount of keys per node 
Under high load we are limited by the _amount_ of keys we need to update
in a single transaction. In this commit we try storing all state
with a single key.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-01-25 15:31:47 +03:00
Pavel Karpy 64a5294b27 [#2200] shard: Do not fetch big objects from blobovniczas 
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-01-25 15:31:47 +03:00
Pavel Karpy 91757329ae [#2200] shard: Fix blobstor obj fetching 
In the previous implementation any non-nil error that preceded object
fetching from blobstor led to iterating over every storage (in other words,
no storage ID information was taken into account). Now storage ID is
skipped only if metabase (storage ID source) returns any error.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-01-25 15:31:47 +03:00
Pavel Karpy cf1a91a758 [#2206] blobovnicza: Use Latin letters in the code 
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-01-25 15:31:47 +03:00
Pavel Karpy c33ad3c474 [#2164] node: Use `reconnect_interval` from config 
Not always the default one.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-01-25 15:31:47 +03:00
Evgenii Stratonikov 0d8366f475 [#2207] object/acl: Return status error for expired session token 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-01-25 15:31:47 +03:00
Evgenii Stratonikov 6451f019d2 [#2203] shard: Do not panic in `Close` after unsuccessful `Init` 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-01-25 15:31:47 +03:00