8a49ace9b6
O_TMPFILE is implemented for all modern FSes and it's much easier and safer to
use. If application crashes in the middle of writing this file would be gone
and won't leave any garbage.
Notice that this implementation makes a different choice wrt EEXIST handling,
generic one always overwrites, while this one keeps the old data.
There is no real performance difference.
SSD (slow&old), XFS, Core i7-8565U:
Sync
```
name old time/op new time/op delta
Put/size=1024,thread=1/fstree-8 1.74ms ± 3% 0.06ms ± 7% -96.31% (p=0.000 n=10+10)
Put/size=1024,thread=20/fstree-8 10.0ms ±41% 1.1ms ±18% -88.95% (p=0.000 n=9+10)
Put/size=1024,thread=100/fstree-8 32.3ms ±60% 6.5ms ±14% -79.97% (p=0.000 n=10+10)
Put/size=1048576,thread=1/fstree-8 17.8ms ±90% 3.4ms ±70% -81.08% (p=0.000 n=10+10)
Put/size=1048576,thread=20/fstree-8 103ms ±174% 112ms ±158% ~ (p=0.971 n=10+10)
Put/size=1048576,thread=100/fstree-8 949ms ±78% 583ms ±132% ~ (p=0.089 n=10+10)
name old alloc/op new alloc/op delta
Put/size=1024,thread=1/fstree-8 3.17kB ± 1% 1.96kB ± 0% -38.09% (p=0.000 n=10+10)
Put/size=1024,thread=20/fstree-8 59.6kB ± 1% 39.2kB ± 1% -34.30% (p=0.000 n=8+10)
Put/size=1024,thread=100/fstree-8 299kB ± 0% 198kB ± 0% -33.90% (p=0.000 n=7+9)
Put/size=1048576,thread=1/fstree-8 3.38kB ± 1% 2.36kB ± 1% -30.22% (p=0.000 n=10+10)
Put/size=1048576,thread=20/fstree-8 65.7kB ± 4% 47.7kB ± 6% -27.27% (p=0.000 n=10+10)
Put/size=1048576,thread=100/fstree-8 351kB ± 8% 245kB ± 8% -30.22% (p=0.000 n=10+10)
name old allocs/op new allocs/op delta
Put/size=1024,thread=1/fstree-8 30.3 ± 2% 21.0 ± 0% -30.69% (p=0.000 n=10+10)
Put/size=1024,thread=20/fstree-8 554 ± 1% 413 ± 0% -25.35% (p=0.000 n=8+10)
Put/size=1024,thread=100/fstree-8 2.77k ± 0% 2.07k ± 0% -25.27% (p=0.000 n=7+10)
Put/size=1048576,thread=1/fstree-8 32.0 ± 0% 25.0 ± 0% -21.88% (p=0.000 n=9+8)
Put/size=1048576,thread=20/fstree-8 609 ± 5% 494 ± 6% -18.93% (p=0.000 n=10+10)
Put/size=1048576,thread=100/fstree-8 3.25k ± 9% 2.50k ± 8% -23.21% (p=0.000 n=10+10)
```
No sync
```
name old time/op new time/op delta
Put/size=1024,thread=1/fstree-8 71.3µs ±10% 59.8µs ±10% -16.21% (p=0.000 n=10+10)
Put/size=1024,thread=20/fstree-8 1.43ms ± 6% 1.22ms ±13% -14.53% (p=0.000 n=10+10)
Put/size=1024,thread=100/fstree-8 8.12ms ± 3% 6.36ms ± 2% -21.67% (p=0.000 n=8+9)
Put/size=1048576,thread=1/fstree-8 1.88ms ±70% 1.61ms ±78% ~ (p=0.393 n=10+10)
Put/size=1048576,thread=20/fstree-8 32.7ms ±28% 34.2ms ±112% ~ (p=0.968 n=9+10)
Put/size=1048576,thread=100/fstree-8 262ms ±56% 226ms ±34% ~ (p=0.447 n=10+9)
name old alloc/op new alloc/op delta
Put/size=1024,thread=1/fstree-8 2.89kB ± 0% 1.96kB ± 0% -32.28% (p=0.000 n=10+10)
Put/size=1024,thread=20/fstree-8 58.2kB ± 0% 39.5kB ± 0% -32.09% (p=0.000 n=8+8)
Put/size=1024,thread=100/fstree-8 291kB ± 0% 198kB ± 0% -32.19% (p=0.000 n=9+9)
Put/size=1048576,thread=1/fstree-8 3.05kB ± 1% 2.13kB ± 1% -30.16% (p=0.000 n=10+9)
Put/size=1048576,thread=20/fstree-8 62.6kB ± 0% 44.3kB ± 0% -29.23% (p=0.000 n=9+9)
Put/size=1048576,thread=100/fstree-8 302kB ± 0% 210kB ± 1% -30.39% (p=0.000 n=9+9)
name old allocs/op new allocs/op delta
Put/size=1024,thread=1/fstree-8 27.0 ± 0% 21.0 ± 0% -22.22% (p=0.000 n=10+10)
Put/size=1024,thread=20/fstree-8 539 ± 0% 415 ± 0% -22.98% (p=0.000 n=10+10)
Put/size=1024,thread=100/fstree-8 2.69k ± 0% 2.07k ± 0% -23.09% (p=0.000 n=9+9)
Put/size=1048576,thread=1/fstree-8 28.0 ± 0% 22.3 ± 3% -20.36% (p=0.000 n=8+10)
Put/size=1048576,thread=20/fstree-8 577 ± 0% 458 ± 0% -20.72% (p=0.000 n=9+9)
Put/size=1048576,thread=100/fstree-8 2.76k ± 0% 2.15k ± 0% -22.05% (p=0.000 n=9+8)
```
HDD (LVM), ext4, Ryzen 5 1600:
Sync
```
│ fs.sync-generic │ fs.sync-linux │
│ sec/op │ sec/op vs base │
Put/size=1024,thread=1/fstree-12 34.70m ± 19% 33.59m ± 16% ~ (p=0.529 n=10)
Put/size=1024,thread=20/fstree-12 188.8m ± 8% 189.2m ± 16% ~ (p=0.739 n=10)
Put/size=1024,thread=100/fstree-12 264.8m ± 22% 273.6m ± 28% ~ (p=0.353 n=10)
Put/size=1048576,thread=1/fstree-12 54.90m ± 14% 47.08m ± 18% ~ (p=0.063 n=10)
Put/size=1048576,thread=20/fstree-12 244.1m ± 14% 220.4m ± 22% ~ (p=0.579 n=10)
Put/size=1048576,thread=100/fstree-12 847.2m ± 5% 893.6m ± 3% +5.48% (p=0.000 n=10)
geomean 164.3m 158.9m -3.29%
│ fs.sync-generic │ fs.sync-linux │
│ B/op │ B/op vs base │
Put/size=1024,thread=1/fstree-12 3.375Ki ± 1% 2.471Ki ± 1% -26.80% (p=0.000 n=10)
Put/size=1024,thread=20/fstree-12 66.62Ki ± 6% 49.21Ki ± 6% -26.15% (p=0.000 n=10)
Put/size=1024,thread=100/fstree-12 319.2Ki ± 1% 230.9Ki ± 2% -27.64% (p=0.000 n=10)
Put/size=1048576,thread=1/fstree-12 3.457Ki ± 1% 2.559Ki ± 1% -25.97% (p=0.000 n=10)
Put/size=1048576,thread=20/fstree-12 66.91Ki ± 1% 49.16Ki ± 1% -26.52% (p=0.000 n=10)
Put/size=1048576,thread=100/fstree-12 338.8Ki ± 2% 252.3Ki ± 3% -25.54% (p=0.000 n=10)
geomean 42.17Ki 31.02Ki -26.44%
│ fs.sync-generic │ fs.sync-linux │
│ allocs/op │ allocs/op vs base │
Put/size=1024,thread=1/fstree-12 33.00 ± 0% 27.00 ± 0% -18.18% (p=0.000 n=10)
Put/size=1024,thread=20/fstree-12 639.5 ± 1% 519.0 ± 2% -18.84% (p=0.000 n=10)
Put/size=1024,thread=100/fstree-12 3.059k ± 1% 2.478k ± 2% -18.99% (p=0.000 n=10)
Put/size=1048576,thread=1/fstree-12 33.50 ± 1% 28.00 ± 4% -16.42% (p=0.000 n=10)
Put/size=1048576,thread=20/fstree-12 638.5 ± 1% 520.0 ± 1% -18.56% (p=0.000 n=10)
Put/size=1048576,thread=100/fstree-12 3.209k ± 2% 2.655k ± 2% -17.28% (p=0.000 n=10)
geomean 405.3 332.1 -18.05%
```
No sync
```
│ fs.nosync-generic │ fs.nosync-linux │
│ sec/op │ sec/op vs base │
Put/size=1024,thread=1/fstree-12 148.2µ ± 20% 136.6µ ± 19% -7.89% (p=0.029 n=10)
Put/size=1024,thread=20/fstree-12 1.140m ± 26% 1.364m ± 16% ~ (p=0.143 n=10)
Put/size=1024,thread=100/fstree-12 11.93m ± 68% 26.89m ± 62% ~ (p=0.123 n=10)
Put/size=1048576,thread=1/fstree-12 1.302m ± 3% 1.287m ± 5% ~ (p=0.481 n=10)
Put/size=1048576,thread=20/fstree-12 77.52m ± 8% 74.07m ± 7% ~ (p=0.278 n=10+9)
Put/size=1048576,thread=100/fstree-12 226.1m ± ∞ ¹
geomean 5.986m 3.434m +18.60% ²
¹ need >= 6 samples for confidence interval at level 0.95
² benchmark set differs from baseline; geomeans may not be comparable
│ fs.nosync-generic │ fs.nosync-linux │
│ B/op │ B/op vs base │
Put/size=1024,thread=1/fstree-12 2.879Ki ± 0% 1.972Ki ± 0% -31.51% (p=0.000 n=10)
Put/size=1024,thread=20/fstree-12 55.94Ki ± 1% 37.90Ki ± 1% -32.25% (p=0.000 n=10)
Put/size=1024,thread=100/fstree-12 272.6Ki ± 0% 182.1Ki ± 9% -33.21% (p=0.000 n=10)
Put/size=1048576,thread=1/fstree-12 3.158Ki ± 0% 2.259Ki ± 0% -28.46% (p=0.000 n=10)
Put/size=1048576,thread=20/fstree-12 58.87Ki ± 0% 41.03Ki ± 0% -30.30% (p=0.000 n=10+9)
Put/size=1048576,thread=100/fstree-12 299.8Ki ± ∞ ¹
geomean 36.71Ki 16.60Ki -31.17% ²
¹ need >= 6 samples for confidence interval at level 0.95
² benchmark set differs from baseline; geomeans may not be comparable
│ fs.nosync-generic │ fs.nosync-linux │
│ allocs/op │ allocs/op vs base │
Put/size=1024,thread=1/fstree-12 28.00 ± 0% 22.00 ± 0% -21.43% (p=0.000 n=10)
Put/size=1024,thread=20/fstree-12 530.0 ± 0% 407.5 ± 1% -23.11% (p=0.000 n=10)
Put/size=1024,thread=100/fstree-12 2.567k ± 0% 1.956k ± 9% -23.77% (p=0.000 n=10)
Put/size=1048576,thread=1/fstree-12 30.00 ± 0% 24.00 ± 0% -20.00% (p=0.000 n=10)
Put/size=1048576,thread=20/fstree-12 553.5 ± 0% 434.0 ± 0% -21.59% (n=10+9)
Put/size=1048576,thread=100/fstree-12 2.803k ± ∞ ¹
geomean 347.9 178.8 -21.99% ²
¹ need >= 6 samples for confidence interval at level 0.95
² benchmark set differs from baseline; geomeans may not be comparable
```
Signed-off-by: Roman Khimov <roman@nspcc.ru>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
130 lines
5.9 KiB
Modula-2
130 lines
5.9 KiB
Modula-2
module git.frostfs.info/TrueCloudLab/frostfs-node
|
|
|
|
go 1.20
|
|
|
|
require (
|
|
code.gitea.io/sdk/gitea v0.17.1
|
|
git.frostfs.info/TrueCloudLab/frostfs-api-go/v2 v2.16.1-0.20240112150928-72885aae835c
|
|
git.frostfs.info/TrueCloudLab/frostfs-contract v0.18.1-0.20240115082915-f2a82aa635aa
|
|
git.frostfs.info/TrueCloudLab/frostfs-observability v0.0.0-20231101111734-b3ad3335ff65
|
|
git.frostfs.info/TrueCloudLab/frostfs-sdk-go v0.0.0-20240126141009-65b4525b3bf0
|
|
git.frostfs.info/TrueCloudLab/hrw v1.2.1
|
|
git.frostfs.info/TrueCloudLab/policy-engine v0.0.0-20240129064140-8d21ab2d99d9
|
|
git.frostfs.info/TrueCloudLab/tzhash v1.8.0
|
|
github.com/cheggaaa/pb v1.0.29
|
|
github.com/chzyer/readline v1.5.1
|
|
github.com/flynn-archive/go-shlex v0.0.0-20150515145356-3f9db97f8568
|
|
github.com/google/uuid v1.6.0
|
|
github.com/hashicorp/golang-lru/v2 v2.0.7
|
|
github.com/klauspost/compress v1.17.4
|
|
github.com/mitchellh/go-homedir v1.1.0
|
|
github.com/mr-tron/base58 v1.2.0
|
|
github.com/multiformats/go-multiaddr v0.12.1
|
|
github.com/nats-io/nats.go v1.32.0
|
|
github.com/nspcc-dev/neo-go v0.105.1
|
|
github.com/olekukonko/tablewriter v0.0.5
|
|
github.com/panjf2000/ants/v2 v2.9.0
|
|
github.com/paulmach/orb v0.11.0
|
|
github.com/prometheus/client_golang v1.18.0
|
|
github.com/spf13/cast v1.6.0
|
|
github.com/spf13/cobra v1.8.0
|
|
github.com/spf13/pflag v1.0.5
|
|
github.com/spf13/viper v1.18.2
|
|
github.com/stretchr/testify v1.8.4
|
|
go.etcd.io/bbolt v1.3.8
|
|
go.opentelemetry.io/otel v1.22.0
|
|
go.opentelemetry.io/otel/trace v1.22.0
|
|
go.uber.org/zap v1.26.0
|
|
golang.org/x/exp v0.0.0-20240119083558-1b970713d09a
|
|
golang.org/x/sync v0.6.0
|
|
golang.org/x/sys v0.16.0
|
|
golang.org/x/term v0.16.0
|
|
google.golang.org/grpc v1.61.0
|
|
google.golang.org/protobuf v1.32.0
|
|
gopkg.in/yaml.v3 v3.0.1
|
|
)
|
|
|
|
require (
|
|
github.com/sagikazarmark/locafero v0.4.0 // indirect
|
|
github.com/sagikazarmark/slog-shim v0.1.0 // indirect
|
|
github.com/sourcegraph/conc v0.3.0 // indirect
|
|
go.mongodb.org/mongo-driver v1.13.1 // indirect
|
|
)
|
|
|
|
require (
|
|
git.frostfs.info/TrueCloudLab/frostfs-crypto v0.6.0 // indirect
|
|
git.frostfs.info/TrueCloudLab/rfc6979 v0.4.0 // indirect
|
|
github.com/antlr4-go/antlr/v4 v4.13.0 // indirect
|
|
github.com/beorn7/perks v1.0.1 // indirect
|
|
github.com/bits-and-blooms/bitset v1.13.0 // indirect
|
|
github.com/cenkalti/backoff/v4 v4.2.1 // indirect
|
|
github.com/cespare/xxhash/v2 v2.2.0 // indirect
|
|
github.com/consensys/bavard v0.1.13 // indirect
|
|
github.com/consensys/gnark-crypto v0.12.2-0.20231222162921-eb75782795d2 // indirect
|
|
github.com/cpuguy83/go-md2man/v2 v2.0.3 // indirect
|
|
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
|
|
github.com/davidmz/go-pageant v1.0.2 // indirect
|
|
github.com/decred/dcrd/dcrec/secp256k1/v4 v4.2.0 // indirect
|
|
github.com/fsnotify/fsnotify v1.7.0 // indirect
|
|
github.com/go-fed/httpsig v1.1.0 // indirect
|
|
github.com/go-logr/logr v1.4.1 // indirect
|
|
github.com/go-logr/stdr v1.2.2 // indirect
|
|
github.com/golang/protobuf v1.5.3 // indirect
|
|
github.com/golang/snappy v0.0.4 // indirect
|
|
github.com/gorilla/websocket v1.5.1 // indirect
|
|
github.com/grpc-ecosystem/go-grpc-middleware/providers/prometheus v1.0.0 // indirect
|
|
github.com/grpc-ecosystem/go-grpc-middleware/v2 v2.0.1 // indirect
|
|
github.com/grpc-ecosystem/grpc-gateway/v2 v2.19.0 // indirect
|
|
github.com/hashicorp/go-version v1.6.0 // indirect
|
|
github.com/hashicorp/golang-lru v1.0.2 // indirect
|
|
github.com/hashicorp/hcl v1.0.0 // indirect
|
|
github.com/holiman/uint256 v1.2.4 // indirect
|
|
github.com/inconshreveable/mousetrap v1.1.0 // indirect
|
|
github.com/ipfs/go-cid v0.4.1 // indirect
|
|
github.com/josharian/intern v1.0.0 // indirect
|
|
github.com/klauspost/cpuid/v2 v2.2.6 // indirect
|
|
github.com/magiconair/properties v1.8.7 // indirect
|
|
github.com/mailru/easyjson v0.7.7 // indirect
|
|
github.com/mattn/go-runewidth v0.0.15 // indirect
|
|
github.com/minio/sha256-simd v1.0.1 // indirect
|
|
github.com/mitchellh/mapstructure v1.5.0 // indirect
|
|
github.com/mmcloughlin/addchain v0.4.0 // indirect
|
|
github.com/multiformats/go-base32 v0.1.0 // indirect
|
|
github.com/multiformats/go-base36 v0.2.0 // indirect
|
|
github.com/multiformats/go-multibase v0.2.0 // indirect
|
|
github.com/multiformats/go-multihash v0.2.3 // indirect
|
|
github.com/multiformats/go-varint v0.0.7 // indirect
|
|
github.com/nats-io/nkeys v0.4.7 // indirect
|
|
github.com/nats-io/nuid v1.0.1 // indirect
|
|
github.com/nspcc-dev/go-ordered-json v0.0.0-20240112074137-296698a162ae // indirect
|
|
github.com/nspcc-dev/neo-go/pkg/interop v0.0.0-20240122090917-ef99a7a9e33f // indirect
|
|
github.com/nspcc-dev/rfc6979 v0.2.0 // indirect
|
|
github.com/pelletier/go-toml/v2 v2.1.1 // indirect
|
|
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
|
|
github.com/prometheus/client_model v0.5.0 // indirect
|
|
github.com/prometheus/common v0.46.0 // indirect
|
|
github.com/prometheus/procfs v0.12.0 // indirect
|
|
github.com/rivo/uniseg v0.4.4 // indirect
|
|
github.com/russross/blackfriday/v2 v2.1.0 // indirect
|
|
github.com/spaolacci/murmur3 v1.1.0 // indirect
|
|
github.com/spf13/afero v1.11.0 // indirect
|
|
github.com/subosito/gotenv v1.6.0 // indirect
|
|
github.com/syndtr/goleveldb v1.0.1-0.20220721030215-126854af5e6d // indirect
|
|
github.com/twmb/murmur3 v1.1.8 // indirect
|
|
github.com/urfave/cli v1.22.14 // indirect
|
|
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.22.0 // indirect
|
|
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.22.0 // indirect
|
|
go.opentelemetry.io/otel/exporters/stdout/stdouttrace v1.22.0 // indirect
|
|
go.opentelemetry.io/otel/metric v1.22.0 // indirect
|
|
go.opentelemetry.io/otel/sdk v1.22.0 // indirect
|
|
go.opentelemetry.io/proto/otlp v1.1.0 // indirect
|
|
go.uber.org/multierr v1.11.0 // indirect
|
|
golang.org/x/crypto v0.18.0 // indirect
|
|
golang.org/x/net v0.20.0 // indirect
|
|
golang.org/x/text v0.14.0 // indirect
|
|
google.golang.org/genproto/googleapis/api v0.0.0-20240123012728-ef4313101c80 // indirect
|
|
google.golang.org/genproto/googleapis/rpc v0.0.0-20240123012728-ef4313101c80 // indirect
|
|
gopkg.in/ini.v1 v1.67.0 // indirect
|
|
lukechampine.com/blake3 v1.2.1 // indirect
|
|
rsc.io/tmplfunc v0.0.3 // indirect
|
|
)
|