Commit graph

54 commits

Author SHA1 Message Date
Evgenii Stratonikov
6936195afa [] util/attributes: Remove excessive slice copy during parsing
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-03-29 16:02:05 +03:00
Alex Vanin
8f476f3c4d [] Adopt neofs-node for non pointer slices in SDK
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-03-16 15:29:31 +03:00
Evgenii Stratonikov
050a4bb2b0 [] *: link TODOs to corresponding issues
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-02-11 12:58:59 +03:00
Evgenii Stratonikov
674f520da7 [] *: Use path/filepath package when working with files
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-02-03 10:12:07 +03:00
Evgenii Stratonikov
5828f43e52 [] util/rand: use single random source
It is much more convenient to skip source creation.
Also fix some bugs:
1. `cryptoSource.Int63()` now returns number in [0, 1<<63) as required
   by `rand.Source` interface.
2. Replace `cryptoSource.Uint63()` with `cryptoSource.Uint64` to allow
   generate uint64 numbers directly (see rand.Source64 docs).

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-01-12 19:29:27 +03:00
Evgenii Stratonikov
c35cdb3684 [] autocomplete: fetch output stream from provided command
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-01-12 18:33:48 +03:00
Evgenii Stratonikov
c01acba3ce [] autocomplete: beautify help message
Indent sections corresponding to different shells for faster navigation
and remove `#` at the line beginning.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-01-12 18:33:48 +03:00
Evgenii Stratonikov
efabffbfd0 [] cmd/*: factor out autocomplet command generation
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-01-12 18:33:48 +03:00
Evgenii Stratonikov
95893927aa *: replace neofs-api-go with neofs-sdk-go
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-11-12 17:29:09 +03:00
Alex Vanin
45f244eb77 [] locode: Use the nearest polygon when continent can't be found
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-11-02 10:30:23 +03:00
Evgenii Stratonikov
0beaed2ef4 [] util: implement Release for PseudWorkerPool
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-10-25 18:53:36 +03:00
Leonard Lyubich
07130855aa [] util: Rename SyncWorkerPool with PseudoWorkerPool
`SyncWorkerPool` name is more appropriate for worker pool of size 1.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-10-14 10:20:39 +03:00
Pavel Karpy
501c78f327 [] util/locode: Fix parsing minutes
Convert minutes of received coordinates
to decimal parts of degree, and do not
use decimal part of float as storage for
minutes: "5915N 01806E" is
59.25N 18.10E, not 59°15'N 18°06'E.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-10-12 22:25:01 +03:00
Alex Vanin
7f074a775e [] util/state: Implement persistent storage for application state
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-09-07 17:22:25 +03:00
Alex Vanin
f9218bf84f [] util/attributes: Restore escape characters after caching the result
As soon as resulting list and cache operate with the attribute pointer,
we can add attribute structure immediately and set restored key and
value of the attribute later.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-09-01 14:29:26 +03:00
Alex Vanin
f09b7e48af [] util/attributes: Fix order of require.Equal arguments
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-09-01 14:29:26 +03:00
Alex Vanin
2d7e9f0c40 [] util/attributes: Return consistent list of attributes
Attributes are linked to each other through parents, so they can
be returned in any order. However, it will be better to return
the list in consistent order to reduce entropy.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-09-01 14:29:26 +03:00
Alex Vanin
d8a04726ad [] util/attributes: Reuse attribute struct to update parents list
Parser should reuse existing attributes from the cache to update list
of the parent links in it. Parent links should be unique.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-09-01 14:29:26 +03:00
Alex Vanin
6527f9157c [] util/attributes: Support escaped symbols
To encode attributes with semicolon or slash, use
backslash as escaped character.

Example:
  User-Agent:NeoFS\/0.23
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-08-04 16:48:54 +03:00
Pavel Karpy
7a10d902be [] Use util.MkdirAllX instead of os.MkdirAll
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-07-30 16:42:22 +03:00
Pavel Karpy
51eeae3640 [] util: Add MkdirAllX func
Add `MkdirAllX` function that wraps `os.MkdirAll`
and adds `x` permission for user and group.
This is used to make all created dirs openable
for user and group regardless of the read config
permissions.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-07-30 16:42:22 +03:00
Pavel Karpy
46a66a21ec [] pkg/util: Fix commentary error
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-07-22 15:12:06 +03:00
Leonard Lyubich
dab33e0f9b [] Update Neo Go to v0.95.3
Add `keys.NEP2ScryptParams()` to `keys.NEP2Decrypt` call arguments.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-29 13:44:59 +03:00
Leonard Lyubich
56d4410913 [] Update to Go 1.16
Changes:

  * replace `iotuil` elements with the ones from `os` package;
  * replace `os.Filemode` with `fs.FileMode`;
  * use `signal.NotifyContext` instead of `NewGracefulContext` (removed).

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-29 13:44:59 +03:00
Pavel Karpy
48827f42d3 [] pkg: Sync method names and commentaries to them
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-06-24 16:10:44 +03:00
Evgenii Stratonikov
1553967328 [] cmd/neofs-ir: use NEP-6 wallet for keys
Also remove neofs-crypto uses from `pkg/innerring`.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-06-15 14:49:59 +03:00
Evgenii Stratonikov
71b87155ef [] *: use stdlib errors package
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-05-19 16:42:54 +03:00
Leonard Lyubich
8d17dab86e [] Refactor serving of prometheus and pprof services
Rename `util/profiler` package to `httputil` and refactor it:

  * simplify utility HTTP server;

  * make more generic server's parameters in order to remove `viper.Viper`
    dependency;

  * use single constructor for creating the pprof and prometheus servers;

  * replace `enabled` config value with empty-check of the network address.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-12 10:46:17 +03:00
Leonard Lyubich
db2e43387b [] logger: Simplify Logger's parameters
Replace `viper.Viper` parameter with `Prm` structure. Currently only logging
level can be parameterized through string setter.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-11 12:35:12 +03:00
Leonard Lyubich
2a970b0125 [] logger: Do not set global fields in constructor
Remove setting of `app_name` and `app_version` fields in `NewLogger`
constructor. Set these fields in Node and IR application in already
constructed log and remove them from viper .

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-11 12:35:12 +03:00
Leonard Lyubich
d17526f8ac [] logger: Abolish non-usable options
Change logger's encoding to `console`, time encoding to `ISO8601TimeEncoder`
and leave all other options as they are in `zap.NewProductionConfig`.

Remove default values of no longer existing options in node/ir config.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-11 12:35:12 +03:00
Alex Vanin
70dcb920b7 [] util/keyer: Prioritize hex decoding over base58
Hex encoded values are often may be misinterpreted as base58
values. Reverse case is quite rare, so it is better to try
hex decoding first.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-03-26 15:27:55 +03:00
Leonard Lyubich
977229eb5a [] locode/csv: Scan subdivision csv-table into memory
Scanning subdivision csv-table entries one-by-one takes significant time and
system resources. To speed up random access to table records, on the first
call, the table is pumped into memory (map). On subsequent calls, I/O
operations are not performed.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-10 21:11:28 +03:00
Leonard Lyubich
307355f165 [] locode/airports: Scan csv table into memory
Scanning csv-table entries one-by-one takes significant time and system
resources. To speed up random access to table records, on the first call,
the table is pumped into memory (map). On subsequent calls, I/O operations
are not performed.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-10 20:18:06 +03:00
Leonard Lyubich
687c7d3b4a [] locode/column: Fix LatitudeHemisphere.North method implementation
North method must return true if hemisphere char is `N`.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-10 20:18:06 +03:00
Leonard Lyubich
9a425924cf [] locode/boltdb: Replace all references to the City with Location
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-10 14:05:03 +03:00
Leonard Lyubich
0d2440649a [] locode/boltdb: Add option to enable read-only mode
Add ReadOnly function that returns Option that enables read-only mode in DB.
RO mode can be used by processes that won't modify the DB in order to not
acquire write flock.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-10 14:05:03 +03:00
Leonard Lyubich
04b06ba539 [] locode/boltdb: Add default timeout to open underlying BoltDB instance
Set timeout option of BoltDB Open operation to 3s in order to prevent
indefinite waiting for file lock.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-10 14:05:03 +03:00
Leonard Lyubich
f88e0866fe [] locode: Implement NeoFS location database based on BoltDB instance
Define NeoFS location database based on BoltDB. Implement methods to save
the record by key (Put) and to read the record by key (Get).

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-09 11:05:55 +03:00
Leonard Lyubich
9669afdfc7 [] locode: Implement polygons DB based on GeoJSON polygons
Define database of Earth's polygons in GeoJSON format. Implement resolving
of geo point to continent in which it is located through point-in-polygon
calculation.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-09 11:05:55 +03:00
Leonard Lyubich
d8c3a0e4f5 [] locode: Implement csv OpenFlights airport database
Implement airport database based on csv OpenFlights table. Implement
UN/LOCODE entry matcher. Implement country namespace.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-09 11:05:55 +03:00
Leonard Lyubich
6829048124 [] locode: Implement csv UN/LOCODE table
Define csv UN/LOCODE table. Implement iterator over table entries. Implement
method that resolves country/subdivision code pair to subdivision name.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-09 11:05:55 +03:00
Leonard Lyubich
cdd1274e1c [] locode: Define the API of location database
Define structure of keys and records of the location database. Define the
interfaces of all components necessary for the formation of the database.
Implement the function of filling the database.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-09 11:05:55 +03:00
Leonard Lyubich
0be35859ed [] locode: Define UN/LOCODE table data types
Define the data types needed to work with LOCODE's in NeoFS (country code,
location code, coordinates). Implement string parsers for new types.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-09 11:05:55 +03:00
Leonard Lyubich
137ef25d3e [] locode: Define structure of the entry of the UN/LOCODE table
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-09 11:05:55 +03:00
Leonard Lyubich
ba03f46316 [] util: Implement salting writer
Implement wrapper over io.Writer that applies binary salt to written data.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-11 18:40:59 +03:00
Leonard Lyubich
58fcb35fb0 [] Use Marshal(JSON)/Unmarshal(JSON) methods for encoding/decoding
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-17 11:56:00 +03:00
Alex Vanin
ded45e1fbc [] Add keyer to neofs-cli
Keyer prints information about private key, public key,
NEO3 Wallet, scripthash. It can generate new private key
or generate multisig address.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-11-09 18:59:21 +03:00
Alex Vanin
7db40eafe0 [] Add precision converter
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-28 13:36:59 +03:00
Leonard Lyubich
9d8576d397 [] util: Add SaltXOR function
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00