Commit graph

155 commits

Author SHA1 Message Date
Leonard Lyubich
8d17dab86e [#493] Refactor serving of prometheus and pprof services
Rename `util/profiler` package to `httputil` and refactor it:

  * simplify utility HTTP server;

  * make more generic server's parameters in order to remove `viper.Viper`
    dependency;

  * use single constructor for creating the pprof and prometheus servers;

  * replace `enabled` config value with empty-check of the network address.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-12 10:46:17 +03:00
Leonard Lyubich
db2e43387b [#493] logger: Simplify Logger's parameters
Replace `viper.Viper` parameter with `Prm` structure. Currently only logging
level can be parameterized through string setter.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-11 12:35:12 +03:00
Alex Vanin
d49bd4b94a Update neofs-api-go to v1.26.0
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-05-10 09:45:11 +03:00
Pavel Karpy
ea781664cf [#488] reputation/eigentrust/calculator: Implement calc wrapper
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Pavel Karpy
dbff8e9b3f [#488] go.mod: Update neofs-api-go lib
Update neofs-api-go version in go.mod
after #275 nil pointer bugfix.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Pavel Karpy
e8885d72f4 [#488] reputation/remoteProvider: Implement intermediate remote
Move common remoteProvider code to cmd/reputation/common.
Hide WriterProvider initialization behind interface and
add implementation of that interface to local and
intermediate packages in cmd/reputation directory.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Pavel Karpy
6ffc109a75 [#488] go.mod: Update neofs-api-go lib
Update neofs-api-go version
in go.mod with epoch field in
`SendIntermediateResultRequest_Body`.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Evgenii Stratonikov
60d74bce67 [#466] cli: support NEP-2 and NEP-6 key formats
NEP-2 is contains WIF encrypted with password.
NEP-6 is format used for neo-go wallets.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-04-23 10:11:47 +03:00
Leonard Lyubich
0ec8bcf6b4 Update neofs-api-go library
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-04-12 13:25:28 +03:00
Alex Vanin
05b45270c8 [#468] Update neo-go to v0.94.1
New neo-go version provides:
- new type for roles in `RoleManagement` contract,
- methods to get keys from `RoleManagement` contract,
- new way to sign notary transaction.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-04-09 11:32:05 +03:00
Alex Vanin
7cf48d4d91 [#452] Update neo-go to latest master
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-04-06 10:35:20 +03:00
Leonard Lyubich
747cfa44e5 [#444] Pull neofs-api-go with reputation changes
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-04-01 14:54:22 +03:00
Alex Vanin
0851ee1ed1 Update neofs-api-go to v1.25.0
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-03-22 12:29:11 +03:00
Alex Vanin
56243fe2b1 [#433] Update neo-go to v0.94.0
This version supports updated network fee calculation.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-03-22 11:45:35 +03:00
Leonard Lyubich
1ce0505575 [#425] Exec go mod tidy
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-03-17 15:00:37 +03:00
Leonard Lyubich
e6f945e61d [#425] Upgrade neofs-api-go version
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-03-17 15:00:37 +03:00
Alex Vanin
ec07792c4e [#427] Update neo-go to pre-released v0.94.0 version
Should be compatible with NEO3 testnet.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-03-17 09:57:40 +03:00
Evgenii Stratonikov
cc7287d6f7 [#422] pkg/services: Cache clients by address only
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-03-15 14:07:53 +03:00
Alex Vanin
883a2e3e77 [#404] Update neo-go to post preview5 version
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-03-04 12:39:15 +03:00
Alex Vanin
1d2832b61a Update neofs-api-go to v1.24.0
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-02-26 18:29:07 +03:00
Leonard Lyubich
9073e198b9 [#398] cmd/node: Serve NetmapService.NetworkInfo RPC
Implement `NetworkInfo` calls on full stack of Netmap services. Current
epoch is read from node local state, magic number is read via `MagicNumber`
call of morph client.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-19 19:16:40 +03:00
Leonard Lyubich
065c663bde Update neo-go to v0.93.0
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-15 10:49:38 +03:00
Leonard Lyubich
d936567249 Update neofs-api-go to v1.23.0
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-15 10:49:38 +03:00
Alex Vanin
d7bd312abd [#373] Update neo-go to pre-v0.93.0
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-02-11 17:31:56 +03:00
Leonard Lyubich
37915b53cf [#362] Pull latest neofs-api-go lib
Adopt new names of constant keys to node attributes from API lib.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-11 14:01:35 +03:00
Leonard Lyubich
9669afdfc7 [#316] locode: Implement polygons DB based on GeoJSON polygons
Define database of Earth's polygons in GeoJSON format. Implement resolving
of geo point to continent in which it is located through point-in-polygon
calculation.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-09 11:05:55 +03:00
Leonard Lyubich
ffbf6b922f [#361] metabase: Support new match types
Support STRING_NOT_EQUAL and NOT_PRESENT match types.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-03 10:32:15 +03:00
Leonard Lyubich
cc88320d6b [#328] container/load: Implement a metrics exchange controller
Implement a component that connects the value stores of the used space of
containers. Implement the Start/Stop operations on it, which will later
become the application handlers of the corresponding events from the
sidechain. The main task of the controller is to temporarily synchronize the
stages of calculating the global estimate of the used space in the
container. The details of the score calculation (the way of collecting /
transmitting local scores, the final score formula and writing to the
contract) are encapsulated in the dependency components, the controller is
abstracted from them.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-02 15:45:04 +03:00
Alex Vanin
66a8bcac1e [#347] Update neofs-api-go to v1.22.2
Contains fix for #347.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-01-27 21:13:55 +03:00
Alex Vanin
f83fbe1422 [#345] Update neofs-api-go to latest master
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-01-25 14:16:50 +03:00
Alex Vanin
3aa12797bd Update neofs-api-go to v1.22.1
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-01-15 18:13:43 +03:00
Leonard Lyubich
85ec633938 [#306] Define and compile proto files for private node service
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-14 12:13:49 +03:00
Leonard Lyubich
1e170c3812 [#234] services/object: Support netmap epoch and lookup dead in read ops
Support processing of NetmapEpoch and NetmapLookupDepth X-headers when
processing object read operations. Placement for operations
Get/Head/GetRange/GetRangeHash/Search is built for the epoch specified in
NetmapEpoch X-header (by default latest). Also the specified operations are
processed until success is achieved for network maps from the past up to
NetmapLookupDepth value. Behavior for default values (zero or missing) left
unchanged.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-12 19:19:11 +03:00
Leonard Lyubich
63fe34af7b [#292] Pull neofs-api-go v1.22.0
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-30 13:22:50 +03:00
Leonard Lyubich
22ae074510 [#291] Update go modules files
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-30 10:27:53 +03:00
Alex Vanin
df116852f2 [#289] Update neo-go version to v0.92.0
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-29 17:42:02 +03:00
Alex Vanin
0855dec9c2 Prepare for v0.14.0-rc.1
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Alex Vanin
44d9f95137 [#276] service/audit: Count Head requests and retries in PoR
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Leonard Lyubich
f7ca4a8dce [#80] cmd/cli: Implement storagegroup commands
Implement PUT/GET/LIST/DELETE commands of storage group module.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:49:27 +03:00
Alex Vanin
feec559e47 [#254] Update neo-go to preview4 compatible version
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Leonard Lyubich
0302dc7490 Pull neofs-api-go v1.21.2
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-24 15:34:20 +03:00
Leonard Lyubich
e53bf574b5 [#279] container: Use new methods to work with container format
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-24 13:38:20 +03:00
Leonard Lyubich
b56cfa4318 Pull neofs-api-go release v1.21.1
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-18 18:58:38 +03:00
Leonard Lyubich
66f9532857 [#264] object/get: Check parent address in linking/last child
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-18 16:42:17 +03:00
Leonard Lyubich
fb50362dcc [#243] services/object: Share common parameters across services
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Alex Vanin
a01262d8bd [#243] service/object: Fix object chain assembly for tombstone body
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
09916f21b5 [#243] cmd/cli: Write tombstone address in Object.Delete cmd
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
510e9ff2ec [#243] core/object: Replace test content of tombstone with an API structure
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
1d23483828 [#235] services/object: Implement new GetRange algorithm
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
f24daa10ff [#233] services/object: Implement new Get algorithm
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Alex Vanin
e2de95e3f6 [#230] metabase: Index split info structures
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
07e998d846 [#216] blobovnicza: Implement main methods and logic
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
b127607ac6 [#176] localstore: Implement primary BlobStor
Implement primary local BLOB storage based on filesystem tree.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
09750484f9 [#176] localstore: Draft storage engine structure and ops
Implement the primary structure and operation of the local object
storage engine.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich
f6e5644b85 Update to latest neofs-api-go
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-01 13:35:53 +03:00
Leonard Lyubich
d94a0eb25e [#221] eacl: Get rid of deprecated methods usage
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-01 11:08:47 +03:00
Alex Vanin
46a43a778f [#208] Update neofs-api-go with split ID support
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-11-30 10:44:15 +03:00
Alex Vanin
3d6c6d8539 Update neofs-api-go version to v1.20.3
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-11-25 15:17:33 +03:00
Leonard Lyubich
e8bc03d92f [#206] cli/container: Use GetEACLWithSignature client method
Use GetEACLWithSignature client method in get-eacl cmd of container CLI in
order to print eACL table and signature regardless of their correctness. The
ability to check the correctness of the signature will be added later.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-25 14:35:27 +03:00
Leonard Lyubich
a1b3da88c6 [#193] Pull latest neofs-api-go
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-24 15:34:30 +03:00
Leonard Lyubich
ffeea2c003 [#177] metabase: Add index by object ID
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-23 09:54:59 +03:00
Alex Vanin
1b74ae45cc Update neofs-api-go to v1.20.2
Fixes small bug with signature check of head
responses in `pkg/client`.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-11-17 17:27:22 +03:00
Leonard Lyubich
1caf15463e [#174] Update to neofs-api-go v1.20.0
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-17 11:56:00 +03:00
Leonard Lyubich
3de8febe57 [#174] Update to latest neofs-api-go changes
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-17 11:56:00 +03:00
Alex Vanin
1c12f6bcbf [#160] Update neofs-api-go to latest version
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-11-16 10:02:12 +03:00
Alex Vanin
65be09d3db [#155] Update neofs-api-go with refactored pkg/netmap
Refactored pkg/netmap package provides JSON converters for
NodeInfo and PlacementPolicy structures, that has been used
by client applications.

It also updates Node structure itself so it is a part of
grpc <-> v2 <-> pkg conversion chain.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-11-06 09:55:05 +03:00
Leonard Lyubich
d74aa807bf Update go.sum
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-03 14:14:38 +03:00
Leonard Lyubich
a3e0a9f74c [#85] core/container: Implement CheckFormat function
Implement function that conducts sanity checks of the container data.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-11-03 14:14:38 +03:00
Alex Vanin
174efc9df3 [#124] Update neo-go to pre-preview4 version
Neo-go does not use smartcontract.Parameter to return values
anymore, so it's convertes partly removed from neofs-node.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-11-02 13:54:31 +03:00
Alex Vanin
c80bce9438 [#132] Implement netmap snapshot getter for inner ring
Later it should be merged with morph/client getters.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-11-02 10:52:52 +03:00
Alex Vanin
b12d26f974 Update neofs-api-go to latest version
Handle errors provided by JSON encoders.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-30 11:28:49 +03:00
Leonard Lyubich
85aacbbb10 [#128] localstorage: Implement primary object metabase
Implement bolt-based metabase that is going to be used in local object
storage. Implement Put/Get/Select methods.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-29 17:34:41 +03:00
Alex Vanin
ca986fc5ff [#105] Update neofs-api-go to latest version
With this version JSON encoder prints deafult
values.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-21 18:22:59 +03:00
Leonard Lyubich
4ce58fb26c Update go.sum
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-21 14:42:51 +03:00
Alex Vanin
e6f04f7785 [#104] Update neofs-api-go with new protobuf API
Also update contains JSON converters for neofs-cli
and fixes bug in container.set-acl command of SDK.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-20 17:31:59 +03:00
Alex Vanin
b7dfc73bbc [#25] Use comma separated container attributes in CLI
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-17 13:16:59 +03:00
Alex Vanin
d4461b9493 [#98] Update neofs-api-go to latest version
This version support stringers and parsers for identity types.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-16 11:26:47 +03:00
Alex Vanin
ace2362e74 [#92] Update neofs-api-go to latest version
With this update SDK Client won't rewrite original
session token of the request with new data.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-15 10:20:10 +03:00
Alex Vanin
fec5c81823 [#84] Update neofs-api-go to latest version
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-09 09:15:18 +03:00
Leonard Lyubich
2b2f5d457b Update go.sum
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-05 09:36:29 +03:00
Leonard Lyubich
1654df4d97 [#61] Update to latest neofs-api-go changes
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
Leonard Lyubich
12d57af998 [#56] Update to latest neofs-api-go changes
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
Alex Vanin
f6904db84f [#32] Use pkg/core interfaces to fetch container and netmap
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:36 +03:00
Alex Vanin
ad36a2cd8f [#32] Use classifier in basic ACL check
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:36 +03:00
Alex Vanin
5045b0c3d4 [#32] Add request sender classifier
ACL has to classify request senders by roles:
- owner of the container,
- request from container or inner ring node,
- any other request.

According to this roles ACL checker use different
bits of basic ACL to grant or deny access.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:36 +03:00
Alex Vanin
ab565b1862 [#32] Add basis of basic ACL check service
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:36 +03:00
Leonard Lyubich
7813b90554 [#52] Update to latest neofs-api-go changes
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
Leonard Lyubich
0490107165 [#50] services/object: Implement GetRange service
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:36 +03:00
Leonard Lyubich
becb9148c1 [#38] sdk/object: Add Object to RawObject converter
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
dc88934034 [#34] Update go.sum
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
c410b23c97 [#33] Update to latest neofs-api-go version
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
e7925fbc1c [#31] placement: Implement container placement traverser
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
121137c62b [#30] object_manager: Implement object transformers
Define object writer interface. Implement payload size limiter that
restricts payload size of the object. Implement object format moulder that
fill verification fields and finalizes object structure.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
9c1c023f05 [#30] Update to latest neofs-api-go and fix conflicts
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Alex Vanin
778dd54244 [#28] Update neofs-api-go to build images
These hotfixes will be altered with later neofs-api-go
updates.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:35 +03:00
Alex Vanin
86b9aefcae [#28] Make storage node configurable
To run storage node at dev-env environment it should have
configurable parameters. To keep `cfg` structures we can
read configuration from env and yml config file with viper
and parse values such as script hashes, fees, keys into
`cfg` structures.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:35 +03:00
001a23eb31 [#23] Support SQL-like queries for placement
JSON format is rather verbose an inconvenient to be
edited by hand. This commit implements SQL-like
language for representing placement policy.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2020-10-02 11:25:35 +03:00
Stanislav Bogatyrev
246a15de35 cli: Add empty neofs-cli app structure
In the following release `neofs-cli` will be used to directly manage NeoFS Node.
All required definitions and interfaces are also moving from `neofs-api` to
`neofs-node` repository, so it's more convinient to have `neofs-cli` here.

Signed-off-by: Stanislav Bogatyrev <stanislav@nspcc.ru>
2020-10-02 11:25:35 +03:00
Leonard Lyubich
03b170237f Remove broken packages
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-10-02 11:25:35 +03:00
Alex Vanin
27fcf2cc1d [#16] Update neo-go and neofs-api-go dependencies
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-10-02 11:25:35 +03:00