# Protocol Documentation ## Table of Contents - [netmap/service.proto](#netmap/service.proto) - Services - [NetmapService](#frost.fs.netmap.NetmapService) - Messages - [LocalNodeInfoRequest](#frost.fs.netmap.LocalNodeInfoRequest) - [LocalNodeInfoRequest.Body](#frost.fs.netmap.LocalNodeInfoRequest.Body) - [LocalNodeInfoResponse](#frost.fs.netmap.LocalNodeInfoResponse) - [LocalNodeInfoResponse.Body](#frost.fs.netmap.LocalNodeInfoResponse.Body) - [NetmapSnapshotRequest](#frost.fs.netmap.NetmapSnapshotRequest) - [NetmapSnapshotRequest.Body](#frost.fs.netmap.NetmapSnapshotRequest.Body) - [NetmapSnapshotResponse](#frost.fs.netmap.NetmapSnapshotResponse) - [NetmapSnapshotResponse.Body](#frost.fs.netmap.NetmapSnapshotResponse.Body) - [NetworkInfoRequest](#frost.fs.netmap.NetworkInfoRequest) - [NetworkInfoRequest.Body](#frost.fs.netmap.NetworkInfoRequest.Body) - [NetworkInfoResponse](#frost.fs.netmap.NetworkInfoResponse) - [NetworkInfoResponse.Body](#frost.fs.netmap.NetworkInfoResponse.Body) - [netmap/types.proto](#netmap/types.proto) - Messages - [Filter](#frost.fs.netmap.Filter) - [Netmap](#frost.fs.netmap.Netmap) - [NetworkConfig](#frost.fs.netmap.NetworkConfig) - [NetworkConfig.Parameter](#frost.fs.netmap.NetworkConfig.Parameter) - [NetworkInfo](#frost.fs.netmap.NetworkInfo) - [NodeInfo](#frost.fs.netmap.NodeInfo) - [NodeInfo.Attribute](#frost.fs.netmap.NodeInfo.Attribute) - [PlacementPolicy](#frost.fs.netmap.PlacementPolicy) - [Replica](#frost.fs.netmap.Replica) - [Selector](#frost.fs.netmap.Selector) - [Scalar Value Types](#scalar-value-types)

Top

## netmap/service.proto ### Service "frost.fs.netmap.NetmapService" `NetmapService` provides methods to work with `Network Map` and the information required to build it. The resulting `Network Map` is stored in sidechain `Netmap` smart contract, while related information can be obtained from other FrostFS nodes. ``` rpc LocalNodeInfo(LocalNodeInfoRequest) returns (LocalNodeInfoResponse); rpc NetworkInfo(NetworkInfoRequest) returns (NetworkInfoResponse); rpc NetmapSnapshot(NetmapSnapshotRequest) returns (NetmapSnapshotResponse); ``` #### Method LocalNodeInfo Get NodeInfo structure from the particular node directly. Node information can be taken from `Netmap` smart contract. In some cases, though, one may want to get recent information directly or to talk to the node not yet present in the `Network Map` to find out what API version can be used for further communication. This can be also used to check if a node is up and running. Statuses: - **OK** (0, SECTION_SUCCESS): information about the server has been successfully read; - Common failures (SECTION_FAILURE_COMMON). | Name | Input | Output | | ---- | ----- | ------ | | LocalNodeInfo | [LocalNodeInfoRequest](#frost.fs.netmap.LocalNodeInfoRequest) | [LocalNodeInfoResponse](#frost.fs.netmap.LocalNodeInfoResponse) | #### Method NetworkInfo Read recent information about the FrostFS network. Statuses: - **OK** (0, SECTION_SUCCESS): information about the current network state has been successfully read; - Common failures (SECTION_FAILURE_COMMON). | Name | Input | Output | | ---- | ----- | ------ | | NetworkInfo | [NetworkInfoRequest](#frost.fs.netmap.NetworkInfoRequest) | [NetworkInfoResponse](#frost.fs.netmap.NetworkInfoResponse) | #### Method NetmapSnapshot Returns network map snapshot of the current FrostFS epoch. Statuses: - **OK** (0, SECTION_SUCCESS): information about the current network map has been successfully read; - Common failures (SECTION_FAILURE_COMMON). | Name | Input | Output | | ---- | ----- | ------ | | NetmapSnapshot | [NetmapSnapshotRequest](#frost.fs.netmap.NetmapSnapshotRequest) | [NetmapSnapshotResponse](#frost.fs.netmap.NetmapSnapshotResponse) | ### Message LocalNodeInfoRequest Get NodeInfo structure directly from a particular node | Field | Type | Label | Description | | ----- | ---- | ----- | ----------- | | body | [LocalNodeInfoRequest.Body](#frost.fs.netmap.LocalNodeInfoRequest.Body) | | Body of the LocalNodeInfo request message | | meta_header | [frost.fs.session.RequestMetaHeader](#frost.fs.session.RequestMetaHeader) | | Carries request meta information. Header data is used only to regulate message transport and does not affect request execution. | | verify_header | [frost.fs.session.RequestVerificationHeader](#frost.fs.session.RequestVerificationHeader) | | Carries request verification information. This header is used to authenticate the nodes of the message route and check the correctness of transmission. | ### Message LocalNodeInfoRequest.Body LocalNodeInfo request body is empty. ### Message LocalNodeInfoResponse Local Node Info, including API Version in use | Field | Type | Label | Description | | ----- | ---- | ----- | ----------- | | body | [LocalNodeInfoResponse.Body](#frost.fs.netmap.LocalNodeInfoResponse.Body) | | Body of the balance response message. | | meta_header | [frost.fs.session.ResponseMetaHeader](#frost.fs.session.ResponseMetaHeader) | | Carries response meta information. Header data is used only to regulate message transport and does not affect response execution. | | verify_header | [frost.fs.session.ResponseVerificationHeader](#frost.fs.session.ResponseVerificationHeader) | | Carries response verification information. This header is used to authenticate the nodes of the message route and check the correctness of transmission. | ### Message LocalNodeInfoResponse.Body Local Node Info, including API Version in use. | Field | Type | Label | Description | | ----- | ---- | ----- | ----------- | | version | [frost.fs.refs.Version](#frost.fs.refs.Version) | | Latest FrostFS API version in use | | node_info | [NodeInfo](#frost.fs.netmap.NodeInfo) | | NodeInfo structure with recent information from node itself | ### Message NetmapSnapshotRequest Get netmap snapshot request | Field | Type | Label | Description | | ----- | ---- | ----- | ----------- | | body | [NetmapSnapshotRequest.Body](#frost.fs.netmap.NetmapSnapshotRequest.Body) | | Body of get netmap snapshot request message. | | meta_header | [frost.fs.session.RequestMetaHeader](#frost.fs.session.RequestMetaHeader) | | Carries request meta information. Header data is used only to regulate message transport and does not affect request execution. | | verify_header | [frost.fs.session.RequestVerificationHeader](#frost.fs.session.RequestVerificationHeader) | | Carries request verification information. This header is used to authenticate the nodes of the message route and check the correctness of transmission. | ### Message NetmapSnapshotRequest.Body Get netmap snapshot request body. ### Message NetmapSnapshotResponse Response with current netmap snapshot | Field | Type | Label | Description | | ----- | ---- | ----- | ----------- | | body | [NetmapSnapshotResponse.Body](#frost.fs.netmap.NetmapSnapshotResponse.Body) | | Body of get netmap snapshot response message. | | meta_header | [frost.fs.session.ResponseMetaHeader](#frost.fs.session.ResponseMetaHeader) | | Carries response meta information. Header data is used only to regulate message transport and does not affect response execution. | | verify_header | [frost.fs.session.ResponseVerificationHeader](#frost.fs.session.ResponseVerificationHeader) | | Carries response verification information. This header is used to authenticate the nodes of the message route and check the correctness of transmission. | ### Message NetmapSnapshotResponse.Body Get netmap snapshot response body | Field | Type | Label | Description | | ----- | ---- | ----- | ----------- | | netmap | [Netmap](#frost.fs.netmap.Netmap) | | Structure of the requested network map. | ### Message NetworkInfoRequest Get NetworkInfo structure with the network view from a particular node. | Field | Type | Label | Description | | ----- | ---- | ----- | ----------- | | body | [NetworkInfoRequest.Body](#frost.fs.netmap.NetworkInfoRequest.Body) | | Body of the NetworkInfo request message | | meta_header | [frost.fs.session.RequestMetaHeader](#frost.fs.session.RequestMetaHeader) | | Carries request meta information. Header data is used only to regulate message transport and does not affect request execution. | | verify_header | [frost.fs.session.RequestVerificationHeader](#frost.fs.session.RequestVerificationHeader) | | Carries request verification information. This header is used to authenticate the nodes of the message route and check the correctness of transmission. | ### Message NetworkInfoRequest.Body NetworkInfo request body is empty. ### Message NetworkInfoResponse Response with NetworkInfo structure including current epoch and sidechain magic number. | Field | Type | Label | Description | | ----- | ---- | ----- | ----------- | | body | [NetworkInfoResponse.Body](#frost.fs.netmap.NetworkInfoResponse.Body) | | Body of the NetworkInfo response message. | | meta_header | [frost.fs.session.ResponseMetaHeader](#frost.fs.session.ResponseMetaHeader) | | Carries response meta information. Header data is used only to regulate message transport and does not affect response execution. | | verify_header | [frost.fs.session.ResponseVerificationHeader](#frost.fs.session.ResponseVerificationHeader) | | Carries response verification information. This header is used to authenticate the nodes of the message route and check the correctness of transmission. | ### Message NetworkInfoResponse.Body Information about the network. | Field | Type | Label | Description | | ----- | ---- | ----- | ----------- | | network_info | [NetworkInfo](#frost.fs.netmap.NetworkInfo) | | NetworkInfo structure with recent information. |

Top

## netmap/types.proto ### Message Filter This filter will return the subset of nodes from `NetworkMap` or another filter's results that will satisfy filter's conditions. | Field | Type | Label | Description | | ----- | ---- | ----- | ----------- | | name | [string](#string) | | Name of the filter or a reference to a named filter. '*' means application to the whole unfiltered NetworkMap. At top level it's used as a filter name. At lower levels it's considered to be a reference to another named filter | | key | [string](#string) | | Key to filter | | op | [Operation](#frost.fs.netmap.Operation) | | Filtering operation | | value | [string](#string) | | Value to match | | filters | [Filter](#frost.fs.netmap.Filter) | repeated | List of inner filters. Top level operation will be applied to the whole list. | ### Message Netmap Network map structure | Field | Type | Label | Description | | ----- | ---- | ----- | ----------- | | epoch | [uint64](#uint64) | | Network map revision number. | | nodes | [NodeInfo](#frost.fs.netmap.NodeInfo) | repeated | Nodes presented in network. | ### Message NetworkConfig FrostFS network configuration | Field | Type | Label | Description | | ----- | ---- | ----- | ----------- | | parameters | [NetworkConfig.Parameter](#frost.fs.netmap.NetworkConfig.Parameter) | repeated | List of parameter values | ### Message NetworkConfig.Parameter Single configuration parameter. Key MUST be network-unique. System parameters: - **AuditFee** \ Fee paid by the storage group owner to the Inner Ring member. Value: little-endian integer. Default: 0. - **BasicIncomeRate** \ Cost of storing one gigabyte of data for a period of one epoch. Paid by container owner to container nodes. Value: little-endian integer. Default: 0. - **ContainerAliasFee** \ Fee paid for named container's creation by the container owner. Value: little-endian integer. Default: 0. - **ContainerFee** \ Fee paid for container creation by the container owner. Value: little-endian integer. Default: 0. - **EpochDuration** \ FrostFS epoch duration measured in Sidechain blocks. Value: little-endian integer. Default: 0. - **HomomorphicHashingDisabled** \ Flag of disabling the homomorphic hashing of objects' payload. Value: true if any byte != 0. Default: false. - **InnerRingCandidateFee** \ Fee for entrance to the Inner Ring paid by the candidate. Value: little-endian integer. Default: 0. - **MaintenanceModeAllowed** \ Flag allowing setting the MAINTENANCE state to storage nodes. Value: true if any byte != 0. Default: false. - **MaxObjectSize** \ Maximum size of physically stored FrostFS object measured in bytes. Value: little-endian integer. Default: 0. This value refers to the maximum size of a **physically** stored object in FrostFS. However, from a user's perspective, the **logical** size of a stored object can be significantly larger. The relationship between the physical and logical object sizes is governed by the following formula ```math \mathrm{Stored\ Object\ Size} \le \frac{ \left(\mathrm{Max\ Object\ Size}\right)^2 }{ \mathrm{Object\ ID\ Size} } ``` This arises from the fact that a tombstone, also being an object, stores the IDs of inhumed objects and cannot be divided into smaller objects, thus having an upper limit for its size. For example, if: * Max Object Size Size = 64 MiB; * Object ID Size = 32 B; then: ```math \mathrm{Stored\ Object\ Size} \le \frac{\left(64\ \mathrm{MiB}\right)^2}{32\ \mathrm{B}} = \frac{2^{52}}{2^5}\ \mathrm{B} = 2^{47}\ \mathrm{B} = 128\ \mathrm{TiB} ``` - **WithdrawFee** \ Fee paid for withdrawal of funds paid by the account owner. Value: little-endian integer. Default: 0. - **MaxECDataCount** \ Maximum number of data shards for EC placement policy. Value: little-endian integer. Default: 0. - **MaxECParityCount** \ Maximum number of parity shards for EC placement policy. Value: little-endian integer. Default: 0. | Field | Type | Label | Description | | ----- | ---- | ----- | ----------- | | key | [bytes](#bytes) | | Parameter key. UTF-8 encoded string | | value | [bytes](#bytes) | | Parameter value | ### Message NetworkInfo Information about FrostFS network | Field | Type | Label | Description | | ----- | ---- | ----- | ----------- | | current_epoch | [uint64](#uint64) | | Number of the current epoch in the FrostFS network | | magic_number | [uint64](#uint64) | | Magic number of the sidechain of the FrostFS network | | ms_per_block | [int64](#int64) | | MillisecondsPerBlock network parameter of the sidechain of the FrostFS network | | network_config | [NetworkConfig](#frost.fs.netmap.NetworkConfig) | | FrostFS network configuration | ### Message NodeInfo FrostFS node description | Field | Type | Label | Description | | ----- | ---- | ----- | ----------- | | public_key | [bytes](#bytes) | | Public key of the FrostFS node in a binary format | | addresses | [string](#string) | repeated | Ways to connect to a node | | attributes | [NodeInfo.Attribute](#frost.fs.netmap.NodeInfo.Attribute) | repeated | Carries list of the FrostFS node attributes in a key-value form. Key name must be a node-unique valid UTF-8 string. Value can't be empty. NodeInfo structures with duplicated attribute names or attributes with empty values will be considered invalid. | | state | [NodeInfo.State](#frost.fs.netmap.NodeInfo.State) | | Carries state of the FrostFS node | ### Message NodeInfo.Attribute Administrator-defined Attributes of the FrostFS Storage Node. `Attribute` is a Key-Value metadata pair. Key name must be a valid UTF-8 string. Value can't be empty. Attributes can be constructed into a chain of attributes: any attribute can have a parent attribute and a child attribute (except the first and the last one). A string representation of the chain of attributes in FrostFS Storage Node configuration uses ":" and "/" symbols, e.g.: `FrostFS_NODE_ATTRIBUTE_1=key1:val1/key2:val2` Therefore the string attribute representation in the Node configuration must use "\:", "\/" and "\\" escaped symbols if any of them appears in an attribute's key or value. Node's attributes are mostly used during Storage Policy evaluation to calculate object's placement and find a set of nodes satisfying policy requirements. There are some "well-known" node attributes common to all the Storage Nodes in the network and used implicitly with default values if not explicitly set: * Capacity \ Total available disk space in Gigabytes. * Price \ Price in GAS tokens for storing one GB of data during one Epoch. In node attributes it's a string presenting floating point number with comma or point delimiter for decimal part. In the Network Map it will be saved as 64-bit unsigned integer representing number of minimal token fractions. * UN-LOCODE \ Node's geographic location in [UN/LOCODE](https://www.unece.org/cefact/codesfortrade/codes_index.html) format approximated to the nearest point defined in the standard. * CountryCode \ Country code in [ISO 3166-1_alpha-2](https://en.wikipedia.org/wiki/ISO_3166-1_alpha-2) format. Calculated automatically from `UN-LOCODE` attribute. * Country \ Country short name in English, as defined in [ISO-3166](https://www.iso.org/obp/ui/#search). Calculated automatically from `UN-LOCODE` attribute. * Location \ Place names are given, whenever possible, in their national language versions as expressed in the Roman alphabet using the 26 characters of the character set adopted for international trade data interchange, written without diacritics . Calculated automatically from `UN-LOCODE` attribute. * SubDivCode \ Country's administrative subdivision where node is located. Calculated automatically from `UN-LOCODE` attribute based on `SubDiv` field. Presented in [ISO 3166-2](https://en.wikipedia.org/wiki/ISO_3166-2) format. * SubDiv \ Country's administrative subdivision name, as defined in [ISO 3166-2](https://en.wikipedia.org/wiki/ISO_3166-2). Calculated automatically from `UN-LOCODE` attribute. * Continent \ Node's continent name according to the [Seven-Continent model](https://en.wikipedia.org/wiki/Continent#Number). Calculated automatically from `UN-LOCODE` attribute. * ExternalAddr Node's preferred way for communications with external clients. Clients SHOULD use these addresses if possible. Must contain a comma-separated list of multi-addresses. For detailed description of each well-known attribute please see the corresponding section in FrostFS Technical Specification. | Field | Type | Label | Description | | ----- | ---- | ----- | ----------- | | key | [string](#string) | | Key of the node attribute | | value | [string](#string) | | Value of the node attribute | | parents | [string](#string) | repeated | Parent keys, if any. For example for `City` it could be `Region` and `Country`. | ### Message PlacementPolicy Set of rules to select a subset of nodes from `NetworkMap` able to store container's objects. The format is simple enough to transpile from different storage policy definition languages. | Field | Type | Label | Description | | ----- | ---- | ----- | ----------- | | replicas | [Replica](#frost.fs.netmap.Replica) | repeated | Rules to set number of object replicas and place each one into a named bucket | | container_backup_factor | [uint32](#uint32) | | Container backup factor controls how deep FrostFS will search for nodes alternatives to include into container's nodes subset | | selectors | [Selector](#frost.fs.netmap.Selector) | repeated | Set of Selectors to form the container's nodes subset | | filters | [Filter](#frost.fs.netmap.Filter) | repeated | List of named filters to reference in selectors | | unique | [bool](#bool) | | Unique flag defines non-overlapping application for replicas | ### Message Replica Number of object replicas in a set of nodes from the defined selector. If no selector set, the root bucket containing all possible nodes will be used by default. | Field | Type | Label | Description | | ----- | ---- | ----- | ----------- | | count | [uint32](#uint32) | | How many object replicas to put | | selector | [string](#string) | | Named selector bucket to put replicas | | ec_data_count | [uint32](#uint32) | | Data shards count | | ec_parity_count | [uint32](#uint32) | | Parity shards count | ### Message Selector Selector chooses a number of nodes from the bucket taking the nearest nodes to the provided `ContainerID` by hash distance. | Field | Type | Label | Description | | ----- | ---- | ----- | ----------- | | name | [string](#string) | | Selector name to reference in object placement section | | count | [uint32](#uint32) | | How many nodes to select from the bucket | | clause | [Clause](#frost.fs.netmap.Clause) | | Selector modifier showing how to form a bucket | | attribute | [string](#string) | | Bucket attribute to select from | | filter | [string](#string) | | Filter reference to select from | ### Clause Selector modifier shows how the node set will be formed. By default selector just groups nodes into a bucket by attribute, selecting nodes only by their hash distance. | Name | Number | Description | | ---- | ------ | ----------- | | CLAUSE_UNSPECIFIED | 0 | No modifier defined. Nodes will be selected from the bucket randomly | | SAME | 1 | SAME will select only nodes having the same value of bucket attribute | | DISTINCT | 2 | DISTINCT will select nodes having different values of bucket attribute | ### NodeInfo.State Represents the enumeration of various states of the FrostFS node. | Name | Number | Description | | ---- | ------ | ----------- | | UNSPECIFIED | 0 | Unknown state | | ONLINE | 1 | Active state in the network | | OFFLINE | 2 | Network unavailable state | | MAINTENANCE | 3 | Maintenance state | ### Operation Operations on filters | Name | Number | Description | | ---- | ------ | ----------- | | OPERATION_UNSPECIFIED | 0 | No Operation defined | | EQ | 1 | Equal | | NE | 2 | Not Equal | | GT | 3 | Greater then | | GE | 4 | Greater or equal | | LT | 5 | Less then | | LE | 6 | Less or equal | | OR | 7 | Logical OR | | AND | 8 | Logical AND | | NOT | 9 | Logical negation | | LIKE | 10 | Matches pattern | ## Scalar Value Types | .proto Type | Notes | C++ Type | Java Type | Python Type | | ----------- | ----- | -------- | --------- | ----------- | | double | | double | double | float | | float | | float | float | float | | int32 | Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint32 instead. | int32 | int | int | | int64 | Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint64 instead. | int64 | long | int/long | | uint32 | Uses variable-length encoding. | uint32 | int | int/long | | uint64 | Uses variable-length encoding. | uint64 | long | int/long | | sint32 | Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int32s. | int32 | int | int | | sint64 | Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int64s. | int64 | long | int/long | | fixed32 | Always four bytes. More efficient than uint32 if values are often greater than 2^28. | uint32 | int | int | | fixed64 | Always eight bytes. More efficient than uint64 if values are often greater than 2^56. | uint64 | long | int/long | | sfixed32 | Always four bytes. | int32 | int | int | | sfixed64 | Always eight bytes. | int64 | long | int/long | | bool | | bool | boolean | boolean | | string | A string must always contain UTF-8 encoded or 7-bit ASCII text. | string | String | str/unicode | | bytes | May contain any arbitrary sequence of bytes. | string | ByteString | str |