diff --git a/CHANGELOG.md b/CHANGELOG.md index 4a82f44..a4cddb2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,11 +1,47 @@ # Changelog +## [2.3.0] - 2021-02-11 - Seonyudo (선유도, 仙遊島) + +This release brings support for nodes to exchange information about disk space +used by each Container. This information will be used by Inner Ring nodes to +calculate basic rewards and payments. + +Another significant change is UN/LOCODE support for node's attributes describing +geographical location. From now on, most of the geographical attributes will be +calculated automatically from a single `UN-LOCODE` attribute in a deterministic +manner. + +### Added + +- Added `container.AnnounceUsedSpace` request for announcing disk space consumed + by container's objects on the node +- Added `Continent` well-known node's attribute +- Added `SubDivCode` well-known node's attribute +- Added `Location` well-known node's attribute +- Added `CounrtyCode` well-known node's attribute +- Added `STRING_NOT_EQUAL` match type +- Added `NOT_PRESENT` match type +- Added JSON names for search request filter fields + +### Changed + +- `Locode` well-known node's attribute renamed to `UN-LOCODE`. It will be used + as a base for calculating most of the node's geographical attributes. +- `Region` well-known node's attribute renamed to `SubDiv` + +### Removed + +- Removed `City` well-known node's attribute +- Removed `Region` well-known node's attribute + ## [2.2.1] - 2021-01-15 Define "well-known" X-headers ### Added -- Description of the format of "well-known" X-headers that affect system behavior + +- Description of the format of "well-known" X-headers that affect system + behavior - X-header key to netmap epoch value - X-header key to netmap lookup depth value @@ -14,9 +50,11 @@ Define "well-known" X-headers Storage Groups based Data Audit updates ### Added + - `audit.DataAuditResult` message for recording audit result is added ### Changed + - `object.ShortHeader` now has `payload_hash` and `homomorphic_hash` fields ## [2.1.1] - 2020-12-17 @@ -24,6 +62,7 @@ Storage Groups based Data Audit updates Minor documentation fixes ### Changed + - Clarify JSON encoding for `ObjectID`, `ContainerID` and `OwnerID` - Clarify object field usage in some requests' eACL filters @@ -32,6 +71,7 @@ Minor documentation fixes Object split and deletion improvements, documentation clarifications. ### Added + - `$Object:objectID` added to the list of available ACL and Search filters - `split_id` field added in `object.Object.header` - `$Object:split.splitID` search filter added @@ -47,10 +87,12 @@ Object split and deletion improvements, documentation clarifications. - `raw` flag added in `object.GetRangeRequest.Body` ### Changed + - Clarified special search index descriptions - Clarified various types encoding formats descriptions ### Removed + - `$Object:CHILDFREE` filter description removed from well-known list - `$Object:LEAF` filter description removed from well-known list @@ -59,11 +101,14 @@ Object split and deletion improvements, documentation clarifications. More "well-known" application attributes and documentation updates. ### Added + - Added "well-known" attributes list for extended ACL and object search filters - Added `Name`, `Timestamp` "well-known" application attributes for container -- Added `Name`, `FileName`, `Timestamp` "well-known" application attributes for object +- Added `Name`, `FileName`, `Timestamp` "well-known" application attributes for + object ### Changed + - `BearerToken.owner_id` field description changed - `Subnet` container attribute now has `__NEOFS__` prefix - Search and ACL filters now have `key` and `value` fields @@ -73,16 +118,19 @@ More "well-known" application attributes and documentation updates. Documentation updates and JSON field names definition ### Added + - Added "well-known" attributes list for `netmap.NodeInfo` - Added "well-known" attributes list for objects - Added "well-known" attributes list for containers - JSON field names defined for most of data structures ### Changed + - Documentation updated for all packages fixing typos and minor inaccuracies - `acl.EACLRecord.Target.key_list` field renamed to `keys` for consistency ### Removed + - Human-written documentation will now be only in [NeoFS Specification](https://github.com/nspcc-dev/neofs-spec) @@ -91,6 +139,7 @@ Documentation updates and JSON field names definition Major API refactoring and simplification. ### Added + - `neo.fs.v2` prefix added to all package names - `container.Attributes` field added - `refs.ContainerID` added as a separate type @@ -102,26 +151,31 @@ Major API refactoring and simplification. - `refs.Version` defined as a separate type - `refs.Version` field added to all messages stored in SmartContracts - `refs.Checksum` defined as a separate type -- `netmap.LocalNodeInfo` request added to get actual information from connected peer +- `netmap.LocalNodeInfo` request added to get actual information from connected + peer ### Changed + - Extended ACL Table format changed - Protobuf definitions style changed to follow Google Style Guide - `System` and `Extended` Object headers are merged into on `object.Header` type - `object.UserHeader` renamed to `object.Header.Attribute` -- `refs.ObjectID` is now a hash of the `object.Header` field, which contains hash of payload +- `refs.ObjectID` is now a hash of the `object.Header` field, which contains + hash of payload - `StorageGroup` information moved to Object's payload -- `netmap.NodeInfo.options` renamed to `netmap.NodeInfo.attributes` and it uses a - separate `netmap.NodeInfo.Attribute` type now. +- `netmap.NodeInfo.options` renamed to `netmap.NodeInfo.attributes` and it uses + a separate `netmap.NodeInfo.Attribute` type now. - `netmap.NodeInfo.Attribute` type now has a list of parents to construct a tree - Session Token renamed to `session.SessionToken` from `session.Token` - All Requests and Responses now have a common "body-meta-verify" structure - Meta and Verification headers now follow Matryoshka-style composition - SessionToken and BearerToken are now part of Meta header - Object placement policy format is simplified and defined in `netmap` package -- `object.Head()` request now returns either short header or full header with a signature +- `object.Head()` request now returns either short header or full header with a + signature ### Removed + - gogoproto is not used anymore - `decimal` package merged into `accounting` package - `query` package merged into `object` package @@ -142,7 +196,8 @@ Major API refactoring and simplification. ### Added -- ```acl.EACLRecord```, ```acl.EACLTable``` messages for the table of extended ACL rules. +- ```acl.EACLRecord```, ```acl.EACLTable``` messages for the table of extended + ACL rules. ## [1.1.0] - 2020-06-18 @@ -201,7 +256,8 @@ Bump major release ### Removed - `Raw` field from `object.GetRequest` and `object.HeadRequest` messages. -- `Token` field from `object.PutRequest.PutHeader` and `object.DeleteRequest` messages. +- `Token` field from `object.PutRequest.PutHeader` and `object.DeleteRequest` + messages. - `VerificationHeader` message. ## [0.7.1] - 2020-04-20 @@ -256,3 +312,4 @@ Bump major release [2.1.1]: https://github.com/nspcc-dev/neofs-api/compare/v2.1.0...v2.1.1 [2.2.0]: https://github.com/nspcc-dev/neofs-api/compare/v2.1.1...v2.2.0 [2.2.1]: https://github.com/nspcc-dev/neofs-api/compare/v2.2.0...v2.2.1 +[2.3.0]: https://github.com/nspcc-dev/neofs-api/compare/v2.2.1...v2.3.0 diff --git a/proto-docs/container.md b/proto-docs/container.md index c5dbe82..6c7d9dd 100644 --- a/proto-docs/container.md +++ b/proto-docs/container.md @@ -8,6 +8,11 @@ - [ContainerService](#neo.fs.v2.container.ContainerService) - Messages + - [AnnounceUsedSpaceRequest](#neo.fs.v2.container.AnnounceUsedSpaceRequest) + - [AnnounceUsedSpaceRequest.Body](#neo.fs.v2.container.AnnounceUsedSpaceRequest.Body) + - [AnnounceUsedSpaceRequest.Body.Announcement](#neo.fs.v2.container.AnnounceUsedSpaceRequest.Body.Announcement) + - [AnnounceUsedSpaceResponse](#neo.fs.v2.container.AnnounceUsedSpaceResponse) + - [AnnounceUsedSpaceResponse.Body](#neo.fs.v2.container.AnnounceUsedSpaceResponse.Body) - [DeleteRequest](#neo.fs.v2.container.DeleteRequest) - [DeleteRequest.Body](#neo.fs.v2.container.DeleteRequest.Body) - [DeleteResponse](#neo.fs.v2.container.DeleteResponse) @@ -68,6 +73,7 @@ rpc Get(GetRequest) returns (GetResponse); rpc List(ListRequest) returns (ListResponse); rpc SetExtendedACL(SetExtendedACLRequest) returns (SetExtendedACLResponse); rpc GetExtendedACL(GetExtendedACLRequest) returns (GetExtendedACLResponse); +rpc AnnounceUsedSpace(AnnounceUsedSpaceRequest) returns (AnnounceUsedSpaceResponse); ``` @@ -122,9 +128,74 @@ storage. | Name | Input | Output | | ---- | ----- | ------ | | GetExtendedACL | [GetExtendedACLRequest](#neo.fs.v2.container.GetExtendedACLRequest) | [GetExtendedACLResponse](#neo.fs.v2.container.GetExtendedACLResponse) | +#### Method AnnounceUsedSpace + +Announce container used space values for P2P synchronization. + +| Name | Input | Output | +| ---- | ----- | ------ | +| AnnounceUsedSpace | [AnnounceUsedSpaceRequest](#neo.fs.v2.container.AnnounceUsedSpaceRequest) | [AnnounceUsedSpaceResponse](#neo.fs.v2.container.AnnounceUsedSpaceResponse) | + + +### Message AnnounceUsedSpaceRequest +Announce container used space + + +| Field | Type | Label | Description | +| ----- | ---- | ----- | ----------- | +| body | [AnnounceUsedSpaceRequest.Body](#neo.fs.v2.container.AnnounceUsedSpaceRequest.Body) | | Body of announce used space request message. | +| meta_header | [neo.fs.v2.session.RequestMetaHeader](#neo.fs.v2.session.RequestMetaHeader) | | Carries request meta information. Header data is used only to regulate message transport and does not affect request execution. | +| verify_header | [neo.fs.v2.session.RequestVerificationHeader](#neo.fs.v2.session.RequestVerificationHeader) | | Carries request verification information. This header is used to authenticate the nodes of the message route and check the correctness of transmission. | + + + + +### Message AnnounceUsedSpaceRequest.Body +Container used space announcement body. + + +| Field | Type | Label | Description | +| ----- | ---- | ----- | ----------- | +| announcements | [AnnounceUsedSpaceRequest.Body.Announcement](#neo.fs.v2.container.AnnounceUsedSpaceRequest.Body.Announcement) | repeated | List of announcements. If nodes share several containers, then announcements transferred in a batch. | + + + + +### Message AnnounceUsedSpaceRequest.Body.Announcement +Announcement contains used space information about single container. + + +| Field | Type | Label | Description | +| ----- | ---- | ----- | ----------- | +| epoch | [uint64](#uint64) | | Epoch number for which container size estimation was produced. | +| container_id | [neo.fs.v2.refs.ContainerID](#neo.fs.v2.refs.ContainerID) | | Identifier of the container. | +| used_space | [uint64](#uint64) | | Used space is a sum of object payload sizes of specified container, stored in the node. It must not include inhumed objects. | + + + + +### Message AnnounceUsedSpaceResponse +Announce container used space + + +| Field | Type | Label | Description | +| ----- | ---- | ----- | ----------- | +| body | [AnnounceUsedSpaceResponse.Body](#neo.fs.v2.container.AnnounceUsedSpaceResponse.Body) | | Body of announce used space response message. | +| meta_header | [neo.fs.v2.session.ResponseMetaHeader](#neo.fs.v2.session.ResponseMetaHeader) | | Carries response meta information. Header data is used only to regulate message transport and does not affect request execution. | +| verify_header | [neo.fs.v2.session.ResponseVerificationHeader](#neo.fs.v2.session.ResponseVerificationHeader) | | Carries response verification information. This header is used to authenticate the nodes of the message route and check the correctness of transmission. | + + + + +### Message AnnounceUsedSpaceResponse.Body +`AnnounceUsedSpaceResponse` has an empty body because announcements are +one way communication. + + + ### Message DeleteRequest diff --git a/proto-docs/netmap.md b/proto-docs/netmap.md index cc27c5a..ab0cde6 100644 --- a/proto-docs/netmap.md +++ b/proto-docs/netmap.md @@ -177,21 +177,37 @@ explicitly set: * Subnet \ String ID of Node's storage subnet. There can be only one subnet served by the Storage Node. -* Locode \ +* UN-LOCODE \ Node's geographic location in [UN/LOCODE](https://www.unece.org/cefact/codesfortrade/codes_index.html) - format approximated to the nearest point defined in standard. -* Country \ + format approximated to the nearest point defined in the standard. +* CountryCode \ Country code in [ISO 3166-1_alpha-2](https://en.wikipedia.org/wiki/ISO_3166-1_alpha-2) - format. Calculated automatically from `Locode` attribute -* Region \ - Country's administative subdivision where node is located. Calculated - automatically from `Locode` attribute based on `SubDiv` field. Presented - in [ISO 3166-2](https://en.wikipedia.org/wiki/ISO_3166-2) format. -* City \ - City, town, village or rural area name where node is located written - without diacritics . Calculated automatically from `Locode` attribute. + format. Calculated automatically from `UN-LOCODE` attribute. +* Country \ + Country short name in English, as defined in + [ISO-3166](https://www.iso.org/obp/ui/#search). Calculated automatically + from `UN-LOCODE` attribute. +* Location \ + Place names are given, whenever possible, in their national language + versions as expressed in the Roman alphabet using the 26 characters of + the character set adopted for international trade data interchange, + written without diacritics . Calculated automatically from `UN-LOCODE` + attribute. +* SubDivCode \ + Country's administrative subdivision where node is located. Calculated + automatically from `UN-LOCODE` attribute based on `SubDiv` field. + Presented in [ISO 3166-2](https://en.wikipedia.org/wiki/ISO_3166-2) + format. +* SubDiv \ + Country's administrative subdivision name, as defined in + [ISO 3166-2](https://en.wikipedia.org/wiki/ISO_3166-2). Calculated + automatically from `UN-LOCODE` attribute. +* Continent \ + Node's continent name according to the [Seven-Continent model] + (https://en.wikipedia.org/wiki/Continent#Number). Calculated + automatically from `UN-LOCODE` attribute. For detailed description of each well-known attribute please see the corresponding section in NeoFS Technical specification. diff --git a/proto-docs/object.md b/proto-docs/object.md index fdf3001..ab07679 100644 --- a/proto-docs/object.md +++ b/proto-docs/object.md @@ -597,6 +597,10 @@ properties: Returns only objects physically stored in the system. This filter is activated if the `key` exists, disregarding the value and matcher type. +Note: using filters with a key with prefix `$Object:` and match type +`NOT_PRESENT `is not recommended since this is not a cross-version approach. +Behavior when processing this kind of filters is undefined. + | Field | Type | Label | Description | | ----- | ---- | ----- | ----------- | @@ -780,6 +784,8 @@ Type of match expression | ---- | ------ | ----------- | | MATCH_TYPE_UNSPECIFIED | 0 | Unknown. Not used | | STRING_EQUAL | 1 | Full string match | +| STRING_NOT_EQUAL | 2 | Full string mismatch | +| NOT_PRESENT | 3 | Lack of key | diff --git a/proto-docs/session.md b/proto-docs/session.md index e1340c9..f2324f4 100644 --- a/proto-docs/session.md +++ b/proto-docs/session.md @@ -84,7 +84,7 @@ Session creation request body | Field | Type | Label | Description | | ----- | ---- | ----- | ----------- | -| owner_id | [neo.fs.v2.refs.OwnerID](#neo.fs.v2.refs.OwnerID) | | Dession initiating user's or node's key derived `OwnerID`. | +| owner_id | [neo.fs.v2.refs.OwnerID](#neo.fs.v2.refs.OwnerID) | | Session initiating user's or node's key derived `OwnerID` | | expiration | [uint64](#uint64) | | Session expiration `Epoch` | @@ -243,7 +243,25 @@ Lifetime parameters of the token. Field names taken from rfc7519. ### Message XHeader -Extended headers for Request/Response. +Extended headers for Request/Response. May contain any user-defined headers +to be interpreted on application level. + +Key name must be unique valid UTF-8 string. Value can't be empty. Requests or +Responses with duplicated header names or headers with empty values will be +considered invalid. + +There are some "well-known" headers starting with `__NEOFS__` prefix that +affect system behaviour: + +* __NEOFS__NETMAP_EPOCH \ + Netmap epoch to use for object placement calculation. The `value` is string + encoded `uint64` in decimal presentation. If set to '0' or not set, the + current epoch only will be used. +* __NEOFS__NETMAP_LOOKUP_DEPTH \ + If object can't be found using current epoch's netmap, this header limits + how many past epochs back the node can lookup. The `value` is string + encoded `uint64` in decimal presentation. If set to '0' or not set, the + current epoch only will be used. | Field | Type | Label | Description |