Airat Arifullin cfb148ea0c [#48 ] object: Introduce parent_split_id and parent_index for EC

* Introduce new fields for `EC`-header for `Object` service;
* Regenerate docs.

Signed-off-by: Airat Arifullin <aarifullin@yadro.com>

2024-05-08 15:02:23 +03:00

53 KiB

Raw Blame History

Protocol Documentation

object/service.proto
- Services
  - ObjectService
- Messages
object/types.proto
- Messages
  - ECInfo
  - ECInfo.Chunk
  - Header
  - Header.Attribute
  - Header.EC
  - Header.Split
  - Object
  - ShortHeader
  - SplitInfo
Scalar Value Types

Top

object/service.proto

Service "neo.fs.v2.object.ObjectService"

ObjectService provides API for manipulating objects. Object operations do not affect the sidechain and are only served by nodes in p2p style.

rpc Get(GetRequest) returns (stream GetResponse);
rpc Put(stream PutRequest) returns (PutResponse);
rpc Delete(DeleteRequest) returns (DeleteResponse);
rpc Head(HeadRequest) returns (HeadResponse);
rpc Search(SearchRequest) returns (stream SearchResponse);
rpc GetRange(GetRangeRequest) returns (stream GetRangeResponse);
rpc GetRangeHash(GetRangeHashRequest) returns (GetRangeHashResponse);
rpc PutSingle(PutSingleRequest) returns (PutSingleResponse);

Method Get

Receive full object structure, including Headers and payload. Response uses gRPC stream. First response message carries the object with the requested address. Chunk messages are parts of the object's payload if it is needed. All messages, except the first one, carry payload chunks. The requested object can be restored by concatenation of object message payload and all chunks keeping the receiving order.

Extended headers can change Get behaviour:

[ __SYSTEM__NETMAP_EPOCH ]
(__NEOFS__NETMAP_EPOCH is deprecated)
Will use the requsted version of Network Map for object placement calculation.
[ __SYSTEM__NETMAP_LOOKUP_DEPTH ]
(__NEOFS__NETMAP_LOOKUP_DEPTH is deprecated)
Will try older versions (starting from __SYSTEM__NETMAP_EPOCH (__NEOFS__NETMAP_EPOCH is deprecated) if specified or the latest one otherwise) of Network Map to find an object until the depth limit is reached.

Please refer to detailed XHeader description.

Statuses:

OK (0, SECTION_SUCCESS):
object has been successfully read;
Common failures (SECTION_FAILURE_COMMON);
ACCESS_DENIED (2048, SECTION_OBJECT):
read access to the object is denied;
OBJECT_NOT_FOUND (2049, SECTION_OBJECT):
object not found in container;
OBJECT_ALREADY_REMOVED (2052, SECTION_OBJECT):
the requested object has been marked as deleted;
CONTAINER_NOT_FOUND (3072, SECTION_CONTAINER):
object container not found;
CONTAINER_ACCESS_DENIED (3074, SECTION_CONTAINER):
access to container is denied;
TOKEN_EXPIRED (4097, SECTION_SESSION):
provided session token has expired.

Name	Input	Output
Get	GetRequest	GetResponse

Method Put

Put the object into container. Request uses gRPC stream. First message SHOULD be of PutHeader type. ContainerID and OwnerID of an object SHOULD be set. Session token SHOULD be obtained before PUT operation (see session package). Chunk messages are considered by server as a part of an object payload. All messages, except first one, SHOULD be payload chunks. Chunk messages SHOULD be sent in the direct order of fragmentation.

Extended headers can change Put behaviour:

[ __SYSTEM__NETMAP_EPOCH
(__NEOFS__NETMAP_EPOCH is deprecated)
Will use the requsted version of Network Map for object placement calculation.

Please refer to detailed XHeader description.

Statuses:

OK (0, SECTION_SUCCESS):
object has been successfully saved in the container;
Common failures (SECTION_FAILURE_COMMON);
ACCESS_DENIED (2048, SECTION_OBJECT):
write access to the container is denied;
LOCKED (2050, SECTION_OBJECT):
placement of an object of type TOMBSTONE that includes at least one locked object is prohibited;
LOCK_NON_REGULAR_OBJECT (2051, SECTION_OBJECT):
placement of an object of type LOCK that includes at least one object of type other than REGULAR is prohibited;
CONTAINER_NOT_FOUND (3072, SECTION_CONTAINER):
object storage container not found;
CONTAINER_ACCESS_DENIED (3074, SECTION_CONTAINER):
access to container is denied;
TOKEN_NOT_FOUND (4096, SECTION_SESSION):
(for trusted object preparation) session private key does not exist or has been deleted;
TOKEN_EXPIRED (4097, SECTION_SESSION):
provided session token has expired.

Name	Input	Output
Put	PutRequest	PutResponse

Method Delete

Delete the object from a container. There is no immediate removal guarantee. Object will be marked for removal and deleted eventually.

Extended headers can change Delete behaviour:

[ __SYSTEM__NETMAP_EPOCH ]
(__NEOFS__NETMAP_EPOCH is deprecated)
Will use the requested version of Network Map for object placement calculation.

Please refer to detailed XHeader description.

Statuses:

OK (0, SECTION_SUCCESS):
object has been successfully marked to be removed from the container;
Common failures (SECTION_FAILURE_COMMON);
ACCESS_DENIED (2048, SECTION_OBJECT):
delete access to the object is denied;
OBJECT_NOT_FOUND (2049, SECTION_OBJECT):
the object could not be deleted because it has not been
found within the container;
LOCKED (2050, SECTION_OBJECT):
deleting a locked object is prohibited;
CONTAINER_NOT_FOUND (3072, SECTION_CONTAINER):
object container not found;
CONTAINER_ACCESS_DENIED (3074, SECTION_CONTAINER):
access to container is denied;
TOKEN_EXPIRED (4097, SECTION_SESSION):
provided session token has expired.

Name	Input	Output
Delete	DeleteRequest	DeleteResponse

Method Head

Returns the object Headers without data payload. By default full header is returned. If main_only request field is set, the short header with only the very minimal information will be returned instead.

Extended headers can change Head behaviour:

[ __SYSTEM__NETMAP_EPOCH ]
(__NEOFS__NETMAP_EPOCH is deprecated)
Will use the requested version of Network Map for object placement calculation.

Please refer to detailed XHeader description.

Statuses:

OK (0, SECTION_SUCCESS):
object header has been successfully read;
Common failures (SECTION_FAILURE_COMMON);
ACCESS_DENIED (2048, SECTION_OBJECT):
access to operation HEAD of the object is denied;
OBJECT_NOT_FOUND (2049, SECTION_OBJECT):
object not found in container;
OBJECT_ALREADY_REMOVED (2052, SECTION_OBJECT):
the requested object has been marked as deleted;
CONTAINER_NOT_FOUND (3072, SECTION_CONTAINER):
object container not found;
CONTAINER_ACCESS_DENIED (3074, SECTION_CONTAINER):
access to container is denied;
TOKEN_EXPIRED (4097, SECTION_SESSION):
provided session token has expired.

Name	Input	Output
Head	HeadRequest	HeadResponse

Method Search

Search objects in container. Search query allows to match by Object Header's filed values. Please see the corresponding NeoFS Technical Specification section for more details.

Extended headers can change Search behaviour:

[ __SYSTEM__NETMAP_EPOCH ]
(__NEOFS__NETMAP_EPOCH is deprecated)
Will use the requested version of Network Map for object placement calculation.

Please refer to detailed XHeader description.

Statuses:

OK (0, SECTION_SUCCESS):
objects have been successfully selected;
Common failures (SECTION_FAILURE_COMMON);
ACCESS_DENIED (2048, SECTION_OBJECT):
access to operation SEARCH of the object is denied;
CONTAINER_NOT_FOUND (3072, SECTION_CONTAINER):
search container not found;
CONTAINER_ACCESS_DENIED (3074, SECTION_CONTAINER):
access to container is denied;
TOKEN_EXPIRED (4097, SECTION_SESSION):
provided session token has expired.

Name	Input	Output
Search	SearchRequest	SearchResponse

Method GetRange

Get byte range of data payload. Range is set as an (offset, length) tuple. Like in Get method, the response uses gRPC stream. Requested range can be restored by concatenation of all received payload chunks keeping the receiving order.

Extended headers can change GetRange behaviour:

[ __SYSTEM__NETMAP_EPOCH ]
(__NEOFS__NETMAP_EPOCH is deprecated)
Will use the requested version of Network Map for object placement calculation.
[ __SYSTEM__NETMAP_LOOKUP_DEPTH ]
(__NEOFS__NETMAP_LOOKUP_DEPTH is deprecated)
Will try older versions of Network Map to find an object until the depth limit is reached.

Please refer to detailed XHeader description.

Statuses:

OK (0, SECTION_SUCCESS):
data range of the object payload has been successfully read;
Common failures (SECTION_FAILURE_COMMON);
ACCESS_DENIED (2048, SECTION_OBJECT):
access to operation RANGE of the object is denied;
OBJECT_NOT_FOUND (2049, SECTION_OBJECT):
object not found in container;
OBJECT_ALREADY_REMOVED (2052, SECTION_OBJECT):
the requested object has been marked as deleted.
OUT_OF_RANGE (2053, SECTION_OBJECT):
the requested range is out of bounds;
CONTAINER_NOT_FOUND (3072, SECTION_CONTAINER):
object container not found;
CONTAINER_ACCESS_DENIED (3074, SECTION_CONTAINER):
access to container is denied;
TOKEN_EXPIRED (4097, SECTION_SESSION):
provided session token has expired.

Name	Input	Output
GetRange	GetRangeRequest	GetRangeResponse

Method GetRangeHash

Returns homomorphic or regular hash of object's payload range after applying XOR operation with the provided salt. Ranges are set of (offset, length) tuples. Hashes order in response corresponds to the ranges order in the request. Note that hash is calculated for XORed data.

Extended headers can change GetRangeHash behaviour:

[ __SYSTEM__NETMAP_EPOCH ]
(__NEOFS__NETMAP_EPOCH is deprecated)
Will use the requested version of Network Map for object placement calculation.
[ __SYSTEM__NETMAP_LOOKUP_DEPTH ]
(__NEOFS__NETMAP_LOOKUP_DEPTH is deprecated)
Will try older versions of Network Map to find an object until the depth limit is reached.

Please refer to detailed XHeader description.

Statuses:

OK (0, SECTION_SUCCESS):
data range of the object payload has been successfully hashed;
Common failures (SECTION_FAILURE_COMMON);
ACCESS_DENIED (2048, SECTION_OBJECT):
access to operation RANGEHASH of the object is denied;
OBJECT_NOT_FOUND (2049, SECTION_OBJECT):
object not found in container;
OUT_OF_RANGE (2053, SECTION_OBJECT):
the requested range is out of bounds;
CONTAINER_NOT_FOUND (3072, SECTION_CONTAINER):
object container not found;
CONTAINER_ACCESS_DENIED (3074, SECTION_CONTAINER):
access to container is denied;
TOKEN_EXPIRED (4097, SECTION_SESSION):
provided session token has expired.

Name	Input	Output
GetRangeHash	GetRangeHashRequest	GetRangeHashResponse

Method PutSingle

Put the prepared object into container. ContainerID, ObjectID, OwnerID, PayloadHash and PayloadLength of an object MUST be set.

Extended headers can change Put behaviour:

[ __SYSTEM__NETMAP_EPOCH
(__NEOFS__NETMAP_EPOCH is deprecated)
Will use the requested version of Network Map for object placement calculation.

Please refer to detailed XHeader description.

Statuses:

OK (0, SECTION_SUCCESS):
object has been successfully saved in the container;
Common failures (SECTION_FAILURE_COMMON);
ACCESS_DENIED (2048, SECTION_OBJECT):
write access to the container is denied;
LOCKED (2050, SECTION_OBJECT):
placement of an object of type TOMBSTONE that includes at least one locked object is prohibited;
LOCK_NON_REGULAR_OBJECT (2051, SECTION_OBJECT):
placement of an object of type LOCK that includes at least one object of type other than REGULAR is prohibited;
CONTAINER_NOT_FOUND (3072, SECTION_CONTAINER):
object storage container not found;
CONTAINER_ACCESS_DENIED (3074, SECTION_CONTAINER):
access to container is denied;
TOKEN_NOT_FOUND (4096, SECTION_SESSION):
(for trusted object preparation) session private key does not exist or has been deleted;
TOKEN_EXPIRED (4097, SECTION_SESSION):
provided session token has expired.

Name	Input	Output
PutSingle	PutSingleRequest	PutSingleResponse

Message DeleteRequest

Object DELETE request

Field	Type	Description
body	DeleteRequest.Body	Body of delete object request message.
meta_header	neo.fs.v2.session.RequestMetaHeader	Carries request meta information. Header data is used only to regulate message transport and does not affect request execution.
verify_header	neo.fs.v2.session.RequestVerificationHeader	Carries request verification information. This header is used to authenticate the nodes of the message route and check the correctness of transmission.

Message DeleteRequest.Body

Object DELETE request body

Field	Type	Label	Description
address	neo.fs.v2.refs.Address		Address of the object to be deleted

Message DeleteResponse

DeleteResponse body is empty because we cannot guarantee permanent object removal in distributed system.

Field	Type	Description
body	DeleteResponse.Body	Body of delete object response message.
meta_header	neo.fs.v2.session.ResponseMetaHeader	Carries response meta information. Header data is used only to regulate message transport and does not affect request execution.
verify_header	neo.fs.v2.session.ResponseVerificationHeader	Carries response verification information. This header is used to authenticate the nodes of the message route and check the correctness of transmission.

Message DeleteResponse.Body

Object DELETE Response has an empty body.

Field	Type	Label	Description
tombstone	neo.fs.v2.refs.Address		Address of the tombstone created for the deleted object

Message GetRangeHashRequest

Get hash of object's payload part

Field	Type	Description
body	GetRangeHashRequest.Body	Body of get range hash object request message.
meta_header	neo.fs.v2.session.RequestMetaHeader	Carries request meta information. Header data is used only to regulate message transport and does not affect request execution.
verify_header	neo.fs.v2.session.RequestVerificationHeader	Carries request verification information. This header is used to authenticate the nodes of the message route and check the correctness of transmission.

Message GetRangeHashRequest.Body

Get hash of object's payload part request body.

Field	Type	Label	Description
address	neo.fs.v2.refs.Address		Address of the object that containing the requested payload range
ranges	Range	repeated	List of object's payload ranges to calculate homomorphic hash
salt	bytes		Binary salt to XOR object's payload ranges before hash calculation
type	neo.fs.v2.refs.ChecksumType		Checksum algorithm type

Message GetRangeHashResponse

Get hash of object's payload part

Field	Type	Description
body	GetRangeHashResponse.Body	Body of get range hash object response message.
meta_header	neo.fs.v2.session.ResponseMetaHeader	Carries response meta information. Header data is used only to regulate message transport and does not affect request execution.
verify_header	neo.fs.v2.session.ResponseVerificationHeader	Carries response verification information. This header is used to authenticate the nodes of the message route and check the correctness of transmission.

Message GetRangeHashResponse.Body

Get hash of object's payload part response body.

Field	Type	Label	Description
type	neo.fs.v2.refs.ChecksumType		Checksum algorithm type
hash_list	bytes	repeated	List of range hashes in a binary format

Message GetRangeRequest

Request part of object's payload

Field	Type	Description
body	GetRangeRequest.Body	Body of get range object request message.
meta_header	neo.fs.v2.session.RequestMetaHeader	Carries request meta information. Header data is used only to regulate message transport and does not affect request execution.
verify_header	neo.fs.v2.session.RequestVerificationHeader	Carries request verification information. This header is used to authenticate the nodes of the message route and check the correctness of transmission.

Message GetRangeRequest.Body

Byte range of object's payload request body

Field	Type	Description
address	neo.fs.v2.refs.Address	Address of the object containing the requested payload range
range	Range	Requested payload range
raw	bool	If `raw` flag is set, request will work only with objects that are physically stored on the peer node.

Message GetRangeResponse

Get part of object's payload

Field	Type	Description
body	GetRangeResponse.Body	Body of get range object response message.
meta_header	neo.fs.v2.session.ResponseMetaHeader	Carries response meta information. Header data is used only to regulate message transport and does not affect request execution.
verify_header	neo.fs.v2.session.ResponseVerificationHeader	Carries response verification information. This header is used to authenticate the nodes of the message route and check the correctness of transmission.

Message GetRangeResponse.Body

Get Range response body uses streams to transfer the response. Because object payload considered a byte sequence, there is no need to have some initial preamble message. The requested byte range is sent as a series chunks.

Field	Type	Description
chunk	bytes	Chunked object payload's range.
split_info	SplitInfo	Meta information of split hierarchy.
ec_info	ECInfo	Meta information for EC object assembly.

Message GetRequest

GET object request

Field	Type	Description
body	GetRequest.Body	Body of get object request message.
meta_header	neo.fs.v2.session.RequestMetaHeader	Carries request meta information. Header data is used only to regulate message transport and does not affect request execution.
verify_header	neo.fs.v2.session.RequestVerificationHeader	Carries request verification information. This header is used to authenticate the nodes of the message route and check the correctness of transmission.

Message GetRequest.Body

GET Object request body

Field	Type	Label	Description
address	neo.fs.v2.refs.Address		Address of the requested object
raw	bool		If `raw` flag is set, request will work only with objects that are physically stored on the peer node

Message GetResponse

GET object response

Field	Type	Description
body	GetResponse.Body	Body of get object response message.
meta_header	neo.fs.v2.session.ResponseMetaHeader	Carries response meta information. Header data is used only to regulate message transport and does not affect request execution.
verify_header	neo.fs.v2.session.ResponseVerificationHeader	Carries response verification information. This header is used to authenticate the nodes of the message route and check the correctness of transmission.

Message GetResponse.Body

GET Object Response body

Field	Type	Description
init	GetResponse.Body.Init	Initial part of the object stream
chunk	bytes	Chunked object payload
split_info	SplitInfo	Meta information of split hierarchy for object assembly.
ec_info	ECInfo	Meta information for EC object assembly.

Message GetResponse.Body.Init

Initial part of the Object structure stream. Technically it's a set of all Object structure's fields except payload.

Field	Type	Description
object_id	neo.fs.v2.refs.ObjectID	Object's unique identifier.
signature	neo.fs.v2.refs.Signature	Signed `ObjectID`
header	Header	Object metadata headers

Message HeadRequest

Object HEAD request

Field	Type	Description
body	HeadRequest.Body	Body of head object request message.
meta_header	neo.fs.v2.session.RequestMetaHeader	Carries request meta information. Header data is used only to regulate message transport and does not affect request execution.
verify_header	neo.fs.v2.session.RequestVerificationHeader	Carries request verification information. This header is used to authenticate the nodes of the message route and check the correctness of transmission.

Message HeadRequest.Body

Object HEAD request body

Field	Type	Description
address	neo.fs.v2.refs.Address	Address of the object with the requested Header
main_only	bool	Return only minimal header subset
raw	bool	If `raw` flag is set, request will work only with objects that are physically stored on the peer node

Message HeadResponse

Object HEAD response

Field	Type	Description
body	HeadResponse.Body	Body of head object response message.
meta_header	neo.fs.v2.session.ResponseMetaHeader	Carries response meta information. Header data is used only to regulate message transport and does not affect request execution.
verify_header	neo.fs.v2.session.ResponseVerificationHeader	Carries response verification information. This header is used to authenticate the nodes of the message route and check the correctness of transmission.

Message HeadResponse.Body

Object HEAD response body

Field	Type	Description
header	HeaderWithSignature	Full object's `Header` with `ObjectID` signature
short_header	ShortHeader	Short object header
split_info	SplitInfo	Meta information of split hierarchy.
ec_info	ECInfo	Meta information for EC object assembly.

Message HeaderWithSignature

Tuple of a full object header and signature of an ObjectID.
Signed ObjectID is present to verify full header's authenticity through the following steps:

Calculate SHA-256 of the marshalled Header structure
Check if the resulting hash matches ObjectID
Check if ObjectID signature in signature field is correct

Field	Type	Label	Description
header	Header		Full object header
signature	neo.fs.v2.refs.Signature		Signed `ObjectID` to verify full header's authenticity

Message PutRequest

PUT object request

Field	Type	Description
body	PutRequest.Body	Body of put object request message.
meta_header	neo.fs.v2.session.RequestMetaHeader	Carries request meta information. Header data is used only to regulate message transport and does not affect request execution.
verify_header	neo.fs.v2.session.RequestVerificationHeader	Carries request verification information. This header is used to authenticate the nodes of the message route and check the correctness of transmission.

Message PutRequest.Body

PUT request body

Field	Type	Label	Description
init	PutRequest.Body.Init		Initial part of the object stream
chunk	bytes		Chunked object payload

Message PutRequest.Body.Init

Newly created object structure parameters. If some optional parameters are not set, they will be calculated by a peer node.

Field	Type	Label	Description
object_id	neo.fs.v2.refs.ObjectID		ObjectID if available.
signature	neo.fs.v2.refs.Signature		Object signature if available
header	Header		Object's Header
copies_number	uint32	repeated	Number of copies of the object to store within the RPC call. By default, object is processed according to the container's placement policy. Can be one of: 1. A single number; applied to the whole request and is treated as a minimal number of nodes that must store an object to complete the request successfully. 2. An ordered array; every number is treated as a minimal number of nodes in a corresponding placement vector that must store an object to complete the request successfully. The length MUST equal the placement vectors number, otherwise request is considered malformed.

Message PutResponse

PUT Object response

Field	Type	Description
body	PutResponse.Body	Body of put object response message.
meta_header	neo.fs.v2.session.ResponseMetaHeader	Carries response meta information. Header data is used only to regulate message transport and does not affect request execution.
verify_header	neo.fs.v2.session.ResponseVerificationHeader	Carries response verification information. This header is used to authenticate the nodes of the message route and check the correctness of transmission.

Message PutResponse.Body

PUT Object response body

Field	Type	Label	Description
object_id	neo.fs.v2.refs.ObjectID		Identifier of the saved object

Message PutSingleRequest

Object PUT Single request

Field	Type	Description
body	PutSingleRequest.Body	Body of put single object request message.
meta_header	neo.fs.v2.session.RequestMetaHeader	Carries request meta information. Header data is used only to regulate message transport and does not affect request execution.
verify_header	neo.fs.v2.session.RequestVerificationHeader	Carries request verification information. This header is used to authenticate the nodes of the message route and check the correctness of transmission.

Message PutSingleRequest.Body

PUT Single request body

Field	Type	Label	Description
object	Object		Prepared object with payload.
copies_number	uint32	repeated	Number of copies of the object to store within the RPC call. By default, object is processed according to the container's placement policy. Every number is treated as a minimal number of nodes in a corresponding placement vector that must store an object to complete the request successfully. The length MUST equal the placement vectors number, otherwise request is considered malformed.

Message PutSingleResponse

Object PUT Single response

Field	Type	Description
body	PutSingleResponse.Body	Body of put single object response message.
meta_header	neo.fs.v2.session.ResponseMetaHeader	Carries response meta information. Header data is used only to regulate message transport and does not affect request execution.
verify_header	neo.fs.v2.session.ResponseVerificationHeader	Carries response verification information. This header is used to authenticate the nodes of the message route and check the correctness of transmission.

Message PutSingleResponse.Body

PUT Single Object response body

Message Range

Object payload range.Ranges of zero length SHOULD be considered as invalid.

Field	Type	Label	Description
offset	uint64		Offset of the range from the object payload start
length	uint64		Length in bytes of the object payload range

Message SearchRequest

Object Search request

Field	Type	Description
body	SearchRequest.Body	Body of search object request message.
meta_header	neo.fs.v2.session.RequestMetaHeader	Carries request meta information. Header data is used only to regulate message transport and does not affect request execution.
verify_header	neo.fs.v2.session.RequestVerificationHeader	Carries request verification information. This header is used to authenticate the nodes of the message route and check the correctness of transmission.

Message SearchRequest.Body

Object Search request body

Field	Type	Label	Description
container_id	neo.fs.v2.refs.ContainerID		Container identifier were to search
version	uint32		Version of the Query Language used
filters	SearchRequest.Body.Filter	repeated	List of search expressions

Message SearchRequest.Body.Filter

Filter structure checks if the object header field or the attribute content matches a value.

If no filters are set, search request will return all objects of the container, including Regular object and Tombstone objects. Most human users expect to get only object they can directly work with. In that case, $Object:ROOT filter should be used.

By default key field refers to the corresponding object's Attribute. Some Object's header fields can also be accessed by adding $Object: prefix to the name. Here is the list of fields available via this prefix:

$Object:version
version
$Object:objectID
object_id
$Object:containerID
container_id
$Object:ownerID
owner_id
$Object:creationEpoch
creation_epoch
$Object:payloadLength
payload_length
$Object:payloadHash
payload_hash
$Object:objectType
object_type
$Object:homomorphicHash
homomorphic_hash
$Object:split.parent
object_id of parent
$Object:split.splitID
16 byte UUIDv4 used to identify the split object hierarchy parts
$Object:ec.parent
If the object is stored according to EC policy, then ec_parent attribute is set to return an id list of all related EC chunks.

There are some well-known filter aliases to match objects by certain properties:

$Object:ROOT
Returns only REGULAR type objects that are not split or that are the top level root objects in a split hierarchy. This includes objects not present physically, like large objects split into smaller objects without a separate top-level root object. Objects of other types like Locks and Tombstones will not be shown. This filter may be useful for listing objects like ls command of some virtual file system. This filter is activated if the key exists, disregarding the value and matcher type.
$Object:PHY
Returns only objects physically stored in the system. This filter is activated if the key exists, disregarding the value and matcher type.

Note: using filters with a key with prefix $Object: and match type NOT_PRESENT is not recommended since this is not a cross-version approach. Behavior when processing this kind of filters is undefined.

Field	Type	Description
match_type	MatchType	Match type to use
key	string	Attribute or Header fields to match
value	string	Value to match

Message SearchResponse

Search response

Field	Type	Description
body	SearchResponse.Body	Body of search object response message.
meta_header	neo.fs.v2.session.ResponseMetaHeader	Carries response meta information. Header data is used only to regulate message transport and does not affect request execution.
verify_header	neo.fs.v2.session.ResponseVerificationHeader	Carries response verification information. This header is used to authenticate the nodes of the message route and check the correctness of transmission.

Message SearchResponse.Body

Object Search response body

Field	Type	Label	Description
id_list	neo.fs.v2.refs.ObjectID	repeated	List of `ObjectID`s that match the search query

Top

object/types.proto

Message ECInfo

Meta information for the erasure-encoded object.

Field	Type	Label	Description
chunks	ECInfo.Chunk	repeated	Chunk stored on the node.

Message ECInfo.Chunk

Field	Type	Description
id	neo.fs.v2.refs.ObjectID	Object ID of the chunk.
index	uint32	Index of the chunk.
total	uint32	Total number of chunks in this split.

Message Header

Object Header

Field	Type	Label	Description
version	neo.fs.v2.refs.Version		Object format version. Effectively, the version of API library used to create particular object
container_id	neo.fs.v2.refs.ContainerID		Object's container
owner_id	neo.fs.v2.refs.OwnerID		Object's owner
creation_epoch	uint64		Object creation Epoch
payload_length	uint64		Size of payload in bytes. `0xFFFFFFFFFFFFFFFF` means `payload_length` is unknown.
payload_hash	neo.fs.v2.refs.Checksum		Hash of payload bytes
object_type	ObjectType		Type of the object payload content
homomorphic_hash	neo.fs.v2.refs.Checksum		Homomorphic hash of the object payload
session_token	neo.fs.v2.session.SessionToken		Session token, if it was used during Object creation. Need it to verify integrity and authenticity out of Request scope.
attributes	Header.Attribute	repeated	User-defined object attributes
split	Header.Split		Position of the object in the split hierarchy
ec	Header.EC		Erasure code chunk information.

Message Header.Attribute

Attribute is a user-defined Key-Value metadata pair attached to an object.

Key name must be an object-unique valid UTF-8 string. Value can't be empty. Objects with duplicated attribute names or attributes with empty values will be considered invalid.

There are some "well-known" attributes starting with __SYSTEM__ (__NEOFS__ is deprecated) prefix that affect system behaviour:

[ __SYSTEM__UPLOAD_ID ]
(__NEOFS__UPLOAD_ID is deprecated)
Marks smaller parts of a split bigger object
[ __SYSTEM__EXPIRATION_EPOCH ]
(__NEOFS__EXPIRATION_EPOCH is deprecated)
The epoch after which object with no LOCKs on it becomes unavailable. Locked object continues to be available until each of the LOCKs expire.
[ __SYSTEM__TICK_EPOCH ]
(__NEOFS__TICK_EPOCH is deprecated)
Decimal number that defines what epoch must produce object notification with UTF-8 object address in a body (0 value produces notification right after object put)
[ __SYSTEM__TICK_TOPIC ]
(__NEOFS__TICK_TOPIC is deprecated)
UTF-8 string topic ID that is used for object notification

And some well-known attributes used by applications only:

Name
Human-friendly name
FileName
File name to be associated with the object on saving
FilePath
Full path to be associated with the object on saving. Should start with a '/' and use '/' as a delimiting symbol. Trailing '/' should be interpreted as a virtual directory marker. If an object has conflicting FilePath and FileName, FilePath should have higher priority, because it is used to construct the directory tree. FilePath with trailing '/' and non-empty FileName attribute should not be used together.
Timestamp
User-defined local time of object creation in Unix Timestamp format
Content-Type
MIME Content Type of object's payload

For detailed description of each well-known attribute please see the corresponding section in NeoFS Technical Specification.

Field	Type	Label	Description
key	string		string key to the object attribute
value	string		string value of the object attribute

Message Header.EC

Erasure code can be applied to any object. Information about encoded object structure is stored in EC header. All objects belonging to a single EC group have the same parent field.

Field	Type	Description
parent	neo.fs.v2.refs.ObjectID	Identifier of the origin object. Known to all chunks.
index	uint32	Index of this chunk.
total	uint32	Total number of chunks in this split.
header_length	uint32	Total length of a parent header. Used to trim padding zeroes.
header	bytes	Chunk of a parent header.
parent_split_id	bytes	As the origin object is EC-splitted its identifier is known to all chunks as parent. But parent itself can be a part of Split (does not relate to EC-split). In this case parent_split_id should be set.
parent_split_index	uint32	Parent's index in Split.

Message Header.Split

Bigger objects can be split into a chain of smaller objects. Information about inter-dependencies between spawned objects and how to re-construct the original one is in the Split headers. Parent and children objects must be within the same container.

Field	Type	Label	Description
parent	neo.fs.v2.refs.ObjectID		Identifier of the origin object. Known only to the minor child.
previous	neo.fs.v2.refs.ObjectID		Identifier of the left split neighbor
parent_signature	neo.fs.v2.refs.Signature		`signature` field of the parent object. Used to reconstruct parent.
parent_header	Header		`header` field of the parent object. Used to reconstruct parent.
children	neo.fs.v2.refs.ObjectID	repeated	List of identifiers of the objects generated by splitting current one.
split_id	bytes		16 byte UUIDv4 used to identify the split object hierarchy parts. Must be unique inside container. All objects participating in the split must have the same `split_id` value.

Message Object

Object structure. Object is immutable and content-addressed. It means ObjectID will change if the header or the payload changes. It's calculated as a hash of header field which contains hash of the object's payload.

For non-regular object types payload format depends on object type specified in the header.

Field	Type	Description
object_id	neo.fs.v2.refs.ObjectID	Object's unique identifier.
signature	neo.fs.v2.refs.Signature	Signed object_id
header	Header	Object metadata headers
payload	bytes	Payload bytes

Message ShortHeader

Short header fields

Field	Type	Description
version	neo.fs.v2.refs.Version	Object format version. Effectively, the version of API library used to create particular object.
creation_epoch	uint64	Epoch when the object was created
owner_id	neo.fs.v2.refs.OwnerID	Object's owner
object_type	ObjectType	Type of the object payload content
payload_length	uint64	Size of payload in bytes. `0xFFFFFFFFFFFFFFFF` means `payload_length` is unknown
payload_hash	neo.fs.v2.refs.Checksum	Hash of payload bytes
homomorphic_hash	neo.fs.v2.refs.Checksum	Homomorphic hash of the object payload

Message SplitInfo

Meta information of split hierarchy for object assembly. With the last part one can traverse linked list of split hierarchy back to the first part and assemble the original object. With a linking object one can assemble an object right from the object parts.

Field	Type	Description
split_id	bytes	16 byte UUID used to identify the split object hierarchy parts.
last_part	neo.fs.v2.refs.ObjectID	The identifier of the last object in split hierarchy parts. It contains split header with the original object header.
link	neo.fs.v2.refs.ObjectID	The identifier of a linking object for split hierarchy parts. It contains split header with the original object header and a sorted list of object parts.

MatchType

Type of match expression

Name	Number	Description
MATCH_TYPE_UNSPECIFIED	0	Unknown. Not used
STRING_EQUAL	1	Full string match
STRING_NOT_EQUAL	2	Full string mismatch
NOT_PRESENT	3	Lack of key
COMMON_PREFIX	4	String prefix match

ObjectType

Type of the object payload content. Only REGULAR type objects can be split, hence TOMBSTONE and LOCK payload is limited by the maximum object size.

String presentation of object type is the same as definition:

REGULAR
TOMBSTONE
LOCK

Name	Number	Description
REGULAR	0	Just a normal object
TOMBSTONE	1	Used internally to identify deleted objects
LOCK	3	Object lock

Scalar Value Types

.proto Type	Notes	C++ Type	Java Type	Python Type
double		double	double	float
float		float	float	float
int32	Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint32 instead.	int32	int	int
int64	Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint64 instead.	int64	long	int/long
uint32	Uses variable-length encoding.	uint32	int	int/long
uint64	Uses variable-length encoding.	uint64	long	int/long
sint32	Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int32s.	int32	int	int
sint64	Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int64s.	int64	long	int/long
fixed32	Always four bytes. More efficient than uint32 if values are often greater than 2^28.	uint32	int	int
fixed64	Always eight bytes. More efficient than uint64 if values are often greater than 2^56.	uint64	long	int/long
sfixed32	Always four bytes.	int32	int	int
sfixed64	Always eight bytes.	int64	long	int/long
bool		bool	boolean	boolean
string	A string must always contain UTF-8 encoded or 7-bit ASCII text.	string	String	str/unicode
bytes	May contain any arbitrary sequence of bytes.	string	ByteString	str

53 KiB Raw Blame History

Protocol Documentation

Table of Contents

object/service.proto

Service "neo.fs.v2.object.ObjectService"

Method Get

Method Put

Method Delete

Method Head

Method Search

Method GetRange

Method GetRangeHash

Method PutSingle

Message DeleteRequest

Message DeleteRequest.Body

Message DeleteResponse

Message DeleteResponse.Body

Message GetRangeHashRequest

Message GetRangeHashRequest.Body

Message GetRangeHashResponse

Message GetRangeHashResponse.Body

Message GetRangeRequest

Message GetRangeRequest.Body

Message GetRangeResponse

Message GetRangeResponse.Body

Message GetRequest

Message GetRequest.Body

Message GetResponse

Message GetResponse.Body

Message GetResponse.Body.Init

Message HeadRequest

Message HeadRequest.Body

Message HeadResponse

Message HeadResponse.Body

Message HeaderWithSignature

Message PutRequest

Message PutRequest.Body

Message PutRequest.Body.Init

Message PutResponse

Message PutResponse.Body

Message PutSingleRequest

Message PutSingleRequest.Body

Message PutSingleResponse

Message PutSingleResponse.Body

Message Range

Message SearchRequest

Message SearchRequest.Body

Message SearchRequest.Body.Filter

Message SearchResponse

Message SearchResponse.Body

object/types.proto

Message ECInfo

Message ECInfo.Chunk

Message Header

Message Header.Attribute

Message Header.EC

Message Header.Split

Message Object

Message ShortHeader

Message SplitInfo

MatchType

ObjectType

Scalar Value Types

53 KiB

Raw Blame History