frostfs-api/proto-docs/object.md
Leonard Lyubich fccd753a5d [#31] refs: Use ContainerID message in all services
Change the type of all fields for the container identifier to refs.ContainerID.
This will allow to follow a single format and not duplicate its description.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-08-06 22:00:01 +03:00

21 KiB

Protocol Documentation

Table of Contents

Top

object/service.proto

Service "object.Service"

Object service provides API for manipulating with the object.

rpc Get(GetRequest) returns (stream GetResponse);
rpc Put(stream PutRequest) returns (PutResponse);
rpc Delete(DeleteRequest) returns (DeleteResponse);
rpc Head(HeadRequest) returns (HeadResponse);
rpc Search(SearchRequest) returns (stream SearchResponse);
rpc GetRange(GetRangeRequest) returns (stream GetRangeResponse);
rpc GetRangeHash(GetRangeHashRequest) returns (GetRangeHashResponse);

Method Get

Get the object from container. Response uses gRPC stream. First response message carry object of requested address. Chunk messages are parts of the object's payload if it is needed. All messages except first carry chunks. Requested object can be restored by concatenation of object message payload and all chunks keeping receiving order.

Name Input Output
Get GetRequest GetResponse

Method Put

Put the object into container. Request uses gRPC stream. First message SHOULD BE type of PutHeader. Container id and Owner id of object SHOULD BE set. Session token SHOULD BE obtained before put operation (see session package). Chunk messages considered by server as part of object payload. All messages except first SHOULD BE chunks. Chunk messages SHOULD BE sent in direct order of fragmentation.

Name Input Output
Put PutRequest PutResponse

Method Delete

Delete the object from a container

Name Input Output
Delete DeleteRequest DeleteResponse

Method Head

Head returns the object without data payload. Object in the response has system header only. If full headers flag is set, extended headers are also present.

Name Input Output
Head HeadRequest HeadResponse

Search objects in container. Version of query language format SHOULD BE set to 1. Search query represented in serialized format (see query package).

Name Input Output
Search SearchRequest SearchResponse

Method GetRange

GetRange of data payload. Range is a pair (offset, length). Requested range can be restored by concatenation of all chunks keeping receiving order.

Name Input Output
GetRange GetRangeRequest GetRangeResponse

Method GetRangeHash

GetRangeHash returns homomorphic hash of object payload range after XOR operation. Ranges are set of pairs (offset, length). Hashes order in response corresponds to ranges order in request. Homomorphic hash is calculated for XORed data.

Name Input Output
GetRangeHash GetRangeHashRequest GetRangeHashResponse

Message DeleteRequest

Field Type Label Description
Address refs.Address Address of object (container id + object id)
OwnerID bytes OwnerID is a wallet address
Meta service.RequestMetaHeader RequestMetaHeader contains information about request meta headers (should be embedded into message)
Verify service.RequestVerificationHeader RequestVerificationHeader is a set of signatures of every NeoFS Node that processed request (should be embedded into message)

Message DeleteResponse

DeleteResponse is empty because we cannot guarantee permanent object removal in distributed system.

Field Type Label Description
Meta service.ResponseMetaHeader ResponseMetaHeader contains meta information based on request processing by server (should be embedded into message)

Message GetRangeHashRequest

Field Type Label Description
Address refs.Address Address of object (container id + object id)
Ranges Range repeated Ranges of object's payload to calculate homomorphic hash
Salt bytes Salt is used to XOR object's payload ranges before hashing, it can be nil
Meta service.RequestMetaHeader RequestMetaHeader contains information about request meta headers (should be embedded into message)
Verify service.RequestVerificationHeader RequestVerificationHeader is a set of signatures of every NeoFS Node that processed request (should be embedded into message)

Message GetRangeHashResponse

Field Type Label Description
Hashes bytes repeated Hashes is a homomorphic hashes of all ranges
Meta service.ResponseMetaHeader ResponseMetaHeader contains meta information based on request processing by server (should be embedded into message)

Message GetRangeRequest

Field Type Label Description
Address refs.Address Address of object (container id + object id)
Range Range Range of object's payload to return
Meta service.RequestMetaHeader RequestMetaHeader contains information about request meta headers (should be embedded into message)
Verify service.RequestVerificationHeader RequestVerificationHeader is a set of signatures of every NeoFS Node that processed request (should be embedded into message)

Message GetRangeResponse

Field Type Label Description
Fragment bytes Fragment of object's payload
Meta service.ResponseMetaHeader ResponseMetaHeader contains meta information based on request processing by server (should be embedded into message)

Message GetRequest

Field Type Label Description
Address refs.Address Address of object (container id + object id)
Meta service.RequestMetaHeader RequestMetaHeader contains information about request meta headers (should be embedded into message)
Verify service.RequestVerificationHeader RequestVerificationHeader is a set of signatures of every NeoFS Node that processed request (should be embedded into message)

Message GetResponse

Field Type Label Description
object Object Object header and some payload
Chunk bytes Chunk of remaining payload
Meta service.ResponseMetaHeader ResponseMetaHeader contains meta information based on request processing by server (should be embedded into message)

Message HeadRequest

Field Type Label Description
Address refs.Address Address of object (container id + object id)
FullHeaders bool FullHeaders can be set true for extended headers in the object
Meta service.RequestMetaHeader RequestMetaHeader contains information about request meta headers (should be embedded into message)
Verify service.RequestVerificationHeader RequestVerificationHeader is a set of signatures of every NeoFS Node that processed request (should be embedded into message)

Message HeadResponse

Field Type Label Description
Object Object Object without payload
Meta service.ResponseMetaHeader ResponseMetaHeader contains meta information based on request processing by server (should be embedded into message)

Message PutRequest

Field Type Label Description
Header PutRequest.PutHeader Header should be the first message in the stream
Chunk bytes Chunk should be a remaining message in stream should be chunks
Meta service.RequestMetaHeader RequestMetaHeader contains information about request meta headers (should be embedded into message)
Verify service.RequestVerificationHeader RequestVerificationHeader is a set of signatures of every NeoFS Node that processed request (should be embedded into message)

Message PutRequest.PutHeader

Field Type Label Description
Object Object Object with at least container id and owner id fields
CopiesNumber uint32 Number of the object copies to store within the RPC call (zero is processed according to the placement rules)

Message PutResponse

Field Type Label Description
Address refs.Address Address of object (container id + object id)
Meta service.ResponseMetaHeader ResponseMetaHeader contains meta information based on request processing by server (should be embedded into message)

Message Range

Field Type Label Description
Offset uint64 Offset of the data range
Length uint64 Length of the data range

Message SearchRequest

Field Type Label Description
ContainerID refs.ContainerID ContainerID carries search container identifier.
Query bytes Query in the binary serialized format
QueryVersion uint32 QueryVersion is a version of search query format
Meta service.RequestMetaHeader RequestMetaHeader contains information about request meta headers (should be embedded into message)
Verify service.RequestVerificationHeader RequestVerificationHeader is a set of signatures of every NeoFS Node that processed request (should be embedded into message)

Message SearchResponse

Field Type Label Description
Addresses refs.Address repeated Addresses of found objects
Meta service.ResponseMetaHeader ResponseMetaHeader contains meta information based on request processing by server (should be embedded into message)

Top

object/types.proto

Message Header

Header groups the information about the NeoFS object.

Field Type Label Description
main Header.Main Main carries the main part of the header. Main MUST NOT be NULL.
extended Header.Extended Extended carries the additional part of the header.

Message Header.Extended

Extended groups additional information about the object. It encapsulates both user and system attributes needed to regulate the NeoFS sub-systems.

Field Type Label Description
integrity Header.Extended.Integrity Integrity carries object integrity evidence.
Attributes Header.Extended.Attribute repeated Attributes carries list of the object attributes in a string key-value format.
CreationEpoch uint64 CreationEpoch carries number of NeoFS epoch on which the object was created.
tombstone Header.Extended.Tombstone Tombstone marks the object to be deleted.
HomomorphicHash bytes HomomorphicHash carries homomorphic hash of the object payload.
storageGroup Header.Extended.StorageGroup StorageGroup marks an object containing information about a storage group.
split Header.Extended.Split Split carries the position of the object in the split hierarchy.

Message Header.Extended.Attribute

Attribute groups the parameters of the object attributes.

Field Type Label Description
Key string Key carries the string key to the object attribute.
Value string Value carries the string value of the object attribute.

Message Header.Extended.Integrity

Integrity groups evidence of the integrity of an object's structure.

Field Type Label Description
PayloadChecksum bytes PayloadChecksum carries the checksum of object payload bytes. Changing any byte of the payload changes the checksum. It is calculated as a SHA-256 hash over payload bytes.
HeaderChecksum bytes HeaderChecksum carries checksum of the object header structure. It covers all object attributes. Changing any field of the object except CreatorKey and ChecksumSignature changes the checksum. PayloadChecksum and HeaderChecksum cannot be merged due to the need to verify the header in the absence of a payload (e.g. in object.Head rpc). It is calculated as a SHA-256 hash over marshaled object header with cut CreatorKey and ChecksumSignature.
SessionToken service.Token SessionToken carries token of the session within which the object was created. If session token is presented in object, it acts as the user's proof of the correctness of the CreatorKey.
CreatorKey bytes CreatorKey carries public key of the object creator in a binary format.
ChecksumSignature bytes ChecksumSignature carries signature of the structure checksum by the object creator.

Message Header.Extended.Split

Split groups information about spawning the object through a payload splitting.

Field Type Label Description
Parent refs.ObjectID Parent carries identifier of the origin object.
Previous refs.ObjectID Previous carries identifier of the left split neighbor.
Next refs.ObjectID Previous carries identifier of the right split neighbor.
Children refs.ObjectID repeated Children carries list of identifiers of the objects generated by splitting the current.
Origin Header Origin carries the header of the origin object.

Message Header.Extended.StorageGroup

StorageGroup groups meta information about a storage group.

Message Header.Extended.Tombstone

Tombstone groups the options for deleting an object.

Message Header.Main

Main groups mandatory information about the object. Message fields are presented in all NeoFS objects.

Field Type Label Description
PayloadLength uint64 PayloadLength carries length of the object payload. Each object has a fixed payload length since it's immutable.
Address refs.Address Address carries object address in the NeoFS system. It encapsulates the object and the container identifiers.
OwnerID bytes OwnerID carries identifier the object owner in a binary format.

Message Object

Object groups the information about the NeoFS object. It consists of payload data with additional service information.

Field Type Label Description
Header Header Header carries the object header.
Payload bytes Payload carries the object payload bytes.

Scalar Value Types

.proto Type Notes C++ Type Java Type Python Type
double double double float
float float float float
int32 Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint32 instead. int32 int int
int64 Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint64 instead. int64 long int/long
uint32 Uses variable-length encoding. uint32 int int/long
uint64 Uses variable-length encoding. uint64 long int/long
sint32 Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int32s. int32 int int
sint64 Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int64s. int64 long int/long
fixed32 Always four bytes. More efficient than uint32 if values are often greater than 2^28. uint32 int int
fixed64 Always eight bytes. More efficient than uint64 if values are often greater than 2^56. uint64 long int/long
sfixed32 Always four bytes. int32 int int
sfixed64 Always eight bytes. int64 long int/long
bool bool boolean boolean
string A string must always contain UTF-8 encoded or 7-bit ASCII text. string String str/unicode
bytes May contain any arbitrary sequence of bytes. string ByteString str