frostfs-api-go/docs/object.md

Protocol Documentation

Table of Contents

• object/service.proto

• Services

  • Service

• Messages

  • DeleteRequest
  • DeleteResponse
  • GetRangeHashRequest
  • GetRangeHashResponse
  • GetRangeRequest
  • GetRangeResponse
  • GetRequest
  • GetResponse
  • HeadRequest
  • HeadResponse
  • PutRequest
  • PutRequest.PutHeader
  • PutResponse
  • SearchRequest
  • SearchResponse

• object/types.proto

  • Messages
    • CreationPoint
    • Header
    • IntegrityHeader
    • Link
    • Object
    • PublicKey
    • Range
    • SystemHeader
    • Tombstone
    • Transform
    • UserHeader

• Scalar Value Types

Top

object/service.proto

Service "object.Service"

Object service provides API for manipulating with the object.

rpc Get(GetRequest) returns (stream GetResponse);
rpc Put(stream PutRequest) returns (PutResponse);
rpc Delete(DeleteRequest) returns (DeleteResponse);
rpc Head(HeadRequest) returns (HeadResponse);
rpc Search(SearchRequest) returns (stream SearchResponse);
rpc GetRange(GetRangeRequest) returns (stream GetRangeResponse);
rpc GetRangeHash(GetRangeHashRequest) returns (GetRangeHashResponse);

Method Get

Get the object from container. Response uses gRPC stream. First response message carry object of requested address. Chunk messages are parts of the object's payload if it is needed. All messages except first carry chunks. Requested object can be restored by concatenation of object message payload and all chunks keeping receiving order.

Name	Input	Output
Get	GetRequest	GetResponse

Method Put

Put the object into container. Request uses gRPC stream. First message SHOULD BE type of PutHeader. Container id and Owner id of object SHOULD BE set. Session token SHOULD BE obtained before put operation (see session package). Chunk messages considered by server as part of object payload. All messages except first SHOULD BE chunks. Chunk messages SHOULD BE sent in direct order of fragmentation.

Name	Input	Output
Put	PutRequest	PutResponse

Method Delete

Delete the object from a container

Name	Input	Output
Delete	DeleteRequest	DeleteResponse

Method Head

Head returns the object without data payload. Object in the response has system header only. If full headers flag is set, extended headers are also present.

Name	Input	Output
Head	HeadRequest	HeadResponse

Method Search

Search objects in container. Version of query language format SHOULD BE set to 1. Search query represented in serialized format (see query package).

Name	Input	Output
Search	SearchRequest	SearchResponse

Method GetRange

GetRange of data payload. Range is a pair (offset, length). Requested range can be restored by concatenation of all chunks keeping receiving order.

Name	Input	Output
GetRange	GetRangeRequest	GetRangeResponse

Method GetRangeHash

GetRangeHash returns homomorphic hash of object payload range after XOR operation. Ranges are set of pairs (offset, length). Hashes order in response corresponds to ranges order in request. Homomorphic hash is calculated for XORed data.

Name	Input	Output
GetRangeHash	GetRangeHashRequest	GetRangeHashResponse

Message DeleteRequest

Field	Type	Label	Description
Address	refs.Address		Address of object (container id + object id)
OwnerID	bytes		OwnerID is a wallet address
Token	session.Token		Token with session public key and user's signature
Meta	service.RequestMetaHeader		RequestMetaHeader contains information about request meta headers (should be embedded into message)
Verify	service.RequestVerificationHeader		RequestVerificationHeader is a set of signatures of every NeoFS Node that processed request (should be embedded into message)

Message DeleteResponse

DeleteResponse is empty because we cannot guarantee permanent object removal in distributed system.

Field	Type	Label	Description
Meta	service.ResponseMetaHeader		ResponseMetaHeader contains meta information based on request processing by server (should be embedded into message)

Message GetRangeHashRequest

Field	Type	Label	Description
Address	refs.Address		Address of object (container id + object id)
Ranges	Range	repeated	Ranges of object's payload to calculate homomorphic hash
Salt	bytes		Salt is used to XOR object's payload ranges before hashing, it can be nil
Meta	service.RequestMetaHeader		RequestMetaHeader contains information about request meta headers (should be embedded into message)
Verify	service.RequestVerificationHeader		RequestVerificationHeader is a set of signatures of every NeoFS Node that processed request (should be embedded into message)

Message GetRangeHashResponse

Field	Type	Label	Description
Hashes	bytes	repeated	Hashes is a homomorphic hashes of all ranges
Meta	service.ResponseMetaHeader		ResponseMetaHeader contains meta information based on request processing by server (should be embedded into message)

Message GetRangeRequest

Field	Type	Label	Description
Address	refs.Address		Address of object (container id + object id)
Range	Range		Range of object's payload to return
Meta	service.RequestMetaHeader		RequestMetaHeader contains information about request meta headers (should be embedded into message)
Verify	service.RequestVerificationHeader		RequestVerificationHeader is a set of signatures of every NeoFS Node that processed request (should be embedded into message)

Message GetRangeResponse

Field	Type	Label	Description
Fragment	bytes		Fragment of object's payload
Meta	service.ResponseMetaHeader		ResponseMetaHeader contains meta information based on request processing by server (should be embedded into message)

Message GetRequest

Field	Type	Label	Description
Address	refs.Address		Address of object (container id + object id)
Raw	bool		Raw is the request flag of a physically stored representation of an object
Meta	service.RequestMetaHeader		RequestMetaHeader contains information about request meta headers (should be embedded into message)
Verify	service.RequestVerificationHeader		RequestVerificationHeader is a set of signatures of every NeoFS Node that processed request (should be embedded into message)

Message GetResponse

Field	Type	Label	Description
object	Object		Object header and some payload
Chunk	bytes		Chunk of remaining payload
Meta	service.ResponseMetaHeader		ResponseMetaHeader contains meta information based on request processing by server (should be embedded into message)

Message HeadRequest

Field	Type	Label	Description
Address	refs.Address		Address of object (container id + object id)
FullHeaders	bool		FullHeaders can be set true for extended headers in the object
Raw	bool		Raw is the request flag of a physically stored representation of an object
Meta	service.RequestMetaHeader		RequestMetaHeader contains information about request meta headers (should be embedded into message)
Verify	service.RequestVerificationHeader		RequestVerificationHeader is a set of signatures of every NeoFS Node that processed request (should be embedded into message)

Message HeadResponse

Field	Type	Label	Description
Object	Object		Object without payload
Meta	service.ResponseMetaHeader		ResponseMetaHeader contains meta information based on request processing by server (should be embedded into message)

Message PutRequest

Field	Type	Label	Description
Header	PutRequest.PutHeader		Header should be the first message in the stream
Chunk	bytes		Chunk should be a remaining message in stream should be chunks
Meta	service.RequestMetaHeader		RequestMetaHeader contains information about request meta headers (should be embedded into message)
Verify	service.RequestVerificationHeader		RequestVerificationHeader is a set of signatures of every NeoFS Node that processed request (should be embedded into message)

Message PutRequest.PutHeader

Field	Type	Label	Description
Object	Object		Object with at least container id and owner id fields
Token	session.Token		Token with session public key and user's signature

Message PutResponse

Field	Type	Label	Description
Address	refs.Address		Address of object (container id + object id)
Meta	service.ResponseMetaHeader		ResponseMetaHeader contains meta information based on request processing by server (should be embedded into message)

Message SearchRequest

Field	Type	Label	Description
ContainerID	bytes		ContainerID for searching the object
Query	bytes		Query in the binary serialized format
QueryVersion	uint32		QueryVersion is a version of search query format
Meta	service.RequestMetaHeader		RequestMetaHeader contains information about request meta headers (should be embedded into message)
Verify	service.RequestVerificationHeader		RequestVerificationHeader is a set of signatures of every NeoFS Node that processed request (should be embedded into message)

Message SearchResponse

Field	Type	Label	Description
Addresses	refs.Address	repeated	Addresses of found objects
Meta	service.ResponseMetaHeader		ResponseMetaHeader contains meta information based on request processing by server (should be embedded into message)

Top

object/types.proto

Message CreationPoint

Field	Type	Label	Description
UnixTime	int64		UnixTime is a date of creation in unixtime format
Epoch	uint64		Epoch is a date of creation in NeoFS epochs

Message Header

Field	Type	Label	Description
Link	Link		Link to other objects
Redirect	refs.Address		Redirect not used yet
UserHeader	UserHeader		UserHeader is a set of KV headers defined by user
Transform	Transform		Transform defines transform operation (e.g. payload split)
Tombstone	Tombstone		Tombstone header that set up in deleted objects
Verify	session.VerificationHeader		Verify header that contains session public key and user's signature
HomoHash	bytes		HomoHash is a homomorphic hash of original object payload
PayloadChecksum	bytes		PayloadChecksum of actual object's payload
Integrity	IntegrityHeader		Integrity header with checksum of all above headers in the object
StorageGroup	storagegroup.StorageGroup		StorageGroup contains meta information for the data audit
PublicKey	PublicKey		PublicKey of owner of the object. Key is used for verification and can be based on NeoID or x509 cert.

Message IntegrityHeader

Field	Type	Label	Description
HeadersChecksum	bytes		HeadersChecksum is a checksum of all above headers in the object
ChecksumSignature	bytes		ChecksumSignature is an user's signature of checksum to verify if it is correct

Message Link

Field	Type	Label	Description
type	Link.Type		Type of link
ID	bytes		ID is an object identifier, is a valid UUIDv4

Message Object

Field	Type	Label	Description
SystemHeader	SystemHeader		SystemHeader describes system header
Headers	Header	repeated	Headers describes a set of an extended headers
Payload	bytes		Payload is an object's payload

Message PublicKey

Field	Type	Label	Description
Value	bytes		Value contains marshaled ecdsa public key

Message Range

Field	Type	Label	Description
Offset	uint64		Offset of the data range
Length	uint64		Length of the data range

Message SystemHeader

Field	Type	Label	Description
Version	uint64		Version of the object structure
PayloadLength	uint64		PayloadLength is an object payload length
ID	bytes		ID is an object identifier, is a valid UUIDv4
OwnerID	bytes		OwnerID is a wallet address
CID	bytes		CID is a SHA256 hash of the container structure (container identifier)
CreatedAt	CreationPoint		CreatedAt is a timestamp of object creation

Message Tombstone

Message Transform

Field	Type	Label	Description
type	Transform.Type		Type of object transformation

Message UserHeader

Field	Type	Label	Description
Key	string		Key of the user's header
Value	string		Value of the user's header

Link.Type

Name	Number	Description
Unknown	0
Parent	1	Parent object created during object transformation
Previous	2	Previous object in the linked list created during object transformation
Next	3	Next object in the linked list created during object transformation
Child	4	Child object created during object transformation
StorageGroup	5	Object that included into this storage group

Transform.Type

Name	Number	Description
Unknown	0
Split	1	Split sets when object created after payload split
Sign	2	Sign sets when object created after re-signing (doesn't used)
Mould	3	Mould sets when object created after filling missing headers in the object

Scalar Value Types

.proto Type	Notes	C++ Type	Java Type	Python Type
double		double	double	float
float		float	float	float
int32	Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint32 instead.	int32	int	int
int64	Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint64 instead.	int64	long	int/long
uint32	Uses variable-length encoding.	uint32	int	int/long
uint64	Uses variable-length encoding.	uint64	long	int/long
sint32	Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int32s.	int32	int	int
sint64	Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int64s.	int64	long	int/long
fixed32	Always four bytes. More efficient than uint32 if values are often greater than 2^28.	uint32	int	int
fixed64	Always eight bytes. More efficient than uint64 if values are often greater than 2^56.	uint64	long	int/long
sfixed32	Always four bytes.	int32	int	int
sfixed64	Always eight bytes.	int64	long	int/long
bool		bool	boolean	boolean
string	A string must always contain UTF-8 encoded or 7-bit ASCII text.	string	String	str/unicode
bytes	May contain any arbitrary sequence of bytes.	string	ByteString	str