frostfs-api/object/service.proto

syntax = "proto3";

package neo.fs.v2.object;

option go_package = "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/object/grpc;object";
option csharp_namespace = "Neo.FileStorage.API.Object";

import "object/types.proto";
import "refs/types.proto";
import "session/types.proto";

// `ObjectService` provides API for manipulating objects. Object operations do
// not affect the sidechain and are only served by nodes in p2p style.
service ObjectService {
  // Receive full object structure, including Headers and payload. Response uses
  // gRPC stream. First response message carries the object with the requested address.
  // Chunk messages are parts of the object's payload if it is needed. All
  // messages, except the first one, carry payload chunks. The requested object can
  // be restored by concatenation of object message payload and all chunks
  // keeping the receiving order.
  //
  // Extended headers can change `Get` behaviour:
  // * __NEOFS__NETMAP_EPOCH \
  //   Will use the requsted version of Network Map for object placement
  //   calculation.
  // * __NEOFS__NETMAP_LOOKUP_DEPTH \
  //   Will try older versions (starting from `__NEOFS__NETMAP_EPOCH` if specified or
  //   the latest one otherwise) of Network Map to find an object until the depth
  //   limit is reached.
  //
  // Please refer to detailed `XHeader` description.
  //
  // Statuses:
  // - **OK** (0, SECTION_SUCCESS): \
  //   object has been successfully read;
  // - Common failures (SECTION_FAILURE_COMMON);
  // - **ACCESS_DENIED** (2048, SECTION_OBJECT): \
  //   read access to the object is denied;
  // - **OBJECT_NOT_FOUND** (2049, SECTION_OBJECT): \
  //   object not found in container;
  // - **OBJECT_ALREADY_REMOVED** (2052, SECTION_OBJECT): \
  //   the requested object has been marked as deleted;
  // - **CONTAINER_NOT_FOUND** (3072, SECTION_CONTAINER): \
  //   object container not found;
  // - **TOKEN_EXPIRED** (4097, SECTION_SESSION): \
  //   provided session token has expired.
  rpc Get(GetRequest) returns (stream GetResponse);

  // Put the object into container. Request uses gRPC stream. First message
  // SHOULD be of PutHeader type. `ContainerID` and `OwnerID` of an object
  // SHOULD be set. Session token SHOULD be obtained before `PUT` operation (see
  // session package). Chunk messages are considered by server as a part of an
  // object payload. All messages, except first one, SHOULD be payload chunks.
  // Chunk messages SHOULD be sent in the direct order of fragmentation.
  //
  // Extended headers can change `Put` behaviour:
  // * __NEOFS__NETMAP_EPOCH \
  //   Will use the requsted version of Network Map for object placement
  //   calculation.
  //
  // Please refer to detailed `XHeader` description.
  //
  // Statuses:
  // - **OK** (0, SECTION_SUCCESS): \
  //   object has been successfully saved in the container;
  // - Common failures (SECTION_FAILURE_COMMON);
  // - **ACCESS_DENIED** (2048, SECTION_OBJECT): \
  //   write access to the container is denied;
  // - **LOCKED** (2050, SECTION_OBJECT): \
  //   placement of an object of type TOMBSTONE that includes at least one locked
  //   object is prohibited;
  // - **LOCK_NON_REGULAR_OBJECT** (2051, SECTION_OBJECT): \
  //   placement of an object of type LOCK that includes at least one object of
  //   type other than REGULAR is prohibited;
  // - **CONTAINER_NOT_FOUND** (3072, SECTION_CONTAINER): \
  //   object storage container not found;
  // - **TOKEN_NOT_FOUND** (4096, SECTION_SESSION): \
  //   (for trusted object preparation) session private key does not exist or has
  // been deleted;
  // - **TOKEN_EXPIRED** (4097, SECTION_SESSION): \
  //   provided session token has expired.
  rpc Put(stream PutRequest) returns (PutResponse);

  // Delete the object from a container. There is no immediate removal
  // guarantee. Object will be marked for removal and deleted eventually.
  //
  // Extended headers can change `Delete` behaviour:
  // * __NEOFS__NETMAP_EPOCH \
  //   Will use the requsted version of Network Map for object placement
  //   calculation.
  //
  // Please refer to detailed `XHeader` description.
  //
  // Statuses:
  // - **OK** (0, SECTION_SUCCESS): \
  //   object has been successfully marked to be removed from the container;
  // - Common failures (SECTION_FAILURE_COMMON);
  // - **ACCESS_DENIED** (2048, SECTION_OBJECT): \
  //   delete access to the object is denied;
  // - **LOCKED** (2050, SECTION_OBJECT): \
  //   deleting a locked object is prohibited;
  // - **CONTAINER_NOT_FOUND** (3072, SECTION_CONTAINER): \
  //   object container not found;
  // - **TOKEN_EXPIRED** (4097, SECTION_SESSION): \
  //   provided session token has expired.
  rpc Delete(DeleteRequest) returns (DeleteResponse);

  // Returns the object Headers without data payload. By default full header is
  // returned. If `main_only` request field is set, the short header with only
  // the very minimal information will be returned instead.
  //
  // Extended headers can change `Head` behaviour:
  // * __NEOFS__NETMAP_EPOCH \
  //   Will use the requsted version of Network Map for object placement
  //   calculation.
  //
  // Please refer to detailed `XHeader` description.
  //
  // Statuses:
  // - **OK** (0, SECTION_SUCCESS): \
  //   object header has been successfully read;
  // - Common failures (SECTION_FAILURE_COMMON);
  // - **ACCESS_DENIED** (2048, SECTION_OBJECT): \
  //   access to operation HEAD of the object is denied;
  // - **OBJECT_NOT_FOUND** (2049, SECTION_OBJECT): \
  //   object not found in container;
  // - **OBJECT_ALREADY_REMOVED** (2052, SECTION_OBJECT): \
  //   the requested object has been marked as deleted;
  // - **CONTAINER_NOT_FOUND** (3072, SECTION_CONTAINER): \
  //   object container not found;
  // - **TOKEN_EXPIRED** (4097, SECTION_SESSION): \
  //   provided session token has expired.
  rpc Head(HeadRequest) returns (HeadResponse);

  // Search objects in container. Search query allows to match by Object
  // Header's filed values. Please see the corresponding NeoFS Technical
  // Specification section for more details.
  //
  // Extended headers can change `Search` behaviour:
  // * __NEOFS__NETMAP_EPOCH \
  //   Will use the requsted version of Network Map for object placement
  //   calculation.
  //
  // Please refer to detailed `XHeader` description.
  //
  // Statuses:
  // - **OK** (0, SECTION_SUCCESS): \
  //   objects have been successfully selected;
  // - Common failures (SECTION_FAILURE_COMMON);
  // - **ACCESS_DENIED** (2048, SECTION_OBJECT): \
  //   access to operation SEARCH of the object is denied;
  // - **CONTAINER_NOT_FOUND** (3072, SECTION_CONTAINER): \
  //   search container not found;
  // - **TOKEN_EXPIRED** (4097, SECTION_SESSION): \
  //   provided session token has expired.
  rpc Search(SearchRequest) returns (stream SearchResponse);

  // Get byte range of data payload. Range is set as an (offset, length) tuple.
  // Like in `Get` method, the response uses gRPC stream. Requested range can be
  // restored by concatenation of all received payload chunks keeping the receiving
  // order.
  //
  // Extended headers can change `GetRange` behaviour:
  // * __NEOFS__NETMAP_EPOCH \
  //   Will use the requsted version of Network Map for object placement
  //   calculation.
  // * __NEOFS__NETMAP_LOOKUP_DEPTH \
  //   Will try older versions of Network Map to find an object until the depth
  //   limit is reached.
  //
  // Please refer to detailed `XHeader` description.
  //
  // Statuses:
  // - **OK** (0, SECTION_SUCCESS): \
  //   data range of the object payload has been successfully read;
  // - Common failures (SECTION_FAILURE_COMMON);
  // - **ACCESS_DENIED** (2048, SECTION_OBJECT): \
  //   access to operation RANGE of the object is denied;
  // - **OBJECT_NOT_FOUND** (2049, SECTION_OBJECT): \
  //   object not found in container;
  // - **OBJECT_ALREADY_REMOVED** (2052, SECTION_OBJECT): \
  //   the requested object has been marked as deleted.
  // - **OUT_OF_RANGE** (2053, SECTION_OBJECT): \
  //   the requested range is out of bounds;
  // - **CONTAINER_NOT_FOUND** (3072, SECTION_CONTAINER): \
  //   object container not found;
  // - **TOKEN_EXPIRED** (4097, SECTION_SESSION): \
  //   provided session token has expired.
  rpc GetRange(GetRangeRequest) returns (stream GetRangeResponse);

  // Returns homomorphic or regular hash of object's payload range after
  // applying XOR operation with the provided `salt`. Ranges are set of (offset,
  // length) tuples. Hashes order in response corresponds to the ranges order in
  // the request. Note that hash is calculated for XORed data.
  //
  // Extended headers can change `GetRangeHash` behaviour:
  // * __NEOFS__NETMAP_EPOCH \
  //   Will use the requsted version of Network Map for object placement
  //   calculation.
  // * __NEOFS__NETMAP_LOOKUP_DEPTH \
  //   Will try older versions of Network Map to find an object until the depth
  //   limit is reached.
  //
  // Please refer to detailed `XHeader` description.
  //
  // Statuses:
  // - **OK** (0, SECTION_SUCCESS): \
  //   data range of the object payload has been successfully hashed;
  // - Common failures (SECTION_FAILURE_COMMON);
  // - **ACCESS_DENIED** (2048, SECTION_OBJECT): \
  //   access to operation RANGEHASH of the object is denied;
  // - **OBJECT_NOT_FOUND** (2049, SECTION_OBJECT): \
  //   object not found in container;
  // - **OUT_OF_RANGE** (2053, SECTION_OBJECT): \
  //   the requested range is out of bounds;
  // - **CONTAINER_NOT_FOUND** (3072, SECTION_CONTAINER): \
  //   object container not found;
  // - **TOKEN_EXPIRED** (4097, SECTION_SESSION): \
  //   provided session token has expired.
  rpc GetRangeHash(GetRangeHashRequest) returns (GetRangeHashResponse);
}

// GET object request
message GetRequest {
  // GET Object request body
  message Body {
    // Address of the requested object
    neo.fs.v2.refs.Address address = 1;

    // If `raw` flag is set, request will work only with objects that are
    // physically stored on the peer node
    bool raw = 2;
  }
  // Body of get object request message.
  Body body = 1;

  // Carries request meta information. Header data is used only to regulate
  // message transport and does not affect request execution.
  neo.fs.v2.session.RequestMetaHeader meta_header = 2;

  // Carries request verification information. This header is used to
  // authenticate the nodes of the message route and check the correctness of
  // transmission.
  neo.fs.v2.session.RequestVerificationHeader verify_header = 3;
}

// GET object response
message GetResponse {
  // GET Object Response body
  message Body {
    // Initial part of the `Object` structure stream. Technically it's a
    // set of all `Object` structure's fields except `payload`.
    message Init {
      // Object's unique identifier.
      neo.fs.v2.refs.ObjectID object_id = 1;

      // Signed `ObjectID`
      neo.fs.v2.refs.Signature signature = 2;

      // Object metadata headers
      Header header = 3;
    }
    // Single message in the response stream.
    oneof object_part {
      // Initial part of the object stream
      Init init = 1;

      // Chunked object payload
      bytes chunk = 2;

      // Meta information of split hierarchy for object assembly.
      SplitInfo split_info = 3;
    }
  }
  // Body of get object response message.
  Body body = 1;

  // Carries response meta information. Header data is used only to regulate
  // message transport and does not affect request execution.
  neo.fs.v2.session.ResponseMetaHeader meta_header = 2;

  // Carries response verification information. This header is used to
  // authenticate the nodes of the message route and check the correctness of
  // transmission.
  neo.fs.v2.session.ResponseVerificationHeader verify_header = 3;
}

// PUT object request
message PutRequest {
  // PUT request body
  message Body {
    // Newly created object structure parameters. If some optional parameters
    // are not set, they will be calculated by a peer node.
    message Init {
      // ObjectID if available.
      neo.fs.v2.refs.ObjectID object_id = 1;

      // Object signature if available
      neo.fs.v2.refs.Signature signature = 2;

      // Object's Header
      Header header = 3;

      // Number of the object copies to store within the RPC call. By default
      // object is processed according to the container's placement policy.
      uint32 copies_number = 4;
    }
    // Single message in the request stream.
    oneof object_part {
      // Initial part of the object stream
      Init init = 1;

      // Chunked object payload
      bytes chunk = 2;
    }
  }
  // Body of put object request message.
  Body body = 1;

  // Carries request meta information. Header data is used only to regulate
  // message transport and does not affect request execution.
  neo.fs.v2.session.RequestMetaHeader meta_header = 2;

  // Carries request verification information. This header is used to
  // authenticate the nodes of the message route and check the correctness of
  // transmission.
  neo.fs.v2.session.RequestVerificationHeader verify_header = 3;
}

// PUT Object response
message PutResponse {
  // PUT Object response body
  message Body {
    // Identifier of the saved object
    neo.fs.v2.refs.ObjectID object_id = 1;
  }
  // Body of put object response message.
  Body body = 1;

  // Carries response meta information. Header data is used only to regulate
  // message transport and does not affect request execution.
  neo.fs.v2.session.ResponseMetaHeader meta_header = 2;

  // Carries response verification information. This header is used to
  // authenticate the nodes of the message route and check the correctness of
  // transmission.
  neo.fs.v2.session.ResponseVerificationHeader verify_header = 3;
}

// Object DELETE request
message DeleteRequest {
  // Object DELETE request body
  message Body {
    // Address of the object to be deleted
    neo.fs.v2.refs.Address address = 1;
 }
  // Body of delete object request message.
  Body body = 1;

  // Carries request meta information. Header data is used only to regulate
  // message transport and does not affect request execution.
  neo.fs.v2.session.RequestMetaHeader meta_header = 2;

  // Carries request verification information. This header is used to
  // authenticate the nodes of the message route and check the correctness of
  // transmission.
  neo.fs.v2.session.RequestVerificationHeader verify_header = 3;
}

// DeleteResponse body is empty because we cannot guarantee permanent object
// removal in distributed system.
message DeleteResponse {
  // Object DELETE Response has an empty body.
  message Body {
    // Address of the tombstone created for the deleted object
    neo.fs.v2.refs.Address tombstone = 1;
  }

  // Body of delete object response message.
  Body body = 1;

  // Carries response meta information. Header data is used only to regulate
  // message transport and does not affect request execution.
  neo.fs.v2.session.ResponseMetaHeader meta_header = 2;

  // Carries response verification information. This header is used to
  // authenticate the nodes of the message route and check the correctness of
  // transmission.
  neo.fs.v2.session.ResponseVerificationHeader verify_header = 3;
}

// Object HEAD request
message HeadRequest {
  // Object HEAD request body
  message Body {
    // Address of the object with the requested Header
    neo.fs.v2.refs.Address address = 1;

    // Return only minimal header subset
    bool main_only = 2;

    // If `raw` flag is set, request will work only with objects that are
    // physically stored on the peer node
    bool raw = 3;
  }
  // Body of head object request message.
  Body body = 1;

  // Carries request meta information. Header data is used only to regulate
  // message transport and does not affect request execution.
  neo.fs.v2.session.RequestMetaHeader meta_header = 2;

  // Carries request verification information. This header is used to
  // authenticate the nodes of the message route and check the correctness of
  // transmission.
  neo.fs.v2.session.RequestVerificationHeader verify_header = 3;
}

// Tuple of a full object header and signature of an `ObjectID`. \
// Signed `ObjectID` is present to verify full header's authenticity through the
// following steps:
//
// 1. Calculate `SHA-256` of the marshalled `Header` structure
// 2. Check if the resulting hash matches `ObjectID`
// 3. Check if `ObjectID` signature in `signature` field is correct
message HeaderWithSignature {
  // Full object header
  Header header = 1 [json_name = "header"];

  // Signed `ObjectID` to verify full header's authenticity
  neo.fs.v2.refs.Signature signature = 2 [json_name = "signature"];
}

// Object HEAD response
message HeadResponse {
  // Object HEAD response body
  message Body {
    // Requested object header, it's part or meta information about split
    // object.
    oneof head{
      // Full object's `Header` with `ObjectID` signature
      HeaderWithSignature header = 1;

      // Short object header
      ShortHeader short_header = 2;

      // Meta information of split hierarchy.
      SplitInfo split_info = 3;
    }
  }
  // Body of head object response message.
  Body body = 1;

  // Carries response meta information. Header data is used only to regulate
  // message transport and does not affect request execution.
  neo.fs.v2.session.ResponseMetaHeader meta_header = 2;

  // Carries response verification information. This header is used to
  // authenticate the nodes of the message route and check the correctness of
  // transmission.
  neo.fs.v2.session.ResponseVerificationHeader verify_header = 3;
}

// Object Search request
message SearchRequest {
  // Object Search request body
  message Body {
    // Container identifier were to search
    neo.fs.v2.refs.ContainerID container_id = 1;

    // Version of the Query Language used
    uint32 version = 2;
    // Filter structure checks if the object header field or the attribute content
    // matches a value.
    //
    // If no filters are set, search request will return all objects of the
    // container, including Regular object, Tombstones and Storage Group
    // objects. Most human users expect to get only object they can directly
    // work with. In that case, `$Object:ROOT` filter should be used.
    //
    // By default `key` field refers to the corresponding object's `Attribute`.
    // Some Object's header fields can also be accessed by adding `$Object:`
    // prefix to the name. Here is the list of fields available via this prefix:
    //
    // * $Object:version \
    //   version
    // * $Object:objectID \
    //   object_id
    // * $Object:containerID \
    //   container_id
    // * $Object:ownerID \
    //   owner_id
    // * $Object:creationEpoch \
    //   creation_epoch
    // * $Object:payloadLength \
    //   payload_length
    // * $Object:payloadHash \
    //   payload_hash
    // * $Object:objectType \
    //   object_type
    // * $Object:homomorphicHash \
    //   homomorphic_hash
    // * $Object:split.parent \
    //   object_id of parent
    // * $Object:split.splitID \
    //   16 byte UUIDv4 used to identify the split object hierarchy parts
    //
    // There are some well-known filter aliases to match objects by certain
    // properties:
    //
    // * $Object:ROOT \
    //   Returns only `REGULAR` type objects that are not split or that are the top
    //   level root objects in a split hierarchy. This includes objects not
    //   present physically, like large objects split into smaller objects
    //   without a separate top-level root object. Objects of other types like
    //   StorageGroups and Tombstones will not be shown. This filter may be
    //   useful for listing objects like `ls` command of some virtual file
    //   system. This filter is activated if the `key` exists, disregarding the
    //   value and matcher type.
    // * $Object:PHY \
    //   Returns only objects physically stored in the system. This filter is
    //   activated if the `key` exists, disregarding the value and matcher type.
    //
    // Note: using filters with a key with prefix `$Object:` and match type
    // `NOT_PRESENT `is not recommended since this is not a cross-version approach.
    // Behavior when processing this kind of filters is undefined.
    message Filter {
      // Match type to use
      MatchType match_type = 1 [json_name = "matchType"];

      // Attribute or Header fields to match
      string key = 2 [json_name = "key"];

      // Value to match
      string value = 3 [json_name = "value"];
    }
    // List of search expressions
    repeated Filter filters = 3;
  }
  // Body of search object request message.
  Body body = 1;

  // Carries request meta information. Header data is used only to regulate
  // message transport and does not affect request execution.
  neo.fs.v2.session.RequestMetaHeader meta_header = 2;

  // Carries request verification information. This header is used to
  // authenticate the nodes of the message route and check the correctness of
  // transmission.
  neo.fs.v2.session.RequestVerificationHeader verify_header = 3;
}

// Search response
message SearchResponse {
  // Object Search response body
  message Body {
    // List of `ObjectID`s that match the search query
    repeated neo.fs.v2.refs.ObjectID id_list = 1;
  }
  // Body of search object response message.
  Body body = 1;

  // Carries response meta information. Header data is used only to regulate
  // message transport and does not affect request execution.
  neo.fs.v2.session.ResponseMetaHeader meta_header = 2;

  // Carries response verification information. This header is used to
  // authenticate the nodes of the message route and check the correctness of
  // transmission.
  neo.fs.v2.session.ResponseVerificationHeader verify_header = 3;
}

// Object payload range.Ranges of zero length SHOULD be considered as invalid.
message Range {
  // Offset of the range from the object payload start
  uint64 offset = 1;

  // Length in bytes of the object payload range
  uint64 length = 2;
}

// Request part of object's payload
message GetRangeRequest {
  // Byte range of object's payload request body
  message Body {
    // Address of the object containing the requested payload range
    neo.fs.v2.refs.Address address = 1;

    // Requested payload range
    Range range = 2;

    // If `raw` flag is set, request will work only with objects that are
    // physically stored on the peer node.
    bool raw = 3;
  }

  // Body of get range object request message.
  Body body = 1;

  // Carries request meta information. Header data is used only to regulate
  // message transport and does not affect request execution.
  neo.fs.v2.session.RequestMetaHeader meta_header = 2;

  // Carries request verification information. This header is used to
  // authenticate the nodes of the message route and check the correctness of
  // transmission.
  neo.fs.v2.session.RequestVerificationHeader verify_header = 3;
}

// Get part of object's payload
message GetRangeResponse {
  // Get Range response body uses streams to transfer the response. Because
  // object payload considered a byte sequence, there is no need to have some
  // initial preamble message. The requested byte range is sent as a series
  // chunks.
  message Body {
    // Requested object range or meta information about split object.
    oneof range_part{
      // Chunked object payload's range.
      bytes chunk = 1;

      // Meta information of split hierarchy.
      SplitInfo split_info = 2;
    }
  }

  // Body of get range object response message.
  Body body = 1;

  // Carries response meta information. Header data is used only to regulate
  // message transport and does not affect request execution.
  neo.fs.v2.session.ResponseMetaHeader meta_header = 2;

  // Carries response verification information. This header is used to
  // authenticate the nodes of the message route and check the correctness of
  // transmission.
  neo.fs.v2.session.ResponseVerificationHeader verify_header = 3;
}

// Get hash of object's payload part
message GetRangeHashRequest {
  // Get hash of object's payload part request body.
  message Body {
    // Address of the object that containing the requested payload range
    neo.fs.v2.refs.Address address = 1;

    // List of object's payload ranges to calculate homomorphic hash
    repeated Range ranges = 2;

    // Binary salt to XOR object's payload ranges before hash calculation
    bytes salt = 3;

    // Checksum algorithm type
    neo.fs.v2.refs.ChecksumType type = 4;
  }
  // Body of get range hash object request message.
  Body body = 1;

  // Carries request meta information. Header data is used only to regulate
  // message transport and does not affect request execution.
  neo.fs.v2.session.RequestMetaHeader meta_header = 2;

  // Carries request verification information. This header is used to
  // authenticate the nodes of the message route and check the correctness of
  // transmission.
  neo.fs.v2.session.RequestVerificationHeader verify_header = 3;
}

// Get hash of object's payload part
message GetRangeHashResponse {
  // Get hash of object's payload part response body.
  message Body {
    // Checksum algorithm type
    neo.fs.v2.refs.ChecksumType type = 1;

    // List of range hashes in a binary format
    repeated bytes hash_list = 2;
  }
  // Body of get range hash object response message.
  Body body = 1;

  // Carries response meta information. Header data is used only to regulate
  // message transport and does not affect request execution.
  neo.fs.v2.session.ResponseMetaHeader meta_header = 2;

  // Carries response verification information. This header is used to
  // authenticate the nodes of the message route and check the correctness of
  // transmission.
  neo.fs.v2.session.ResponseVerificationHeader verify_header = 3;
}