Go implementation of FrostFS SDK
Find a file
Denis Kirillov 59b49dd7e6 [#165] pool: update WaitForContainerPresence
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-03-21 12:41:08 +03:00
.github/workflows [#37] ci: Add github workflows 2021-10-15 17:37:18 +03:00
accounting [#170] accounting: Remove message sub-string from V2-methods 2022-03-21 12:37:53 +03:00
acl [#106] Add extra descriptions for constants 2022-01-11 11:01:37 +03:00
audit [#170] audit: Refactor and document package functionality 2022-03-21 12:37:53 +03:00
checksum [#54] tests: unify test generator names 2021-12-01 10:34:54 +03:00
client [#170] accounting: Remove message sub-string from V2-methods 2022-03-21 12:37:53 +03:00
container [#168] container: Remove pointer tricks from setAttribute 2022-03-15 16:59:59 +03:00
eacl [#168] eacl: Replace pointer slices with non-pointer slices 2022-03-15 16:59:59 +03:00
logger [#26] *: Move pkg content to root 2021-07-28 13:47:27 +03:00
netmap [#168] netmap: Add simple selector benchmark 2022-03-15 16:59:59 +03:00
object [#172] object: Allow to marshal SplitInfo to JSON 2022-03-15 17:00:25 +03:00
owner [#134] owner: Add helpers for working script hash 2022-02-24 15:53:42 +03:00
policy [#168] policy: Adopt replacement of pointer slices with struct slices 2022-03-15 16:59:59 +03:00
pool [#165] pool: update WaitForContainerPresence 2022-03-21 12:41:08 +03:00
reputation [#168] reputation: Replace []*reputation.Trust with []reputation.Trust 2022-03-15 16:59:59 +03:00
resolver [#93] Return NNS resolver 2021-12-14 15:27:04 +03:00
session [#158] client: Support object context in session token setter 2022-03-03 14:27:29 +03:00
signature [#157] signature: Change scheme selection 2022-03-03 09:39:21 +03:00
storagegroup [#168] storagegroup: Replace []*oid.ID with []oid.ID 2022-03-15 16:59:59 +03:00
subnet [#54] tests: unify test generator names 2021-12-01 10:34:54 +03:00
token [#150] util/signature: Simplify public interface 2022-02-25 12:42:38 +03:00
util/signature [#157] util/signature: Revive removed features 2022-03-03 09:39:21 +03:00
version [#54] tests: unify test generator names 2021-12-01 10:34:54 +03:00
.gitattributes [#3] policy: use ANTLRv4 parser generator 2021-06-15 11:42:14 +03:00
.gitignore Add vendor directory to gitignore 2021-07-07 17:20:36 +03:00
.golangci.yml [#37] ci: Replace golint by revive 2021-10-15 17:37:18 +03:00
go.mod Update neofs-api-go to v2.12.1 2022-03-15 17:18:05 +03:00
go.sum Update neofs-api-go to v2.12.1 2022-03-15 17:18:05 +03:00
LICENSE Initial commit 2021-02-25 11:35:04 +03:00
Makefile [nspcc-dev/neofs-node#166] *: Add project files 2021-05-20 23:57:51 +03:00
README.md [#131] client: Name all methods and types the same way 2022-02-18 17:01:08 +03:00

neofs-sdk-go

Go implementation of NeoFS SDK. It contains high-level version-independent wrappers for structures from neofs-api-go as well as helper functions for simplifying node/dApp implementations.

Repository structure

accounting

Contains fixed-point Decimal type for performing balance calculations.

eacl

Contains Extended ACL types for fine-grained access control. There is also a reference implementation of checking algorithm which is used in NeoFS node.

checksum

Contains Checksum type encapsulating checksum as well as it's kind. Currently Sha256 and Tillich-Zemor hashsum are in use.

owner

owner.ID type represents single account interacting with NeoFS. In v2 version of protocol it is just raw bytes behing base58-encoded address in Neo blockchain. Note that for historical reasons it contains version prefix and checksum in addition to script-hash.

token

Contains Bearer token type with several NeoFS-specific methods.

resolver

In NeoFS there are 2 types of name resolution: DNS and NNS. NNS stands for Neo Name Service is just a contract deployed on a Neo blockchain. Basically, NNS is just a DNS-on-chain which can be used for resolving container nice-names as well as any other name in dApps. See our CoreDNS plugin for the example of how NNS can be integrated in DNS.

session

To help lightweight clients interact with NeoFS without sacrificing trust, NeoFS has a concept of session token. It is signed by client and allows any node with which a session is established to perform certain actions on behalf of the user.

client

Contains client for working with NeoFS.

c, _ := client.New(
    client.WithAddress("localhost:40005"), // endpoint address
    client.WithDefaultPrivateKey(key),     // private key for request signing
    client.WithNeoFSErrorHandling(),       // enable erroneous status parsing
    client.WithTLSConfig(&tls.Config{}))   // custom TLS configuration
    
ctx, cancel := context.WithTimeout(context.Background(), 5 * time.Second)
defer cancel()

res, err := c.BalanceGet(ctx, owner)
if err != nil {
    return
}

fmt.Printf("Balance for %s: %s\n", owner, res.Amount())

Response status

In NeoFS every operation can fail on multiple levels, so a single error doesn't suffice, e.g. consider a case when object was put on 4 out of 5 replicas. Thus, all request execution details are contained in Status returned from every RPC call. dApp can inspect them if needed and perform any desired action. In the case above we may want to report these details to the user as well as retry an operation, possibly with different parameters. Status wire-format is extendable and each node can report any set of details it wants. The set of reserved status codes can be found in NeoFS API. There is also a client.WithNeoFSErrorHandling() to seamlessly convert erroneous statuses into Go error type.

policy

Contains helpers allowing conversion of placing policy from/to JSON representation and SQL-like human-readable language.

p, _ := policy.Parse(`
    REP 2
    SELECT 6 FROM F
    FILTER StorageType EQ SSD AS F`)

// Convert parsed policy back to human-readable text and print.
println(strings.Join(policy.Encode(p), "\n"))

netmap

Contains CRUSH-like implementation of container node selection algorithm. Relevant details are described in this paper http://ceur-ws.org/Vol-2344/short10.pdf . Note that it can be outdated in some details.

netmap/json_tests subfolder contains language-agnostic tests for selection algorithm.

import (
    "github.com/nspcc-dev/neofs-sdk-go/netmap"
    "github.com/nspcc-dev/neofs-sdk-go/object"
)

func placementNodes(addr *object.Address, p *netmap.PlacementPolicy, neofsNodes []netmap.NodeInfo) {
    // Convert list of nodes in NeoFS API format to the intermediate representation.
    nodes := netmap.NodesFromInfo(nodes)

    // Create new netmap (errors are skipped for the sake of clarity). 
    nm, _ := NewNetmap(nodes)

    // Calculate nodes of container.
    cn, _ := nm.GetContainerNodes(p, addr.ContainerID().ToV2().GetValue())

    // Return list of nodes for each replica to place object on in the order of priority.
    return nm.GetPlacementVectors(cn, addr.ObjectID().ToV2().GetValue())
}

pool

Simple pool for managing connections to NeoFS nodes.

acl, checksum, version, signature

Contain simple API wrappers.

logger

Wrapper over zap.Logger which is used across NeoFS codebase.

util

Utilities for working with signature-related code.