TrueCloudLab/frostfs-node
20
1
Fork 28
Code Issues 107 Pull requests 4 Projects Releases 2 Packages 4 Activity Actions

[#676] services/container: Cache the results of read operations

Browse source 
In previous implementation Container service handlers didn't cache the
results of `Get` / `GetEACL` / `List` operations. As a consequence of this,
high load on the service caused neo-go client's connection errors. To avoid
this there is a need to use cache. Object service already uses `Get` and
`GetEACL` caches.

Implement cache of `List` results. Share already implemented cache of Object
service with the Container one. Provide new instance of read-only container
storage (defined as an interface)to morph executor's constructor on which
container service is based. Write operations remained unchanged.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
This commit is contained in:
Leonard Lyubich 2021-08-30 14:16:41 +03:00 • committed by Alex Vanin
parent c54f524df9
commit e738699fcc
5 changed files with 115 additions and 30 deletions
Download patch file Download diff file Expand all files Collapse all files

48
cmd/neofs-node/cache.go
View file

@ -9,8 +9,10 @@ import (
containerSDK "github.com/nspcc-dev/neofs-api-go/pkg/container"
cid "github.com/nspcc-dev/neofs-api-go/pkg/container/id"
netmapSDK "github.com/nspcc-dev/neofs-api-go/pkg/netmap"
"github.com/nspcc-dev/neofs-api-go/pkg/owner"
"github.com/nspcc-dev/neofs-node/pkg/core/container"
"github.com/nspcc-dev/neofs-node/pkg/core/netmap"
"github.com/nspcc-dev/neofs-node/pkg/morph/client/container/wrapper"
"github.com/nspcc-dev/neofs-node/pkg/services/object/acl/eacl"
)
@ -227,3 +229,49 @@ func (s *lruNetmapSource) getNetMapByEpoch(epoch uint64) (*netmapSDK.Netmap, err
func (s *lruNetmapSource) Epoch() (uint64, error) {
return s.netState.CurrentEpoch(), nil
}
// wrapper over TTL cache of values read from the network
// that implements container lister.
type ttlContainerLister ttlNetCache
func newCachedContainerLister(w *wrapper.Wrapper) *ttlContainerLister {
const (
containerListerCacheSize = 100
containerListerCacheTTL = 30 * time.Second
)
lruCnrListerCache := newNetworkTTLCache(containerListerCacheSize, containerListerCacheTTL, func(key interface{}) (interface{}, error) {
var (
id *owner.ID
strID = key.(string)
)
if strID != "" {
id = owner.NewID()
err := id.Parse(strID)
if err != nil {
return nil, err
}
}
return w.List(id)
})
return (*ttlContainerLister)(lruCnrListerCache)
}
func (s *ttlContainerLister) List(id *owner.ID) ([]*cid.ID, error) {
var str string
if id != nil {
str = id.String()
}
val, err := (*ttlNetCache)(s).get(str)
if err != nil {
return nil, err
}
return val.([]*cid.ID), nil
}