Use TTL for blobovnicza tree cache #866

New Issue

Closed

opened 2023-12-13 10:51:03 +00:00 by dstepanov-yadro · 7 comments

dstepanov-yadro commented

2023-12-13 10:51:03 +00:00

Collaborator

Blobovnicza tree holds blobovniczas opened in cache even if they are not used. It consumes system resources.

Expected Behavior

Blobovnicza tree must close unused blobovniczas in cache after some amount of time.

Current Behavior

Blobovnicza tree holds unused blobovniczas in cache.

Blobovnicza tree holds blobovniczas opened in cache even if they are not used. It consumes system resources. ## Expected Behavior Blobovnicza tree must close unused blobovniczas in cache after some amount of time. ## Current Behavior Blobovnicza tree holds unused blobovniczas in cache.

dstepanov-yadro added the

bug

frostfs-node

triage

labels 2023-12-13 10:51:03 +00:00

fyrchik commented

2023-12-14 11:02:53 +00:00

Owner

Why is it labeled as bug?

Why is it labeled as `bug`?

dstepanov-yadro commented

2023-12-14 12:36:33 +00:00

Poster

Collaborator

Resource using without any reason looks like a bug.

acid-ant self-assigned this 2023-12-14 17:06:46 +00:00

fyrchik added

enhancement

and removed

triage

bug

labels 2023-12-18 07:26:10 +00:00

fyrchik commented

2023-12-18 07:27:19 +00:00

Owner

It was intended (because we use LRU cache) and the reason is that it was use "less than 1000 blobovniczas ago".
Let's fix this, but I have removed the label bug -- this is very much a new feature to me.

It was intended (because we use LRU cache) and the reason is that it was use "less than 1000 blobovniczas ago". Let's fix this, but I have removed the label bug -- this is very much a new feature to me.

acid-ant referenced a pull request that will close this issue

2023-12-18 09:55:06 +00:00

node: Use TTL for blobovnicza tree cache #875

fyrchik referenced this issue from a commit

2023-12-19 16:36:37 +00:00

[#866] Use TTL for blobovnicza tree cache

fyrchik closed this issue

2023-12-19 16:36:41 +00:00

fyrchik added this to the v0.38.0 milestone 2023-12-22 07:23:44 +00:00

acid-ant commented

2024-02-15 08:52:30 +00:00

Collaborator

Test case:

Set new parameter in node config for blobovnicza:

...
shard:
      default: 
        blobovnicza: 
          opened_cache_ttl: 1m  # ttl for opened database file 
...

Put file in blobovnicza tree
Wait for a period ~ opened_cache_ttl
There are should be no opened blobovnicza's db files by node process

Test case: - Set new parameter in `node` config for `blobovnicza`: ``` ... shard: default: blobovnicza: opened_cache_ttl: 1m # ttl for opened database file ... ``` - Put file in blobovnicza tree - Wait for a period ~ `opened_cache_ttl` - There are should be no opened blobovnicza's db files by `node` process

acid-ant reopened this issue

2024-03-01 16:20:25 +00:00

acid-ant commented

2024-03-01 16:23:20 +00:00

Collaborator

Reverted due to the problem with hashicorp's expirable cache.
Details - #1004
Will be fixed in scope of #1017

Reverted due to the problem with hashicorp's expirable cache. Details - https://git.frostfs.info/TrueCloudLab/frostfs-node/issues/1004 Will be fixed in scope of https://git.frostfs.info/TrueCloudLab/frostfs-node/pulls/1017

acid-ant referenced this issue

2024-03-01 16:37:47 +00:00

Revert "[#866] Use TTL for blobovnicza tree cache" #1018

dstepanov-yadro referenced this issue from a commit

2024-03-04 06:55:54 +00:00

Revert "[#866] Use TTL for blobovnicza tree cache"

acid-ant referenced this issue

2024-03-04 19:58:26 +00:00

blobovnicza: Use TTL for blobovnicza tree cache #1017

acid-ant commented

2024-03-04 20:02:30 +00:00

Collaborator

Checked the following list of caches:

https://github.com/bluele/gcache - the same behavior as for hashicorp
https://github.com/dgraph-io/ristretto - no expiration functionality
https://github.com/golang/groupcache - no expiration functionality
https://github.com/coocood/freecache - no expiration functionality(only for Set)
required to build expiration logic under the cache
no internal mechanism to remove from cache expired elements
https://github.com/patrickmn/go-cache - no "onevict" functionality
https://github.com/allegro/bigcache - stored only byte[]
https://github.com/Yesphet/lru-cache - no "onevict" functionality
https://github.com/tigerinus/gocache - no "onevict" functionality
https://github.com/erwint/ttlcache - no lru
https://github.com/jellydator/ttlcache - no lru

https://github.com/go-pkgz/expirable-cache implements LRU and allow deleting expired in external routine.

Checked the following list of caches: - https://github.com/bluele/gcache - the same behavior as for hashicorp - https://github.com/dgraph-io/ristretto - no expiration functionality - https://github.com/golang/groupcache - no expiration functionality - https://github.com/coocood/freecache - no expiration functionality(only for Set) required to build expiration logic under the cache no internal mechanism to remove from cache expired elements - https://github.com/patrickmn/go-cache - no "onevict" functionality - https://github.com/allegro/bigcache - stored only byte[] - https://github.com/Yesphet/lru-cache - no "onevict" functionality - https://github.com/tigerinus/gocache - no "onevict" functionality - https://github.com/erwint/ttlcache - no lru - https://github.com/jellydator/ttlcache - no lru https://github.com/go-pkgz/expirable-cache implements LRU and allow deleting expired in external routine.

fyrchik modified the milestone from v0.38.0 to v0.39.0

2024-03-05 07:41:49 +00:00

fyrchik commented

2024-05-14 14:10:00 +00:00

Owner

Closed via #1017

fyrchik closed this issue

2024-05-14 14:10:02 +00:00