Performance when configuration is large and lots of churn

[BanovMiroslav]

Welcome

• Yes, I've searched for similar issues on GitHub and found none.
• Yes, I've included all information below (version, config, etc).

Description of the problem

I have around 50k items in my redis backend for my traefik cluster and am trying to prepare for even more and the system I am building is doing a lot of aggressive archival and fast provisioning, so there is a lot of churn. I am also doing load test and so provisioning and deleting lots of things on bulk, so getting even more churn from that. But as the load test is creating more routes, things get slower, both the redis and the traefik cluster are getting high CPU use. Here are some stats in just 4.5 days.

Command	Calls
SCAN	1.46 billion
MGET	568k
KEYS	51k
SET/DEL (actual writes)	~300k

Version of Valkeyrie

latest v1.2.1 has the problem too

Version of store

Redis 7.1

Go environment

Code example or link to a public repository

[ldez]

Fixed by #11, #12, #13