Elevated and spiky request time with `CACHEOPS_INSIDEOUT`

[nicwolff]

We activated CACHEOPS_INSIDEOUT hoping to eliminate Redis timeouts that we think are related to invalidation (we're running the reaper but I think our traffic is just outpacing it).

Unfortunately we saw immediate spikes in request time, and continued high Lua memory usage and some Redis timeouts.

We expected some increase in read-request time since every cache lookup now does an extra MGET to get the conj nonces and calculate the checksum – but this was much slower, and the variance was very high.

And, we thought Lua invalidation calls would be faster since it's not scanning and deleting the long conj key sets. We're still seeing other Redis calls time out though.

Is this expected with INSIDEOUT?

[Suor]

Sorry for slow response. This is not what I would expect, I didn't test INSIDEOUT mode on any bigger caches though.

I'm afraid I won't be of much help with any theoretic advice here) For once it would be good to know which particular requests cause memory usage and timeouts. Some other metrics will help too, like min, max, average Redis response for different type of requests, i.e. get/write and maybe hit/miss.

There are two changes in how cache is managed though, which might or might not affected this:

1. Unlinks used to be mass ones and now it case by case, this may influence Redis gc behavior.
2. Cache values live longer now as they are not unlinked by events but only on access or when they expire.

The point 2 may means that you should have substantially different settings both cacheops ones and Redis ones, i.e. lower expire timeouts in cacheops and proper maxmemory and friends in Redis.

P.S. Looking into these docs might help.