RebalancePolicy and allocation failures

[OmerKepten]

Hello,
I see allocation failures during my run. Here you mention setting the RebalancePolicy to default.

1. by RebalancePolicy, do you mean RebalanceStrategy?
2. what is the default policy? currently we use HitsPerSlab. Does it not trigger rebalancing per allocation failure?
3. I have a single-sized dataset. Does this mean allocation will only fail once the cache completely used up its full size?
4. is there a way to directly trigger slab rebalancing / eviction from slab once i see an allocation failure?
   Thanks!

[OmerKepten]

Some additional questions:

1. what will happen if i do not set slab key sizes when i create the pool? will a single slab be created and used for all key sizes or will new slabs be created depending on the dataset? per the example at https://cachelib.org/docs/Cache_Library_Architecture_Guide/slab_rebalancing/#:~:text=For%20CacheLib%2C%20each%20slab%20is,allocation%20classes%20have%20any%20memory. it sounds a single slab will be at work, and will be able to consume all the cache memory, until a newly sized value shows up. What is the granularity of the classes in this case (i.e. when you decide a new class is required?)?
2. what will happen if i create multiple slabs, but only fill the cache with constant sized values?
   Thanks in advance

[haowu14]

If you don't specify a list of allocation classes at creation, cachelib uses the default parameters here and the logic here to generate a list of sizes.

By default, there is no slabs in any allocation classes. As traffic comes in, cachelib figures out the allocation class it belongs to by looking at its size and finding the smallest allocation class that can hold this item. Then that allocation class will ask for a slab from the slab allocator (assuming the cache isn't full and there's still free slabs. Otherwise, this is an allocation failure). Then that allocation class now has one slab (4MB).
In your workload, it seems all the slab space should've gone to only one allocation class.

[haowu14]

If you have a single-sized dataset, rebalancing strategy likely won't be taking effect because there is only one allocation class as mentioned here.

Having allocation failure is not just that the cache is full, but also all the eviction attempts failed. If interested, you can take a look at this section.
The code reference, especially the line numbers, are unfortunately outdated.

Regarding rebalancing:

Yes, I'll fix the wording and change RebalancePolicy to RebalanceStrategy.
By default:
The strategy is to move slabs from the allocation class with the most slabs, to the one allocation class with the most allocation failure. If these two happen to be the same, there will be no rebalancing.
The pool rebalancer will wake up on ever allocation failure by default, regardless of the rebalancing strategy you use. You can disable it before creating cachelib by setting disableForcedWakeup = true. https://github.com/facebook/CacheLib/blob/main/cachelib/allocator/CacheAllocatorConfig.h#L250.

[OmerKepten]

Thank you for helpful answers!

1. regarding the comments here - if the allocation triggers an eviction, would the allocation operation wait until the eviction is done, and will return a valid handle if successful? or will it return with an invalid handle, as the eviction is done by a different thread? from the code it looks like it waits, but i wanted to make sure :)
2. what are possible reasons for eviction failure? could it be related in any way to the way the cache was configured? What is the best way to ensure every eviction attempt will succeed? is setting setEvictionSearchLimit to 0 a good way?
3. From your answers i understand it is harmless to create multiple slabs and not use all of them, correct? this is useful for when the dataset is not priorly known

Thanks again for your help!

[haowu14]

1. Yes it waits. It also will attempt a number of evictions instead of just one (configured here).
2. Most common reason is an item is still has an active handle when the allocation thread tries to evict it. An active handle on the item is created when cache->find(itemKey) returns a hit. The handle stays active as long as it is in scope. Depending on your application, you can make a copy of the cached value and release the handle earlier.
3. Correct. There might be a small benefit of specifying your allocation class if you know exactly your workload in terms of size. (Note that the allocation size takes both key size and value size).

[pbhandar2]

Closing this discussion as it has been inactive. If this is still relevant, then please create a new discussion post and we will get back to you as soon as we can. Thank you!