S3FIFO Eviction Algorithm

[williamnixon20]

This PR adds the S3FIFO eviction algorithm, as described here.

Running cachebench results show that S3FIFO achieves the highest hit rate among all evaluated algorithms. Throughput wise, it performs similarly with TinyLFU.

Our implementation adapts the existing TinyLFUMM.

Algorithm Detail
The algorithm maintains three structures:

• S — a small queue for newly inserted items (10% of size)
• M — a main FIFO queue (90% of size)
• G — a ghost queue for items evicted from S (size of M), only tracks the object key hash.

The general idea of the algorithm is:

• Evict early: items in S are aggressively filtered to not pollute the main cache.
• Promote late: promotion/reinsertion occurs only when an accessed item reaches the tail.

Each object carries one access bit that records whether it has been accessed since it was inserted or last inspected.

Items in the tail are:

• If accessed and in S, moved to M.
• If accessed and in M, reinserted to head of M.
• If not accessed, evict. If this item is evicted from S, move it to the ghost queue.

Insertions are done as such:

• If the item exists in ghost, directly insert to M.
• Else insert to S.

We try to do eviction from S if the size > expected size (10%). Else, we try to evict from main.

Eviction iterator starts from tail and skips accessed items. Promotions (reinsertions) are done 1x before iterator is created.

Benchmark:
We evaluated S3FIFO using the test configurations in test_configs/hit_ratio with 48 threads and an 8 GB cache. We compared this against other existing MMContainers(LRU2Q, LRU, and TinyLFU). We use the DebugRel release version for testing.

The results can be found here
S3 FIFO Benchmark (2).pdf

Miss ratio:
S3FIFO consistently outperforms LRU, LRU2Q, and TinyLFU across all workloads.

Throughput:
Throughput is comparable to TinyLFU.

Notes:

• Adds an atomic FIFO ghost queue implemented as a bucketed hash table storing only object hashes. Each entry is tagged with a logical timestamp (number of inserts into G). Entries expire based on FIFO order: if current time is N, any entry with timestamp < (N − |G|) is considered expired. Entries are removed either when accessed (consume-on-hit) or when overwritten due to hash collisions.