compaction

In all fairness, the compaction is a major pain in Mongo.

Our use of multiple databases got hung up on leveldb's single compaction thread.

I do need to review the compaction algorithms to see if there is benefit to Basho.

Or Cassandra compaction locks up a node so hard its peers think its down, turning your cluster into a recursive tire fire of GC.

These holes are not adequately re-used and a compaction is required to make that space available again.

And compaction is trivially non-blocking in a CoW dataset.

GCs struggle much more with long-lived objects, which eventually require compaction, whose cost rises linearly with the size of the live data set. This is the cause of the problems with large heaps mentioned in the comments below.

Unfortunately, compaction also requires that nodes be taken out of service.

Most of these objects are in the eden space, but we can run into memory fragmentation issues, which leads to eventual long compaction cycles.

I get a little worked up about this because I cant think of another database that handles compaction this poorly, but feel free to correct me if Im wrong.

There, not only are there trees and original buildings, but the businesses as housing are intermingled, plus they go up, allowing a much greater compaction of space.

The initial memtable compaction places this data in its own segment and then we can push that segment down to the deepest level without ever compacting it again.

RocksDB implemented multi-threaded write, which means that parallel compactions on different parts of the database can occur simultaneously.

Nothing hugely special springs to mind - there's some memory compaction issue that is still being discussed, but I don't think there's anything particularly scary or special that stands out over everything else in the -rc1.

While there are a number of great generic solutions for monitoring the health of processes, services, and VMs, there's a level of requisite application monitoring that needs to be deployed for each service too. Need to monitor the query plans and cache hits for PostGREs, the JMX metrics for any JVM application, compaction and read/write latency for Cassandra, etc...Setting up that level of monitoring and _actually using it_ on a day-to-day basis for a large number of different platforms is expensive and cumbersome.

.1 RocksDB replaced the Level-style compaction of leveldb with UniversalStyleCompaction that has reduced write amplification.