MVCC has landed

I will soon be releasing the first alpha on JBoss Cache 3.0.0 - codenamed Naga after the Naga Jolokia pepper, naturally keeping with the pepper-naming theme. This is an alpha and doesn't nearly have all of what the GA release will have, but it does have two very significant features which I would like to share with the world. One is MVCC, and the other is a new configuration file format - which I've left for Mircea Markus to talk about in his post.

UPDATE: This has now been released. See my post on the release.

So on with MVCC. This is something people have been waiting for, for a long while now. Historic locking schemes - pessimistic locking and optimistic locking - have always been somewhat slow, inefficient, and synchronization-heavy. So we've redesigned concurrency and locking in the cache from scratch and have MVCC. Our designs for MVCC have been online for a while now. I do need to update the design page with more implementation details though, which did differ slightly from the original designs.

Feature Summary

In a nutshell, MVCC is a new locking scheme (MultiVersion Concurrency Control, something that most good database implementations have been using for a while now). our implementation is particularly interesting in that it is completely lock-free for readers. This makes it very efficient for a read-mostly system like a cache. The summary of features are:

• Writers don't block readers. At all.
• Completely lock-free for readers.
• Writers are fail-fast (so no DataVersionExceptions at commit time, like when using our old Optimistic Locking implementation)
• Deadlock-safe (thanks to non-blocking readers)
• Memory efficient since we use lock striping (for writer locks) rather than one-lock-per-node.
• Write locks implemented using the JDK's AbstractQueuedSynchronizer - which makes it free of implicit locks and synchronized blocks.
• Locks held on Fqns and not Nodes. Which solves problems with concurrent create-and-removes seen occasionally with Pessimistic Locking under high load.

So what does this mean to everyone? Basically, a much more efficient locking scheme (expect statistics and comparisons here soon), with all the best that the previous two locking schemes had to offer (fail-fast - PL, concurrent readers and writers, deadlock-free - OL).

Switching it on

In the upcoming alpha, you enable this by setting your node locking scheme to MVCC:

Configuration c = new Configuration(); c.setNodeLockingScheme(Configuration.NodeLockingScheme.MVCC);

MVCC allows two isolation levels - READ_COMMITTED and REPEATABLE_READ. All other isolation levels - if configured - will be upgraded or downgraded accordingly. There are a further two configuration elements for MVCC: concurrencyLevel and writeSkewCheck.

Concurrency levels

Concurrency level is a tuning parameter that helps determine the number of segments used when creating a collection of shared locks for all writers. This is similar to the concurrencyLevel parameter passed in to the JDK's ConcurrentHashMap constructor. Unlike CHMs, this defaults to 50 - and should be tuned accordingly. Bigger = greater concurrency but more memory usage - thanks to additional segments.

Write skews

A write skew is a condition where with REPEATABLE_READ semantics, a transaction reads a value, and then changes to to something based on what was read, but another thread had changed it between the two operations. Consider:

// start transaction

line 1: int counterValue = readSomething();
line 2: writeSomething(counterValue + 1);

// end transaction

Since MVCC offers concurrent reads and writes, thread T1 could be at line 1, another thread (T2) could be at line 2 updating the value after T1 has read it. Then when T1 comes to writing the value, this would overwrite the update that T2 had made. This is a write-skew.

Now depending on your application, you may not care about such overwrites, and hence by default we don't do anything about it and allow the overwrite to continue. However, there are still plenty of cases where write skews are bad. If the writeSkewCheck configuration parameter is set to true, an exception is thrown every time a write skew is detected. So in the above example, when T1 attempts to writeSomething(), an exception will be thrown forcing T1 to retry its transaction.

What about other locking schemes?

With our final release, we hope to make MVCC the default option. Optimistic and Pessimistic locking will still be available but deprecated, and will be removed in a future release based on the success of MVCC.

Great! Where can I get it?

I will announce the release on this blog soon. Once the release is available, please download it and try it out - feedback is very important, especially in highly concurrent environments under a lot of stress.

If you are that impatient, you're welcome to check out JBoss Cache core trunk from SVN and have a play-around. :-)

Cheers
Manik

2.2.0 "Poblano" and the evolution of JBoss Cache

So we've finally cut JBoss Cache 2.2.0.Beta1. This is meant to be a minor release on 2.1.0, with some new features (JBCACHE-1258 and JBCACHE-1320 - both involving better cleanup of buddy backup regions when buddy groups no longer exist or when data is gravitated away), and some bug fixes and minor enhancements. And, pretty severe internal refactoring.

Before I bore you with details, here's a summary. The refactoring has provided us with a much more flexible architecture, where everything is far easier to extend, build upon and unit test, and in the process we've gained on average a 15% performance increase over 2.1.0, based on your cache mode. Enough said. :-) Read on for gory details, or skip to the bottom for benchmarks download links and the like.

Now the gory details. The original architecture of JBoss Cache involved a CacheImpl class, which contained most of the core functionality, and a series of interceptors which added aspects such as replication, cache loading and locking among others. Invocations were made on CacheImpl (exposed via a Cache interface), which created an invocation object - a subclass of JGroups' MethodCall object - and passed it up the interceptor chain. The final interceptor would then invoke an internal method on CacheImpl via reflection.

This gradually simplified (in 2.1.0, Alegrias) to use a separate delegate class which implemented the Cache interface, which pushed an invocation up the interceptor chain to end up as an invocation on the CacheImpl, as well as a ComponentRegistry which managed lifecycle and dependencies between different components in the cache such as the BuddyManager, RPCManager, NodeFactory, Notifier, etc. and inject these into interceptors that declare their dependencies on such components. This made things a lot more flexible and extensible, necessary to develop the host of new features we have planned for upcoming releases.

With Poblano, we've implemented a design by Mircea Markus, which uses Command/Visitor pattern and double dispatch to achieve strongly typed callbacks into interceptors. This involves scrapping the CacheImpl altogether, and implementing each public API method as a VisitableCommand. VisitableCommands can be visited by interceptors (which now implement Visitor), each visitXXX() method on each interceptor receiving a strongly typed VisitableCommand implementation with strongly typed payloads and parameters. The final interceptor - the CallInterceptor - now simply invokes VisitableCommand.perform(). Commands also have all the knowledge they need on how to roll back, via a rollback() method, which removes the need for an undo log for each transaction. The modification list is adequate for performing rollbacks as well.

Apart from the minor gains in stronger type safety (no more casting of MethodCall arguments) and doing away with the need for reflection in the CallInterceptor, the primary benefit of this approach is much better code readability, maintainability and unit testability (how many "-ity"s can you use on once sentence?!). Interceptors and commands alike can now be isolated and unit tested, rather than end-to-end functional tests for every combination of configurations, and the overall architecture is much easier to work with when adding features.

In terms of backward compatibility, even with the new architecture this release is 100% compatible with other 2.X releases, in terms of wire protocol (a 2.2.0 cache instance can talk to a 2.1.0 one, thanks to a new Marshaller implementation that is aware of and can translate between commands and MethodCalls being replicated) as well as in terms of plugins and extensions. Custom interceptors extending the abstract (and now deprecated) Interceptor class will still work, although the preferred approach now is to extend CommandInterceptor and use the strongly typed callbacks.

And now for the big surprise - as I said before, the real goal behind this was to improve the architecture for ease of use, robustness and maintainability, and as long as we didn't regress in terms of performance, I would have been quite happy with things. After the last week benchmarking and profiling things, we've had a very pleasant surprise, with pretty interesting performance gains. I'll let the numbers speak for themselves - Mircea has published his benchmark numbers here, on this wiki page, along with extensive information on how the test was run. Along with pretty charts. We all like pretty charts. :-)

So Poblano is now available as a beta, and I would very strongly encourage everyone to download and try this out. We want to make sure this release works as well for everyone as we hope it will. Check out the release notes, userguide and javadocs and benchmarks while you download the release, and comment and feedback on the user forums!

Enjoy!
Manik