Distributed Channel Cache: Initial POC

There are two main goals for the initial POC:

• Performance profiling for SG nodes using memcached bucket
• Validate that the existing in-memory cache can be swapped out

High-level design - Initial POC

Note: The following describes an initial implemention, intended to evaluate the above criteria. See next steps for more discussion on the full set of functionality, including multiple cache writers, advanced sequence processing, and improved notification.

The core change is to maintain the change cache in a memcached bucket, instead of in memory. Cache entries will define a key based on channel name and sequence number to allow fast, key-based retrieval of sequences for a given channel. All sync gateway nodes will do their cache reads from this bucket.

Initially, assign a single node in the SG cluster as the cache writer. Cache writer works the TAP feed as today, and writes to the channel cache bucket for each sequence/channel combination. The cache writer does the same sequence buffering that's done today, to ensure that contents of the channel cache are complete . The cache writer periodically writes channel clock information to the channel cache bucket (count of writes or high sequence per channel), for use in notification.

Simplifications made for Initial POC

• Single cache writer, defined based on server config
  • no election of cache writer
  • no failover/recovery for cache writer
• Cache writer always starts at zero with empty cache bucket. No handling for stop/start of cache writer, appending to existing bucket
• No cache expiry/pruning
• Cache readers poll the cache for change notification - no push notification

Cache Writer

One SG node will be identified as the cache writer, based on entries in the Sync Gateway config (similar to the way shadowing is defined today). The config will define the memcached bucket connection information for the cache writer.

The cache writer will buffer the tap feed in the same way the current change cache does.

Open issues/discussions

One doc or multiple docs per channel in channel cache bucket

There have been discussions debating whether the representation of the channel cache in the memcached bucket should be a single document per channel (containing the cache contents), or multiple documents per channel.

At this point the benefits of the multiple-document approach outweigh the benefits of the single-doc approach.

Benefits of single-doc-per-channel cache

• Simpler notification to readers (watch for updates to those docs)

Benefits of multiple-doc-per-channel cache

• Cache readers don't need to parse a single document to evaluate a 'get changes since sequence' operation.
• No contention on channel cache documents for high traffic channels, multiple cache writers
• No limit on cache size due to document size limit (1 MB for memcached bucket)

Handling late-arriving sequences (recently fixed in issue #525)

A distributed channel cache will need to handle late-arriving sequences in the same way the current cache does. The in-memory approach for late sequence handling may not be a good fit for a distributed cache, however. Currently each channel cache maintains a set of late-arriving sequences, for use by continuous _changes feeds. In a nutshell this means that each channel cache maintains two caches - the regular cache and the late-arriving cache. My feeling is that this functionality doesn't need to be part of the initial POC (as it adds a decent amount of scope), and should be addressed in a subsequent phase, once we've finalized how we're going to handle multiple cache writers.

Implementation Notes

Change Cache API used for feed processing

• DocChanged

Change Cache API used during _changes processing

• GetChangesInChannel(channel, options)
• getOldestSkippedSequence()

Change Cache API used for other processing

• GetChangeLog(channelname, since) - used by handleDumpChannel

Channel Cache API used during feed processing

• addToCache

Channel Cache API used during _changes processing

• InitLateSequenceClient
• GetLateSequencesSince
• ReleaseLateSequenceClient
• GetCachedChanges(options)
• GetChanges(options)

Channel Cache API used during other processing

• PruneCache