关于Dalli

21 October 2014

Dalli is a high performance pure Ruby client for accessing memcached servers. It works with memcached 1.4+ only as it uses the newer binary protocol. It should be considered a replacement for the memcache-client gem.

项目的名字取义自Salvador Dali的名画《永恒的记忆(The Persistence of Memory)》。最初的开发是受到了CouchBase的资助。

Design

I decided to write Dalli after maintaining memcache-client for two years for a few specific reasons:

在维护了memcache-client两年多后，由于一些特殊的原因，决定创作Dalli:

  1. memcache-client的代码老旧且质量差，代码的主体是单个1000行的rb文件
  2. 许多不常使用的选项使得代码库复杂化
  3. 没有提供单独的附加监控的钩子(hooks)
  4. 使用旧的文本协议，降低了运行性能

对应的，Dalli的特点:

  1. Dalli使用一个精确且简单的算法选择服务器，在TB级别的memcached集群中，和memcache-client效果相当
  2. 在Ruby 1.9.2中，Dalli比极度优化的memcache-client快了20%
  3. 包含了显式的”chokepoint(阻塞点)”方法，其可以处理所有的请求。并且可通过监视工具(NewRelic, Rack::Bug等)添加钩子方法，从而追踪内存的使用
  4. supports SASL for use in managed environments, e.g. Heroku.
  5. 提供带有恢复和适应的时耗的故障转移

Supported Ruby versions and implementations

Dalli should work identically on:

  • JRuby 1.6+
  • Ruby 1.9.3+
  • Rubinius 2.0

If you have problems, please enter an issue.

Installation and Usage

Remember, Dalli requires memcached 1.4+. You can check the version with memcached -h. Please note that the memcached version that Mac OS X Snow Leopard ships with is 1.2.8 and it won’t work. Install memcached 1.4.x using Homebrew with

注意，Dalli需要使用memcached 1.4以上的版本，可通过memcached -h查看memcahed的版本。具体的memcached的安装方法，比如在Mac OSX上，使用brew安装:

brew install memcached

在ubuntu上，使用apt安装如下:

apt-get install memcached

可以使用如下的代码验证memcached的安装:

gem install dalli

require 'dalli'  
options = { :namespace => "app_v1", :compress => true }  
dc = Dalli::Client.new('localhost:11211', options)  
dc.set('abc', 123)  
value = dc.get('abc')

The test suite requires memcached 1.4.3+ with SASL(Simple Authentication and Security Layer) enabled (brew install memcached –enable-sasl ; mv /usr/bin/memcached /usr/bin/memcached.old). Currently only supports the PLAIN mechanism.

测试用例需要1.4.3以上的启动了SASL的memcached，当前执行PLAIN机制。

Dalli has no runtime dependencies and never will. You can optionally install the ‘kgio’ gem to give Dalli a 20-30% performance boost.

Dalli没有运行时依赖，但可以通过安装’kgio’gem包来获得20-30%的性能提速。

Usage with Rails 3.x and 4.x

在Gemfile文件中:

gem 'dalli'

在config/environments/production.rb文件中:

config.cache_store = :dalli_store

Here’s a more comprehensive example that sets a reasonable default for maximum cache entry lifetime (one day), enables compression for large values and namespaces all entries for this rails app. Remove the namespace if you have multiple apps which share cached values.

config.cache_store = :dalli_store, 'cache-1.example.com', 'cache-2.example.com',
  { :namespace => NAME_OF_RAILS_APP, :expires_in => 1.day, :compress => true }

If your servers are specified in ENV["MEMCACHE_SERVERS"] (e.g. on Heroku when using a third-party hosted addon), simply provide nil for the servers:

config.cache_store = :dalli_store, nil, { :namespace => NAME_OF_RAILS_APP, :expires_in => 1.day, :compress => true }

To use Dalli for Rails session storage that times out after 20 minutes, in config/initializers/session_store.rb:

For Rails >= 3.2.4:

Rails.application.config.session_store ActionDispatch::Session::CacheStore, :expire_after => 20.minutes

For Rails 3.x:

require 'action_dispatch/middleware/session/dalli_store'
Rails.application.config.session_store :dalli_store, :memcache_server => ['host1', 'host2'], :namespace => 'sessions', :key => '_foundation_session', :expire_after => 20.minutes

Dalli does not support Rails 2.x.

Multithreading and Rails

If you use Puma or another threaded app server, as of Dalli 2.7, you can use a pool of Dalli clients with Rails to ensure the Rails.cache singleton does not become a source of thread contention. You must add gem 'connection_pool' to your Gemfile and add :pool_size to your dalli_store config:

config.cache_store = :dalli_store, 'cache-1.example.com', { :pool_size => 5 }

You can then use the Rails cache as normal or check out a Dalli client directly from the pool:

Rails.cache.fetch('foo', :expires_in => 300) do
  'bar'
end

Rails.cache.dalli.with do |client|
  # client is a Dalli::Client instance which you can
  # use ONLY within this block
end

Configuration

Dalli::Client accepts the following options. All times are in seconds.

expires_in: Global default for key TTL. Default is 0, which means no expiry.

namespace: If specified, prepends each key with this value to provide simple namespacing. Default is nil.

failover: Boolean, if true Dalli will failover to another server if the main server for a key is down. Default is true.

threadsafe: Boolean. If true Dalli ensures that only one thread is using a socket at a given time. Default is true. Set to false at your own peril.

serializer: The serializer to use for objects being stored (ex. JSON). Default is Marshal.

compress: Boolean, if true Dalli will gzip-compress values larger than 1K. Default is false.

compression_min_size: Minimum value byte size for which to attempt compression. Default is 1K.

compression_max_size: Maximum value byte size for which to attempt compression. Default is unlimited.

compressor: The compressor to use for objects being stored. Default is zlib, implemented under Dalli::Compressor. If serving compressed data using nginx’s HttpMemcachedModule, set memcached_gzip_flag 2 and use Dalli::GzipCompressor

keepalive: Boolean. If true, Dalli will enable keep-alive for socket connections. Default is true.

socket_timeout: Timeout for all socket operations (connect, read, write). Default is 0.5.

socket_max_failures: When a socket operation fails after socket_timeout, the same operation is retried. This is to not immediately mark a server down when there’s a very slight network problem. Default is 2.

socket_failure_delay: Before retrying a socket operation, the process sleeps for this amount of time. Default is 0.01. Set to nil for no delay.

down_retry_delay: When a server has been marked down due to many failures, the server will be checked again for being alive only after this amount of time. Don’t set this value too low, otherwise each request which tries the failed server might hang for the maximum socket_timeout. Default is 1 second.

value_max_bytes: The maximum size of a value in memcached. Defaults to 1MB, this can be increased with memcached’s -I parameter. You must also configure Dalli to allow the larger size here.

username: The username to use for authenticating this client instance against a SASL-enabled memcached server. Heroku users should not need to use this normally.

password: The password to use for authenticating this client instance against a SASL-enabled memcached server. Heroku users should not need to use this normally.

Features and Changes

By default, Dalli is thread-safe. Disable thread-safety at your own peril.

Dalli does not need anything special in Unicorn/Passenger since 2.0.4. It will detect sockets shared with child processes and gracefully reopen the socket.

Note that Dalli does not require ActiveSupport or Rails. You can safely use it in your own Ruby projects.

View the Client API



傲娇的使用Disqus