Memcache Client/Server Module

Introduction

In this tutorial we will create a memcache client-server application that is based on the RPC communication protocol and predefined DataKernel modules.

You can find full example sources on GitHub.

Memcache Client and Server Modules

First of all, consider the initial DataKernel implementation of these modules, because our application will be built with their help:

Note: This implementation covers only the basic usage. You may add more features as your application requires.

Create Client and Server

Now, let’s write our own server:

public class MemcacheLikeServer extends Launcher {
	@Inject
	WorkerPool.Instances<RpcServer> instances;

	@Provides
	WorkerPool workerPool(WorkerPools workerPools) {
		return workerPools.createPool(3);
	}

	@Provides
	Config config() {
		return Config.create()
				.with("memcache.buffers", "4")
				.with("memcache.bufferCapacity", "64mb");
	}

	@Override
	protected Module getModule() {
		return Module.create()
				.install(ServiceGraphModule.create())
				.install(MemcacheMultiServerModule.create())
				.install(WorkerPoolModule.create());
	}

	@Override
	protected void run() throws Exception {
		awaitShutdown();
	}

	public static void main(String[] args) throws Exception {
		MemcacheLikeServer server = new MemcacheLikeServer();
		server.launch(args);
	}
}
  • The number of server shards equals to the number of workerPools.
  • As for the memcache functionality - we specify the number of buffers and their capacity in the config.
  • Config is used to pass everything MemcacheMultiServerModule needs to handle upcoming requests.

Our client will create put and get requests:

public class MemcacheLikeClient extends Launcher {
	@Provides
	Eventloop eventloop() {
		return Eventloop.create();
	}

	@Provides
	RawMemcacheClientAdapter rawMemcacheClientAdapter(RawMemcacheClient client) {
		return new RawMemcacheClientAdapter(client);
	}

	@Provides
	Config config() {
		return Config.create()
				.with("protocol.compression", "false")
				.with("client.addresses", "localhost:9000, localhost:9001, localhost:9002");
	}

	@Inject
	RawMemcacheClientAdapter client;

	@Inject
	Eventloop eventloop;

	@Override
	protected Module getModule() {
		return Module.create()
				.install(ServiceGraphModule.create())
				.install(MemcacheClientModule.create())
				.install(ConfigModule.create()
						.printEffectiveConfig()
						.rebindImport(new Key<CompletionStage<Void>>() {},
								new Key<CompletionStage<Void>>(OnStart.class) {}));
	}

	@Override
	protected void run() throws ExecutionException, InterruptedException {
		String message = "Hello, Memcached Server";

		CompletableFuture<Void> future = eventloop.submit(() ->
				sequence(
						() -> Promises.all(range(0, 25).mapToObj(i ->
								client.put(i, message))),
						() -> Promises.all(range(0, 25).mapToObj(i ->
								client.get(i).whenResult(res -> System.out.println(i + " : " + res))))));
		future.get();
	}

	public static void main(String[] args) throws Exception {
		Launcher client = new MemcacheLikeClient();
		client.launch(args);
	}
}
  • Since MemcacheClientModule uses Rendezvous Hashing Strategy to achieve agreement for requests between shards of servers, in client we ought to provide addresses of these shards - 9001, 9002, 9003 ports.
  • In the eventloop we ask to put a message in the current i of the bytes[i] array, and get it back from the corresponding cell.
  • So the client will perform these operations asynchronously for three shards, therefore we will receive a disordered output block as a result.

Running the application

  • First, open and run the MemcacheLikeServer to launch the server;
  • To launch the client, run main() method of MemcacheLikeClient;

You will see the following output in the server’s console (... omit identical messages):

Server on port #9000 accepted message!
Server on port #9000 accepted message!
...
Server on port #9000 accepted message!
Server on port #9002 accepted message!
Server on port #9002 accepted message!
...
Server on port #9002 accepted message!
Server on port #9001 accepted message!
...
Server on port #9001 accepted message!

In the client console you will see that the client received asynchronous responses from the server:

    0 : Hello, Memcached Server
    3 : Hello, Memcached Server
    4 : Hello, Memcached Server
    6 : Hello, Memcached Server
    8 : Hello, Memcached Server
    11 : Hello, Memcached Server
    13 : Hello, Memcached Server
    15 : Hello, Memcached Server
    18 : Hello, Memcached Server
    22 : Hello, Memcached Server
    1 : Hello, Memcached Server
    2 : Hello, Memcached Server
    7 : Hello, Memcached Server
    9 : Hello, Memcached Server
    12 : Hello, Memcached Server
    16 : Hello, Memcached Server
    20 : Hello, Memcached Server
    21 : Hello, Memcached Server
    23 : Hello, Memcached Server
    24 : Hello, Memcached Server
    5 : Hello, Memcached Server
    10 : Hello, Memcached Server
    14 : Hello, Memcached Server
    17 : Hello, Memcached Server
    19 : Hello, Memcached Server