Memcache Client/Server Module

Introduction

In this tutorial we will create memcache client-server application that is based on RPC communication protocol and predefined DataKernel modules.

You can find full example sources on GitHub.

Memcache Client and Server Modules

First of all, consider the initial DataKernel implementation of these modules, because our application will be built with their help:

Note, this implementation covers the basic usage. You may add more features as your application requires.

Create Client and Server

Now, let’s write own server:

public class MemcacheLikeServer extends Launcher {

	@Provides
	Eventloop eventloop() { return Eventloop.create(); }

	@Provides
	Config config() {
		return Config.create()
				.with("memcache.buffers", "8")
				.with("memcache.bufferCapacity", "512")
				.with("server.listenAddresses", "localhost:8080")
				.overrideWith(Config.ofProperties(System.getProperties()).getChild("config"));
	}

	@Inject
	RpcServer memcacheServer;

	@Inject
	Eventloop eventloop;

	@Override
	protected Module getModule() {
		return combine(ServiceGraphModule.create(),
				ConfigModule.create()
						.printEffectiveConfig()
						.rebindImport(new Key<CompletionStage<Void>>() {}, new Key<CompletionStage<Void>>(OnStart.class) {}),
				MemcacheServerModule.create());
	}

	@Override
	protected void run() throws Exception {
		eventloop.run();
		memcacheServer.listen();
	}

	public static void main(String[] args) throws Exception {
		MemcacheLikeServer server = new MemcacheLikeServer();
		server.launch(args);
	}
}
  • As for the memcache functionality - we specify the number of buffers and their capacity in the config.
  • Pass with config everything that MemcacheServerModule needs to handle the upcoming requests.
  • Then make RPC memcacheServer listen on the 8080 port.

Our client will create put and get requests:

public class MemcacheLikeClient extends Launcher {
	@Provides
	Eventloop eventloop() {
		return Eventloop.create();
	}

	@Provides
	Config config() {
		return Config.create()
				.with("protocol.compression", "false")
				.with("client.addresses", "localhost:9010, localhost:9020, localhost:9030")
				.overrideWith(Config.ofProperties(System.getProperties()).getChild("config"));
	}

	@Inject
	RawMemcacheClient client;

	@Inject
	Eventloop eventloop;

	@Override
	protected Module getModule() {
		return combine(ServiceGraphModule.create(),
				ConfigModule.create()
						.printEffectiveConfig()
						.rebindImport(new Key<CompletionStage<Void>>() {}, new Key<CompletionStage<Void>>(OnStart.class) {}),
				MemcacheClientModule.create());
	}

	@Override
	protected void run() {
		eventloop.submit(() -> {
			byte[][] bytes = new byte[25][1];
			for (int i = 0; i < 25; ++i) {
				bytes[i] = new byte[]{(byte) i};
			}
			for (int i = 0; i < 25; ++i) {
				final int idx = i;
				byte[] message = "Strong and smart people always be successful".getBytes();
				Promise<Void> put = client.put(bytes[i], new Slice(message));
				put.whenComplete((res, e) -> {
					if (e != null) {
						e.printStackTrace();
						return;
					}
					System.out.println("Request sent : " + idx);
				});
			}
			for (int i = 0; i < 25; ++i) {
				final int idx = i;
				Promise<Slice> byteBufPromise = client.get(bytes[idx]);
				byteBufPromise.whenComplete((resp, e1) -> {
					if (e1 != null) {
						e1.printStackTrace();
						return;
					}
					System.out.println("Got back from [" + idx + "] : " +
							new String(resp.array(), resp.offset(), resp.length(), UTF_8));
				});
			}
		});
	}

	public static void main(String[] args) throws Exception {
		Launcher client = new MemcacheLikeClient();
		client.launch(args);
	}
}
  • Since MemcacheClientModule uses Rendezvous Hashing Strategy to achieve agreement for requests between shards of servers, in client we ought to provide addresses of these shards - 9010, 9020, 9030 ports.
  • In eventloop we ask to put a message in the current i of the bytes[i] array, and get it back from the corresponding cell.
  • So client will perform these operations asynchronously for three shards, that’s why as the result we will receive three disordered output blocks.

Running the application

  • Launch server - run MemcacheLikeServer main() method.
  • Create and run compound configuration. For IntelliJ IDEA:
    • Add three configurations for each address Run -> Edit configurations -> Add Application -> Run/Debug Configurations -> Main class: MemcacheLikeServer -> VM options: -ea -Dconfig.server.listenAddresses=localhost:9010. Changing port number correspondingly.
    • Create compound configuration Run -> Edit configurations -> Add Compound and add three configurations from previous step.
    • Run compound configuration.
  • Launch client - run MemcacheLikeClient main() method.