Serializer Module

Serializer module is a bytecode generator of extremely fast and space efficient serializers, which are used for transferring data over wire or persisting it into a file system or database.

You can add Serializer module to your project by inserting dependency in pom.xml:

<dependency>
    <groupId>io.datakernel</groupId>
    <artifactId>datakernel-serializer</artifactId>
    <version>3.0.0-SNAPSHOT</version>
</dependency>
  • Schema-less approach - for maximum performance and compactness (unlike other serializers, there is no overhead in typed values)
  • Implemented using runtime bytecode generation, to be compatible with dynamically created classes (like intermediate POJOs created with Codegen module)

A common usage for a serializer is to pass some serialized class instances through the network to remote machines for further processing. This approach is used in RPC, Dataflow and LSMT OLAP Cube modules.


Examples

  1. Simple Object Serialization Example - serialization and deserialization of a simple object.
  2. Complex Object Serialization Example - serialization and deserialization of a more complex object which contains nullable fields, map, list and a two-dimensional array.
  3. Fixed Size Fields Serialization Example - example of serialization and deserialization of an object with fixed size fields.
  4. Generics And Interfaces Serialization Example - example of using generics and interfaces with serializers and deserializers.
Note: To run the examples, you need to clone DataKernel from GitHub:
$ git clone https://github.com/softindex/datakernel
And import it as a Maven project. Before running the examples, build the project.

All of the examples consistently serialize and deserialize objects of different complexity.

If you run the Simple Object Serialization Example, you’ll get the following output:

10 10
abc abc
20 20
30 30
40 40
123 123

The first column represents values of testData1, the second - of testData2. testData2 was created with the help of serialization and deserialization of the testData1.

In order to create classes whose instances can be serialized/deserialized, special annotations should be used:

  • @Serialize annotation with order number on property getter. Ordering provides better compatibility in case classes are changed.
  • @Deserialize annotation with property name (which should be same as in getter) in constructor.
  • @SerializeNullable on properties that can have null values.

For example,

public static class TestDataSimple {
	public TestDataSimple(@Deserialize("finalInt") int finalInt,
			@Deserialize("finalString") String finalString) {
		this.finalInt = finalInt;
		this.finalString = finalString;
	}

	@Serialize(order = 0)
	public final int finalInt;

	@Serialize(order = 1)
	public final String finalString;

	private int i;
	private Integer iBoxed;

	private int getterInt;
	private String getterString;

	@Serialize(order = 2)
	public int getI() {
		return i;
	}


	...

Next, we need to create our BinarySerializer with the help of BinaryBuilder:

BinarySerializer<T> serializer = SerializerBuilder
		.create(getContextClassLoader())
		.build(typeToken);

Generated bytecode of BinarySerializer will be saved in your working directory.


Complex Object Serialization Example is an example of serialization and deserialization of a more complex object, which contains nullable fields, map, list and a two-dimensional array. If you run the example, you’ll receive the following output:

null null
abc abc
[a, null, b] [a, null, b]
2 2
[a, null] [a, null]
null null
{1=abc, 2=null, null=xyz} {null=xyz, 1=abc, 2=null}

You can explore the example here.


Generics and Interfaces Serialization Example represents how to use generics and interfaces with serializers and deserializers. If you run the example, you’ll receive the following output:

2 2 
10 a, 10 a
20 b, 20 b

You can explore the example here.


If you run Fixed Size Fields Serialization Example, you’ll get the following output:

[abc, null, 123, superfluous] [abc, null, 123]
[1, 2, 3, 4] [1, 2, 3, 4]

As you can see in the first line, dataObject2 differs from dataObject1. This is because @SerializeFixedSize annotation was set at value 3 for the String array. Thus, “superfluous” was removed from the array while serialization:

public static class TestDataFixedSize {
	@Serialize(order = 0)
	@SerializeFixedSize(3)
	@SerializeNullable(path = {0})
	public String[] strings;

	@Serialize(order = 1)
	@SerializeFixedSize(4)
	public byte[] bytes;
}