Serializer Module

Serializer module is a bytecode generator of extremely fast and space efficient serializers, which are used for transferring data over wire or persisting it into a file system or database.

You can add Serializer module to your project by inserting dependency in pom.xml:

<dependency>
    <groupId>io.datakernel</groupId>
    <artifactId>datakernel-serializer</artifactId>
    <version>3.0.0-SNAPSHOT</version>
</dependency>
  • Schema-less approach - for maximum performance and compactness (unlike other serializers, there is no overhead in typed values)
  • Implemented using runtime bytecode generation, to be compatible with dynamically created classes (like intermediate POJOs created with Codegen module)

A common usage for a serializer is to pass some serialized class instances through the network to remote machines for further processing. This approach is used in RPC, Dataflow and LSMT OLAP Cube modules. Serialization process can be configured via annotations.


You can explore Serializer examples here

This module on GitHub repository


Benchmark

We have conducted a benchmark of our serializers using the methodology and data structures described here.

Specifically, the test called SIMPLE/SPECIFIC has been run (described here).

Results are presented in the table below.

Table column names meaning:

  • create - time required to set up objects (specifically, to copy the objects)
  • ser - serialization time
  • deser - deserialization time
  • total - serialization + deserialization time
  • size - the size of the serialized data
  • +dfl - the size of the serialized data compressed with Java's built-in implementation of DEFLATE (zlib)

Time is in nanoseconds, size is in bytes.

create ser deser total size +dfl
datakernel serializer 272 1335 1590 2924 216 131
protostuff 305 1798 2624 4422 239 150
kryo-manual 219 2063 2723 4786 211 131
protobuf/protostuff 311 2098 2979 5077 239 149
protostuff-manual 260 2007 3438 5445 239 150
wobly 135 3382 2261 5643 251 151
protobuf/protostuff-runtime 216 2631 3134 5765 241 150
protostuff-graph 316 2811 3075 5885 239 150
kryo-opt 212 2725 3165 5890 209 129
kryo-flat-pre 215 2703 3209 5912 212 132