ETL Module

ETL is a near real-time async data processing system. Unlike traditional ETL (extraction-transformation-loading) systems, this module processes massive parallel FS streams of data with its distributed worker servers and then commits partially processed results into commit graph of OT. These commits are continuously merged into a single coherent result using merge and conflict resolution strategies provided by OT.

You can add ETL module to your project by inserting dependency in pom.xml:

<dependency>
    <groupId>io.datakernel</groupId>
    <artifactId>datakernel-etl</artifactId>
    <version>3.0.0-SNAPSHOT</version>
</dependency>

This module on GitHub repository