Cloud components of DataKernel Framework


DataKernel Cloud components provide tools to create solutions of different scales: from small private file storage to high-loaded OLAP Cube systems. It also includes Operational Transformations, CRDT and RPC implementations.

Components Description
RPC High-performance and fault-tolerant remote procedure call module for building distributed applications with an extremely efficient asynchronous binary RPC streaming protocol.
FS Basis for building scalable remote file storage with implementation of caching and fast asynchronous file I/O based on Java NIO. Utilizes CSP for fast and reliable file transfer.
OT This module allows to build collaborative software systems based on Git-like approach combined with automatic conflict resolution, utilizing a special algorithm for operational transformations.
LSM Tree Aggregation Log-structured merge-tree table which stores aggregate functions and designed for OLAP workload.
LSM Tree OLAP Cube Multidimensional OLAP (Online Analytical Processing) database with a predefined set of dimensions, measures, and log-structured merge-tree tables containing pre-aggregated data. LSM Tree database efficiently executes multi-dimensional analytical queries.
Dataflow Distributed stream-based batch processing engine for Big Data applications. Contains tools to work with data sets which can span multiple partitions.
CRDT Conflict-free replicated data type implementation (specifically, state-based CRDT). Contains tools to create collaborative editing applications using CRDT approach to merge data that comes from multiple nodes into a single CRDT structure.
Multilog This module manages the integrity of log files stored in a distributed file system and allows to work with them as if they were stored in a single place.
ETL Processes logs using operational transformations. Uses the OT module to persist logs and resolve conflicts.