FS Module is a basis for building efficient, scalable remote file storage, supporting data redundancy, rebalancing and resharding. The key principle is effective data immutability, which enables simple design, low-overhead and aggressive caching. This technology allows to create high-throughput data storage and processing applications. All data in FS is organized in file systems.
You can add the FS module to your project by inserting a dependency in
This module includes FsClient interface which represents a client with upload, download, move, delete and list operations. There are several implementations of the interface:
Also, there is a RemoteFsServer which is an implementation of AbstractServer for Cloud-FS.
To run the examples in an IDE, you need to clone DataKernel:
And import it as a Maven project. Check out branch v3.0. Before running the example, build the project (Ctrl + F9 for IntelliJ IDEA).
ServerSetupExample class which is located at datakernel -> examples -> cloud-> fs, and run its main()
FileUploadExample class which is located in the same directory, and run its main() method.
FileDownloadExample class which is located in the same directory, and also run its main() method.
In the example we upload an “example.txt” file to the server and then download it back as a “download_example.txt”.
Let’s have a closer look at Server Setup Example. To make setup and launching as simple as possible, there is a
RemoteFsServerLauncher from Launchers module. It allows to simply setup an FS server:
File upload and download examples have alike implementations. Both of them extend
Launcher and thus
implement run() method which defines the main behaviour of the launcher.
Also, both of the examples utilize CSP module - uploader uses
ChannelFileReader while downloader uses
They allow to asynchronously read/write data from/to files.