This is a simple implementation of a Distributed File System using Gifford Quorum Voting as a way to achieve replica consensus written using Thrift RPC and Java. This is mostly a P2P system so each File Server(Node) involved has the same functionality. This system supports multiple clients and for some simplicity, one of the nodes additionally serves as a centralized Coordinator to build Quorums for each operation and for sequentially ordering concurrent Writes to the same file. However, writes to different files as well as all reads and list operations happen concurrently.
The Client is able to issue commands(read file, write file and list files) from a CLI and additionally, use a "simulate read/write" feature wherein it fires said type of requests with random delays and payloads. This is useful for load testing our system.
For more information, look at the problem description.
The code is well documented with very verbose logging and solid exception handling.
This Java Properties File stores many of the values needed to dynamically configure the system, including:
- addresses and port numbers of each of the nodes.
- filesystem paths.
- node counts, quorum size for read/write.
- coordinator id.
- background synchronization delay.
There is a makefile provided that can be used to easily build and run the project. YOU NEED JAVA 1.8+ AND THRIFT 0.9+ TO RUN THIS SYSTEM
- Check to make sure the address, ports and other options are as desired in the config file.
- Run “make clean && make” to rebuild the source.
- Run “make cli” to run the client.
- Run “make n NUM=” to run the file server nodes.
- starts at 0 and should be set based on what address and port number that node runs on.
- it's used to index into the comma separated list of addresses and port numbers provided in the config file.