Storage Engine

##The Storage Engine idea:

• DataStore - a class and a Bacula SD resource
• SD can configure any number of DataStores
• SD->Device can use a DataStore as an archive storage target
• DataStore parameters:
  • Name - a name of the datastore resource
  • Type - a type of the DataStore, cound be any from Local, Remote
  • ArchDir - location of the storage data on filesystem
  • MaxSize - maximum size of the datastore which cannot be exceeded
  • Replication* - a datastore replication parameters - TBD
• A DataStore is required for data deduplication engine
• Standard Bacula Device resource can write to the defined DataStore
  • it will require additional SD/Device parameter: DataStore
  • what about DeviceType parameter?

DataStore archive directory format:

• /$ArchDir/meta - all metadata (db/hash, cache, etc.)
• /$ArchDir/data - all raw data files
  • /$ArchDir/data/$blocksize/XX.block - data block file
• /$ArchDir/xlog - transaction logs

DataStore API:

• Hash/Digest (SHA2) addressing:
  • read/write
  • ref/unref
  • query
  • queryref
• Volume name + block number addressing:
  • open/close
  • read/write
  • lseek
  • ref_volume
  • unref_volume
• General calls
  • commit

bacula-sd.conf:

DataStore {
   Name = DS1
   Type = Local
   Path = /backup/datastore
   MaxSize = 1TB
}

Device {
   Name = Storage0
   Device Type = ...
   Datastore = DS1
   ...
}