Skip to content

Commit

Permalink
Update crate documentation, more consistent formatting
Browse files Browse the repository at this point in the history
Signed-off-by: Victor Porof <[email protected]>
  • Loading branch information
victorporof committed Jul 21, 2020
1 parent d33e79f commit 9fe69b1
Show file tree
Hide file tree
Showing 38 changed files with 658 additions and 590 deletions.
7 changes: 6 additions & 1 deletion .rustfmt.toml
Original file line number Diff line number Diff line change
@@ -1,4 +1,9 @@
comment_width = 90
force_multiline_blocks = true
imports_layout = "Vertical"
max_width = 120
match_block_trailing_comma = true
max_width = 120
merge_imports = true
reorder_impl_items = true
use_small_heuristics = "Off"
wrap_comments = true
31 changes: 18 additions & 13 deletions Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,23 +1,28 @@
[package]
name = "rkv"
version = "0.11.1"
authors = ["Richard Newman <[email protected]>", "Nan Jiang <[email protected]>", "Myk Melez <[email protected]>", "Victor Porof <[email protected]>"]
edition = "2018"
license = "Apache-2.0"
description = "a simple, humane, typed Rust interface to LMDB"
authors = [
"Richard Newman <[email protected]>",
"Nan Jiang <[email protected]>",
"Myk Melez <[email protected]>",
"Victor Porof <[email protected]>",
]
categories = ["database"]
description = "A simple, humane, typed key-value storage solution"
documentation = "https://docs.rs/rkv"
edition = "2018"
exclude = ["/tests/envs/*"]
homepage = "https://github.com/mozilla/rkv"
repository = "https://github.com/mozilla/rkv"
readme = "README.md"
keywords = ["lmdb", "database", "storage"]
categories = ["database"]
exclude = ["/tests/envs/*"]
license = "Apache-2.0"
name = "rkv"
readme = "README.md"
repository = "https://github.com/mozilla/rkv"
version = "0.11.1"

[features]
default = ["db-dup-sort", "db-int-key"]
backtrace = ["failure/backtrace", "failure/std"]
db-dup-sort = []
db-int-key = []
default = ["db-dup-sort", "db-int-key"]
with-asan = ["lmdb-rkv/with-asan"]
with-fuzzer = ["lmdb-rkv/with-fuzzer"]
with-fuzzer-no-link = ["lmdb-rkv/with-fuzzer-no-link"]
Expand All @@ -32,17 +37,17 @@ lazy_static = "1.0"
lmdb-rkv = "0.14"
log = "0.4"
ordered-float = "1.0"
serde = { version = "1.0", features = ["derive", "rc"] }
serde = {version = "1.0", features = ["derive", "rc"]}
serde_derive = "1.0"
url = "2.0"
uuid = "0.8"

# Get rid of failure's dependency on backtrace. Eventually
# backtrace will move into Rust core, but we don't need it here.
[dependencies.failure]
version = "0.1"
default_features = false
features = ["derive"]
version = "0.1"

[dev-dependencies]
byteorder = "1"
Expand Down
9 changes: 3 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,6 @@ The [rkv Rust crate](https://crates.io/crates/rkv) is a simple, humane, typed ke

## ⚠️ Warning ⚠️

The LMDB backend is currently unstable and crash-prone. We're attempting to fix these crashes in bugs [1538539](https://bugzilla.mozilla.org/show_bug.cgi?id=1538539), [1538541](https://bugzilla.mozilla.org/show_bug.cgi?id=1538541) and [1550174](https://bugzilla.mozilla.org/show_bug.cgi?id=1550174).

To use rkv in production/release environments at Mozilla, you may do so with the "SafeMode" backend, for example:

```rust
Expand All @@ -23,9 +21,9 @@ let shared_rkv = manager.get_or_create(path, Rkv::new::<SafeMode>).unwrap();
...
```

The "SafeMode` backend performs well, with two caveats: the entire database is stored in memory, and write transactions are synchronously written to disk on commit.
The "SafeMode" backend performs well, with two caveats: the entire database is stored in memory, and write transactions are synchronously written to disk (only on commit).

In the future, it will be advisable to switch to a different backend with better performance guarantees. We're working on either fixing the LMDB crashes, or offering more choices of backend engines (e.g. SQLite).
In the future, it will be advisable to switch to a different backend with better performance guarantees. We're working on either fixing some LMDB crashes, or offering more choices of backend engines (e.g. SQLite).

## Use

Expand All @@ -49,8 +47,7 @@ There are several features that you can opt-in and out of when using rkv:

By default, `db-dup-sort` and `db-int-key` features offer high level database APIs which allow multiple values per key, and optimizations around integer-based keys respectively. Opt out of these default features when specifying the rkv dependency in your Cargo.toml file to disable them; doing so avoids a certain amount of overhead required to support them.

If you specify the `backtrace` feature, backtraces will be enabled in "failure"
errors. This feature is disabled by default.
If you specify the `backtrace` feature, backtraces will be enabled in "failure" errors. This feature is disabled by default.

To aid fuzzing efforts, `with-asan`, `with-fuzzer`, and `with-fuzzer-no-link` configure the build scripts responsible with compiling the underlying backing engines (e.g. LMDB) to build with these LLMV features enabled. Please refer to the official LLVM/Clang documentation on them for more informatiuon. These features are also disabled by default.

Expand Down
16 changes: 9 additions & 7 deletions examples/iterator.rs
Original file line number Diff line number Diff line change
Expand Up @@ -7,17 +7,19 @@
//!
//! cargo run --example iterator
use std::fs;
use std::str;
use std::{
fs,
str,
};

use tempfile::Builder;

use rkv::backend::{
Lmdb,
LmdbDatabase,
LmdbEnvironment,
};
use rkv::{
backend::{
Lmdb,
LmdbDatabase,
LmdbEnvironment,
},
Manager,
Rkv,
SingleStore,
Expand Down
14 changes: 7 additions & 7 deletions examples/simple-store.rs
Original file line number Diff line number Diff line change
Expand Up @@ -11,14 +11,14 @@ use std::fs;

use tempfile::Builder;

use rkv::backend::{
BackendStat,
Lmdb,
LmdbDatabase,
LmdbEnvironment,
LmdbRwTransaction,
};
use rkv::{
backend::{
BackendStat,
Lmdb,
LmdbDatabase,
LmdbEnvironment,
LmdbRwTransaction,
},
Manager,
Rkv,
StoreOptions,
Expand Down
44 changes: 16 additions & 28 deletions src/backend.rs
Original file line number Diff line number Diff line change
Expand Up @@ -16,48 +16,36 @@ mod traits;
pub use common::*;
pub use traits::*;

pub use impl_lmdb::DatabaseImpl as LmdbDatabase;
pub use impl_lmdb::EnvironmentBuilderImpl as Lmdb;
pub use impl_lmdb::EnvironmentImpl as LmdbEnvironment;
pub use impl_lmdb::ErrorImpl as LmdbError;
pub use impl_lmdb::IterImpl as LmdbIter;
pub use impl_lmdb::{
DatabaseFlagsImpl as LmdbDatabaseFlags,
DatabaseImpl as LmdbDatabase,
EnvironmentBuilderImpl as Lmdb,
EnvironmentFlagsImpl as LmdbEnvironmentFlags,
WriteFlagsImpl as LmdbWriteFlags,
};
pub use impl_lmdb::{
EnvironmentImpl as LmdbEnvironment,
ErrorImpl as LmdbError,
InfoImpl as LmdbInfo,
StatImpl as LmdbStat,
};
pub use impl_lmdb::{
IterImpl as LmdbIter,
RoCursorImpl as LmdbRoCursor,
RwCursorImpl as LmdbRwCursor,
};
pub use impl_lmdb::{
RoTransactionImpl as LmdbRoTransaction,
RwCursorImpl as LmdbRwCursor,
RwTransactionImpl as LmdbRwTransaction,
StatImpl as LmdbStat,
WriteFlagsImpl as LmdbWriteFlags,
};

pub use impl_safe::DatabaseImpl as SafeModeDatabase;
pub use impl_safe::EnvironmentBuilderImpl as SafeMode;
pub use impl_safe::EnvironmentImpl as SafeModeEnvironment;
pub use impl_safe::ErrorImpl as SafeModeError;
pub use impl_safe::IterImpl as SafeModeIter;
pub use impl_safe::{
DatabaseFlagsImpl as SafeModeDatabaseFlags,
DatabaseImpl as SafeModeDatabase,
EnvironmentBuilderImpl as SafeMode,
EnvironmentFlagsImpl as SafeModeEnvironmentFlags,
WriteFlagsImpl as SafeModeWriteFlags,
};
pub use impl_safe::{
EnvironmentImpl as SafeModeEnvironment,
ErrorImpl as SafeModeError,
InfoImpl as SafeModeInfo,
StatImpl as SafeModeStat,
};
pub use impl_safe::{
IterImpl as SafeModeIter,
RoCursorImpl as SafeModeRoCursor,
RwCursorImpl as SafeModeRwCursor,
};
pub use impl_safe::{
RoTransactionImpl as SafeModeRoTransaction,
RwCursorImpl as SafeModeRwCursor,
RwTransactionImpl as SafeModeRwTransaction,
StatImpl as SafeModeStat,
WriteFlagsImpl as SafeModeWriteFlags,
};
6 changes: 3 additions & 3 deletions src/backend/impl_lmdb/environment.rs
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,8 @@ use crate::backend::traits::{
pub struct EnvironmentBuilderImpl(lmdb::EnvironmentBuilder);

impl<'b> BackendEnvironmentBuilder<'b> for EnvironmentBuilderImpl {
type Error = ErrorImpl;
type Environment = EnvironmentImpl;
type Error = ErrorImpl;
type Flags = EnvironmentFlagsImpl;

fn new() -> EnvironmentBuilderImpl {
Expand Down Expand Up @@ -69,13 +69,13 @@ impl<'b> BackendEnvironmentBuilder<'b> for EnvironmentBuilderImpl {
pub struct EnvironmentImpl(lmdb::Environment);

impl<'e> BackendEnvironment<'e> for EnvironmentImpl {
type Error = ErrorImpl;
type Database = DatabaseImpl;
type Error = ErrorImpl;
type Flags = DatabaseFlagsImpl;
type Stat = StatImpl;
type Info = InfoImpl;
type RoTransaction = RoTransactionImpl<'e>;
type RwTransaction = RwTransactionImpl<'e>;
type Stat = StatImpl;

fn open_db(&self, name: Option<&str>) -> Result<Self::Database, Self::Error> {
self.0.open_db(name).map(DatabaseImpl).map_err(ErrorImpl)
Expand Down
6 changes: 4 additions & 2 deletions src/backend/impl_lmdb/error.rs
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,10 @@

use std::fmt;

use crate::backend::traits::BackendError;
use crate::error::StoreError;
use crate::{
backend::traits::BackendError,
error::StoreError,
};

#[derive(Debug)]
pub struct ErrorImpl(pub(crate) lmdb::Error);
Expand Down
22 changes: 12 additions & 10 deletions src/backend/impl_lmdb/flags.rs
Original file line number Diff line number Diff line change
Expand Up @@ -8,16 +8,18 @@
// CONDITIONS OF ANY KIND, either express or implied. See the License for the
// specific language governing permissions and limitations under the License.

use crate::backend::common::{
DatabaseFlags,
EnvironmentFlags,
WriteFlags,
};
use crate::backend::traits::{
BackendDatabaseFlags,
BackendEnvironmentFlags,
BackendFlags,
BackendWriteFlags,
use crate::backend::{
common::{
DatabaseFlags,
EnvironmentFlags,
WriteFlags,
},
traits::{
BackendDatabaseFlags,
BackendEnvironmentFlags,
BackendFlags,
BackendWriteFlags,
},
};

#[derive(Debug, Eq, PartialEq, Copy, Clone, Default)]
Expand Down
4 changes: 2 additions & 2 deletions src/backend/impl_lmdb/transaction.rs
Original file line number Diff line number Diff line change
Expand Up @@ -27,8 +27,8 @@ use crate::backend::traits::{
pub struct RoTransactionImpl<'t>(pub(crate) lmdb::RoTransaction<'t>);

impl<'t> BackendRoTransaction for RoTransactionImpl<'t> {
type Error = ErrorImpl;
type Database = DatabaseImpl;
type Error = ErrorImpl;

fn get(&self, db: &Self::Database, key: &[u8]) -> Result<&[u8], Self::Error> {
self.0.get(db.0, &key).map_err(ErrorImpl)
Expand All @@ -51,8 +51,8 @@ impl<'t> BackendRoCursorTransaction<'t> for RoTransactionImpl<'t> {
pub struct RwTransactionImpl<'t>(pub(crate) lmdb::RwTransaction<'t>);

impl<'t> BackendRwTransaction for RwTransactionImpl<'t> {
type Error = ErrorImpl;
type Database = DatabaseImpl;
type Error = ErrorImpl;
type Flags = WriteFlagsImpl;

fn get(&self, db: &Self::Database, key: &[u8]) -> Result<&[u8], Self::Error> {
Expand Down
6 changes: 4 additions & 2 deletions src/backend/impl_safe/database.rs
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,10 @@ use serde_derive::{
Serialize,
};

use super::snapshot::Snapshot;
use super::DatabaseFlagsImpl;
use super::{
snapshot::Snapshot,
DatabaseFlagsImpl,
};
use crate::backend::traits::BackendDatabase;

#[derive(Debug, Eq, PartialEq, Copy, Clone, Hash)]
Expand Down
32 changes: 17 additions & 15 deletions src/backend/impl_safe/environment.rs
Original file line number Diff line number Diff line change
Expand Up @@ -8,18 +8,20 @@
// CONDITIONS OF ANY KIND, either express or implied. See the License for the
// specific language governing permissions and limitations under the License.

use std::borrow::Cow;
use std::collections::HashMap;
use std::fs;
use std::path::{
Path,
PathBuf,
};
use std::sync::Arc;
use std::sync::{
RwLock,
RwLockReadGuard,
RwLockWriteGuard,
use std::{
borrow::Cow,
collections::HashMap,
fs,
path::{
Path,
PathBuf,
},
sync::{
Arc,
RwLock,
RwLockReadGuard,
RwLockWriteGuard,
},
};

use id_arena::Arena;
Expand Down Expand Up @@ -55,8 +57,8 @@ pub struct EnvironmentBuilderImpl {
}

impl<'b> BackendEnvironmentBuilder<'b> for EnvironmentBuilderImpl {
type Error = ErrorImpl;
type Environment = EnvironmentImpl;
type Error = ErrorImpl;
type Flags = EnvironmentFlagsImpl;

fn new() -> EnvironmentBuilderImpl {
Expand Down Expand Up @@ -188,13 +190,13 @@ impl EnvironmentImpl {
}

impl<'e> BackendEnvironment<'e> for EnvironmentImpl {
type Error = ErrorImpl;
type Database = DatabaseImpl;
type Error = ErrorImpl;
type Flags = DatabaseFlagsImpl;
type Stat = StatImpl;
type Info = InfoImpl;
type RoTransaction = RoTransactionImpl<'e>;
type RwTransaction = RwTransactionImpl<'e>;
type Stat = StatImpl;

fn open_db(&self, name: Option<&str>) -> Result<Self::Database, Self::Error> {
if Arc::strong_count(&self.ro_txns) > 1 {
Expand Down
12 changes: 8 additions & 4 deletions src/backend/impl_safe/error.rs
Original file line number Diff line number Diff line change
Expand Up @@ -8,13 +8,17 @@
// CONDITIONS OF ANY KIND, either express or implied. See the License for the
// specific language governing permissions and limitations under the License.

use std::fmt;
use std::io;
use std::{
fmt,
io,
};

use bincode::Error as BincodeError;

use crate::backend::traits::BackendError;
use crate::error::StoreError;
use crate::{
backend::traits::BackendError,
error::StoreError,
};

#[derive(Debug)]
pub enum ErrorImpl {
Expand Down
Loading

0 comments on commit 9fe69b1

Please sign in to comment.