hak5
/
bolt
mirror of https://github.com/hak5/bolt.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
An embedded key/value database for Go.
75 Commits
1 Branch
9 Tags
1.5 MiB
Go 99.9%
Makefile 0.1%
 
 
Go to file
Ben Johnson 7bb878ff69 Mmap remap. 2014-02-12 11:49:57 -07:00
LICENSE Initial commit 2013-12-20 10:26:14 -08:00
Makefile Add freelist. 2014-02-10 14:04:01 -07:00
NOTES Add RWTransaction.Put(). 2014-02-01 12:30:37 -05:00
README.md Cursor iteration. 2014-02-11 09:17:48 -07:00
TODO Add RWTransaction.Put(). 2014-02-01 12:30:37 -05:00
assert.go Add freelist. 2014-02-10 14:04:01 -07:00
bucket.go Add freelist. 2014-02-10 14:04:01 -07:00
buckets.go Clean up. 2014-02-06 16:21:36 -07:00
buckets_test.go Rename sys ☞ buckets. 2014-02-05 22:15:47 -07:00
const.go Clean up API. 2014-01-31 12:22:58 -05:00
cursor.go Cursor iteration. 2014-02-11 09:17:48 -07:00
db.go Mmap remap. 2014-02-12 11:49:57 -07:00
db_test.go Mmap remap. 2014-02-12 11:49:57 -07:00
doc.go Clean up API. 2014-01-31 12:22:58 -05:00
error.go Clean up API. 2014-01-31 12:22:58 -05:00
error_test.go Add freelist. 2014-02-10 14:04:01 -07:00
freelist.go Add freelist. 2014-02-10 14:04:01 -07:00
freelist_test.go Add freelist. 2014-02-10 14:04:01 -07:00
meta.go Add freelist. 2014-02-10 14:04:01 -07:00
meta_test.go Add freelist. 2014-02-10 14:04:01 -07:00
node.go Mmap remap. 2014-02-12 11:49:57 -07:00
node_test.go Fix quick tests. 2014-02-05 07:56:13 -07:00
os.go Add freelist. 2014-02-10 14:04:01 -07:00
os_test.go Intermediate. 2014-01-27 10:11:54 -05:00
page.go Add freelist. 2014-02-10 14:04:01 -07:00
page_test.go Add freelist. 2014-02-10 14:04:01 -07:00
quick_test.go Cursor iteration. 2014-02-11 09:17:48 -07:00
rwtransaction.go Mmap remap. 2014-02-12 11:49:57 -07:00
rwtransaction_test.go Add freelist. 2014-02-10 14:04:01 -07:00
syscall_darwin.go Mmap remap. 2014-02-12 11:49:57 -07:00
syscall_darwin_test.go Mmap remap. 2014-02-12 11:49:57 -07:00
syscall_linux.go Mmap remap. 2014-02-12 11:49:57 -07:00
syscall_linux_test.go Mmap remap. 2014-02-12 11:49:57 -07:00
transaction.go Add freelist. 2014-02-10 14:04:01 -07:00
transaction_test.go Cursor iteration. 2014-02-11 09:17:48 -07:00
warn.go Add RWTransaction.Put(). 2014-02-01 12:30:37 -05:00

README.md

Bolt Build Status Coverage Status Project status

Overview

Bolt is a pure Go key/value store inspired by Howard Chu and the LMDB project. The goal of the project is to provide a simple, fast, and reliable database for projects that don't require a full database server such as Postgres or MySQL. It is also meant to be educational. Most of us use tools without understanding how the underlying data really works.

Since Bolt is meant to be used as such a low-level piece of functionality, simplicity is key. The API will be small and only center around getting values and setting values. That's it. If you want to see additional functionality added then we encourage you submit a Github issue and we can discuss developing it as a separate fork.

Simple is the new beautiful.

— Tobias Lütke

Project Status

Bolt is currently in development and is currently not functional.

Comparing Bolt vs LMDB

Bolt is inspired by LMDB so there are many similarities between the two:

  1. Both use a B+Tree data structure.

  2. ACID semantics with fully serializable transactions.

  3. Lock-free MVCC support using a single writer and multiple readers.

There are also several differences between Bolt and LMDB:

  1. LMDB supports more additional features such as multi-value keys, fixed length keys, multi-key insert, direct writes, and bi-directional cursors. Bolt only supports basic Get(), Put(), and Delete() operations and unidirectional cursors.

  2. LMDB databases can be shared between processes. Bolt only allows a single process to use a database at a time.

  3. LMDB is written in C and extremely fast. Bolt is written in pure Go and, while it's fast, it is not as fast as LMDB.

  4. LMDB is a more mature library and is used heavily in projects such as OpenLDAP.

So why use Bolt? The goal of Bolt is provide a simple, fast data store that is easily integrated into Go projects. The library does not require CGO so it is compatible with go get and you can easily build static binaries with it. We are not accepting additional functionality into the library so the API and file format are stable. Bolt also has near 100% unit test coverage and also includes heavy black box testing using the testing/quick package.

Internals

The Bolt database is meant to be a clean, readable implementation of a fast single-level key/value data store. This section gives an overview of the basic concepts and structure of the file format.

B+ Tree

Bolt uses a data structure called an append-only B+ tree to store its data. This structure allows for efficient traversal of data.

TODO: Explain better. :)

Pages

Bolt stores its data in discrete units called pages. The page size can be configured but is typically between 4KB and 32KB.

There are several different types of pages:

  • Meta pages - The first two pages in a database are meta pages. These are used to store references to root pages for system buckets as well as keep track of the last transaction identifier.

  • Branch pages - These pages store references to the location of deeper branch pages or leaf pages.

  • Leaf pages - These pages store the actual key/value data.

  • Overflow pages - These are special pages used when a key's data is too large for a leaf page and needs to spill onto additional pages.