This commit adds Windows support to Bolt. Windows memory maps return an address instead of a byte
slice so the DB.data field had to be refactored to be a pointer to a large byte array.
This commit changes Tx.Check() to return a channel through which check errors are returned. This allows
errors to be found before checking the entire data file.
This commit removes the DB.Check() function and instead makes the user decide
whether a transaction should be writable or read-only. Tx.Check() is not safe
to use concurrently on a read-only transaction, however, it significantly
improves the performance of it.
This commit fixes the freelist so that it frees from the beginning of the data file
instead of the end. It also adds a fast path for pages which can be allocated from
the first free pages and it includes read transaction stats.
This commit adds the ability to adjust the fill percentage for splitting nodes. This
works by setting a threshold that is a percentage of a total page size. When that
threshold is crossed during a split then a new node is created.
This is primarily beneficial for append-only workloads.
Fixes#163.
This commit adds support for writing small buckets directly inline to their value in
their parent's leaf node. Previously, subbuckets would simply have a bucket header
stored in their parent bucket which pointed to the root page. This required that
every bucket use at least a single page. This has a high overhead for buckets with
only one or two small items.
Inline buckets checks subbuckets to see if they only have a small amount of data
(about 1kb) and no subbuckets. If these conditions are met then the bucket's root
node is written to a fake page which is simply a pointer to the end of the bucket's
header.
Fixes#124.
This commit consolidates some of the smaller files into some of the larger files.
The smaller files cluttered the file tree and made it harder to see the logical
groupings of structs.
This commit changes the local Tx variables from "t" to "tx". This is partly
for consistency with external documentation but also because it just
annoys me for some reason.
This commit adds checksums to the meta pages on every write. When the
database loads, it verifies the checksums on the meta pages and returns
an error if either one is corrupt.
In the future, it should fallback to the previous meta page but for right
now it just hard fails. This is at least preferable to opening the database
and getting a random error or further corruption.
Fixes#25.
This commit adds performance counters for each transaction which are rolled
up to the database level on each commit/rollback. Counters are meant to be
a very fast way to track what is going on in the database. A few timers are
also added in areas where the time.Now() overhead is not noticible.
The DB.Stat() function is now deprecated since the `bolt` CLI now performs
similar functions.
Fixes#108.
This commit adds advisory locking via flock() to the database file. This ensures that two separate
processes cannot both open the same data file which would cause corruption.
Fixes#110.
Previously, a two file descriptors were used for the database: file & metafile. The "file" file
descriptor was used for async writes while the "metafile" file descriptor was used with O_SYNC
writes. This commit changes that so that there's only one file descriptor and it uses fdatasync()
to synchronize writes.
Well, this is embarassing. Somehow the freelist was never getting written after each commit.
This commit fixes that and fixes a small reporting issue with "bolt pages".
I consolidated the DB.Tx() and DB.RWTx() calls into a single
DB.Begin(writable bool) call. This is more consistent with the
database/sql library.
I also changed the DB.Do() and DB.With() call to DB.Update() and
DB.View(), respectively. This is more intuitive and more inline with
other database verbiage.
Commit d2173f5f0e removed the complete
os & syscall mocking layer as overly complex. This commit adds back
the simplest possible thing: hooks to control the database file
writes.
Missing tests: TestDBOpenMetaFileError, TestDBMmapStatError.
These are harder to test without more extensive mocking.
Conflicts:
db_test.go
Write errors are often delayed and reported only by the close.
The extra close in defer on success is harmless, (*os.File).Close
protects itself against multiple closes, and this way it's immediately
obvious there is no code path that would leak open files.
Transaction created from Do() and With() are now considered "managed".
Managed transactions cannot be manually committed or rolled back since
the Do() and With() functions provide that functionally automatically.
Previously, a Tx could be manually committed and then any changes after
that would be lost.
Changes munmap to return an error and the DB now implements io.Closer.
I also removed all the OS and Syscall mocking because it's causing issues.
Corrupt file tests need to be recreated but directly using the file system
instead.
Functions such as DB.Put(), DB.Get(), and DB.Delete() were originally
added to be easy to use, however, after implementing Bolt in multiple
projects I have found these ease-of-use functions useless. Nearly
every use case requires multiple calls in a single transaction.
Using the DB ease of use functions turned out to be an antipattern.
I changed the Transaction/RWTransaction types to Tx/RWTx, respectively. This makes the naming
more consistent with other packages such as database/sql. The txnid is changed to txid as well.
Fixes#54. Previously the DB was calculating a minimum mmap size but
using the wrong variable after it calculated the size. This commit
changes the DB to use the correct variable.
Ben Johnson
Martin Kobetic
Steven Normore
Tommi Virtanen