The max mmap size was previous unchecked which resulted in a panic once
the maximum size was reached. This commit adds a check for the max size
when re-mapping and returns an error if the new map will exceed the size.
Thanks to Tamás Gulácsi for testing out the change on i386.
OpenBSD does not include a UBC kernel and writes must be synchronized
with the msync(2) syscall. In addition, the NoSync field of the DB
struct should be ignored on OpenBSD, since unlike other platforms,
missing msyncs will result in data corruption.
Depends on PR #258.
Fixes#257.
This commit moves the DB.FillPercent field to Bucket.FillPercent. This
allows the fill percentage to be specified per-bucket, per-tx. This
value is not persisted and should be set whenever using it.
This commit adds the DB.NoSync flag to skip fsync() calls on each commit. This should only
be used for bulk loading as it can corrupt your database in the event of a system failure.
Initial tests show it can provide a 2x speed up for sequential inserts.
This commit adds a defer handler to ensure that transactions are always closed out - even
in the event of a panic within user code. It's recommended that applications always fail
in the event of a panic but some packages such as net/http will automatically recover
which is a problem (IHMO).
This commit fixes an issue on Windows where the database was doubling
when it was re-opened. This occurred because Windows has to truncate the
file to the mmap size and the mmap resizing code was doubling the size
whenever the DB size was at the next threshold. This has been changed so
that the DB size will double only when the DB size is above the next
threshold.
This commit adds a cache to the freelist which combines the available free pages and pending free pages in
a single map. This was added to improve performance where freelist.isFree() was consuming 70% of CPU time
for large freelists.
This commit adds an explicit DefaultOptions variable for additional documentation.
Open() can still be passed a nil options which will cause options to be change to
the DefaultOptions variable. This change also allows options to be set globally for
an application if more than one database is being opened in a process.
This commit also moves all errors to errors.go so that the godoc groups them together.
This commit changes Open() to provide an additional Options argument. The options
argument currently only has a Timeout which will cause the Open() to return
ErrTimeout if a file lock cannot be obtained in time.
Fixes#207.
This commit reverts merge-split and fixes the node.split() to do a multi-page split. This issue
caused problems with bulk loading because it would split into a small page and a very large page.
The very large page, in turn, would be an arbitrary size so when it was freed later it would be
difficult to reuse and would cause serious fragmentation issues.
This commit adds Windows support to Bolt. Windows memory maps return an address instead of a byte
slice so the DB.data field had to be refactored to be a pointer to a large byte array.
This commit changes Tx.Check() to return a channel through which check errors are returned. This allows
errors to be found before checking the entire data file.
This commit removes the DB.Check() function and instead makes the user decide
whether a transaction should be writable or read-only. Tx.Check() is not safe
to use concurrently on a read-only transaction, however, it significantly
improves the performance of it.
This commit fixes the freelist so that it frees from the beginning of the data file
instead of the end. It also adds a fast path for pages which can be allocated from
the first free pages and it includes read transaction stats.
This commit adds the ability to adjust the fill percentage for splitting nodes. This
works by setting a threshold that is a percentage of a total page size. When that
threshold is crossed during a split then a new node is created.
This is primarily beneficial for append-only workloads.
Fixes#163.
This commit adds support for writing small buckets directly inline to their value in
their parent's leaf node. Previously, subbuckets would simply have a bucket header
stored in their parent bucket which pointed to the root page. This required that
every bucket use at least a single page. This has a high overhead for buckets with
only one or two small items.
Inline buckets checks subbuckets to see if they only have a small amount of data
(about 1kb) and no subbuckets. If these conditions are met then the bucket's root
node is written to a fake page which is simply a pointer to the end of the bucket's
header.
Fixes#124.
This commit consolidates some of the smaller files into some of the larger files.
The smaller files cluttered the file tree and made it harder to see the logical
groupings of structs.
This commit changes the local Tx variables from "t" to "tx". This is partly
for consistency with external documentation but also because it just
annoys me for some reason.
This commit adds checksums to the meta pages on every write. When the
database loads, it verifies the checksums on the meta pages and returns
an error if either one is corrupt.
In the future, it should fallback to the previous meta page but for right
now it just hard fails. This is at least preferable to opening the database
and getting a random error or further corruption.
Fixes#25.
This commit adds performance counters for each transaction which are rolled
up to the database level on each commit/rollback. Counters are meant to be
a very fast way to track what is going on in the database. A few timers are
also added in areas where the time.Now() overhead is not noticible.
The DB.Stat() function is now deprecated since the `bolt` CLI now performs
similar functions.
Fixes#108.
This commit adds advisory locking via flock() to the database file. This ensures that two separate
processes cannot both open the same data file which would cause corruption.
Fixes#110.
Previously, a two file descriptors were used for the database: file & metafile. The "file" file
descriptor was used for async writes while the "metafile" file descriptor was used with O_SYNC
writes. This commit changes that so that there's only one file descriptor and it uses fdatasync()
to synchronize writes.
Well, this is embarassing. Somehow the freelist was never getting written after each commit.
This commit fixes that and fixes a small reporting issue with "bolt pages".
I consolidated the DB.Tx() and DB.RWTx() calls into a single
DB.Begin(writable bool) call. This is more consistent with the
database/sql library.
I also changed the DB.Do() and DB.With() call to DB.Update() and
DB.View(), respectively. This is more intuitive and more inline with
other database verbiage.
Ben Johnson
Josh Rickmar
Martin Kobetic
Steven Normore