when accessing the node data we used to use cast to
*[maxAllocSize]byte, which breaks if we try to go across maxAllocSize boundary.
This leads to occasional panics.
Sample stacktrace:
```
panic: runtime error: slice bounds out of range
goroutine 1 [running]:
github.com/boltdb/bolt.(*node).write(0xc208010f50, 0xc27452a000)
$GOPATH/src/github.com/boltdb/bolt/node.go:228 +0x5a5
github.com/boltdb/bolt.(*node).spill(0xc208010f50, 0x0, 0x0)
$GOPATH/src/github.com/boltdb/bolt/node.go:364 +0x506
github.com/boltdb/bolt.(*node).spill(0xc208010700, 0x0, 0x0)
$GOPATH/src/github.com/boltdb/bolt/node.go:336 +0x12d
github.com/boltdb/bolt.(*node).spill(0xc208010620, 0x0, 0x0)
$GOPATH/src/github.com/boltdb/bolt/node.go:336 +0x12d
github.com/boltdb/bolt.(*Bucket).spill(0xc22b6ae880, 0x0, 0x0)
$GOPATH/src/github.com/boltdb/bolt/bucket.go:535 +0x1c4
github.com/boltdb/bolt.(*Bucket).spill(0xc22b6ae840, 0x0, 0x0)
$GOPATH/src/github.com/boltdb/bolt/bucket.go:502 +0xac2
github.com/boltdb/bolt.(*Bucket).spill(0xc22f4e2018, 0x0, 0x0)
$GOPATH/src/github.com/boltdb/bolt/bucket.go:502 +0xac2
github.com/boltdb/bolt.(*Tx).Commit(0xc22f4e2000, 0x0, 0x0)
$GOPATH/src/github.com/boltdb/bolt/tx.go:150 +0x1ee
github.com/boltdb/bolt.(*DB).Update(0xc2080e4000, 0xc24d077508, 0x0, 0x0)
$GOPATH/src/github.com/boltdb/bolt/db.go:483 +0x169
```
It usually happens when working with large (50M/100M) values.
One way to reproduce it is to change maxAllocSize in bolt_amd64.go to 70000 and run the tests.
TestBucket_Put_Large crashes.
This commit expands calls to _assert() that use variadic arguments. These calls require conversion to interface{} so there
was a large number of calls to Go's internal convT2E() function. In some profiling this was taking over 20% of total runtime.
I don't remember seeing this before Go 1.4 so perhaps something has changed.
This commit moves the DB.FillPercent field to Bucket.FillPercent. This
allows the fill percentage to be specified per-bucket, per-tx. This
value is not persisted and should be set whenever using it.
This commit fixes a bug where a root split on a very large insert would
cause an overflow of the root node. The problem was that the new root
was not split when it was created so a new root with more than 64K child
nodes would overflow the page.count (uint16).
This fixes an issue where split nodes can be double spilled. This is typically
not noticable but it can have large effects when bulk inserting as double
spilled nodes will get added to the freelist which will grow quickly.
This commit fixes an issue where large nodes would take up most of
the insert time because the entire node size would be calculated to
check if it could fit in a page or not. This changes the behavior
so that a node's size is only calculated up to the size it needs to
check and then returns.
This commit reverts merge-split and fixes the node.split() to do a multi-page split. This issue
caused problems with bulk loading because it would split into a small page and a very large page.
The very large page, in turn, would be an arbitrary size so when it was freed later it would be
difficult to reuse and would cause serious fragmentation issues.
This commit fixes a bug where deletions that caused merge-left rebalances were updating
the parent node which caused a node to "reappear" even after it had been deleted. This was
fixed in merge-right rebalances a while ago but merge-left is less frequent so it was
missed until now.
Many thanks to Jordan Sherer (@jsherer) for finding and reporting the bug.
This commit changes the node.split() functionality to check if the next node has
available space and, if so, it will merge the newly split keys into the next node.
Previously, keys could be continually put into the left side of a split causing that
first half to split off small right side nodes. This was especially problematic with
databases with a high fill percent.
This commit adds a check to prevent circular dependencies in branch nodes. If a circular dependency occurs
then a panic will be called and the commit will be prevented. This only works for a single branch level
and will not recursively search the tree.
This commit adds the ability to adjust the fill percentage for splitting nodes. This
works by setting a threshold that is a percentage of a total page size. When that
threshold is crossed during a split then a new node is created.
This is primarily beneficial for append-only workloads.
Fixes#163.
This commit adds support for writing small buckets directly inline to their value in
their parent's leaf node. Previously, subbuckets would simply have a bucket header
stored in their parent bucket which pointed to the root page. This required that
every bucket use at least a single page. This has a high overhead for buckets with
only one or two small items.
Inline buckets checks subbuckets to see if they only have a small amount of data
(about 1kb) and no subbuckets. If these conditions are met then the bucket's root
node is written to a fake page which is simply a pointer to the end of the bucket's
header.
Fixes#124.
This commit adds performance counters for each transaction which are rolled
up to the database level on each commit/rollback. Counters are meant to be
a very fast way to track what is going on in the database. A few timers are
also added in areas where the time.Now() overhead is not noticible.
The DB.Stat() function is now deprecated since the `bolt` CLI now performs
similar functions.
Fixes#108.
I changed the Transaction/RWTransaction types to Tx/RWTx, respectively. This makes the naming
more consistent with other packages such as database/sql. The txnid is changed to txid as well.
Ben Johnson
sasha-s
Martin Kobetic