hak5
/
bolt
mirror of https://github.com/hak5/bolt.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
622 Commits
1 Branch
9 Tags
1.5 MiB
Commit Graph

35 Commits (v1.1.0)

Author SHA1 Message Date
Martin Kobetic 04a3e85793 Merge sorted pgids rather than resorting everything 2015-06-16 13:48:54 -06:00
sasha-s bdc109bdc7 fix `slice bounds out of range`/maxAllocSize bugs 
when accessing the node data we used to use cast to
*[maxAllocSize]byte, which breaks if we try to go across maxAllocSize boundary.

This leads to occasional panics.
Sample stacktrace:
```
panic: runtime error: slice bounds out of range

goroutine 1 [running]:
github.com/boltdb/bolt.(*node).write(0xc208010f50, 0xc27452a000)
    $GOPATH/src/github.com/boltdb/bolt/node.go:228 +0x5a5
github.com/boltdb/bolt.(*node).spill(0xc208010f50, 0x0, 0x0)
    $GOPATH/src/github.com/boltdb/bolt/node.go:364 +0x506
github.com/boltdb/bolt.(*node).spill(0xc208010700, 0x0, 0x0)
    $GOPATH/src/github.com/boltdb/bolt/node.go:336 +0x12d
github.com/boltdb/bolt.(*node).spill(0xc208010620, 0x0, 0x0)
    $GOPATH/src/github.com/boltdb/bolt/node.go:336 +0x12d
github.com/boltdb/bolt.(*Bucket).spill(0xc22b6ae880, 0x0, 0x0)
    $GOPATH/src/github.com/boltdb/bolt/bucket.go:535 +0x1c4
github.com/boltdb/bolt.(*Bucket).spill(0xc22b6ae840, 0x0, 0x0)
    $GOPATH/src/github.com/boltdb/bolt/bucket.go:502 +0xac2
github.com/boltdb/bolt.(*Bucket).spill(0xc22f4e2018, 0x0, 0x0)
    $GOPATH/src/github.com/boltdb/bolt/bucket.go:502 +0xac2
github.com/boltdb/bolt.(*Tx).Commit(0xc22f4e2000, 0x0, 0x0)
    $GOPATH/src/github.com/boltdb/bolt/tx.go:150 +0x1ee
github.com/boltdb/bolt.(*DB).Update(0xc2080e4000, 0xc24d077508, 0x0, 0x0)
    $GOPATH/src/github.com/boltdb/bolt/db.go:483 +0x169
```
It usually happens when working with large (50M/100M) values.
One way to reproduce it is to change maxAllocSize in bolt_amd64.go to 70000 and run the tests.
TestBucket_Put_Large crashes.
 2015-04-16 11:58:02 -07:00
Ben Johnson 349dacfd44 Update max alloc size on 386 arch. 2015-02-27 08:45:05 -07:00
Ben Johnson b9c28b721a Increase max array size to 2GB. 
This commit changes the maxAllocSize from 256GB to 2GB to handle large
values. It was previously 0xFFFFFFF and I tried adding one more "F" but
it caused an "array too large" error. I played around with the value
some more and found that 0x7FFFFFFF (2GB) is the highest allowed value.

This does not affect how the data is stored. It is simply used for type
converting pointers to array pointers in order to utilize zero copy
from the mmap.
 2015-02-26 16:01:58 -07:00
Ben Johnson 515bf03ff9 Fix i686 build error. 2014-06-18 11:53:01 -06:00
Ben Johnson fd7b0efeb5 Increase max nodes per page. 
This commit changes the maxNodesPerPage constant so that it is essentially unlimited.
Previously, a single large transaction could create more nodes than the mock array
could handle.

Fixes #188.
 2014-06-10 09:38:40 -06:00
Ben Johnson 12b36fe70c Fix freelist allocate(). 2014-05-19 14:11:32 -06:00
Ben Johnson 782ead0dbf Fix freelist allocation direction. 
This commit fixes the freelist so that it frees from the beginning of the data file
instead of the end. It also adds a fast path for pages which can be allocated from
the first free pages and it includes read transaction stats.
 2014-05-19 12:08:33 -06:00
Ben Johnson 698b07b074 Add nested buckets. 
This commit adds the ability to create buckets inside of other buckets.
It also replaces the buckets page with a root bucket.

Fixes #56.
 2014-04-11 12:36:54 -06:00
Ben Johnson 0e4d77d424 Add 'bolt pages'. 2014-03-21 22:34:54 -06:00
Ben Johnson a1f43f4d60 Allow reads of unflushed nodes. 
This commit allows cursors to read updated values from within the
RWTransaction.
 2014-03-01 09:13:59 -07:00
Ben Johnson 8ad59edd02 API Documentation. 2014-02-13 10:58:27 -07:00
Ben Johnson 509e93dff4 Add freelist. 2014-02-10 14:04:01 -07:00
Ben Johnson 84939c21f6 Refactor node lookup. 2014-02-07 15:03:29 -07:00
Ben Johnson 0ed3dc3071 Rename sys ☞ buckets. 2014-02-05 22:15:47 -07:00
Ben Johnson 1a17a2cf1e Add RWTransaction.Put(). 2014-02-01 12:30:37 -05:00
Ben Johnson 26f6fefead Add RWTransaction.write(). 2014-01-30 22:20:50 -05:00
Ben Johnson 149d48fb9e Fix leaf/branch deserialization. 2014-01-30 00:11:46 -05:00
Ben Johnson 4ad445aa85 Add freelist page type. 2014-01-29 22:50:29 -05:00
Ben Johnson 4fc84daf2a Refactor meta.copy() and page.init(). 2014-01-29 22:35:58 -05:00
Ben Johnson bb7a93f750 Add tpage.read() test. 2014-01-28 16:18:16 -05:00
Ben Johnson ef590ecdfb lpage 2014-01-27 22:22:37 -05:00
Ben Johnson 192649f453 Intermediate. 2014-01-27 10:11:54 -05:00
Ben Johnson 1baa6d576a Initialize transaction/rwtransaction. 2014-01-26 15:29:06 -07:00
Ben Johnson 73ab1d420d TODO 2014-01-24 16:32:18 -07:00
Ben Johnson 20b26eac78 TODO 2014-01-24 12:51:56 -07:00
Ben Johnson bce3e667df Intermediate commit. 2014-01-21 15:00:48 -07:00
Ben Johnson 153372abd4 Refactoring to RWCursor, RWTxn, and branch/leaf nodes and pages. 2014-01-17 15:23:39 -07:00
Ben Johnson cec7b942e7 Add system buckets. 2014-01-14 13:01:02 -07:00
Ben Johnson 79d9b6bb5a Begin Transaction.Cursor(). 2014-01-13 10:35:04 -07:00
Ben Johnson f1d7fe5b08 Finish open coverage. 2014-01-13 08:25:56 -07:00
Ben Johnson ee24437bfc Initial db.open. 2014-01-11 22:51:01 -07:00
Ben Johnson df8333328f DB.Open(), pages, and meta. 2014-01-10 07:32:12 -07:00
Ben Johnson f922c1d2bc Move all C code into repo. 2014-01-09 09:07:10 -07:00
Ben Johnson ebc9f0da9e Basic types. 2014-01-08 08:06:17 -07:00