Commit Graph

41 Commits (1395a04d97d598ad435d89c5ef1734956b2c9f00)

Author SHA1 Message Date
Ben Johnson 6e977e2f19 Only update rebalance time if nodes are rebalanced.
This commit changes TxStats.RebalanceTime to only update if there are nodes that have been
rebalanced. Previously, the RebalanceTime would include a small amount of time to check if
there were nodes to rebalance but this is confusing to users and not terribly helpful.
2014-07-15 07:26:02 -06:00
Ben Johnson ac4838472d Recover from panics appropriately.
This commit adds a defer handler to ensure that transactions are always closed out - even
in the event of a panic within user code. It's recommended that applications always fail
in the event of a panic but some packages such as net/http will automatically recover
which is a problem (IHMO).
2014-07-11 09:54:10 -06:00
Ben Johnson def455554b Add freelist cache.
This commit adds a cache to the freelist which combines the available free pages and pending free pages in
a single map. This was added to improve performance where freelist.isFree() was consuming 70% of CPU time
for large freelists.
2014-06-30 08:01:41 -06:00
Ben Johnson 642b104396 Add DefaultOptions variable.
This commit adds an explicit DefaultOptions variable for additional documentation.
Open() can still be passed a nil options which will cause options to be change to
the DefaultOptions variable. This change also allows options to be set globally for
an application if more than one database is being opened in a process.

This commit also moves all errors to errors.go so that the godoc groups them together.
2014-06-22 12:44:20 -06:00
Martin Kobetic 571f201672 split the freelist page count stats to free and pending 2014-06-20 14:53:25 +00:00
Martin Kobetic 8a386756df fix up freelist stats naming and add FreeAlloc 2014-06-18 18:10:42 +00:00
Martin Kobetic 4918ce8301 drop mergeStats and move freelist stats update to Tx 2014-06-17 19:30:10 +00:00
Martin Kobetic c105316292 add freelist stats to db stats 2014-06-17 18:40:56 +00:00
Ben Johnson 4db99647eb Fix freelist rollback. 2014-06-13 15:50:47 -06:00
Ben Johnson f448639ce4 Check for freelist overflow. 2014-06-13 07:56:10 -06:00
Ben Johnson 63373660bc Add fallback for O_DIRECT in Tx.Copy().
This commit adds the ability for Bolt to fallback to using a regular file open if Tx.Copy()
errors while opening with O_DIRECT. This only affects Linux.
2014-06-05 09:58:41 -06:00
Ben Johnson 4e31e9d8f9 Merge pull request #177 from tv42/tx-copy-rollback
Do not attempt manual transaction rollback in Tx.Copy
2014-05-28 13:31:41 -06:00
Ben Johnson 754966bea0 Optimize Tx.Check().
This commit removes several memory allocations occurring on every page and also caches the freelist map used when iterating over the pages. This results in significantly better performance.
2014-05-28 12:50:48 -06:00
Tommi Virtanen 394862d541 Do not attempt manual transaction rollback in Tx.Copy
The typical use these days is with a managed transaction, via db.View.

The first case (error when re-opening database file) is not tested;
it is harder to instrument, and I have other plans for it.
2014-05-28 10:15:49 -07:00
Ben Johnson b789691976 Add streaming check.
This commit changes Tx.Check() to return a channel through which check errors are returned. This allows
errors to be found before checking the entire data file.
2014-05-28 10:31:22 -06:00
Ben Johnson 92a9f2e200 Remove DB.Check(). Allow read-only Tx.Check().
This commit removes the DB.Check() function and instead makes the user decide
whether a transaction should be writable or read-only. Tx.Check() is not safe
to use concurrently on a read-only transaction, however, it significantly
improves the performance of it.
2014-05-27 11:31:55 -06:00
Ben Johnson bfccbb2cb5 Avoid trashing page cache on Tx.Copy().
This commit change the database copy to use O_DIRECT so that the Linux page
cache is not trashed during a backup. This is only available on Linux.
2014-05-23 11:40:05 -06:00
Martin Kobetic 8cd1c60f7c review comments 2014-05-21 16:57:29 +00:00
Martin Kobetic 0eda6c9f15 add tx.Size() and ensure tx.Copy() copies exactly that many bytes 2014-05-21 15:43:11 +00:00
Martin Kobetic 519d65228e move Copy and CopyFile from DB to Tx 2014-05-21 15:08:37 +00:00
Ben Johnson 1f5fb0208b Add strict mode. 2014-05-14 18:08:55 -06:00
Ben Johnson 25fea2fd9f Refactor split/spill. 2014-05-03 16:21:28 -06:00
Ben Johnson e3957cd0de Add Tx.Cursor().
This commit adds the Cursor() function to Tx. This allows iteration on the root bucket
in a similar way to iteration on other buckets.

Fixes #141.
2014-04-29 07:25:14 -06:00
Ben Johnson 2505b9a7dc Return bucket from CreateBucket() functions.
This commit changes the API for:

    Tx.CreateBucket()
    Tx.CreateBucketIfNotExists()
    Bucket.CreateBucket()
    Bucket.CreateBucketIfNotExists()

These functions now return the *Bucket and error instead of just the error.
2014-04-15 23:45:06 -04:00
Ben Johnson 698b07b074 Add nested buckets.
This commit adds the ability to create buckets inside of other buckets.
It also replaces the buckets page with a root bucket.

Fixes #56.
2014-04-11 12:36:54 -06:00
Ben Johnson 12204df0b5 Rename internal local Tx variables.
This commit changes the local Tx variables from "t" to "tx". This is partly
for consistency with external documentation but also because it just
annoys me for some reason.
2014-04-04 12:03:04 -06:00
Ben Johnson 394e42e3eb Add Tx.OnCommit() handler.
This commit adds the ability to execute a function after a transaction has
successfully committed.
2014-04-04 07:59:24 -06:00
Ben Johnson 686b6a3341 Add performance counters.
This commit adds performance counters for each transaction which are rolled
up to the database level on each commit/rollback. Counters are meant to be
a very fast way to track what is going on in the database. A few timers are
also added in areas where the time.Now() overhead is not noticible.

The DB.Stat() function is now deprecated since the `bolt` CLI now performs
similar functions.

Fixes #108.
2014-04-02 16:03:03 -06:00
Ben Johnson 4ef19124d1 Consolidate file and metafile descriptors.
Previously, a two file descriptors were used for the database: file & metafile. The "file" file
descriptor was used for async writes while the "metafile" file descriptor was used with O_SYNC
writes. This commit changes that so that there's only one file descriptor and it uses fdatasync()
to synchronize writes.
2014-04-02 13:50:03 -06:00
Ben Johnson 440b89418f Write freelist after each commit.
Well, this is embarassing. Somehow the freelist was never getting written after each commit.
This commit fixes that and fixes a small reporting issue with "bolt pages".
2014-03-31 08:52:19 -06:00
Ben Johnson d8e4cffa12 Fix bucket reclamation.
The bucket page is allocated separately from the rest of the pages but the old bucket pages were
not being added to the freelist. This change fixes that and adds a simple check for database
consistency. More advanced consistency checks can be added in the future.

Fixes #82.
2014-03-25 07:25:00 -06:00
Ben Johnson 59fde2f664 Error refactoring.
Fixed up a few error issues and refactored out the Error type.
2014-03-24 08:31:15 -06:00
Tommi Virtanen e9b2cab0fa Re-add tests for write failures
Commit d2173f5f0e removed the complete
os & syscall mocking layer as overly complex. This commit adds back
the simplest possible thing: hooks to control the database file
writes.

Missing tests: TestDBOpenMetaFileError, TestDBMmapStatError.
These are harder to test without more extensive mocking.

Conflicts:
	db_test.go
2014-03-24 07:47:33 -06:00
Tommi Virtanen cb896f6525 Check spill error in Commit 2014-03-23 13:27:37 -07:00
Tommi Virtanen bfb02aec20 Check meta page write error in Commit 2014-03-23 13:27:36 -07:00
Ben Johnson 482f00fdfc Add ErrTxClosed error.
Commit/Rollback and mutable calls on Tx and Bucket now return ErrTxClosed
if the transaction has already been committed or rolled back. Non-mutable
calls have added an assertion to check if the transaction is closed which
will cause a panic. I don't want to introduce an error return for accessor
methods that are being used improperly so I think the panic is appropriate.
2014-03-23 12:20:16 -06:00
Ben Johnson 76f6ead6b0 Mark Do()/With() transaction as managed.
Transaction created from Do() and With() are now considered "managed".
Managed transactions cannot be manually committed or rolled back since
the Do() and With() functions provide that functionally automatically.
Previously, a Tx could be manually committed and then any changes after
that would be lost.
2014-03-23 10:34:53 -06:00
Tommi Virtanen 5ce378b046 Call fdatasync/fsync after writing out non-meta pages
This avoids a case where writes can be reordered so meta page is
written before a page it refers to, potentially causing a corrupt
database after a power loss or kernel crash.
2014-03-22 20:47:08 -07:00
Ben Johnson 0e4d77d424 Add 'bolt pages'. 2014-03-21 22:34:54 -06:00
Ben Johnson 62cf02e21a Fix Tx.Buckets() sort order.
@tv42 reported an issue with bucket names returning incorrectly. Not sure if
this fixes the issue but it is necessary anyway.
2014-03-13 15:08:59 -06:00
Ben Johnson c551e45a47 Consolidate Tx and RWTx. 2014-03-08 20:40:48 -07:00