Warning

This benchmark was a first attempt to compare the space-efficiency of version-control systems. It did so by importing only the last version of a project, not the full history. The results are therefore highly biased, and should not be seen as "real-life use-case".

Bazaar Benchmarking Results

This page shows how Bazaar stacks up on 40-50 real Use Cases on 33 real projects. Great performance is an important part of overall usability so our goal is to make Bazaar as fast as we can, without sacrificing correctness or other key parts of our user interface.

Note: Early versions of Bazaar were too slow for large projects but it's speed is now orders of magnitude better with the release of Bazaar 1.x. If you like Bazaar's feature set and usability more than any other tool, speed is no longer a reason to select another tool. We're not done improving performance but the 3 big pieces of technology (smart server, dirstate and packs) are now in place so tuning particular Use Cases from here, if required, is 10x easier/faster.

The Projects

Projects range in size from 300 files to 100,000 files representing a who's who of open source projects: drupal-5.5, wordpress-2.3.1, mediawiki-1.11.0, bugzilla-3.0.2, k3b-1.0.4, amarok-1.4.7, inkscape-0.45.1, apache-ant-1.7.0, pidgin-2.3.1, moin-1.5.8, httpd-2.0.61, samba-3.0.28, ruby-1.8.6, gcc-core-4.2.2, xorg-server-1.4, xen-3.1.0, hibernate-20071218, parrot-0.5.0, postgresql-8.3beta4, Python-2.5.1, Joomla-1.5.Beta2, perl-5.9.4,wine-0.9.51, gimp-2.4.3, mysql-5.0.54-20071214, Zope-3.3.1, php-5.2.5, Plone-3.0.4, glassfish-v2-b52g, linux-2.6.23.12, firefox-2.0.0.11, OOo_2.3.1_src_core.

The project archives used are available here: http://people.ubuntu.com/~ianc/benchmarks/src/. If you would like to benchmark Bazaar on your project's code base, download the benchmarking suite used from https://launchpad.net/bzr-usertest/.

Note: The first 30 projects above are considered typical. The last 3 have over 20,000 files and are therefore considered large. For large projects, the optimal way of developing may be different, e.g. developers may want to reuse a checkout.

The Use Cases

For each project, the (bzr-usertest) local script is run 2-3 times and the results are presented for the best of those runs. Here's what the script does in a nutshell:

• unpacks a source code tree
• imports it into Bazaar
• makes another branch and works on it: add, commit, edit, commit, status, diff, log, etc.
• creates a checkout and works on it: add, commit, edit, commit
• creates a lightweight checkout and works on it: add, commit, edit, commit

Disk space usage is also reported as a ratio (versus the size of the original unpacked source) and a total. The final results are presented as CSV files for viewing and processing as you see fit.

Note: The local script still needs some enhancements to cover more operations, particularly merge. That will be done soon.

The Computers

Bazaar 1.1dev (21-Dec-2007)

Summary: branch much faster.

Bazaar 1.0

Summary: Fast enough for all but the very largest projects. Efficiency of disk space usage is very good, without using hard links.

Other comparisons

If you have recently (Jan 2008 or later) benchmarked Bazaar against Git or Mercurial, please add a link to your comparison below.

• Bazzar vs Git on the Linux Kernel

• Distributed Version Control Systems: A Not-So-Quick Guide Through

• A bzr, git and hg. Its conclusion is that Bazaar is in average more than twice less space-efficient than Git.

Note: Given the rapid development rate of Bazaar (and competitive tools), it is recommended that comparisons done prior to 2008 be redone again before being added above.

Disk Space Efficiency

Bazaar's pack format is highly efficient as shown in our comparison benchmarks against Mercurial and Git.