Generating useful history logs from Mercurial

I recently decided to keep my software under a proper configuration control system, and for reasons I won’t go into I picked Mercurial.

Something I wanted to do was to make Mercurial produce change history for my projects, but with a “useful” output. That’s to say that the end user of some software won’t care that I fixed a typo in a comment and added some source files while developing a new feature. But these are the sort of messages that get saved by developers when commiting changes.

I’ve seen the same request on several forums, but never found a decent answer, so I did some investigation and came up with a rather simple solution using branches.
The problem is this – when a change is made in Mercurial and committed to the repository, a message is stored. This message generally describes the nature of the change, for example “Added CNewClass to implement new functionality”. When developing a new feature, or fixing a bug, several commits are often made. The question is how to make the following log:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
changeset:   7:4d234647ab55
summary:     Added line to Task3 file
 
changeset:   6:79b7a9f587e9
summary:     Fixed typo in Task3 file
 
changeset:   5:d39e60a1c3a1
summary:     New Task3 file required
 
changeset:   4:4ee84903f8b1
summary:     Implemented Task2
 
changeset:   3:ab1eed767c5a
summary:     Simple fix for Task 2
 
changeset:   2:69f7cd7dcea2
summary:     Forgot the full stop
 
changeset:   1:433d36065ba3
summary:     Added line to Task 1 file

… become something useful to the end user:

1
2
3
* Implemented Task3
* Implemented Task2
* Implemented Task1

Ok, so calling them something other than TaskN would be useful, but this is just an example.

It turns out it’s quite straightforward, makes use of Mercurials Named Branches, and doesn’t need any extensions.

Let’s assume that we work from a base repository, and builds are performed on the “default” branch. We wish to produce a history log file when we perform this build, where the format of this file is the version number or “tag”, followed by all “useful” changes between that and the previous version, repeated for each released version. You’ve all seen the kind of thing.

We’ll start with the base repository which contains only the initial setup files (.hgignore, perhaps some Development Environment files). This should provide us with the first changeset, made on the default branch.

1
2
3
4
5
$ hg init base
$ cd base
  (generate default .hgignore file)
$ hg add .
$ hg commit -m "Project Started"

To start we wish to implement a feature – “Feature 1″. To do this, we will clone the repository, update to the correct start point (which will be done by default), and start a named branch. This is the important part, and something that is easily forgotten.

1
2
3
4
5
$ cd ..
$ hg clone base dev
$ cd dev
$ hg update
$ hg branch feature1

We now make several changes, and can commit any number of times, providing we only perform the commit on the new branch “feature1″. When happy with our changes we can close the branch (this can be done later if need-be), and merge our changes into the default branch:

1
2
3
$ hg commit -m "Feature1 Complete" --close-branch
$ hg update default
$ hg merge Feature1

Assuming there are no conflicts, or any conflicts are easily resolvable, we now commit the merge, and it’s here that we specify the text that will eventually go into our change log. We also ensure that our push creates our new named branch:

1
2
$ hg commit -m "Implemented new feature XYZ"
$ hg push --new-branch

I’ve missed out a few steps here, such as re-pulling the base repository to ensure we merge to the correct point, but the essentials are there.

The result of this is a rather complex change tree, but importantly the “default” branch contains only the results of the merges from the named branches. This means we can use Mercurials “log” command to get just the history that we want:

1
$ hg log -b default

That shows how to list just the main changes rather than the “in-development” changes. How can we turn this into something resembling a useful change history? There are several issues, and these can be solved with the use of Mercurials Revision Set Specification, and its simple templating of log output.

You can download here a repository export which contains several changes on several named branches. You can use it directly, clone it, or unbundle it into an empty repository.

1
2
3
$ hg init changetest
$ cd changetest
$ hg unbundle ../changetree.hg

The repository simulates a small development where two releases (tagged “v0″ and “v1″) have been made, containing a total of 5 “useful” changes, which we want to include in the history log. The tags have been made on the “default” branch, and so using the log command from above, using the built-in “compact” style, we get:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
$ hg log -b default --style compact
 
20[tip]:-1   093ac87b5929   2011-02-21 13:31 +0000   icabod
Added tag v0 for changeset 789f34ba8be0
 
19:17   8b7195a6db1c   2011-02-21 13:22 +0000   icabod
Added tag v1 for changeset 31a0b04db271
 
17[v1]:11,16   31a0b04db271   2011-02-18 14:56 +0000   icabod
Implemented Task5
 
11:9,10   3de78560136f   2011-02-18 14:53 +0000   icabod
Implemented Task4
 
9[v0]:5,8   789f34ba8be0   2011-02-18 14:47 +0000   icabod
Implemented Task3
 
5:3,4   4ee84903f8b1   2011-02-18 14:42 +0000   icabod
Implemented Task2
 
3:0,2   67e566e92c80   2011-02-18 14:39 +0000   icabod
Implemented Task1
 
0   9518f7639580   2011-02-18 14:38 +0000   icabod
Project Started

It’s not particularly readable, and contains several log items that we don’t really want: “Project Started” and the changesets where we’ve added the tags.

The additional, undesirable changes can be filtered using the the Revision Set Specification – full details can be found by running hg help revsets, but for now we’ll use this:

1
$ hg log -b default -r "sort(!9518f7639580 & !tip & !file(.hgtags),-rev)"

We’ll break down the various aspects:

  • -r allows us to specify the changesets that we wish to include. The definition of the changesets follows in quotes.
  • !9518f7639580 removes a changeset, in our case the first changeset. The value itself will differ depending on your repository.
  • !tip removes the “tip” tag, which always points to the latest changeset.
  • !file(.hgtags) filters out any changesets that modify the .hgtags file, which means any changesets which add a tag.
  • All of this is wrapped in a sort(...,-rev) which sorts the remaining unfiltered changeset in reverse order of revision.

What does this give us?

1
2
3
4
5
6
7
8
9
10
11
12
13
14
17[v1]:11,16   31a0b04db271   2011-02-18 14:56 +0000   icabod
Implemented Task5
 
11:9,10   3de78560136f   2011-02-18 14:53 +0000   icabod
Implemented Task4
 
9[v0]:5,8   789f34ba8be0   2011-02-18 14:47 +0000   icabod
Implemented Task3
 
5:3,4   4ee84903f8b1   2011-02-18 14:42 +0000   icabod
Implemented Task2
 
3:0,2   67e566e92c80   2011-02-18 14:39 +0000   icabod
Implemented Task1

Better. It now contains just the changesets that we’re interested in, but the format isn’t great, and the split of changes into versions is difficult to see. This we can fix with the following template file:

1
2
3
4
5
$ cat history.template
changeset = '{tags}* {desc|fill68|strip|tabindent}\n'
start_tags = '\n'
tag = '{tag},'
last_tag = '{tag} : {node|short} @ {date|shortdate}\n'

This template lists any tags associated with a changeset, followed by a description of the changes themselves. The tags are formatted to show the tag name, followed by the changeset hash (so that we can recreate it easily), and the date that the tag was made. We now have:

1
2
3
4
5
6
7
8
9
10
$ hg log -b default -r "sort(!9518f7639580 & !tip & !file(.hgtags),-rev)" --style ./history.template
 
v1 : 31a0b04db271 @ 2011-02-18
* Implemented Task5
* Implemented Task4
 
v0 : 789f34ba8be0 @ 2011-02-18
* Implemented Task3
* Implemented Task2
* Implemented Task1

And there you have it. Using named branches, and Mercurials templating system, we can easily produce a useful history log for inclusion in a software package. No extensions needed, and no loss of history that comes with “pruning” branches, or squashing several changesets into one.

There are some caveats to using this approach however.

  • It relies on the programmer ensuring that they perform their work on a named branch rather than the default branch. It is easy to forget.
  • Each named branch must have a different name – which may cause issues if two people use the same name for different changes.
  • If you are working on a large project, it is probably better, and safer, to use a proper change-tracking system.

Essentially it all comes down to discipline (not that kind). Other than that… have fun.

One Comment

  1. IcyBee
    Posted March 3, 2011 at 4:02 pm | Permalink

    If all “important” changes to the default branch are merges from other branches, you could use:

    hg log -b default -m

    …and you wouldn’t have to filter out the initial version, the tip, or the tags.

    Also, you don’t necessarily need a separate named branch for each feature – you could just keep merging from a development branch.

2 Trackbacks

  1. [...] contain only “complete” changes. The idea stems from a post on my blog about getting useful history logs from Mercurial. However this isn’t just about generating logs, but about a structured way of [...]

  2. [...] contain only “complete” changes. The idea stems from a post on my blog about getting useful history logs from Mercurial. However this isn’t just about generating logs, but about a structured way of [...]