Good Practice Guide

Software Management

Contents

Introduction

This good practice guide aims to provide information about the different tools available for software management on HECToR. It will give an overview of the types of tools available and the reasons why they are useful. The main focus will be on source code control and build systems.

Details of the capabilities of specific tools available on HECToR will be provided along with instructions on how to access them. We will not provide full details on how to use each of the tools as this is outside the scope of a good practice guide. We will however provide links to relevant sources of learning materials and further information.

Source Code Control

Source code control systems (often called version control systems) allow software to be developed by multiple people in a disciplined way. Changes made to code can be tracked, so that it is easy to "undo" changes if they prove not to be beneficial. Branches can be made so that different versions of the software can develop independently - for example, a new version of the software might be developed by adding new features, while just bug fixes might be applied to a released "version" of the same software.

The source code control system will enable multiple developers to work on the same code at the same time, managing associated problems, and ensuring that the integrity of the code is maintained by making it difficult for one developer to accidentally overwrite changes made by another developer. If two developers happen to make changes to the same file, the system may be able to automatically merge the two sets of changes.

Records are kept of which developer made which changes, so if necessary it is possible to find out who was responsible for any particular change. In addition, log messages are kept which can help you remember why changes were made.

Even if there is only one developer there are still advantages to using a source code management system; the system enables the developer to track changes over time, and recover older versions.

There are four source code control systems available on HECToR; CVS, SVN, Git and Bazaar. Any one of these may be suitable for your project and more details of each are given in the following sections. It is good practice to use source code control.

CVS

CVS (Concurrent Versions System) is a free open source version control system. It employs a client-server architecture, whereby a server stores the current version of the project as well as its history. Clients then connect to this server to "check out" a working copy of the project, make their changes and then "check in" those changes. Multiple developers can work on the project at the same time by making modifications to their own working copy and then submitting these changes to the server.

The server is responsible for combining your changes with the latest version, keeping track of version numbers and storing user supplied comments. If your working copy "conflicts" with the latest version some user intervention may be necessary to deal with the conflict before the server will accept the commit. To avoid these situations users are advised to regularly update their working copy against the latest version on the server.

We will not attempt to provide full details of how to use CVS here. Instead more information on CVS can be found in this CVS Manual. It covers setting up your repository, adding/deleting/renaming files and directories, branches, browsing the repository history, and much more.

The CVS commands are available on HECToR automatically without the need to load any modules.

SVN

SVN (Subversion) is another open source version control system which employs the client-server model. It was designed as "a compelling replacement for CVS" and as such has a very similar set of commands while overcoming many of the limitations of CVS. Of special importance is the improved conflict resolution in SVN over CVS. Other changes include revision numbers being for trees rather than individual files, version of directories within trees and much more.

If you do not already use a version control system and choose to start using a client-sever based version control system it is recommended you use SVN. For those currently using CVS and wishing to switch to SVN there is a tool available, cvs2svn.

A very good and full guide to using SVN for your project can be found in this SVN Book. To make the SVN commands available on HECToR you will need to load the SVN module.

Git

Git is a free and open source version control system designed to be fast and small. Unlike CVS and SVN, Git employs a distributed model rather than a client-server model of version control. With Git, every developer has a local copy of the full development history. They have full revision tracking abilities all the time with no need for access to a central server.

Changes are copied from one developer's local repository to another developer's as an additional development branch. This additional development branch can then be merged with any locally developed branches to get an updated version of the code.

As Git uses the distributed model there is much branching and merging and Git does this very efficiently. Git is also very fast for fetching revision histories due to the information being held locally. It performs especially well for large projects as Git does not get much slower as the project history grows.

There are numerous Git tutorials, from quick start guides to the full manual, available on the Git Tutorials.

To make the Git commands available on HECToR you will need to load the Git module.

Bazaar

Bazaar (bzr) is a free version control system which supports both the client-server model and the distributed model. Supporting both models makes Bazaar very flexible and suitable for most projects. As with Git branching and merging is done very efficiently by Bazaar. The Bazaar commands are very similar to those of SVN/CVS.

For more details on Bazaar and how to use it please see the Bazaar documentation.

To make the Bazaar commands available on HECToR you will need to load the Bazaar (bzr) module.

Build Tools

Build tools are designed to automate many of the tasks a software developer will do on a daily basis. The most obvious task is the compiling and linking of source code into a binary. In a large project involving a number of source files and linking against many libraries the build process can become complex.

There will be numerous dependencies between the files meaning they need to compiled and linked in a particular order. You may also want to have a series of different builds which require a different set of compiler options (debugging or release builds) or linking against different libraries. As the complexity starts to grow issuing all the commands on the command line becomes tedious and is prone to error. Implementing a build system using one of the build tools detailed here will manage many of these complexities by automating much of the build process.

Other advantages include:

  1. The build tools can eliminate redundant work from the build process. Only those files which have changed or depend on files which have changed need to be re-compiled and most build tools can work this out automatically.
  2. Builds tools make life easier for end users who want to build and install the package. By using a build tool this process becomes automated and the user does not need any knowledge of the underlying build process.
  3. These tools can also be used to do much more than compiling code. For example they can be used for running tests, producing release packages and creating documentation or release notes.
There are several build tools available on HECToR: make, Autotools, CMake and qmake. Any one of these may be suitable for your project. More details of these are given in the following sections.

GNU Make

Make is one of the oldest and most widespread build tools available. The developer must write a Makefile which lists all the files needed to construct the executable and how to go about doing it.

A Makefile consists of a set of rules, targets and dependencies. "Rules" tell make what commands to issue to create a specific "target". It also lists the "dependencies" of each "target", which are those files which the commands issued in the "rule" require.

When you type "make" on the command line Make will attempt to build the first target in the Makefile. You can also specify which target for Make to build by specifying the target on the command line. Make will follow the rules specified in the Makefile to build the desired target. It will also issue any rules to build other targets which are listed as dependencies for the target you've asked it to build.

Makefiles can be as simple or as complex as you like. For full details see the GNU Make Manual. The manual gives full details of the Make system as well as a series of example Makefiles which can be adapted for your project.

GNU Autotools

Autotools (GNU Build System) is a set of programming tools, comprised of automake, autoconf and libtool. These tools are best used together to form a complete build system for your project. Autotools is a fairly heavy-weight build system which may not be suitable for many projects. It requires the learning of higher level languages to describe the desired Makefile and configure script behaviour. It does however provide solutions to many of the problems of writing portable cross-platform software.

Automake produces portable Makefiles for use with GNU Make, described above. The developer specifies their build needs in a Makefile.am file. Automake then transforms this into a Makefile.in file which is taken by the configure script (generated with Autoconf) to produce the final Makefile for the user's system.

Autoconf is the tool for producing a "configure" script for your project. A configure script will be run prior to the installation of your software. It inspects the user's system and then generates appropriate Makefiles and header files for the system. The configure script is there to make life easier for users of your software. Autoconf does not automatically generate a configure script but instead relies on the developer specifying how the configure script should behave in a configure.ac file. Autoconf can then take the configure.ac file and turn it into a portable configure script.

Libtool is used to create portable libraries, either static or dynamic. Creating portable libraries can be difficult as systems often have different shared library formats, different flags to create libraries or may not support shared libraries at all. Keeping track of this in a Makefile is a very complex task. Libtool attempts to provide a solution to this problem. Libtool does this by providing a new library format (.la) which abstracts all others. Libtool provides a wrapper around the compiler and linker to specify the correct system specific flags so that in your Makefile.am you create and link against .la files. In this way all system specific details are dealt with by Libtools.

For a full explanation on using these tools see the Autotools Tutorial.

CMake

CMake (Cross Platform Make) is another open source build system. It is a series of tools designed to build, test and package software. The tools available include CMake, CTest and CPack. CMake controls the software compilation process using simple platform and compiler independent configuration files (mainly your CMakeLists file). CMake uses these files to generate makefiles and workspaces suitable for the compiler environment you choose. CMake is a powerful tool with support for complicated environments requiring system configuration, pre-processor generation, code generation, and template instantiation.

Numerous quality tutorials, manuals and guides can found on the CMake Documentation page.

The full suite of CMake tools are available on HECToR by loading the CMake module.

References

Source Code Control:

Build Tools:
Wed Jul 31 15:59:12 BST 2013