- Project tools
-
-
-
- How do I...
-
| Category |
Featured projects |
| scm |
Subversion,
Subclipse,
TortoiseSVN,
RapidSVN
|
| issuetrack |
Scarab |
| requirements |
xmlbasedsrs |
| design |
ArgoUML |
| techcomm |
SubEtha,
eyebrowse,
midgard,
cowiki |
| construction |
antelope,
scons,
frameworx,
build-interceptor,
propel,
phing
|
| testing |
maxq,
aut
|
| deployment |
current |
| process |
ReadySET |
| libraries |
GEF,
Axion,
Style,
SSTree
|
| Over 500 more tools... |
|
SVNChecker
SVNChecker is a framework for Subversion pre-commit and
post-commit hook scripts.
The SVNChecker allows tightly integration Subversion
into existing work and development environments and to
perform a variety of checks of the committed source code. The
major use cases are the integration of Subversion with issue
and bug tracking systems and the check of source code
against coding standards. But, the SVNChecker can be used
for any other use case where a tight integration with the
source code management is necessary.
SVNChecker is designed and developed as an open extensible
framework. By writing appropriate plug-ins, it can be
extended to almost any kind of "check" (for example, to
check source code or to query external databases). This can
be used to enforce a variety of regulations of the software
development process. By developing "handler" plug-ins, the
result of checks can be handled in almost all needed ways
(for example, for sending emails, updating databases, or
running any given external software).
Chapter 1. Architecture
The basic architecture of SVNChecker is displayed in the
following graphic. It provides a flexible framework which is
easily extendable by Checks and Handlers:
The entry point for SVNChecker is the capability of
Subversion to support hook scripts which are called when
specific events are triggered by the repository. There are
nine different kinds of hook scripts, three deal with the
commit process, two with the change of revision properties
and four with the handling of locks. For a complete
reference of the hook scripts take a look at the
"Repository Hooks"
chapter in the Subversion manual. For now it is enough to
know about the pre-commit and post-commit hook.
The pre-commit script is called whenever a user wants to
commit his changes to the repository. A complete snapshot of
how the repository would look after this commit, a so called
transaction, is made. The transaction also includes
information about the author of the commit, the commit
message and a list of changed files. You can perform checks
on this transaction and reject it if doesn't conform to your
rules.
The post-commit script is called whenever a transaction was
completed and a new revision exists. A revision contains the
same information as a transaction with the minor difference
that the revision was already committed and can't be undone
anymore. But you still might want to send a log message
containing a list of all changed files to a mailing list for
example.
In the following the term transaction will be used for both,
a real transaction and a revision, since the SVNChecker
provides an abstraction layer which handles both cases
uniformly.
Based on this transaction a list of checks, e.g. code
convention examination, verification of access rights or
validation of the commit message, can be performed. Each
check returns an exit code, which is zero for success and
bigger than zero for failure, and a message including
further details.
After each check was performed, the message, the exit code
and the transaction are passed to the configured handlers. A
handler is an extension which processes the output of the
checks, e.g. it can send mails, create a log file or print
the output to the console.
The configuration of the SVNChecker is very flexible. You
can define the checks for the pre-commit and post-commit
cases and a list of handlers for each check. Furthermore
each check and handler has access to the configuration for
its own purpose. If you use SVNChecker for more than one
repository you can define a global system configuration with
fixed values which cannot be overwritten by the local
configuration in each repository. For bigger repositories
you can configure different profiles affecting different
parts of the repository. Each profile can have its own
configuration and share some default values with the other
profiles. One use-case may be stricter checks for the trunk
and laxer rules for branches.
Chapter 2. Installation
SVNChecker requires at least python 2.4 and Subversion
1.2. It runs on both, Windows and *nix. Some checks may
depend on external tools. For the requirements of the
individual checks take a look at the "Available Modules"
section.
After extracting the SVNChecker package to an arbitrary
directory, you have to create two files in your
Subversion hooks directory. On *nix systems those files
are named pre-commit and post-commit and must be set
executable. They contain the following content, where
%pathtosvnchecker% is replaced by the directory to which
you unpacked the SVNChecker and %hook% is either
PreCommit or PostCommit (case-sensitive):
#!/bin/sh
/usr/bin/env python %pathtosvnchecker%/Main.py %hook% $1 $2 || exit 1
On Windows systems those files are named pre-commit.bat and
post-commit.bat with the following contents,
%pathtopython% replaced by the path to the python
executable:
%pathtopython% %pathtosvnchecker%/Main.py %hook% $1 $2 || exit 1
Chapter 3. Configuration
The SVNChecker can use any combination of the following
two configuration files. The first one is a global
configuration file which affects all repositories
installed on the same system and must be located in the
SVNChecker directory, just aside of "Main.py", named
"svncheckerconfig.ini". The second one is a repository
local configuration file which must be located in the
"hooks" directory of a repository and names
"svncheckerconfig.ini" as well. If only one of those
configuration files exist, this only one will be used.
If both exist, the options from the global file will
override the options from the local one. The file format
is a simple ini-like format with categories, which are
used for profiles, and key=value pairs.
The basic configuration of the SVNChecker is very
simple. It starts with a category, named "Default".
After that you need to specify which checks to activate
for the pre-commit and post-commit hooks. For each check
you then activate the desired handlers for the success
and failure cases. An example is shown below:
[Default]
Main.PreCommitChecks=Pylint, XMLValidator
Main.PostCommitChecks=Log
Pylint.SuccessHandlers=Console, Mail
Pylint.FailureHandlers=Console, Mail
XMLValidator.SuccessHandlers=Console, Mail
XMLValidator.FailureHandlers=Console, Mail
Log.SuccessHandlers=Console, Mail
Log.FailureHandlers=Console, Mail
You might have seen in the previous example that the
flexibility is very high, you can configure a success
and failure handlers for each check. But this can make
the configuration file very big and unclear, typing the
same code every time although you want to use the same
handlers for all checks. Furthermore one configuration
option may be required by more then one check, e.g. the
path to the java binary. To support those cases the
SVNChecker does not only look for the real name of the
specified configuration option. If it can't find the
complete name it will substitute the first part of the
check, the one before the "." with "Main". So the
previous example could have also been written as:
[Default]
Main.PreCommitChecks=Pylint, XMLValidator
Main.PostCommitChecks=Log
Main.SuccessHandlers=Console, Mail
Main.FailureHandlers=Console, Mail
If your repository grows or you manage different trunks
and branches, you might want to configure specific
settings for different parts of the repository. In order
to do that the SVNChecker support profiles. Profiles
allow you to configure different settings of the
SVNChecker for different files or paths in the
repository. Profiles are defined by categories in the
configuration file and must have a "Main.Regex"
configuration option which specifies a regular
expression which matches all files this profile is valid
for. All files that didn't match a profile will be
checked against the "Default" profile. Here an example
profile that disables all PreCommitChecks for branches:
[Default]
Main.PreCommitChecks=Pylint, XMLValidator
Main.PostCommitChecks=Log
Main.SuccessHandlers=Console, Mail
Main.FailureHandlers=Console, Mail
[Branches]
Main.Regex=^branches/.*$
Main.PreCommitChecks=
If a specified variable can't be found in the given
profile the one from the default profile will be
checked. In this example Main.PostCommitChecks will be
taken from the default profile.
Sometimes it might be useful to know the path of the
hooks directory of the repository in your configuration
file, e.g. when writing log files. For this purpose you
can use the string "%HOOKS%", which will be replaced by
the real directory during runtime.
Chapter 4. Available Modules
With the help of this check you can restrict or
grant access for all or selected files only to
specified users. The configuration options are:
- AccessRights.Rules
List of rules that should be
checked. Those are just identifiers
for the following options. (Default="")
- AccessRights.%RULE%.CheckFiles
List of regular expressions that
match files that should be checked
by this rule. (Default=".*")
- AccessRights.%RULE%.IgnoreFiles
List of regular expressions that
match files that should not be
checked by this rule. (Default="")
- AccessRights.%RULE%.AllowUsers
List of users that may commit
changes to the files. (Default="")
- AccessRights.%RULE%.DenyUsers
List of users that may not commit
changes to the files. (Default="")
This check rejects commits which contain files which
already exist in the repository with the same name
but one or more characters in a different case.
This check checks-out files from the repository to
the local filesystem. SECURITY: Please be careful to
not accidently overwrite any file. The configuration
options are:
- Checkout.Entries
List of entries that should be
checked-out. Those are just
identifiers for the following
options. (Default="")
- Checkout.%ENTRY%.Source
Name of the file in the repository.
- Checkout.%ENTRY%.Destination
Destination, including name, where
to put the file on the local
filesystem.
This check runs checkstyle over a given set of
files. The configuration options are:
- Checkstyle.Java
Path to the java binary.
- Checkstyle.Classpath
Path to the checkstyle-all.jar
- Checkstyle.ConfigFile
Path to the checkstyle configuration file.
- Checkstyle.CheckFiles
List of regular expressions that
match files that should be checked
by this rule. (Default=".*\.java")
- Checkstyle.IgnoreFiles
List of regular expressions that
match files that should not be
checked by this rule. (Default="")
This check tests if a specified list of keywords is
set for a specified set of files. The configuration
options are:
- Keywords.Keywords
Keywords which should be checked.
- Keywords.Keywords.CheckFiles
List of regular expressions that
match files that should be checked
by this rule. (Default=".*")
- Keywords.Keywords.IgnoreFiles
List of regular expressions that
match files that should not be
checked by this rule. (Default="")
Generates a log message. The configuration options
are:
- Log.ViewVcUrl
Url to viewvc to include in the log
message. (Default="")
Checks if the given mantis id(s) exists, is set to
the status "in_progress" and is assigned to the
currently committing user. The Mantis ID must be
specified in the log message and match the pattern
'MANTIS([:#]|[\s\-_]ID) ([0-9]+)'. The configuration
options are:
- Mantis.CheckInProgress
Whether to check the "in_progress"
status. (Default=True)
- Mantis.CheckHandler
Whether to check if the issue is
assigned to the currently committing
user. (Default=True)
- Mantis.URL
URL to the mantisconnect script.
- Mantis.User
Username to login in mantis.
- Mantis.Password
Password to login in mantis.
This check runs pylint over a given set of
files. The configuration options are:
- Pylint.ConfigFile
Path to the pylint configuration file. (Default="")
- Pylint.CheckFiles
List of regular expressions that
match files that should be checked
by this rule. (Default=".*\.py")
- Pylint.IgnoreFiles
List of regular expressions that
match files that should not be
checked by this rule. (Default="")
This check tests if a unit test exists for a given
java class. The java class must follow the pattern
/main/.../Class.java and the unit test must follow
the pattern /main/.../TestClass.java. Interfaces are
omitted. The configuration options are:
- UnitTests.CheckFiles
List of regular expressions that
match files that should be checked
by this rule. (Default=".*\.java")
- UnitTests.IgnoreFiles
List of regular expressions that
match files that should not be
checked by this rule. (Default="")
This check checks if a file is a valid xml file.
- XMLValidator.CheckFiles
List of regular expressions that
match files that should be checked
by this rule. (Default=".*\.xml")
- XMLValidator.IgnoreFiles
List of regular expressions that
match files that should not be
checked by this rule. (Default="")
Prints the output either to sys.stderr or
sys.stdout.
Print the output to a given file. Configuration
options are:
- %Check%.SuccessFile
Path to the file to print the output
in the case of success.
- %Check%.FailureFile
Path to the file to print the output
in the case of failure.
Mails the output to the given addresses. The configuration options are:
- %Check%.SuccessAddresses
Mail addresses to send the output in
the case of success.
- %Check%.FailureAddresses
Mail addresses to send the output in
the case of failure.
Appends the message as bug note to the given mantis
id(s). The Mantis ID must be specified in the
logmessage and match the pattern
'MANTIS([:#]|[\s\-_]ID) ([0-9]+)'. Available
configuration options are:
- Mantis.URL
URL to the mantisconnect script.
- Mantis.User
Username to login in mantis.
- Mantis.Password
Password to login in mantis.
Chapter 5. Developers Guide
Developing Checks and Handlers to extend the SVNChecker is a very easy
process. Each of them must only implement one method which
does the actual work. As helper classes the current
Transaction and Configuration is provided to them.
The Transaction class gives you all information you need
to know about the commit and the repository. It has the
following methods:
- getUserID()
Returns a string with the username of
the current transaction.
- getFiles(checkList, ignoreList)
Returns a map of all modified files. The
keys of the map are the filenames. The
value of each key of the map is the associated
attribute, which can be one of the
default
svnlook changed
attributes.
CheckList must be a list of regular
expressions for files which should be
included, it defaults to [".*"].
IgnoreList must be a list of regular
expressions for files which should be
ignored, it defaults to [].
- getFile(filename)
Returns the path to a temporary copy of
a file in the repository.
- fileExists(filename, ignoreCase=False)
Returns whether a file exists in the
current transaction or revision of the
repository, optionally case-insensitive.
- getCommitMsg()
Returns the commit message. - getRevision()
Return the id of the revision or
transaction.
- getProperty(keyword, filename)
Returns a specified property of a file.
- hasProperty(keyword, filename)
Checks if a given file has the given
property.
- listProperties(filename)
Returns a list of names of the
properties for a file.
The Configuration class provides you with all required
methods to access configuration variables:
- getString(var, default=None)
Returns a variable as string. If the
variable does not exist and default is
set default will be returned, otherwise
a NoSuchConfigurationValueError will be
raised.
- getArray(var, default=None)
Returns a variable as array. The
configuration string is split by the ','
character. If the variable does not
exist and default is set default will be
returned, otherwise a
NoSuchConfigurationValueError will be
raised.
- getBoolean(var, default=None)
Returns a variable as boolean. True,
true, 1 will return True. False, false,
0 will return False. If the variable
does not match the pattern a ValueError
will be raised. If the variable does not
exist and default is set default will be
returned, otherwise a
NoSuchConfigurationValueError will be
raised.
- getInteger(var, default=None)
Returns a variable as integer. If the
variable cannot be converted to an
integer a ValueError will be raised. If
the variable does not exist and default
is set default will be returned,
otherwise a
NoSuchConfigurationValueError will be
raised.
To implement a new Check you only need to write a python
module containing a method named "run", with two
parameters, "transaction" and "config". The return value
must be a tuple containing the return message and the
exit code. You have to put the check into the "checks"
package of the SVNChecker. Here an example of a simple
check, testing if a xml file is parseable:
from xml.dom import minidom
from xml.parsers import expat
def run(transaction, config):
check = config.getArray("XMLValidator.CheckFiles", [".*\.xml"])
ignore = config.getArray("XMLValidator.IgnoreFiles", [])
files = transaction.getFiles(check, ignore)
msg= ""
for filename, attribute in files.iteritems():
if attribute in ["A", "U"]:
try:
minidom.parse(transaction.getFile(filename))
except expat.ExpatError, e:
msg += "XML Validation error in file %r: %s" % (filename, e)
if msg:
return (msg, 1)
else:
return ("", 0)
To implement a new Handler you only need to write a
python module containing a method named "run", with
five parameters, "transaction", "config", "check", "msg"
and "exitCode". "check" is the name of the just executed
check. Here an example of a simple handler, printing the
output to the console:
import sys
separator = "\n" + "=" * 80 + "\n"
def run(transaction, config, check, msg, exitCode):
if (exitCode == 1):
out = sys.stderr
else:
out = sys.stdout
out.write(separator)
out.write(msg)
out.write(separator)
Chapter 6. Changelog
Fixed bug in the pylint check that made the check pass although it failed.
Made the "Pylint.ConfigFile" configuration option optional, a default pylintrc will be used if you don't specify it.
Fixed a bug in the transaction module that returned an empty file when calling getFile() more than once.
Fixed a bug in the transaction module to avoid a deadlock when receiving big files.
Make the SVNChecker compatible with python 2.4.
This is the first big revision of the SVNChecker. The
interfaces to implement a Check or Handler were changed.
Furthermore the Transaction and Config classes have been
improved. If you develop your own modules take a look at
the updated "Developers Guide". A lot of checks have
been enhanced, e.g. Checkstyle and Pylint are faster
now, and new Checks were added, including Checkout and
CaseInsensitiveFilenameClash. For a complete reference
of the available Checks and Handlers and their
configuration options take a look at the "Available
Modules" section. The configuration now supports a
system-wide file which can define configuration options
which are fixed for all installed repositories. If you
want to use the hooks directory in the configuration
file you can now use %HOOKS%, which will automatically
be replaced by the real directory. With the help of a
new test-environment all modules have undergone a better
testing for improved stability. A complete list of fixed
issues is available via the issue tracker at the tigris
project page.
|