The purpose of this article is to show how to write tests in GAP. Either for your own code, or code in the GAP standard library.
I will begin by assuming you have installed GAP, either the latest release, or a development version
GAP's test file format looks like the result of running gap. This is intensional. Here is an example test file which tests the
+ operator. We have purposefully added two tests which are incorrect,
1 + 0 (which isn't
6) and the final test (which is formatted incorrectly).
gap> 1 + 2; 3 gap> 1 + (-1); 0 gap> 1 + 0; 6 gap> [1,2] + 10; [ 11, 12 ] gap> [1,2] + [20,30]; [ 21, 32 ] gap>  + ; [ ] gap>  + ; 
If you save this to a file called
gapsimpletest.tst, you can run it as follows, and see the failing tests.
gap> Test("gapsimpletest.tst"); ########> Diff in gap.tst:5 # Input is: 1 + 0; # Expected output: 6 # But found: 1 ######## ########> Diff in gap.tst:13 # Input is:  + ; # Expected output:  # But found: [ 3 ] ######## false
The first failing test isn't surprising,
6 after all. The second test shows an important fact – GAP tests for exact string matching, not object equivalence.
Sometimes we want to write tests which can't really be written all on one line. In this case, we can use ‘> ' to start each new line. If a line should produce no output (for example if we end it with
;;), then we move straight on to the next
gap> statement. We can also start lines with a
# to give comments, but only at the beginning of the file, or after a blank line. Let's put all of these things together:
# Checking + gap> 1 + 1 + > 1; 2 # Checking - # This first line produces no output # The second prints out 'x' gap> x := 1 - 1;; gap> x; 0
Earlier we mentioned that GAP's test checks the output string, not that the actual objects produced are equivalent. In some cases, this is exactly what we want – when we are checking how GAP prints for instance. In other cases, it can lead to fragile tests.
A good way to see this is by looking at three ways we can get GAP to print out the symmetric group on 50 points:
gap> g := SymmetricGroup(50); Sym( [ 1 .. 50 ] ) gap> h := Group([ (1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20, > 21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40, > 41,42,43,44,45,46,47,48,49,50), (1,2) ]); <permutation group with 2 generators> gap> g = h; true gap> Size(h); 30414093201713378043612608166064768844377641568960512000000000000 gap> h; <permutation group of size 30414093201713378043612608166064768844377641568960512000000000000 with 2 generators>
If we construct the symmetric group with
SymmetricGroup, then we can see the group is output as
Sym( [1 .. 50 ] ). However, if we just give the generators GAP just remembers it has a
<permutation group with 2 generators>. Finally, if we take that group and ask for it's size, GAP remembers that size and starts printing that out as well.
This causes a problem for writing tests – you have to know exactly which one GAP is going to output, and also hope no internal change to GAP causes it to learn the size (for example), which doesn't change the result of your test, but does change the output.
The other possibility is that a test could break, without changing the text output. For example imagine that we recorded some function should return
<permutation group with 2 generators>. That function's return value could change to any group with two generators, without ever noticing the error.
The solution? The easiest option is to write tests which explictly check the correct object is returned, and just print
true, for example, to check
gap> Intersection(SymmetricGroup(40), AlternatingGroup(20)) = AlternatingGroup(20); true
It is perfectly legal to use
Read to read other code while testing. One technique I often use is to write functions which should output nothing if the tests succeed. Firstly, we will write a file with functions which test intersection. If these functions succeed, they will print nothing
slowInt := function(g1,g2) local perms; perms := Filtered(g1, p -> p in g2); return Group(perms); end;; testIntersect := function() local g1, g2, slowint; for g1 in AllPrimitiveGroups(NrMovedPoints, [1..8]) do for g2 in AllPrimitiveGroups(NrMovedPoints, [1..8]) do if Intersection(g1,g2) <> slowInt(g1,g2) then Print(g1, " and ", g2, "\n"); fi; od; od; end;
Then we can write a trivial test which makes sure this function runs correctly, and prints nothing.
gap> Read("testintersect.g");; gap> testIntersect();
Is generating many random inputs worthwhile? Yes! Here is a bug which was fixed in GAP – the following code didn't work:
Stabilizer(SymmetricGroup(5), [1,2,1,2], OnTuples); # Should be: Group([(3,5),(4,5)]) # Used to be: Group(())
This code was broken in GAP 4.7.5 when:
It is very unlikely (I think) someone would have manually constructed a test that hit this case – but it is easy to hit it if you just brute force randomly chosen groups and lists!
There are two common testing features I haven't covered yet. The first is the function
TestDirectory. This function just runs all tests in a directory (including tests in all sub-directories, recursively). This is useful for testing a whole project.
The second feature measures the speed uses tests for timing, using GAPstones. You will often notice tests start with a line like
START_TEST("file.tst"); and end with
END_TEST("file.tst", 10000);. These lines are used to record how long the file takes to run. My advice is to ignore these lines GAPstones aren't actually useful for measuring the speed of GAP code.