t
Folders and files
| Name | Name | Last commit date | ||
|---|---|---|---|---|
===============================================================================
| ______ ______ ______ |
| /\ __ \ /\__ _\ /\__ _\ |
| \ \ \/\ \ __ __ ______ ______ \/_/\ \/ ______ ___\/_/\ \/ ______ |
| \ \ __//\ \/\ \/\ __ \/\ ___\ \ \ \/\ ___\/\ ___\ \ \/\ ___\ |
| \ \ \/ \ \ \_\ \ \ \/\ \ \___ \ \ \ \ \ ___\ \___ \ \ \ \___ \ |
| \ \__\ \ \____/\ \____ \/\_____\ \ \_\ \_____\/\_____\ \_\/\_____\ |
| \/__/ \/___/ \/___/\ \/____/ \/_/\/____/ \/____/ \/_/\/____/ |
| /\____/ |
| \/___/ |
| |
===============================================================================
Welcome to the Pugs Test suite.
Pugs is currently being developed in a highly test-driven manner. Tests are
written for both implemented and unimplemented features and are based roughly
on the Perl6 Language Synopsis documents. All are welcome and encouraged to
contribute to this test suite. It is the hope that not only will this test
suite be used for Pugs, but that it will eventually become the Perl6 test
suite itself.
-------------------------------------------------------------------------------
Getting Started
-------------------------------------------------------------------------------
Here are some basic guidelines to help you get started writing tests for Pugs.
- Prerequisites
Please read the Perl 6 Spec first, namely, the Synopses:
http://dev.perl.org/perl6/doc/synopsis.html
or directly from the SVN repos:
http://svn.perl.org/perl6/doc/trunk/design/syn
A good grasp of the Perl 6 language itself is very important to writing good
tests.
- If you are unsure of something, don't hesitate to ask.
If you have a question about a test, written or unwritten, log on to #perl6 on
irc.freenode.net or send an email to perl6-compiler and ask someone about it.
If you have read the synopses very carefully and are still unsure about a
perl6 language element, we encourage you to ask on #perl6 or email the
perl6-language list and get clarification. Pugs and Perl6 are group efforts
and asking questions is a good thing.
- What to test
A number of Pugs hackers on #perl6 run regular smoke tests, and you can run
your own using 'make smoke'. The smoke test produces an HTML graph of what tests
are passing, and what aren't. This can be a good place to start. There's a smoke
server on the web:
http://m19s28.vlinux.de/cgi-bin/pugs-smokeserv.pl?
There is also a cross linking which is made between the tests and the synopses.
It's generated by running the 'util/smartlinks.pl' script. You
can generate an HTML version of the latest Synopses cross-referenced with test
file snippets from the test suite using the following command:
$ util/smartlinks.pl t/*/*.t t/*/*/*.t
There's a cross-referenced version of Synopses at
http://perlcabal.org/syn/
which is automatically updated by the smartlinks.pl script on feather once
an hour.
- Use the Test module.
We have created a basic Test module found in ext/Test/lib/Test.pm. It is
written in Perl 6 and implements the TAP protocol (and so can be used with
Test::Harness). The module has its own documentation and I encourage you to
read it.
- Pugs tests should have a non-she-bang line of "use v6-alpha;".
This line helps both Test::Harness as well as the 'prove6' utility when
running Pugs tests.
- Place tests in the appropriate folder.
We have recently undergone a re-organization of the test suite in order to
make it easier to find what has and has not been tested. It is important as
the test suite grows that we try to keep this organization. If you have a test
and are unsure of where to put it, ask on #perl6 for help, or put it in the
general/ folder and email perl6-compiler and let us know.
Warning: please grep the whole test suite to ensure the tests you want to add
are not already in some file.
- Run the test file with util/prove6 to test your tests.
util/prove6 t/foo/bar.t
Make sure the outputs are what you *can* expect even if your tests fail.
- If possible, please smart link your tests to the Synopsis
Smart links are explained on the wiki:
http://rakudo.org/perl6/index.cgi?smart_linking
And in the POD of smartlinks.pl:
perldoc util/smartlinks.pl
You may also find the following two pugs.blogs.com posts helpful:
http://pugs.blogs.com/pugs/2006/08/integrating_the.html
http://pugs.blogs.com/pugs/2006/09/check_smoke_res.html
http://pugs.blogs.com/pugs/2006/09/the_benefits_of.html
Everytime you have added smartlinks to some .t file, remember to run
util/smartlinks.pl to verify the links' validity:
$ perl util/smartlinks.pl --check t/some/test.t
By default, the freshness of the synopses will checked every time. Use "--fast"
to skip that step.
- Dealing with parse failures
When developing tests for features that have not been implemented yet,
often you find yourself writing code that doesn't compile. Don't get
stuck on this: wrap the new code with eval and test it anyway. Just make
sure that the test fails as long as the eval does, and until the feature
has been implemented correctly.
Sometimes code is so futuristic, it can even confuse eval. We call this a
"hard parsefail". When this happens, comment out the failing code, but mark
it so it doesn't get forgotten, like so:
todo :pugs<6.28.0>, v6_pm<0.110>;
flunk("FIXME parsefail");
#ok eval('my code here');
Or another alternate style is as follows:
ok eval('# $code.which(%will, @fail)');
ok eval(q{
blah blah blah
});
is(eval(q{
my $val;
# some code here...
$val;
}), $expected, 'description');
which essentially comments out your eval, and returns 'undef' to ok().
- When TODO and when not TODO.
All of the functions in the Test module also have a 'todo' function. (See
Test.pm's documentation for more details.)
The general rule about todo tests is that if the feature is not
yet implemented, it is TODO. But if a feature is broken, or a bug is found
then the tests should fail and *not* be TODO.
The only exception to this rule is that we TODO all failing tests before
each point release. This is so 'make test' will succeed :)
Remember, the failing test *is* your bug report.
# vim: ft=text