Difference between revisions of "Testing MPICH"
(→Running the configure tests) |
(→Running the coding checks) |
||
| Line 225: | Line 225: | ||
An explanation of some of the rules that are applied by the style checker are | An explanation of some of the rules that are applied by the style checker are | ||
| − | in the [[ | + | in the [[Coding Standards]]. |
==== Finding Specific Classes of Problems ==== | ==== Finding Specific Classes of Problems ==== | ||
Revision as of 21:11, 25 July 2008
Contents
- 1 Testing MPICH2
- 1.1 Using the MPI test suites with MPICH2
- 1.2 Changing the default tests
- 1.3 Location of test output
- 1.4 Running the Special Tests
- 1.5 Running the configure tests
- 1.6 Running the coding checks
- 1.7 Running the global symbols checks
- 1.8 Running and understanding the coverage tests
- 1.9 Troubleshooting the tests
- 1.10 Sources for the Test Suites
- 1.11 Running the LLNL I/O Test
Testing MPICH2
Using the MPI test suites with MPICH2
Basic tests can be performed using the MPICH2 tests
make testing
The results are in test/summary.xml. The XML style file for these results is in test/TestResults.xsl.
To run the full set of tests and add the results to the
[[http://www-unix.mcs.anl.gov/mpi/mpich1/micronotes/mpich2-status/index.htm%7Ctest
results web page]], simply run
/home/MPI/nightly/basictests .
This will not update the web page however. Run
/home/MPI/nightly/updatesum to update the test results web page.
To customize the tests run by basictests, the following
command-line options may be used:
- -tables=pm-device
mpd-ch3 rmpd-ch3 mpd-ch3:ssm mpd-ch3:shm gforker-ch3 random
- -cc=name, -cxx=name, -fc=name, -f90=name
Specifies different compilers for C, C++, Fortran 77, and Fortran 90
- -arch=name
Sets the "architecture name". This option should be used to ensure that the results are given a unique output name. For example
-arch=Linuxgcc31
for IA32 Linux using gcc version 3.1 . The file name for the results is made up of the arch, process manager, device, and date. This option is particularly important because the columns of the web page are determined by this option.
- -soft=name
Add the "soft" option. For example, to use the PGI compilers at MCS, you need to execute the command
soft add +pgi
This option causes basictests to perform this step
- -mpich2dir=path
Sets an alternate location for the source of MPICH2 (the default is /home/MPI/testing/mpich2/mpich2).
- -datadir=path
Sets an alternate location for the output files. The default is /home/MPI/nightly/runs.
- -tests=list
Selects the tests to run. The default is to run all four test sets and is
equivalent to specifying -tests=mpich:mpicxx:intel:mpich2.
For example, to run the tests on your own copy of MPICH2, storing the data in a private directory, for just the mpd process manager and ch3 device, use
/home/MPI/nightly/basictests -mpich2dir=/home/me/mympich2 \
-datadir=/home/me/myresults \
-tables=mpd
To do: add information on using all five test suites, including the MPI I/O test suite.
Example uses of basictests
- Basic use
Run the tests and let basictests choose the compilers and the test name (used to label a column on the web page)
/home/MPI/nightly/basictests
- Alternate Compilers
This example shows how to select the Portland Group compilers.
Note the use of the -soft option to ensure that the
correct paths are set up for using Portland Group compilers and the
use of the -arch option to give this test a unique name
/home/MPI/nightly/basictests -soft=pgi-5.2 -cc=pgcc -fc=pgf77 -cxx=pgCC -f90=pgf90 -arch=IA32-Linux-PG
Changing the default tests
The tests that are run by default are choosen in the script /home/MPI/nightly/basictests. To add a new architecture to the tests, simply run basictests on a new platform; the results will be automatically added to the web page [[1]].
To add a new device and/or process manager, you need to edit two files. In /home/MPI/nightly/basictests, find the line
tables="mpd-ch3 mpd-ch3:ssm mpd-ch3:shm mpd-ch3:sshm gforker-ch3 random"
Add the process manager and device pair that you want to test, following the form processmanager-device. For a device that has additional options, add these with a colon after the device name. For example, to add testing with the gforker process manager for the shared memory version of the ch3 device, add gforker-ch3:shm to tables:
tables="mpd-ch3 mpd-ch3:ssm mpd-ch3:shm mpd-ch3:sshm gforker-ch3 gforker-ch3:shm random"
You must also update /home/MPI/nightly/updatesum to add
the new test to the web page. Add the same entry to the
@tables array in updatesum. As this is a
perl script, the syntax is slightly different. Continuing with the
example above, we add "gforker-ch3:shm" to @tables:
@tables = ( "mpd-ch3",
"mpd-ch3:ssm", "mpd-ch3:shm", "mpd-ch3:sshm",
"gforker-ch3",
"gforker-ch3:shm",
"random" );
That's it. The next time basictests runs, the new process manager and device pair will be added to the tests, and the web page will reflect the new tests.
Location of test output
The output of the tests run by basictests is copied into the directory /home/MPI/nightly/runs in a file with a name that contains the architecture, process manager, device, date, and test name. For example, the files for IA32 under Linux, with the GNU compilers, using the MPD process manager and the ch3 device, and started on April 1st, 2005 at 2200 hours, are
IA32-Linux-GNU-mpd-ch3-2005-04-01-22-00.xml IA32-Linux-GNU-mpd-ch3-make-2005-04-01-22-00.htm IA32-Linux-GNU-mpd-ch3-2005-04-01-22-00-testsumm.xml IA32-Linux-GNU-mpd-ch3-2005-04-01-22-00-testsumm-fail.xml IA32-Linux-GNU-mpd-ch3-2005-04-01-22-00-testsumm-cxx.xml IA32-Linux-GNU-mpd-ch3-2005-04-01-22-00-testsumm-cxx-fail.xml IA32-Linux-GNU-mpd-ch3-2005-04-01-22-00-testsumm-intel.xml IA32-Linux-GNU-mpd-ch3-2005-04-01-22-00-testsumm-intel-fail.xml IA32-Linux-GNU-mpd-ch3-2005-04-01-22-00-testsumm-mpich2.xml IA32-Linux-GNU-mpd-ch3-2005-04-01-22-00-testsumm-mpich2-fail.xml
The first of these is the entire output file. The second contains
only the output of make that was might contain warnings or errors
about the make. The remaining eight files contain the test results
and the failure-only results (the files with the -fail
extension).
If more details are required, you will need to check the directory in which the tests were run. By default (see basictests), this is in /sandbox/USER/cb, where USER is the name of the person that ran the tests.
Running the Special Tests
The [| Special tests] are run
with the script /home/MPI/nightly/specialtest -dir=rundir, where
rundir is the directory in which specialtest writes
the log files. The -help option to specialtest
describes the options that can be used to run a subset of tests or to select
different combinations of process managers or devices.
For example, to run the tests on your own copy of MPICH2 and place the results in a private location, use
/home/MPI/nightly/specialtest -srcdir=/home/me/mympich2 \
-dir=/sandbox/me \
-webdir=/home/me/mytests
The results can then be viewed by opening /home/me/mytests/index.htm in a web browser. <p> To update the special tests output on the general web page, simply run
/home/MPI/nightly/specialtest
This will use the MPICH2 source in /home/MPI/testing/mpich2/mpich2.
Always run this test in a separate directory; do not run it in /home/MPI/nightly.
Running the configure tests
The configure tests are run with the script /home/MPI/nightly/testoptions.
To update the special tests output on the general web page, simply run
/home/MPI/nightly/testoptions
This will use the MPICH2 source in /home/MPI/testing/mpich2/mpich2.
Running the coding checks
To run the coding checks, change to a directory that contains MPICH2 and execute the following command:
/home/gropp/projects/software/buildsys/src/codingcheck -conf=mpich2 \
-skipfiles=src/mpid/globus,src/mpe2,src/mpid/rdma \
-checktest src examples test
(This assumes that you are running on one of the MCS division Linux systems; you can install the buildsys tools and using codingcheck from your own installation if you need to run the coding check on a different system). The report of problems detected is sent to standard output. The above command will produce a relatively comprehensive report. See the use of the codingcheck command in /home/MPI/nightly/updatetodo for the current command use to update the web page; this command may skip additional directories and may use additional options (such as -rmchecks=cppdefines) to tune the output.
An updated report is generated every morning as one of the steps performed by /home/MPI/nightly/updatetodo and placed into /mcs/www/mpi/mpi/mpich2/todo/coding-problems.txt (accessible as [[2]]).
An explanation of some of the rules that are applied by the style checker are in the Coding Standards.
Finding Specific Classes of Problems
Currently, the coding checker produces a unified report about all
problems. To find specific problems, you can use grep or
view the report (it is a text file) in your favorite editor. Some
simple examples are:
- Find errors
These are uses that are viewed as erroneous. In many cases, these are uses of either MPI or PMPI routines where NMPI names must be used (NMPI routines ensure that errors are properly handled).
grep 'Error:' /mcs/www/mpi/mpich2/todo/coding-problems.txt
- Find non-conforming ifdef names
These find C preprocessor names that may conflict with other system header files
grep Warning: /mcs/www/mpi/mpich2/todo/coding-problems.txt | egrep 'ifn?def'
- Find missing style headers
Each file should have a style header to avoid unnecessary reformatting when the file is edited (e.g., to maintain a consistent level of indentation).
grep 'style header missing' /mcs/www/mpi/mpich2/todo/coding-problems.txt
- Find suspicious function usage
There are a number of functions, such as strcat, that
should never appear in good code because of the potential for memory
overwrites (even if the code is surrounded by length checks, any
reader of the code must take the time to ensure that those checks are
correct; it is better to avoid them). Other routines, such as
malloc, interfere with debugging tools that are built
into the code. Yet other functions, such as setvbuf or
snprintf, are either not portable to all operating
systems or are only part of C99 or later, and may pose portability
problems for users with older (e.g., C90) compilers.
grep 'Warning: found ' /mcs/www/mpi/mpich2/todo/coding-problems.txt grep 'Caution: found ' /mcs/www/mpi/mpich2/todo/coding-problems.txt
- Find missing Copyright statements
To get a quick list of files that are missing a copyright statement:
grep 'Copyright statement missing' /mcs/www/mpi/mpich2/todo/coding-problems.txt
- Find files that use a particular function
For example, to find all files that use malloc
grep malloc /mcs/www/mpi/mpich2/todo/coding-problems.txt | grep -v Warning:
Running the global symbols checks
To update the web page that reports on the global symbols, run
/home/MPI/nightly/checkglobs
Like the other tests, this works with the source files in /home/MPI/testing/mpich2/mpich2. The program itself is a simple perl script; if you wish to change the tests for yourself, make a copy and update the values of the following variables, found near the top of the file:
- $mpich2src
Location of MPICH2 source
- $builddir
Directory to be used to build MPICH2 as part of identifying global symbols
- $reportdir
Direction into which the result web page is placed
- $checksym
Location of program that is used to check for global symbols, along with any command-line arguments
- $includeLogFiles
Set to 1 if the configure and make log files should be copied to $reportdir
There are several easy ways to get a quick look at the global
symbols. The easiest is to use the nm program on
libmpich.a. However, you will need to filter the output
to find any non-conforming names. For example, under Linux,
nm libmpich.a | grep " C " | grep -v mpipriv
will show the names of any uninitialized global variables.
Finding non-conforming names is harder, primarily because the list of
allowed prefixes is fairly large. Instead of trying to list all
prefixes and using grep -v, you can use
/homes/gropp/projects/software/buildsys/src/checkforglobs -mpich2 libmpich.a
When testing for global symbols, make sure that you include tests with weak symbols disabled to ensure that any internal routines that are static only when weak symbols are supported have conforming names.
Running and understanding the coverage tests
The easiest way to run the coverage tests is with the script
/home/MPI/nightly/getcoverage. This uses the version of
MPICH2 that is updated every night in
/home/MPI/testing/mpich2/mpich2, and uses the MPICH2
test suite and the three additional test suites that are in
/home/MPI/testing/tsuites. Run this script with
/home/MPI/nightly/getcoverage -updateweb
The best way to view the coverage data for the entire project is through the <a href="http://www-unix.mcs.anl.gov/mpi/mpich2/todo/coverage/index.htm">web page</a>. Code with a blue background has been ignored in counting covered and uncovered lines; this is typically error handling and reporting code or experimental code that is not expected to be executed. Code with a red background is code that should have been executed by the tests but was not (in some cases, this code was not marked as error handling or reporting code; such code should be fixed by annotating the source code to mark the code as error handling).
To update the coverage information for a single file, the easiest
approach is to use /home/MPI/nightly/getcoverage to
create the correct versions of the libraries and get the initial
coverage results. To add to the results, simply compile a program
with the coverage-enabled libraries (by default, this is in
/sandbox/$LOGNAME/mpich2-cov) and run it. This will
cause the coverage data files (files with extensions da) to
be updated. To generate a text file describing the coverage, you can
normally use
gcov foo.c
in the directory that contains the file foo.da (you'll
also see the files foo.bb and foo.bbg).
This will produce a file foo.c.gcov. Lines that are
marked with ##### have not be executed (see man
gcov for more information on the gcov tool).
When running the /home/MPI/nightly/getcoverage tool from a cron
job, the following options are useful:
-
-quiet- Suppresses all output -
-logfile=name- Sends all output to the named file.
If -updateweb is also selected, then the -logfile
option also copies the contents of the logfile into logfile.txt
in the designated web directory.
Troubleshooting the tests
Stopping the tests
Several of the test suites allow you to stop them by creating a file. When
the test suite detects the existance of the file, the tests are aborted. A
good way to create these files is with date so that it is easy to
see when the file was originally created.
- mpich
$HOME/.stopmpichtests
- mpich2
.stoptest in the top-level testing directory (e.g.,
mpich2/test/mpi
Sources for the Test Suites
The test suites should be accessible through the <a href="http://www.mcs.anl.gov/mpi/mpi-test/tsuite.html">test suite web page</a>. Local to Argonne, the test suites are available from the CVS repository:
| Test Suite | CVS Repository | CVS Module |
|---|---|---|
| Intel | /home/MPI/cvsMaster | IntelMPITEST |
| C++ | /home/MPI/cvsMaster | mpicxxtest |
| LLNL I/O | /home/MPI/cvsMaster | Testmpio |
| MPICH | /home/MPI/cvsMaster | mpich/examples/test |
| MPICH2 | /home/MPI/cvsMaster | mpich2-01/test/mpi |
Running the LLNL I/O Test
To run the LLNL I/O test, do the following:
- Get a current version. For example,
cvs -d /home/MPI/cvsMaster export -D now Testmpio
- Update the
Makefile. For MPICH2, all you should need to
do is to reset the value of the variable MPIHOME.
- Make a
t1subdirectory in the test directory:
mkdir t1
Without this step, the tests will fail with "invalid file name" messages.
- Run the test with
make testing. If you have trouble with
this (for example, it fails if you direct the output into a file), run the test manually as follows (assuming a C-shell):
setenv MPIO_USER_PATH `pwd`/t1
mpiexec -n 4 testmpio 1 |& tee test.log
- Interpret the results. This test does not provide a pass/fail summary,
