Difference between revisions of "Creating a Release"

From Mpich
Jump to: navigation, search
(Update MPICH website)
(Update MPICH website)
 
(51 intermediate revisions by 7 users not shown)
Line 4: Line 4:
  
 
=== Check the Release Status ===
 
=== Check the Release Status ===
See the page https://trac.mpich.org/projects/mpich/roadmap for the relevant release to ensure that all pending bugs have been fixed.
+
See the page https://github.com/pmodels/mpich/milestones for the relevant release to ensure that all pending bugs have been fixed.
  
 
=== Ensure that all testing has been performed ===
 
=== Ensure that all testing has been performed ===
Line 15: Line 15:
 
# The special tests (See [[http://wiki.mpich.org/mpich/index.php/Testing_MPICH#Running_the_Special_Tests]] ); these are important for testing that various options and compiler choices work
 
# The special tests (See [[http://wiki.mpich.org/mpich/index.php/Testing_MPICH#Running_the_Special_Tests]] ); these are important for testing that various options and compiler choices work
 
# The "random" tests in the nightly test should not indicate any problems; these are used to test correct handling of the configure options
 
# The "random" tests in the nightly test should not indicate any problems; these are used to test correct handling of the configure options
# Ensure that the Debugger interface works; <code>(cd src/mpi/debugger ; make tvtest)</code> can be used as a start; a check with <code>totalview</code> should also be performed
+
# Ensure that the Debugger interface works; <code>(configure mpich with --enable-debuginfo; make src/mpi/debugger/tvtest)</code> can be used as a start; a check with <code>totalview</code> should also be performed (e.g.: "totalview mpiexec -a -n 4 src/mpi/debugger/tvtest" and make sure that message queue status works fine)
 +
# Ensure that <code>make dist</code> works for the MPI testsuite.  That is, make sure <code>cd test/mpi ; make dist</code> produces a tar file that can be unpacked and the test suite built against an installed MPI (not necessarily MPICH, but that is usually sufficient for this test).
 
# Ensure that there are no performance artifacts, both for point-to-point and collective operations.
 
# Ensure that there are no performance artifacts, both for point-to-point and collective operations.
  
 
=== Update all necessary files and commit them to the trunk ===
 
=== Update all necessary files and commit them to the trunk ===
  
* '''Update the version numbers (including <code>current:revision:age</code> ABI) in <code>maint/version.m4</code>'''. Note: don't update the release date, as it will be automatically replaced during tarball generation.
+
* '''Update the version number in <code>maint/version.m4</code>'''. Note: don't update the release date, as it will be automatically replaced during tarball generation.
* Update the <code>CHANGES</code> file. You can find the commits that went in by going through the svn log information.
+
* '''[STABLE RELEASE ONLY]''' Update the ABI '''<code>current:revision:age</code>''' string in '''<code>maint/version.m4</code>''' Pre-releases have an ABI string of 0:0:0 to avoid getting accidentally linked in place of stable libraries.
 +
* Update the <code>CHANGES</code> file. You can find the commits that went in by going through the git log information.
 +
** Before committing <code>CHANGES</code>, it is recommended to send <code>CHANGES</code>, at least, to core@mpich.org in order to make sure everything is clear.
 
* Update the <code>RELEASE_NOTES</code> file. This requires input from everyone, and generally requires asking each person (in person) if the current restrictions still apply, and if any new ones should be added.
 
* Update the <code>RELEASE_NOTES</code> file. This requires input from everyone, and generally requires asking each person (in person) if the current restrictions still apply, and if any new ones should be added.
* Update the <code>README</code> if necessary.
+
* Update the <code>README.vin</code> if necessary.
 +
 
 +
=== Create a final tarball for the release using the release.pl script ===
 +
* Before using release.pl, make sure these tools are ready: latex, [http://web.engr.illinois.edu/~wgropp/projects/software/sowing/ doctext] and [http://mvertes.free.fr/download/ txt2man] (from [http://freecode.com/projects/txt2man freecode])
 +
** txt2man can be installed using package manager. For example,
 +
<pre>
 +
  apt-get install txt2man (on Ubuntu)
 +
  brew install txt2man (on Mac)
 +
</pre>
 +
 
 +
* Get the <code>release.pl</code> script in <code>maint</code> from the trunk and run it as
 +
<pre>
 +
./release.pl --branch=[git_branch_to_use]  --version=[version] --git-repo=[path_to_git_repository]
 +
</pre>
 +
 +
E.g., 
 +
<pre>
 +
./release.pl --branch=master --version=3.1.2 --git-repo=https://github.com/pmodels/mpich
 +
</pre>
 +
 
 +
Notes:
 +
* The release.pl requires specific autotools to run. Even with a newer version, you may not be able to run the script. The script did it on purpose to test it maintain the correctness of ABI. You will need the exact version of autotools to build and release the tarball.
 +
* While we use tags are in the format "vX.Y.Z", the version format should not have the leading "v", so it should be "X.Y.Z".
  
 
=== Create a release tag ===
 
=== Create a release tag ===
Line 40: Line 65:
 
* '''It is strongly recommended''' that you create the release tarball locally and completely test it before pushing the tag to the origin repository.
 
* '''It is strongly recommended''' that you create the release tarball locally and completely test it before pushing the tag to the origin repository.
 
   <code> make testing </code>
 
   <code> make testing </code>
* Once the candidate tarball has been thoroughly tested and inspected, push the tag to origin:  
+
* Once the candidate tarball has been thoroughly tested and inspected, push the tag to the mpich and hydra repositories:  
   <code> git push origin tag vX.Y.Z </code>
+
   <code> git push mpich tag vX.Y.Z </code>
  
 
* Long term release branches are still discouraged at this time, though they can easily be created after the fact as long as the tag exists.  One easy way to do this in git is: <code>git push origin vX.Y.Z:refs/heads/maint-vX.Y.Z</code>, which will create a new branch on the remote "origin" named "maint-vX.Y.Z" which starts at the same commit as the tag "vX.Y.Z".  Local branches can easily be created to track this remote branch in the usual way: <code>git branch maint-vX.Y.Z --track origin/maint-vX.Y.Z ; git checkout maint-vX.Y.Z</code> (or the equivalent shorthand: <code>git checkout maint-vX.Y.Z</code>, as long as that branch name only exists in exactly one remote).
 
* Long term release branches are still discouraged at this time, though they can easily be created after the fact as long as the tag exists.  One easy way to do this in git is: <code>git push origin vX.Y.Z:refs/heads/maint-vX.Y.Z</code>, which will create a new branch on the remote "origin" named "maint-vX.Y.Z" which starts at the same commit as the tag "vX.Y.Z".  Local branches can easily be created to track this remote branch in the usual way: <code>git branch maint-vX.Y.Z --track origin/maint-vX.Y.Z ; git checkout maint-vX.Y.Z</code> (or the equivalent shorthand: <code>git checkout maint-vX.Y.Z</code>, as long as that branch name only exists in exactly one remote).
 
Files corresponding to the tags are accessible through the web URL http://git.mpich.org/mpich.git/blob/vX.Y.Z:/<filename>
 
 
=== Create a final tarball for the release ===
 
 
* Get the <code>release.pl</code> script in <code>maint</code> from the trunk and run it as
 
  ./release.pl --branch=[git_branch_to_use] --pbranch=[git_branch_for_previous_version] --version=[version] --remote-git-repo=[path_to_git_repository]
 
 
Notes:
 
* While we use tags are in the format "vX.Y.Z", the version format should not have the leading "v", so it should be "X.Y.Z".
 
* [path_to_git_repository] should point to your local copy of the remote origin/master repository.
 
  
 
=== Update MPICH website ===
 
=== Update MPICH website ===
Line 60: Line 74:
 
* Create directory for tarballs
 
* Create directory for tarballs
 
<pre>
 
<pre>
cd /mcs/mpich/tarballs
+
  cd /mcs/mpich/tarballs
mkdir X.Y.Z
+
  mkdir X.Y.Z
 +
</pre>
 +
* Copy MPICH and Hydra tarballs into new directory, e.g.,
 +
<pre>
 +
  cp mpich-3.1.2.tar.gz /mcs/mpich/tarballs/3.1.2
 +
  cp hydra-3.1.2.tar.gz /mcs/mpich/tarballs/3.1.2
 +
</pre>
 +
* Copy a shortlog containing changes since the last stable release. Shortlog can be generated with, e.g.,
 +
<pre>
 +
git log --no-merges --format="format:[%cd] %s" --date="format:%Y-%m-%d" v3.4.2..v4.0a2 > shortlog
 +
</pre>
 +
* For full release update documentation as well: README.txt, installguide.pdf, userguide.pdf  (with <code>mpich-[version]-</code> prefix)
 +
<pre>
 +
  mkdir ~/sandbox-3.1.2 && cd ~/sandbox-3.1.2
 +
  tar zxvf /mcs/mpich/tarballs/3.1.2/mpich-3.1.2.tar.gz
 +
  cp ./mpich-3.1.2/README  /mcs/mpich/tarballs/3.1.2/mpich-3.1.2-README.txt
 +
  cp ./mpich-3.1.2/doc/installguide/install.pdf  /mcs/mpich/tarballs/3.1.2/mpich-3.1.2-installguide.pdf
 +
  cp ./mpich-3.1.2/doc/userguide/user.pdf  /mcs/mpich/tarballs/3.1.2/mpich-3.1.2-userguide.pdf
 
</pre>
 
</pre>
* Copy MPICH and Hydra tarballs into new directory
 
 
* Update the downloads svn repository
 
* Update the downloads svn repository
 
<pre>
 
<pre>
svn add X.Y.Z
+
  cd /mcs/mpich/tarballs/
svn commit
+
  svn add X.Y.Z
 +
  svn commit
 +
  chmod -R g+w X.Y.Z
 
</pre>
 
</pre>
 
* Update website to reflect new release: downloads page, main page, and new page: https://www.mpich.org/wp-admin/
 
* Update website to reflect new release: downloads page, main page, and new page: https://www.mpich.org/wp-admin/
** For full release update documentation as well: README.txt, installguide.pdf, userguide.pdf and windevguide.pdf (with <code>mpich-[version]-</code> prefix)
+
** Create a new announcement post (in Left Box, News & Events).
** Also for full release - update the online Manpages. We keep multiple versions in /mcs/mpich/docs. Change the "latest" symlink to point to the new version and update https://www.mpich.org/documentation/manpages/
+
** Change the version number in "Settings -> MPICH Release (at bottom of the left)"
 +
* Also for full release - update the online Manpages. Change the "latest" symlink to point to the new version.
 +
<pre>
 +
  cp -rf ~/sandbox-X.Y.Z/mpich-X.Y.Z/www /mcs/mpich/docs/vX.Y.Z
 +
  cd /mcs/mpich/docs
 +
  rm -f latest
 +
  ln -sf vX.Y.Z latest
 +
  chgrp -h mpi latest
 +
</pre>
 +
* Update links at https://www.mpich.org/documentation/manpages/
  
=== Mark the milestone complete on TRAC ===
+
=== Mark the milestone complete on GitHub ===
  
* If there are any pending tickets, move them to the next release.
+
* If there are any pending issues, move them to the next release.
  
 
=== Update any package managers we're responsible for ===
 
=== Update any package managers we're responsible for ===
Line 95: Line 136:
  
 
=== Send out the release announcement to announce@mpich.org and party! Yay! ===
 
=== Send out the release announcement to announce@mpich.org and party! Yay! ===
 
The older version of these release instructions can be found here: http://www.mcs.anl.gov/research/projects/mpich2/olddeveloper/release.htm
 
 
 
<!-- vim: set ft=wikipedia : -->
 
<!-- vim: set ft=wikipedia : -->

Latest revision as of 14:10, 11 June 2021


Creating the release has the following steps:

Check the Release Status

See the page https://github.com/pmodels/mpich/milestones for the relevant release to ensure that all pending bugs have been fixed.

Ensure that all testing has been performed

This includes both the results of the nightly tests and the special tests that are used to check that common configuration options continue to work and that the coding standards have been followed.

TODO: There was a comprehensive checklist of the prerequisites for a release. That needs to be placed here. Recent releases have not checked this list. What follows is a partial list

Check all tests. This includes

  1. The four major test suites (MPICH, C++, Intel, MPICH2)
  2. The special tests (See [[1]] ); these are important for testing that various options and compiler choices work
  3. The "random" tests in the nightly test should not indicate any problems; these are used to test correct handling of the configure options
  4. Ensure that the Debugger interface works; (configure mpich with --enable-debuginfo; make src/mpi/debugger/tvtest) can be used as a start; a check with totalview should also be performed (e.g.: "totalview mpiexec -a -n 4 src/mpi/debugger/tvtest" and make sure that message queue status works fine)
  5. Ensure that make dist works for the MPI testsuite. That is, make sure cd test/mpi ; make dist produces a tar file that can be unpacked and the test suite built against an installed MPI (not necessarily MPICH, but that is usually sufficient for this test).
  6. Ensure that there are no performance artifacts, both for point-to-point and collective operations.

Update all necessary files and commit them to the trunk

  • Update the version number in maint/version.m4. Note: don't update the release date, as it will be automatically replaced during tarball generation.
  • [STABLE RELEASE ONLY] Update the ABI current:revision:age string in maint/version.m4 Pre-releases have an ABI string of 0:0:0 to avoid getting accidentally linked in place of stable libraries.
  • Update the CHANGES file. You can find the commits that went in by going through the git log information.
    • Before committing CHANGES, it is recommended to send CHANGES, at least, to core@mpich.org in order to make sure everything is clear.
  • Update the RELEASE_NOTES file. This requires input from everyone, and generally requires asking each person (in person) if the current restrictions still apply, and if any new ones should be added.
  • Update the README.vin if necessary.

Create a final tarball for the release using the release.pl script

  • Before using release.pl, make sure these tools are ready: latex, doctext and txt2man (from freecode)
    • txt2man can be installed using package manager. For example,
   apt-get install txt2man (on Ubuntu)
   brew install txt2man (on Mac)
  • Get the release.pl script in maint from the trunk and run it as
 ./release.pl --branch=[git_branch_to_use]  --version=[version] --git-repo=[path_to_git_repository]

E.g.,

 ./release.pl --branch=master --version=3.1.2 --git-repo=https://github.com/pmodels/mpich

Notes:

  • The release.pl requires specific autotools to run. Even with a newer version, you may not be able to run the script. The script did it on purpose to test it maintain the correctness of ABI. You will need the exact version of autotools to build and release the tarball.
  • While we use tags are in the format "vX.Y.Z", the version format should not have the leading "v", so it should be "X.Y.Z".

Create a release tag

(NOTE: below are the notes from when we managed our source in SVN and the project was still called "MPICH2", though they never 100% accurately reflected reality. Since the switch to Git, we haven't been following this model quite as closely. Also, some of the instructions below don't make as much sense in Git. These instructions are helpful history, but not how we do things now.)

  • A branch is only created once for a complete release cycle, i.e., 1.0.7rc1, 1.0.7rc2, 1.0.7, 1.0.7p1, etc., will have just one branch. At what point a release branch is to be created is not decided yet (e.g., alpha releases, beta releases, RCs, full release), but we currently do that at the time of the RC release. This essentially means an svn-copy of the trunk to the branches/release directory.
  • If this is the first RC for this release cycle, create a branch. Branches are named as mpich-X.Z.Y, where X, Y, Z are the major, minor and revision version numbers.
  • If this is not the first RC, merge all required changes from the trunk to the release branch.
  • A tag is created for every release from the branch. Tag is essentially an svn-copy of the branch to the tags/release directory. Tags are named as mpich-X.Y.Z[ext], where [ext] is a1/a2/.. (for alpha releases), rc1/rc2/.. (for RC releases), etc.

This is how we manage release tags and branches under git:

  • tag the release version in your local git repository:
  git tag -a -m "tagging 'vX.Y.Z'" vX.Y.Z [COMMIT_HASH] ([COMMIT_HASH] defaults to HEAD if omitted)
  • It is strongly recommended that you create the release tarball locally and completely test it before pushing the tag to the origin repository.
  make testing 
  • Once the candidate tarball has been thoroughly tested and inspected, push the tag to the mpich and hydra repositories:
  git push mpich tag vX.Y.Z 
  • Long term release branches are still discouraged at this time, though they can easily be created after the fact as long as the tag exists. One easy way to do this in git is: git push origin vX.Y.Z:refs/heads/maint-vX.Y.Z, which will create a new branch on the remote "origin" named "maint-vX.Y.Z" which starts at the same commit as the tag "vX.Y.Z". Local branches can easily be created to track this remote branch in the usual way: git branch maint-vX.Y.Z --track origin/maint-vX.Y.Z ; git checkout maint-vX.Y.Z (or the equivalent shorthand: git checkout maint-vX.Y.Z, as long as that branch name only exists in exactly one remote).

Update MPICH website

  • Create directory for tarballs
  cd /mcs/mpich/tarballs
  mkdir X.Y.Z
  • Copy MPICH and Hydra tarballs into new directory, e.g.,
  cp mpich-3.1.2.tar.gz /mcs/mpich/tarballs/3.1.2
  cp hydra-3.1.2.tar.gz /mcs/mpich/tarballs/3.1.2
  • Copy a shortlog containing changes since the last stable release. Shortlog can be generated with, e.g.,
git log --no-merges --format="format:[%cd] %s" --date="format:%Y-%m-%d" v3.4.2..v4.0a2 > shortlog
  • For full release update documentation as well: README.txt, installguide.pdf, userguide.pdf (with mpich-[version]- prefix)
  mkdir ~/sandbox-3.1.2 && cd ~/sandbox-3.1.2
  tar zxvf /mcs/mpich/tarballs/3.1.2/mpich-3.1.2.tar.gz
  cp ./mpich-3.1.2/README  /mcs/mpich/tarballs/3.1.2/mpich-3.1.2-README.txt
  cp ./mpich-3.1.2/doc/installguide/install.pdf  /mcs/mpich/tarballs/3.1.2/mpich-3.1.2-installguide.pdf
  cp ./mpich-3.1.2/doc/userguide/user.pdf  /mcs/mpich/tarballs/3.1.2/mpich-3.1.2-userguide.pdf
  • Update the downloads svn repository
  cd /mcs/mpich/tarballs/
  svn add X.Y.Z
  svn commit
  chmod -R g+w X.Y.Z
  • Update website to reflect new release: downloads page, main page, and new page: https://www.mpich.org/wp-admin/
    • Create a new announcement post (in Left Box, News & Events).
    • Change the version number in "Settings -> MPICH Release (at bottom of the left)"
  • Also for full release - update the online Manpages. Change the "latest" symlink to point to the new version.
  cp -rf ~/sandbox-X.Y.Z/mpich-X.Y.Z/www /mcs/mpich/docs/vX.Y.Z
  cd /mcs/mpich/docs
  rm -f latest
  ln -sf vX.Y.Z latest
  chgrp -h mpi latest

Mark the milestone complete on GitHub

  • If there are any pending issues, move them to the next release.

Update any package managers we're responsible for

At the moment, this is just Homebrew. To do this, you need to:

  • Fork the original on github to get your own version
    • Yes, this has to be done on GitHub itself
  • Make a branch to make your changes to the MPICH
    • Doing this in master will only bring pain. Don't do it.
  • Make the changes to Library/Formula/mpich2.rb
    • At the moment, this formula is still named mpich2 and can't be changed.
    • There is however, an alias to mpich so people can just say that they want to install that.
  • Unless there have been any major changes to the way people should build MPICH, all this requires is changing the URL for the formula to whatever the new URL is.
    • If there have been new changes to the way people do the usual configure/make/make install setup, the rest of the script may need to change.
    • If there are new options that we should include in the package manager versions (don't read "add all new options here"), then they can be added.
  • Push your changes to your own fork on GitHub
  • Send a pull-request to the original mxcl repository

Send out the release announcement to announce@mpich.org and party! Yay!