This document explains processes for maintaners of the JSR 308 (type annotations) tools.
Contents:
jsr308-langtools and basic-jsr308-langtools are the same, if one removes all edits from mernst, mali, mahmood, notnoop. That means that they should be updated in the same way. I presume one should update basic-jsr308-langtools from OpenJDK, then jsr308-langtools from basic-jsr308-langtools.
# First, make sure Sun has not broken the OpenJDK version # (Sun's repository is at http://hg.openjdk.java.net/jdk7/jdk7/langtools/ ). cd ~/research/types/openjdk-langtools hg fetch cd make; unset jsr308_imports; ant -e -Dboot.java.home=$HOME/java/jdk -Dtarget.java.home=$HOME/java/jdk -Djtreg.home=$HOME/research/types/jtreg clean build-all-tools jtreg # Next, make sure that we have not broken our version: cd ~/research/types/basic-jsr308-langtools hg fetch cd make; unset jsr308_imports; ant -e -Dboot.java.home=$HOME/java/jdk -Dtarget.java.home=$HOME/java/jdk -Djtreg.home=$HOME/research/types/jtreg clean build-all-tools jtreg # Incorporate OpenJDK changes into basic. # There are tags like "jdk7-b108" that I should take advantage of rather # than just blind "hg fetch" to whatever is current. cd ~/research/types/basic-jsr308-langtools ## Alternately: hg fetch http://hg.openjdk.java.net/jdk7/tl/langtools/ ## which gets patches faster (jdk7/jdk7 version is synced every 3 weeks), ## which can be good or bad. hg fetch http://hg.openjdk.java.net/jdk7/jdk7/langtools/ cd make; unset jsr308_imports; ant -e -Dboot.java.home=$HOME/java/jdk -Dtarget.java.home=$HOME/java/jdk -Djtreg.home=$HOME/research/types/jtreg clean build-all-tools jtreg # Incorporate basic changes into jsr308-langtools cd ~/research/types/jsr308-langtools hg fetch https://code.google.com/p/basic.jsr308-langtools/ cd make; unset jsr308_imports; ant -e -Dboot.java.home=$HOME/java/jdk -Dtarget.java.home=$HOME/java/jdk -Djtreg.home=$HOME/research/types/jtreg clean build-javac build-javap build-javadoc build-doclets javadoc-javac jtreg # Do not commit any changes to jsr308-langtools yet! (make -C $qn) && (make -C $pl/java check-nullness check-interning) && (make -C $inv check-nullness check-interning) # If the above passes, commit changes to jsr308-langtools.
In addition, run
diff -u jsr308-langtools/dist/bin/javac checker-framework/checkers/binary/javac
and incorporate any recent Oracle javac changes into the Checker Framework's copy.
This section explains how to make a public release of the JSR 308 langtools and checkers.
You should first update jsr308-langtools from recent OpenJDK changes.
The release process is mostly automated, but you should expect to spend 30 to 60 minutes for each release you make. You will actually make the release twice — first to a testing site, and then to the live site.
[I'm not sure whether this is a good or a bad idea.] You may wish to make the release as a fresh user that has no customizations.
To make a release of the JSR 308 langtools and checkers, you need:
/cse/www2/types
directory on a cs.washington.edu machinehevea
and bibtex
)public_html
(for
testing the release)types_www
Unix groupYou should have all of the above if you use a Washington CSE machine
(e.g., jicama
or godwit
).
export anno=/scratch/$USER/jsr308-release export PATH=${PATH}:$anno/plume-lib/bin export PLUME_LIB=$anno/plume-lib export TEXINPUTS=.:/homes/gws/mernst/tex/sty:/homes/gws/mernst/tex:/uns/lib/hevea:..: export EDITOR=emacs export PERLLIB=${PERLLIB}:/homes/gws/mernst/bin/src/perl/lib/perl5/site_perl/5.10.0 export PERL5LIB=${PERL5LIB}:/homes/gws/mernst/bin/src/perl/lib/perl5/site_perl/5.10.0 # If dia and hevea are not on your path, add them, from either /uns or Michael Ernst's directory: export PATH=${PATH}:/homes/gws/mernst/bin/install/dia-0.97.1-install/bin:/homes/gws/mernst/bin/install/hevea-1.10/bin
Create a new directory. Place the directory in a non-NFS location such as a scratch partition, because a local disk is faster than NFS.
mkdir -p $anno cd $anno
Check out the jsr308-langtools and checker-framework repositories:
hg clone https://code.google.com/p/jsr308-langtools/ jsr308-langtools hg clone https://code.google.com/p/checker-framework/ checker-framework hg clone https://code.google.com/p/plume-lib/ plume-lib hg clone https://code.google.com/p/annotation-tools/ annotation-tools
Copy some files from the JSR 308 site as the basis for a testing site:
cd $anno/checker-framework/release ant -f release.xml site-copy
and answer the prompt accordingly. The site will be created in
~/www/jsr308test
by default.
[TODO: make it possible to be created elsewhere, for users with limited
disk space. Or, just link from that directory to elsewhere.]
[This is actually done by the Python script, so it is not strictly necessary. Does the Python script also check "hg outgoing"? Does it need to?] Ensure that your checkouts are up to date and you have no uncommitted changes. Each of these commands should succeed, printing "no changes found":
cd $anno/plume-lib && hg fetch && hg outgoing cd $anno/jsr308-langtools && hg fetch && hg outgoing cd $anno/annotation-tools && hg fetch && hg outgoing cd $anno/checker-framework && hg fetch && hg outgoingIf any "fetch" fails, you can see the differences with:
cd $anno/plume-lib && hg diff cd $anno/jsr308-langtools && hg diff cd $anno/annotation-tools && hg diff cd $anno/checker-framework && hg diffIf you want to revert the have to start over and have changed, then do:
cd $anno/jsr308-langtools && hg revert --all cd $anno/checker-framework && hg revert --alland then redo the above.
Ensure that all regression tests pass.
Run:
# Only necessary if you did not run the regression tests in this directory. cd $anno/jsr308-langtools/make ant # Depends on the JSR 308 javac. cd $anno/annotation-tools/annotation-file-utilities ant zipfile
TODO: merge this into the main script: in python or, better, in the Ant file.
Make a release of the Annotation File Utilities before doing the JSR308/Checker Framework release. This should ensure that the file date for the AFU links is correctly updated. After the JSR308/Checker Framework release, ensure that the AFU release dates are correct.
Edit the changelogs ($anno/jsr308-langtools/doc/changelog-jsr308.txt and $anno/checker-framework/checkers/changelog-checkers.txt) to indicate what has changed in this version. (Hopefully, this has already been done by each person who made a recent change.) The changelogs should focus on user-visible changes, or very significant implementation changes that may affect users.
Every release, update file $anno/jsr308-langtools/doc/changelog-jsr308.txt, the changelog for the JSR 308 (Type Annotations) reference implementation; the release scripts run make -C $anno/jsr308-langtools/doc web which publishes the implementation.
If there were changes in the specification, update its changelog in file jsr308-changes.html. To release the specification, run: make -C $anno/jsr308-langtools/doc web-spec
Note: The release script below auto-populate the changelog files with changeset description since last release, and prompts to clean up the info.
[TODO: Update the the instructions here to use tags instead]
To double-check that the changelogs are adequate, run the following commands, and read the files in ~/tmp that they create:
Determine the revision number for the previous release (TODO: we should tag releases instead) by examining the output of
(cd $anno/checker-framework; hg log .hgtags) (cd $anno/jsr308-langtools; hg log .hgtags)
and setting environment variables accordingly:
export CF_PREV_REVISION=78cb6185e063 export JL_PREV_REVISION=f448d3583dff
mkdir -p ~/tmp # Diff the manuals against the previous versions: cd $anno/checker-framework/checkers/manual hg diff -w -r$CF_PREV_REVISION . > ~/tmp/checkers-manual-diff cd $anno/jsr308-langtools hg diff -w -r$JL_PREV_REVISION README-jsr308.html > ~/tmp/langtools-manual-diff # Use the version control logs since the last release: # In Emacs, it's useful to do: (delete-matching-paragraphs "automated merge with " nil nil t) cd $anno/checker-framework hg log -v -r$CF_PREV_REVISION:tip > ~/tmp/checkers-log cd $anno/jsr308-langtools hg log -r$JL_PREV_REVISION:tip > ~/tmp/langtools-log hg log -r$JL_PREV_REVISION:tip --style changelog > ~/tmp/langtools-log-with-filenames
Execute:
cd $anno/checker-framework/release python -m release
and follow the instructions it provides.
If you are running on a system without access to the /cse/www2/types directory, then when the sanity check fails, you can do:
(cd $HOME/www; scp -r jsr308test mernst@godwit.cs.washington.edu:./www/)
(which puts the Checker Framework distribution at http://homes.cs.washington.edu/~mernst/jsr308test/checker-framework/current/checkers-manual.html), and then run a second time with
python -m release -Dsanitycheck.dry.url=http://homes.cs.washington.edu/~mernst/jsr308test/checker-framework/releases/1.1.1
(It would be possible to just run the sanitycheck target directly:
ant -f release.xml -Dsanitycheck.dry.url=http://homes.cs.washington.edu/~mernst/jsr308test/checker-framework/releases/1.1.1 -Drelease.ver=1.1.1 sanitycheck
but that fails to check for broken links.)
Update the Checker Plugin for Eclipse.
The basic instructions are contained in the README file. The project is hosted at eclipselabs.org.
These instructions are deprecated in favor of the python release script. The material will be deleted in due time. But, for now there is information here that is not captured in the python release script.
export anno=$HOME/research/types export CFVERSION=1.0.3
export anno=$HOME/research/types export CFVERSION=1.0.3
Ensure that all regression tests pass.
Edit the changelogs ($anno/jsr308-langtools/doc/changelog-jsr308.txt and $anno/checker-framework/checkers/changelog-checkers.txt) to indicate what has changed in this version. (Hopefully, this has already been done by each person who made a recent change.) The changelogs should focus on user-visible changes, or very significant implementation changes that may affect users.
To double-check that the changelogs are adequate, run the following commands, and read the files in ~/tmp that they create:
Determine the revision number for the previous release (TODO: we should tag releases instead) by examining the output of
(cd $anno/checker-framework; hg log checkers/changelog-checkers.txt) (cd $anno/jsr308-langtools; hg log /doc/changelog-jsr308.txt)
and setting environment variables accordingly:
export CF_PREV_REVISION=78cb6185e063 export JL_PREV_REVISION=f448d3583dff
mkdir -p ~/tmp # Diff the manuals against the previous versions: cd $anno/checker-framework/checkers/manual hg diff -w -r$CF_PREV_REVISION . > ~/tmp/checkers-manual-diff cd $anno/jsr308-langtools hg diff -w -r$JL_PREV_REVISION README-jsr308.html > ~/tmp/langtools-manual-diff # Use the version control logs since the last release: # In Emacs, it's useful to do: (delete-matching-paragraphs "automated merge with " nil nil t) cd $anno/checker-framework hg log -v -r$CF_PREV_REVISION:tip > ~/tmp/checkers-log cd $anno/jsr308-langtools hg log -r$JL_PREV_REVISION:tip > ~/tmp/langtools-log hg log -r$JL_PREV_REVISION:tip --style changelog > ~/tmp/langtools-log-with-filenames
Ensure that your checkouts are up to date and you have no uncommitted changes. Each one of these hg commands should print "no changes found":
cd $anno/checker-framework hg fetch hg outgoing cd $anno/jsr308-langtools hg fetch hg outgoing
Update your release checkout:
cd $anno/checker-framework hg fetch cd ../jsr308-langtools hg fetch
There shouldn't be any conflicts, since the release scripts shouldn't make any serious local modifications (aside from version number changes, which you'll check in later in the process).
Ensure that the properties for the dry-run release is set appropriately in checker-framework/release/release.properties, set as follows:
web.dry.root.dir
: the location of your site copy, e.g.,
~/www/jsr308test
sanitycheck.dry.url
: the URL to access the releases
directory of your site copy, e.g.,
http://homes.cs.washington.edu/~${USER}/jsr308test/releases
(You'll do a dry-run of the release to your site copy, and will only update the live site after you've sanity-checked the release.)
Do a dry-run of the release:
cd $anno/checker-framework/release ant -f release.xml -Drelease.ver=$CFVERSION clean web sanitycheck
The web
step will run ant clean build-javac
build-javap
for langtools, and ant clean dists
all-tests
for checkers, so if the build is broken or any
checker tests fail, you won't be able to continue. (If this is the
case, you should fix the issue, or contact someone who can via checker-framework-dev@googlegroups.com.)
The sanitycheck
step unzips the releases from the website, then
rebuilds both projects and runs tests on the source and binary versions
from the release.
If all steps pass and you see BUILD SUCCESSFUL
, then
proceed. Otherwise, fix the problems and repeat the ant command.
Sanity-check the release. If there are any problems, correct them in the repository and re-make the release.
As a new user with no customizations (say, in a virtual machine; Mike uses ssh test@localhost), follow the Checker Framework short installation instructions by executing these commands:
cd wget -nv -N ${JSR308_WEB}/Makefile-jsr308-install JSR308_WEB=http://homes.cs.washington.edu/~REAL-USER-NAME/jsr308test/checker-framework/current \ make -f Makefile-jsr308-install
cd $anno/checker-framework/release ant -f release.xml -Drelease.ver=$CFVERSION -Drelease.is.real=true clean web checklinks sanitycheck
The same process as in the dry-run occurs; chances are, if the dry-run passed successfully, this step should pass as well.
cd wget -nv -N http://types.cs.washington.edu/jsr308/Makefile-jsr308-install make -f Makefile-jsr308-install
Check in any changes made to version numbers and dates as part of the release process. (Should this be done before the release, so that its data is consistent?)
cd $anno/jsr308-langtools hg diff # As always, review the changes output by "hg diff" before committing. # Typical commit message: hg commit -m "Update version number to $CFVERSION" hg push cd $anno/checker-framework hg diff # As always, review the changes output by "hg diff" before committing. # Typical commit message: hg commit -m "Update version number to $CFVERSION" hg push
cp -pf .../checkers-quals.jar ${inv}/java/lib cvs commit ${inv}/java/lib/checkers-quals.jar -m "checkers-quals.jar version $CFVERSION" cp -pf .../checkers-quals.jar ${pl}/java/lib hg commit ${pl}/java/lib/checkers-quals.jar -m "checkers-quals.jar version $CFVERSION"
Send a release announcement to jsr308-discuss@groups.google.com. The announcement should indicate what is new in the release, and should include the changelog entries from
checker-framework/checkers/changelog-checkers.txt jsr308-langtools/doc/changelog-jsr308.txt