[JSR308] Suggestion (and example) for annotations that take any annotation or for annotation inheritance

[Jonathan Aldrich]

Hello,

I appreciate Joe sharing his perspective on this issue.  I work with 
Ciera and Nels at CMU; they and others in my group have found that 
annotation inheritance and/or annotations that take another annotation 
are useful for a wide range of analyses, and so I wanted to respond to 
some of the points Joe made and better explain why we think this 
extension would be worth its weight.

Regarding JSR 175, I definitely appreciate its focus on the needs of 
"Enterprise Edition" Java.  On the other hand, JSR 308 is at least in 
part aimed at adapting annotations, in minor ways, to make them more 
suitable for program analysis.  Whether annotation-based program 
analysis becomes mainstream is unknown--but if it does, as I and many 
other believe it will, it will benefit all programmers, not just 
Enterprise programmers.  Thus JSR 308 could have a wider impact than JSR 
175.  Furthermore, although I'm not an Enterprise edition expert, there 
are good reasons for thinking that Enterprise users might benefit from 
annotations that take another annotation as well (see below).

I believe allowing annotations to take arbitrary annotations as 
arguments is a smaller change than allowing annotations wherever types 
go (the primary focus of the proposal at present).  Furthermore, it will 
eventually be needed by almost all of the program analyses/type systems 
targeted by JSR 308.  Finally, this one change, by itself, can express 
much of the the JSR 308 existing proposal (I'm not arguing we should 
throw the rest out, only that this piece carries its weight well).  Let 
me explain these one by one.



FIRST: annotations taking other annotations as parameters is a 
(comparatively) small change.  The existing JSR 308 proposal changes the 
Java language syntax.  Merely permitting annotations to extend other 
annotations would not change the language syntax.  The existing JSR 308 
proposal changes the class file format.  Joe raised a great question 
about whether annotations taking other annotations as parameters would 
require changing the class file format as well.  As far as I can tell, 
from the specification of the Java 5 classfile format, it would not:

http://docs.oracle.com/javase/specs/jvms/se5.0/ClassFileFormat-Java5.pdf

Specifically, annotations are stored in an "annotation" structure in the 
classfile.  This structure contains a type index (the type of the 
annotation) and a list of element_value_pairs.  The value in the 
element_value_pair is an element_value, which is a tagged union, and if 
the tag is "@" for annotation, then the union stores an "annotation" 
structure.  That's the annotation structure above--which in fact 
contains a type index, that in the case of "annotations taking arbitrary 
annotations" tells us what type the nested annotation is.  Ironically, 
in the current classfile format, this is wasted space, as it could be 
inferred from the surrounding annotation type, but it nicely supports 
this extension.  Thus, "annotations containing arbitrary annotations" 
need change neither the syntax of the language nor the classfile format. 
  It could be that there are other issues lurking, and Joe is right that 
we need to investigate this in detail, but to me it initially looks like 
a simpler change than the other elements of JSR 308.



SECOND: such an extension would be of very wide utility.  In the last 
week, four people have suggested this.  Two of them are my students, who 
are developing entirely different type systems, with independent and 
somewhat different needs for annotations within annotations.  I have at 
least one more example that I'll give below.  This, and that two others 
unrelated to me have proposed variants of the same idea, suggests we're 
on to something general.

In fact, the most important use case for "annotations taking arbitrary 
annotations" is annotation parameters that don't correspond to a type 
parameter.  Imagine that you have developed some kind of container class 
that is designed to hold things of a specific type, not of an arbitrary 
type.  This may be realistic in cases where the container needs to 
interact with the things it contains in domain-specific ways.  In this 
case, there is no reason to have a type parameter on the container. 
However, it may still make lots of sense to have annotations on the 
contained thing.

As a concrete example, another one of my students is using annotations 
to implement an ownership type system.  Ownership types can be used for 
lots of things, including finding design errors, eliminating race 
conditions, etc.  Imagine a MySubject object that has a list of 
MyObserver objects.  The MySubject knows the MyObserver type, and 
doesn't need a type parameter for the particular kind of observer, but 
it does need to know who owns its observers.  Right now, we have to 
express this in an ugly string-based syntax:

@OwnedBy("owned<shared>") MySubject subject

Here "owned" is the owner of MySubject, while "shared" is the owner of 
the observers stored in MySubject.  We could add a dummy parameter to 
MySubject and use JSR 308 as follows:

@Owned
MySubject< @Shared MyObserver> subject

First, note the benefits of JSR308: I was able to avoid parsing a 
string, and I can use the more structured @Owned and @Shared instead of 
"owned" and "shared."  But it is very unpleasant, to the point of being 
unmaintainable in large systems where there might be multiple such 
parameters, to add unneeded type parameters just to add an annotation 
parameter.  A much better solution is to use annotations:

@Owned
@OwnershipArgs({@Shared})
MySubject subject

Almost any analysis is likely to come up against this issue.  To pick a 
few from the JSR 308 spec, containers without type parameters could 
benefit in:
  * Titanium, where contained elements are @local
  * dependent types, where the contained elements are arrays of a @Size
  * javari/immutability, where the contained elements are immutable
  * non-nullness, where the contained elements are non-null
  * taintedness, where the contained elements are tainted
  * etc. for just about any type qualifier

Another great example in the JSR 308 spec, which is independent of the 
"annotation parameterization without type parameterization" is units of 
measurement.  Instead of @Units("m/s") it might be nice to expose the 
structure of the units with annotations: @Units(value={@Meters}, 
dividedBy={@Seconds})  This won't work without "annotations taking 
arbitrary annotations."

As a further (conceptual) motivation, JSR 175 was originally developed 
for "Enterprise Edition" Java--but when Enterprise users aren't using 
annotations, they're using XML configuration files.  Interestingly, the 
most important semantic difference between JSR 175 annotations and XML 
files is that XML files allow recursive structure--XML elements 
containing arbitrary XML elements--whereas JSR 175 does not allow 
annotations to contain arbitrary annotations.  I am not an Enterprise 
Java user, but I suspect that there would be a lot of use cases in 
Enterprise Java for recursively structured annotations--in particular I 
imagine complex issues like database persistence and security could 
probably benefit.

So if JSR 308 is designed to facilitate (at least in part) static 
checkers, nearly all of them would benefit from "annotations taking 
arbitrary annotations," and although I don't have examples, it's 
reasonable to think that other uses of annotations would benefit as well.



THIRD, "annotations taking arbitrary annotations" can express much of 
what's in JSR 308--not as cleanly, to be sure, but still allowing users 
to express annotations on most uses of a type.  Instead of the current 
proposal:

Map<@NonNull String, @NonEmpty List<@Readonly Document>> files;

we could write:

@Params({@NonNull, @AnnotAndParams(@NonEmpty, {@Readonly})})
Map<String, List<Document>> files;


Similar ideas would work for arrays, type bounds, wildcards, class 
inheritance, throws clauses, and method receivers.  It would not work 
for new statements, casts, or generic type arguments; syntax extensions 
a la JSR 308 are needed there.

Now, the JSR 308 syntax is *definitely* more readable, and I am not for 
a moment suggesting that we give it up.  But the fact that "annotations 
taking arbitrary annotations" is by itself, likely simpler than the rest 
of JSR 308 to implement, is broadly applicable, and can express much of 
what the rest of JSR 308 does, seems to me like a good argument that it 
pulls its own weight.  It arguably may fall higher than the rest of JSR 
308 on the "worse is better" scale.



Joe also said that work needs to be done in ensuring this can be 
feasibly implemented, beyond the short analysis above; I agree, and I 
hope we can contribute to that.  But first I'd like to get a broader 
sense for if the JSR 308 community sees this as a valuable proposal.

Thanks,

Jonathan



P.S. Regarding "anything that requires instanceof is suspicious" - 
functional programmers who use pattern matching would disagree, and I 
suspect instanceof is harmless in many of the use cases for this 
extension.  But from an OO extensibility perspective, the "right change" 
is actually not "annotations that take arbitrary annotations" but rather 
"annotations that inherit from other annotations."  In the latter case, 
no instanceof is necessary, as ordinary dispatch or the visitor pattern 
can be used.  But "annotations that inherit from other annotations" is 
more complex than "annotations that take arbitrary annotations", so a 
"worse is better" perspective suggests that we might prefer "annotations 
that take arbitrary annotations" despite the instanceof issue.  If 
there's a groundswell of support for annotations that inherit from other 
annotations, and the technical issues are solvable, I'd be delighted.