JEP 181: Nest-Based Access Control

Summary

Introduce nests, an access-control context that aligns with the existing notion of nested types in the Java programming language. Nests allow classes that are logically part of the same code entity, but which are compiled to distinct class files, to access each other's private members without the need for compilers to insert accessibility-broadening bridge methods.

Non-Goals

This JEP is not concerned with large scales of access control, such as modules.

Motivation

Many JVM languages support multiple classes in a single source file (such as Java's nested classes), or translate non-class source artifacts to class files. From a user perspective, however, these are generally considered to be all in "the same class", and therefore users expect them to share a common access control regime. To preserve these expectations, compilers frequently have to broaden the access of private members to package, through the addition of access bridges: an invocation of a private member is compiled into an invocation of a compiler-generated package-private method in the target class, which in turn accesses the intended private member. These bridges subvert encapsulation, slightly increase the size of a deployed application, and can confuse users and tools. A formal notion of a group of class files forming a nest, where nest mates share a common access control mechanism, allows the desired result to be directly achieved in a simpler, more secure, more transparent manner.

The notion of a common access control context arises in other places as well, such as the host class mechanism in Unsafe.defineAnonymousClass(), where a dynamically loaded class can use the access control context of a host. A formal notion of nest membership would put this mechanism on firmer ground (but actually providing a supported replacement for defineAnonymousClass() would be a separate effort.)

Description

The Java Language Specification allows classes and interfaces to be nested within each other. Within the scope of a top-level declaration (JLS 7.6), any number of types can appear nested. These nested types have unrestricted access to each other (JLS 6.6.1), including to private fields, methods, and constructors. We can describe a top-level type, plus all types nested within it, as forming a nest, and two members of a nest are described as nestmates.

The private access is complete (undifferentiated, flat) within the whole declaration of the containing top-level type. (One can think of this as a top-level type defining a sort of "mini-package", within which extra access is granted, even beyond that provided to other members of the same Java package.)

Today private access between nestmates is not permitted by the JVM access rules. To provide the permitted access a Java source code compiler has to introduce a level of indirection. For example, an invocation of a private member is compiled into an invocation of a compiler-generated package-private, bridging method in the target class, which in turn invokes the intended private method. These access bridges are generated only as needed to satisfy the member accesses requested within the nest.

A further consequence of the lack of JVM support for private access within a nest, is that core reflection also denies access. A reflective method invocation (using java.lang.reflect.Method.invoke) from one nestmate to another throws IllegalAccessError (unless access control has been disabled). This is surprising given that reflective invocations should behave the same as source level invocations. Similarly, the MethodHandle API rejects a direct "lookup" of a private nestmate method, but provides special support through Lookup.in to allow the source level invocation semantics to be expressed.

By codifying the notion of nestmates and the associated access rules within the JVM, we simplify the job of Java source code compilers, tighten the existing access checks and remove surprising behaviour from the core reflection and MethodHandle APIs. We also allow for future enhancements to take advantage of the "nest" notion. For example:

• In generic specialization, each specialized type could be created as a nestmate of the generic type.
• A safe and supported replacement for the Unsafe.defineAnonymousClass() API could create the new class as a nestmate of an existing class.
• The concept of "sealed classes" could be effected by only allowing subclasses that are nestmates.
• Truly private nested types could be effected (presently private nested types are defined with package-access).

Nest Class File Attributes

The existing classfile format defines the InnerClasses and EnclosingMethod attributes (JVMS 4.7.6 and 4.7.7) to allow Java source code compilers, like javac, to reify the source level nesting relationship. Each nested type is compiled to its own class file, with different class files "linked" by the value of these attributes. While these attributes are enough for the JVM to determine nestmate-ness, they are not directly suitable for access control and are inherently tied to a single Java language concept.

To allow for a broader, more general, notion of nestmates beyond simply Java language nested types, and for the sake of efficient access control checking, it is proposed to modify the class file format to define two new attributes. One nest member (typically the top-level class) is designated as the nest host, and contains an attribute (NestMembers) to identify the other statically known nest members. Each of the other nest members has an attribute (NestHost) to identify its nest host.

JVM Access Control for Nestmates

We will adjust the JVM's access rules by adding something like the following clause to JVMS 5.4.4:

A field or method R is accessible to a class or interface D if and only if any of the following conditions are true:

• ...
• R is private and is declared in a different class or interface C, and C and D, are nestmates.

For types C and D to be nestmates they must have the same nest host. A type C claims to be a member of the nest hosted by D, if it lists D in its NestHost attribute. The membership is validated if D also lists C in its NestMembers attribute. D is implicitly a member of the nest that it hosts.

A class with no NestHost or NestMembers attribute, implicitly forms a nest with itself as the nest host, and sole nest member.

The loosened access rules would affect access checks during the following activities:

• Resolving fields and methods (JVMS 5.4.3.2, etc.)
• Resolving method handle constants (JVMS 5.4.3.5)
• Resolving call site specifiers (JVMS 5.4.3.6)
• Checking Java language access by instances of java.lang.reflect.AccessibleObject
• Checking access during queries to java.lang.invoke.MethodHandles.Lookup

With the change to the access rules, and with suitable adjustments to byte code rules, we can allow simplified rules for generating invocation bytecodes:

• invokespecial for private nestmate constructors,
• invokevirtual for private non-interface, nestmate instance methods,
• invokeinterface for private interface, nestmate instance methods; and
• invokestatic for private nestmate, static methods

This relaxes the existing constraint that private interface methods must be invoked using invokespecial (JVMS 6.5) and more generally allows invokevirtual to be used for private method invocation, rather than adding to the complex usage rules surrounding invokespecial. Similar changes can be made to the semantics of MethodHandle invocation (which mirrors the invocation byte code constraints).

Nest Membership Validation

Nest membership must be validated before an access check relying on nestmate access can proceed. This may happen as late as the time of access of the member, or as early as verification time of the class, or somewhere in-between, such as JIT compilation of a method. Nest membership validation requires loading of the nest host class if it has not already been loaded. To avoid potentially unnecessary class loading, nest membership validation should be performed as late as possible i.e. at the time of the access check. This mitigates the impact of the incompatibility introduced by requiring that the nest host class be present if nestmate access is relied upon.

To preserve integrity of the nest it is proposed that, at least initially, it is prohibited to modify the nest classfile attributes using any form of class transformation or class redefinition.

Nestmate Reflection API

As we are introducing new classfile attributes it is customary to provide a means to inspect/query those attributes using core reflection. This is currently envisaged as three methods in java.lang.Class: getNestHost, getNestMembers, and isNestmateOf.

Affected Specifications and APIs

The proposed changes, while conceptually simple, affect all specifications and API's that explicitly, or implicitly, involve access control, or relate to method invocation modes. These include:

• The Java Virtual Machine Specification
  • Classfile attribute changes
  • Access control rule changes
  • Invocation bytecode rule changes
• Core reflection
  • Method invocation rules
  • Field access rules
• MethodHandle lookup rules
• Class transformation/re-definition: JVM TI and java.lang.instrument API, JDWP and JDI (com.sun.jdi.VirtualMachine)
  • Prohibit modification of the nest related classfile attributes
• Pack200 Specification
  • Recognize the new classfile attributes

Impact on Java Source Code Compilers

The proposed changes simplify the rules for mapping Java source constructs to class files, and so have a number of affects on a Java source code compiler that chooses to utilize them:

• Proper generation of the nest related classfile attributes
• Elision of the previously needed access bridge methods and generation of direct member access instructions for private nestmate members
• Issuing of the correct/appropriate invocation bytecodes
• The ability to change other synthetic methods to be private rather than package-private (or even eliminate them, or replace them with shared-but-private method handle constants)

The javac compiler will be updated to fully utilize nestmates when generating the latest version classfiles. (Older versions will be generated as they are today using access bridges etc.)

Impact on Other Tools

Any tool that operates on classfiles, or which generates or processes bytecodes is potentially impacted by these changes. At a minimum such tools must tolerate the presence of the new classfile attributes and allow for the change in bytecode rules. For example:

• The javap classfile inspection tool,
• The Pack200 implementation, and
• The ASM bytecode manipulation framework, which is also used internally in the JDK.

Open Issues

The extra complexity of access checking is something that has to be examined. In particular issues surrounding the resolution of the nest host class and the errors that can arise. We have already encountered and addressed a problem where a compiler thread needed to load a nest host class - which is not permitted in the compiler thread. We need to ensure the implementation is able to handle these conditions, and ensure that the specification is not impacted by their introduction.

Alternatives

We can continue generating bridge methods in the Java compiler, as needed. This is a hard process to predict. For example, Project Lambda had difficulty resolving method handle constants in the presence of inner classes, leading to a new type of bridge method. Because compiler-generated bridge methods are tricky and unpredictable, they are also buggy and hard to analyze by various tools, including decompilers and debuggers.

The initial proposal considered using the existing InnerClasses and EnclosingMethod attributes to establish nestmate-ship. But introducing specific nestmate related attributes both makes nestmates more general than only relating to language-level nested types, and permits a more efficient implementation. Additionally, if we had opted for eager nest membership validation checks it would have changed the semantics of the existing attributes and that would have been a compatibility issue. While the javac compiler will likely keep the "inner classes" and "nest members" attributes aligned, this is a compiler choice, and the JVM will treat them completely independently.

There was discussion about how best to express the nesting relationship through the classfile attributes before settling on the current approach. One suggestion, for a de-centralised approach, was for each nest to be identified by a UUID. That discussion concluded as follows:

There are two parts to such a proposall:

1. New naming convention for nests, based on UUIDs. This is a new concept in the JVM, and would require new infrastructure to manage (generate, transcode, verify, reflect, debug). That means new bugs and new attack surfaces. In the absence of a decisive benefit, it's better to reuse existing name spaces, and (in particular) the JVM's type name dictionary.

2. Unidirectional links. The UUID, being a pure identity with no content, does not contain a list of its nest members. The nest members point to the nest (via the UUID). Any class can inject itself into a nest (in the same package) simply by mentioning the appropriate UUID. Unidirectional linkage means that there is no way to enumerate a nest. This complicates some optimizations (based on sealed types). Security and seal-ability of nests is reduced to that of packages. PRIVATE becomes just an alias for default-scope access control.

Sorry, but neither part of this is appealing to me, compared with the current proposal.

Testing

We will need an extensive set of JVM tests to verify the new access rules and adjustments to the byte code semantics to support nestmates.

Similarly we will need additional tests for core reflection, method handles, var-handles, and external access API's like JDWP, JVM TI and JNI.

Since no language changes are proposed here, no new language compliance tests are needed.

Adequate functional tests for nestmates will arise naturally from language compliance tests, after the javac compiler is modified to take advantage of nestmate access.

Risks and Assumptions

The new rules will have to be associated with a new class file version number, as we need to ensure that a Java source compiler only generates bytecodes relying on the new attributes and rules, when targeting a JVM that understands them. A corollary to that is that a JVM will only recognize and act upon the new attributes, if they appear in a classfile with a suitable version number. A new classfile version imposes a burden on the tools in the broader Java ecosystem, but we do not expect Nestmates to be the sole technology, in the targeted JDK release, that will be relying on a new class file version number.

Loosening access presents little conformance risk. All Java language accesses that compile and run today, will compile and run with the nestmate changes, with no changes to the source code. Code that disables access checking (via setAccessible) for reflective access to nestmates today, will continue to run correctly with the nestmate changes - but can be updated to not disable access checking.

Compliance tests that check for prohibited behaviour can fail is some cases. For example:

• Direct reflective access to private nest methods currently fails (unless access checking is disabled) but will "unexpectedly" succeed with these changes applied.
• A test that invokeinterface can't be used for a private interface method will now fail because with these changes it can be used.

There is little or no risk to user compatibility, since the proposal loosens access. However, if users have "discovered" and exploited access bridge methods, they will be unable to do so after the bridges are dropped. Such a risk is very small, since bridge methods do not have stable names in the first place.

There is little or no risk to system integrity, since the proposed rules confer new access only within a single runtime package. By removing the need for bridge methods, potential access between distinct top-level classes will be systematically decreased.

Nest membership validation requires the presence of the nest-host class, even if that class is itself unused (except as a container for the nest members). This can have impact in three areas:

1. The order of class loading can change as the nest host may be needed for an access check earlier than when any direct use of the nest host occurs. This is not expected to be a problem as the class is only loaded, not initialized, and dependencies on class loading order (as distinct from class initialization order) are very rare.

2. This can impact tests/applications that strip out unused classes from their distributed forms, and the nest host is unused. By leaving nest membership validation until the time that a nestmate access check is needed we aim to minimize the impact of this issue, but in some cases the end user will have to change the way they distribute their code. We think this is a very small risk as it isn't common to use a top-level class purely as a stateless container, containing only static nested types, where the nested types will rely on private access to each other.

3. Resolution of the nest host also introduces class loading (and the potential for associated exceptions) into the JVM's access checking logic. This is primarily a concern for JVM implementors. Care must be taken to ensure all paths that can lead to a VM access check either preclude the possibility of loading a nest host, or else can cope with it. Similarly for the potential exceptions that can occur. From the user perspective there is very little risk due to this as Java code rarely makes assumptions about when and where class loading may occur, and exceptions will only occur if there are badly formed class files.