JEP 220: Modular Run-Time Images

AuthorMark Reinhold
OwnerAlan Bateman
TypeFeature
ScopeSE
StatusClosed / Delivered
Release9
JSR376
Discussionjigsaw dash dev at openjdk dot java dot net
EffortL
DurationL
BlocksJEP 200: The Modular JDK
JEP 261: Module System
Relates toJEP 162: Prepare for Modularization
JEP 201: Modular Source Code
JEP 282: jlink: The Java Linker
Reviewed byAlan Bateman, Alex Buckley, Chris Hegarty, Mandy Chung, Paul Sandoz
Created2014/10/23 15:05
Updated2017/09/22 20:16
Issue8061971

Summary

Restructure the JDK and JRE run-time images to accommodate modules and to improve performance, security, and maintainability. Define a new URI scheme for naming the modules, classes, and resources stored in a run-time image without revealing the internal structure or format of the image. Revise existing specifications as required to accommodate these changes.

Goals

Success Metrics

Modular run-time images equivalent to the JRE, JDK, and Compact Profile images of the immediately-preceding JDK 9 build must not regress on a representative set of startup, static footprint, and dynamic footprint benchmarks.

Non-Goals

Motivation

Project Jigsaw aims to design and implement a standard module system for the Java SE Platform and to apply that system to the Platform itself, and to the JDK. Its primary goals are to make implementations of the Platform more easily scalable down to small devices, improve the security and maintainability, enable improved application performance, and provide developers with better tools for programming in the large.

This JEP is the third of four JEPs for Project Jigsaw. The earlier JEP 200 defines the structure of the modular JDK, and JEP 201 reorganizes the JDK source code into modules. A later JEP, 261, introduces the actual module system.

Description

Current run-time image structure

The JDK build system presently produces two types of run-time images: A Java Runtime Environment (JRE), which is a complete implementation of the Java SE Platform, and a Java Development Kit (JDK), which embeds a JRE and includes development tools and libraries. (The three Compact Profile builds are subsets of the JRE.)

The root directory of a JRE image contains two directories, bin and lib, with the following content:

A JDK image includes a copy of the JRE in its jre subdirectory and contains additional subdirectories:

The root directory of a JDK image, or of a JRE image that is not embedded in a JDK image, also contains various COPYRIGHT, LICENSE and README files and also a release file that describes the image in terms of simple key/value property pairs, e.g.,

JAVA_VERSION="1.9.0"
OS_NAME="Linux"
OS_VERSION="2.6"
OS_ARCH="amd64"

New run-time image structure

The present distinction between JRE and JDK images is purely historical, a consequence of an implementation decision made late in the development of the JDK 1.2 release and never revisited. The new image structure eliminates this distinction: A JDK image is simply a run-time image that happens to contain the full set of development tools and other items historically found in the JDK.

A modular run-time image contains the following directories:

The root directory of a modular run-time image also contains the release file, which is generated by the build system. To make it easy to tell which modules are present in a run-time image the release file includes a new property, MODULES, which is a space-separated list of the names of those modules. The list is topologically ordered according to the modules' dependence relationships, so the java.base module is always first.

Removed: The endorsed-standards override mechanism

The endorsed-standards override mechanism allowed implementations of newer versions of standards maintained outside of the Java Community Process, or of standalone APIs that are part of the Java SE Platform yet continue to evolve independently, to be installed into a run-time image.

The endorsed-standards mechanism was defined in terms of a path-like system property, java.endorsed.dirs, and a default value for that property, $JAVA_HOME/lib/endorsed. A JAR file containing a newer implementation of an endorsed standard or standalone API can be installed into a run-time image by placing it in one of the directories named by the system property, or by placing it in the default lib/endorsed directory if the system property is not defined. Such JAR files are prepended to the JVM's bootstrap class path at run time, thereby overriding any definitions stored in the run-time system itself.

A modular image is composed of modules rather than JAR files. Going forward, endorsed standards and standalone APIs are supported in modular form only, via the concept of upgradeable modules. We have therefore removed the endorsed-standards override mechanism, including the java.endorsed.dirs system property and the lib/endorsed directory. To help identify any existing uses of this mechanism the compiler and the launcher now fail if this system property is set, or if the lib/endorsed directory exists.

Removed: The extension mechanism

The extension mechanism allowed JAR files containing APIs that extend the Java SE Platform to be installed into a run-time image so that their contents are visible to every application that is compiled with or runs on that image.

The mechanism was defined in terms of a path-like system property, java.ext.dirs, and a default value for that property composed of $JAVA_HOME/lib/ext and a platform-specific system-wide directory (e.g, /usr/java/packages/lib/ext on Linux). It worked in much the same manner as the endorsed-standards mechanism except that JAR files placed in an extension directory were loaded by the run-time environment's extension class loader, which is a child of the bootstrap class loader and the parent of the system class loader, which actually loads the application to be run from the class path. Extension classes therefore could not override the JDK classes loaded by the bootstrap loader but they were loaded in preference to classes defined by the system loader and its descendants.

The extension mechanism was introduced in JDK 1.2, which was released in 1998, but in modern times we have seen little evidence of its use. This is not surprising, since most Java applications today place the libraries that they need directly on the class path rather than require that those libraries be installed as extensions of the run-time system.

It is technically possible, though awkward, to continue to support the extension mechanism in the modular JDK. To simplify both the Java SE Platform and the JDK we have removed the extension mechanism, including the java.ext.dirs system property and the lib/ext directory. To help identify any existing uses of this mechanism the compiler and the launcher now fail if this system property is set, or if the lib/ext directory exists. The compiler and the launcher ignore the platform-specific system-wide extension directory by default, but if the -XX:+CheckEndorsedAndExtDirs command-line option is specified then they fail if that directory exists and is not empty.

Several features associated with the extension mechanism were retained, since they are useful in their own right:

Removed: rt.jar and tools.jar

The class and resource files previously stored in lib/rt.jar, lib/tools.jar, lib/dt.jar, and various other internal JAR files are now stored in a more efficient format in implementation-specific files in the lib directory. The format of these files is not specified and is subject to change without notice.

The removal of rt.jar and similar files leads to three distinct problems:

  1. Existing standard APIs such as the ClassLoader::getSystemResource method return URL objects to name class and resource files inside the run-time image. For example, when run on JDK 8 the code

    ClassLoader.getSystemResource("java/lang/Class.class");

    returns a jar URL of the form

    jar:file:/usr/local/jdk8/jre/lib/rt.jar!/java/lang/Class.class

    which, as can be seen, embeds a file URL to name the actual JAR file within the run-time image. The getContent method of that URL object can be used to retrieve the content of the class file, via the built-in protocol handler for the jar URL scheme.

    A modular image does not contain any JAR files, so URLs of the above form make no sense. The specifications of getSystemResource and related methods, fortunately, do not require the URL objects returned by these methods actually to use the JAR scheme. They do, however, require that it be possible to load the content of a stored class or resource file via these URL objects.

  2. The java.security.CodeSource API and security-policy files use URLs to name the locations of code bases that are to be granted specified permissions. Components of the run-time system that require specific permissions are currently identified in the lib/security/java.policy file via file URLs. The elliptic-curve cryptography provider, e.g., is identified as

    file:${java.home}/lib/ext/sunec.jar

    which, obviously, has no meaning in a modular image.

  3. IDEs and other kinds of development tools require the ability to enumerate the class and resource files stored in a run-time image, and to read their contents. Today they often do this directly by opening and reading rt.jar and similar files. This is, of course, not possible with a modular image.

New URI scheme for naming stored modules, classes, and resources

To address the above three problems a new URL scheme, jrt, can be used to name the modules, classes, and resources stored in a run-time image without revealing the internal structure or format of the image.

A jrt URL is a hierarchical URI, per RFC 3986, with the syntax

jrt:/[$MODULE[/$PATH]]

where $MODULE is an optional module name and $PATH, if present, is the path to a specific class or resource file within that module. The meaning of a jrt URL depends upon its structure:

These three forms of jrt URLs address the above problems as follows:

  1. APIs that presently return jar URLs now return jrt URLs. The above invocation of ClassLoader::getSystemResource, e.g., now returns the URL

    jrt:/java.base/java/lang/Class.class

    A built-in protocol handler for the jrt scheme ensures that the getContent method of such URL objects retrieves the content of the named class or resource file.

  2. Security-policy files and other uses of the CodeSource API can use jrt URLs to name specific modules for the purpose of granting permissions. The elliptic-curve cryptography provider, e.g., can now be identified by the jrt URL

    jrt:/jdk.crypto.ec

    Other modules that are currently granted all permissions but do not actually require them can trivially be de-privileged, i.e., given precisely the permissions they require.

  3. A built-in NIO FileSystem provider for the jrt URL scheme ensures that development tools can enumerate and read the class and resource files in a run-time image by loading the FileSystem named by the URL jrt:/, as follows:

    FileSystem fs = FileSystems.getFileSystem(URI.create("jrt:/"));
byte[] jlo = Files.readAllBytes(fs.getPath("modules", "java.base",
                                           "java/lang/Object.class"));

    The top-level modules directory in this filesystem contains one subdirectory for each module in the image. The top-level packages directory contains one subdirectory for each package in the image, and that subdirectory contains a symbolic link to the subdirectory for the module that defines that package.

    For tools that support the development of code for JDK 9 but which themselves run on JDK 8, a copy of this filesystem provider suitable for use on JDK 8 is placed in the lib directory of JDK 9 run-time images, in a file named jrt-fs.jar.

(The jrt URL protocol handler does not return any content for URLs of the second and third forms.)

Build-system changes

The build system produces the new run-time image format described above, using the Java linker (JEP 282).

We took the opportunity here, finally, to rename the images/j2sdk-image and images/j2re-image directories to images/jdk and images/jre, respectively.

Minor specification changes

JEP 162, implemented in JDK 8, made a number of changes to prepare the Java SE Platform and the JDK for the modularization work described here and in related JEPs. Among those changes were the removal of normative specification statements that require certain configuration files to be looked up in the lib directory of run-time images, since those files are now in the conf directory. Most of the SE-only APIs with such statements were so revised as part of Java SE 8, but some APIs shared across the Java SE and EE Platforms still contain such statements:

In Java SE 9, these statements no longer mandate the lib directory.

Testing

Some existing tests made direct use of run-time image internals (e.g., rt.jar) or refer to system properties (e.g., java.ext.dirs) that no longer exist. These tests have been fixed.

Early-access builds containing the changes described here were available throughout the development of the module system. Members of the Java community were strongly encouraged to test their tools, libraries, and applications against these builds to help identify compatibility issues.

Risks and Assumptions

The central risks of this proposal are ones of compatibility, summarized as follows:

Dependences

This JEP is the third of four JEPs for Project Jigsaw. It depends upon JEP 201, which reorganized the JDK source code into modules and upgraded the build system to compile modules. It also depends upon earlier preparatory work done in JEP 162, implemented in JDK 8.