JVM Class Data Sharing

This post describes how to enable Class Data Sharing (CDS) for a java app and what are the benefits of doing so.

I’ve recently seen a talk about CDS by Volker Simonis which was the inspiration to write this blogpost.

Table of Contents

What is Class Data Sharing?
Prepare CDS
- Intro — we need OpenJDK 10
Makefile
Experiment
- Let’s see how the classes are loaded
- Memory usage
Recap
Conclusions
- Ideas for future posts

Class Data Sharing is a JVM feature, which allows multiple JVMs to share loaded classes (and some other things) via shared memory.

A basic CDS was available with Sun JVM since java 1.5. It was limited to system classes and serial GC only, so not widely applicable. In Oracle JDK 9 this finally became useful, with support of other GC algorithms and application classes. It was a commercial feature until JDK 10, where it became opensource.

CDS creates a memory mapped file, caching the internal representation of specified classes. This file (and its memory) can therefore be shared between JVMs, plus the loading of classes from the internal representation is faster than from a regular jar file.

Class Data Sharing

It is worth noting that IBM Open J9 had a similar feature for a long time.

In this blog post we’ll investigate how to prepare such a CDS archive and look at potential benefits. We’ll use elasticsearch docker image to conduct the experiment.

Prepare CDS #

All of the code presented here is posted on github.

Intro — we need OpenJDK 10 #

Let’s use elasticsearch docker image to test the impact of class data sharing.

First, we need to repackage it with openjdk 10.

## Dockerfile-openjdk10
FROM docker.elastic.co/elasticsearch/elasticsearch-oss:6.2.4

COPY pkg/ /app/pkg

RUN cd /app/pkg && \
    tar -zxf openjdk-10.0.1_linux-x64_bin.tar.gz && \
    mkdir -p /opt/java && \
    mv jdk-10.0.1 /opt/java

ENV JAVA_HOME /opt/java/jdk-10.0.1

Let’s create a Makefile, we’ll add more to it as we go:

## Makefile
.PHONY: help
help:
    @ echo "# Elasticsearch and Class Data Sharing (CDS) experiment"
    @ echo
    @ echo "build-jdk10         - repackage elasticsearch-oss with openjdk10"
    @ echo 
    @ echo "clean-cache         - nuke the class list and cache"
    @ echo "generate-class-list - list classes used by elasticsearch"
    @ echo "dump-class-cache    - dump the classes from the list to cache file"
    @ echo "build-cds           - repackge the class cache in the elasticsearch docker container"
    @ echo
    @ echo "run-{cds/nocds}     - run elasticsearch with/without class data sharing"
    @ echo

pkg/openjdk-10.0.1_linux-x64_bin.tar.gz:
    @ wget -P pkg https://download.java.net/java/GA/jdk10/10.0.1/fb4372174a714e6b8c52526dc134031e/10/openjdk-10.0.1_linux-x64_bin.tar.gz

pkg: pkg/openjdk-10.0.1_linux-x64_bin.tar.gz

.PHONY: build-jdk10
build-jdk10: pkg
    @ docker build -f Dockerfile-openjdk10 -t "ikupczynski/elasticsearch-oss:6.2.4-openjdk10" .

And now we can get our repackaged image:

make build-jdk10

Create a list of classes used by Elasticsearch #

We’re going to start elastics, make it respond to a simple request and let it log all the classes it uses to a file -XX:DumpLoadedClassList=/app/cache/elasticsearch_appcds. We need to specifically enable the support for application classes with -XX:+UseAppCDS, otherwise will limit ourselves to jdk classes only.

CWD := $(shell pwd)

.PHONY: clean-cache
clean-cache:
    @ rm -rf cache/elasticsearch_appcds*
    @ echo "Nuked cache"

cache/elasticsearch_appcds.cls:
    @ touch cache/elasticsearch_appcds.cls
    @ export ES_JAVA_OPTS="-XX:+UseAppCDS \
            -XX:DumpLoadedClassList=/app/cache/elasticsearch_appcds.cls" && \
        docker run \
            -d --name generate-class-list \
            -p 9200:9200 -p 9300:9300 \
            -e "discovery.type=single-node" \
            -v $(CWD)/cache:/app/cache \
            --env ES_JAVA_OPTS \
            -it "ikupczynski/elasticsearch-oss:6.2.4-openjdk10"
    @ echo "Waiting until elasticsearch starts"
    @ bin/wait-on-elastic
    @ docker rm -f generate-class-list
    @ echo "Class list generated. Number of classes: "
    @ wc -l cache/elasticsearch_appcds.cls

We start elasticsearch in the background (docker run -d) and wait until it starts responding to requests with a following script:

until $(curl --output /dev/null --silent --head --fail localhost:9200); do
    printf '.'
    sleep 1
done

If we run make cache/elasticsearch_appcds.cls we’ll get a list of approximately 7k classes used by elasticsearch.

$ make cache/elasticsearch_appcds.cls
e2afbb966e1ad64d1296ada7c67a1038301834f8419aafa419ebbf74c7cd499b
Waiting until elasticsearch starts
...........generate-class-list
Class list generated. Number of classes:
    7111 cache/elasticsearch_appcds.cls

$ head -n 5 cache/elasticsearch_appcds.cls
java/lang/Object
java/lang/String
java/io/Serializable
java/lang/Comparable
java/lang/CharSequence

$ tail -n 5 cache/elasticsearch_appcds.cls
org/elasticsearch/common/io/Streams$FlushOnCloseOutputStream
org/elasticsearch/Version$DeclaredVersionsHolder
com/fasterxml/jackson/core/io/JsonStringEncoder
com/fasterxml/jackson/core/util/ByteArrayBuilder
org/elasticsearch/common/bytes/PagedBytesReference$1

Note that the list is generate while your app is running, so if you load any classes manually later on, you should exercise this codepath. In our case we just wait until elasticsearch can respond to a simple rest request. This is not perfect, but good enough for our experiment.

JVM error #

Normally, the next step would be to take the elasticsearch_appcds class list and use it to populate the cache file, however I’ve hit some limit (either a JVM bug, or more likely, some memory limits or docker-jvm interplay) and I needed to trim down the class list by 1/7 to around 6k. See error.org for more details.

I leave the root cause investigation for some later time. Now, let’s just trim the class list.

## Workaround of the JVM error
cache/elasticsearch_appcds.cls-thin: cache/elasticsearch_appcds.cls
    @ head -n 6218 cache/elasticsearch_appcds.cls > cache/elasticsearch_appcds.cls-thin
    @ echo "Filtered the class list. Number of classes: "
    @ wc -l cache/elasticsearch_appcds.cls-thin

generate-class-list: cache/elasticsearch_appcds.cls-thin

Prepopulate the class cache #

Now that we have the class list, let’s use it to pre-populate the class cache. Contrary to the previous step, here the jvm won’t run the app. It will start a process to go through the class list, load each of the classes and dump them to the cache file and stop the jvm.

cache/elasticsearch_appcds.jsa: generate-class-list
    @ touch cache/elasticsearch_appcds.jsa
    @ export ES_JAVA_OPTS="-Xshare:dump \
            -XX:+UseAppCDS \
            -XX:SharedClassListFile=/app/cache/elasticsearch_appcds.cls-thin \
            -XX:+UnlockDiagnosticVMOptions \
            -XX:SharedArchiveFile=/app/cache/elasticsearch_appcds.jsa" && \
        docker run \
            --rm --name dump-class-cache \
            -e "discovery.type=single-node" \
            -v $(CWD)/cache:/app/cache \
            --env ES_JAVA_OPTS \
            -it "ikupczynski/elasticsearch-oss:6.2.4-openjdk10"

dump-class-cache: generate-class-list cache/elasticsearch_appcds.jsa

Let’s try it out.

$ make dump-class-cache
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
narrow_klass_base = 0x0000000800000000, narrow_klass_shift = 3
Allocated temporary class space: 1073741824 bytes at 0x00000008c0000000
Allocated shared space: 3221225472 bytes at 0x0000000800000000
Loading classes to share ...
(...)
Loading classes to share: done.
Rewriting and linking classes ...
Preload Warning: Removed error class: org.apache.logging.log4j.core.async.AsyncLoggerContext
Rewriting and linking classes: done
Number of classes 7211
    instance classes   =  7087
    obj array classes  =   116
    type array classes =     8
Updating ConstMethods ... done.
Removing unshareable information ... done.
Scanning all metaspace objects ...
Allocating RW objects ...
Allocating RO objects ...
Relocating embedded pointers ...
Relocating external roots ...
Dumping symbol table ...
Relocating SystemDictionary::_well_known_klasses[] ...
(...)
Removing java_mirror ... done.
mc  space:     18184 [  0.0% of total] out of     20480 bytes [ 88.8% used] at 0x0000000800000000
rw  space:  18630320 [ 22.6% of total] out of  18632704 bytes [100.0% used] at 0x0000000800005000
ro  space:  31856600 [ 38.6% of total] out of  31858688 bytes [100.0% used] at 0x00000008011ca000
md  space:      6160 [  0.0% of total] out of      8192 bytes [ 75.2% used] at 0x000000080302c000
od  space:  31914104 [ 38.7% of total] out of  31916032 bytes [100.0% used] at 0x000000080302e000
total    :  82425368 [100.0% of total] out of  82436096 bytes [100.0% used]

And the resulting file is 80Mb of pre-loaded classes:

$ ls -lah cache/elasticsearch_appcds.jsa
-r--r--r--  1 igor  staff    79M May 20 20:52 cache/elasticsearch_appcds.jsa

Package it with the container #

We have the class cache, now we can package it with the container.

Here is the dockerfile:

## Dockerfile-cds
FROM ikupczynski/elasticsearch-oss:6.2.4-openjdk10

COPY cache/ /app/cache

As you can see it is quite simple, it just copies the cache to the container.

And the complimentary Makefile addition:

CDS_IMAGE = ikupczynski/elasticsearch-oss:6.2.4-cds

.PHONY: build-cds
build-cds: dump-class-cache
    @ docker build -f Dockerfile-cds -t $(CDS_IMAGE) .

I’ve build and pushed the image to dockerhub, so you can carry on from here with

docker pull ikupczynski/elasticsearch-oss:6.2.4-cds

Convenience targets to run elasticsearch #

Let’s define some make targets to let us run elasticsearch both with and without CDS.

RUN_NO_CDS = export ES_JAVA_OPTS="-Xshare:off \
            -Xlog:class+load:file=/app/logs/classload-nocds.log " && \
        docker run \
            -e "discovery.type=single-node" \
            -v $(CWD)/cache:/app/cache \
            -v $(CWD)/logs:/app/logs \
            --env ES_JAVA_OPTS \
            -it 

.PHONY: clean-nocds-logs
clean-nocds-logs:
    @ rm -f logs/classload-nocds.log*

.PHONY: run-nocds
run-nocds: clean-nocds-logs
    @ $(RUN_NO_CDS) --rm $(CDS_IMAGE)

## `time-nocds` given mostly for illustration, it is not acurate
.PHONY: time-nocds
time-nocds:
    @ $(RUN_NO_CDS) -p 9200:9200 --name run-no-cds -d $(CDS_IMAGE)
    @ echo "Timing the wait on elastic"
    @ time bin/wait-on-elastic
    @ docker rm -f run-no-cds

RUN_CDS = export ES_JAVA_OPTS="-Xshare:on \
            -Xlog:class+load:file=/app/logs/classload-cds.log \
            -XX:+UseAppCDS \
            -XX:SharedClassListFile=/app/cache/elasticsearch_appcds.cls-thin \
            -XX:+UnlockDiagnosticVMOptions \
            -XX:SharedArchiveFile=/app/cache/elasticsearch_appcds.jsa" && \
        docker run \
            -e "discovery.type=single-node" \
            -v $(CWD)/cache:/app/cache \
            -v $(CWD)/logs:/app/logs \
            --env ES_JAVA_OPTS \
            -it

.PHONY: clean-cds-logs
clean-cds-logs:
    @ rm -f logs/classload-cds.log*

.PHONY: run-cds
run-cds:
    @ $(RUN_CDS) --rm $(CDS_IMAGE)

## `time-cds` given mostly for illustration, it is not acurate
.PHONY: time-cds
time-cds:
    @ $(RUN_CDS) -p 9200:9200 --name run-cds -d $(CDS_IMAGE)
    @ echo "Timing the wait on elastic"
    @ time bin/wait-on-elastic
    @ docker rm -f run-cds

Now we can run elasticsearch with or without CDS:

make run-cds
make run-nocds

Experiment #

Let’s see how the classes are loaded #

We can run both versions in the terminal, and then inspect logs:

$ make time-nocds
## ...

$ make time-cds
## ...


$ grep 'org.elasticsearch.bootstrap.Bootstrap ' logs/*
logs/classload-cds.log:[1.370s][info][class,load] org.elasticsearch.bootstrap.Bootstrap source: shared objects file
logs/classload-nocds.log:[2.696s][info][class,load] org.elasticsearch.bootstrap.Bootstrap source: file:/usr/share/elasticsearch/lib/elasticsearch-6.2.4.jar

As we can see, with the class data sharing, we’ve loaded the main class from shared objects file, which means the class cache we’ve created. In case of nocds the file is loaded from the jar. All as expected.

We also see that the cds version loaded the main class over a second faster, and this number was fairly consistent across various runs I did to write this post. I wouldn’t trust it, as it wasn’t a rigorous benchmark — I run on a dev laptop with many variables to account for. If you are in a business of (re-)starting java apps often, generate a class cache and try to compare startup times for you use case / app.

Let’s also see how many classes where actually loaded from cache.

$ grep -c 'shared objects file' logs/classload-cds.log
2028

Memory usage #

Let’s start with no class data sharing:

$ make run-nocds  # 4 times, different terminals
## (...)


$ docker stats
CONTAINER ID        NAME                CPU %               MEM USAGE / LIMIT     MEM %               NET I/O             BLOCK I/O           PIDS
371268a8263b        vigilant_neumann    0.91%               1.168GiB / 7.787GiB   15.00%              858B / 0B           0B / 254kB          28
5137554b5e86        pensive_kepler      0.16%               1.164GiB / 7.787GiB   14.95%              858B / 0B           0B / 254kB          28
9fd0a493d7a2        vibrant_albattani   0.18%               1.174GiB / 7.787GiB   15.08%              858B / 0B           0B / 254kB          28
5cfb3f1d7a8e        vibrant_shtern      0.20%               1.17GiB / 7.787GiB    15.03%              1.04kB / 0B         8.19kB / 254kB      28

Reported mem usage: 1.164, 1.168, 1.170, 1.174 [GiB]

And with class data sharing

$ make run-cds  # 4 times
## (...)

$ docker stats
CONTAINER ID        NAME                       CPU %               MEM USAGE / LIMIT     MEM %               NET I/O             BLOCK I/O           PIDS
4e29db4f3dca        inspiring_volhard          0.97%               1.154GiB / 7.787GiB   14.82%              718B / 0B           8.19kB / 254kB      28
2841355336be        awesome_shockley           0.31%               1.153GiB / 7.787GiB   14.81%              788B / 0B           0B / 254kB          28
1fe95a004665        naughty_goldberg           0.39%               1.171GiB / 7.787GiB   15.04%              788B / 0B           0B / 254kB          28
513955f2d550        compassionate_montalcini   0.27%               1.152GiB / 7.787GiB   14.80%              968B / 0B           0B / 254kB          28

Reported mem usage: 1.152, 1.153, 1.154, 1.171 [GiB]

Again, not really a benchmark, but we can see a 5--20 MiB improvement.

Recap #

To create a class data archive, we need to:

Run the application, while dumping the class list

-XX:+UseAppCDS -XX:DumpLoadedClassList=/class-list.cls

Generate the archive file from this list

-Xshare:dump \
   -XX:+UseAppCDS \
   -XX:SharedClassListFile=/class-list.cls \
   -XX:+UnlockDiagnosticVMOptions \
   -XX:SharedArchiveFile=/archive.jsa

Run the app with the archive

-Xshare:on \
   -XX:+UseAppCDS \
   -XX:SharedClassListFile=/class-list.cls \
   -XX:+UnlockDiagnosticVMOptions \
   -XX:SharedArchiveFile=/archive.jsa

The internal representation may be architecture dependent, so should be generated on an env close to the target.

As a result of that, few dozen megabytes worth of java classes can be shared across JVMs; plus we’ll shave a few seconds from the startup time. In case of elasticsearch, these are not huge savings. On the other hand, if you run multiple copies of your app, or this is a script, where lower startup time improves the UX greatly, you should consider using Class Data Sharing. Coincidentally, if you run a serverless platform, both of these conditions apply.

Conclusions #

Class data sharing allows for shorter startup times and lower memory usage (if running multiple instances). Volker Simonis reports 30% statup time improvement for JRuby. If you need to start your app often (e.g. scripts) or run multiple copies of it (orchestration, serverless), and you are OpenJDK 10 compatible (for application class sharing) give it a try.

Ideas for future posts #

Why does JVM complain about full 7k list of classes for elasticsearch?
What are the gains with scala, which is known to generate a lot of classes.

>> Home

2018/05/29

What is Class Data Sharing? #