A Trace-based Java JIT Compiler Retrofitted from a Method-based ...

IBM Research - Tokyo

A Trace-based Java JIT Compiler Retrofitted from a Method-based Compiler Hiroshi Inoue†, Hiroshige Hayashizaki†, Peng Wu‡ and Toshio Nakatani† † IBM Research – Tokyo ‡ IBM Research – T.J. Watson Research Center

April 6, 2011 | CGO at Chamonix, France

© 2011 IBM Corporation

IBM Research - Tokyo

Goal and Motivation Goals 1. develop an efficient trace-based Java JIT compiler (trace-JIT) based on existing mature method-based JIT compiler (method-JIT) 2. understand the benefits and drawbacks of the trace-JIT against the method-JIT Why not method-JIT? – Limited optimization opportunities in larger application with a flat execution profile (no hot spots) Can trace-JIT provide more optimization opportunities than method-JIT for such applications?

2

A Trace-based Java JIT Compiler Retrofitted from a Method-based Compiler

© 2011 IBM Corporation

IBM Research - Tokyo

Background: Trace-based Compilation Using a Trace, a hot path identified at runtime, as a basic unit of compilation method f

method entry if (x != 0) rarely executed

trace A

trace entry

if (x != 0)

frequently executed

while (!end)

trace exit (side exit)

frequently executed

while (!end)

do something

return 3

a hot path

trace exit

A Trace-based Java JIT Compiler Retrofitted from a Method-based Compiler

include a hot path only © 2011 IBM Corporation

IBM Research - Tokyo

Motivating Example A trace can span multiple methods – Free from method boundaries – In large server workloads, there are deep (>100) layers of methods method 1

method 2

method 3

method 4

Web Container

JavaServer Pages

Enterprise JavaBeans

JDBC Driver

ok?

error!

search for a handler

log?

4

ok?

error!

search for a bean

print

log?

print

ok?

error!

create an SQL

log?

print

A Trace-based Java JIT Compiler Retrofitted from a Method-based Compiler

ok?

error!

send the SQL to DB

log?

print

© 2011 IBM Corporation

IBM Research - Tokyo

Outline Motivation Background Trace-JIT Architecture Performance Evaluation Future work and Summary

5

A Trace-based Java JIT Compiler Retrofitted from a Method-based Compiler

© 2011 IBM Corporation

IBM Research - Tokyo

Baseline Method-JIT Components Java VM

interpreter interpreter

compiled compiled method method dispatcher dispatcher

garbage collector class libraries

compilation request for frequently executed methods

code code cache cache

Method-JIT compiled code

code code generator generator IR IR generator generator

6

optimizers optimizers

A Trace-based Java JIT Compiler Retrofitted from a Method-based Compiler

© 2011 IBM Corporation

IBM Research - Tokyo

Our Trace-JIT Architecture Java VM

interpreter interpreter

trace trace dispatcher dispatcher

garbage collector class libraries

execution events Tracing runtime

trace trace selection selection engine engine trace (Java bytecode)

hash hash map map

code code cache cache

(e.g. compiled code address)

compiled code Trace-JIT

trace trace side side exit exit elimination elimination

code code generator generator

IR IR generator generator

optimizers optimizers

new new component component modified modified component component unmodified component

7

A Trace-based Java JIT Compiler Retrofitted from a Method-based Compiler

© 2011 IBM Corporation

IBM Research - Tokyo

Our Trace-JIT Architecture identify two types of hot paths

Java VM

interpreter interpreter

trace trace dispatcher dispatcher linear trace

cyclic trace

class libraries

execution events Tracing runtime

trace trace selection selection engine engine

garbage collector

A A

hash hash map map

A A

code code cache cache

B

trace (Java bytecode)

B B

(e.g. compiled codeBaddress)

exit

compiled code

Trace-JIT

trace trace side side exit exit elimination elimination

stub code code generator generator exit

IR IR generator generator

optimizers optimizers

stub exitnew new component component modified modified component component unmodified component

8

A Trace-based Java JIT Compiler Retrofitted from a Method-based Compiler

© 2011 IBM Corporation

IBM Research - Tokyo

Our Trace-JIT Architecture Java VM

interpreter interpreter

trace trace dispatcher dispatcher

garbage collector class libraries

execution events Tracing runtime

trace trace selection selection engine engine trace (Java bytecode)

hash hash map map

code code cache cache

(e.g. compiled code address)

compiled code Trace-JIT

trace trace side side exit exit elimination elimination

code code generator generator

IR IR generator generator

optimizers optimizers

new new component component modified modified component component unmodified component

9

A Trace-based Java JIT Compiler Retrofitted from a Method-based Compiler

© 2011 IBM Corporation

IBM Research - Tokyo

Our Trace-JIT Architecture Java VM

interpreter interpreter

trace trace dispatcher dispatcher

Java design classstack libraries compatible with interpreter

execution events Tracing runtime

trace trace selection selection engine engine trace (Java bytecode)

garbage collector

hash hash map map (e.g. compiled code address)

cache code cache tocode reduce overhead in JIT ↔ interpreter transitions compiled code

Trace-JIT

trace trace side side exit exit elimination elimination

code code generator generator

IR IR generator generator

optimizers optimizers

new new component component modified modified component component unmodified component

10

A Trace-based Java JIT Compiler Retrofitted from a Method-based Compiler

© 2011 IBM Corporation

IBM Research - Tokyo

Our Trace-JIT Architecture Java VM

interpreter interpreter

trace trace dispatcher dispatcher

garbage collector class libraries

execution events Tracing runtime

trace trace selection selection engine engine trace (Java bytecode)

hash hash map map

code code cache cache

(e.g. compiled code address)

compiled code Trace-JIT

trace trace side side exit exit elimination elimination

code code generator generator

IR IR generator generator

optimizers optimizers

new new component component modified modified component component unmodified component

11

A Trace-based Java JIT Compiler Retrofitted from a Method-based Compiler

© 2011 IBM Corporation

IBM Research - Tokyo

Technical challenge in reusing a method-based compiler for trace-JIT Scope mismatch problem In method-JIT, – local variables must be dead at the start and the end of compilation scope In trace-JIT – local variables may live at the start and the end of compilation scope Live range of local variables does not match with compilation scope in trace-JIT

12

A Trace-based Java JIT Compiler Retrofitted from a Method-based Compiler

© 2011 IBM Corporation

IBM Research - Tokyo

Solving the scope mismatch problem dead store elimination (DSE) as an example Is this dead store? (no use in the trace)

void prepend(e) { p = head; do { tail = p; p = p->next; } while (p != NULL); tail->next = e; e->next = NULL; }

13

No!

compilation scope (= trace)

we analyze outside the compilation scope to identify liveness at the end of compilation scope

A Trace-based Java JIT Compiler Retrofitted from a Method-based Compiler

© 2011 IBM Corporation

IBM Research - Tokyo

Analyze outside the compilation scope We identify all live variables at each compilation scope boundary point – trace head, trace exit points For each boundary point, we analyze the method that includes the point – mostly in live range analyzer and use-def analyzer in the framework, not in each optimizer entry

method 1

method 2

method 3

method 4

Web Container

JavaServer Pages

Enterprise JavaBeans

JDBC Driver

exit

ok?

error!

search for a handler

log? exit print

ok?

exit

error!

search for a bean

log? exitprint

exit

ok?

error!

ok?

exit

error!

create an SQL

send the SQL to DB

log? exit print

log? exitprint

exit 14

A Trace-based Java JIT Compiler Retrofitted from a Method-based Compiler

© 2011 IBM Corporation

IBM Research - Tokyo

Our Trace-JIT Architecture Java VM

interpreter interpreter

trace trace dispatcher dispatcher

garbage collector class libraries

execution events Tracing runtime

trace trace selection selection engine engine trace (Java bytecode)

hash code hash map map code cache cache Apply simple one-path value propagation to exploit simple (e.g. compiled code address) topologies of traces compiled code

Trace-JIT

trace trace side side exit exit elimination elimination IR IR generator generator

It removes potential side exits to reduce IR tree size and code code generator generator compilationnew time component new component optimizers optimizers

modified modified component component unmodified component

15

A Trace-based Java JIT Compiler Retrofitted from a Method-based Compiler

© 2011 IBM Corporation

IBM Research - Tokyo

Outline Motivation Background Trace-JIT Architecture Performance Evaluation Future work and Summary

16

A Trace-based Java JIT Compiler Retrofitted from a Method-based Compiler

© 2011 IBM Corporation

IBM Research - Tokyo

Performance Evaluation Hardware: IBM BladeCenter JS22 – 4 cores (8 SMT threads) of POWER6 4.0GHz – 16 GB system memory Software: – AIX 6.1 – Method-JIT: IBM JDK for Java 6 (32 bit) – Trace-JIT: Our Trace-JIT based on the same IBM JDK • used only standard optimization level (-Xjit:optlevel=warm) • 512 MB Java heap with large page enabled • generational garbage collector (gencon)

Benchmark: – DaCapo benchmark suite 9.12

17

A Trace-based Java JIT Compiler Retrofitted from a Method-based Compiler

© 2011 IBM Corporation

IBM Research - Tokyo

Steady state performance 1.4 1.2

our trace-JIT

1.0 0.8 0.6 0.4 0.2

x lu se ar ch pm d su nf lo w to m tra c de a t be an s xa l ge an om ea n

de

n

lu in

jy th o

h2

fo p

0.0 av ro ra ba tik ec lip se

relative performance over method-JIT.

higher is faster

method-JIT

Trace-JIT was 22% slower to 26% faster than method-JIT 18

A Trace-based Java JIT Compiler Retrofitted from a Method-based Compiler

© 2011 IBM Corporation

IBM Research - Tokyo

Execution time breakdown avrora

Native library JIT-compiled code

0.9

0.9

0.8

0.8

0.7 0.6 0.5 0.4 0.3 0.2

0.7 0.6 0.5 0.4 0.3 0.2

0.1

0.1

0.0

0.0

method-JIT trace-JIT

shorter is faster

Runtime

1.0

tomcat

normalized CPU time.

GC

1.0

normalized CPU time.

OS Kernel

normalized CPU time.

jython

1.3 1.2 1.1 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0.0

method-JIT trace-JIT

method-JIT trace-JIT

☺ Trace-JIT often (not always) shows better JITted code performance (blue parts) Trace-JIT incurs larger runtime overhead (orange parts)

19

A Trace-based Java JIT Compiler Retrofitted from a Method-based Compiler

© 2011 IBM Corporation

IBM Research - Tokyo

jython

avrora

tomcat

normalized CPU timenormalized CPU timenormalized CPU time

Execution time breakdown by trace length 0.15 cyclic trace 0.10

linear trace

0.05 0.00

0.40 cyclic trace

0.30 0.20 these 0.10

more inlining effect than method-JIT with inlining

0.00 0.40

cyclic trace larger compilation scope yields ☺ less runtime overhead due to transitionlinear trace ☺ more compiler optimization opportunities

0.30 0.20 0.10 0.00

0

shorter trace 20

linear trace

long traces typically crossed 80+ method boundaries

500

1000

1500

trace length in number of Java bytecodes

A Trace-based Java JIT Compiler Retrofitted from a Method-based Compiler

2000

longer trace © 2011 IBM Corporation

IBM Research - Tokyo

Outline Motivation Background Trace-JIT Architecture Performance Evaluation Future work and Summary

21

A Trace-based Java JIT Compiler Retrofitted from a Method-based Compiler

© 2011 IBM Corporation

IBM Research - Tokyo

Optimization opportunities and challenges Opportunities – Potentially larger compilation scope than method-JIT – Simple control flow • main path of a trace is a very large extended basic block

– Explicit control flow • like method inlining

– More specialization • type specialization, value specialization etc

Challenges – Interaction between trace selection and optimizations • e.g. Loop optimizations

22

A Trace-based Java JIT Compiler Retrofitted from a Method-based Compiler

© 2011 IBM Corporation

IBM Research - Tokyo

Future work: Effective Loop Optimization in trace-JIT More loop optimizations in trace-JIT – backward-branch-based cyclic trace identification is not suitable for loop optimizations need to enhance trace selection algorithm to maximize the optimization opportunities

23

backward branch

exit:

loop preheader (e.g. initialization of loop variable) is not included in a cyclic trace. a cyclic trace

loop:

... iconst_0 istore_1 iload_1 iconst_4 if_icmpge exit iinc 2, 1 iinc 1, 1 goto loop ...

Java code: for (int i=0; i