Design Automation for Streaming Systems - Berkeley Reconfigurable ...

Design Automation for Streaming Systems IA

IB

Eylon Caspi University of California, Berkeley OA

OB

12/2/05

Outline ♦ Streaming for Hardware ♦ From Programming Model to Hardware Model ♦ Synthesis Methodology for FPGA • Streams, Queues, SFSM Logic ♦ Characterization of 7 Multimedia Apps ♦ Optimizations • Pipelining, Placement, Queue Sizing, Decomposition

12/2/05

Eylon Caspi

2

Large System Design Challenges ♦ Devices growing with Moore’s Law

• AMD Opteron dual core CPU: ~230M transistors • Xilinx Virtex 4 / Altera Stratix-II FPGAs: ~200K LUTs ♦ Problems of DSM, large systems

• Growing interconnect delay, timing closure ♦

“Routing delays typically account for 45% to 65% of the total path delays” (Xilinx Constraints Guide)

• Slow place-and-route • Design complexity • Designs do not scale well on next gen. device; must redesign ♦ Same problems in FPGAs

12/2/05

Eylon Caspi

3

Limitations of RTL ♦ RTL = Register Transfer Level ♦ Fully exposed timing behavior

• always @(posedge clk) ... ♦ Laborious, error prone ♦ Unpredictable interconnect delay

• How deep to pipeline? • Redesign on next-gen device ♦ Undermines reuse ♦ Existing solutions

• Modular design • Floorplanning • Physical synthesis • Hierarchical CAD • Latency insensitive communication 12/2/05

Eylon Caspi

4

Streams ♦ A better communication abstraction ♦ Streams connect modules

Stream Memory

Module (compute)

• FIFO buffered channel (queue) • Blocking read • Timing independent (deterministic) ♦ Robust to communication delay

• Pipeline across long distances • Robust to unknown delay Post-placement pipelining ♦ Alternate transport (packet switched NOC) ♦

♦ Flexibly timed module interfaces

• Robust to module optimization (pipeline, reschedule, etc.) ♦ Enhances modular design + reuse 12/2/05

Eylon Caspi

5

Streaming Applications ♦ Persistent compute structure (infrequent changes) ♦ Large data sets, mostly sequential access ♦ Limited feedback ♦ Implement with deep, system level pipelining ♦ E.g. DSP, multimedia

♦ JPEG Encoder: 12/2/05

Eylon Caspi

6

Ad Hoc Streaming ♦ Every module needs streaming flow control • Block if inputs not available, output not ready to receive ♦ Every stream needs queueing • Pipeline to match interconnect delay • Queue to absorb delay mismatch, dynamic rates ♦ Manual implementation, in HDL • Laborious (flow control, queues) • Error prone (deadlock if violate protocol, queue too small) • No automation (pipeline depth, queue choice / width / depth) ♦ Interconnect / queue IP (e.g. OCP / Sonics Bus) • Still no automation 12/2/05

Eylon Caspi

7

Systematic Streaming ♦ Strong stream semantics: Process Networks

• Stream = FIFO channel with (flavor of) blocking read • E.g. Kahn Process Networks, E.g. Dataflow Process Networks (E.A.Lee)

♦ Streams as programming primitive

• Language support hides flow control ♦ Compiler support

• Compiler generated flow control • Compiler controlled pipelining, queue depth, queue impl. • Compiler optimizations (e.g. module merging, partitioning) ♦ Benefits

• Easy, correct, high performance • Portable • Paging / Virtualization is a logical extension (Automatic page partitioning) 12/2/05

Eylon Caspi

8


12/2/05

Eylon Caspi

9

SCORE Model

Stream Computations Organized for Reconfigurable Execution

♦ Application = Graph of stream-connected operators ♦ Operator

= Process with local state

♦ Stream

= FIFO channel, unbounded capacity, blocking read

♦ Segment

= Memory, accessed via streams

Segment

Operator (SFSM)

♦ Dynamics:

• Dynamic I/O rates • Dynamic graph construction

Stream

(omitted in this work)

12/2/05

Eylon Caspi

10

SCORE Programming Model: TDF ♦ TDF = behavioral language for

• SFSM Operators (Streaming Extended Finite State Machine) • Static operator graphs ♦ State machine for • Sequencing, branching

• Firing control ♦ Firing semantics

• In state X, wait for X’s inputs, then evaluate X’s action i

j state foo (i, j): o = i + j; goto bar; }

12/2/05

o

Eylon Caspi

11

SCORE / TDF Process Networks ♦ A process from M inputs to N outputs, unified stream type S (i.e. SM→SN) ♦ SFSM = (Σ, σ0, σ, R, fNS, fO) • Σ = Set of states • σ0 ∈ Σ = Initial state • σ∈Σ = Present state • R ⊆ (Σ × SM) = Set of firing rules • fNS : R→Σ = Next state function • fO : R→SN = Output function ♦ Similar to dataflow process networks [Lee+Parks, IEEE May ‘95], but with stateful actors 12/2/05

Eylon Caspi

12

Related Streaming Models ♦ Streaming Models • Kahn PN, DFPN, BDF, SDF, CSDF, HDF, StreamsC, YAPI, Catapult C, SHIM

♦ Streaming Platforms • Pleiades, Philips VSP, Imagine, TRIPS ♦ How do we differ? • Stateful processes • Deterministic • Dynamic dataflow rates (FSM nodes) • Direct synthesis to hardware • Bounded Buffers 12/2/05

Eylon Caspi

13

Streaming Platforms ♦ FPGA

(this work)

♦ Paged FPGA • Page = fixed size partition,

connected by streams • Stylized page-to-page interconnect • Hierarchical PAR

♦ Paged, Virtual FPGA (SCORE) • Time shared pages • Area abstraction (virtually large) ♦ Multiprocessor on Chip ♦ Heterogeneous 12/2/05

Eylon Caspi

14

The Compilation Problem Programming Model: TDF

Execution Model: FPGA

• Communicating SFSMs

• Single circuit / configuration

- unrestricted size, # IOs, timing

- one or more clocks

• Unbounded stream buffering

• Fixed size queues

memory segment

TDF operator

Compile

stream 12/2/05

FPGA

Big semantic gap

Eylon Caspi

15

The Semantic Gap ♦ Semantic gap between TDF, HW ♦ Need to bind:

• • • • • • •

Compile

FPGA

Stream protocol Stream pipelining Queue implementation Queue depths SFSM synthesis style (behavioral synthesis) Memory allocation Primary I/O

♦ SCORE device binds some implementation decisions (custom hardware), raw FPGA does not ♦ Want to characterize cost of implementation decisions 12/2/05

Eylon Caspi

16


12/2/05

Eylon Caspi

17

Compilation Tool Flow Application • Local optimization • System optimization • Queue sizing

TDF

tdfc

• Pipeline extraction • SFSM partitioning / merging • Pipelining

• Generate flow ctl, streams, queues

Verilog

Synplify

• Behavioral Synthesis • Retiming

EDIF (Unplaced LUTs, etc.)

Xilinx ISE

• Slice packing • Place and route

Device Configuration

Bits 12/2/05

Eylon Caspi

18

Wire Protocol for Streams ♦

D = Data,

V = Valid,

B = Backpressure

♦

Synchronous transaction protocol

• Producer asserts V when D ready, Consumer deasserts B when ready • Transaction commits if (¬B ∧ V) at clock edge • Encode EOS E as extra D bit (out of band, easy to enqueue)

Clk D (Data),

Producer

E (EOS)

V (Valid)

D

Consumer

B (Backpressure)

12/2/05

Eylon Caspi

V B

19

Operator Firing ♦ In state X, fire if

• Inputs desired by X are ready (Valid, EOS) • Outputs emitted by X are ready (Backpressure) ♦ Firing guard / control flow

• if (iv && !ie && !ob) begin ib=0; ov=1; ...

end

id,e iv ib

Op

od,e ov ob

♦ Subtlety: master, slave

• Operator is slave ♦

To synchronize streams, (1) wait for flow control in, (2) fire / emit out

• Connecting two slaves would deadlock • Need master (queue) between every pair of operators 12/2/05

Eylon Caspi

20

SFSM Synthesis

B V E D

Control Data registers Stream I/O

FSM ♦ Implemented as Behavioral Verilog, using state ‘case’ in FSM and DP

Datapath For State 1

Datapath For State 2

♦ FSM handles firing control, branching ♦ FSM sends state to DP ♦ DP sends bool. flags to FSM

Datapath

12/2/05

21 B V E D

FSM Module, Firing Control TDF:

Verilog FSM Module:

12/2/05

foo (input input output { state one ... }

unsigned[16] x, unsigned[16] y, unsigned[16] o) (x, eos(y)) : o=x+1;

module foo_fsm (clock, reset, x_e, x_v, x_b, y_e, y_v, y_b, o_e, o_v, o_b, state, statecase); ... always @* begin x_b_=1; y_b_=1; o_e_=0; o_v_=0; state_reg_ = state_reg; Default statecase_ = statecase_stall; did_goto_ = 0; case (state_reg) state_one: begin if (x_v && !x_e && y_v && y_e && !o_b) statecase_ = statecase_1; x_b_=0; y_b_=0; o_v_=1; o_e_=0; end ... end // always @* Eylon Caspi endmodule // foo_fsm

is stall

Firing condition(s) for state one begin

Stream flow ctl for state one 22

Data-Path Module TDF:

Verilog Data-path Module:

12/2/05

foo (input input output { state one ... }

unsigned[16] x, unsigned[16] y, unsigned[16] o) (x, eos(y)) : o=x+1;

module foo_dp (clock, reset, x_d, y_d, o_d, state, statecase); ... always @* begin o_d_=16’bx; Default did_goto_ = 0; case (state) state_one: begin if (statecase_ == statecase_1) o_d_ = (x_d + 1’d1); end ... end // always @* endmodule // foo_dp

Eylon Caspi

begin

is stall

Firing condition(s) for state one Data-path for state one

23

Stream Buffers (Queues) ♦ Systolic • Cascade of depth-1

stages (or depth-N)

♦ Shift register • Put: shift all entries • Get: tail pointer ♦ Circular buffer • Memory with

head / tail pointers

12/2/05

Eylon Caspi

24

Enabled Register Queue ♦ Systolic, depth-1 stage

iB

♦ 1 state bit (empty/full) = V

iV

iD

oV

oD

en

♦ Shift in data unless: • Full and downstream not ready to consume queued element

♦ Area  1 FF per data bit • On FPGA  1 LUT cell per data bit • Depth-1 (single stage) nearly free,

oB

since FFs pack with logic

♦ Speed: as fast as FF • But combinationally connects producer + consumer via B 12/2/05

Eylon Caspi

25

Xilinx SRL16 ♦ SRL16 = Shift register of depth 16 in one 4-LUT cell • Shift register of arbitrary width: parallel SRL16, arbitrary depth: cascade SRL16

♦ Improve queue density by 16x

4-LUT Mode 12/2/05

SRL16 Mode Eylon Caspi

26

Shift Register Queue ♦ State: empty bit + capacity counter

iB iV iD,E

♦ Data stored in shift register

• In at position 0 • Out at position Address ♦ Address = number of stored elements minus 1 ♦ Synplify infers SRL16E from Verilog array

0

+1

-1

FSM en

Address

Empty

Shift Reg

• Parameterized depth, width ♦ Flow control

• ov = (State==Non-Empty) • ib = !(Address==Depth-1)

NonEmpty

=Depth-1

=0

full

♦ Performance improvements

zero

• Registered data out • Registered flow control • Specialized, pre-computed fullness

12/2/05

oB oV oD,E Eylon Caspi

27

SRL Queue with Registered Data Out ♦ Registered data out

iB iV iD,E

• od (clock-to-Q delay) • Non-retimable

♦ Data output register extends shift register ♦ Bypass shift register when queue empty ♦ 3 States ♦ Address = number of stored elements minus 2 ♦ Flow control

0

+1

-1

Empty

en

Address

Shift Reg

One

=Depth-2 More

full zero

=0 Data Out

• ov = !(State==Empty) • ib = (Address ==Depth-2)

12/2/05


28

SRL Queue with Registered Flow Ctl. ♦ Registered flow ctl.

iB iV iD,E

• ov (clock-to-Q delay) • ib (clock-to-Q delay) • Non-retimable

0

♦ Flow control

• ov_next = !(State_next

en

Address

• ib_next =

♦ Based on precomputed fullness

-1

Empty

==Empty)

(Address_next ==Depth-2)

+1

Shift Reg

One

More

• full_next =

full_next full zero

=Depth-2

=0 Data Out

(Address_next ==Depth-2) oB oV oD,E

12/2/05

Eylon Caspi

29

SRL Queue with Specialized, Pre-Computed Fullness ♦ Speed up critical full pre-computation by special-casing full_next for each state

iB iV iD,E

♦ Flow control

• ov_next = !(State_next ==Empty) • ib_next = full_next

♦ zero pre-computation is less critical ♦ Result

• >200MHz unless very

large (e.g. 128 x 128) • All output delays are clock-to-Q • Area ≈ 3 x (SRL16E area) 12/2/05

0

+1

-1

Empty

en

Address

Shift Reg

One

full =Depth-3

=0

More

zero

Data Out


30

SRL Queue Speed

12/2/05

Eylon Caspi

31

SRL Queue Area

12/2/05

Eylon Caspi

32

Page Synthesis ♦ Page = Cluster of Operator(s) + Queues ♦ SFSMs

Op 1 FSM

• One or more per page • Further decomposed into

Op 1 Datapath

FSM, data-path

♦ Page Input Queues

• Deep • Drain pipelined page-topage streams before reconfiguration

Page Input Queue(s)

Queue

♦ In-page Queues

• Shallow ♦ Separately Synthesizable Modules

Op 2 FSM

• Separate characterization • Consider custom resources 12/2/05

Eylon Caspi

Op 2 Datapath 33

Page Synthesis ♦ Module Hierarchy • Individual SFSMs

(combinational cores)

Op 1 FSM

• Local / output queues

Op 1 Datapath

• Operators and

local / output queues

• Input queues • Page

Page Input Queue(s)

Queue

Op 2 FSM 12/2/05

Eylon Caspi

Op 2 Datapath 34


12/2/05

Eylon Caspi

35

Tool Flow, Revisited ♦

Separate compilation for application, SFSMs • Page • SFSM • Datapath • FSM

Application

tdfc

Verilog

Synplify EDIF

Tool Options • Identical queuing for every stream (SRL16 based, depth 16) • I/O boundary regs (for Xilinx static timing analysis) • Synplify 8.0 • Target 200MHz • Optimize: FSM, retiming, pipelining • Retain monolithic FSM encodings

(Unplaced LUTs, etc.)

Xilinx ISE Device Configuration

Bits 12/2/05

Eylon Caspi

• ISE 6.3i • Constrain to minimum square area, at least max slice packing + 20%, expand if fail PAR • Device: XC2VP70 -7 36

PAR Flow for Minimum Area EDIF

Constraints

♦

EDIF netlist from Synplify

♦

Constraints file

• Page area • Target Period

ngdbuild

♦

ngdbuild:

• Convert netlist EDIF → NGD

map

♦

map:

• Pack LUTs, MUXes, etc. into slices Ok? yes

no

trce

Target packed timing

par Ok? yes

trce

Target packed slices

no

Target 1 extra row/col

♦

trce: (pre-PAR)

• Static timing analysis, logic only ♦

par:

• Place and route

♦

trce: (post-PAR)

• Static timing analysis

37

SCORE Applications ♦ 7 Multimedia Applications / 279 Operators • MPEG, JPEG, Wavelet, IIR • Written by Joe Yeh

• Mostly feed-forward streaming • Constant consumption / production ratios, except compressors (ZLE, Huffman) Streams Application

Speed

Area

%Area

%Area

SFSMs

Segments

In

Local

Out

(MHz)

(4-LUT cells)

FSMs

Queues

IIR

8

0

1

7

1

166

1,922

3.4%

27.7%

JPEG Decode

9

1

1

41

8

47

7,442

7.0%

28.7%

JPEG Encode

11

4

8

42

1

57

6,728

7.5%

36.9%

MPEG Encode IP

80

16

6

231

1

47

41,472

5.5%

39.7%

MPEG Encode IPB

114

17

3

313

1

50

65,772

5.2%

40.5%

Wavelet Encode

30

6

1

50

7

106

8,320

10.1%

32.0%

Wavelet Decode

27

6

7

49

1

109

8,712

8.5%

29.6%

Total

279

50

27

733

20

140,368

5.9%

38.3%

12/2/05

Eylon Caspi

38

Page Area DCT, IDCT

♦

87% of SFSMs are smaller than 512 LUTs — by design

♦

FSMs small

12/2/05

♦ Datapaths dominate in most large pages Eylon Caspi

39

Page Speed

43%

47%

♦

FSMs (flow control) are fast, never critical

♦

Queues are critical for 1/3 fastest pages

12/2/05

Eylon Caspi

10%

♦ Datapaths dominate 40


12/2/05

Eylon Caspi

41

Improving Performance, Area ♦ Local (module) Optimization • Traditional compiler optimization ♦

(const folding, CSE, etc.)

• Datapath pipelining / loop scheduling • Granularity transforms ♦

(composition / decomposition)

♦ System Level Optimization • Interconnect pipelining • Shrink / remove queues • Area-time transformations ♦ 12/2/05

(rate matching, serialization, parallelization) Eylon Caspi

42

Pipelining With Streams ♦ Datapath pipelining • Add registers at output (or input) • Retime into place ♦ Harder in practice (FSM, cycles) • Add registers at strategic locations • Rewrite control • Avoid violating communication protocol ♦ Stream pipelining • Add registers on streams • Retime into datapath • Modify queues, not processes 12/2/05

Eylon Caspi

DP

FSM DP

FSM DP

43

Logic Pipelining ♦ Add L pipeline registers to D, V ♦ Retime backwards

• This pipelines feed-forward parts of producer’s data-path ♦ Stale flow control may overflow queue (by L) ♦ Modify queue to emit back-pressure when empty slots ≤ L ♦ No manual modification of processes Retime

Producer

12/2/05

D

(Data)

D

V

(Valid)

B

(Backpressure)

Queue with L Reserve

Eylon Caspi

V

Consumer

B 44

Logic Relaying + Retiming ♦ Break-up deep logic in a process ♦ Relay through enabled register queue(s) ♦ Retime registers into adjacent process

• This pipelines feed-forward parts of process’s datapath • Can retime into producer or consumer ♦ No manual modification of processes Retime

Producer

D

D

V

V en

B 12/2/05

B Eylon Caspi

D

Original Queue

V

Consumer

B 45

Benefits, Limitations ♦ Benefits • Simple to implement, relies only on retiming • Sufficient for many cases, e.g. DCT, IDCT ♦ Limitations • Feed-forward only (weaker than loop sched.) • Resource sharing obfuscates retiming opportunities ♦ Extends to interconnect pipelining • Do not retime into logic — register placement only • Also pipeline B, modify queue

12/2/05

Eylon Caspi

46

Pipelining Configuration Retime Retime

D V

Queue with Reserve

D

D

V

V

Lp B

Retime

SFSM

D

D

D

V

V

V

en

en

B

B

B

B

B

Input side Logic Relaying

Logic Pipelining

Output side Logic Relaying

Li

Lp

Lr

♦ Pipeline depth parameters: Li+Lp+Lr ♦ Uniform pipelining: same depths for every stream 12/2/05

Eylon Caspi

47

Speedup from Logic Pipelining

Enabled Regs (Lr)

12/2/05

Eylon Caspi

D FFs (Lp)

48

Expansion from Logic Pipelining

Enabled Regs (Lr)

12/2/05

Eylon Caspi

D FFs (Lp)

49

Some Things Are Better Left Unpipelined ♦ Page speedup:

♦ Page expansion:

♦ Initially fast pages should not be pipelined 12/2/05

Eylon Caspi

50

Page Specific Logic Pipelining ♦ Separate pipelining of each SFSM ♦ Assumption: application speed = slowest page speed

• Critical Page ♦ Repeatedly improve slowest page until no further improvement is possible ♦ Page improvement heuristics

• Greedy Lr : • Greedy Lp : • Max :

Add one level of pipelining in 0+0+Lr Add one level of pipelining in 1+Lp+0 Pipeline to best page speed (brute force)

♦ Greedy heuristics may end early

• Non-monotonicity: adding a level of pipelining may slow page 12/2/05

Eylon Caspi

51

Speedup from Page Specific

Enabled Regs (Lr)

12/2/05

D FFs (Lp)

Eylon Caspi

52

Expansion from Page Specific

Enabled Regs (Lr)

12/2/05

D FFs (Lp)

Eylon Caspi

53

Interconnect Delay ♦ Critical routing delay grows with circuit size • Routing delay for an application: avg. 45% - 56% • Routing delay for its slowest page: avg. 40% - 50% • Ratio (appl. to slowest page): avg. 0.99x - 1.34x ♦

Averaged over 7 apps / varies with logic pipelining

♦ Modular design helps • Retain critical routing delay of page, not application • Page-to-page delays (streams) can be pipelined

12/2/05

Eylon Caspi

54

Interconnect Pipelining ♦ Add W pipeline registers to D, V, B

• Mobile registers for placer

• Not retimable

♦ Stale flow control may overflow queue (by 2W)

• Staleness = total delay on B-V feedback loop = 2W ♦ Modify downstream queue to emit back-pressure when empty slots ≤ 2W Long distance

Producer

D

(Data)

V

(Valid)

D

Queue with 2W Reserve

B (Backpressure) 12/2/05

V

Consumer

B Eylon Caspi

55

Speedup from Interconnect Pipelining

12/2/05

Eylon Caspi

56

Speedup from Interconnect Pipelining, No Area Constraint

12/2/05

Eylon Caspi

57

Expansion from Interconnect Pipelining, No Area Constraint

12/2/05

Eylon Caspi

58

Interconnect Register Allocation ♦ Commercial FPGAs / tool flows

• • • •

No dedicated interconnect registers Allocation: add to netlist, slice pack, place-and-route If pack registers with logic  limited register mobility If pack registers alone  area overhead

♦ Better: Post-placement register allocation

• Weaver et al., “Post-Placement C-Slow Retiming for the Xilinx • • • •

Virtex FPGA,” FPGA 2003 Allocation: PAR, c-slow, retime, scavenge registers, reroute No area overhead (scavenge registers from existing placement) Better performance, since know routing delay Modification for streaming: ♦

12/2/05

PAR, pipeline, retime, scavenge registers, reroute, modify queue depths (configuration specialization) Eylon Caspi

59

Throughput Modeling ♦ Pipelining feedback loops may reduce throughput (tokens per clock period) • Which loops / streams are critical? ♦ Throughput model for PN • Feedback cycle C with M tokens, N pipe delays, has token period: TC = M/N

• Overall token period: T = maxC {TC } • Available slack: CycleSlackC = (T - TC) • Generalize to multi-rate, dynamic rate by unfolding

TC1 = 3

TC2 = 2

equivalent single-rate PN 12/2/05

Eylon Caspi

60

Throughput Aware Optimizations ♦ Throughput aware placement • Adapt [Singh+Brown, FPGA 2002] • Stream slack: Te = maxC s.t. e∈C {TC } • Stream net criticality: crit = 1 - ((T - Te) / T) ♦ Throughput aware pipelining • Pipeline stream w/o exceeding slack • Pipeline module s.t. depth does not exceed any output stream slack

♦ Pipeline balancing (by retiming) ♦ Process Serialization • Serial arithmetic for process with low throughput, high slack 12/2/05

Eylon Caspi

61

Stream Buffer Sizing ♦ Fixed size buffers in hardware

• For minimum area • For performance

(want smallest feasible queue) (want deep enough to avoid stalls from producer-consumer timing mismatch)

♦ Semantic gap

• Buffers are unbounded in TDF, • •

• 12/2/05

bounded in HW Small buffer may create artificial deadlock (bufferlock) Theorem: memory bound is undecidable for a Turing complete process network In practice, our buffering requirements are small Eylon Caspi

Bounded x= y=

x

=x

y

=y

Unbounded x= y=

x

=x

y

=y

62

Dealing with Undecidability ♦ Handle unbounded streams

• Buffer expansion [Parks ‘95] ♦

Detect bufferlock, expand buffers

• Hardware implementation ♦

Buffer expansion = rewire to another queue

♦

Storage in off-chip memory or queue bank

♦ Guarantee depth bound for some cases

• User depth annotation • Analysis ♦

Identify compatible SFSMs with balanced schedules

♦ Detect bufferlock and fail 12/2/05

Eylon Caspi

63

Interface Automata

de Alfaro + Henzinger, Symp. Found. SW Eng. (FSE) 2001

♦ A finite state machine that transitions on I/O actions

• Not input-enabled (not every I/O on every cycle) ♦ G = (V, E, Ai, Ao, Ah, Vstart)

• • • •

Ai Ao Ah E

= = = ⊂

input actions output actions internal actions V x (Ai ∪ Ao ∪ Ah) x V

x? (in CSP notation) y! ” z; ” (transition on action)

♦ Execution trace = (v, a, v, a, …)

s st; t

S

s?

12/2/05

o!

s o

F

f?

t

f

T’

S’

sf; f

T

t?

(non-deterministic branching)

F’

o! Eylon Caspi

select

o 64

Automata Composition ♦ Composition ~ product FSM with synchronization (rendezvous) on common actions Composition edges:

x

A

y

B

x?

z A

(I)

A’

step A on unshared action

(ii) step B on unshared action (iii) step both on shared action

y! x?

Compatible Composition → Bounded Memory

12/2/05

AB

B

x? A’B

AB

y; z!

y? B’

z!

z! AB’

A’B’

A’B

y! y? z! x?

z! AB’

y? A’B’

x?

y!

Automata Composition

Direct Product

65

Stream Buffer Bounds Analysis ♦ Given a process network, find minimum buffer sizes to avoid bufferlock ♦ Buffer (queue) is also automaton

x

i

A

♦ Symbolic Park’s algorithm

o

B

y

• Compose network using •

arbitrary buffer sizes If deadlock, try larger sizes

x

i

A

♦ Practical considerations: avoiding state explosion

• Multi-action automata • Know which streams to expand first • Compose pairwise in clever order ♦

12/2/05

0

Eylon Caspi

i? 1

o!

Q

o

B

y

i?

Composition is associative

• Cull states reaching deadlock • Partition system

Q x

2

o! i?o! 66

SFSM Decomposition ♦

(Partitioning)

Why decompose

• To improve locality • To fit into custom page resources ♦

Decomposition by state clustering

• 1 state (i.e. 1 cluster) active at a time ♦

Cluster states to contain transitions

• Fast local transitions, slow external trans. • Formulation: minimize cut of transition probability under area, I/O constraints

♦

Similar to:

• • • • 12/2/05

VLIW trace scheduling [Fisher ‘81] FSM decomp. for low power [Benini/DeMicheli ISCAS ‘98] GarpCC HW/SW partitioning [Callahan ‘00] VM/cache code placement Eylon Caspi

State flow Data flow 67

Early SFSM Decomposition Results ♦ Approach 1: Balanced, multi-way, min-cut

• Modified Wong FBB [Yang+Wong, ACM ‘94] • Edge weight is mix: c*(transition probability) + (1-c)*(wire bits) • Poor at simultaneous I/O constraint + cut optimization ♦ Approach 2: Spectral order + Extent cover

• Spectral ordering clusters connected components in 1D ♦

Minimize squared weighted distance, weight is mix (as above)

• Then choose area + I/O feasible extents [start, end] •

using dynamic programming Effective for partitioning to custom page resources

♦ Under 2% external transitions

• Amdahl’s law: few slow transitions ⇒ small performance loss • Achievable with either approach 12/2/05

Eylon Caspi

68

Summary ♦ Streaming addresses large system design challenges

• Growing interconnect delay • Flexibly timed module interfaces

• Design complexity • Reuse

♦ Methodology to compile SCORE applications to Virtex-II Pro

• Language + compiler support for streaming ♦ Characterized 7 applications on Virtex-II-Pro

• Queue area ~38% ♦

• Flow control FSM area ~6%

Improve by merging SFSMs, eliminating queues

♦ Stream pipelining

• For logic

• For interconnect

♦ Stream based optimizations

• Pipelining • Queue sizing • Placement • Serialization 12/2/05

• Module Merging • Partitioning

Eylon Caspi

69

Supplemental Material

12/2/05

Eylon Caspi

70

TDF ∈ Dataflow Process Networks ♦ Dataflow Process Networks [Lee+Parks, IEEE May ‘95]

• Process enabled by set of firing rules: R = { r1, r2, … , RK } • Firing rule = set of input patterns: ri = ( ri,1, ri,2 , … , ri,M ) • Feedback arc for state • Firing rule(s) per state

state

♦ DF process for a TDF operator: process

Patterns match state + input presence ♦ E.g. for state σ : rσ = ( [σ], rσ,1, rσ,2 , … ) ♦ Patterns: rσ,j = [*] if input j is in input signature of state σ rσ,j = ⊥ if input j is not in input signature of state σ ♦

• Single firing rule per state = DFPN sequential firing rules • Multiple firing rules per state translate the same way, with restrictions to retain determinism

12/2/05

Eylon Caspi

71

SFSM Partitioning Transform ♦

Only 1 partition active at a time

• Transform to activate via streams ♦

A

C

B

D

New state in each partition: “wait”

• Used when not active • Waits for activation

from other partition(s) • Has one input signature (firing rule) per activator

♦

Firing rules are not sequential, but determinism guaranteed

• Only 1 possible activator ♦

A

Activation streams from given source to given dest. partitions can be merged + binary-encoded

12/2/05

Eylon Caspi

B

{C,D}

{A,B}

Wait

Wait

AB

CD

{C,D}

{A,B}

C

D

72

Virtual Paged Hardware (SCORE) ♦

Compute model has unbounded resources

♦

Paging

♦

Efficient virtualization

• Programmer does not target a particular device size • “Compute pages” swapped in/out (like virtual memory) • Page context = thread (FSM to block on stream access) • Amortize reconfiguration cost over an entire input buffer • Requires “working sets” of tightly-communicating pages to fit on device

compute pages

buffers

Transform

Quantize

RLE

Encode

73