Software Techniques for Programmable Data Plane ... - Open-NFP

Software Techniques for Programmable Data Plane Virtualization David Hancock ([email protected])

Flux Research Group

©2017 Open-NFP

1

Overview Motivation Approach Controller (“Control Plane Hypervisor”) HyPer4.p4 (“Data Plane Hypervisor”) Demo Wrap-up / Questions

©2017 Open-NFP

Flux Research Group

2

Motivation Programmable data planes ▪  How dynamic are they? Can we insert functionality on demand without disrupting important flows?

Multi-tenant environments Virtualization ▪  Isolation ▪  Abstraction •  Composition •  Introspection •  Standard functionality ©2017 Open-NFP

Flux Research Group

3

Approach Software approach to expanding PDP device feature set HyPer4.p4: “Data Plane Hypervisor” Controller: “Control Plane Hypervisor” ▪  Slice definition / provisioning ▪  Slice management •  •  •  • 

Compiler Loader Rule interpreter Composer

©2017 Open-NFP

Flux Research Group

4

func1.p4

func.on configura.on

func.on defini.on

HyPer4 func2.p4 func3.p4 compiler HP4 defini.on

HyPer4.p4

func1 rules func2 rules func3 rules

rule interpreter

controller

composer

HP4 configura.on

HyPer4 table entries HyPer4 table entries HyPer4 table rules

P4 device HyPer4 M-A table

M-A table

M-A table

... parser

©2017 Open-NFP

deparser

5

P4

P4 device

©2017 Open-NFP

Flux Research Group

6

P4

func.on defini.on

func.p4

P4 device func M-A table

M-A table

M-A table

... parser

©2017 Open-NFP

deparser

Flux Research Group

7

P4

func.on defini.on

func.on configura.on

func.p4

func table entries func table entries func table rules

P4 device func M-A table

M-A table

M-A table

... parser

©2017 Open-NFP

deparser

Flux Research Group

8

HyPer4

HP4 defini.on

HyPer4.p4

P4 device HyPer4 M-A table

M-A table

M-A table

... parser

©2017 Open-NFP

deparser

Flux Research Group

9

func.on defini.on

HyPer4 func1.p4 func2.p4 func3.p4 compiler HP4 configura.on

HP4 defini.on

HyPer4 table entries HyPer4 table entries HyPer4 table rules

HyPer4.p4

P4 device HyPer4 M-A table

M-A table

M-A table

... parser

©2017 Open-NFP

deparser

Flux Research Group

10

func1.p4

func.on configura.on

func.on defini.on

HyPer4 func2.p4 func3.p4 compiler

func1 rules func2 rules func3 rules

rule interpreter HP4 configura.on

HP4 defini.on

HyPer4 table entries HyPer4 table entries HyPer4 table rules

HyPer4.p4

P4 device HyPer4 M-A table

M-A table

M-A table

... parser

©2017 Open-NFP

deparser

Flux Research Group

11

func1.p4

func.on configura.on

func.on defini.on

HyPer4 func2.p4 func3.p4 compiler HP4 defini.on

HyPer4.p4

func1 rules func2 rules func3 rules

rule interpreter

controller

composer

HP4 configura.on

HyPer4 table entries HyPer4 table entries HyPer4 table rules

P4 device HyPer4 M-A table

M-A table

M-A table

... parser

©2017 Open-NFP

deparser

12

HyPer4.p4: Packet Tagging action a_set_context(vdev_ID, vingress) { modify_field(meta.vdev_ID, vdev_ID); modify_field(meta.vingress, vingress); } table tset_context { reads { standard_metadata.ingress_port : exact; } actions { a_set_context; } } control setup { if (meta.vdev_ID == 0) { apply(tset_context); } // … ©2017 Open-NFP

Flux Research Group

13

func1.p4

func.on configura.on

func.on defini.on

HyPer4 func2.p4 func3.p4 compiler HP4 defini.on

HyPer4.p4

func1 rules func2 rules func3 rules

rule interpreter

controller

composer

HP4 configura.on

HyPer4 table entries HyPer4 table entries HyPer4 table rules

P4 device HyPer4 M-A table

M-A table

M-A table

... parser

©2017 Open-NFP

deparser

14

HyPer4 Controller

©2017 Open-NFP

Flux Research Group

15

Northbound API Administrator ▪  create_device ▪  create_slice ▪  grant_lease ▪  list_devices ▪  list_slices ▪  reset_device ▪  revoke_lease ©2017 Open-NFP

\



Flux Research Group

16

Northbound API Slice Manager ▪  create_virtual_device ▪  destroy_virtual_device ▪  interpret ▪  interpret_file

\ ▪  lease insert | append | remove | replace [args] ▪  list_devs ▪  list_vdev ▪  list_vdevs ©2017 Open-NFP

Flux Research Group

17

Southbound API Device class defines interface ▪  send_command •  do_table_add •  do_table_modify •  do_table_delete

▪  various multicast-related methods

Device subclasses implement ▪  Bmv2-SSwitch ▪  Agilio

©2017 Open-NFP

Flux Research Group

18

Compiler/Interpreter Task: P4 programs à “object-code-like” HyPer4 rulesets à rule interpreter guidance Subtasks: ▪  metadata field addressing •  All metadata fields consolidated into HyPer4.p4’s meta.data field •  Reads/writes done with bitmasks and bitshifts applied to meta.data

▪  parsing ▪  matching ▪  actions ▪  control flow ©2017 Open-NFP

Flux Research Group

19

Compiler/Interpreter default: extract 20 B

Subtasks: Parsing parser start { extract(ethernet); return select(ethernet.EtherType) { 0x0806: parse_arp; default : ingress; } } parser parse_arp { extract(arp); return ingress; } ©2017 Open-NFP

parser parse_control

0x0806 in PR[12-13]: extract 42 B

first pass

second pass

parse_select:1-20 default

import_PR 20 or 42 B imported

pipeline_config valid_headers: 0b11 or 0b10 [ethernet][arp]

ingress M+A Flux Research Group

20

Compiler/Interpreter Subtasks: Matching

©2017 Open-NFP

Flux Research Group

21

HyPer4.p4: Matching Exact matching vs arbitrary fields: ▪  Use ternary matching •  rules include mask

▪  Strategy: apply mask to pr.data to isolate relevant bits

©2017 Open-NFP

Flux Research Group

22

HyPer4.p4: Actions match result: set match id, primitive code set_primitive_metadata invoke table specific to primitive & param types

triggered action executes primitive update state more primitives? to next match ©2017 Open-NFP

Flux Research Group

23

Loader “Object code”: tset_pipeline_config a_set_pipeline :[vdev ID] 1: [STDMETA_INGRESS_PORT_EXACT] 0x80[9x00s] 0

t_mod_41 mod_stdmeta_egressspec_stdmeta_ingressport : [vdev ID] 11 4 0&&&0:[MAX PRIORITY]

©2017 Open-NFP

Flux Research Group

24

Loader “Object code”:

virtualized func@on ID parse_state ac@on table (end of “setup”) tset_pipeline_config a_set_pipeline :[vdev ID] 1: [STDMETA_INGRESS_PORT_EXACT] 0x80[9x00s] 0 egress handling mode ID of 1st match table valid headers bitmap (0b1000…: ethernet only) t_mod_41 mod_stdmeta_egressspec_stdmeta_ingressport : [vdev ID] 11 4 0&&&0:[MAX PRIORITY]

©2017 Open-NFP

Flux Research Group

25

Loader “Object code”: tset_pipeline_config a_set_pipeline :[vdev ID] 1: [STDMETA_INGRESS_PORT_EXACT] 0x80[9x00s] 0

table (modify_field) ac@on t_mod_41 mod_stdmeta_egressspec_stdmeta_ingressport : [vdev ID] 11 4 0&&&0:[LOWEST PRIORITY] match ID ternary match ac@on ID virtualized prim priority func@on ID subtype ©2017 Open-NFP

Flux Research Group

26

Loader “Object code”: tset_pipeline_config a_set_pipeline :[vdev ID] 1: [STDMETA_INGRESS_PORT_EXACT] 0x80[9x00s] 0

t_mod_41 mod_stdmeta_egressspec_stdmeta_ingressport : [vdev ID] 11 4 0&&&0:[MAX PRIORITY]

©2017 Open-NFP

Flux Research Group

27

Loader Bmv2-SS API Commands: table_add tset_pipeline_config a_set_pipeline 1 1 => 5 0x80000000000000000000 0

table_add t_mod_41 mod_stdmeta_egressspec_stdmeta_ingressport 1 11 4 0&&&0 => 2147483646

©2017 Open-NFP

Flux Research Group

28

HyPer4.p4: Composition Support End of ingress ▪  Match on vegress ▪  Virtual forwarding? Set egress to ingress port, set virt forwarding flag

Egress ▪  Virt forwarding flag set? •  Match on vegress •  Set next vdev_ID, vingress •  Recirculate with field list that includes key fields

©2017 Open-NFP

Flux Research Group

29

Composition Abstractions virtual network:

vdev chains:

physical interfaces

f1

f2

hp4

f3

●  Very flexible ●  Complicated to implement and manage ©2017 Open-NFP

f1

f1

f2

f2

f3

f3

●  Flexible ●  Simpler to implement and manage

Flux Research Group

30

Chains

ChainLease.interfaces

ChainLease.chain

©2017 Open-NFP

setup f1 f2 f3

phys ingress → (vdev, virt ingress) 1 2 1

1: f1…(n-1)(packet, state) → virt egress 2: virt egress → vdev

2 1 2

1: fn(packet, state) → virt egress 2: virt egress → phys egress

Flux Research Group

31

Egress Conflict Resolution Types of Forwarding Abstractions ▪  return-to-sender: single port ▪  bump-in-the-wire [+ return-to-sender]: two ports ▪  switch: (packet, state) à egress: two+ ports VDev sequence results in (sometimes unintentional) egress overrides Resolution: ▪  Enforce switch at the end ▪  Slice manager indicates preferences with respect to egress modifications for each vdev via controller command option: •  always write to egress_spec •  conditionally write to egress_spec (if void) •  never write to egress_spec ©2017 Open-NFP

Flux Research Group

32

Possibilities Migration Cloning Maintain rulesets separately, attach to compatible vdevs ondemand Automatically evolving functions Linked functions (change to one begets [a]symmetrical change to another across the network)

©2017 Open-NFP

Flux Research Group

33

HyPer4.p4: Clean up Handle virtual networking Egress filtering (avoid broadcast storms) Handle checksum (e.g., IPv4) Resize parsed representation (in case of add_header / remove_header) Write pr.data back to parsed representation ©2017 Open-NFP

Flux Research Group

34

HyPer4.p4: Code Generation P4 compiler / target restriction: ONE reference per table ▪  Want: arbitrary functionality at arbitrary points in the pipeline ▪  Strategy: many functionally redundant copies, differing in name only ▪  Want: configurable support for specified # stages, specified # primitives / stage ▪  Strategy: define .p4 for single stage, single primitive, add tokens directing processing tool to produce desired configuration

©2017 Open-NFP

Flux Research Group

35

HyPer4.p4: Code Generation [+ sloop +] control stage[+X+] { match_[+X+](); // match.p4 [+ nif +] if(meta_ctrl.stage_state != COMPLETE) { switch_primitivetype_[+X+][+Y+](); apply(tstg[+X+][+Y+]_update_state); [+ endnif +] } [+ endsloop +]

©2017 Open-NFP

control stage1 { match_1(); if(meta_ctrl.stage_state != COMPLETE) { switch_primitivetype_11(); apply(tstg11_update_state); if(meta_ctrl.stage_state != COMPLETE) { switch_primitivetype_12(); apply(tstg12_update_state); // … } control stage2 { match_2(); if(meta_ctrl.stage_state != COMPLETE) {

Flux Research Group

36

HyPer4.p4: Objections, Rebuttals Performance Concerns ▪  May be addressed with custom targets ▪  May be addressed with more sensible default parsing behavior and streamlined match-action pipeline ▪  May be addressed by using programmable parser + fixed pipeline, or fixed parser + programmable pipeline ▪  Should not get in the way of exploring high level abstractions made possible by vPDPs

©2017 Open-NFP

Flux Research Group

37

Demo

©2017 Open-NFP

Flux Research Group

38

Wrap-up Motivation Approach Controller (“Control Plane Hypervisor”) HyPer4.p4 (“Data Plane Hypervisor”) ▪  [email protected]:dhankook/hp4-src.git ▪  README.md provides commit used for CoNEXT 2016 paper

Demo

©2017 Open-NFP

Flux Research Group

39

Questions Questions?

©2017 Open-NFP

Flux Research Group

40

HyPer4.p4: Parsing // not shown: header_type definitions header a_t a; header b_t b; parser start { extract(a); return select(a.field_3) { 0x01 : parse_b; default : ingress; } } parser parse_b { extract(b); return ingress; }

a_type header b_type

field_1

field_2

field_4 field_1

field_3 field_5

field_2

field_3

field_4

payload

P4 source includes references to named fields

parsed representation: a | a, b

©2017 Open-NFP

Flux Research Group

41

HyPer4.p4: Parsing Strategy: - 

generic 8-bit header type

- 

ext[100]: array of such headers

ext[0]

ext[1] ext[2]

ext[3]

ext[4] ext[5]

ext[6] ext[7]

ext[8]

ext[10] ext[11]

ext[9]

payload

©2017 Open-NFP

Flux Research Group

42

HyPer4.p4: Parsing Strategy: - 

generic 8-bit header type

- 

ext[100]: array of such headers

ext[0]

- 

metadata fields: #bytes, parse_state

ext[4] ext[5]

ext[6] ext[7]

ext[8]

ext[10] ext[11]

ext[1] ext[2]

ext[9]

ext[3]

payload

©2017 Open-NFP

Flux Research Group

43

HyPer4.p4: Parsing Strategy: - 

generic 8-bit header type

- 

ext[100]: array of such headers

ext[0]

- 

metadata fields: #bytes, parse_state

ext[4] ext[5]

ext[6] ext[7]

- 

Parser: extract default -> 100 bytes

ext[8]

ext[10] ext[11]

-  #bytes guides parser

ext[1] ext[2]

ext[9]

ext[3]

payload

©2017 Open-NFP

Flux Research Group

44

HyPer4.p4: Parsing Strategy: - 

generic 8-bit header type

- 

ext[100]: array of such headers

ext[0]

- 

metadata fields: #bytes, parse_state

ext[4] ext[5]

ext[6] ext[7]

- 

Parser: extract default -> 100 bytes

ext[8]

ext[10] ext[11]

-  #bytes guides parser - 

©2017 Open-NFP

Complex parsing logic moved to Ingress: parse_state may trigger update of #bytes + resubmit

Flux Research Group

ext[1] ext[2]

ext[9]

ext[3]

payload

45

HyPer4.p4: Parsing Strategy: - 

generic 8-bit header type

- 

ext[100]: array of such headers

ext[0]

- 

metadata fields: #bytes, parse_state

ext[4] ext[5]

ext[6] ext[7]

- 

Parser: extract default -> 100 bytes

ext[8]

ext[10] ext[11]

-  #bytes guides parser - 

Complex parsing logic moved to Ingress: parse_state may trigger update of #bytes + resubmit

ext[1] ext[2]

ext[9]

ext[3]

payload

parsed representation: all valid elements of ext -  Inconvenient to work with; copy to single very wide metadata field ©2017 Open-NFP

Flux Research Group

46

HyPer4.p4: Parsing packet parser

ingress

parse 20

©2017 Open-NFP

Flux Research Group

47

HyPer4.p4: Parsing packet parser

parse 20

ingress

#bytes

> 20

©2017 Open-NFP

Flux Research Group

48

HyPer4.p4: Parsing packet parser

parse 20

#bytes

ingress

parse _state

> 20 parse more

©2017 Open-NFP

Flux Research Group

49

HyPer4.p4: Parsing packet parser

parse 20

#bytes

ingress

parse _state

not shown: inspect PR

> 20 parse more

©2017 Open-NFP

Flux Research Group

50

HyPer4.p4: Parsing packet parser

ingress update #bytes, resubmit

parse 20

#bytes

> 20 parse more

©2017 Open-NFP

parse _state

not shown: inspect PR

con@nue

Flux Research Group

51