Basic idea. â¡. Similar to binomial heaps, but less rigid structure. â¡. Binomial heap: eagerly consolidate trees afte
Priority Queues Performance Cost Summary
Fibonacci Heaps
Lecture slides adapted from: • Chapter 20 of Introduction to Algorithms by Cormen, Leiserson, Rivest, and Stein.
Operation
Linked List
Binary Heap
Binomial Heap
Fibonacci Heap †
Relaxed Heap
make-heap
1
1
1
1
1
is-empty
1
1
1
1
1
insert
1
log n
log n
1
1
delete-min
n
log n
log n
log n
log n
decrease-key
n
log n
log n
1
1
delete
n
log n
log n
log n
log n
union
1
n
log n
1
1
find-min
n
1
log n
1
1
n = number of elements in priority queue
† amortized
• Chapter 9 of The Design and Analysis of Algorithms by Dexter Kozen.
Theorem. Starting from empty Fibonacci heap, any sequence of a1 insert, a2 delete-min, and a3 decrease-key operations takes O(a1 + a2 log n + a3) time. COS 423 Theory of Algorithms • Kevin Wayne • Spring 2007
2
Priority Queues Performance Cost Summary
Operation
Linked List
Binary Heap
Binomial Heap
Fibonacci Heap †
Relaxed Heap
make-heap
1
1
1
1
1
is-empty
1
1
1
1
1
insert
1
log n
log n
1
1
delete-min
n
log n
log n
log n
log n
decrease-key
n
log n
log n
1
1
delete
n
log n
log n
log n
log n
union
1
n
log n
1
1
find-min
n
1
log n
1
1
Fibonacci Heaps History. [Fredman and Tarjan, 1986] Ingenious data structure and analysis. Original motivation: improve Dijkstra's shortest path algorithm from O(E log V ) to O(E + V log V ). !
!
V insert, V delete-min, E decrease-key
Basic idea. Similar to binomial heaps, but less rigid structure. Binomial heap: eagerly consolidate trees after each insert. !
n = number of elements in priority queue
!
† amortized
Hopeless challenge. O(1) insert, delete-min and decrease-key. Why?
!
3
Fibonacci heap: lazily defer consolidation until next delete-min.
4
Fibonacci Heaps: Structure
Fibonacci Heaps: Structure
each parent larger than its children
Fibonacci heap. Set of heap-ordered trees. Maintain pointer to minimum element. Set of marked nodes.
Fibonacci heap. Set of heap-ordered trees. Maintain pointer to minimum element. Set of marked nodes.
!
!
!
!
!
!
find-min takes O(1) time
roots
17
30
24
26
heap-ordered tree
23
7
3
46
18
35
Heap H
min
52
17
30
41
26
23
7
3
46
18
35
Heap H
44
39
24
52
41
44
39
5
6
Fibonacci Heaps: Structure
Fibonacci Heaps: Notation
Fibonacci heap. Set of heap-ordered trees. Maintain pointer to minimum element. Set of marked nodes.
Notation. n = number of nodes in heap. rank(x) = number of children of node x. rank(H) = max rank of any node in heap H. trees(H) = number of trees in heap H. marks(H) = number of marked nodes in heap H.
!
!
!
!
!
!
!
use to keep heaps flat (stay tuned)
!
min
17
30
Heap H
24
26
35
23
7
trees(H) = 5
3
46
18 marked
39
52
17
30
41 Heap H
44 7
marks(H) = 3
24
26
35
n = 14
23
min
rank = 3
7
3
46
18 marked
39
52
41
44 8
Fibonacci Heaps: Potential Function
Insert !(H) !=!trees(H) + 2 " marks(H) potential of heap H
trees(H) = 5
17
30
marks(H) = 3
24
26
23
7
3
46
18
35
Heap H
min
!(H) = 5 + 2"3 = 11
marked
52
41
44
39
9
10
Fibonacci Heaps: Insert
Fibonacci Heaps: Insert
Insert. Create a new singleton tree. Add to root list; update min pointer (if necessary).
Insert. Create a new singleton tree. Add to root list; update min pointer (if necessary).
!
!
!
!
insert 21
insert 21
21
17
30
Heap H
24
26
35
46
23
min
7
min
3
18
39
52
17
30
41 Heap H
44 11
24
26
35
46
23
7
3
21
18
39
52
41
44 12
Fibonacci Heaps: Insert Analysis Actual cost. O(1)
Delete Min
!(H) !=!trees(H) + 2 " marks(H) potential of heap H
Change in potential. +1 Amortized cost. O(1)
min
17
30
24
26
23
7
46
18
35
Heap H
3
21
52
41
44
39
13
14
Linking Operation
Fibonacci Heaps: Delete Min
Linking operation. Make larger root be a child of smaller root.
Delete min. Delete min; meld its children into root list; update min. Consolidate trees so that no two roots have same rank. !
!
larger root
3
15
56
24
77 tree T1
still heap-ordered
smaller root
18
52
39
41
44 tree T2
min
3
56
15
18
24
39
52
7
41
44
30
24
26
46
23
17
3
18
52
41
77 35
tree T'
15
39
44
16
Fibonacci Heaps: Delete Min
Fibonacci Heaps: Delete Min
Delete min. Delete min; meld its children into root list; update min. Consolidate trees so that no two roots have same rank.
Delete min. Delete min; meld its children into root list; update min. Consolidate trees so that no two roots have same rank.
!
!
!
!
min
min
7
30
24
26
23
17
18
52
39
46
current
41
7
44
30
35
24
26
23
17
18
52
39
46
41
44
35
17
18
Fibonacci Heaps: Delete Min
Fibonacci Heaps: Delete Min
Delete min. Delete min; meld its children into root list; update min. Consolidate trees so that no two roots have same rank.
Delete min. Delete min; meld its children into root list; update min. Consolidate trees so that no two roots have same rank.
!
!
!
!
rank
0
min
1
2
rank
3
0
min
current 7
30
24
26
46
23
17
18
39
52
2
3
current
41
7
44
30
35
1
24
26
46
23
17
18
39
52
41
44
35
19
20
Fibonacci Heaps: Delete Min
Fibonacci Heaps: Delete Min
Delete min. Delete min; meld its children into root list; update min. Consolidate trees so that no two roots have same rank.
Delete min. Delete min; meld its children into root list; update min. Consolidate trees so that no two roots have same rank.
!
!
!
!
rank
0
1
2
rank
3
0
min
1
2
3
min
7
30
24
26
46
23
17
18
52
39
current
41
7
44
30
35
24
26
46
23
17
18
current
52
39
41
44
35
link 23 into 17 21
22
Fibonacci Heaps: Delete Min
Fibonacci Heaps: Delete Min
Delete min. Delete min; meld its children into root list; update min. Consolidate trees so that no two roots have same rank.
Delete min. Delete min; meld its children into root list; update min. Consolidate trees so that no two roots have same rank.
!
!
!
!
rank
0
1
2
rank
3
0
1
2
3
min
current min
7
30
24
26
46
current
17
18
23
39
52
41
24
44
26
35
35
link 17 into 7
46
17
7
18
30
39
52
41
44
23
link 24 into 7 23
24
Fibonacci Heaps: Delete Min
Fibonacci Heaps: Delete Min
Delete min. Delete min; meld its children into root list; update min. Consolidate trees so that no two roots have same rank.
Delete min. Delete min; meld its children into root list; update min. Consolidate trees so that no two roots have same rank.
!
!
!
!
rank
0
1
2
rank
3
0
1
2
3
current
current
min
26
24
17
46
23
min
7
18
30
39
52
41
44
26
35
24
17
46
23
7
18
30
39
52
41
44
35 25
26
Fibonacci Heaps: Delete Min
Fibonacci Heaps: Delete Min
Delete min. Delete min; meld its children into root list; update min. Consolidate trees so that no two roots have same rank.
Delete min. Delete min; meld its children into root list; update min. Consolidate trees so that no two roots have same rank.
!
!
!
!
rank
0
1
2
rank
3
0
1
2
3
current
current
min
26
24
17
46
23
min
7
18
30
39
52
41
44
26
24
17
46
23
7
18
30
39
52
41
44
link 41 into 18 35
35 27
28
Fibonacci Heaps: Delete Min
Fibonacci Heaps: Delete Min
Delete min. Delete min; meld its children into root list; update min. Consolidate trees so that no two roots have same rank.
Delete min. Delete min; meld its children into root list; update min. Consolidate trees so that no two roots have same rank.
!
!
!
!
rank
0
1
2
rank
3
0
1
2
3
current
current
min
min
7
26
24
17
46
23
52
18
41
30
7
39
44
26
35
24
17
46
23
52
18
41
30
39
44
35 29
30
Fibonacci Heaps: Delete Min
Fibonacci Heaps: Delete Min Analysis
Delete min. Delete min; meld its children into root list; update min. Consolidate trees so that no two roots have same rank.
Delete min.
!(H) !=!trees(H) + 2 " marks(H)
!
potential function
!
Actual cost. O(rank(H)) + O(trees(H)) O(rank(H)) to meld min's children into root list. O(rank(H)) + O(trees(H)) to update min. O(rank(H)) + O(trees(H)) to consolidate trees. !
!
!
min
7
52
Change in potential. O(rank(H)) - trees(H) trees(H' ) # rank(H) + 1 since no two trees have same rank. $!(H) # rank(H) + 1 - trees(H).
18
!
!
24
17
41
30
39
Amortized cost. O(rank(H)) 26
46
23
44
stop 35 31
32
Fibonacci Heaps: Delete Min Analysis Q. Is amortized cost of O(rank(H)) good?
Decrease Key
A. Yes, if only insert and delete-min operations. In this case, all trees are binomial trees. This implies rank(H) # lg n. !
!
we only link trees of equal rank
B0
B1
B2
B3
A. Yes, we'll implement decrease-key so that rank(H) = O(log n).
33
34
Fibonacci Heaps: Decrease Key
Fibonacci Heaps: Decrease Key
Intuition for deceasing the key of node x. If heap-order is not violated, just decrease the key of x. Otherwise, cut tree rooted at x and meld into root list. To keep trees flat: as soon as a node has its second child cut, cut it off and meld into root list (and unmark it).
Case 1. [heap order not violated] Decrease key of x. Change heap min pointer (if necessary).
!
!
!
!
!
min
min
7 marked node: one child already cut
26
24
17
46
30
23
21
18
38
39
41
7
24
26
52
17
46 29
30
23
21
18
38
39
41
52
x
35
88
72
35 35
88
72
decrease-key of x from 46 to 29 36
Fibonacci Heaps: Decrease Key
Fibonacci Heaps: Decrease Key
Case 1. [heap order not violated] Decrease key of x. Change heap min pointer (if necessary).
Case 2a. [heap order violated] Decrease key of x. Cut tree rooted at x, meld into root list, and unmark. If parent p of x is unmarked (hasn't yet lost a child), mark it; Otherwise, cut p, meld into root list, and unmark (and do so recursively for all ancestors that lose a second child).
!
!
!
!
!
min
min
7
24
17
23
21
18
38
39
41
7
24
17
23
21
18
38
39
41
p
26
29
30
26
52
29 15
x
35
88
30
52
x
decrease-key of x from 46 to 29
72
35
88
decrease-key of x from 29 to 15
72
37
38
Fibonacci Heaps: Decrease Key
Fibonacci Heaps: Decrease Key
Case 2a. [heap order violated] Decrease key of x. Cut tree rooted at x, meld into root list, and unmark. If parent p of x is unmarked (hasn't yet lost a child), mark it; Otherwise, cut p, meld into root list, and unmark (and do so recursively for all ancestors that lose a second child).
Case 2a. [heap order violated] Decrease key of x. Cut tree rooted at x, meld into root list, and unmark. If parent p of x is unmarked (hasn't yet lost a child), mark it; Otherwise, cut p, meld into root list, and unmark (and do so recursively for all ancestors that lose a second child).
!
!
!
!
!
!
x
min
7
24
17
23
21
18
38
15
39
41
72
min
7
24
p
26
15
17
23
21
18
38
39
41
p
30
26
52
30
52
x
35
88
72
decrease-key of x from 29 to 15
35 39
88
decrease-key of x from 29 to 15 40
Fibonacci Heaps: Decrease Key
Fibonacci Heaps: Decrease Key
Case 2a. [heap order violated] Decrease key of x. Cut tree rooted at x, meld into root list, and unmark. If parent p of x is unmarked (hasn't yet lost a child), mark it; Otherwise, cut p, meld into root list, and unmark (and do so recursively for all ancestors that lose a second child).
Case 2b. [heap order violated] Decrease key of x. Cut tree rooted at x, meld into root list, and unmark. If parent p of x is unmarked (hasn't yet lost a child), mark it; Otherwise, cut p, meld into root list, and unmark (and do so recursively for all ancestors that lose a second child).
!
!
!
!
!
x
!
min
15
min
7
72
24
23
21
38
15
39
41
72
7
24
17
23
21
18
38
39
41
p
mark parent
26
35
17
18
30
p
52
decrease-key of x from 29 to 15
88
35 5
x
26
30
52
decrease-key of x from 35 to 5
88
41
42
Fibonacci Heaps: Decrease Key
Fibonacci Heaps: Decrease Key
Case 2b. [heap order violated] Decrease key of x. Cut tree rooted at x, meld into root list, and unmark. If parent p of x is unmarked (hasn't yet lost a child), mark it; Otherwise, cut p, meld into root list, and unmark (and do so recursively for all ancestors that lose a second child).
Case 2b. [heap order violated] Decrease key of x. Cut tree rooted at x, meld into root list, and unmark. If parent p of x is unmarked (hasn't yet lost a child), mark it; Otherwise, cut p, meld into root list, and unmark (and do so recursively for all ancestors that lose a second child).
!
!
!
!
!
!
min
min
15
7
72
24
p
x
5
26
88
17
30
23
21
18
38
15
39
41
72
x
5
7
24
p
52
decrease-key of x from 35 to 5
26
88 43
17
30
23
21
18
38
39
41
52
decrease-key of x from 35 to 5 44
Fibonacci Heaps: Decrease Key
Fibonacci Heaps: Decrease Key
Case 2b. [heap order violated] Decrease key of x. Cut tree rooted at x, meld into root list, and unmark. If parent p of x is unmarked (hasn't yet lost a child), mark it; Otherwise, cut p, meld into root list, and unmark (and do so recursively for all ancestors that lose a second child).
Case 2b. [heap order violated] Decrease key of x. Cut tree rooted at x, meld into root list, and unmark. If parent p of x is unmarked (hasn't yet lost a child), mark it; Otherwise, cut p, meld into root list, and unmark (and do so recursively for all ancestors that lose a second child).
!
!
!
!
!
min
15
min
x
5
72
!
7
24
second child cut
26
p
17
23
30
21
18
38
15
39
41
72
x
p
5
26
88
7
24
52
17
23
30
38
39
41
52
decrease-key of x from 35 to 5
88
21
18
decrease-key of x from 35 to 5 45
46
Fibonacci Heaps: Decrease Key
Fibonacci Heaps: Decrease Key
Case 2b. [heap order violated] Decrease key of x. Cut tree rooted at x, meld into root list, and unmark. If parent p of x is unmarked (hasn't yet lost a child), mark it; Otherwise, cut p, meld into root list, and unmark (and do so recursively for all ancestors that lose a second child).
Case 2b. [heap order violated] Decrease key of x. Cut tree rooted at x, meld into root list, and unmark. If parent p of x is unmarked (hasn't yet lost a child), mark it; Otherwise, cut p, meld into root list, and unmark (and do so recursively for all ancestors that lose a second child).
!
!
!
!
!
min
15
72
x
p
5
26
88
!
min
7
p'
second child cut
24
17
30
23
21
18
38
15
39
41
72
52
x
p
p'
p''
5
26
24
7
88
don't mark parent if it's a root
17
30
decrease-key of x from 35 to 5
23
21
18
38
39
41
52
decrease-key of x from 35 to 5 47
48
Fibonacci Heaps: Decrease Key Analysis Decrease-key.
Analysis
!(H) !=!trees(H) + 2 " marks(H) potential function
Actual cost. O(c) O(1) time for changing the key. O(1) time for each of c cuts, plus melding into root list. !
!
Change in potential. O(1) - c trees(H') = trees(H) + c. marks(H') # marks(H) - c + 2. $! # c + 2 " (-c + 2) = 4 - c. !
!
!
Amortized cost. O(1)
49
50
Analysis Summary Insert. Delete-min. Decrease-key.
O(1) O(rank(H)) O(1) †
Fibonacci Heaps: Bounding the Rank Lemma. Fix a point in time. Let x be a node, and let y1, …, yk denote its children in the order in which they were linked to x. Then:
†
x
† amortized
$0 if i = 1 rank (yi ) " % & i # 2 if i " 1 y1
y2
…
yk
Key lemma. rank(H) = O(log n). !
number of nodes is exponential in rank
Pf. !
!
!
!
51
When yi was linked into x, x had at least i -1 children y1, …, yi-1. Since only trees of equal rank are linked, at that time rank(yi)!= rank(xi) % i - 1. Since then, yi has lost at most one child. Thus, right now rank(yi) % i - 2. or yi would have been cut
52
Fibonacci Heaps: Bounding the Rank
Fibonacci Heaps: Bounding the Rank
Lemma. Fix a point in time. Let x be a node, and let y1, …, yk denote its children in the order in which they were linked to x. Then:
Lemma. Fix a point in time. Let x be a node, and let y1, …, yk denote its children in the order in which they were linked to x. Then:
x
x
$0 if i = 1 rank (yi ) " % & i # 2 if i " 1
$0 if i = 1 rank (yi ) " % & i # 2 if i " 1 y1
…
y2
yk
y1
!
y2
…
yk
!
Def. Let Fk be smallest possible tree of rank k satisfying property.
Def. Let Fk be smallest possible tree of rank k satisfying property. F4
F0
F1
F2
F3
F4
F5
1
2
3
5
8
13
F6
F5
8
13
8 + 13 = 21
53
54
Fibonacci Heaps: Bounding the Rank Lemma. Fix a point in time. Let x be a node, and let y1, …, yk denote its children in the order in which they were linked to x. Then:
Fibonacci Numbers
x
$0 if i = 1 rank (yi ) " % & i # 2 if i " 1 y1
y2
…
yk
!
Def. Let Fk be smallest possible tree of rank k satisfying property. Fibonacci fact. Fk % &k, where & = (1 + '5) / 2 ( 1.618. Corollary. rank(H) # log& n .
golden ratio
55
56
Fibonacci Numbers: Exponential Growth
Fibonacci Numbers and Nature
Def. The Fibonacci sequence is: 1, 2, 3, 5, 8, 13, 21, … #1 % Fk = $ 2 % F +F & k-1 k-2
if k = 0 if k = 1 if k " 2
slightly non-standard definition
Lemma. Fk % &k, where & = (1 + '5) / 2 ( 1.618. !
Pf. [by induction on k] Base cases: F0 = 1 % 1, F1 = 2 % &. Inductive hypotheses: Fk % &k and Fk+1 % &k + 1
pinecone
!
!
Fk+2
= " = = =
Fk + Fk+1 # k + # k+1 # k (1 + # ) # k (# 2 ) # k+2
cauliflower
(definition) (inductive hypothesis) (algebra) (&2 = & + 1) (algebra) 57
58
!
Fibonacci Heaps: Union Union. Combine two Fibonacci heaps.
Union
Representation. Root lists are circular, doubly linked lists.
min
23
30
Heap H' 59
24
26
35
min
17
7
46
3
18 Heap H''
39
52
21
41
44 60
Fibonacci Heaps: Union
Fibonacci Heaps: Union
Union. Combine two Fibonacci heaps.
Actual cost. O(1)
Representation. Root lists are circular, doubly linked lists.
Change in potential. 0
!(H) !=!trees(H) + 2 " marks(H) potential function
Amortized cost. O(1)
min
23
30
24
26
35
17
7
46
3
18 Heap H
min
39
52
21
23
30
41
24
26
17
46
3
18
35
44
7
Heap H
52
39
21
41
44
61
62
Fibonacci Heaps: Delete Delete node x. decrease-key of x to -).
Delete
!(H) !=!trees(H) + 2 " marks(H)
!
!
delete-min element in heap.
potential function
Amortized cost. O(rank(H)) O(1) amortized for decrease-key. O(rank(H)) amortized for delete-min. !
!
63
64
