1. How to Multiply integers, matrices, and polynomials. COS 423. Spring 2007
slides by Kevin Wayne. 2. Complex Multiplication. Complex multiplication. (a + bi)
...
Complex Multiplication Complex multiplication. (a + bi) (c + di) = x + yi.
How to Multiply
Grade-school. x = ac - bd, y = bc + ad.
integers, matrices, and polynomials
4 multiplications, 2 additions
Q. Is it possible to do with fewer multiplications?
COS 423 Spring 2007 slides by Kevin Wayne
1
2
Complex Multiplication
Integer Multiplication
Complex multiplication. (a + bi) (c + di) = x + yi. Grade-school. x = ac - bd, y = bc + ad. 4 multiplications, 2 additions
Q. Is it possible to do with fewer multiplications? A. Yes. [Gauss] x = ac - bd, y = (a + b) (c + d) - ac - bd. 3 multiplications, 5 additions
Section 5.5
Remark. Improvement if no hardware multiply.
3
Integer Addition
Integer Multiplication
Addition. Given two n-bit integers a and b, compute a + b. Grade-school. !(n) bit operations.
Multiplication. Given two n-bit integers a and b, compute a " b. Grade-school. !(n2) bit operations. 1 1 0 1 0 1 0 1
1
1
1
1
1
1
0
1
" 0 1 1 1 1 1 0 1
1
1
0
1
0
1
0
1
1 1 0 1 0 1 0 1
+
0
1
1
1
1
1
0
1
0 0 0 0 0 0 0 0 0
1
0
1
0
1
0
0
1
0
1 1 0 1 0 1 0 1 0 1 1 0 1 0 1 0 1 0 1 1 0 1 0 1 0 1 0 1 1 0 1 0 1 0 1 0 1 1 0 1 0 1 0 1 0
Remark. Grade-school addition algorithm is optimal.
0 0 0 0 0 0 0 0 0 0 1 1 0 1 0 0 0 0 0 0 0 0 0 0 1
Q. Is grade-school multiplication algorithm optimal? 5
6
Divide-and-Conquer Multiplication: Warmup
Recursion Tree
To multiply two n-bit integers a and b: Multiply four !n-bit integers, recursively. Add and shift to obtain result.
lg n $ 21+ lg n #1 ' 2 T (n) = " n 2 k = n & ) = 2n # n % 2 #1 ( k=0
" 0 if n = 0 T (n) = # $ 4T (n /2) + n otherwise
!
!
!
assume n is a power of 2
(
Ex. !
)(
)
a = 10001101
b = 11100001
a1
b1
T(n/2)
T(n/2)
T(n/4) T(n/4)
T(n/4)
T(n/4)
...
4(n/2)
T(n/2)
T(n/4)
T(n/4) T(n/4)
T(n/4)
16(n/4) ...
b0
...
...
recursive calls
4k (n / 2k)
T(n / 2k)
T (n) = 4T (n /2) + "(n) # T (n) = "(n 2 ) 123 1424 3 add, shift
T(2)
!
T(n/2)
...
a0
n
! T(n)
a = 2 n / 2 " a1 + a0 b = 2 n / 2 " b1 + b0 a b = 2 n / 2 " a1 + a0 2 n / 2 " b1 + b0 = 2 n " a1b1 + 2 n / 2 " (a1b0 + a0 b1 ) + a0 b0
7
T(2)
T(2)
T(2)
...
T(2)
T(2)
T(2)
T(2)
4
lg n
(1) 8
Karatsuba Multiplication
Karatsuba Multiplication
To multiply two n-bit integers a and b: Add two !n bit integers. Multiply three !n-bit integers, recursively. Add, subtract, and shift to obtain result.
To multiply two n-bit integers a and b: Add two !n bit integers. Multiply three !n-bit integers, recursively. Add, subtract, and shift to obtain result.
!
!
!
!
!
a = 2 n / 2 " a1 b = 2 n / 2 " b1 ab = 2 n " a1b1 = 2 n " a1b1
+ + + +
!
a = 2 n / 2 " a1 b = 2 n / 2 " b1 ab = 2 n " a1b1 = 2 n " a1b1
a0 b0 2 n / 2 " (a1b0 + a0 b1 ) + a0 b0 2 n / 2 " ( (a1 + a0 ) (b1 + b0 ) # a1b1 # a0 b0 ) + a0 b0 2
1
1
3
3
+ + + +
1
!
a0 b0 2 n / 2 " (a1b0 + a0 b1 ) + a0 b0 2 n / 2 " ( (a1 + a0 ) (b1 + b0 ) # a1b1 # a0 b0 ) + a0 b0 2
1
3
3
!
Theorem. [Karatsuba-Ofman 1962] Can multiply two n-bit integers in O(n1.585) bit operations. T (n) " T ( #n /2$ ) + T ( %n /2& ) + T ( 1+ %n /2& ) + '(n) ( T (n) = O(n lg 3 ) = O(n1.585 ) 14243 14444444244444443 recursive calls
add, subtract, shift
9
10
!
Karatsuba: Recursion Tree lg n
" 0 if n = 0 T (n) = # $ 3T (n /2) + n otherwise
!
T (n) = " n k=0
( 23 )
k
Integer Division
$ 3 1+ lg n #1 ' () ) = 3n lg 3 # 2n = n& 2 3 & ) % 2 #1 (
assume n is a power of 2 n
!T(n)
T(n/2)
T(n/4) T(n/4)
T(n/2)
T(n/4)
T(n/4) T(n/4)
3(n/2)
T(n/2)
T(n/4)
T(n/4) T(n/4)
T(n/4)
9(n/4) ...
...
...
...
T(2)
T(2)
T(2)
...
k
3 (n / 2 )
T(n / 2k)
T(2)
Section 30.3
k
T(2)
T(2)
T(2)
T(2)
3
lg n
(1) 11
Integer Division
Newton's Method
Integer division. Given two integer s and t of at most n bits each, compute the quotient and remainder: q = #s / t$ , r = s mod t.
Goal. Given a function f (x), find a value x* such that f(x*) = 0.
Ex.
Newton's method. Start with initial guess x0. f (xi ) . Compute a sequence of approximations: xi+1 = xi "
!
!
sufficiently smooth
s = 1,000, t = 110 % q = 9, r = 10. s = 4,905,648,605,986,590,685, t = 100 % r = 85.
!
!
f #(xi )
Long division. !(n2). !
xi
f (x)
xi+1
x
Q. Is grade-school long division algorithm optimal?
Convergence. No guarantees in general. 13
14
Integer Division: Newton's Method
Integer Division: Newton's Method Example
Goal. Given two integer s and t compute q = #s / t$.
Ex. t = 7
Our approach: Newton's method. Approximately compute x = 1 / t using exact arithmetic. !
1 f (x) = t " x xi+1 = 2xi " txi2
!
After not !too many iterations, quotient q is either #s xk$ or &s xk'.
1 x = 2xi " txi2
!
x0 = 0.1
!
x1 = 0.13
!
x2 = 0.1417
!
x3 = 0.142847770
!
x4 = 0.14285714224218970
!
x5 = 0.14285714285714285449568544449737
!
x6 = 0.14285714285714285714285714285714280809023113867839307631644158170
f (x) = t " number of digits of accuracy doubles after each iteration
xi+1
!
Ex. s/t = 123456/7 !
!
15
s x5 = 17636.57142857142824461934223586731072000
Correct answer is either 17636 or 17637.
16
Integer Division: Newton's Method
Analysis L1. Iterates converge monotonically. 1 1 < x 0 " x1 " x 2 " L " . 2t t
(q, r) = NewtonDivision(s, t)
Pf. [by induction on i]
Choose x to be unique fractional power of 2 in interval (1 / (2t), 1 / t ]
!
repeat lg n times
!
Base case: by construction,
1 1 < x0 " 2t t
Inductive hypothesis: !
1 1 < x 0 " x1 " L " xi " . 2t t
x! ( 2x – t x2 !
set q = #s x $ or q = &s x ' set r = s – q t
xi+1
= = # =
2
2xi " t xi ! xi (2 " t xi ) xi (2 " t (1/t)) xi
xi+1
= = = #
2xi " t xi2 (2xi " t xi2 "1/t) +1/t "t(xi "1/t) 2 +1/t 1/t
inductive hypothesis
(monotonic) 17
(bounded)
Analysis
Analysis
L2. Iterates converge quadratically to 1 / t :
Pf. [by induction on i] !
Base case: by construction,
1 " txi
