Array Structured Object Types for Mathematical ...

2 downloads 0 Views 115KB Size Report
+ empirical optimization schemes (e.g. ATLAS). Å  special purpose languages such as. Matlab, Mathematica, Maple, R etc. Oberon, Zonnon: ideal languages for ...
Array Structured Object Types for Mathematical Programming F. Friedrich, J. Gutknecht ETH Zürich

Joint Modular Languages Conference, Oxford, 2006

Array Structured Object Types Department of Computer Science

Outline

Š Introduction Š Built-in array types Š Custom array structured object types

Friday, 11 May 2007

JMLC 2006

2

Array Structured Object Types Department of Computer Science

Objective

MathOberon (& Zonnon for .NET): Concept for intuitive and efficient mathematical programming within an object-oriented language

Friday, 11 May 2007

JMLC 2006

3

Array Structured Object Types Department of Computer Science

Existing concepts for mathematical programming Š

multi-purpose languages such as Fortran, APL, ZPL, Chapel, etc. + empirical optimization schemes (e.g. ATLAS)

Š

special purpose languages such as Matlab, Mathematica, Maple, R etc.

Oberon, Zonnon: ideal languages for mathematical modeling

Friday, 11 May 2007

JMLC 2006

4

Array Structured Object Types Department of Computer Science

Requisites Š Efficiency. Incorporate expert knowledge about structure of algorithms

Š Notational simplicity. Readable notation that conforms with usual mathematical conventions.

Š Structural simplicity. Programmer must not need to handle system matters like pointer arithmetics and memory management

.

Š Extensibility. Possibility to add arbitrary functionality on an implementation level

Š Safety. Typical safety features, such as range- and type-checking.

Friday, 11 May 2007

JMLC 2006

5

Array Structured Object Types Department of Computer Science

New Concepts Š Built-in array types @ @ @ @

ranged indices as array designators built-in operators value semantics notation near to that of Matlab

Š Custom array structured object types @ mimic the behaviour of arrays @ provide flexibility

Friday, 11 May 2007

JMLC 2006

6

Array Structured Object Types Department of Computer Science

Example VAR A,B: ARRAY [*,*] OF REAL; C: ARRAY [3,3] OF REAL; value semantics ...NEW(B,6,6) ..

Š optimization possible (e.g. SSE2 instructions) Š intuitive & close to mathematical notation (* initiallyallocation LEN(A)=0 *) Š implicit of the l.h.s.possible

built-in operator

ranged indices

A := C` * B[1..3,2..4]; transposition

0

1

1

1

1

1

4

3

3

1

2

3

1

0

1

1

1

1

6

5

5

4

1

2

1

1

0

1

1

1

6

5

5

2

3

4

1

1

1

0

0

1

1

1

1

1

0

1

A

C`

*

B Friday, 11 May 2007

JMLC 2006

7

Array Structured Object Types Department of Computer Science

Example: matrix-multiplication optimization plain C++, naive approach

Plain Oberon, naive approach optimized C++ naive approach

SSE2 assembler inline in Oberon

Processor cycles of matrix multiplication. Oberon vs. C++ vs. optimized C++ vs. Oberon using SSE2 instructions, source: Diploma Thesis Baumgartner

Friday, 11 May 2007

JMLC 2006

8

Array Structured Object Types Department of Computer Science

Example: sub-domains Small part of SVD in classical notation VAR u: ARRAY [*,*] OF REAL; s,h: REAL; i,j,k,l,m,n: LONGINT; (* ... *) FOR j := l TO n DO long and not s := 0.0; intuitive FOR k := i TO m DO s := s + u[k, i] * u[k, j] END; Computationally FOR k := i TO m DO expensive u[k, j] := u[k, j] + s/h * u[k, i] END; END;

Friday, 11 May 2007

JMLC 2006

9

Array Structured Object Types Department of Computer Science

Example: sub-domains Replacement of inner loops by operators on substructures VAR u: ARRAY [*,*] OF REAL; s,h: REAL; i,j,l,m,n: LONGINT; (* ... *) pseudo scalar product

FOR j := l TO n DO s := u[i..m,i] +* u[i..m,j]; u[i..m,j] := u[i..m,j] + s / h* u[i..m,i]; END;

other operators + , -, * , / , MOD , DIV , .* , ./ , +* , **, ./, ABS , MIN , MAX , SUM , ` etc.

Šreadability Šrange checks on sub-domains Šidentification of independent pieces of code (potential parallelisation)

Friday, 11 May 2007

JMLC 2006

10

Array Structured Object Types Department of Computer Science

Using custom functions PROCEDURE MySum(CONST X: ARRAY [*,*] OF REAL): REAL; VAR i,j: LONGINT; sum: REAL; BEGIN .... (stack)

designator(A)

... adr inc0 len0 inc1 len1

baseadr inc(A,0) len(A,0) inc(A,1) len(A,1)

c := MySum(A[a..b,c..d]); Friday, 11 May 2007

JMLC 2006

different ways to pass an array: PROCEDURE P (CONST X: ARRAY [*,*] OF B); PROCEDURE P (VAR X: ARRAY [*,*] OF B); PROCEDURE P (X: ARRAY [*,*] OF B); PROCEDURE P (...): ARRAY [*,*] OF B; 11

Array Structured Object Types Department of Computer Science

Custom array structured types Array structured object type: Value object type whose signature explicitly contains the array structure, designed for all cases where an array cannot be stored in one linear piece of memory Example: sparse matrices Š Š Š Š Š

Definition like an object No inheritance and polymorphism Mimic the behaviour of built-in arrays Operator overloading on module level Prevent the misuse of mathematical ‘indexers’ for general purposes

Friday, 11 May 2007

JMLC 2006

12

Array Structured Object Types Department of Computer Science

Custom Array Types: Sparse Matrix Usage VAR A: ARRAY [10,10] OF REAL; B:SparseMatrix; i: LONGINT; (* ... *) NEW(B,1000,1000); FOR i := 0 TO 999 BY 10 DO B[i..i+9,i..i+9] := A; END; TYPE

SparseMatrix*= ARRAY [..,..] OF REAL VAR d: Data; len0,len1: LONGINT; (* Data defined elsewhere *)

PROCEDURE NEW(i,j: LONGINT); (* allocation *) PROCEDURE LEN(i: LONGINT): LONGINT; (* shape *) PROCEDURE "[]"(i,j: LONGINT): REAL; PROCEDURE "[]"(i,j: LONGINT; r: REAL);

PROCEDURE "[]"(a1..b1 BY c1,a2..b2 BY c2: LONGINT): ARRAY [..,..] OF REAL; PROCEDURE "[]"(a1..b1 BY c1,a2..b2 BY c2: LONGINT; CONST A: ARRAY [..,..] OF REAL);

END SparseMatrix; PROCEDURE ’*’ (A,B: SparseMatrix): SparseMatrix;

Friday, 11 May 2007

JMLC 2006

13

Array Structured Object Types Department of Computer Science

Custom Array Types: Sparse Matrix Interface 2d array structure with TYPE element type real SparseMatrix*= ARRAY [..,..] OF REAL VAR d: Data; len0,len1: LONGINT; (* Data defined elsewhere *) PROCEDURE NEW(i,j: LONGINT); (* allocation *) PROCEDURE LEN(i: LONGINT): LONGINT; (* shape *) PROCEDURE "[]"(i,j: LONGINT): REAL; PROCEDURE "[]"(i,j: LONGINT; r: REAL);

element access

PROCEDURE "[]"(a1..b1 BY c1,a2..b2 BY c2: LONGINT): ARRAY [..,..] OF REAL; PROCEDURE "[]"(a1..b1 BY c1,a2..b2 BY c2: LONGINT; CONST A: ARRAY [..,..] OF REAL); sub-structure access

END SparseMatrix; PROCEDURE ’*’ (A,B: SparseMatrix): SparseMatrix;

Friday, 11 May 2007

JMLC 2006

operator overloading

14

Array Structured Object Types Department of Computer Science

Some implementation details Š Geometric information is kept separate from array data ∆ runtime measurements have indicated a gain of speed for large arrays  codesize sometimes larger compared with pointer oriented approach

Š Built-in operators implemented as (low-level) library ∆ easily extensible  procedure call overhead (in particular w.r.t. small arrays)

Š Strict value semantics: Array descriptors immutable in sub-procedures ∆ protection against concurrent access ∆ no GC corruption  no re-allocation in sub-procedures possible unless wrapped in an object

Friday, 11 May 2007

JMLC 2006

15

Array Structured Object Types Department of Computer Science

Achievements Š notational compactness in linear algebra and imaging applications Š language with high potential w.r.t optimization while preserving safety Š extensibility and flexibility guaranteed by a clean implementation of custom array structured types Š planned for integration into Zonnon for .NET

Friday, 11 May 2007

JMLC 2006

16

Array Structured Object Types Department of Computer Science

thank you for your attention

Friday, 11 May 2007

JMLC 2006

17

Array Structured Object Types Department of Computer Science

Discarded Ideas Š no separate type for the domain S Š no variable lower bound Š no super-dynamic features (like appending, insertion and deletion of elements) Š no free boundary conditions (like ‘mirrored’, ‘wrapped’ etc.) Š no properties (like ‘diagonal’, ‘symmetric’ etc.)

Friday, 11 May 2007

JMLC 2006

18

Array Structured Object Types Department of Computer Science

Concepts of the built-in arrays Š Ranged indices (‘i..j BY k’) as array designators. @ readabilty @ efficiency: identification of independent pieces of code @ safety: range checks on sub-domains

Š Value semantics @ structural simplicity: memory allocation and pointer mechanisms behind the scenes @ clearness of operations (assignment, equality) @ protection against unintentional concurrent access

Š Notation near to that of Matlab @ acceptance

Š Operators are part of the language Friday, 11 May 2007

JMLC 2006

19

Array Structured Object Types Department of Computer Science

Oberon ideal language for mathematical modeling Š clearity and readabilty Š more abstract and closer to mathematics than system near languages Š experiences / libraries @ Voyager (G.Sawitzki, Heidelberg): data analysis, statistics, visualisation, stochastic simulation @ Ants (FF, Munich): random fields, image analysis, mathematical modelling in the life sciences @ CAT/CAPO (A.Freed, Cleveland, FF): linear algebra package @ Mathematical modelling in medicine (P.Hunziker, Basel): wavelets, image analysis, linear algebra

Friday, 11 May 2007

JMLC 2006

20

Array Structured Object Types Department of Computer Science

Matrix multiplication: naive approach VAR A,B,Res: ARRAY [*,*] OF REAL; i,j,k: LONGINT; temp: REAL; (* ... *) (* check shapes *) FOR i := 0 TO LEN(L,0)-1 DO FOR j := 0 TO LEN(R,1)-1 DO temp := 0; FOR k := 0 TO LEN(R,0)-1 DO temp := temp + L[i,k]*R[k,j]; END; Res[i,j] := temp; END; END;

Friday, 11 May 2007

JMLC 2006

long and not intuitive Computationally expensive, cache misses may occur

21

Array Structured Object Types Department of Computer Science

Example: matrix-multiplication 2. Replacement of inner loop VAR A,B,Res: ARRAY [*,*] OF REAL; i,j: LONGINT; (* ... *) (* check shapes *) FOR i := 0 TO LEN(L,0)-1 DO FOR j := 0 TO LEN(R,1)-1 DO Res[i,j] := L[i,..] +* R[..,j]; (* scalar product *) END; END; Optimization possible Friday, 11 May 2007

JMLC 2006

22

Array Structured Object Types Department of Computer Science

matrix-multiplication as operator Important operators are made available by the language VAR A,B,Res: ARRAY [*,*] OF REAL; (* ... *) transposition

Res := A * B`; matrix multiplication

Friday, 11 May 2007

Š optimization possible (e.g. SSE2 instructions) Š intuitive Š close to mathematical notation Š implicit allocation possible JMLC 2006

23

Array Structured Object Types Department of Computer Science

Scope Ideal case: development of theoretically better algorithms providing lower complexity. Additionally: by exploiting structure of the problem code can be made clearer and more efficient Š Objective: Object oriented concept of arraystructured types. Š Goal: intuitive and efficient mathematical programming. Š Used Language: (Active) Oberon Friday, 11 May 2007

JMLC 2006

24

Array Structured Object Types Department of Computer Science

Code snippet: Singular Value Decomposition scale := 0; FOR k := i TO m - 1 DO scale := scale + ABS( u[k, i] ) END; IF (scale # 0) THEN FOR k := i TO m - 1 DO u[k, i] := u[k, i]/scale; s := s + u[k, i]*u[k, i]; END; f := u[i, i]; IF f >= 0 THEN g := -sqrt( s ) ELSE g := sqrt( s ) END; h := f*g - s; u[i, i] := ( f - g); FOR j := l TO n - 1 DO s := 0; FOR k := i TO m - 1 DO s := s + u[k, i]*u[k, j] END; f := s/h; FOR k := i TO m - 1 DO u[k, j] := u[k, j] + f*u[k, i] END; END; FOR k := i TO m - 1 DO u[k, i] := u[k, i]*scale END; END

Friday, 11 May 2007

JMLC 2006

25

Array Structured Object Types Department of Computer Science

Code snippet: Singular Value Decomposition scale := SUM(ABS(u[i..m-1,i])); IF (scale # 0) THEN u[i..m-1,i] := u[i..m-1,i]/scale; s := s + u[i..m-1,i] +* u[i..m-1,i]; f := u[i, i]; IF f >= 0 THEN g := -sqrt( s ) ELSE g := sqrt( s ) END; h := f*g - s; u[i, i] := ( f - g); FOR j := l TO n - 1 DO s := u[i..m-1,i] +* u[i..m-1,j]; f := s/h; u[i..m-1,j] := u[i..m-1,j] + f*u[i..m-1,i] END; u[i..m-1, i] := u[i..m-1, i]*scale END

Friday, 11 May 2007

JMLC 2006

26