+ empirical optimization schemes (e.g. ATLAS). Å special purpose languages such as. Matlab, Mathematica, Maple, R etc. Oberon, Zonnon: ideal languages for ...
Array Structured Object Types for Mathematical Programming F. Friedrich, J. Gutknecht ETH Zürich
Joint Modular Languages Conference, Oxford, 2006
Array Structured Object Types Department of Computer Science
Outline
Introduction Built-in array types Custom array structured object types
Friday, 11 May 2007
JMLC 2006
2
Array Structured Object Types Department of Computer Science
Objective
MathOberon (& Zonnon for .NET): Concept for intuitive and efficient mathematical programming within an object-oriented language
Friday, 11 May 2007
JMLC 2006
3
Array Structured Object Types Department of Computer Science
Existing concepts for mathematical programming
multi-purpose languages such as Fortran, APL, ZPL, Chapel, etc. + empirical optimization schemes (e.g. ATLAS)
special purpose languages such as Matlab, Mathematica, Maple, R etc.
Oberon, Zonnon: ideal languages for mathematical modeling
Friday, 11 May 2007
JMLC 2006
4
Array Structured Object Types Department of Computer Science
Requisites Efficiency. Incorporate expert knowledge about structure of algorithms
Notational simplicity. Readable notation that conforms with usual mathematical conventions.
Structural simplicity. Programmer must not need to handle system matters like pointer arithmetics and memory management
.
Extensibility. Possibility to add arbitrary functionality on an implementation level
Safety. Typical safety features, such as range- and type-checking.
Friday, 11 May 2007
JMLC 2006
5
Array Structured Object Types Department of Computer Science
New Concepts Built-in array types @ @ @ @
ranged indices as array designators built-in operators value semantics notation near to that of Matlab
Custom array structured object types @ mimic the behaviour of arrays @ provide flexibility
Friday, 11 May 2007
JMLC 2006
6
Array Structured Object Types Department of Computer Science
Example VAR A,B: ARRAY [*,*] OF REAL; C: ARRAY [3,3] OF REAL; value semantics ...NEW(B,6,6) ..
optimization possible (e.g. SSE2 instructions) intuitive & close to mathematical notation (* initiallyallocation LEN(A)=0 *) implicit of the l.h.s.possible
built-in operator
ranged indices
A := C` * B[1..3,2..4]; transposition
0
1
1
1
1
1
4
3
3
1
2
3
1
0
1
1
1
1
6
5
5
4
1
2
1
1
0
1
1
1
6
5
5
2
3
4
1
1
1
0
0
1
1
1
1
1
0
1
A
C`
*
B Friday, 11 May 2007
JMLC 2006
7
Array Structured Object Types Department of Computer Science
Example: matrix-multiplication optimization plain C++, naive approach
Plain Oberon, naive approach optimized C++ naive approach
SSE2 assembler inline in Oberon
Processor cycles of matrix multiplication. Oberon vs. C++ vs. optimized C++ vs. Oberon using SSE2 instructions, source: Diploma Thesis Baumgartner
Friday, 11 May 2007
JMLC 2006
8
Array Structured Object Types Department of Computer Science
Example: sub-domains Small part of SVD in classical notation VAR u: ARRAY [*,*] OF REAL; s,h: REAL; i,j,k,l,m,n: LONGINT; (* ... *) FOR j := l TO n DO long and not s := 0.0; intuitive FOR k := i TO m DO s := s + u[k, i] * u[k, j] END; Computationally FOR k := i TO m DO expensive u[k, j] := u[k, j] + s/h * u[k, i] END; END;
Friday, 11 May 2007
JMLC 2006
9
Array Structured Object Types Department of Computer Science
Example: sub-domains Replacement of inner loops by operators on substructures VAR u: ARRAY [*,*] OF REAL; s,h: REAL; i,j,l,m,n: LONGINT; (* ... *) pseudo scalar product
FOR j := l TO n DO s := u[i..m,i] +* u[i..m,j]; u[i..m,j] := u[i..m,j] + s / h* u[i..m,i]; END;
other operators + , -, * , / , MOD , DIV , .* , ./ , +* , **, ./, ABS , MIN , MAX , SUM , ` etc.
readability range checks on sub-domains identification of independent pieces of code (potential parallelisation)
Friday, 11 May 2007
JMLC 2006
10
Array Structured Object Types Department of Computer Science
Using custom functions PROCEDURE MySum(CONST X: ARRAY [*,*] OF REAL): REAL; VAR i,j: LONGINT; sum: REAL; BEGIN .... (stack)
designator(A)
... adr inc0 len0 inc1 len1
baseadr inc(A,0) len(A,0) inc(A,1) len(A,1)
c := MySum(A[a..b,c..d]); Friday, 11 May 2007
JMLC 2006
different ways to pass an array: PROCEDURE P (CONST X: ARRAY [*,*] OF B); PROCEDURE P (VAR X: ARRAY [*,*] OF B); PROCEDURE P (X: ARRAY [*,*] OF B); PROCEDURE P (...): ARRAY [*,*] OF B; 11
Array Structured Object Types Department of Computer Science
Custom array structured types Array structured object type: Value object type whose signature explicitly contains the array structure, designed for all cases where an array cannot be stored in one linear piece of memory Example: sparse matrices
Definition like an object No inheritance and polymorphism Mimic the behaviour of built-in arrays Operator overloading on module level Prevent the misuse of mathematical ‘indexers’ for general purposes
Friday, 11 May 2007
JMLC 2006
12
Array Structured Object Types Department of Computer Science
Custom Array Types: Sparse Matrix Usage VAR A: ARRAY [10,10] OF REAL; B:SparseMatrix; i: LONGINT; (* ... *) NEW(B,1000,1000); FOR i := 0 TO 999 BY 10 DO B[i..i+9,i..i+9] := A; END; TYPE
SparseMatrix*= ARRAY [..,..] OF REAL VAR d: Data; len0,len1: LONGINT; (* Data defined elsewhere *)
PROCEDURE NEW(i,j: LONGINT); (* allocation *) PROCEDURE LEN(i: LONGINT): LONGINT; (* shape *) PROCEDURE "[]"(i,j: LONGINT): REAL; PROCEDURE "[]"(i,j: LONGINT; r: REAL);
PROCEDURE "[]"(a1..b1 BY c1,a2..b2 BY c2: LONGINT): ARRAY [..,..] OF REAL; PROCEDURE "[]"(a1..b1 BY c1,a2..b2 BY c2: LONGINT; CONST A: ARRAY [..,..] OF REAL);
END SparseMatrix; PROCEDURE ’*’ (A,B: SparseMatrix): SparseMatrix;
Friday, 11 May 2007
JMLC 2006
13
Array Structured Object Types Department of Computer Science
Custom Array Types: Sparse Matrix Interface 2d array structure with TYPE element type real SparseMatrix*= ARRAY [..,..] OF REAL VAR d: Data; len0,len1: LONGINT; (* Data defined elsewhere *) PROCEDURE NEW(i,j: LONGINT); (* allocation *) PROCEDURE LEN(i: LONGINT): LONGINT; (* shape *) PROCEDURE "[]"(i,j: LONGINT): REAL; PROCEDURE "[]"(i,j: LONGINT; r: REAL);
element access
PROCEDURE "[]"(a1..b1 BY c1,a2..b2 BY c2: LONGINT): ARRAY [..,..] OF REAL; PROCEDURE "[]"(a1..b1 BY c1,a2..b2 BY c2: LONGINT; CONST A: ARRAY [..,..] OF REAL); sub-structure access
END SparseMatrix; PROCEDURE ’*’ (A,B: SparseMatrix): SparseMatrix;
Friday, 11 May 2007
JMLC 2006
operator overloading
14
Array Structured Object Types Department of Computer Science
Some implementation details Geometric information is kept separate from array data ∆ runtime measurements have indicated a gain of speed for large arrays codesize sometimes larger compared with pointer oriented approach
Built-in operators implemented as (low-level) library ∆ easily extensible procedure call overhead (in particular w.r.t. small arrays)
Strict value semantics: Array descriptors immutable in sub-procedures ∆ protection against concurrent access ∆ no GC corruption no re-allocation in sub-procedures possible unless wrapped in an object
Friday, 11 May 2007
JMLC 2006
15
Array Structured Object Types Department of Computer Science
Achievements notational compactness in linear algebra and imaging applications language with high potential w.r.t optimization while preserving safety extensibility and flexibility guaranteed by a clean implementation of custom array structured types planned for integration into Zonnon for .NET
Friday, 11 May 2007
JMLC 2006
16
Array Structured Object Types Department of Computer Science
thank you for your attention
Friday, 11 May 2007
JMLC 2006
17
Array Structured Object Types Department of Computer Science
Discarded Ideas no separate type for the domain S no variable lower bound no super-dynamic features (like appending, insertion and deletion of elements) no free boundary conditions (like ‘mirrored’, ‘wrapped’ etc.) no properties (like ‘diagonal’, ‘symmetric’ etc.)
Friday, 11 May 2007
JMLC 2006
18
Array Structured Object Types Department of Computer Science
Concepts of the built-in arrays Ranged indices (‘i..j BY k’) as array designators. @ readabilty @ efficiency: identification of independent pieces of code @ safety: range checks on sub-domains
Value semantics @ structural simplicity: memory allocation and pointer mechanisms behind the scenes @ clearness of operations (assignment, equality) @ protection against unintentional concurrent access
Notation near to that of Matlab @ acceptance
Operators are part of the language Friday, 11 May 2007
JMLC 2006
19
Array Structured Object Types Department of Computer Science
Oberon ideal language for mathematical modeling clearity and readabilty more abstract and closer to mathematics than system near languages experiences / libraries @ Voyager (G.Sawitzki, Heidelberg): data analysis, statistics, visualisation, stochastic simulation @ Ants (FF, Munich): random fields, image analysis, mathematical modelling in the life sciences @ CAT/CAPO (A.Freed, Cleveland, FF): linear algebra package @ Mathematical modelling in medicine (P.Hunziker, Basel): wavelets, image analysis, linear algebra
Friday, 11 May 2007
JMLC 2006
20
Array Structured Object Types Department of Computer Science
Matrix multiplication: naive approach VAR A,B,Res: ARRAY [*,*] OF REAL; i,j,k: LONGINT; temp: REAL; (* ... *) (* check shapes *) FOR i := 0 TO LEN(L,0)-1 DO FOR j := 0 TO LEN(R,1)-1 DO temp := 0; FOR k := 0 TO LEN(R,0)-1 DO temp := temp + L[i,k]*R[k,j]; END; Res[i,j] := temp; END; END;
Friday, 11 May 2007
JMLC 2006
long and not intuitive Computationally expensive, cache misses may occur
21
Array Structured Object Types Department of Computer Science
Example: matrix-multiplication 2. Replacement of inner loop VAR A,B,Res: ARRAY [*,*] OF REAL; i,j: LONGINT; (* ... *) (* check shapes *) FOR i := 0 TO LEN(L,0)-1 DO FOR j := 0 TO LEN(R,1)-1 DO Res[i,j] := L[i,..] +* R[..,j]; (* scalar product *) END; END; Optimization possible Friday, 11 May 2007
JMLC 2006
22
Array Structured Object Types Department of Computer Science
matrix-multiplication as operator Important operators are made available by the language VAR A,B,Res: ARRAY [*,*] OF REAL; (* ... *) transposition
Res := A * B`; matrix multiplication
Friday, 11 May 2007
optimization possible (e.g. SSE2 instructions) intuitive close to mathematical notation implicit allocation possible JMLC 2006
23
Array Structured Object Types Department of Computer Science
Scope Ideal case: development of theoretically better algorithms providing lower complexity. Additionally: by exploiting structure of the problem code can be made clearer and more efficient Objective: Object oriented concept of arraystructured types. Goal: intuitive and efficient mathematical programming. Used Language: (Active) Oberon Friday, 11 May 2007
JMLC 2006
24
Array Structured Object Types Department of Computer Science
Code snippet: Singular Value Decomposition scale := 0; FOR k := i TO m - 1 DO scale := scale + ABS( u[k, i] ) END; IF (scale # 0) THEN FOR k := i TO m - 1 DO u[k, i] := u[k, i]/scale; s := s + u[k, i]*u[k, i]; END; f := u[i, i]; IF f >= 0 THEN g := -sqrt( s ) ELSE g := sqrt( s ) END; h := f*g - s; u[i, i] := ( f - g); FOR j := l TO n - 1 DO s := 0; FOR k := i TO m - 1 DO s := s + u[k, i]*u[k, j] END; f := s/h; FOR k := i TO m - 1 DO u[k, j] := u[k, j] + f*u[k, i] END; END; FOR k := i TO m - 1 DO u[k, i] := u[k, i]*scale END; END
Friday, 11 May 2007
JMLC 2006
25
Array Structured Object Types Department of Computer Science
Code snippet: Singular Value Decomposition scale := SUM(ABS(u[i..m-1,i])); IF (scale # 0) THEN u[i..m-1,i] := u[i..m-1,i]/scale; s := s + u[i..m-1,i] +* u[i..m-1,i]; f := u[i, i]; IF f >= 0 THEN g := -sqrt( s ) ELSE g := sqrt( s ) END; h := f*g - s; u[i, i] := ( f - g); FOR j := l TO n - 1 DO s := u[i..m-1,i] +* u[i..m-1,j]; f := s/h; u[i..m-1,j] := u[i..m-1,j] + f*u[i..m-1,i] END; u[i..m-1, i] := u[i..m-1, i]*scale END
Friday, 11 May 2007
JMLC 2006
26
