Clojure - A small introduction - JUG Münster Wiki

Clojure A small introduction Alex Ott http://alexott.net

M¨ unster JUG, July 2010

Alex Ott (alexott@gmail .com)

Clojure


1 / 40

About me... Professional software developer & architect Working with different programming languages – C/C++, Lisps, Haskell, Erlang, ... About 20 years of Lisp experience, including commercial projects Development on Unix/Linux platforms Author of articles on programming (including Clojure), Emacs, etc. Participate in many open source projects, including Incanter, labrepl, etc. Maintainer of “Planet Clojure”


Clojure


2 / 40

Outline 1

What is Clojure?

2

Language basics

3

Interoperability with Java

4

Concurrent programming

5

Clojure in real life

6

Sources of information

7

Examples


Clojure


3 / 40

What is Clojure? Lisp-like language, created by Rich Hickey and announced in 2007th Designed to work on existing platforms Functional programming language, with immutable data Special features for concurrent programming Open sourced with liberal license Already used in many projects, including commercial


Clojure


4 / 40

Why new language? Lisp, but without compatibility with previous versions Immutable data and more support for pure functional programming comparing with other Lisps Support for concurrent programing inside language Better integration with target platforms: JVM & .Net


Clojure


5 / 40

Main features Dynamically typed language Very simple syntax, like in other Lisps (code as data) Support for interactive development Designed in terms of abstractions Metaprogramming Multimethods and protocols (in version 1.2) Compiled into byte code of target platforms Access to big number of available libraries


Clojure


6 / 40

Differences from other Lisps Changes in syntax, less parentheses Changes in naming More first-class data structures – maps, sets, vectors Immutable, by default, data Ability to link meta-information with variables and functions “Lazy” collections There is no reader macros Case-sensitive names Lack of tail call optimization (JVM’s limitation) – explicit loops loop/recur Exceptions instead of signals and restarts


Clojure


7 / 40

Base data types Integer numbers of any size – 14235344564564564564 Rational numbers – 26/7 Real numbers – 1.2345 and BigDecimals – 1.2345M Strings (String class from Java) – "hello world" Characters (Character from Java) – \a, \b, . . . Regular expressions – #"[abc]*" Boolean – true & false nil – the same as null in Java, not an empty list as in Lisp Symbols – test, var1, . . . Keywords – :test, :hello, . . .


Clojure


8 / 40

Data structures Separate syntax for different collections: Lists – (1 2 3 "abc") Vectors – [1 2 3 "abc"] Maps – {:key1 1234 :key2 "value"} Sets – #{:val1 "text" 1 2 10}

Sequence – abstraction to work with all collections (including classes from Java/.Net) Common set of functions to work with sequences Lazy operations on sequences Vectors, maps, and sets are functions – to simplify access to data in them Persistent collections Transient collections – performance optimization defstruct – optimization of map to work with complex data structures Alex Ott (alexott@gmail .com)

Clojure


9 / 40

Syntax Very simple, homoiconic syntax – program is usual data structure Special procedure (reader) process text and produce data structures All objects represent themselves except symbols and lists Symbols link value with “variable” Lists represent forms, that could be: Special form – def, let, if, loop, . . . Macros Function call or expression, that could be used as function (maps, keywords, returned functional values, . . . )

To get list as-is you need to “quote” it (with quote or ’)


Clojure


10 / 40

Code example ( defn f i b o ( [ ] ( concat [1 1] ( f i b o 1 1))) ([ a b] ( l e t [ n (+ a b ) ] ( l a z y −s e q ( c o n s n ( f i b o b n ) ) ) ) ) ) ( t a k e 100000000 ( f i b o ) ) ( defn vrange2 [ n ] ( loop [ i 0 v ( transient [])] ( i f (< i n ) ( recur ( inc i ) ( conj ! v i )) ( persistent ! v ))))


Clojure


11 / 40

Life cycle of Clojure program Code is read from file, or REPL, or from other code Reader transforms program into data structures Macros are expanded Resulting code is evaluated/compiled Produced bytecode is executed by JVM File Data structures

Code Reader

Evaluator/ Compiler

bytecode

JVM

Data Data structures structures

Code

Execution Macroexpand

Developer (REPL) Alex Ott (alexott@gmail .com)

Clojure


12 / 40

Functions Functions are first-class objects Function can have variable number of arguments Function’s definition: defn – top-level functions, fn – anonymous functions Simplified syntax for anonymous functions – #(code). For example: (map #(.toUpperCase %) ["test" "hello"]) (map #(vector %1 %2) [1 2 3] [4 5 6]) You can specify tests & pre/post-conditions. For example, ( d e f n c o n s t r a i n e d −s q r [ x ] { : p r e [ ( pos ? x ) ] : p o s t [( > % 1 6 ) , (< % 2 2 5 ) ] } (∗ x x ) ) Each function is compiled into separate Java class Each function implements Runnable interface Alex Ott (alexott@gmail .com)

Clojure


13 / 40

Metaprogramming & macros Macros receives source code and returns new code, that will compiled Big part of language is written using macros Variable number of arguments (same as functions) Code could be generated with list functions, but it’s much handy to use quasi-quote – ‘ and substitution operators – ~ and ~@ The # suffix in names is used to generate unique names macroexpand-1 & macroexpand are used to debug macros


Clojure


14 / 40

Macros examples (1) The standard macros when: ( d e f m a c r o when [ t e s t & body ] ( l i s t ’ i f t e s t ( c o n s ’ do body ) ) ) when used as: ( when ( pos ? a ) ( p r i n t l n " p o s i t i v e ") (/ b a ) ) will expanded into: ( i f ( pos ? a ) ( do ( p r i n t l n " p o s i t i v e ") (/ b a ) ) )


Clojure


15 / 40

Macros examples (2) The standard macros and ( d e f m a c r o and ( [ ] true ) ([ x] x) ( [ x & next ] ‘ ( l e t [ and# ~x ] ( i f and# ( and ~@next ) and # ) ) ) ) will expanded into different code, depending on number of arguments: user> user> user> ( let∗ ( if

( macroexpand ’ ( and ) ) ( macroexpand ’ ( and (= ( macroexpand ’ ( and (= [ and__4457__auto__ (= and__4457__auto__ ( c l o j u r e . c o r e / and (= 3 and__4457__auto__ ) )


Clojure

==> t r u e 1 2 ) ) ) ==> (= 1 2 ) 1 2 ) (= 3 3 ) ) ) ==> 1 2)] 3)) M¨ unster JUG, July 2010

16 / 40

Polymorphism & multimethods Extensibility Multimethods aren’t bound exclusively to types/classes Dispatching is performed using results of user-specified dispatch function Dispatching on several parameters Differs from CLOS – absence of :before, :after, . . . defmulti – is analog of defgeneric in Common Lisp Defined as ( defmulti func−name dispatch−fn) + set of implementations (via defmethod) You can also define hierarchical relationships with derive, and use them in dispatch (via isa?)


Clojure


17 / 40

Multimethods examples (1) Simple dispatching using data type/class: ( d e f m u l t i m−e x a m p l e c l a s s ) ( d e f m e t h od m−e x a m p l e S t r i n g [ t h i s ] ( p r i n t l n " This i s s t r i n g ’" t h i s " ’")) ( d e f m e t h od m−e x a m p l e j a v a . u t i l . C o l l e c t i o n [ t h i s ] ( p r i n t " This i s c o l l e c t i o n ! " ) ) and we’ll get: (m−e x a m p l e " H e l l o " ) ==> " T h i s i s s t r i n g ’ H e l l o ’ " (m−e x a m p l e [ 1 2 3 ] ) ==> " T h i s i s c o l l e c t i o n ! " (m−e x a m p l e ’ ( 1 2 3 ) ) ==> " T h i s i s c o l l e c t i o n ! "


Clojure


18 / 40

Multimethods examples (2) ( defmulti encounter ( fn [ x y ] [ ( : Species x ) ( : Species y ) ] ) ) ( d e f m e t h od e n c o u n t e r [ : Bunny : L i o n ] [ b l ] : run−away ) ( d e f m e t h od e n c o u n t e r [ : L i o n : Bunny ] [ l b ] : e a t ) ( d e f m e t h od e n c o u n t e r [ : L i o n : L i o n ] [ l 1 l 2 ] : f i g h t ) ( d e f m e t h od e n c o u n t e r [ : Bunny : Bunny ] [ b1 b2 ] : mate ) ( d e f b1 { : S p e c i e s : Bunny } ) ( d e f b2 { : S p e c i e s : Bunny } ) ( def l 1 { : S p e c i e s : Lion }) ( def l 2 { : S p e c i e s : Lion }) ( encounter ( encounter ( encounter ( encounter

b1 b1 l1 l1

b2 ) l1 ) b1 ) l2 )


==> ==> ==> ==>

: mate : run−away : eat : fight Clojure


19 / 40

Protocols & Datatypes Introduced in version 1.2 (will released shortly) Much faster than multimethods Allows to write “Clojure in Clojure” Dispatching is done only on data types Similar to type classes in Haskell Java interface is created for each protocol defrecord & deftype define new data types extend-protocol & extend-type bind protocol(s) with data types (not only with defined in Clojure!) reify is used to implement protocols and interfaces for “once-used types”


Clojure


20 / 40

Protocol’s examples ( d e f p r o t o c o l H e l l o " Test of p r o t o c o l " ( h e l l o [ t h i s ] " h e l l o function ")) ( d e f r e c o r d B [ name ] H e l l o ( h e l l o [ t h i s ] ( s t r " H e l l o " ( : name t h i s ) " ! " ) ) ) ( h e l l o (B . " U s e r " ) ) ==> " H e l l o U s e r ! " ( e x t e n d −p r o t o c o l H e l l o S t r i n g ( hello [ this ] ( str " Hello " this "!"))) ( h e l l o " w o r l d " ) ==> " H e l l o w o r l d ! " ( e x t e n d −p r o t o c o l H e l l o j a v a . l a n g . O b j e c t ( h e l l o [ t h i s ] ( s t r " H e l l o ’" t h i s " ’ ! (" ( type t h i s ) " ) " ) ) ) ( h e l l o 1 ) ==> " H e l l o ’ 1 ’ ! ( c l a s s j a v a . l a n g . I n t e g e r ) " Alex Ott (alexott@gmail .com)

Clojure


21 / 40

Miscellaneous Metadata – #^{} in 1.0 & 1.1, or simply ^{} in 1.2 Optional type specification (type hints) – #^Type or ^Type You can specify tests directly in function’s declaration Scope management Access and change of metadata from code Don’t change value equality

Namespaces: are first-class objects are used to organize code into libraries

Data destructuring (function arguments, etc.) ( l e t [ [ a b & c : as e ] [1 2 3 4 ] ] [ a b c e ] ) ==> [ 1 2 ( 3 4 ) [ 1 2 3 4 ] ] ( l e t [ { : k e y s [ a b c ] : a s m : o r {b 3}} { : a 5 : c 6 } ] [ a b c m] ) ==> [ 5 3 6 { : a 5 : c 6 } ]


Clojure


22 / 40

Interoperability with Java Two-way interoperability with target platform: Creation of instances – new or Class. Call of Java methods – ., .., doto Class & interface generation – gen-class, gen-interface, or proxy (anonymous class)

You can execute Clojure code in Java programs Separate functions to work with Java arrays: make-array – array creation aget/aset – access to array’s elements amap/areduce – iteration over array’s elements

memfn allows to use Java methods as arguments of map, filter, etc. The set! special form to set values inside class Generation and catch of exception – throw & catch


Clojure


23 / 40

Interoperability examples Instance creation: ( new j a v a . u t i l . Date ) ( j a v a . u t i l . Date . ) Call of Java methods and access to data: ( . s u b s t r i n g " H e l l o World " 0 5 ) ==> " H e l l o " ( . " H e l l o World " s u b s t r i n g 0 5 ) ==> " H e l l o " Math/ PI ==> 3 . 1 4 1 5 9 2 6 5 3 5 8 9 7 9 3 ( I n t e g e r / p a r s e I n t " 4 2 " ) ==> 42 .. is used to chained calls: ( . . System g e t P r o p e r t i e s ( g e t " o s . name " ) ) ==> "Mac System . g e t P r o p e r t i e s ( ) . g e t ( " o s . name " ) doto – call of several methods for one object: ( d o t o ( j a v a . u t i l . HashMap . ) ( . p u t " a " 1 ) ( . p u t "b" 2 ) ) Alex Ott (alexott@gmail .com)

Clojure


24 / 40

Concurrent programming Special language features, that provides data mutation: Refs – synchronous, coordinated change Agents – asynchronous, not coordinated Atoms – synchronous, not coordinated Vars – local for current thread @ (deref) is used to get access to data

Parallel code execution future pmap, pvalues & pcalls Native threads

Synchronization with promise


Clojure


25 / 40

State and Identity Clojure separates concepts of state and identity State (value) isn’t changed! More information at http://clojure.org/state Code (past)

Code (present)

Code (future)

Identity (ref/agent/atom name)

State S1 [1 2 3 4]


State S2 [1 2 3 4 5 6 7]

Clojure

State S3 [?????]


26 / 40

Refs & STM Coordinated change of data from several threads Software Transaction Memory provides atomicity, consistency, and isolation Changes could be made only inside transaction You can add function for data validation Ability to add watcher function ( def counters ( r e f {})) ( d e f n get −c o u n t e r [ k e y ] ( @counters key 0)) ( d e f n i n c r e m e n t −c o u n t e r [ k e y ] ( dosync ( a l t e r c o u n t e r s a s s o c key ( i n c ( @counters key 0 ) ) ) ) ) Alex Ott (alexott@gmail .com)

Clojure


27 / 40

Agents Asynchronous change of data – “fire and forget” Thread pools for data update functions: send – for “fast” functions, and send-off – for “heavyweight” functions (separate thread pool) send & send-off get function that will applied to agent’s current state Validators and watchers You can wait until finish of all updates ( def acounters ( agent {})) ( d e f n i n c r e m e n t −a c o u n t e r [ k e y ] ( send a c o u n t e r s a s s o c key ( i n c ( @counters key 0 ) ) ) ) ( d e f n get −a c o u n t e r [ k e y ] ( @acounters key 0)) Alex Ott (alexott@gmail .com)

Clojure


28 / 40

Vars & Atoms Atoms Synchronous data change without coordination Data are changed by applying function to current state of atom Function could be called several times Function shouldn’t have side effects!

Vars Provide data change only in current thread binding links new values with symbols Change affects all functions, called from current thread Be careful with lazy sequences!

( def ∗ var ∗ 5) ( defn foo [ ] ∗ var ∗) ( f o o ) ==> 5 ( b i n d i n g [ ∗ v a r ∗ 1 0 ] ( f o o ) ) ==> 10


Clojure


29 / 40

Parallel code execution future executes given code in separate thread @ is used to get results of code execution @ blocks current thread, if results aren’t available yet results of code execution are cached

promise is used for synchronization between parts of programs deliver sets value of promise @ reads value, or blocks execution if value isn’t set yet

pmap, pvalues & pcalls are used for “heavyweight” operations, that could be performed in parallel


Clojure


30 / 40

Clojure in real life Existing infrastructure: IDEs Build tools Debugging and related tools Libraries Code repositories

Clojure/core – commercial support & consulting Clojure is used in commercial projects since 2008 Sources of information


Clojure


31 / 40

Infrastructure: IDEs and build tools Supported by most of popular IDEs: Emacs + SLIME (most popular now) VimClojure NetBeans Eclipse IntelliJ IDEA

Build tools: Clojure support in Maven, Ant and Gradle Leiningen – written in Clojure, extensible and very simple


Clojure


32 / 40

Infrastructure: libraries & repositories Simple access to big number of existing Java libraries Clojure-specific libraries: Clojure-Contrib – “semi-standard” libraries Compojure, Ring, Scriptjure – Web development ClojureQL – databases Incanter – R-like environment and libraries other – see at http://clojars.org

Repositories: build.clojure.org clojars.org


Clojure


33 / 40

Commercial projects FlightCaster – machine learning, etc. Relevance, Inc. – consulting Runa – marketing services Sonian Networks – archiving solutions BackType – social media analytics DRW Trading Group – trading and finance DocuHarvest project by Snowtide Informatics Systems, Inc. ThorTech Solutions – scalable, mission-critical solutions TheDeadline project by freiheit.com (http://www.freiheit.com/) and many more. . .


Clojure


34 / 40

Sources of information (1) Main sites: Site of the language – http://clojure.org Planet Clojure – http://planet.clojure.in the #clojure IRC channel at freenode.net The labrepl project (http://github.com/relevance/labrepl) – learning environment Try-Clojure (http://www.try-clojure.org/) – you can execute Clojure code via Web-browser http://clojuredocs.org/ – documentation and examples

German resources: Book in German will released in 2010 fall (http://www.clojure-buch.de/) clojure-de mailing list (http://groups.google.com/group/clojure-de) videos about Clojure in German (http://www.rheinjug.de/videos/ gse.lectures.app/Talk.html#Clojure)


Clojure


35 / 40

Sources of information (2) Books: Programming Clojure – 2009th, Clojure v. 1.0 Practical Clojure. The Definitive Guide – 2010th, Clojure, v. 1.2 The Joy of Clojure (beta) Clojure in Action (beta) Clojure Programming on Wikibooks Clojure Notes on RubyLearning

Video-lectures & screencasts Clojure user groups around the world Clojure study courses (on-site in USA & Europe, or online)


Clojure


36 / 40

Examples. Hadoop Hadoop word count example is only lines of code (using clojure-hadoop), instead dozens for Java... All low-level details are handled by macros ( n s c l o j u r e −hadoop . e x a m p l e s . w o r d c o u n t 3 ( : import ( java . u t i l StringTokenizer )) ) ( d e f n my−map [ k e y v a l u e ] ( map ( f n [ t o k e n ] [ t o k e n 1 ] ) ( e n u m e r a t i o n −s e q ( S t r i n g T o k e n i z e r . v a l u e ) ) ) ) ( d e f n my−r e d u c e [ k e y v a l u e s −f n ] [ [ k e y ( r e d u c e + ( v a l u e s −f n ) ) ] ] )


Clojure


37 / 40

Examples. Cascalog Domain specific language to query data in Hadoop with joins, aggregates, custom operations, subqueries, etc. Interactive work with data, using Clojure Available from http://github.com/nathanmarz/cascalog Example: query for persons younger than 30: (?

Clojure - A small introduction - JUG Münster Wiki

Clojure - A small introduction - JUG Münster Wiki

Suggest Documents

Clojure In Small Pieces - Timothy Daly

Clojure In Small Pieces - Timothy Daly

Clojure In Small Pieces - Timothy Daly

Content strategy: a brief introduction - Wiki

Clojure: A Critical Dissection - KTH

Hadoop + Clojure

Clojure data visualization Clojure C web visualization - Keming Labs

Asynchronous Events in Clojure

Bouillon: a wiki-wiki social web - CiteSeerX

Clojure Development with JBoss

The joy of clojure

MAPS - Jug Bay Wetlands Sanctuary

STM in Clojure

Clojure Web Development - Jfokus

Clojure and concurrency

ESM Wiki [ESM Wiki]

Introduction to Small Engines

Jug Kettle - Applica Use and Care Manuals

Using Clojure in Linguistic Computing

Boston Clojure Meetup - Google Groups

Lessons Learned from Adopting Clojure

Brede Wiki: Neuroscience data structured in a wiki

Lessons Learned from Adopting Clojure

Using Clojure in Linguistic Computing