Page 1. Clojure, Functional. Programming and Data at. uSwitch.com. Paul Ingles.
@pingles. Page 2. Page 3. Data Systems. Page 4. Data Systems.
Clojure, Functional Programming and Data at uSwitch.com Paul Ingles @pingles
Data Systems
Functional Data Systems
Everything as Data
Code as Data (conj fpdays-attendees me)
Data as Data {:name "Paul Ingles" :mood (repeat :happy)}
Builds as Data (defproject energy-comparison "1.0.0" :description "pricing service" :dependencies [[org.clojure/clojure "1.4.0"] ...)
Everything as Data • Data, in its most essential form • Few simple abstractions • map,
seq, fn
• General library • map,
filter, sort, take
Logs as Data?
Logs as Data?
0.0.0.0 - - [22/May/2012:09:00:03 +0000] "GET / HTTP/1.1" 304 0 "SomeUserAgent"
Logs as Files
/var/log/nginx/access.log
current power < potential power
log crunching
event processing
fn
S0
fn
S1
Time
S2
Identity
log crunching
event processing
Dysfunctional Data
Complexity Themes
Complexity Themes • Confound storage and processing of data • Current state only • Data in its non-essential form
current power < potential power
Data Muddling Energy
Replication, ETL
Insurance
Credit Cards
Broadband
Session DB
Warehouse
Source: http://www.irregulartimes.com/panopt.html
Energy
Insurance
Integration DB
Credit Cards
Broadband
data in its essential form?
simple abstractions?
Functional Inspiration
The Joy of Clojure • Functional • Separation of computation and data • Immutable state • Data as data • Time, state and identity
Users as Data?
user = fn(interactions)
User fn
S0
fn
S1
Time
S2
User (Identity)
data > current state
data outlives applications
current state snapshots
event processing
“It is better to skip the massive upfront time and expense, focusing instead on making it very fast and easy to add new data sources or new elements to existing sources.”
Source: http://thinkrelevance.com/blog/2012/04/04/big-data-reference-model
data warehousing
data ecosystem
data ecosystem
Producer
Producer
Producer
Aggregator
Consumer
Consumer Consumer
events carrying smaller, simpler pieces of data
{:user-id 1234 :email "
[email protected]"}
{:user-id 1234 :postcode "NW3 2LB"}
{:method :GET :uri “/” :user-id 1234}
{:user-id 1234 :energy-consumption {:gas {:units :kwh :value 20500}}}
Dysfunctional Data
Functional Data
Functional Data • Immutability • Time, state and identity • Isolate storage and processing • Essence of data
Datomic
Examples...
http://github.com/pingles/clj-esper
$ ./print.sh -u
(->> (events aggregator) (filter (criteria :user-id "pingles")))
Functional Data at uSwitch.com
Thank you!
Thank you! @pingles http://www.oobaloo.co.uk
![[06;53;11] - Free Download Programming Clojure ... - WordPress.com](https://m.moam.info/img/260x300/065311-free-download-programming-clojure-wordpress_647a2f94097c476a028c78cd.jpg)
