Interactive Data Visualization for the Web. An Introduction to Designing With D3.
Scott Murray. Publisher: O'Reilly Media (March, 2013). ISBN-10: 1449339735 ...
Data Visualization Lecture 09 (Final) Colman McMahon
[email protected]
2012/13
1
Contact Details Colman McMahon
►
Email -
[email protected]
►
Notes - www.thedata.co/teaching/dit-viz-20122
►
Sharing - facebook.com/TheDataCo
Lectures: ►
Tuesdays 18:30-21:30, KA-3-006
2012/13
3
Agenda Housekeeping ►
Download latest version of “D3 Tips and Tricks” (March, 2013) and “Interactive Data Visualization”
►
Sign-in sheet
►
Notices
►
Lecture dates
Lecture 1) Recap 2) Examine Basic Graph and understand the code 3) Create a database-driven D3 graph
2012/13
4
Notices ►
Accenture's "Analytics Innovation Centre" (Dublin) want to hire 180 data specialists between now and 2015. From analyst to manager levels, MSc's/PhD's/post-docs, industry experts (particularly finance). R, SAS, SPSS, Matlab, and other technologies.
►
http://lnkd.in/WUYB9p
►
Hiring manager: David Rohan »
Email:
[email protected]
►
2012/13
5
Lecture Dates http://www.dit.ie/academicaffairsandregistrar/calendar
2012/13
Week 01
Lecture 01 - 29/01/13
Week 02
Lecture 02 - 05/02/13
Week 03
Lecture 03 - 12/02/13
Week 04
Lecture 04 - 19/02/13
Week 05
Lecture 05 - 26/02/13
Week 06
Lecture 06 - 05/03/13
Week 07
Lecture 07 - 12/03/13
Week 08
Lecture 08 - 19/03/13
Week 09
Easter Hols - 26/03/13
Week 10
Easter Hols - 02/04/13
Week 11
Lecture 09 - 09/04/13
Week 12
Lecture 10 - 16/04/13
Week 13
Review Wk - 23/04/13
Week 14
Lecture 11 - 30/04/13
Week 15
Lecture 12 - 07/05/13
St. Patrick's Weekend
May Bank Holiday Weekend
6
Module - Syllabus INTRO
Lecture 01
Introductions, course overview Warm-up visualization exercise
PART 1 Principles
Lecture 02
Illinsky & Steele - Chs 1-3 Install R and R-Studio, install colour palette
Lecture 03
Illinsky & Steele - Chs 4-6 Install Inkscape + mini-tutorial
Lecture 04
Yau - Ch 2 "Handling Data" Install Python, Beautiful Soup + screen scraping
Lecture 05
Yau - Ch 4 "Patterns Over Time" Lab tutorial
Lecture 06
Yau - Ch 6 "Relationships" Lab tutorial
Lecture 07
Yau - Ch 8 "Spatial Relationships" (excl. animation) Lab tutorial
Lecture 08
Introduction, Technology Fundamentals, Setup
Lecture 09
Data, Drawing with Data
Lecture 10
Updates, Transitions, Interactivity
Lecture 11
Geomapping
Lecture 12
Presentations, review, exam prep
PART 2 R & Inkscape
PART 3 D3.js
END 2012/13
7
Agenda Housekeeping ►
Download latest version of “D3 Tips and Tricks” (March, 2013) and “Interactive Data Visualization”
►
Sign-in sheet
►
Notices
►
Lecture dates
Lecture 1) Recap 2) Examine Basic Graph and understand the code 3) Create a database-driven D3 graph
2012/13
9
D3.js d3js.org
2012/13
10
Interactive Data Visualization for the Web An Introduction to Designing With D3
Scott Murray Publisher: O'Reilly Media (March, 2013) ISBN-10: 1449339735
2012/13
11
D3 Tips and Tricks D3noob
http://www.d3noob.org/p/d3noob-downloads.html https://leanpub.com/D3-Tips-and-Tricks
2012/13
12
Technology Fundamentals 1) The Web 2) HTML 3) Document Object Model (DOM) 4) CSS 5) JavaScript 6) Scalable Vector Graphics (SVG)
Some extremely basic definitions to follow... 2012/13
13
Install Local Webserver
Doubleclick....
(if a warning dialog appears click the "Allow" option to install XAMPP)
14
2012/13
Source: http://sawmac.com/xampp/
Getting & Installing D3 ►
Follow instructions Murray, Chapter 4
►
Download the latest version here:
►
http://d3js.org/d3.v3.zip
►
Unzip d3.js into “htdocs” folder
2012/13
15
2012/13
16
2012/13
17
Agenda Housekeeping ►
Download latest version of “D3 Tips and Tricks” (March, 2013) and “Interactive Data Visualization”
►
Sign-in sheet
►
Notices
►
Lecture dates
Lecture 1) Recap 2) Examine Basic Graph and understand the code 3) Create a database-driven D3 graph
2012/13
18
"Chain syntax" ►
By “chaining” methods together with periods (.) we can perform several actions in a single line of code
►
This code:
►
Could be written:
2012/13
19
Murray, “Interactive Data Visualization for the Web”
The Basic Graph
2012/13
21
HTML Looks like a wrapper for the CSS and JavaScript...
identifies the file (and location) that needs to be loaded to get D3 up and running
2012/13
22
SVG canvas
Page
2012/13
23
CSS Cascading Style Sheets - controls the look/feel/presentation of the content
2012/13
24
Changing CSS values
2012/13
25
D3 script - steps 1) Set up margins and graph area 2) Format the date 3) Scales and Ranges 4) Set the axes 5) Add data to the line function 6) Add the SVG canvas 7) Get the data 8) Scale the range of the data 9) Draw the “valueline” path 10) Draw the X and Y axes
2012/13
26
D3 JavaScript code
2012/13
27
D3 JavaScript code
2012/13
28
D3 JavaScript code Setting up margins and graph area Formatting date Scales and Ranges Setting axes
Adding data to the line function
Adding the SVG canvas 2012/13
29
Setting up margins and graph area
2012/13
30
D3 JavaScript code Setting up margins and graph area Formatting date Scales and Ranges Setting axes
Adding data to the line function
Adding the SVG canvas 2012/13
31
Scaling ►
The purpose of these portions of the script is to ensure that the data we ingest fits onto our graph correctly
►
First we make sure that any quantity we specify on the x axis fits onto our graph. var x = d3.time.scale().range([0, width]);
►
Here we set our variable that will tell D3 where to draw something on the x axis. By using the d3.time.scale() function we make sure that D3 knows to treat the values as date / time entities
►
Then we specify the range that those values will cover (.range) and we specify the range as being from 0 to the width of our graphing area
►
Then we do the same for the Y axis.
var y = d3.scale.linear().range([height, 0]); 2012/13
32
Scaling
2012/13
33
►
Useful to do if the data was concentrated in a
Scaling
narrow range of values that are quite distant from zero
2012/13
35
D3 JavaScript code Setting up margins and graph area Formatting date Scales and Ranges Setting axes
Adding data to the line function
Adding the SVG canvas 2012/13
36
Axes ►
Generates the visual elements of the axes, including lines, labels, and ticks
2012/13
37
D3 JavaScript code Setting up margins and graph area Formatting date Scales and Ranges Setting axes
Adding data to the line function
Adding the SVG canvas 2012/13
38
Adding data to the line function
►
Each time this line function is called on, it will go through the data will assign coordinates to ‘date’ and ‘close’ pairs using the ‘x’ and ‘y’ functions
2012/13
39
D3 JavaScript code Setting up margins and graph area Formatting date Scales and Ranges Setting axes
Adding data to the line function
Adding the SVG canvas 2012/13
40
Adding the SVG canvas ►
We are ‘appending’ an SVG element (a canvas that we are going to draw things on) to the element of the HTML page »
D3 needs have a space defined for it to draw things • can give the space a name, attributes and positions
2012/13
41
Getting our data
Scale data range Add “valueline” path
Add X and Y axes
2012/13
42
the data file to load (its “url”)
invokes the d3.tsv request
Tells the script to carry out a function on the data (which will now be called ‘data’)
d3.tsv("data/data.tsv", function(error, data) { data.forEach(function(d) { Sets the data variable that is being dealt with (slightly confusingly called, ‘data’) and tells the block of code that, for each group within the ‘data’ array it should carry out a function on it
d.date = parseDate(d.date); d.close = +d.close;
Takes the raw date information from the .tsv file in a specific row and converts it into a format that D3 can then process
});
sets the ‘close’ variable to a numeric value (if it isn’t already) using the ‘+’ operator
("%d%b%y")
This block of code - picked up a file with data in it of a particular type (.tsv) and ensured that it is formatted in a way that the rest of the script can use correctly 2012/13
43
Getting our data
Scale data range Add “valueline” path
Add X and Y axes
2012/13
44
Setting Domain range ►
We haven’t actually told D3 what the range of the data is.
►
The .domain function is designed to let D3 know what the scope of the data will be »
►
this is what is then passed to the scale function
X-axis - the domain for the x-axis, values will be determined by the d3.extent function which in turn is acting on a separate function which looks through all the ‘date’ values that occur in the ‘data’ array »
►
from the 26th of March 2012 till 1st of May 2012
Y-axis - sorts through all the ‘close’ values in the ‘data’ array and returns the largest one »
2012/13
domain is from 0 to 636.23
45
Getting our data
Scale data range Add “valueline” path
Add X and Y axes
2012/13
46
Add “valueline” path Paths - the outline of a shape which can be filled, stroked, used as a clipping path, or any
►
combination of the three
(http://www.w3.org/TR/SVG/paths.html)
►
The svg.append("path") portion adds a new path element
►
valueline(data) passes the ‘valueline’ array (with its x and y coordinates) to the path element »
2012/13
Creates an .svg element which is a path going from one set of ‘valueline’ coordinates to another
47
Getting our data
Scale data range Add “valueline” path
Add X and Y axes
2012/13
48
Drawing axis on canvas ►
Covered formatting of axis components earlier - this part is about getting those components drawn onto the canvas...
2012/13
49
“Basic Graph” http://localhost/simple-graph.html
2012/13
50
D3 script - steps 1) Set up margins and graph area 2) Format the date 3) Scales and Ranges 4) Set the axes 5) Add data to the line function 6) Add the SVG canvas 7) Get the data 8) Scale the range of the data 9) Draw the “valueline” path 10) Draw the X and Y axes
2012/13
51
Applying Encodings ►
Add Axis Labels
►
Add a title
►
Smoothing graph lines
►
Add grid lines
►
Make a dashed line
►
Fill area under the graph
►
Add more than one line
►
Multiple axes
►
Rotate text labels for the x-axis
►
Format a date / time (skip)
2012/13
52
Dynamics features ►
Randomised data (Murray, ch6)
►
Updates, Transitions, Motion (Murray, ch 9)
►
Interactivity (Murray, ch 10)
2012/13
53
Agenda Housekeeping ►
Download latest version of “D3 Tips and Tricks” (March, 2013) and “Interactive Data Visualization”
►
Sign-in sheet
►
Notices
►
Lecture dates
Lecture 1) Recap 2) Examine Basic Graph and understand the code 3) Create a database-driven D3 graph
2012/13
54
D3 + database
2012/13
55
Steps involved 1) Create a new MySQL database 2) Create a new user 3) Create a table 4) Create columns 5) Import data file (data.tsv) 6) Put php script (“data2.php”) into PHP folder 7) Amend the D3 script to call “data2.php” 8) “data.php” converts data in MySQL database into JSON format 9) D3 script ingests JSON formatted data and populates? graph
2012/13
56
2012/13
57
Create new database
homedb
2012/13
58
New database created
2012/13
59
Add a user to database
Click on “Privileges” and select ‘Add a new user’
2012/13
60
New User details
homedbuser homedbuser
2012/13
61
Create a table called “data2” with 3 columns Choose “data2” as a name since we will put the same data as we have in the “data2.tsv” file. (three columns for the “date”, “close” and “open” columns in the data2.tsv file)
2012/13
62
Set up data columns
...and “Save” 2012/13
63
Click on the table name in the left hand panel, and the details of it in the main panel (no data, yet)
2012/13
64
Importing data into MySQL ►
In phpMyAdmin, click on the ‘Import’ tab and choose your data file
MySQL
►
“data2.tsv” reformatted into CSV »
replacing all the tabs with commas
»
removing the header line
Import
Data2.tsv ►
In example files folder (under “data”)
2012/13
65
2012/13
66
2012/13
67
Querying the database 1) Extract the information we want 2) In a chosen format (json)
►
SQL Syntax »
2012/13
SELECT `date`, `close` FROM `data2`
68
Querying the database (php – MySQL – json - D3) ►
php script will... 1) run the query 2) extract data from the database 3) format it into JSON 4) make available to D3
D3.js
2012/13
php
MySQL
JSON
70
data2.php
2012/13
72
data2.php
►
data2.php - a separate file
►
Put in folder called php in web root directory (alongside the data directory)
2012/13
73
Getting the data into d3.js ►
Replaced the part of the code that read in the data file as data.tsv with the equivalent that reads the php/data2.php file in as json
2012/13
74
Querying the database (php – MySQL – json - D3)
D3.js
2012/13
php
MySQL
JSON
75
Database driven D3 graph And here is the result! MySQL
2012/13
76
Additional... ►
Encodings, e.g.
2012/13
»
Add Axis Labels
»
Add a title
»
Smoothing graph lines
»
Add grid lines
»
etc.
77
Data Visualization Lecture 09 (Final) Colman McMahon
[email protected]
2012/13
78
