ETL Benchmarks .... Test 4: File Input Delimited > Table Output Oracle BULK . ....
This document constitutes Version 1.1 of the ETL Benchmark, as version 1.0 ...
Pg 1
MANAPPS
ETL Benchmarks V 1.1 Comparing DATASTAGE SERVER 7.5 DATASTAGE PX 7.5 TALEND OPEN STUDIO 2.4.1 INFORMATICA 8.1.1 PENTAHO DATA INTEGRATOR 3.0.0
[email protected] V 1.1 2008/10/20
ETL Benchmarks
Pg 2
MANAPPS
This document is published under the Creative Commons license: http://creativecommons.org/licenses/by/3.0/us/
You are free: to Share — to copy, distribute, display, and perform the work
to Remix — to make derivative works
Under the following conditions:
Attribution. You must attribute the work in the manner specified by the author or licensor (but not in any way that suggests that they endorse you or your use of the work). For any reuse or distribution, you must make clear to others the license terms of this work. The best way to do this is with a link to this web page. Any of the above conditions can be waived if you get permission from the copyright holder. Apart from the remix rights granted under this license, nothing in this license impairs or restricts the author's moral rights.
V 1.1 2008/10/20
ETL Benchmarks
Pg 3
MANAPPS
Table of Contents You are free: .................................................................................................................................... 2 Under the following conditions:...................................................................................................... 2 Table of Contents .................................................................................................................................... 3 General comments .................................................................................................................................. 5 Hardware Configuration.......................................................................................................................... 6 Test 1: File Input Delimited > File Output Delimited............................................................................... 8 Scenario:.............................................................................................................................................. 8 Test results: ....................................................................................................................................... 14 Test 2: File Input Delimited > Table MySQL Output.............................................................................. 15 Scenario:............................................................................................................................................ 15 Test results: ....................................................................................................................................... 18 Test 3: Table Oracle Input > File Output Delimited............................................................................... 18 Scenario:............................................................................................................................................ 18 Test results: ....................................................................................................................................... 25 Test 4: File Input Delimited > Table Output Oracle BULK ..................................................................... 26 Scenario:............................................................................................................................................ 26 Test results: ....................................................................................................................................... 32 Test 5: File Input Delimited > Transform > File Output Delimited ........................................................ 33 Scenario:............................................................................................................................................ 33 Tests result: ....................................................................................................................................... 45 Test 6: Table Input Oracle > Aggregation > Table Output Oracle (ELT) ................................................ 46 Scenario:............................................................................................................................................ 46 Test results: ....................................................................................................................................... 52 Test 7: Tables Input Oracle > Transformation > Tables Output Oracle (ELT)........................................ 53 Scenario:............................................................................................................................................ 53 Test results: ....................................................................................................................................... 59 Test 8: File Input Delimited > Sort > File Output Delimited .................................................................. 60
V 1.1 2008/10/20
ETL Benchmarks
Pg 4
MANAPPS
Scenario:............................................................................................................................................ 60 Tests result: ....................................................................................................................................... 66 Test 9: File Input Delimited > Aggregate > File Output Delimited ........................................................ 70 Scenario:............................................................................................................................................ 70 Tests result: ....................................................................................................................................... 77 Test 10: File Input Delimited > Lookup > File Output Delimited ........................................................... 80 Scenario:............................................................................................................................................ 80 Tests result: ....................................................................................................................................... 92 Test 11: File Input Delimited > Lookup > File Output Delimited && rejects ......................................... 96 Scenario:............................................................................................................................................ 96 Tests result: ..................................................................................................................................... 109
V 1.1 2008/10/20
ETL Benchmarks
Pg 5
MANAPPS
General comments This document constitutes Version 1.1 of the ETL Benchmark, as version 1.0 showed inaccurate tests results for the PowerCenter solution powered by Informatica, as our tests were carried out with inadequate settings for this product. An expert from Informatica suggested adapted settings, and the same tests were run again on the same environment, in order to preserve the benchmarking basis between all compared ETL tools. Use of this settings on the Informatica PowerCenter solution greatly improve the results obtained by this solution on the same ETL benchmark tests, as detailed in this corrected version of our benchmark. This Version 1.1 of the benchmark thus includes the updated results and comparison between all tested tools, and Annexe1 details the changes in the use of the Informatica software.
We are open to comments from all tested editors, but also to other publishers, and are ready to give access to our testing conditions in order to allow them to verify the results obtained by their products and to suggest applicable best practices. For the tests with DataStage PX, we used 2 nodes to take advantage of the dual cores and of the parallelization feature of the tool.
Results: Even if it is difficult to give results for this kind of benchmark, and we think that each test is different, some people ask us to give a global synthesis of those tests.
Global performance: As requested by some people after the issue of version 1.0 of this ETL Benchmark, we have assigned, for each test, a specific number of points to the tested solutions (5 points to the best, 4 to the second…1 to the fifth). According to this scenario, results are as follows: o First: Informatica 8.1.1 (353 points) o Second: Talend Open Studio 2.4.1 (333 points) o Third: IBM Datastage PX 7.5 (239 points)
V 1.1 2008/10/20
ETL Benchmarks
Pg 6
MANAPPS o Fourth: IBM Dataserver 7.5 (199 points) o Fifth: Pentaho Data Integration 3.0.0 (148 points) Below are the detailed results: TOS 2.4.1 Test1 Test2 Test3 Test4 Test5 Test6 Test7 Test8 Test8.2 Test8.3 Test9 Test9.2 Test9.3 Test10 Test10.2 Test10.3 Test10.4 Test11 Test11.2 Test11.3 Test12 Test12.2 Test12.3
Total
13 0 13 8 15 15 11 13 12 12 12 16 12 20 20 16 12 20 20 16 20 20 17
PDI 3.0.0 7 0 3 7 4 4 3 12 13 12 6 5 8 7 6 6 4 7 6 6 8 7 7
IBM DS 7.5 19 0 7 12 13 10 7 5 4 4 15 12 13 12 6 6 8 10 6 6 13 6 5
IBM DS PX 7.5 8 0 9 5 12 5 8 14 15 15 12 9 11 10 13 14 17 8 12 13 6 11 12
INFA PWC 8.1.1 16 0 11 13 18 12 15 16 18 17 17 19 16 13 16 18 19 16 16 19 13 16 19
333
148
199
239
353
In terms of intuitiveness and ease of use, Talend Open Studio and DataStage Server are ahead of the pack. DataStage PX comes in the third position, Informatica in fourth and the least intuitive is Pentaho Data Integrator. Our main reason for this assessment of Pentaho is mostly linked to the many parameters that need to be learnt. However, we think that if you invest lots of time in it, it could become an powerful tool.
Open Source ETL & Parallelization: Pentaho Data Integrator claims the first position here. It is easier to parallelize with PDI. We did however fine some issues with the way the tool lets you to parallelize all the components, but some results are inconsistent.
Hardware Configuration
V 1.1 2008/10/20
ETL Benchmarks
Pg 7
MANAPPS
OS: Windows XP Pro SP2 CPU: Intel Core2 Duo 2 GHz JVM 1.6.0_87 RAM: 4 Go
V 1.1 2008/10/20
ETL Benchmarks
Pg 8
MANAPPS
Test 1: File Input Delimited > File Output Delimited
Scenario: Reading X lines from a file input delimited and writing in a file output delimited. File input delimited extract:
V 1.1 2008/10/20
ETL Benchmarks
Pg 9
MANAPPS TALEND OPEN STUDIO Job name: file_input_delimited__file_output_delimited
Job
Schema of file_input_delimited
V 1.1 2008/10/20
ETL Benchmarks
Pg 10
MANAPPS PENTAHO DATA INTEGRATION Job name: file_input_delimited__file_output_delimited
Job
Schema of file_input_delimited
V 1.1 2008/10/20
ETL Benchmarks
Pg 11
MANAPPS DATASTAGE SERVER Job name: file_input_delimited__file_output_delimited
Job
Schema of file_input_delimited
V 1.1 2008/10/20
ETL Benchmarks
Pg 12
MANAPPS DATASTAGE PX Job name: PX_file_input_delimited__file_output_delimited
Job
Schema of file_input_delimited
V 1.1 2008/10/20
ETL Benchmarks
Pg 13
MANAPPS INFORMATICA Job name: file_input_delimited__file_output_delimited
Job
Schema of file_input_delimited
V 1.1 2008/10/20
ETL Benchmarks
Pg 14
MANAPPS
Test results:
Test 1: File Input Delimited > File Output Delimited
Lines TOS 2.4.1 PDI 3.0.0 IBM DS 7.5 IBM DS PX 7.5 INFA PWC 8.1.1
Statistics: Number of lines
TOS 2.4.1
100 000 1,00 2,00 2,00 3,40 2,00
1 000 000 5 000 000 7,80 39,10 15,50 83,80 4,00 12,50 12,00 40,00 7,00 18,00
PDI 3.0.0 DataStage 7.5 DataStage PX 7.5
ratio compared with TOS 2.4.1
100 000 1 000 000
20 000 000 162,09 417,80 66,00 150,00 74,00
2 1,99
2 0,51
3,4 1,54
Informatica 8.1.1
2 0,9
V 1.1 2008/10/20
ETL Benchmarks
Pg 15
MANAPPS 5 000 000 20 000 000
2,14 2,58
0,32 0,41
1,02 0,93
0,46 0,47
Test 2: File Input Delimited > Table MySQL Output
Scenario: Reading X lines from a file input delimited and writing into a table output MySQL. Comments: DataStage 7.5, DataStage PX 7.5 and Informatica 8.1.1 are not tested for this use case. To begin, the test has been done with default parameters. To optimize the performances, the commit parameter has been learned. To finish, the job has been parallelize. To parallelize with TOS 2.4.1, we just have to cut through our file input delimited (With the header and the limit parameters) and parallelize two sub‐jobs. With PDI 3.0.0, we just have to increment the number of copy. TOS 2.4.1 permits to use the extended insert, which is a MySQL feature. This feature limits the number of database accesses and increases the performances. With this feature, TOS 2.4.1 is 6 times faster.
V 1.1 2008/10/20
ETL Benchmarks
Pg 16
MANAPPS TALEND OPEN STUDIO Job name: file_input_delimited__table_output_mysql
Job (Multi‐Thread Execution checked on Job Settings)
Schema of file_input_delimited
V 1.1 2008/10/20
ETL Benchmarks
Pg 17
MANAPPS PENTAHO DATA INTEGRATION Job name: file_input_delimited__table_output_mysql
Job
Schema of file_input_delimited
V 1.1 2008/10/20
ETL Benchmarks
Pg 18
MANAPPS
Test results:
Test 2: File Input Delimited > Table MySQL Output
Lines TOS 2.4.1 PDI 3.0.0 TOS 2.4.1 with Extended Insert
100 000 15,26 14,90 2,60
1 000 000 144,50 151,80 25,00
5 000 000 731,78 843,90 129,00
Statistics: Number of lines
TOS 2.4.1
PDI 3.0.0
TOS 2.4.1 Extended Insert
ratio compared with TOS 2.4.1
100 000 1 000 000 5 000 000
0,98 1,05 1,15
0,18 0,17 0,18
Test 3: Table Oracle Input > File Output Delimited
Scenario:
V 1.1 2008/10/20
ETL Benchmarks
Pg 19
MANAPPS Reading X lines from a table output Oracle and writing into a file output delimited.
V 1.1 2008/10/20
ETL Benchmarks
Pg 20
MANAPPS TALEND OPEN STUDIO Job name: table_input_oracle__file_output_delimited
Job
Schema of table_input_oracle
V 1.1 2008/10/20
ETL Benchmarks
Pg 21
MANAPPS PENTAHO DATA INTEGRATION Job name: table_input_oracle__file_output_delimited
Job SCHEMA VIEWER NOT POSSIBLE Schema of table_input_oracle
V 1.1 2008/10/20
ETL Benchmarks
Pg 22
MANAPPS DATASTAGE SERVER Job name: table_input_oracle__file_output_delimited
Job
Schema of table_input_oracle
V 1.1 2008/10/20
ETL Benchmarks
Pg 23
MANAPPS DATASTAGE PX Job name: PX_table_input_oracle__file_output_delimited
Job
Schema of table_input_oracle
V 1.1 2008/10/20
ETL Benchmarks
Pg 24
MANAPPS INFORMATICA Job name: table_input_oracle__file_output_delimited
Job
Schema of table_input_oracle
V 1.1 2008/10/20
ETL Benchmarks
Pg 25
MANAPPS
Test results:
Test 3: Table Oracle Input > File Output Delimited
Lines TOS 2.4.1 PDI 3.0.0 IBM DS 7.5 IBM DS PX 7.5 INFA PWC 8.1.1
Statistics: Number of lines
TOS 2.4.1
100 000 2,25 4,78 4,00 4,00 5
500 000 6,26 21,20 11,00 8,00 6
1 000 000 14,25 37,40 19,00 15,00 9
PDI 3.0.0 DataStage 7.5 DataStage PX 7.5
ratio compared with TOS 2.4.1
100 000 500 000 1 000 000
2,12 3,39 2,62
1,78 1,76 1,33
1,78 1,28 1,05
Informatica 8.1.1
2 0,95 0,63
V 1.1 2008/10/20
ETL Benchmarks
Pg 26
MANAPPS
Test 4: File Input Delimited > Table Output Oracle BULK
Scenario: Reading X lines from a file input delimited and writing into a table output Oracle BULK.
V 1.1 2008/10/20
ETL Benchmarks
Pg 27
MANAPPS TALEND OPEN STUDIO Job name: file_input_delimited__table_output_oracle_bulk
Job
V 1.1 2008/10/20
ETL Benchmarks
Pg 28
MANAPPS PENTAHO DATA INTEGRATION Job name: file_input_delimited__table_output_oracle_bulk
Job
Schema of file_input_delimited
V 1.1 2008/10/20
ETL Benchmarks
Pg 29
MANAPPS DATASTAGE SERVER Job name: file_input_delimited__table_output_oracle_bulk
Job
Schema of file_input_delimited
V 1.1 2008/10/20
ETL Benchmarks
Pg 30
MANAPPS DATASTAGE PX Job name: PX_file_input_delimited__table_output_oracle_bulk
Job
Schema of file_input_delimited
V 1.1 2008/10/20
ETL Benchmarks
Pg 31
MANAPPS INFORMATICA Job name: file_input_delimited__table_output_oracle_bulk
Job
Schema of file_input_delimited
V 1.1 2008/10/20
ETL Benchmarks
Pg 32
MANAPPS
Test results:
Test 4: File Input Delimited > Table Output Oracle BULK
Lines TOS 2.4.1 PDI 3.0.0 IBM DS 7.5 IBM DS PX 7.5 INFA PWC 8.1.1
Statistics: Number of lines
TOS 2.4.1
100 000 4,36 2,60 3,00 6,00 4
1 000 000 22,12 30,60 18,00 27,00 7
2 000 000 49,66 72,70 40,00 55,00 11
PDI 3.0.0 DataStage 7.5 DataStage PX 7.5
ratio compared with TOS 2.4.1
100 000 1 000 000 2 000 000
0,6 1,38 1,46
0,69 0,81 0,8
1,38 1,22 1,11
Informatica 8.1.1
0,92 0,31 0,22
V 1.1 2008/10/20
ETL Benchmarks
Pg 33
MANAPPS
Test 5: File Input Delimited > Transform > File Output Delimited
Scenario: Reading X lines from a file input delimited and writing in a file output delimited after some changes. Changes list: • The field `rate` content is multiplied by 100. • The new field `name` is a concatenation (`firstname`+ « » +`lastname`). • The fields `address` content is converted to uppercase. Comments: Pentaho Data Integration hasn’t any graphic component to transform data. Thus, we have to use a custom code component. The used language is JavaScript. The four others ETL got a transformer to do this. Talend Open Studio got a custom code too, named tJavaRow or tPerlRow.
V 1.1 2008/10/20
ETL Benchmarks
Pg 34
MANAPPS TALEND OPEN STUDIO Job name: file_input_delimited__transformation__file_output_delimited
Job
Schema of file_input_delimited
Schema of file_output_delimited
V 1.1 2008/10/20
ETL Benchmarks
Pg 35
MANAPPS
tMap
V 1.1 2008/10/20
ETL Benchmarks
Pg 36
MANAPPS PENTAHO DATA INTEGRATION Job name: file_input_delimited__transformation__file_output_delimited
Job
Schema of file_input_delimited
Schema of file_output_delimited
V 1.1 2008/10/20
ETL Benchmarks
Pg 37
MANAPPS
JavaScript Custom Code
Select Values
Select Values
V 1.1 2008/10/20
ETL Benchmarks
Pg 38
MANAPPS DATASTAGE SERVER Job name: file_input_delimited__transformation__file_output_delimited
Job
Schema of file_input_delimited
Schema of file_output_delimited
V 1.1 2008/10/20
ETL Benchmarks
Pg 39
MANAPPS
Transformer
V 1.1 2008/10/20
ETL Benchmarks
Pg 40
MANAPPS DATASTAGE PX Job name: PX_file_input_delimited__transformation__file_output_delimited
Job
Schema of file_input_delimited
Schema of file_output_delimited
V 1.1 2008/10/20
ETL Benchmarks
Pg 41
MANAPPS
Transformer
V 1.1 2008/10/20
ETL Benchmarks
Pg 42
MANAPPS INFORMATICA Job name: file_input_delimited__transformation__file_output_delimited
Job
Schema of file_input_delimited
V 1.1 2008/10/20
ETL Benchmarks
Pg 43
MANAPPS
Schema of file_output_delimited
V 1.1 2008/10/20
ETL Benchmarks
Pg 44
MANAPPS
Mapping
V 1.1 2008/10/20
ETL Benchmarks
Pg 45
MANAPPS
Tests result:
Test 5: File Input Delimited > Transform > File Output Delimited
Lines TOS 2.4.1 PDI 3.0.0 IBM DS 7.5 IBM DS PX 7.5 INFA PWC 8.1.1
Statistics: Number of lines
TOS 2.4.1
100 000 1,30 5,30 2,00 4,75 3,00
1 000 000 8,50 51,00 10,00 11,33 6,00
5 000 000 43,10 259,40 56,00 41,00 17,00
PDI 3.0.0 DataStage 7.5 DataStage PX 7.5
ratio compared with TOS 2.4.1
100 000 1 000 000 5 000 000 20 000 000
20 000 000 183,13 1126,10 178,00 155,00 74,00
4,07 6 6,02 6,16
1,54 1,18 1,3 0,97
3,65 1,33 0,95 0,84
Informatica 8.1.1
2,3 0,7 0,39 0,4
V 1.1 2008/10/20
ETL Benchmarks
Pg 46
MANAPPS
Test 6: Table Input Oracle > Aggregation > Table Output Oracle (ELT)
Scenario: Reading X lines from tables input Oracle and writing into another tables output Oracle (ELT Mod). Comments: Only Talend Open Studio permits to use an ELT mod. Informatica got the Push Down Optimization, but I didn’t find this feature on the tool.
V 1.1 2008/10/20
ETL Benchmarks
Pg 47
MANAPPS
TALEND OPEN STUDIO Job names: ELT__table_input_oracle__aggregate_group_by_age_count__table_output_oracle
Job (ELT)
Schema of table_input_oracle
V 1.1 2008/10/20
ETL Benchmarks
Pg 48
MANAPPS PENTAHO DATA INTEGRATION Job name: table_input_oracle__aggregate_group_by_age_count__table_output_oracle
Job SCHEMA VIEWER NOT POSSIBLE Schema of table_input_oracle
V 1.1 2008/10/20
ETL Benchmarks
Pg 49
MANAPPS DATASTAGE SERVER Job name: table_input_oracle__aggregate_group_by_age_count__table_output_oracle
Job
Schema of table_input_oracle
V 1.1 2008/10/20
ETL Benchmarks
Pg 50
MANAPPS
DATASTAGE PX Job name: PX_table_input_oracle__aggregate_group_by_age_count__table_output_oracle
Job
Schema of table_input_oracle
V 1.1 2008/10/20
ETL Benchmarks
Pg 51
MANAPPS INFORMATICA Job name: table_input_oracle__aggregate_group_by_age_count__table_output_oracle
Job
Schema of table_input_oracle
V 1.1 2008/10/20
ETL Benchmarks
Pg 52
MANAPPS
Test results:
Test 6: Table Input Oracle > Aggregation > Table Output Oracle (ELT)
Lines TOS 2.4.1 PDI 3.0.0 IBM DS 7.5 IBM DS PX 7.5 INFA PWC 8.1.1
Statistics: Number of lines
TOS 2.4.1
100 000 1,24 4,26 2,40 8,00 4
500 000 1,4 22,26 8,00 12,00 3
1 000 000 1,69 47,80 13,67 17,50 4
PDI 3.0.0 DataStage 7.5 DataStage PX 7.5
ratio compared with TOS 2.4.1
100 000 500 000 1 000 000
3,44 15,9 28,28
1,94 5,71 8,09
6,45 8,57 10,36
Informatica 8.1.1
3,22 2,14 2,36
V 1.1 2008/10/20
ETL Benchmarks
Pg 53
MANAPPS
Test 7: Tables Input Oracle > Transformation > Tables Output Oracle (ELT)
Scenario: Reading X lines from tables input Oracle and writing into another tables output Oracle (ELT Mod) after some changes.
V 1.1 2008/10/20
ETL Benchmarks
Pg 54
MANAPPS TALEND OPEN STUDIO Job name: table_input_oracle__elt__table_output_oracle
Job (ELT)
Schema of table_lookup_oracle
Schema of table_input_oracle
V 1.1 2008/10/20
ETL Benchmarks
Pg 55
MANAPPS PENTAHO DATA INTEGRATION Job name: table_input_oracle__elt__table_output_oracle
Job SCHEMA VIEWER NOT POSSIBLE Schema of table_lookup_oracle SCHEMA VIEWER NOT POSSIBLE Schema of table_input_oracle
V 1.1 2008/10/20
ETL Benchmarks
Pg 56
MANAPPS DATASTAGE SERVER Job name: table_input_oracle__elt__table_output_oracle
Job
Schema of table_lookup_oracle
Schema of table_input_oracle
V 1.1 2008/10/20
ETL Benchmarks
Pg 57
MANAPPS DATASTAGE PX Job name: PX_table_input_oracle__elt__table_output_oracle
Job
Schema of table_lookup_oracle
Schema of table_input_oracle
V 1.1 2008/10/20
ETL Benchmarks
Pg 58
MANAPPS INFORMATICA Job name: table_input_oracle__elt__table_output_oracle
Job
Schema of table_lookup_oracle
V 1.1 2008/10/20
ETL Benchmarks
Pg 59
MANAPPS Schema of table_input_oracle
Test results:
Test 7: Tables Input Oracle > Transformation > Tables Output Oracle (ELT)
Lines TOS 2.4.1 PDI 3.0.0 IBM DS 7.5 IBM DS PX 7.5 INFA PWC 8.1.1
Statistics: Number of lines
TOS 2.4.1
100 000 5,99 38,35 12,70 15,00 5
1 000 000 52,72 382,60 116,00 47,50 14
PDI 3.0.0 DataStage 7.5 DataStage PX 7.5
ratio compared with TOS 2.4.1
100 000 500 000 1 000 000
500 000 23,26 201,60 65,00 30,50 9
6,4 8,67 7,26
2,12 2,79 2,2
Informatica 8.1.1
2,5 1,31 0,9
0,83 0,39 0,27
V 1.1 2008/10/20
ETL Benchmarks
Pg 60
MANAPPS
Test 8: File Input Delimited > Sort > File Output Delimited
Scenario: Reading X lines from a file input delimited and writing in a file input delimited sorted. Sorts list: • • •
Order by the integer field `age` ASC. Order by the string field `firstname` ASC. Order by the fields `age` and `firstname` ASC.
Comments: With the version used, I can’t do sort in memory with Pentaho Data Integrator. But the feature is present on latest version. On Talend Open Studio, with a large volume (5 000 000 and 20 000 000), we have to use the component tExternalSort which use GNU sort, a sort software.
V 1.1 2008/10/20
ETL Benchmarks
Pg 61
MANAPPS TALEND OPEN STUDIO Job names: • file_input_delimited__sort_on_age__file_output_delimited • file_input_delimited__sort_on_firstname__file_output_delimited • file_input_delimited__sort_on_firstname_and_age__file_output_delimited
Job
Schema of file_input_delimited
V 1.1 2008/10/20
ETL Benchmarks
Pg 62
MANAPPS PENTAHO DATA INTEGRATION Job names: • file_input_delimited__sort_on_age__file_output_delimited • file_input_delimited__sort_on_firstname__file_output_delimited • file_input_delimited__sort_on_firstname_and_age__file_output_delimited
Job
Schema of file_input_delimited
V 1.1 2008/10/20
ETL Benchmarks
Pg 63
MANAPPS DATASTAGE SERVER Job names: • file_input_delimited__sort_on_age__file_output_delimited • file_input_delimited__sort_on_firstname__file_output_delimited • file_input_delimited__sort_on_firstname_and_age__file_output_delimited
Job
Schema of file_input_delimited
V 1.1 2008/10/20
ETL Benchmarks
Pg 64
MANAPPS DATASTAGE PX
Job names: • PX_file_input_delimited__sort_on_age__file_output_delimited • PX_file_input_delimited__sort_on_firstname__file_output_delimited • PX_file_input_delimited__sort_on_firstname_and_age__file_output_delimited
Job
Schema of file_input_delimited
V 1.1 2008/10/20
ETL Benchmarks
Pg 65
MANAPPS INFORMATICA Job names: • file_input_delimited__sort_on_age__file_output_delimited • file_input_delimited__sort_on_firstname__file_output_delimited • file_input_delimited__sort_on_firstname_and_age__file_output_delimited
Job
Schema of file_input_delimited
V 1.1 2008/10/20
ETL Benchmarks
Pg 66
MANAPPS
Tests result:
Test 8: File Input Delimited > Sort > File Output Delimited Sorted by Age
Sorted by age Lines TOS 2.4.1 PDI 3.0.0 IBM DS 7.5 IBM DS PX 7.5 INFA PWC 8.1.1
Statistics: Number of lines
TOS 2.4.1
100 000 1,44 3,63 4,20 4,00 5,00
1 000 000 15,73 32,85 60,70 16,25 13,00
20 000 000 1016,03 668,20 492,67 201,00
PDI 3.0.0 DataStage 7.5 DataStage PX 7.5
ratio compared with TOS 2.4.1
100 000 1 000 000 5 000 000
5 000 000 188,21 155,95 267,70 64,50 50,00
2,51 2,09 0,83
2,92 3,86 1,42
Informatica 8.1.1
2,78 1,03 0,34
3,47 0,82 0,26
V 1.1 2008/10/20
ETL Benchmarks
Pg 67
MANAPPS 20 000 000
0,66
0,48
+++
0,2
Test 8: File Input Delimited > Sort > File Output Delimited Sort By First Name
Sorted by firstname Lines TOS 2.4.1 PDI 3.0.0 IBM DS 7.5 IBM DS PX 7.5 INFA PWC 8.1.1
Statistics: Number of lines
TOS 2.4.1
100 000 1,69 3,40 6,00 4,00 4,00
1 000 000 18,05 31,20 58,00 16,00 13,00
20 000 000 1071,20 739,20 624,00 223,00
PDI 3.0.0 DataStage 7.5 DataStage PX 7.5
ratio compared with TOS 2.4.1
100 000 1 000 000
5 000 000 168,46 157,15 426,00 57,00 51,00
2,01 1,73
3,55 3,21
Informatica 8.1.1
2,37 0,89
2,36 0,72
V 1.1 2008/10/20
ETL Benchmarks
Pg 68
MANAPPS 5 000 000 20 000 000
0,93 0,69
2,53 +++
0,34 0,58
0,3 0,21
Test 8: File Input Delimited > Sort > File Output Delimited Sort By First Age, Name
Sorted by age & firstname Lines TOS 2.4.1 PDI 3.0.0 IBM DS 7.5 IBM DS PX 7.5 INFA PWC 8.1.1
Statistics: Number of lines
TOS 2.4.1
100 000 1,33 3,22 7,33 4,50 5,00
1 000 000 17,40 29,27 60,00 16,33 13,00
PDI 3.0.0 DataStage 7.5 DataStage PX 7.5
ratio compared with TOS 2.4.1
100 000 1 000 000 5 000 000
5 000 000 225,03 159,10 360,00 59,00 49,00
2,42 1,68 0,71
5,51 3,45 1,6
20 000 000 1007,00 842,20 582,50 211,00
Informatica 8.1.1
3,38 0,94 0,26
3,75 0,74 0,22
V 1.1 2008/10/20
ETL Benchmarks
Pg 69
MANAPPS 20 000 000
0,84
+++
0,58
0,21
V 1.1 2008/10/20
ETL Benchmarks
Pg 70
MANAPPS
Test 9: File Input Delimited > Aggregate > File Output Delimited
Scenario: Reading X lines from a file input delimited, achieving an aggregation and writing the operations result in a file output delimited. 1 – Group by the field `age`; Operation: COUNT. 2 – Group by the field `age`; Operations: COUNT, SUM(rate), AVG(rate), MIN(rate), MAX(rate). 3 – Group by the field `firstname`; Operations: COUNT. Comments: When the output flow is too big (aggregate by firstname with big volume here), we have to use the tSortedAggregateRow on Talend Open Studio. This component sorts rows before the aggregation. On this case, Pentaho Data Integrator failed.
V 1.1 2008/10/20
ETL Benchmarks
Pg 71
MANAPPS TALEND OPEN STUDIO
Job names: • file_input_delimited__aggregate_group_by_age_count__file_output_delimited • file_input_delimited__aggregate_group_by_age_count_sum_avg_min_max__file_o utput_delimited • file_input_delimited__aggregate_group_by_firstname_count__file_output_delimit ed
Job
Job using the tExternalSortRow component
V 1.1 2008/10/20
ETL Benchmarks
Pg 72
MANAPPS
Schema of file_input_delimited
Schema of file_output_delimited file_input_delimited__aggregate_group_by_age_count__file_output_delimited
V 1.1 2008/10/20
ETL Benchmarks
Pg 73
MANAPPS PENTAHO DATA INTEGRATION
Job names: • file_input_delimited__aggregate_group_by_age_count__file_output_delimited • file_input_delimited__aggregate_group_by_age_count_sum_avg_min_max__file_o utput_delimited • file_input_delimited__aggregate_group_by_firstname_count__file_output_delimit ed
Job
Schema of file_input_delimited
Schema of file_output_delimited file_input_delimited__aggregate_group_by_age_count__file_output_delimited
V 1.1 2008/10/20
ETL Benchmarks
Pg 74
MANAPPS DATASTAGE SERVER
Job names: • file_input_delimited__aggregate_group_by_age_count__file_output_delimited • file_input_delimited__aggregate_group_by_age_count_sum_avg_min_max__file_o utput_delimited • file_input_delimited__aggregate_group_by_firstname_count__file_output_delimit ed
Job
Schema of file_input_delimited
Schema of file_output_delimited file_input_delimited__aggregate_group_by_age_count__file_output_delimited
V 1.1 2008/10/20
ETL Benchmarks
Pg 75
MANAPPS DATASTAGE PX
Job names: • PX_file_input_delimited__aggregate_group_by_age_count__file_output_delimited • PX_file_input_delimited__aggregate_group_by_age_count_sum_avg_min_max__fi le_output_delimited • PX_file_input_delimited__aggregate_group_by_firstname_count__file_output_deli mited
Job
Schema of file_input_delimited
Schema of file_output_delimited file_input_delimited__aggregate_group_by_age_count__file_output_delimited
V 1.1 2008/10/20
ETL Benchmarks
Pg 76
MANAPPS INFORMATICA
Job names: • file_input_delimited__aggregate_group_by_age_count__file_output_delimited • file_input_delimited__aggregate_group_by_age_count_sum_avg_min_max__file_o utput_delimited • file_input_delimited__aggregate_group_by_firstname_count__file_output_delimit ed
Job
Schema of file_input_delimited
Schema of file_output_delimited file_input_delimited__aggregate_group_by_age_count__file_output_delimited
V 1.1 2008/10/20
ETL Benchmarks
Pg 77
MANAPPS
Tests result:
Test 9: File Input Delimited > Aggregate > File Output Delimited Group by age (count)
Group by Age (Count) Lines TOS 2.4.1 PDI 3.0.0 IBM DS 7.5 IBM DS PX 7.5 INFA PWC 8.1.1
Statistics: Number of lines
TOS 2.4.1
100 000 0,62 2,70 2,00 4,00 3,00
1 000 000 6,99 26,53 6,00 6,50 5,00
20 000 000 124,16 466,50 128,00 78,00 27,00
PDI 3.0.0 DataStage 7.5 DataStage PX 7.5
ratio compared with TOS 2.4.1
100 000 1 000 000
5 000 000 30,05 134,30 21,00 21,33 8,00
4,35 3,8
3,23 0,86
Informatica 8.1.1
6,45 0,93
4,84 0,72
V 1.1 2008/10/20
ETL Benchmarks
Pg 78
MANAPPS 5 000 000 20 000 000
4,47 3,76
0,7 1,03
0,71 0,63
0,27 0,22
Test 9: File Input Delimited > Aggregate > File Output Delimited
Group by Age (Count, Sum(Rate), Avg(Rate), Min(Rate), Max(Rate))
Group by Age (Count, Sum(Rate), Avg(Rate), Min(Rate), Max(Rate)) Lines TOS 2.4.1 PDI 3.0.0 IBM DS 7.5 IBM DS PX 7.5 INFA PWC 8.1.1
Statistics: Number of lines
TOS 2.4.1
100 000 0,84 2,60 2,00 11,25 2,00
5 000 000 37,61 138,30 50,00 33,50 12,00
PDI 3.0.0 DataStage 7.5 DataStage PX 7.5
ratio compared with TOS 2.4.1
100 000 1 000 000 5 000 000
1 000 000 7,44 25,20 11,00 15,33 6,00
3,1 3,39 3,68
2,38 1,48 1,33
20 000 000 139,12 426,00 184,00 254,33 38,00
Informatica 8.1.1
13,39 2,06 0,89
2,38 0,8 0,31
V 1.1 2008/10/20
ETL Benchmarks
Pg 79
MANAPPS 20 000 000
3,06
1,32
1,91
0,27
Test 9: File Input Delimited > Aggregate > File Output Delimited Group by FirstName (Count)
Group by FirstName (Count) Lines TOS 2.4.1 PDI 3.0.0 IBM DS 7.5 IBM DS PX 7.5 INFA PWC 8.1.1
Statistics: Number of lines
TOS 2.4.1
100 000 0,86 2,70 2,00 4,50 4
1 000 000 7,89 29,70 14,00 11,00 9
PDI 3.0.0 DataStage 7.5 DataStage PX 7.5
ratio compared with TOS 2.4.1
100 000 1 000 000 5 000 000 20 000 000
5 000 000 198,79 162,30 68,00 40,00 23
3,14 3,76 0,82 0,59
2,33 1,77 0,34 0,46
20 000 000 928,08 544,00 424,00 505,00 85
Informatica 8.1.1
5,23 1,39 0,2 0,54
4,65 1,14 012 0,092
V 1.1 2008/10/20
ETL Benchmarks
Pg 80
MANAPPS
Test 10: File Input Delimited > Lookup > File Output Delimited
Scenario: Reading X lines from a file input delimited, looking up to another file input delimited, for 4 fields using id_client column. Writing the jointure result into a file output delimited.
V 1.1 2008/10/20
ETL Benchmarks
Pg 81
MANAPPS TALEND OPEN STUDIO Job name: file_input_delimited__file_lookup_delimited__file_output_delimited
Job
Schema of file_input_delimited
Schema of file_lookup_delimited
V 1.1 2008/10/20
ETL Benchmarks
Pg 82
MANAPPS
Schema file_output_delimited
tMap Component
V 1.1 2008/10/20
ETL Benchmarks
Pg 83
MANAPPS PENTAHO DATA INTEGRATION Job name: file_input_delimited__file_lookup_delimited__file_output_delimited
Job
Schema of file_input_delimited
Schema of file_lookup_delimited
V 1.1 2008/10/20
ETL Benchmarks
Pg 84
MANAPPS
Schema of file_output_delimited
Mapping Component
V 1.1 2008/10/20
ETL Benchmarks
Pg 85
MANAPPS DATASTAGE SERVER Job name: file_input_delimited__file_lookup_delimited__file_output_delimited
Job
Schema of file_input_delimited
V 1.1 2008/10/20
ETL Benchmarks
Pg 86
MANAPPS
Schema of file_lookup_delimited
Schema file_output_delimited
V 1.1 2008/10/20
ETL Benchmarks
Pg 87
MANAPPS
Transformer Component
V 1.1 2008/10/20
ETL Benchmarks
Pg 88
MANAPPS DATASTAGE PX Job name: PX_file_input_delimited__file_lookup_delimited__file_output_delimited
Job
Schema of file_input_delimited
V 1.1 2008/10/20
ETL Benchmarks
Pg 89
MANAPPS
Schema of file_lookup_delimited
Schema file_output_delimited
Transformer Component
V 1.1 2008/10/20
ETL Benchmarks
Pg 90
MANAPPS INFORMATICA Job name: file_input_delimited__file_lookup_delimited__file_output_delimited
Job
Schema of file_input_delimited
Schema of file_lookup_delimited
V 1.1 2008/10/20
ETL Benchmarks
Pg 91
MANAPPS
Schema file_output_delimited
Transformer Component
V 1.1 2008/10/20
ETL Benchmarks
Pg 92
MANAPPS
Tests result:
Test 10: File Input Delimited > Lookup > File Output Delimited Lookup 100 000 rows ~7MB
Lookup 100 000 rows ~7MB Lines TOS 2.4.1 PDI 3.0.0 IBM DS 7.5 IBM DS PX 7.5 INFA PWC 8.1.1
Statistics: Number of lines
TOS 2.4.1
100 000 1,45 4,14 5,00 5,00 5,00
1 000 000 6,39 21,40 10,60 12,20 11,00
PDI 3.0.0 DataStage 7.5 DataStage PX 7.5
ratio compared with TOS 2.4.1
100 000 1 000 000 5 000 000
5 000 000 28,72 87,60 33,00 40,00 32,00
2,86 3,35 3,05
3,45 1,66 1,15
20 000 000 108,37 288,90 139,00 122,00 116,00
Informatica 8.1.1
3,45 1,91 1,39
3,44 1,72 1,11
V 1.1 2008/10/20
ETL Benchmarks
Pg 93
MANAPPS 20 000 000
2,67
1,28
1,13
1,07
Test 10: File Input Delimited > Lookup > File Output Delimited Lookup 500 000 rows ~34MB
Lookup 500 000 rows ~34MB Lines TOS 2.4.1 PDI 3.0.0 IBM DS 7.5 IBM DS PX 7.5 INFA PWC 8.1.1
Statistics: Number of lines
TOS 2.4.1
100 000 3,9 7,90 28,00 7,00 4,00
1 000 000 8,89 24,50 33,00 13,00 11,00
PDI 3.0.0 DataStage 7.5 DataStage PX 7.5
ratio compared with TOS 2.4.1
100 000 1 000 000 5 000 000 20 000 000
5 000 000 32,36 97,40 56,00 40,00 33,00
2,03 2,76 3,01 2,52
7,18 3,71 1,73 1,69
20 000 000 115,67 291,10 195,00 122,00 122,00
Informatica 8.1.1
1,79 1,46 1,24 1,05
1,03 1,24 1,02 1,05
V 1.1 2008/10/20
ETL Benchmarks
Pg 94
MANAPPS
Test 10: File Input Delimited > Lookup > File Output Delimited Lookup 1 000 000 rows ~68MB
Lookup 1 000 000 rows ~68MB Lines 100 000 1 000 000 TOS 2.4.1 9,86 14,26 PDI 3.0.0 14,50 32,20 IBM DS 7.5 68,30 80,00 IBM DS PX 7.5 9,25 15,00 INFA PWC 8.1.1 5,00 12,00
Statistics: Number of lines
TOS 2.4.1
20 000 000 121,44 487,25 203,00 123,00 142,00
PDI 3.0.0 DataStage 7.5 DataStage PX 7.5
ratio compared with TOS 2.4.1
100 000 1 000 000 5 000 000 20 000 000
5 000 000 38,6 116,60 102,00 40,00 35,00
1,47 2,26 3,02 4,01
6,93 5,61 2,64 1,67
Informatica 8.1.1
0,94 1,05 1,04 1,01
0,51 0,84 0,91 1,16
V 1.1 2008/10/20
ETL Benchmarks
Pg 95
MANAPPS
Test 10: File Input Delimited > Lookup > File Output Delimited Lookup 5 000 000 rows ~365MB
Lookup 5 000 000 rows ~365MB Lines TOS 2.4.1 PDI 3.0.0 IBM DS 7.5 IBM DS PX 7.5 INFA PWC 8.1.1
Statistics: Number of lines
TOS 2.4.1
100 000 56,51
1 000 000 69,1
5 000 000 199,26
20 000 000 557,1
369,00 24,00 11,00
407,00 30,00 14,00
496,00 55,00 42,00
973,00 134,00 141,00
PDI 3.0.0 DataStage 7.5 DataStage PX 7.5
ratio compared with TOS 2.4.1
100 000 1 000 000 5 000 000 20 000 000
Failed Failed Failed Failed
6,53 5,89 2,49 1,75
Informatica 8.1.1
0,42 0,43 0,28 0,24
0,19 0,2 0,21 0,25
V 1.1 2008/10/20
ETL Benchmarks
Pg 96
MANAPPS
Test 11: File Input Delimited > Lookup > File Output Delimited && rejects
Scenario: Reading X lines from a file input delimited, looking up to another file input delimited, for 4 fields using id_client column. Writing the jointure result into a file output delimited and the output rejects into another files output delimited. 1 – Filter rejects: `age` content File Output Delimited && rejects Lookup 1 000 000 rows ~68MB + Filter 18 years
Lookup 1 000 000 rows ~68MB Lines 100 000 1 000 000 TOS 2.4.1 10,2 15,22 PDI 3.0.0 14,10 32,35 IBM DS 7.5 66,00 68,00 IBM DS PX 7.5 9,00 18,00 INFA PWC 8.1.1 6,00 14,00
Statistics: Number of lines
TOS 2.4.1
20 000 000 126,63 319,05 220,00 153,33 130,00
PDI 3.0.0 DataStage 7.5 DataStage PX 7.5
ratio compared with TOS 2.4.1
100 000 1 000 000 5 000 000 20 000 000
5 000 000 38,31 111,35 95,00 51,00 34,00
1,38 2,13 2,91 2,52
6,47 4,47 1,7 1,74
Informatica 8.1.1
0,88 1,18 1,33 1,21
0,59 0,92 0,89 1,03
V 1.1 2008/10/20
ETL Benchmarks
Pg 113
MANAPPS
TALEND OPEN STUDIO Job name: file_input_delimited__file_lookup_delimited__file_output_delimited__rejects_and_innerjoin_rejects _file_output_delimited
Job
Schema of file_input_delimited
V 1.1 2008/10/20
ETL Benchmarks
Pg 114
MANAPPS
Schema of file_lookup_delimited
Schema of file_output_delimited (age>=18) Schema of file_output_delimited (age_file_lookup_delimited > file_output_delimited__rejects && innerjoin_rejects_file_output_delimited Lookup 1 000 000 rows ~68MB
Lookup 1 000 000 rows ~68MB Lines 100 000 1 000 000 TOS 2.4.1 10,98 15,18 PDI 3.0.0 13,30 27,35 IBM DS 7.5 38,49 90,40 IBM DS PX 7.5 13,00 19,00 INFA PWC 8.1.1 6 13
Statistics: Number of lines
TOS 2.4.1
20 000 000 126,57 413,45 231,00 134,00 131
PDI 3.0.0 DataStage 7.5 DataStage PX 7.5
ratio compared with TOS 2.4.1
100 000 1 000 000 5 000 000 20 000 000
5 000 000 38,49 79,00 108,00 49,00 37
1,21 1,8 2,05 3,27
3,51 5,96 2,81 1,83
Informatica 8.1.1
1,18 1,25 1,27 1,06
0,55 0,86 0,96 1,04
V 1.1 2008/10/20
ETL Benchmarks
Pg 130
MANAPPS
Annex 1: Informatica settings and results This annex presents the settings changes made by Informatica and limitations they have found
Comments and amendment done on the basic PowerCenter 8.1.1 installation: *** Since the 'benchmark' machine is a tiny laptop with limited ressource (XP 32bit, Core2 Duo CPU and 3,43 GB of RAM) we've done following change: ‐ Auto‐Memory deactivation: MaxMem at 0 in the Default Session Config ‐ High Availability storage deactivation: EnableHAStorage at No for the 'Integration Service ‐ Metadata Manager and Reporting Service deactivation *** Configuration amendments : ‐ Unix environment variable INFA_DEFAULT_DOMAIN added ‐ Custom variable FileRdrTreatNullCharAs on the Integration Service added (NULL character are encountered in source data files) *** Standard Oracle 10g (10.1.0.2.0) Database installation with: sga_max_size=164MB pga_aggregate_target=115MB
Comments and "bestpractices" for the tests: Test 1: File Input Delimited > File Output Delimited - dynamic partitioning at 2 with more than 5 millions rows This is a Disk Bounded test Test 2: File Input Delimited > Table MySQL Output Not Applicable Test 3: Table Oracle Input > File Output Delimited - no partitioning as it's too small in volume and short in time Test 4: File Input Delimited > Table Output Oracle BULK
V 1.1 2008/10/20
ETL Benchmarks
Pg 131
MANAPPS - commit size at 100000 - dynamic partitioning at 2 with 2 millions rows This is a Disk Bounded test Test 5: File Input Delimited > Transform > File Output Delimited - function "CONCAT(CONCAT(firstname,' '),lastname)" is replaced by "firstname || ' ' || lastname" - dynamic partitioning at 2 with more than 5 millions rows This is a Disk Bounded test Test 6: Table Input Oracle > Aggregation > Table Output Oracle (ELT) - no partitioning as it's too small in volume and short in time Oracle database is not 'tuned' for ELT mode Test 7: Tables Input Oracle > Transformation > Tables Output Oracle (ELT) - commit size at 50000 - no partitioning as it's too small in volume and short in time Oracle database is not 'tuned' for ELT mode Test 8: File Input Delimited > Sort > File Output Delimited - sorter memory adjustment This is a memory limited test at 20 millions rows (2 pass sort are required) and also disk limited sometime Test 9: File Input Delimited > Aggregate > File Output Delimited - dynamic partitioning at 2 with more than 5 millions rows in source - aggregator memory adjustment This is a CPU bounded test Test 10: File Input Delimited > Lookup > File Output Delimited - dynamic partitioning at 2 with more than 5 millions rows in source or lookup - lookup memory adjustment - lookup in the flow with hash partitioning point This is a CPU bounded test Test 11: File Input Delimited > Lookup > File Output Delimited && rejects - use of router in place of filters - dynamic partitioning at 2 with more than 5 millions rows in source - lookup memory adjustment - lookup in the flow with hash partitioning point This is a CPU bounded test Test 12: file_input_delimited >_file_lookup_delimited > file_output_delimited__rejects && innerjoin_rejects_file_output_delimited - use of router in place of filters - dynamic partitioning at 2 with more than 5 millions rows in source - lookup memory adjustment - lookup in the flow with hash partitioning point This is a CPU bounded test
V 1.1 2008/10/20
ETL Benchmarks
