Comments on: ParallelReader versus competitors http://blog.cloveretl.com/2009/10/26/parallelreader-versus-competitors/ Life, the Universe, CloverETL and everything ... Tue, 19 Jan 2010 18:17:45 +0000 http://wordpress.com/ hourly 1 By: ParallelReader Versus Competitors Finish « CloverETL's Blog http://blog.cloveretl.com/2009/10/26/parallelreader-versus-competitors/#comment-54 ParallelReader Versus Competitors Finish « CloverETL's Blog Wed, 09 Dec 2009 15:09:21 +0000 http://blog.cloveretl.com/?p=223#comment-54 [...] transformations were run on my laptop with Windows Vista Home Premium. For detail information see part 1 and part [...] [...] transformations were run on my laptop with Windows Vista Home Premium. For detail information see part 1 and part [...]

]]> By: ParallelReader Versus Competitors Part 2 « CloverETL's Blog http://blog.cloveretl.com/2009/10/26/parallelreader-versus-competitors/#comment-47 ParallelReader Versus Competitors Part 2 « CloverETL's Blog Wed, 11 Nov 2009 16:08:42 +0000 http://blog.cloveretl.com/?p=223#comment-47 [...] pm Before we will release a complete comparison of open source ETL tools and after a success of my previous blog post I decided to publish the second transformation that we used in the [...] [...] pm Before we will release a complete comparison of open source ETL tools and after a success of my previous blog post I decided to publish the second transformation that we used in the [...]

]]> By: Matt Casters http://blog.cloveretl.com/2009/10/26/parallelreader-versus-competitors/#comment-40 Matt Casters Fri, 30 Oct 2009 10:56:09 +0000 http://blog.cloveretl.com/?p=223#comment-40 Hi Peter, You were right obviously, we don't have an in memory aggregation step in Pentaho Data Integration. The reason for that is quite simple if you think about it actually: we do our reporting and analyses in other parts of the Pentaho software stack. For performance driven (immediate) response times I would recommend Pentaho Analyses (Mondrian) in that regard. It's also possible to report directly on the data from Pentaho Data Integration too: http://michaeltarallo.blogspot.com/2009/07/pentaho-goes-to-movies-data-integration.html However, as with all benchmarks, there is this hidden implication that since one performance metric is bad, the whole tool must be bad. :-) So suppose we *would* have an in memory group by, what would the performance *then* be? Well, I created a new JIRA feature request in your honor this morning: http://jira.pentaho.com/browse/PDI-2804 And, since the code is pretty trivial compared to the functionality of the streaming version, I implemented it in 4.0 as well. It's unfortunately not possible to add the step to the stable 3.2 branches and I lack the time (and user requests) to create it as a plugin. As such, if you or a reader wants to play with it, get a recent (preview) build over here: http://ci.pentaho.com/job/Kettle/ The result is more in-line with yours for a CPU-bound process. Pentaho Data Integration did it in 45 seconds, but your system is a lot slower (1.66Ghz vs 2.33Ghz) http://imagebin.ca/img/oo-MhJ.jpg Get the transformation here : http://www.kettle.be/dloads/Pentaho2.ktr The transformation displayed is interesting in the sense that you can use data partitioning to parallelize the aggregation step as well. That gives you maybe another 5-10% gain. Maybe that's something you should try too. The run-time for the transformation is really a bit too short to draw any major conclusions. Maybe we should try with larger data sets to flush out other bottlenecks. :-) On a side note: it takes Kettle 100 seconds to bulk load the data into MySQL. Executing the query takes MySQL 43 seconds, even when you put an index on the group columns, so I don't think either tool is doing that bad of a job. Again, thanks for the puzzle/challenge. I certainly had my fun with it :-) Best of luck with CloverETL, it looks like you're doing a great job. Kind regards, Matt Hi Peter,

You were right obviously, we don’t have an in memory aggregation step in Pentaho Data Integration.
The reason for that is quite simple if you think about it actually: we do our reporting and analyses in other parts of the Pentaho software stack.
For performance driven (immediate) response times I would recommend Pentaho Analyses (Mondrian) in that regard.
It’s also possible to report directly on the data from Pentaho Data Integration too: http://michaeltarallo.blogspot.com/2009/07/pentaho-goes-to-movies-data-integration.html

However, as with all benchmarks, there is this hidden implication that since one performance metric is bad, the whole tool must be bad. :-)

So suppose we *would* have an in memory group by, what would the performance *then* be?
Well, I created a new JIRA feature request in your honor this morning: http://jira.pentaho.com/browse/PDI-2804
And, since the code is pretty trivial compared to the functionality of the streaming version, I implemented it in 4.0 as well.
It’s unfortunately not possible to add the step to the stable 3.2 branches and I lack the time (and user requests) to create it as a plugin.

As such, if you or a reader wants to play with it, get a recent (preview) build over here: http://ci.pentaho.com/job/Kettle/

The result is more in-line with yours for a CPU-bound process. Pentaho Data Integration did it in 45 seconds, but your system is a lot slower (1.66Ghz vs 2.33Ghz) http://imagebin.ca/img/oo-MhJ.jpg
Get the transformation here : http://www.kettle.be/dloads/Pentaho2.ktr

The transformation displayed is interesting in the sense that you can use data partitioning to parallelize the aggregation step as well. That gives you maybe another 5-10% gain. Maybe that’s something you should try too. The run-time for the transformation is really a bit too short to draw any major conclusions. Maybe we should try with larger data sets to flush out other bottlenecks. :-)

On a side note: it takes Kettle 100 seconds to bulk load the data into MySQL.
Executing the query takes MySQL 43 seconds, even when you put an index on the group columns, so I don’t think either tool is doing that bad of a job.

Again, thanks for the puzzle/challenge. I certainly had my fun with it :-)
Best of luck with CloverETL, it looks like you’re doing a great job.

Kind regards,
Matt

]]> By: Matt Casters http://blog.cloveretl.com/2009/10/26/parallelreader-versus-competitors/#comment-39 Matt Casters Thu, 29 Oct 2009 20:41:52 +0000 http://blog.cloveretl.com/?p=223#comment-39 Thanks for the puzzle Peter, I'll have a look tomorrow. Thanks for the puzzle Peter, I’ll have a look tomorrow.

]]> By: Petr Uher http://blog.cloveretl.com/2009/10/26/parallelreader-versus-competitors/#comment-38 Petr Uher Thu, 29 Oct 2009 20:39:07 +0000 http://blog.cloveretl.com/?p=223#comment-38 Alternative link for download added. http://www.filefactory.com/file/a0732fg/n/ParallelReaderComparison_zip Alternative link for download added.
http://www.filefactory.com/file/a0732fg/n/ParallelReaderComparison_zip

]]> By: Petr Uher http://blog.cloveretl.com/2009/10/26/parallelreader-versus-competitors/#comment-37 Petr Uher Thu, 29 Oct 2009 20:29:25 +0000 http://blog.cloveretl.com/?p=223#comment-37 Yes, it is but it isn't my fault. It's Pentaho's "feature". Pentaho "Group by" step requires sorted input, CloverETL's and Talend's aggregates don't. Yes, it is but it isn’t my fault. It’s Pentaho’s “feature”. Pentaho “Group by” step requires sorted input, CloverETL’s and Talend’s aggregates don’t.

]]> By: Matt Casters http://blog.cloveretl.com/2009/10/26/parallelreader-versus-competitors/#comment-36 Matt Casters Thu, 29 Oct 2009 20:21:10 +0000 http://blog.cloveretl.com/?p=223#comment-36 OK, since I know that the reading is fast enough, I can only guess by looking at the image that Kettle is being hurt by the sort BEFORE the aggregate. OK, since I know that the reading is fast enough, I can only guess by looking at the image that Kettle is being hurt by the sort BEFORE the aggregate.

]]> By: Petr Uher http://blog.cloveretl.com/2009/10/26/parallelreader-versus-competitors/#comment-35 Petr Uher Thu, 29 Oct 2009 20:12:08 +0000 http://blog.cloveretl.com/?p=223#comment-35 It looks like rapidshare is too busy. I'll reupload it to another server and add download link to the post. It looks like rapidshare is too busy. I’ll reupload it to another server and add download link to the post.

]]> By: Matt Casters http://blog.cloveretl.com/2009/10/26/parallelreader-versus-competitors/#comment-34 Matt Casters Thu, 29 Oct 2009 20:06:38 +0000 http://blog.cloveretl.com/?p=223#comment-34 By the way, I'm still trying to download the file without too much luck. I'll give it another try later. By the way, I’m still trying to download the file without too much luck. I’ll give it another try later.

]]> By: Matt Casters http://blog.cloveretl.com/2009/10/26/parallelreader-versus-competitors/#comment-33 Matt Casters Thu, 29 Oct 2009 20:05:23 +0000 http://blog.cloveretl.com/?p=223#comment-33 The file lineitems.tbl has an extra trailing column. (according to the CSV standard) Define a dummy value at the end and you're fine. The file lineitems.tbl has an extra trailing column. (according to the CSV standard) Define a dummy value at the end and you’re fine.

]]>