We have prepared English translation of the original document with comments.
As of today (Mar 31st), Clover Engine 2.7 branch has been created and the testing/QA process has started. Within approx 2 weeks, brand new version of CloverETL will be ready. It brings many small new features and bug fixes, but also several significant improvements – mostly in speed.
The aging ExtSort component is being replaced by new FastSort, which can bring up to 2.5 times the performance of old ExtSort. I am sure, there will be special post on this blog by FastSort’s developer Pavel Najvar, who will explan in detail where he found those hidden 250% of speed.
There are also speed improvements in our Universal Data Reader (reader of text data, delimited or fixed). We thoroughly profiled its code and were able to find 20-25% of additional speed. This puts us even farher in front of competition !
Can ETL be used for something useful – aside extract, transform & load ?
I played with an idea that common applications, which include data entry & data processing, can be split into simple, web based data entry application (optionally also data presentation) and business logic implemented as an ETL process. With CloverETL, the transformation piece can be also called as a WEB application – thank to Clover’s “launch services” which seamlessly convert any transformation into WEB service where data are passed-in and sent-out through HTTP protocol (POST request).
The whole idea is based on fact, that data processing is easily (visually) designed & debugged using ETL tool. Unfortunately, the data entry is usually out of scope for most ETL tools (are there any which support it ?) therefore WEB based “glue” with data entry forms is the easiest way.
Other option is to use the modern curse of IT – Excel. Excel is popular data entry & data presentation tool – why not to exploit it ?
I wholeheartedly believe that using this approach, development time can be cut to half as opposed to standard way of coding it in PHP or as a J2EE application. Also further modifications can be done more quickly. Visual programming is here, finally !
PS: I am going to bid for a project using this approach. So if I convince the customer, in two months I should know what this idea stands for….