CloverETL can be used not only in enterprise environment, but also in sport and entertainment industry. Prague hosted 10th FIMBA World Maxibasketball Championship in the first week of july. More than 160 teams from 31 countries took part in this popular event.
Match data (results and statistics) was transmitted in xls format, the most popular format for these purposes I guess. Although the excel files are user-friendly, for automatical processing are slightly inconvenient. CloverETL transformed xls data and stored them into a database to be used in more comfortable way from data engineer’s point of view.
Customers often tend to have obscure requirements. In a recent project we faced an interesting issue. Output records had to be split into unknown number of excel files according to their category. In addition, records within each file should have been written in datasheets according to their subcategory. The number of subcategories varied from 1 to 1024, so the whole solution seemed to me quite impractical.
Fortunately, we could solve the customer’s requirement very easily using CloverETL. For example, there are (among others) two fields – category and subcategory – in your metadata coming into XLSWriter. Then, if you set File URL in the form of filename_#.xls, Data sheet set as $subcategory and Partition key as category, the writer will split records into files according to the categories and into datasheets according to the subcategories.
Finally, the customer came to a conclusion that one file with many records is better than dozens datasheets within dozens files with very few records.
Settings of XLS_WRITER
