An Approach for Implementation of Cost Effective Automated Data Warehouse System

Abstract

Technology has derived humongous growth in generation of the data and need of Data Warehouse. The performance of business decisions depends upon the optimality of the operations performed on data and design of its Data Ware House (DWH) system. Mostly, DWH system’s design methodologies focus on Extraction, Transformation and Loading (ETL) and related processes. Big Data paradigm is emerging as a big challenge to ETL and causing performance degradation and frequent re-configuration to meet day-to-day operational tasks. Recent research claims on automations of ETL processes have emerged as possible solution in Big Data paradigm. In this proposal, DWH system process is considered as a complete and unified tool which covers all intermediate steps from source data to report generation; for analysis and decision making. This work attempts the automation of source data extraction, transformation & loading and reporting. A cost effective DWH system solution is proposed by avoiding the use of commercial tools without compromising the performance.