An Approach for Implementation of Cost Effective Automated Data Warehouse System

Abstract

Technology has resulted in humongous growth in generation of data and need of Data Warehouse. The performance of business decisions depends upon the optimality of the operations performed on data and supporting data warehouse system (DWH). Mostly, data warehouse system design methodologies focus on Extraction, Transformation and Loading (ETL) and ETL processes. Big Data paradigm is emerging as a big challenge to ETL and causing performance degradation and frequent reconfiguration to meet day to day operational tasks. Recent research claims on automations of ETL processes have emerged as possible solution in Big Data paradigm. Through this proposal, data warehouse system process is considered as a complete unified tool covering all intermediate steps from source data to report generation; for analysis and decision making. All steps such as, source data extraction automation, transformation & loading automation and reporting automation are attempted. Cost effective data warehouse system solution is proposed by avoiding use of commercial tools without compromising on performance.