SnapLogic research claims that 83 percent of organisations are not fully satisfied with the performance and output of their data management and data warehousing initiatives.
IT leaders cite a growing number of disconnected applications and data sources, outdated legacy systems, and slow and manual data movement as reasons for their frustration, all of which are stalling progress and costing them millions.
The study, You can read the research report, “The State of Data Management – Why Data Warehouses Fail” was conducted by independent research firm Vanson Bourne. It found that the average organisation has 115 distinct applications and data sources across their enterprise, but almost half of them are “siloed” and disconnected from one another. Respondents expressed clear concern, with 89 per cent of IT Decision Makers (ITDMs) from organisations where these apps and systems aren’t integrated worried that these data silos are holding them back. Indeed, ITDMs confirmed they are losing, on average, more than $1 million USD annually due to poor data management.
Still, organisations are committed to building a data-driven enterprise and are funding their data management initiatives accordingly. Over three quarters (76 percent) of survey respondents indicated that their companies have increased their data budgets over the past year.
ITDMs are investing in data storage solutions such as data warehouses and data lakes to help them make the most of their valuable data assets and deliver on the promise of agile analytics and actionable business insights. With this data, respondents hope to achieve faster and more informed decision making (70 percent), an improvement in business operations (64 percent), accelerated innovation and new business development (63 percent), an enhanced customer experience (61 percent), and more efficient and productive employees (57 percent). However, the process of getting data into a data warehouse or data lake is not always straightforward, all too often inhibiting progress and success.
Key research findings:
- Nearly nine out of ten ITDMs experience challenges trying to load data into data warehouses, with the biggest inhibitors being legacy technology (49 per cent), complex data types and formats (44 per cent), data silos (40 per cent), and data access issues tied to regulatory requirements (40 per cent).
- Nearly half of the data migrated into organisations’ data warehouses or other storage solutions requires cleaning before it can be useful. This likely contributes to the more than four hours of time wasted on average, per employee per week, resolving issues around missing data, duplicate data, or data that needs to be reformatted.
- ITDMs indicated that, on average, 42 per cent of their data management and warehousing processes are currently being done manually, but could be automated, which would save valuable time, resources, and money.
- As a result of these issues, almost all respondents (93 per cent) believe improvements are needed in how they collect, manage, and analyse data.
CTO at SnapLogic Craig Stewart said: “We are not only in a period of exponential data growth, but also one in which data lies at the heart of the successful adoption of technologies such as AI, automation, and advanced analytics. This means that having an effective data management and warehousing strategy is business-critical”.
Stewart said: “To get data warehousing right, organisations must break down data silos, retire legacy tools, automate manual processes, and accelerate the integration and movement of data across the enterprise. Only then can teams across the business truly harness the full power of data to drive better decisions, actions, and outcomes.”
