Data transformations relies on having quality, enriched data accessible to all at the right velocity and at the right time. Nilesh Deshmukh (CIO of PibyThree) talks about the challenges with data and the opportunities for successful data transformation
Q. Which challenges are most often encountered with Data?
A. Accessing, Cleansing, and aggregating data. It begins with data duplicity, data in multiple formats, data stored in multiple old legacy systems. Many organizations do not realize the potential of a good Data Catalog. On top of that there are multiple policies around data storage in different departments.
Q. And Data Provenance?
A. Data ownership and provenance comes into picture once organization has the vision to bring out insights from it and insights can only come from Quality Data.
Q. So, Data Quality should be addressed first?
A. Absolutely. Once basic segmentation is in place, organizations may consider evolving into more analytical and micro market-segmentation approaches as they gather better data, have enough entities in their inventory.
Q. So, ideally, how much time span should an organization look at for data transformation?
A. Timeline is different for every organization because every entity within any organization has different volume variety and velocity of data. Speed should not be the key motivation. Actionable insights should be the vision and all efforts should be towards that effect. Quality data and ease of Data sharing (data democratization) is the key.
Q. So where should organizations begin?
A. Legacy data systems and legacy infrastructure may not help expedite data transformation. Modernizing the infrastructure and revisiting data architectures is the key. Cloud or on-prem, revisiting architectures to help speedup transformation is key. This will expedite data cleansing/enrichment thereby paving way for munging of data. Once Quality data is accessible and read to be converted to actionable insights with the right infra, more monetization biz cases can be addressed. Having the right vision and the right people to deliver will help. And if cloud can be made part of the ecosystem, the better it is to scale up and scale out.
Q. Cloud-native, data-led architectures can help businesses?
A. Many cloud-native PaaS and SaaS services exist which make implementation easier, are easily scalable, provide the right data platform and reduce Capex burden.
Q. Final words
A. When the data doesn't reach the business in on-time or the correct data isn't delivered, that is when problems begin. There needs to be proper Data governance, proper data lineage and traceability. Unless that is achieved no data transformation will be successful even with a cloud-native architecture.
