Rethinking Data Catalogs: The Promise and Pitfalls
The promise of data catalogs, a single source of truth for your organisation's data, often clashes with the reality of under-utilised features, redundancy across various data catalog solutions across teams, adoption challenges and a lack of clear strategy.
This talk will pose some critical questions concerning current approaches of choosing and implementing data catalogs:
- Do you actually need a Data catalog?
- Are data catalogs becoming just glorified registries without much practical use?
- Why do organisations find themselves juggling multiple catalogs?
- Is there a synergy between System Catalog & Data Catalog?
- How do you identify the right fit and what are the considerations?
- How to measure success for a data catalog?
We'll dissect the reasons behind these challenges and share our experience of implementing data catalogs across different organisations.
See this talk and many more by getting your ticket to PyCon AU now!
I want a ticket!
I'm Harmeet, a lead data engineer at Thoughtworks who's been lucky enough to work with some incredibly smart people in building data and ML platforms & products . Over the years, I've had the opportunity to lead teams in building data and ML platforms, products, and work on organisational transformations including evolving data function operating models to scale. I’ve dabbled in various industries, including energy, accounting, airlines, retail and many more , helping teams mature their data and ML capabilities. <br /> Beyond my professional life, I'm a co-host of the data engineering Melbourne meetup and also had the opportunity to tech review an O'Reilly book 'Effective Machine Learning Teams'.
Vishal is a Senior Data Consultant with DevOps skills who has worked across a range of industries. He has experience in establishing Cloud Infrastructure Foundations, Event Driven Data Lake, Data Visualisation, Master Data Management, Data Quality, Data Governance frameworks and Data Mesh. He is passionate about real time event driven distributed systems. Vishal has used these experiences to enable use cases which help businesses realise real value from data.