Serpents and Ducks: wrangling data with Python and DuckDB
Got a billion rows of data in a weird file format? Wishing you could wrangle a dataframe from a geospatial dataset? A bit lost interacting with a remote API? Let’s wrangle some data with Python and DuckDB.
DuckDB executes analytical SQL queries without the need for a server. DuckDB features a deep and deceptively simple integration with the Python ecosystem, allowing us to query, wrangle, and output data, alongside all your favourite Python tools.. Its powerful analytical features and rich integrations position DuckDB as an invaluable tool for anyone working with analytical data in Python, helping you solve complex problems with ease and elegance. In this practical talk, we’ll introduce DuckDB, a fast and versatile analytical database to keep in your data toolkit. We’ll go through how to use the DuckDB Python client effectively, taking advantage of DuckDB’s efficient data processing features, as well as its integrations with libraries like Pandas and Ibis.
See this talk and many more by getting your ticket to PyCon AU now!
I want a ticket!
Ned is a lead data science engineer at Thoughtworks Australia. He’s worked across a range of sectors and domains, applying machine learning, natural language processing, and data analysis & visualisation to business challenges and opportunities. Ned has used these experiences to develop strategies for making effective use of data & AI for identifying and framing the business value of data science and analytics initiatives. Ned is also a co-author of Getting Started with DuckDB, recently published by Packt.
Simon Aubury serves as an Associate Director of Data Platforms at Simple Machines. Simon describes his job to his children as being a “data geek”. Although it doesn’t impress his kids, he is proud to have worked around the globe building highly available distributed data systems for finance, transport, health care, insurance and telecommunications clients.