Practical Data Science on Databricks

Develop the must-have skills required for any data scientist to get the best results from Azure Databricks. In this book, you’ll get to grips with Databricks, enabling you to power-up your organization’s data science applications. We’ll walk through applying the Databricks AI and ML stack to real-world use cases for natural language processing, computer vision, time series data, and more. We’ll dive deep into the complete model development life cycle for data ingestion and analysis, and get familiar with the latest offerings of AutoML, Feature Store, and MLStudio, on the Databricks platform. You’ll get hands-on experience implementing repeatable ML operations (MLOps) pipeline using MLFlow, track model training and key metrics, and explore real-time ML, anomaly detection, and streaming analytics with Delta lake and Spark Structured Streaming. Starting with an overview of Data Science use cases across different organizations and industries, you will then be introduced to feature stores, feature tables, and how to access them. You will see why AutoML is important and how to create a baseline model with AutoML within Databricks. Utilizing the ML Flow model registry to manage model versioning and transition to production will be covered, along with detecting and protecting against model drift in production environments. By the end of the book, you will know how to set up your Databricks ML development and deployment as a CI/CD pipeline. In this book we are going to specifically focus on the tools catering to the Data Scientist persona. Readers who want to learn how to successfully build and deploy end-end Data Science projects using the Databricks cloud agnostic unified analytics platform will benefit from this book, along with AI and Machine Learning practitioners.