Enjoy examples of Azure Data Factory with this thorough guide to creating pipelines to move data from one area to another, monitoring, data flows, development, test, production examples and security. A really well paced tutorial into aspects of ADF.
I was interested in what offerings Microsoft has for cloud based data processing, movement and transformation.
Azure Data Factory (ADF) is an low-code, serverless ,ELT tool which runs on top of Spark clusters.
As I'm familiar with ETL I wanted a book which would show me how to use ADF in real scenarios. This is where this book comes in.
This book takes a tutorial-first approach to showing you around ADF. From setup to configuration, pipeline builds, to output creation. It also covers automation and movement from of pipelines from different environments (traditionally dev - uat - production)
When explaining an aspect of ADF, if there is a corresponding or similar aspect in Microsoft SQL Server Integration Services, it will highlight this. So this book is ideal for current SSIS developers.
After reading background on ADF, you will dive headlong into creating your first instance of ADF. From there you will create linked services, datasets, activities. This will includes variables, parameters and expressions.
You will go through moving data into and around Azure services automatically. Transform data nativly usingADF data flows and Power Query data wrangling. Master flow-of-control and triggers for tightly orchestrated pipeline execution. Publish and monitor pipelines easily and with confidence.
It is an understatement how easy this book is to read and understand. By the end of around chapter three I had a grounding in understanding setting up, configuring and creating pipelines.
I would recommend this book for anyone who is familiar with data flows (as in data flowing), ETL systems, or just generally interested in how to set up and configure an ADF instance to run productionised code.
One of the more useful aspects it covers is parameters and variables in linked services and datasets so you don't create multiple linked services of the same type, and instead use variables and parameters.
