data / azure
Barnardo’s
Data-pipeline engineering to turn source data into usable downstream information.
My role
Data pipeline engineering
Project context
Data engineering
Contribution areas
Engineering & systems
The situation
Data needed to be ingested and transformed before it could support downstream use. The work sat in the processing pipeline rather than in a customer-facing interface.
My contribution
My work covered data-pipeline architecture and implementation, using Azure Data Factory and medallion-style processing.
The work
Worked on the ingestion and transformation stages, separating the processing of source data from the information prepared for later use.
The result
Data-pipeline architecture and implementation using Azure Data Factory, with a medallion-style approach to processing.