Modern Data Medieval Methods

“I was recently asked to make a few suggestions on how to incorporate the practices familiar from “terrestrial” development such as source control, continuous integration, and automated testing into the brave new world of cloud-based data analytics, on a project incorporating Azure Data Factory, Azure Data Lake, and Azure SQL Data Warehouse.

Many of the tools associated with these offerings seem to be aimed at the elusive “Data Professional” rather than the jobbing developer, which creates a conflict between ease of use for “non-programmers” and integration with well-known processes for more experienced developers.

This talk will discuss some of the challenges faced in developing this proof of concept, and solutions found using tools and techniques old and new.”

Leave a Reply