Azure Machine Learning and the Team Data Science Process – Part 1

The Team Data Science Process allows you to have a repeatable, controlled progression for analytics projects. You can use it with any Data Science technologies, and Microsoft has a full suite of products you can use for AI programming.

Microsoft Azure Machine Learning Services have several components that assist in large-scale AI programming, Deep Learning, and more. There are three main components:

Microsoft Azure has a few components for creating and deploying AI solutions:

1. An Experimentation Service – A service that logs and tracks your experiments in a central location.
2. A Model Management Service – Uses Azure Storage and the Azure Container Registry to log the model environment necessary for scoring, and allows versioning.
3. Docker, Conda - Technologies that provide an abstracted, contained environment. You use this for both Experimentation and Deploying trained Models.
4. Azure Container Service (AKS) – A scalable Docker environment you can use to host your trained models as a web service. (Note: You have other deployment options, but this service is fully managed for you)
5. Application Insights – Along with Azure Logs and Storage, tracks the information from your environment.

Docker is used as a contained environment for two functions:

1. Running Experiments to create a trained Machine Learning Model
2. Hosting the web service for users to access your trained Model.

Local Tools and Environments

1. A Local Development Environment – A Command-Line Interface of “az ml” commands used to create experiments, set up environments, deploy your models, and track and query all parts of the system.
2. Azure Workbench – A graphical environment that you can use to create and manage your experiments, as well as a full data-preparation-by-example tool. The installation of this tool gives you the az ml commands, the conda python environment controls, git integration and many other required components.
3. Visual Studio Code – A multi-platform coding environment, in which you can install the Azure AI tools – this makes VS Code work in a similar way to Azure Workbench.

You can also use your own Integrated Development Environments (IDE’s) for the ML code you write. You would simply install the Azure ML Workbench, edit the code in your IDE, and use the az ml commands to control your project.

In the next installment, I’ll cover how these pieces are used to facilitate the Team Data Science Process by mapping each component.