Developing Universal Windows app using WinML model exported from Custom Vision

[日本語版はこちら]

Cognitive Services Custom Vision service is pre-build and customizable image classification & now object detection machine learning models builder, just uploading some photos to detect. You can use your ML models as Web API, also download in several format to use in your app, like CoreML for iOS, TensorFlow for Android, ONNX-based WinML for Windows, and now Docker Files to deploy to Edge devices.

ONNX-based WinML format enables to add ML model into your Windows Universal (UWP) app without using Rest API call.

To create WinML-type model in Custom Vision Service, select "compact" type domain when create new project. Or just change domain type to "compact" from settings and re-train model.

Once compact-type model trained, it should be downloadable from "export" button. Select "WinML" and download "*.onnx" file.

Visual Studio tools for AI (for free) with Visual Studio 2017 is also great tool for ONNX support, it automatically generate class libraries only to add ONNX file to Visual Studio project. Better to be visible to replace long Custom vision model name to friendly name ...

Auto-generated library uses Windows.AI.MachineLearning library. It works as to create LearningModel (Create_*modelname*_Model method), evaluate ModelInput and return ModelOutput. ModelInput is defined as VideoFrame.

Here is my sample code how to call my onnx model named as "FruitDetection.onnx";

private async Task EvaluateVideoFrameAsync(VideoFrame inputFrame)
{
    if (inputFrame != null)
    {
        try
        {
            // Get ONNX file
            string modelPath
                = Path.Combine(Windows.ApplicationModel.Package.Current.InstalledLocation.Path, "FruitDetection.onnx");
            var modelFile = await StorageFile.GetFileFromPathAsync(modelPath);

            // Create WinML Model
            var model = new FruitDetectionModel();
            var learninModel = await model.CreateFruitDetectionModel(modelFile);

            // Set image(VideoFrame)
            var input = new FruitDetectionModelInput
            {
                data = inputFrame
            };

            // Detect image
            var output = await learninModel.EvaluateAsync(input);

            ResultText.Text = output.classLabel[0];
            ResultText.Visibility = Visibility.Visible;
        }
        catch(Exception ex)
        {
        }
    }
}

This "FruitDetection.onnx" is fruits detection model created by Custom Vision Service as below;

Here is sample UWP app (whole source code) using this onnx model to detect photo selected from local files.