Test Object Detection AI Model

In another blog, we discussed, how to train and publish the object detection AI Model. This blog will discuss how to consume that AI model in a real scenario.

For our demo, we will use SharePoint as a source to send the input.

• The first step is for a user to create list item and upload the image to SharePoint
• The next step is Power automate flow will be triggered.
• The next step is to Power automate send the data or file content to AI Model.
• The Next AI model returns the result/output.
• Finally, Power automate update the result in SharePoint List, So that we can verify the data

Visual representation for the data flow

Step by Step to consume AI model through Power Automate

1. Create a SharePoint list with the following fields. These are the objects/data elements that will be collected from the image and AI.
   1. Soccer Ball Count – Integer
   2. Confidence – Decimal
   3. Analyze Complete – Boolean
      
2. Create automated cloud flow
   1. Go to https://make.powerapps.com
   2. Select the appropriate environment
   3. Create New Solution (If not exist already)
      • Note: The default environment has some restrictions
   4. Click New to create Cloud Flow
      
   5. Source: SharePoint
   6. Trigger: When an item is created or modified 
       
   7. Enter the name for the flow
      
   8.  Add another action to get SharePoint Attachments
      1. Select the same Site and list
      2. The id should from the Trigger step
         
   9.  Add action to get SharePoint File Content
      • Select the same site
      • File Identifier should ‘Id’ from the “Get attachments” step
        
   10. Add action to AI Builder
       1. Action : Predict
          
   11. Select your Object detection AI model that we trained on our last blog
       • Set the image as the file content from Step #9
         
   12.  Add action to update item in SharePoint
       • Source : SharePoint
       • Action : Update Item
       • Set the Id and Title from the trigger
       • Update other fields
         • Soccer Ball Count
           • Formula:
             length(outputs(‘Predict’)?[‘body/responsev2/predictionOutput/results’])
         • Confidence % :
           mul(items(‘Apply_to_each_2’)?[‘confidence’],100)
         • Analyze Complete – This is to avoid recursive (infinite loop)
           
       •  You may noticed another “Apply to each 2” added since we may collect or predict multiple object from the same image 
3. Test the AI Model
   1. Go to SharePoint
   2. Add List item
   3. Click “Save”
   4. Test Image
       
   5. We should be able to see the result in few minutes at max
   6. Here is the result
      
   7.  Confidence level is 78%. We need to train with more images to increase the confidence level.

Thanks for visiting. Happy Learning!