How-to: Template Matching

In this exercise, you will get to try out template matching.

Updated Instructions (2024)

Open Notebook on Google Colab

Navigate to https://github.com/dionny/ai-tutorial-notebooks/blob/main/template_matching.ipynb and click on "Open in Colab" at the top.

Run Template Matching

Follow the steps on the notebook, executing each of the Python code blocks in the order in which they appear. To execute a code block, click the Play icon to the left of the block.

Note: After executing the first code block, an upload file widget will appear. This widget allows you to upload an image file from your computer. This, in turn, is the image that will be queried.

Note: After executing the second code block, an upload file widget will appear. This widget allows you to upload an image file from your computer. This, in turn, is the image that will be used as a template.

Widget For Uploading a File

After uploading a file, simply follow the instructions in the Jupyter notebook, executing each of the code blocks in the order of which they appear.

Viewing the Results

Viewing the Results

When viewing the results, you will be able to see a Screenshot Image with the matching bounding box along with a Confidence Score for the match.

Discussion Topic: How useful do you think this would be for testing purposes? Note that this approach allows us to locate elements on an application screen without using a Document Object Model (DOM) or similar page source for mobile applications.

What are some drawbacks of relying on the DOM or on page source?

Discussion Topic: How reliable was the template matching algorithm? How about when changing the various different switches?

Congratulations!

For making it even further. Not only have you used deep learning models, but you've also got a simpler, less computationally-intensive technique in your toolbox.

Legacy Instructions (Deprecated)

At a Glance

In this exercise, you will:

Set Up Your Jupyter Server

Navigate to https://jupyterhub.dionny.dev and make sure you arrive at the following login screen:

Jupyter Server Login Screen

Enter the following credentials:

  • Username should be your e-mail address or first + last name.

  • Password should be admin.

Your e-mail information is not being collected. We need to ensure your username is unique when considering all tutorial attendees.

Click on Sign In.

After a few short moments, your unique Jupyter notebook server will be created for you.

Launch Notebook

Double click the template_matching.ipynb notebook.

Your Own Unique Jupyter Server

Run Template Matching

Follow the steps on the notebook, executing each of the Python code blocks in the order of which they appear. To execute a code block, first click the block, then click the Play icon.

Executing a Code Block on Jupyter

Note: After executing the first code block, an upload file widget will appear. This widget allows you to upload an image file from your computer. This, in turn, is the image that will be queried.

Note: After executing the second code block, an upload file widget will appear. This widget allows you to upload an image file from your computer. This, in turn, is the image that will be used as a template.

Widget For Uploading a File

After uploading a file, simply follow the instructions in the Jupyter notebook, executing each of the code blocks in the order of which they appear.

Viewing the Results

When viewing the results, you will be able to see a Screenshot Image with the matching bounding box along with a Confidence Score for the match.

Discussion Topic: How useful do you think this would be for testing purposes? Note that this approach allows us to locate elements on an application screen without using a Document Object Model (DOM) or similar page source for mobile applications.

What are some drawbacks of relying on the DOM or on page source?

Discussion Topic: How reliable was the template matching algorithm? How about when changing the various different switches?

Congratulations!

For making it even further. Not only have you used deep learning models, but you've also got a simpler, less computationally-intensive technique in your toolbox.

PreviousHow-to: AI Using Appium

Last updated

Was this helpful?