How-to: Build Your Own Classifier

In this demo, we will see how we can build our own classifier for use in Appium.

Install all Dependencies

First, clone the repository located here.

cd classifier-builder
conda create -n python2 python=2.7
conda activate python2
pip install tensorflow==1.12
pip install tensorflow_hub==0.2

Run Sample Classification

cd classifier-builder
cd sample_run
python run_model.py --image cart.png

The classifier will output a set of labels along with a confidence score associated with each label. In the example above, we are challenging a pre-trained sample model with an image of a shopping cart icon it has never seen before.

Training Data Organization

The training data provided with the project is organized as follows:

• There is a training_images directory that contains the training data.

• Within the training_images directory, there are several subdirectories.

• Each subdirectory is named after a label. For example, there is a cart subdirectory.

• Each subdirectory contains example icons for the associated label, that have been collected across many applications

• There is also a _negative subdirectory that contains many example images of things that are not examples for any of the other labels (e.g., negative examples).

Training Your Classifier

First, you will need to collect additional training data and manipulate the provided training data, either by:

• Adding additional examples for any of the already provided labels,

• Creating new subdirectories for new labels, and populating the new subdirectories with example images,

• Deleting already provided labels or examples, or,

• Adding more negative examples.

Once you are satisfied with the dataset, you may execute the training as follows:

python retrain.py --image_dir training_images/ \
--output_graph output/saved_model.pb \
--output_labels output/saved_model.pbtxt \
--how_many_training_steps 4000 \
--learning_rate 0.30 \
--testing_percentage 25 \
--validation_percentage 25 \
--eval_step_interval 50 \
--train_batch_size 2000 \
--test_batch_size -1 \
--validation_batch_size -1 \
--bottleneck_dir /tmp/bottleneck

There are many configurable parameters when running the training step. Choosing adequate parameters for your given dataset is outside the scope of this tutorial. The parameters provided above work well in practice for the dataset we have provided.

The training step is generally fairly time-consuming. In fact, the first time you execute the above command, it will take 2-3 hours to execute, as it builds a cached network on the first run.

Using Your Classifier

After your classifier has finished training according to your provided parameters, two files will be created:

• output/saved_model.pb

• output/saved_model.pbtxt

Run Within Builder Project

To run your classifier just like we did under the sample_run folder, we can simply replace the models under that folder.

cp output/saved_model.pb sample_run/test_ai_model/saved_model.pb
cp output/saved_model.pbtxt sample_run/test_ai_model/saved_model.pbtxt
cd sample_run
python run_model.py --image cart.png

Note that you may use any image you'd like besides cart.png

Run Within Appium

Make sure you run through the steps under Demo 1: AI using Appium first:

How-to: AI Using Appium

To use your classifier in place of the classifier provided by Test.ai out of the box, simply replace the models that are installed by the test-ai-classifier node package:

cp output/saved_model.pb \
~/.nvm/versions/node/v14.16.1/lib/node_modules/test-ai-classifier/model/group1-shard1of1

Then, try using the classifier just like we did for Demo 1.