Train a tensorflow lite model to detect plant diseases and integrate it in an android app

Even though this tutorial has primarily been designed for mac OS, the basic commands and instruction stays same across all platforms

This tutorial will consist of two parts :

1) Training the model using python3, pip3, virtualenv
2) Using the trained model in our android application

1) Training the model using python3,pip3,virtualenv


First of all go here and install python3,pip3,virtualenv

If you get an error like this

AttributeError: module 'tensorflow.contrib.eager' has no attribute 'enable_eager_execution'

This happens because Eager execution may be missing from the tensorflow version that you are using.

Please run the following command and install tensorflow to fix the above error.


python3 -m pip install --upgrade https://storage.googleapis.com/tensorflow/mac/cpu/tensorflow-1.12.0-py3-none-any.whl 
After having successfully completed the steps mentioned above, perform the following steps

1) First of all activate the virtual environment that we have already prepared by running the following command.

source ./venv/bin/activate
Download to get the complete directory setup here blogger_ml.zip. If you already have downloaded the project blogger_ml.zipjump to step 4 or else continue to next step(step 2).

2) The retrain script is from the TensorFlow Hub repobut it is not installed as part of the pip package. So for simplicity I've included it inside blogger_ml.zip. You can run the script using the python command.

3)Now create a folder blogger_ml and inside it, create another folder tf_files, now extract the contents of above scripts.zip to tf_files. Inside the folder tf_files, create another folder train_models and add folders for each labels which you would like to create. Inside each labelled folder, add a minimum of 20 images corresponding to that particular label. In our case here we will be adding three folders with label name apple cedar rust, peach bacterial spot, bell pepper bacterial spot(these are just plant disease names, so that we could classify each image based on plant diseases). Download it here train_models.zip and extract it into tf_files

4) Now in the terminal we have already activated the virtual environment(step 1). Now change the directory to blogger_ml in the terminal(by doing cd command).

5) Now type the following commands to set the image size and architecture for the model to be trained


IMAGE_SIZE=224
ARCHITECTURE="mobilenet_0.50_${IMAGE_SIZE}"

 6) You are now all set to train the model. Run the following command


python -m scripts.retrain \
  --bottleneck_dir=tf_files/bottlenecks \
  --how_many_training_steps=500 \
  --model_dir=tf_files/models/ \
  --summaries_dir=tf_files/training_summaries/"${ARCHITECTURE}" \
  --output_graph=tf_files/retrained_graph.pb \
  --output_labels=tf_files/retrained_labels.txt \
  --architecture="${ARCHITECTURE}" \
  --image_dir=tf_files/train_models

If you have come across the error "ZeroDivisionError: integer division or modulo by zero", this probably means that you haven't added a minimum of 20 images under each label. Try adding at least 20 images per label and you should be good.


Now just wait some time for the training process to complete. This would take a while. Once the training process is complete goto  blogger_ml> tf_files, here you can see that two files retrained_graph.pb, retrained_labels.txt have been auto generated as a result of the training process.

You can now confirm that the trained model is working fine by running the following command


python -m scripts.label_image \
    --graph=tf_files/retrained_graph.pb  \
    --image=tf_files/sample_image_directory/sample.jpg
Note that you will need to create a directory named sample_image_directory and add an image named sample.jpg inside it to run the above command. If everything goes well, you would see an output with confidence level associated with each label

Now we need to convert the retrained_graph.pb file to .lite extention file so that we can use it in our android application. You will need a tool called toco to convert .pb to .lite . Perform the following steps for conversion

1) type the following to set image size


IMAGE_SIZE=224

2) Now run the following command



toco \
  --graph_def_file=tf_files/retrained_graph.pb \
  --output_file=tf_files/optimized_graph.lite \
  --input_format=TENSORFLOW_GRAPHDEF \
  --output_format=TFLITE \
  --input_shape=1,${IMAGE_SIZE},${IMAGE_SIZE},3 \
  --input_array=input \
  --output_array=final_result \
  --inference_type=FLOAT \
  --input_data_type=FLOAT

If everything went well, you could find optimized_graph.lite inside blogger_ml> tf_files


2) Using the trained model in our android application


Now that we have two files optimized_graph.lite , retrained_labels.txt inside blogger_ml> tf_files, we can now use these files inside our android application to build a camera application which scans the leaves of a plant and detect the diseases associated with it.

Now go here and download the sample android project. Now extract the project and go to
tensorflow-for-poets-2 >> android >> tflite . Import the tflite directory to android studio. Now, you will have the following directory structure in android studio


Now rename your optimized_graph.lite , retrained_labels.txt files created above to graph.lite, labels.txt respectively and then replace it with the existing files in assets folder shown in the image.

Now clean and build the project. Run it.

Now try scanning images using the above app and checkout the results.

I will be posting a tutorial shortly on how to build leaf scanning android app using the above generated tensorflow lite model.

For now, you may check the following video demo of an app where i have used the above tensorflow model.

8 comments:

  1. Hi. I am stuck at step 6 which is to train the model.. i got the error ImportError: No module named '_pywrap_tensorflow_internal'pls help

    ReplyDelete
  2. Hi. I am stuck at step 6 which is to train the model.. i got the error ImportError: No module named '_pywrap_tensorflow_internal'pls help

    ReplyDelete
    Replies
    1. have you installed an appropriate version of tensorflow that is supported by your python version

      Delete
    2. Hello I am using anaconda with 3.6.8 and with the latest tensorflow version. how can I run it? Where do I run it.

      Delete
    3. I use windows how can I run those command (setting image size and such)

      Delete
  3. Can I get the windows version of the commands?

    ReplyDelete
  4. Hi! I'm here again. Why does the toco command don't work? I got the error: No module named 'tensorflow.contrib.lite.python.tflite_convert'

    ReplyDelete
  5. Can these be used in multiple object detection?

    ReplyDelete