amc:ss2023:group-v:start
Differences
This shows you the differences between two versions of the page.
Next revision | Previous revision | ||
amc:ss2023:group-v:start [2023/07/23 21:46] – created ainul-danish-zulhusni.bin-azrin | amc:ss2023:group-v:start [2023/07/25 21:06] (current) – shreevarshan.sivakumar | ||
---|---|---|---|
Line 1: | Line 1: | ||
- | **Smart water meter with AI on the edge** | + | ====== |
+ | |||
+ | |||
+ | |||
+ | //**by Ainul Danish Zulhusni Bin Azrin (28977), Shreevarshan Sivakumar (31230), Yacine Aider (30913)**// | ||
+ | |||
+ | |||
+ | =====1. Introduction====== | ||
- | **1. Introduction: | ||
Integrating artificial intelligence (AI) with edge devices has changed the game in the quickly developing IoT (Internet of Things) industry. Smart water meter reading, which enables effective monitoring and management of water consumption in homes and businesses, is one of the promising applications of AI on the Edge. In this context, we'll look at how to use the ESP32-CAM, a powerful and reasonably priced microcontroller with a camera module, to enable real-time water meter reading with artificial intelligence capabilities and seamlessly integrate it with Home Assistant, a well-liked home automation platform, using the MQTT (Message Queuing Telemetry Transport) protocol. The ESP32-CAM is a flexible microcontroller with built-in Wi-Fi, Bluetooth, and a camera module. It is the perfect device to use in a variety of IoT projects due to its small size and low power consumption. We can do on-device AI operations like image identification and data processing directly at the edge by utilizing the processing capacity of ESP32-CAM, eliminating the need on cloud services and assuring faster and more dependable real-time data analysis. | Integrating artificial intelligence (AI) with edge devices has changed the game in the quickly developing IoT (Internet of Things) industry. Smart water meter reading, which enables effective monitoring and management of water consumption in homes and businesses, is one of the promising applications of AI on the Edge. In this context, we'll look at how to use the ESP32-CAM, a powerful and reasonably priced microcontroller with a camera module, to enable real-time water meter reading with artificial intelligence capabilities and seamlessly integrate it with Home Assistant, a well-liked home automation platform, using the MQTT (Message Queuing Telemetry Transport) protocol. The ESP32-CAM is a flexible microcontroller with built-in Wi-Fi, Bluetooth, and a camera module. It is the perfect device to use in a variety of IoT projects due to its small size and low power consumption. We can do on-device AI operations like image identification and data processing directly at the edge by utilizing the processing capacity of ESP32-CAM, eliminating the need on cloud services and assuring faster and more dependable real-time data analysis. | ||
Line 8: | Line 15: | ||
Home Assistant is a well-liked open-source platform for home automation that unifies the management of smart devices and services. The ESP32-CAM can easily connect with Home Assistant by utilizing MQTT, a compact messaging standard. The MQTT broker hosted on Home Assistant can receive the meter reading data that has been processed on the edge by the AI model. | Home Assistant is a well-liked open-source platform for home automation that unifies the management of smart devices and services. The ESP32-CAM can easily connect with Home Assistant by utilizing MQTT, a compact messaging standard. The MQTT broker hosted on Home Assistant can receive the meter reading data that has been processed on the edge by the AI model. | ||
- | The ESP32-CAM and Home Assistant communicate through the MQTT broker. It makes message exchange more effective and guarantees dependable data transport between devices. As an MQTT broker, | + | The ESP32-CAM and Home Assistant communicate through the MQTT broker. It makes message exchange more effective and guarantees dependable data transport between devices. As an MQTT broker, |
+ | |||
+ | ===== 2. Materials | ||
- | **2.Materials: | + | • **ESP 32-CAM module** |
- | • **__ESP 32-CAM module__** | ||
The ESP32-CAM module (Fig 1) is a small development board that supports cameras and Wi-Fi and is based on the ESP32 microcontroller. It has GPIO pins, a camera interface, and a USB-to-TTL converter for programming. It is frequently utilized for IoT initiatives, | The ESP32-CAM module (Fig 1) is a small development board that supports cameras and Wi-Fi and is based on the ESP32 microcontroller. It has GPIO pins, a camera interface, and a USB-to-TTL converter for programming. It is frequently utilized for IoT initiatives, | ||
Line 20: | Line 29: | ||
- | • **__ESP32-CAM AI-Thinker MB Programmer__** | + | • **ESP32-CAM AI-Thinker MB Programmer** |
A shield called the ESP32-CAM AI-Thinker MB programmer (Fig 2) can be connected to the GPIOs on the ESP32-CAM board. The CH340C USB to serial chip is included with the programmer. As a result, the use of the shield' | A shield called the ESP32-CAM AI-Thinker MB programmer (Fig 2) can be connected to the GPIOs on the ESP32-CAM board. The CH340C USB to serial chip is included with the programmer. As a result, the use of the shield' | ||
Line 29: | Line 38: | ||
- | • **__16 Gb SD card and SD card reader:__** | + | • **16 Gb MicroSD |
For purposes such as Storage of AI Models and Data, Resource Constraints, | For purposes such as Storage of AI Models and Data, Resource Constraints, | ||
- | • **__Micro | + | • **Micro USB to USB A cable** |
The cable (Fig 3) was necessary to connect to the programmer to the USB slots in the laptop to perform actions needed for examples, flashing the firmware and accessing AI on the edge. | The cable (Fig 3) was necessary to connect to the programmer to the USB slots in the laptop to perform actions needed for examples, flashing the firmware and accessing AI on the edge. | ||
Line 42: | Line 51: | ||
| | ||
- | • **__3D housing:__** | + | • **3D housing** |
A design inspired from the web was taken and modified to fit the needs of the water meter sample provided. The output of the printed file is seen in Fig 4. | A design inspired from the web was taken and modified to fit the needs of the water meter sample provided. The output of the printed file is seen in Fig 4. | ||
Line 52: | Line 61: | ||
- | **2.1Methods:** | + | ===== 3. Methods ===== |
- | **1. Downloading the AI on the edge source code zip from github:** | ||
- | This was the necessary file that was used to work on. This was unzipped into a location as per needs. Ensured was that the SD card folder in the version downloaded was not empty and downloaded was also partitions.bin, | + | ==== 3.1. Downloading |
- | **2. Flashing using the python based esptool:** | ||
- | • A python environment is needed, used for this was anaconda prompt. Installed was esptool (Fig 5). | + | This was the necessary file that was used to work on. This was unzipped into a location as per needs. Ensured was that the SD card folder in the version downloaded was not empty and downloaded was also partitions.bin, |
+ | |||
+ | ==== 3.2. Flashing using the python based esptool ==== | ||
+ | |||
+ | |||
+ | • A python environment is needed, used for this was anaconda prompt. Installed was esptool (Fig 5a). | ||
+ | |||
+ | < | ||
Code snippet: pip install esptool | Code snippet: pip install esptool | ||
+ | </ | ||
{{: | {{: | ||
- | Fig 5: Installing esptool. Source: own image - Shreevarshan | + | Fig 5a: Installing esptool. Source: own image - Shreevarshan |
- | • Setup tools was installed using code: pip install | + | • Setup tools was installed using code: pip install |
{{: | {{: | ||
- | Fig 5: Installing setup tools. Source: own image - Shreevarshan | + | Fig 5b: Installing setup tools. Source: own image - Shreevarshan |
• The ESP 32 module was connected to the MB programmer to put it in boot mode and then the following commands were used to erase the flash (Fig 6) and write the flash bootloader, partitions and firmware (Fig 7). For writing these files, it is necessary to make sure to be on the correct directory to call the function, where these files reside. (Fig 7). | • The ESP 32 module was connected to the MB programmer to put it in boot mode and then the following commands were used to erase the flash (Fig 6) and write the flash bootloader, partitions and firmware (Fig 7). For writing these files, it is necessary to make sure to be on the correct directory to call the function, where these files reside. (Fig 7). | ||
The code snippets for: | The code snippets for: | ||
+ | < | ||
Erasing: python -m esptool –chip esp32 erase_flash (uses python to use esptool to erase flash on esp 32 module). | Erasing: python -m esptool –chip esp32 erase_flash (uses python to use esptool to erase flash on esp 32 module). | ||
+ | |||
Flashing: python -m esptool –chip esp32 write_flash 0x01000 bootloader.bin 0x08000 partitions.bin 0x10000 firmware.bin | Flashing: python -m esptool –chip esp32 write_flash 0x01000 bootloader.bin 0x08000 partitions.bin 0x10000 firmware.bin | ||
+ | </ | ||
{{: | {{: | ||
Line 88: | Line 106: | ||
Fig 7: Writing flash after choosing the correct directory. Source: own image - Shreevarshan | Fig 7: Writing flash after choosing the correct directory. Source: own image - Shreevarshan | ||
- | **3. Fomatting the SD card and copying the SD card files:** | + | ==== 3.3. Fomatting the SD card and copying the SD card files ==== |
- | • The SD card was supposed to be formatted in the FAT 32 type. Then the files from the SD folder on the AI on the edge was copied into the SD card. The was opened using a text editor and the SSID and password of the network was given (Fig 8) in this case, used was a hotspot from the PC which was worked on to get easier connections and to get the IP address of the watermeter i.e., ESP module for accessing AI on the edge. | + | • The SD card was supposed to be formatted in the FAT 32 type. Then the files from the SD card folder on the AI on the edge was copied into the SD card. The was opened using a text editor and the SSID and password of the network was given (Fig 8) in this case, used was a hotspot from the PC which was worked on to get easier connections and to get the IP address of the watermeter i.e., ESP module for accessing AI on the edge. |
{{: | {{: | ||
Line 96: | Line 114: | ||
Fig 8: Edited SSID and password on “wlan” file. Source: own image - Shreeevarshan | Fig 8: Edited SSID and password on “wlan” file. Source: own image - Shreeevarshan | ||
- | **4. Getting the IP of the “watermeter”- ESP module:** | + | ==== 3.4. Getting the IP of the “watermeter”- ESP module |
The SD card is replaced into the ESP module and the Micro USB cable is connected to the laptop and the ESP module. It connected to the credentials provided, when it connects to the hotspot, the LED on the ESP lights up. Under the settings of Windows, the IP of the connected devices are available under the personal hotspot and from there the IP can be taken to open AI on the edge (Fig 9). | The SD card is replaced into the ESP module and the Micro USB cable is connected to the laptop and the ESP module. It connected to the credentials provided, when it connects to the hotspot, the LED on the ESP lights up. Under the settings of Windows, the IP of the connected devices are available under the personal hotspot and from there the IP can be taken to open AI on the edge (Fig 9). | ||
Line 104: | Line 123: | ||
Fig 9: Getting the IP of the “watermeter”. Source: own image - Shreevarshan | Fig 9: Getting the IP of the “watermeter”. Source: own image - Shreevarshan | ||
- | **5. AI on the edge and New reference creation:** | + | ==== 3.5. AI on the edge and New reference creation |
• The obtained IP of the “watermeter” is pasted on the search bar of the browser and the AI on the edge is accessed. When the page is accessed, It already had a pre available reference image of the water meter reading. Next step was to create an own reference image by clicking “create new reference”. This accesses the camera on the ESP to take a picture of the reference that is provided (Fig 10). The ESP setup as above was fixed to the 3D mount which was printed to take pictures of the reference readings (Fig 11). | • The obtained IP of the “watermeter” is pasted on the search bar of the browser and the AI on the edge is accessed. When the page is accessed, It already had a pre available reference image of the water meter reading. Next step was to create an own reference image by clicking “create new reference”. This accesses the camera on the ESP to take a picture of the reference that is provided (Fig 10). The ESP setup as above was fixed to the 3D mount which was printed to take pictures of the reference readings (Fig 11). | ||
Line 116: | Line 136: | ||
Fig 11: Creating a new reference image. Source: own image - Shreevarshan | Fig 11: Creating a new reference image. Source: own image - Shreevarshan | ||
- | **6. Creating Alignment marks, ROI and analog references:** | + | ==== 3.6. Creating Alignment marks, ROI and analog references |
• The next step was to create the alignment marks, in this case used was” m3 “ from the image provided as reference(Fig 12) and click on the update reference image and then save, so this process is updated (Fig 13). | • The next step was to create the alignment marks, in this case used was” m3 “ from the image provided as reference(Fig 12) and click on the update reference image and then save, so this process is updated (Fig 13). | ||
Line 130: | Line 151: | ||
Fig 13: Updating the reference. Source: own image - Shreevarshan | Fig 13: Updating the reference. Source: own image - Shreevarshan | ||
- | • **Proceeding to analyse digits and Analog:** | + | • **Proceeding to analyse |
- | There were three digits that could be added as references, further more Number references could be added by clicking “New” and “new ROI after current”. It was not working to provide correct results when decimal places were to be analyses | + | There were three digits that could be added as references, further more Number references could be added by clicking “New” and “new ROI after current”. It was not working to provide correct results when decimal places were to be analysed |
{{: | {{: | ||
Line 138: | Line 159: | ||
Fig 14: Referencing digits. Source: own image- Shreevarshan | Fig 14: Referencing digits. Source: own image- Shreevarshan | ||
- | {{: | + | {{: |
Fig 15: Referencing the analogs. Source: own image - Shreevarshan | Fig 15: Referencing the analogs. Source: own image - Shreevarshan | ||
- | **7. Analysing new image readings with time frames:** | + | ==== 3.7. Analysing new image readings with time frames |
- | The device was completely configured and set with the reference image provided. This setup was later used to read the image to check if it gives correct readings. To do this, “Data” dropdown menu was chosen and then recognition was clicked to take a new picture to get the readings. | + | The device was completely configured and set with the reference image provided. This setup was later used to read the image to check if it gives correct readings. To do this, “Data” dropdown menu was chosen and then recognition was clicked to take a new picture to get the readings. |
{{: | {{: | ||
Line 154: | Line 175: | ||
Fig 17: Configuring the MQTT and Home Assistant. Source: own image - Shreevarshan | Fig 17: Configuring the MQTT and Home Assistant. Source: own image - Shreevarshan | ||
- | **8. Working with Home Assistant** | + | ==== 3.8. Working with Home Assistant |
This project focuses to utilize the many benefits of Home Assistant while also using MQTT to relay data from AI On the Edge water meter. | This project focuses to utilize the many benefits of Home Assistant while also using MQTT to relay data from AI On the Edge water meter. | ||
- | • Since Home Assistant wasn’t already available, first step was to download Home Assistant into the system. Below are the steps needed to run Home Assistant and MQTT. | + | Since Home Assistant wasn’t already available, first step was to download Home Assistant into the system. Below are the steps needed to run Home Assistant and MQTT. |
Among the various methods of installation of Home assistant, Oracle VM was used(Virtual Machine) to operate and get Home Assistant in the system. Oracle VM (Virtual Box) allows the user to enable cross-platform virtualization software. This means that the user can extend their existing computer to run multiple operating systems than the already built-in software, with this, we were able to use Linux, to download, and operate Home Assistant to the PC. Firstly, Oracle VM was downloaded (Virtual Box) and the settings were configured. Then, starting the VM. The IP Address for Home Assistant becomes available (Fig 18, Fig 19). Later, an account was created and the setup was started. | Among the various methods of installation of Home assistant, Oracle VM was used(Virtual Machine) to operate and get Home Assistant in the system. Oracle VM (Virtual Box) allows the user to enable cross-platform virtualization software. This means that the user can extend their existing computer to run multiple operating systems than the already built-in software, with this, we were able to use Linux, to download, and operate Home Assistant to the PC. Firstly, Oracle VM was downloaded (Virtual Box) and the settings were configured. Then, starting the VM. The IP Address for Home Assistant becomes available (Fig 18, Fig 19). Later, an account was created and the setup was started. | ||
- | {{: | + | {{: |
Fig 18: Configuration settings of Oracle VM, source: own image – Ainul Danish | Fig 18: Configuration settings of Oracle VM, source: own image – Ainul Danish | ||
- | {{: | + | {{: |
- | Figure | + | Fig 19: IP Address for Home Assistant, source: own image – Ainul Danish |
Home Assistant can be configured to view all relative home devices in any home. In Fig 20 are all the relative devices that are connected and available at home. More integration can be added to the Home Assistant for more advance smart home services. | Home Assistant can be configured to view all relative home devices in any home. In Fig 20 are all the relative devices that are connected and available at home. More integration can be added to the Home Assistant for more advance smart home services. | ||
- | {{: | + | {{: |
Fig 20: Devices available in-Home Assistant, source: own image – Ainul Danish | Fig 20: Devices available in-Home Assistant, source: own image – Ainul Danish | ||
Line 179: | Line 200: | ||
* **MQTT:** When Home Assistant installed and running, proceeded with was MQTT. It acts as the medium to send data from the publisher (ESP32 cam water meter) to the subscriber (Home Assistant). This enables MQTT to send data reading of the water meter. | * **MQTT:** When Home Assistant installed and running, proceeded with was MQTT. It acts as the medium to send data from the publisher (ESP32 cam water meter) to the subscriber (Home Assistant). This enables MQTT to send data reading of the water meter. | ||
- | {{: | + | {{: |
- | Figure | + | Fig 21: Downloading Mosquito broker in Add-Ons, source: own image – Ainul Danish |
* **Testing the broker:** Any sample topic was added in the publish and listening field (ensure that the topic is the same in both the fields). Proceeded with clicking on Start Listening. In the payload field, any message was typed in and clicked on Publish. The same message received and appearing at the bottom (Fig 22) confirms the working of the broker. | * **Testing the broker:** Any sample topic was added in the publish and listening field (ensure that the topic is the same in both the fields). Proceeded with clicking on Start Listening. In the payload field, any message was typed in and clicked on Publish. The same message received and appearing at the bottom (Fig 22) confirms the working of the broker. | ||
- | {{: | + | {{: |
- | Figure | + | Fig 22: Testing whether the MQTT broker is functioning, |
- | * **Uploading codes:** After MQTT being set, uploaded was the relative codes into the Home Assistant for ESP32 Cam to read and send the data. Stated here are the Home Assistant configurations and finding the “configuration.yaml” file. One of the best ways is to download the “Studio Code Server” add-on from Home Assistant, as it helps to navigate around Home Assistant such as searching for files much easier. This leads to the configuration files (Fig 23). | + | * **Uploading codes:** After MQTT being set, uploaded was the relative codes into the Home Assistant for ESP32 Cam to read and send the data. Stated here are the Home Assistant configurations and finding the “**configuration.yaml**” file. One of the best ways is to download the “Studio Code Server” add-on from Home Assistant, as it helps to navigate around Home Assistant such as searching for files much easier. This leads to the configuration files (Fig 23). Unfortunately, |
- | {{: | + | {{: |
- | Figure 6: “Configuration.yaml” with the help of Studio Code Server to upload the code. Source: own image – Ainul Danish | + | Fig 23: “**Configuration.yaml**” with the help of Studio Code Server to upload the code. Source: own image – Ainul Danish |
- | **__Results and Discussion: | ||
- | The results were successfully obtained and the data of readings is also saved to use these later for post processing. During every time interval set, the device takes the image and does the recognition and saves the data and displays it as well on the overview. As previously mentioned in Step 7. Of methods, it was hard to integrate the Home Assistant to AI on the edge. Working on different laptops and networks could have been the issue. There is already the Home Assistant and MQTT broker available created by the group members, which can be integrated, in this case, it was not possible. | + | ===== 4. Results |
+ | |||
- | **__Issues Detected: | + | The results were successfully obtained and the data of readings is also saved to use these later for post processing. During every time interval set, the device takes the image and does the recognition and saves the data and displays it as well on the overview. As previously mentioned in Step 3.7, because the AI on the edge and Home Assistant was done on separate laptops, further coding for Home Assistant had to be halt only until Configuration.yaml part. Unfortunately, |
- | An issue that frequently | + | Some methods that could be used to maintain best practise are listed: |
- | The AI on the edge was not always very responsive while working, often crashed. If it worked on Windows it didn’t work on a Mac. The ESP also didn’t connect often to the WLAN at home, that was an additional reason to use the personal hotspot. Further issues with Home Assistant | + | |
+ | -> | ||
+ | |||
+ | -> | ||
+ | |||
+ | ->Forums and online resources often contain valuable insights and solutions contributed by experienced developers and users.Regularly updating the firmware and libraries on the ESP32-CAM and Home Assistant can potentially resolve bugs and issues, ensuring compatibility with the latest advancements. | ||
+ | |||
+ | ===== 5. Issues Detected ===== | ||
+ | |||
+ | |||
+ | An issue that frequently | ||
+ | The AI on the edge was not always very responsive while working, often crashed. If it worked on Windows it didn’t work on a Mac. The ESP also didn’t connect often to the WLAN at home, that was an additional reason to use the personal hotspot. Futhermore, the addition of new ROI fields in the digits(eg. decimal places) and analog references did not provide correct results when rebooted and the page was unresponsive during recognition when any new ROI was introduced. Further issues | ||
+ | |||
+ | ===== 6. References | ||
- | **__References: | ||
https:// | https:// | ||
- | https:// | + | https:// |
+ | |||
+ | https:// | ||
https:// | https:// | ||
+ | https:// |
amc/ss2023/group-v/start.1690141619.txt.gz · Last modified: 2023/07/23 21:46 by ainul-danish-zulhusni.bin-azrin