Differences

This shows you the differences between two versions of the page.

--- amc:ss2023:group-v:start [2023/07/24 20:58] – shreevarshan.sivakumar
+++ amc:ss2023:group-v:start [2023/07/25 21:06] (current) – shreevarshan.sivakumar
@@ Line 1: / Line 1: @@
-**Smart water meter with AI on the edge**
+====== Smart water meter with AI on the edge & Home Assistant ======
+//**by Ainul Danish Zulhusni Bin Azrin (28977), Shreevarshan Sivakumar (31230), Yacine Aider (30913)**//
+=====1. Introduction======
-**by Ainul Danish Zulhusni Bin Azrin (28977), Shreevarshan Sivakumar (31230), Yacine Aider (30913)**
-**__1. Introduction:__**
 Integrating artificial intelligence (AI) with edge devices has changed the game in the quickly developing IoT (Internet of Things) industry. Smart water meter reading, which enables effective monitoring and management of water consumption in homes and businesses, is one of the promising applications of AI on the Edge. In this context, we'll look at how to use the ESP32-CAM, a powerful and reasonably priced microcontroller with a camera module, to enable real-time water meter reading with artificial intelligence capabilities and seamlessly integrate it with Home Assistant, a well-liked home automation platform, using the MQTT (Message Queuing Telemetry Transport) protocol. The ESP32-CAM is a flexible microcontroller with built-in Wi-Fi, Bluetooth, and a camera module. It is the perfect device to use in a variety of IoT projects due to its small size and low power consumption. We can do on-device AI operations like image identification and data processing directly at the edge by utilizing the processing capacity of ESP32-CAM, eliminating the need on cloud services and assuring faster and more dependable real-time data analysis.
@@ Line 12: / Line 17: @@
 The ESP32-CAM and Home Assistant communicate through the MQTT broker. It makes message exchange more effective and guarantees dependable data transport between devices. As an MQTT broker, Home Assistant can receive the water meter readings sent by the ESP32-CAM and show the data on its user interface in real time. Customers may easily keep an eye on how much water they use and create automated rules based on the results.
-**__2. Materials:__**
+===== 2. Materials  =====
-• **__ESP 32-CAM module__**
+• **ESP 32-CAM module**
 The ESP32-CAM module (Fig 1) is a small development board that supports cameras and Wi-Fi and is based on the ESP32 microcontroller. It has GPIO pins, a camera interface, and a USB-to-TTL converter for programming. It is frequently utilized for IoT initiatives, security cameras, and AI edge applications.
@@ Line 23: / Line 29: @@
-• **__ESP32-CAM AI-Thinker MB Programmer__**
+• **ESP32-CAM AI-Thinker MB Programmer**
 A shield called the ESP32-CAM AI-Thinker MB programmer (Fig 2) can be connected to the GPIOs on the ESP32-CAM board. The CH340C USB to serial chip is included with the programmer. As a result, the use of the shield's USB interface to program the ESP32-CAM is possible. The shield also includes BOOT (IO0) and RESET (IO0) buttons. This was helpful for quickly flashing the ESP32-CAM or resetting it.
@@ Line 32: / Line 38: @@
-• **__16 Gb SD card and SD card reader:__**
+• **16 Gb MicroSD card and MicroSD card reader**
 For purposes such as Storage of AI Models and Data, Resource Constraints, Flexibility and Upgradability, Offline AI Processing, Data Logging and Retrieval an SD card was used. A reader was used to read the and write to the SD card as the laptops used were not having an SD card slot.
-• **__Micro USB to USB A cable:__**
+• **Micro USB to USB A cable**
 The cable (Fig 3) was necessary to connect to the programmer to the USB slots in the laptop to perform actions needed for examples, flashing the firmware and accessing AI on the edge.
@@ Line 45: / Line 51: @@
-• **__3D housing:__**
+• **3D housing**
 A design inspired from the web was taken and modified to fit the needs of the water meter sample provided. The output of the printed file is seen in Fig 4.
@@ Line 55: / Line 61: @@
-**__3. Methods:__**
+===== 3. Methods =====
+==== 3.1. Downloading the AI on the edge source code zip from github  ====
-**3.1. Downloading the AI on the edge source code zip from github:**
 This was the necessary file that was used to work on. This was unzipped into a location as per needs. Ensured was that the SD card folder in the version downloaded was not empty and downloaded was also partitions.bin, bootloader.bin and firmware.bin from the github separately. Link to github can be accessed here: https://github.com/jomjol/AI-on-the-edge-device
-**3.2. Flashing using the python based esptool:**
+==== 3.2. Flashing using the python based esptool ====
 • A python environment is needed, used for this was anaconda prompt. Installed was esptool (Fig 5a).
+<file>
 Code snippet: pip install esptool
+</file>
 {{:amc:ss2023:group-v:code_snippet_.jpg?400|}}
@@ Line 79: / Line 91: @@
 The code snippets for:
+<file>
 Erasing: python -m esptool –chip esp32 erase_flash (uses python to use esptool to erase flash on esp 32 module).
 Flashing: python -m esptool –chip esp32 write_flash 0x01000 bootloader.bin 0x08000 partitions.bin 0x10000 firmware.bin
+</file>
 {{:amc:ss2023:group-v:erasing_flash.jpg?400|}}
@@ Line 92: / Line 106: @@
 Fig 7: Writing flash after choosing the correct directory. Source: own image - Shreevarshan
-**3.3. Fomatting the SD card and copying the SD card files:**
+==== 3.3. Fomatting the SD card and copying the SD card files  ====
 • The SD card was supposed to be formatted in the FAT 32 type. Then the files from the SD card folder on the AI on the edge was copied into the SD card. The was opened using a text editor and the SSID and password of the network was given (Fig 8) in this case, used was a hotspot from the PC which was worked on to get easier connections and to get the IP address of the watermeter i.e., ESP module for accessing AI on the edge.
@@ Line 100: / Line 114: @@
 Fig 8: Edited SSID and password on “wlan” file. Source: own image - Shreeevarshan
-**3.4. Getting the IP of the “watermeter”- ESP module:**
+==== 3.4. Getting the IP of the “watermeter”- ESP module  ====
 The SD card is replaced into the ESP module and the Micro USB cable is connected to the laptop and the ESP module. It connected to the credentials provided, when it connects to the hotspot, the LED on the ESP lights up. Under the settings of Windows, the IP of the connected devices are available under the personal hotspot and from there the IP can be taken to open AI on the edge (Fig 9).
@@ Line 108: / Line 123: @@
 Fig 9: Getting the IP of the “watermeter”. Source: own image - Shreevarshan
-**3.5. AI on the edge and New reference creation:**
+==== 3.5. AI on the edge and New reference creation  ====
 • The obtained IP of the “watermeter” is pasted on the search bar of the browser and the AI on the edge is accessed. When the page is accessed, It already had a pre available reference image of the water meter reading. Next step was to create an own reference image by clicking “create new reference”. This accesses the camera on the ESP to take a picture of the reference that is provided (Fig 10). The ESP setup as above was fixed to the 3D mount which was printed to take pictures of the reference readings (Fig 11).
@@ Line 120: / Line 136: @@
 Fig 11: Creating a new reference image. Source: own image - Shreevarshan
-**3.6. Creating Alignment marks, ROI and analog references:**
+==== 3.6. Creating Alignment marks, ROI and analog references  ====
 • The next step was to create the alignment marks, in this case used was” m3 “ from the image provided as reference(Fig 12) and click on the update reference image and then save, so this process is updated (Fig 13).
@@ Line 146: / Line 163: @@
 Fig 15: Referencing the analogs. Source: own image - Shreevarshan
-**3.7. Analysing new image readings with time frames:**
+==== 3.7. Analysing new image readings with time frames ====
 The device was completely configured and set with the reference image provided. This setup was later used to read the image to check if it gives correct readings. To do this, “Data” dropdown menu was chosen and then recognition was clicked to take a new picture to get the readings.  It first initialises the device, then takes the image, then aligns and then the data it reads can be available in the overview (Fig 16). This Process of image processing was configured to every 3 minutes under the “Settings” dropdown and then clicking on “configuration”. When scrolled to the last, the time interval can be set to take pictures. Also here the MQTT and Home Assistant could be set up to send this data to the dashboard (Fig 17). In this case, the page was very unresponsive to do the configurations of the MQTT and Home Assistant as the work was done on different laptops and the ESP module did not work on the other laptop which had the Home Assistant installed.
@@ Line 158: / Line 175: @@
 Fig 17: Configuring the MQTT and Home Assistant. Source: own image - Shreevarshan
-**3.8. Working with Home Assistant**
+==== 3.8. Working with Home Assistant  ====
 This project focuses to utilize the many benefits of Home Assistant while also using MQTT to relay data from AI On the Edge water meter.
-• Since Home Assistant wasn’t already available, first step was to download Home Assistant into the system. Below are the steps needed to run Home Assistant and MQTT.
+Since Home Assistant wasn’t already available, first step was to download Home Assistant into the system. Below are the steps needed to run Home Assistant and MQTT.
 Among the various methods of installation of Home assistant, Oracle VM was used(Virtual Machine) to operate and get Home Assistant in the system. Oracle VM (Virtual Box) allows the user to enable cross-platform virtualization software. This means that the user can extend their existing computer to run multiple operating systems than the already built-in software, with this, we were able to use Linux, to download, and operate Home Assistant to the PC. Firstly, Oracle VM was downloaded (Virtual Box) and the settings were configured. Then, starting the VM. The IP Address for Home Assistant becomes available (Fig 18, Fig 19). Later, an account was created and the setup was started.
@@ Line 200: / Line 217: @@
-**__4. Results and Discussion:__**
+===== 4. Results and Discussion =====
 The results were successfully obtained and the data of readings is also saved to use these later for post processing. During every time interval set, the device takes the image and does the recognition and saves the data and displays it as well on the overview. As previously mentioned in Step 3.7, because the AI on the edge and Home Assistant was done on separate laptops, further coding for Home Assistant had to be halt only until Configuration.yaml part. Unfortunately, the ESP32 CAM was unable to connect to the laptop that had Home Assistant installed and set up. Therefore, no further progress could be made. However, a firm belief from the team is that the project may have had a  successful integration to view data on Home Assistant if everything wasn't done separately by group members and on different networks and laptops, as no issue was encountered in setting up Home Assistant.
@@ Line 212: / Line 230: @@
 ->Forums and online resources often contain valuable insights and solutions contributed by experienced developers and users.Regularly updating the firmware and libraries on the ESP32-CAM and Home Assistant can potentially resolve bugs and issues, ensuring compatibility with the latest advancements.
-**__5. Issues Detected:__**
+===== 5. Issues Detected =====
 An issue that frequently occurred was an error called “Brownout Trigger detected”. This error essentially means that, there is a temporary drop in voltage in the power supply, which is usually below the normal operating voltage range of the system. So, the programmer/machine is limiting and inhibiting itself from uploading/running the code, as it would potentially cause issues for the ESP32Cam. Therefore, the cam was unable to work. While this error kept happening with using the cam with the UARTsBee programmer. The possible explanations as to why this error occur were: 1) the USB cable is of poor quality, and possibly too long. 2) The computer’s USB port cannot supply enough power to the board. 3) The ESP32 Cam itself is defective. 4) Other components in the circuit are not correctly wired, affecting the power supply. One of the solutions tried was to connect ESP32-CAM power supply from 3.3V to 5V from the UARTsBee, which still had no impact.
 The AI on the edge was not always very responsive while working, often crashed. If it worked on Windows it didn’t work on a Mac. The ESP also didn’t connect often to the WLAN at home, that was an additional reason to use the personal hotspot. Futhermore, the addition of new ROI fields in the digits(eg. decimal places) and analog references did not provide correct results when rebooted and the page was unresponsive during recognition when any new ROI was introduced. Further issues encountered with Home Assistant were discussed in Step 3.7 of methods, above.
-**__6. References:__**
+===== 6. References  =====
 https://github.com/jomjol/AI-on-the-edge-device