HSRW EOLab Students Wiki

User Tools

Site Tools

Trace:
amc:ss2023:group-d:start

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
amc:ss2023:group-d:start [2023/07/25 10:08] – [b. UartSBee v5.0 Module] 29907_students.hsrwamc:ss2023:group-d:start [2023/07/25 20:23] (current) – [Table] 29907_students.hsrw
Line 15: Line 15:
 ====a. ESP32-CAM====  ====a. ESP32-CAM==== 
  
-The ESP32-CAM is a popular and cheap development board based on the ESP32 microcontroller. Its dimensions of 27mm*40.5mm*4.5mm make it small but a powerful tool. It also comes with a built-in camera module and microSD card slot and has Wi-Fi and Bluetooth connectivity feature with a dual-core processor, making it suitable for a wide range of IoT (Internet of Things) applications, remote controlling, or inspection systems.\\ +The ESP32-CAM is a popular and cheap development board based on the ESP32 microcontroller. Its dimensions of 27mm*40.5mm*4.5mm make it small but a powerful tool. It also comes with a built-in camera module and microSD card slot and has Wi-Fi and Bluetooth connectivity feature with a dual-core processor, making it suitable for a wide range of IoT (Internet of Things) applications, remote controlling, or inspection systems. 
-More details at: [[https://docs.ai-thinker.com/en/esp32-cam]]\\ +More details at: [[https://docs.ai-thinker.com/en/esp32-cam]] 
-^{{:amc:ss2023:group-d:esp32-cam.jpeg?300|}}^\\ +^ {{:amc:ss2023:group-d:esp32-cam.jpeg?300}}                                                                                                    ^ 
-|**Figure 1.** ESP32-CAM \\ Source: [[https://www.nutsvolts.com/magazine/article/build-a-video-camera-using-the-esp32-cam-board]]\\ |+| **Figure 1.** ESP32-CAM (front and back)\\ Source: https://www.nutsvolts.com/magazine/article/build-a-video-camera-using-the-esp32-cam-board  |
  
  
Line 26: Line 26:
 ==== b. UartSBee v5.0 Module ==== ==== b. UartSBee v5.0 Module ====
 The ESP32-CAM used in this project does not come with its own programmer hence UartSBee v5.0 module was used. The UartSBee v5.0 module is a compact and versatile UART (universal asynchronous receiver-transmitter)-to- USB (universal serial bus) adapter. It enables easy communication between microcontrollers (like ESP32-CAM in our case) or other UART-enabled devices and a computer (PC/laptop) through USB. This module is commonly used in electronics projects for debugging, programming, and data exchange, making it a convenient tool for developers or hobbyists. For this project UART of FT232 was used. The ESP32-CAM used in this project does not come with its own programmer hence UartSBee v5.0 module was used. The UartSBee v5.0 module is a compact and versatile UART (universal asynchronous receiver-transmitter)-to- USB (universal serial bus) adapter. It enables easy communication between microcontrollers (like ESP32-CAM in our case) or other UART-enabled devices and a computer (PC/laptop) through USB. This module is commonly used in electronics projects for debugging, programming, and data exchange, making it a convenient tool for developers or hobbyists. For this project UART of FT232 was used.
-More details at: [[https://wiki.seeedstudio.com/UartSBee_v5/]]\\ +More details at: [[https://wiki.seeedstudio.com/UartSBee_v5/]] 
-\\+
 ^ {{:amc:ss2023:group-d:uartsbee_v5_new.jpg?300}}                                          ^ ^ {{:amc:ss2023:group-d:uartsbee_v5_new.jpg?300}}                                          ^
 | **Figure 2.** UartSBee v5.0 Module \\ Source: https://wiki.seeedstudio.com/UartSBee_v5/  | | **Figure 2.** UartSBee v5.0 Module \\ Source: https://wiki.seeedstudio.com/UartSBee_v5/  |
Line 44: Line 44:
 In the ideal case, a watermeter provided by the local authorities is used. For this project, the device was trained on the pictures of water meters with different reading levels on them. In the ideal case, a watermeter provided by the local authorities is used. For this project, the device was trained on the pictures of water meters with different reading levels on them.
  
-^{{:amc:ss2023:group-d:watermeter.jpg?300|}}^\\+^{{:amc:ss2023:group-d:watermeter.jpg?300|}}^
 |**Figure 3.** Example Watermeter\\ Source: [[https://wasser.badenovanetze.de/wasser-allgemein/wasserzaehler/]]| |**Figure 3.** Example Watermeter\\ Source: [[https://wasser.badenovanetze.de/wasser-allgemein/wasserzaehler/]]|
  
Line 51: Line 51:
 ==== f. Microcontroller housing ==== ==== f. Microcontroller housing ====
  
-To mount the setup onto the watermeter, a cardboard housing of length 10 cm and 9 cm diameter was designed. This housing helps to create a controlled environment with a fixed distance from the watermeter and also no external light coming in. Also, the lens was focused on that the object. +To mount the setup onto the watermeter, a cardboard housing of length 10 cm and 9 cm diameter was designed. This housing helps to create a controlled environment with a fixed distance from the watermeter and also no external light coming in. Also, the lens was focused on the object by anticlockwise rotation of the lens. The focus calibration was performed hit and trial method, as different focal length requires different calibration.
  
 ^ {{:amc:ss2023:group-d:housing.jpg?200}}             ^ ^ {{:amc:ss2023:group-d:housing.jpg?200}}             ^
Line 62: Line 62:
 ==== a. Hardware Setup ==== ==== a. Hardware Setup ====
  
-Using jumper cables, ESP32-CAM was connected to the FTDI programmer. The connection setup can be seen in Figure 5 below. GPIO 0 and GND are both connected in ESP32-CAM to upload the code (Program). After programming, it can be unplugged.+Using jumper cables, ESP32-CAM was connected to the FTDI programmer. The connection setup can be seen in Figure 5 below. Figure 6 is the actual experimental setup of this project. Whereas in Figure 7, the schematic representation of the ESP32-CAM and FTDI programmer is shown. GPIO 0 and GND are both connected in ESP32-CAM to upload the code (Program). After the programming code is uploaded, Transfer-receive and programming cables can be unplugged. Only 5v/Vcc and GND connection is required now for the device to function.
  
 ^{{:amc:ss2023:group-d:esp-32_and_programmer_connection.jpg?400|}}^ ^{{:amc:ss2023:group-d:esp-32_and_programmer_connection.jpg?400|}}^
 |**Figure 5.** ESP32-CAM and Programmer connection Sketch| |**Figure 5.** ESP32-CAM and Programmer connection Sketch|
-\\ 
  
  
Line 72: Line 71:
 |**Figure 6.** ESP32-CAM and Programmer connection with USB-A Cable| |**Figure 6.** ESP32-CAM and Programmer connection with USB-A Cable|
  
-..\\ + 
-..\\ +^{{:amc:ss2023:group-d:schematic_drawing_of_connection.jpg?400|}}^ 
-..\\ +|**Figure 7.** Schematic drawing of connection for programming|
-..\\ +
-..\\ +
-..\\ +
-..\\ +
-..\\ +
-..\\ +
-..\\ +
-..\\ +
-..\\ +
-..\\ +
-..\\ +
-..\\ +
-..\\ +
-..\\ +
-..\\ +
-^^ +
-|**Figure 7.** Schematic drawing of connection|+
  
  
Line 97: Line 79:
 ==== b. Software Setup ==== ==== b. Software Setup ====
  
-For this project, the software provided in the AI-on-the-edge-device GitHub repository was implemented. The complete documentation provided by the author can be found under the following link. **[[https://jomjol.github.io/AI-on-the-edge-device-docs/]]**+For this project, the software provided in the AI on the edge device GitHub repository was implemented. The complete documentation provided by the author can be found under the following link. **[[https://jomjol.github.io/AI-on-the-edge-device-docs/]]**
  
 Steps to flash the device with the above-mentioned software are given below: Steps to flash the device with the above-mentioned software are given below:
-Open the web installer in Chrome or Edge: 
  
   - Download the program from the following link. [[https://github.com/jomjol/AI-on-the-edge-device]]    - Download the program from the following link. [[https://github.com/jomjol/AI-on-the-edge-device]] 
Line 120: Line 101:
 ^{{:amc:ss2023:group-d:reference_image_for_documentation.jpg?400|}}^ ^{{:amc:ss2023:group-d:reference_image_for_documentation.jpg?400|}}^
 |**Figure 9.** Reference Image setup| |**Figure 9.** Reference Image setup|
-\\ + 
-2. After that, alignment marks should be set up. It is a reference position for the software to search for the analog and digit regions of interest as observed in Figure 10 below.+2. After that, alignment marks should be set up. It is a reference position for the software to search for the analog and digit regions of interest, which is the unit of measurement in the watermeter's case. These alignment marks are always the same, hence first of all this is recognized by the program, and then the analog and digits in fixed x and y displacements from these marks. The alignment setup of m<sup>3</sup> as individual 'm' and <sup>'3'</sup> is observed in Figure 10 below.
 ^{{:amc:ss2023:group-d:reference_image_for_documentation_2.jpg?400|}}{{:amc:ss2023:group-d:reference_image_for_documentation_3.jpg?400|}}^^ ^{{:amc:ss2023:group-d:reference_image_for_documentation_2.jpg?400|}}{{:amc:ss2023:group-d:reference_image_for_documentation_3.jpg?400|}}^^
 |**Figure 10.** Alignment marks setup for 'm' and <sup>'3'</sup> as constant reference position| |**Figure 10.** Alignment marks setup for 'm' and <sup>'3'</sup> as constant reference position|
-\\ + 
-3. Then, the region of interest for the digits has to be set up, it is done by selecting x and y coordinates for every numeric digit of interest. It is seen in Figure 11 below.+3. Then, the region of interest for the digits (numbers) has to be set up, it is done by selecting x and y coordinates for every numeric digit of interest. It is seen in Figure 11 below.
 ^{{:amc:ss2023:group-d:reference_image_for_documentation_4.jpg?400|}}^ ^{{:amc:ss2023:group-d:reference_image_for_documentation_4.jpg?400|}}^
 |**Figure 11.** Digit region of interest setup| |**Figure 11.** Digit region of interest setup|
  
-4. Likewise, the analog region of interest is marked, as shown in Figure 12. In this project, we are only interested in one digit after decimal. For more precise measurement, other analog meters can also be marked as regions of interest.+4. Likewise, the analog region of interest is marked, as shown in Figure 12. Analog needles give the numbers after the comma for a reading, hence the more needles we introduce the more precise the reading gets. In this project, we are only interested in one digit after decimal. For more precise measurement, other analog meters can also be marked as regions of interest.
 ^{{:amc:ss2023:group-d:reference_image_for_documentation_5.jpg?400|}}^ ^{{:amc:ss2023:group-d:reference_image_for_documentation_5.jpg?400|}}^
 |**Figure 12.** Analog region of interest setup| |**Figure 12.** Analog region of interest setup|
Line 136: Line 117:
  
 =====4. Results ===== =====4. Results =====
-After reboot, the device now automatically captures the image and using its optical character recognition technology converts regions of interest into data readings. The device read the different samples provided as follows. Figure 13 shows the experimental setup of the device over the sample image. And Figure 14 shows the different readings when the device was placed over the different sample images. The value on the top right of the image is the actual post processed value, and above the individual digits and the analog needle are the pre processing initial read values.+After reboot, the device now automatically captures the image and using its optical character recognition technology converts regions of interest into data readings. The device read the different samples provided as follows. Figure 13 shows the experimental setup of the device over the sample image. 
  
 ^{{:amc:ss2023:group-d:experimental_setup_device_over_sample.jpeg?400|}}^ ^{{:amc:ss2023:group-d:experimental_setup_device_over_sample.jpeg?400|}}^
 |**Figure 13.** Experimental setup of the device over the sample image| |**Figure 13.** Experimental setup of the device over the sample image|
 +
 +Figure 14 shows the different readings when the device was placed over the different sample images. The value on the top right of the image is the actual post-processed value, and above the individual digits and the analog needle are the pre-processing initial read values.
  
 ^{{:amc:ss2023:group-d:reference_image_for_documentation_6.jpg?400|}}^ ^{{:amc:ss2023:group-d:reference_image_for_documentation_6.jpg?400|}}^
Line 159: Line 142:
  
 Now that the data is ready, it is only saved locally for 3 days. This limits our scope of communication. This is why, this project was further extended to remote monitoring using the Message Queuing Telemetry Transport (MQTT) protocol. MQTT is a lightweight publish-subscribe machine-to-machine network protocol for message queuing services. As it is an Internet of Things (IoT) protocol, it is suitable for the application of this project. There are 2 roles in this protocol i.e., a client and a broker. In MQTT any conventional server is called an MQTT broker and the clients can connect with it. They can have different roles like a subscriber or publisher. In the scenario of this project, watermeter is a publisher, as it shares its readings with the broker. Now, any client that subscribes to the relevant topics would get the data forwarded from the broker. The MQTT setup in the device can be configured under configuration in the settings menu, this has been shown in Figure 17, where the reading has been forwarded to the broker and credentials for access have been set up.  MQTTX is an application used in the demonstration of this project. In Figure 18, an MQTT deployment can be seen where the watermeter and the MQTTX application both connect to the broker. In Figure 19, Remote data reception by the MQTTX application according to the subscription can be seen. The data is the published data by the watermeter under various topics. The client application can subscribe to any of these topics and the complete list of topics of the watermeter is documented on the MQTT API page: Now that the data is ready, it is only saved locally for 3 days. This limits our scope of communication. This is why, this project was further extended to remote monitoring using the Message Queuing Telemetry Transport (MQTT) protocol. MQTT is a lightweight publish-subscribe machine-to-machine network protocol for message queuing services. As it is an Internet of Things (IoT) protocol, it is suitable for the application of this project. There are 2 roles in this protocol i.e., a client and a broker. In MQTT any conventional server is called an MQTT broker and the clients can connect with it. They can have different roles like a subscriber or publisher. In the scenario of this project, watermeter is a publisher, as it shares its readings with the broker. Now, any client that subscribes to the relevant topics would get the data forwarded from the broker. The MQTT setup in the device can be configured under configuration in the settings menu, this has been shown in Figure 17, where the reading has been forwarded to the broker and credentials for access have been set up.  MQTTX is an application used in the demonstration of this project. In Figure 18, an MQTT deployment can be seen where the watermeter and the MQTTX application both connect to the broker. In Figure 19, Remote data reception by the MQTTX application according to the subscription can be seen. The data is the published data by the watermeter under various topics. The client application can subscribe to any of these topics and the complete list of topics of the watermeter is documented on the MQTT API page:
-[[https://jomjol.github.io/AI-on-the-edge-device-docs/MQTT-API/]]\\+[[https://jomjol.github.io/AI-on-the-edge-device-docs/MQTT-API/]]
  
 ^{{:amc:ss2023:group-d:mqtt_2.jpg?400|}}^ ^{{:amc:ss2023:group-d:mqtt_2.jpg?400|}}^
Line 169: Line 152:
 ^{{:amc:ss2023:group-d:mqtt_client.png?400|}}{{:amc:ss2023:group-d:mqtt_client2.png?400|}}^ ^{{:amc:ss2023:group-d:mqtt_client.png?400|}}{{:amc:ss2023:group-d:mqtt_client2.png?400|}}^
 |**Figure 19.** Remote data reception according to the subscription (main value and timestamp)| |**Figure 19.** Remote data reception according to the subscription (main value and timestamp)|
 +
 +For this project, only two topics i.e.,  the reading after post-processing and time stamp were pursued. Some other topics that could be also subscribed hence, remotely monitored are listed with their respective subscription topic and description in Table 1.
  
 |**Table 1:** Some topics and their description|| |**Table 1:** Some topics and their description||
Line 177: Line 162:
 | watermeter/main/changeabsolut  | Gives the difference between the previous and actual read value.                     | | watermeter/main/changeabsolut  | Gives the difference between the previous and actual read value.                     |
  
-For this project, only two topics i.e.,  the reading after post-processing and time stamp were pursued. Some other topics that could be also subscribed hence, remotely monitored are listed with their respective subscription topic and description in Table 1.  
  
 =====6. Best practice suggestions =====  =====6. Best practice suggestions ===== 
Line 193: Line 177:
 The main issue is still the reliability and accuracy because it cannot be always made sure that the device reads the correct values from the meter. In the above examples, we could see that the digits could be properly recognized but there was always a discrepancy with the analog needle values. Other than that, there is also the question of reliability in terms of maintaining the service, as the AI software doesn't always successfully detect the meter reading. In the course of this project, the image provided was high in contrast, and the possibility of reflection is minimal due to printed samples. But in a real-world scenario, the meter might not always be read or read correctly, due to reflections, brightness level, moisture, or even low contrast in the meter numbers, even if the reference image is calibrated to the best possible level.  The main issue is still the reliability and accuracy because it cannot be always made sure that the device reads the correct values from the meter. In the above examples, we could see that the digits could be properly recognized but there was always a discrepancy with the analog needle values. Other than that, there is also the question of reliability in terms of maintaining the service, as the AI software doesn't always successfully detect the meter reading. In the course of this project, the image provided was high in contrast, and the possibility of reflection is minimal due to printed samples. But in a real-world scenario, the meter might not always be read or read correctly, due to reflections, brightness level, moisture, or even low contrast in the meter numbers, even if the reference image is calibrated to the best possible level. 
  
-These issues can be tackled by using fully digital flow meters that can share their reading via numerous digital data transfer methods. Infrared reading is common with radiators and electric meters, where the values can be accurately transmitted. But it cannot be done in real-time. Another thing that could be further looked into in the future would be to introduce a battery-based current source and further introduce deep sleep for sustainable energy consumption. +These issues can be tackled by using fully digital flow meters that can share their reading via numerous digital data transfer methods. Infrared reading is common with radiators and electric meters, where the values can be accurately transmitted. But it cannot be done in real-time. Another thing that could be further looked into in the future would be to introduce a battery-based current source and moreover introduce deep sleep for sustainable energy consumption in the long deployment. Additionally, data retention technology like the introduction of data banks to save data from different meters could help widen the appeal of such installations in more devices.
  
-All in all, if the user is looking for a cheap solution that works with existing infrastructure and is able to maintain the accuracy and reliability of the readings, then this method will definitely be suitable.+All in all, if the user is looking for a cheap solution that works with existing infrastructure and is able to maintain the accuracy and reliability of the readings, then this method will definitely be suitable. It was undoubtedly difficult to realize this project in a limited amount of time and with insufficient technological expertise, but with enough devotion and guidance from the instructors, it was completed successfully. 
  
 =====8. References ===== =====8. References =====
amc/ss2023/group-d/start.1690272517.txt.gz · Last modified: 2023/07/25 10:08 by 29907_students.hsrw