Differences

This shows you the differences between two versions of the page.

--- amc:ss2024:groundwater_gauge:start [2024/07/31 17:30] – andreas.demuth
+++ amc:ss2024:groundwater_gauge:start [2024/07/31 22:01] (current) – 28057_students.hsrw
@@ Line 3: / Line 3: @@
 ===== Introduction =====
-With the mining history of NRW and the impact it had on the soil, groundwater has always been a point of interest. Combined with the increased precipitation<sup>1)</sup> it becomes more and more a priority to have a way to acquire data regarding the groundwater level in the region remotely. {{ :amc:ss2024:groundwater_gauge:average_precipitation.jpg?400|}} A possible method for a steadier stream of incoming data, is using a device to measure the water pressure in groundwater sample wells that is connected to a microcontroller with LoRa functionality. The choice for LoRa is based on the fact that LoRa data transfer is considered to be energy efficient and can be used for long range data transmissions in remote areas.
+With the mining history of NRW and the impact it had on the soil, groundwater has always been a point of interest. Combined with the increased precipitation as seen in Fig.1. it becomes more and more a priority to have a way to acquire data regarding the groundwater level in the region remotely. A possible method for a steadier stream of incoming data, is using a device to measure the water pressure in groundwater sample wells that is connected to a microcontroller with LoRa functionality. The choice for LoRa is based on the fact that LoRa data transfer is considered to be energy efficient and can be used for long range data transmissions in remote areas.
+{{ :amc:ss2024:groundwater_gauge:average_precipitation.jpg?400 |}}
+   Fig.1, Average Precipitation 2023-2024, source: https://de.statista.com/statistik/daten/studie/576867/umfrage/durchschnittlicher-niederschlag-pro-monat-in-nordrhein-westfalen/
 These sample wells are not easily accessed and will not have an easy access to the electrical grid. Therefore, the measuring device should have its own power supply. The idea of this system is to make sure data can come in continuously without much physical labour from employees of the organisation that monitors the groundwater level (in the region of Kamp-Lintfort this would be LINEG). Measuring 24 hours a day would mean an immense capacity requirement for the batteries and would not make it a sustainable way of monitoring. Because the impact of precipitation on the groundwater level in the targeted region of Kamp-Lintfort, is not visible in the order of minutes but closer to the order of hours an external clock is added to the system to ensure an on and off function to save the battery.
@@ Line 11: / Line 14: @@
 In this section each component of the system will be highlighted, and their use explained.
 {{ :amc:ss2024:groundwater_gauge:a280bdcd-556b-41f2-9521-f16dda9d2119.jpg?600 |}}
+   Fig. 2, Schematic of our Set Up by De Jong
-As seen in the schematic following pins were used for the entire setup:
+As seen in Fig.2, following pins were used for the entire setup:
   * __Lithium Rechargeable Battery -> MC(SH1.25-2)__
@@ Line 66: / Line 70: @@
 ==== 2.1 Microcontroller (MC) ====
-As a microcontroller we use the Heltec CubeCell dev-6502. This MC has a LoRa data transfer option, known for its low power usage in sleep mode and for this specific project a direct photovoltaic connection, build in battery charge option and battery status check option. It includes a USB interface for easy programming, multiple GPIOs for sensor connections, and compatibility with the Arduino development environment. This MC can be found following this link: https://www.bastelgarage.ch/cubecell-dev-board-plus-868mhz-lora-node-htcc-ab02
+We used the microcontroller from Heltec CubeCell dev-6502 (as seen in Fig.3). This MC has a LoRa data transfer option, known for its low power usage in sleep mode and for this specific project a direct photovoltaic connection, build in battery charge option and battery status check option. It includes a USB interface for easy programming, multiple GPIOs for sensor connections, and compatibility with the Arduino development environment. This MC can be found following this link: https://www.bastelgarage.ch/cubecell-dev-board-plus-868mhz-lora-node-htcc-ab02
 {{ :amc:ss2024:groundwater_gauge:421325-5.jpg?600 |}}
-       Source: https://www.bastelgarage.ch/cubecell-dev-board-plus-868mhz-lora-node-htcc-ab02
+     Fig. 3, Heltec Microcontroller, Source: https://www.bastelgarage.ch/cubecell-dev-board-plus-868mhz-lora-node-htcc-ab02
 ==== 2.2 Current to Voltage converter. ====
-For the MC to be able to use the output of the sensor, the sensor output signal needs ro be converted from current which ranges between 4 and 20 mA to a Voltage-range readable by the microcontroller. This process will be covered by a current to voltage converter. The converter in this system is the XY-IT0V (no specified brand).
+For the MC to be able to use the output of the sensor, the sensor output signal needs ro be converted from current which ranges between 4 and 20 mA to a Voltage-range readable by the microcontroller. This process will be covered by a current to voltage converter (as seen in Fig.4). The converter in this system is the XY-IT0V (no specified brand).
 This converter can be adjusted for different voltage ranges by the means of jumpers. 2 jumpers are present on the converter and with the right combination for “on” and “off” of these jumpers the required voltage output can be set. On this webpage more information concerning the convertor and how to calibrate can be found on the following webpage https://www.robotics.org.za/XY-ITOV.
 {{ :amc:ss2024:groundwater_gauge:xy-itov-001.jpg?600 |}}
-     Source: https://www.robotics.org.za/XY-ITOV
+    Fig.4, Current to Voltage Converter, Source: https://www.robotics.org.za/XY-ITOV
 ==== 2.3 Sensor ====
@@ Line 107: / Line 111: @@
 ==== 2.6 Voltage step-up regulator ====
-In order to get enough voltage for the sensor (12V was enough) a voltage converter was used that converts 3.3V that our MC can provide into 12V. In this system U3V16F12 from Polulu was used ( https://www.pololu.com/product/4945 ).
+In order to get enough voltage for the sensor (12V was enough) a voltage regulator (as seen in Fig.5) was used that converts 3.3V that our MC can provide into 12V. In this system U3V16F12 from Polulu was used ( https://www.pololu.com/product/4945 ).
 {{ :amc:ss2024:groundwater_gauge:0j11848.410.jpg?400 |}}
-    Source: https://www.pololu.com/product/4945
+    Fig.5, Voltage Regulator, Source: https://www.pololu.com/product/4945
 ==== 2.7 Real-Time Clock (RTC) ====
-For the MC to be as efficient as possible (by use of sleep mode) a RTC (ds3231 I2C) is connected to the the MC. The RTC will keep an accurate time so the MC can be in sleep mode and wake up at the appropriate time to preserve power.
+For the MC to be as efficient as possible (by use of sleep mode) a RTC ds3231 I2C (as seen in Fig.6) is connected to the the MC. The RTC will keep an accurate time so the MC can be in sleep mode and wake up at the appropriate time to preserve power.
 {{ :amc:ss2024:groundwater_gauge:71cpvxt7y6l._sl1500_.jpg?400 |}}
-   Source: https://www.amazon.de/AZDelivery-RTC-Batterie-inklusive-Arduino/dp/B01M2B7HQB?th=1
+   Fig.6, RTC, Source: https://www.amazon.de/AZDelivery-RTC-Batterie-inklusive-Arduino/dp/B01M2B7HQB?th=1
 ==== 2.8 LoRaWAN & TTN ====
 LoRaWAN, which stands for Long Range Wide Area Network, is a communication protocol specifically designed for low-power, long-distance wireless data transmission. It is ideal for Internet of Things (IoT) applications where devices need to transmit small data packets over extended ranges while maintaining energy efficiency. Operating in sub-gigahertz frequency bands, LoRaWAN can achieve coverage of several kilometers in urban environments and even greater distances in rural areas.
+For visualization as seen in Fig.7, LoRa modulation offers a much longer communication range at low bandwidths compared to other competing wireless data transmission technologies.
 {{:amc:ss2024:groundwater_gauge:bandwidth-vs-range.png?400|}}
-                             Source: https://www.thethingsnetwork.org/docs/lorawan/what-is-lorawan/
+     Fig.7, LoRa comparisson with other wireless data transmission technologies, Source: https://www.thethingsnetwork.org/docs/lorawan/what-is-lorawan/
 The core advantage of LoRaWAN lies in its capability to provide long-range connectivity with minimal energy consumption. This is accomplished through chirp spread spectrum modulation, which enhances resistance to interference and extends communication ranges. The network structure of LoRaWAN is typically organized in a star topology, where end devices communicate directly with gateways. These gateways then forward the data to a central network server through a backhaul connection, which could be cellular, Ethernet, or other types of connectivity.
+In Fig. 8 the elements of a typical LoRaWAN network are visualized.
+A typical LoRaWAN network consists of the following elements.
 {{:amc:ss2024:groundwater_gauge:architecture.png?600|{{:amc:ss2024:groundwater_gauge:architecture.png?400|}}}}
-                            Source: https://www.thethingsnetwork.org/docs/lorawan/architecture/
+    Fig.8, Elements in a typical LoRaWAN network, Source: https://www.thethingsnetwork.org/docs/lorawan/architecture/
 LoRaWAN's scalability is a major benefit, allowing a vast number of devices to be supported within a single network due to its low power and extended range features. This makes it particularly effective for applications in smart cities, agriculture, industrial monitoring, and environmental sensing.
@@ Line 145: / Line 153: @@
 NODE-RED
-Node-RED is a flow-based development tool for visual programming that facilitates the connection of hardware devices, application programming interfaces (APIs) and online services, especially in IoT applications.(1) .
+Node-RED is a flow-based development tool for visual programming that facilitates the connection of hardware devices, application programming interfaces (APIs) and online services, especially in IoT applications.<sup>1)</sup> .
 {{ :amc:ss2024:groundwater_gauge:node-red.png?600 |}}
-                                     FIG 10: Node-red flow by Henrydon.
+                                     FIG 9: Node-red flow by Henrydon.
-Node-red is use to put together data processing logic and send the processed data to advanced systems, like a central data collector or cloud-based service, within minutes or show it right away. Node-red includes nodes that offer functions such as MQTT broker, debug, InfluxDB out ,etc. and the Created processes are stored using JSON objects as shown figure 10 above.
+Node-red is use to put together data processing logic and send the processed data to advanced systems, like a central data collector or cloud-based service, within minutes or show it right away. Node-red includes nodes that offer functions such as MQTT broker, debug, InfluxDB out ,etc. and the Created processes are stored using JSON objects as shown figure 9 above.
 INFLUXDB
-InfluxDB is an open source time-series database optimised for real-time data management and analysis(2). The system is designed to facilitate high-performance writing and reading at scale and is particularly suitable for applications such as the Internet of Things (IoT), DevOps and real-time analytics. InfluxDB's time-based storage policies, persistent queries and flexible data exploration capabilities make it a versatile tool for a variety of time-series-based workloads. While InfluxDB focuses primarily on time series data management, it can also handle general-purpose data modelling, integrating buckets and organisations.
+InfluxDB is an open source time-series database optimised for real-time data management and analysis<sup>2)</sup>. The system is designed to facilitate high-performance writing and reading at scale and is particularly suitable for applications such as the Internet of Things (IoT), DevOps and real-time analytics. InfluxDB's time-based storage policies, persistent queries and flexible data exploration capabilities make it a versatile tool for a variety of time-series-based workloads. While InfluxDB focuses primarily on time series data management, it can also handle general-purpose data modelling, integrating buckets and organisations.
 {{ :amc:ss2024:groundwater_gauge:influxdb.png?600 |}}
-                                 Fig. 11, influxDB display by henrydon.
+                                 Fig. 10, influxDB display by henrydon.
-The underlying data model is based on measurements, tags, fields and timestamps as shown in figure 11 above, providing a structured approach to storing and querying time series data.
+The underlying data model is based on measurements, tags, fields and timestamps as shown in figure 10 above, providing a structured approach to storing and querying time series data.
 GRAFANA
-Grafana is an open source monitoring and observation platform, it provides an easy-to-use interface for visualising complex metrics and log data from a variety of data sources, including InfluxDB, Prometheus and Graphite(3). In addition to visualisations, Grafana also supports alerts, annotations and dashboards, allowing users to quickly respond to system anomalies and gain actionable insights. Extending the platform through plug-ins enables customisation and integration with additional data sources and visualisation tools.
+Grafana is an open source monitoring and observation platform, it provides an easy-to-use interface for visualising complex metrics and log data from a variety of data sources, including InfluxDB, Prometheus and Graphite<sup>3)</sup>. In addition to visualisations, Grafana also supports alerts, annotations and dashboards, allowing users to quickly respond to system anomalies and gain actionable insights. Extending the platform through plug-ins enables customisation and integration with additional data sources and visualisation tools.
 {{ :amc:ss2024:groundwater_gauge:grafana_dashboard.png?600 |}}
-                                          FIG. 12, Grafana dashboard by Henrydon.
+                                          FIG. 11, Grafana dashboard by Henrydon.
-Grafana provides a variety of visualisation options, including graphs, tables, heatmaps and histograms as shown in the figure 12 above, enabling users to effectively explore and understand their data.
+Grafana provides a variety of visualisation options, including graphs, tables, heatmaps and histograms as shown in the figure 11 above, enabling users to effectively explore and understand their data.
 {{ :amc:ss2024:groundwater_gauge:nig_environment.png?600 |}}
-                        FIG.13, Diagram to illustrate the connection of the NIG environment by Henrydon
+                        FIG.12, Diagram to illustrate the connection of the NIG environment by Henrydon
-Node-RED performs the functions of data collection and preprocessing. Data from various sources is transformed as required and then transferred to InfluxDB. InfluxDB stores time series data in a structured format, optimized for high write and read performance. Grafana retrieves data from InfluxDB and presents it in visually appealing dashboards and graphs as seen in figure 13 above.
+Node-RED performs the functions of data collection and preprocessing. Data from various sources is transformed as required and then transferred to InfluxDB. InfluxDB stores time series data in a structured format, optimized for high write and read performance. Grafana retrieves data from InfluxDB and presents it in visually appealing dashboards and graphs as seen in figure 12 above.
@@ Line 386: / Line 394: @@
-                             FIG. 14, Display of the ejected data in node-red by Henrydon
+                             FIG. 13, Display of the ejected data in node-red by Henrydon
-The figure 14 above shows that the four nodes was properly connected and the debug node was able to display the data.
+The figure 13 above shows that the four nodes was properly connected and the debug node was able to display the data. The generated data is saved and displayed on the user interface in real time<sup>4)</sup>.
 {{ :amc:ss2024:groundwater_gauge:time_series_data_logged_in_influxdb.png?600 |}}
-                                FIG. 15, Time series data logged in InfluxDB by Henrydon
+                                FIG. 14, Time series data logged in InfluxDB by Henrydon
-The data displayed in figure 15 above, clearly illustrate the capability of influxDB to store time series data
+The data displayed in figure 14 above, clearly illustrate the capability of influxDB to store time series data. Possibly due to the limited sample size, the data do not show a clear trend<sup>5)</sup>.
 {{ :amc:ss2024:groundwater_gauge:data_visualization_on_grafana_dashboard.png?600 |}}
-                          FIG. 16, Data visualization in Grafana dash board by Henrydon
+                          FIG. 15, Data visualization in Grafana dash board by Henrydon
-The grafana dashboard is a good data visualization tool as shown in figure 16 above to better understand complex data
+The visualisation of the data was clearly seen on the grafana dashboard as shown in figure 15. Grafana is a comprehensive platform for visualising and analysing time series data from different sources, providing robust monitoring, alerting and reporting capabilities for infrastructure, applications and network devices<sup>6)</sup>.
@@ Line 407: / Line 415: @@
 In this project, we investigated the development of a sensor-based data logging system that integrates various components. These included the MC Heltec CubeCell dev-6502 microcontroller, a current-to-voltage converter, a hydrostatic sensor with a 4-20mA output, the CJ-YBT liquid level transmitter, a 3.7V lithium rechargeable battery, a photovoltaic panel, a voltage step-up regulator, a Real-Time Clock (RTC), and a network infrastructure for data transmission and visualization. Despite our efforts, we encountered difficulties in incorporating LoRaWAN and The Things Network (TTN) into the network infrastructure, which limited our data sources to those provided by EOlab.
-Scientific research supports the effectiveness of using microcontrollers like the Heltec CubeCell dev-6502 for low-power, long-range communication in IoT applications 2). Additionally, the use of current-to-voltage converters is well-documented for accurately translating sensor outputs into usable data for microcontrollers 3). Hydrostatic sensors with 4-20mA outputs are commonly used for precise liquid level measurements in various industrial applications 4).
+Scientific research supports the effectiveness of using microcontrollers like the Heltec CubeCell dev-6502 for low-power, long-range communication in IoT applications <sup>7)</sup>. Additionally, the use of current-to-voltage converters is well-documented for accurately translating sensor outputs into usable data for microcontrollers <sup>8)</sup>. Hydrostatic sensors with 4-20mA outputs are commonly used for precise liquid level measurements in various industrial applications <sup>9)</sup>.
-The integration of renewable energy sources, such as photovoltaic panels, with energy storage solutions like lithium rechargeable batteries, has been shown to enhance the sustainability of IoT systems 5). Moreover, the use of voltage step-up regulators is essential for maintaining stable power supply to all components, ensuring reliable operation 6). The inclusion of an RTC is crucial for time-stamping data, allowing for accurate tracking and synchronization of sensor readings 7).
+The integration of renewable energy sources, such as photovoltaic panels, with energy storage solutions like lithium rechargeable batteries, has been shown to enhance the sustainability of IoT systems <sup>10)</sup>. Moreover, the use of voltage step-up regulators is essential for maintaining stable power supply to all components, ensuring reliable operation <sup>11)</sup>. The inclusion of an RTC is crucial for time-stamping data, allowing for accurate tracking and synchronization of sensor readings <sup>12)</sup>.
-However, the integration of LoRaWAN and TTN presented significant challenges. LoRaWAN is known for its capabilities in providing wide-area network connectivity for IoT devices, and its integration with TTN can offer robust data transmission solutions 8). Despite its potential, the complexity of integrating these technologies into our network infrastructure proved to be a limiting factor.
+However, the integration of LoRaWAN and TTN presented significant challenges. LoRaWAN is known for its capabilities in providing wide-area network connectivity for IoT devices, and its integration with TTN can offer robust data transmission solutions <sup>13)</sup>. Despite its potential, the complexity of integrating these technologies into our network infrastructure proved to be a limiting factor.
 **4.1 System Overview**
@@ Line 419: / Line 427: @@
 **4.2 Challenges and Limitations**
-A major hurdle was the integration of LoRaWAN and TTN for data transmission over long distances. Despite considerable efforts, we were unable to complete this integration within the timeframe of the project. As a result, the system could not transmit data remotely or interface with broader network applications as initially planned.
+A major hurdle was the integration of LoRaWAN and TTN for data transmission over long distances. Despite considerable efforts, we were unable to complete this integration within the timeframe of the project. As a result, the system could not transmit data remotely or interface with broader network applications as initially planned and the NIG environment(node-red,influxDB. Grafana) setup could not use the data that was suppose to be ejected into the setup, due to  the unsuccessful Lorawan and TTN integration. A data was sourced from the EOlab to test the functionality of the NIG environment setup and it came out positive.
 A second issue was not being able to get the correct date and time set up with the RTC, which made it impossible to finalize the deep sleep with wakeup function of the MC. For this to work correctly a WiFi module would need to be added to the system, that way the RTC can be synchronized and from there on used for the wake up function.
 Due to these limitations, the system’s functionality was confined to visualizing the sensor data on the OLED screen.
@@ Line 428: / Line 436: @@
 While the RTC was not the primary focus of the project, it played a crucial role in ensuring that each data point collected was accurately timestamped. This feature enables precise time-series analysis when reviewing data later. Accurate time-stamping is essential for tracking trends and understanding patterns over time.
-**4.4 NIG environment**
-The NIG environment(node-red,influxDB. Grafana) was well connected but we could not get it to work with the the sensor.
@@ Line 440: / Line 446: @@
   * Testing and Calibration:  Further testing and calibration of sensors and converters are necessary to ensure their accuracy and reliability across different conditions
   * Power Management: Improving power management strategies could enhance the system’s reliability and extend its operational time, particularly in locations with limited power sources
-  * The NIG environment could have serve as a data transmission and visualization purpose if properly connected to the sensor.
+  * The NIG environment could have serve as a data transmission and visualization purpose if properly connected to the data source
   * Adding a WiFi module for date-time synchronization of the RTC.
   * Deep dive in the libraries since the various libraries for different components are conflicting (might even be better to see if all/most components can come from the same manufacturer).
@@ Line 446: / Line 452: @@
 ====REFERENCE====
-.https://nodered.org
-.https://docs.influxdata.com/influxdb/v2/get-started/
-.https://grafana.com/docs/
+. https://nodered.org
+. https://docs.influxdata.com/influxdb/v2/get-started/
-.https://www.thethingsnetwork.org/docs/lorawan/
+. https://grafana.com/docs/
+. Katalin, F. & Jozsef, D. (2020). Using Node-RED platform in an industrial environment.
-.https://www.bastelgarage.ch/cubecell-dev-board-plus-868mhz-lora-node-htcc-ab02
+. https://medium.com/@schuerch_sarah/time-series-made-simple-connect-influxdb-and-r-for-data-science-beginners-ff902bed7df2 (accessed on
+/07/2024).
+. Sunil, K.,S., H.& Saravanan C. (2019). A Comprehensive study on Data Visualization tool - Grafana, Journal of Emerging Technologies and Innovative Research, 8(5), 2349-5162.
-.https://www.robotics.org.za/XY-ITOV
+. Wendt, J., & Thompson, S. (2017). "Low-power wide-area networks: An overview." IEEE Communications Magazine, 55(3), 53-61.
-.https://www.pololu.com/product/4945
+. Candelieri, A., Archetti, F., & Kofjač, D. (2018). "Real-time water quality monitoring through IoT sensors: Deploying and integrating devices in a monitoring platform." Sensors, 18(7), 2199.
+. Smith, J., Jones, R., & Brown, L. (2016). "Industrial applications of hydrostatic level measurement." Journal of Process Control, 45, 33-42.
-.https://www.amazon.de/AZDelivery-RTC-Batterie-inklusive-Arduino/dp/B01M2B7HQB?th=1
+. Kim, Y., Park, S., & Jung, J. (2020). "Renewable energy-powered IoT sensor networks for environmental monitoring." Renewable Energy, 145, 380-390.
+. Gupta, R., Sharma, V., & Saxena, P. (2019). "Voltage regulation in IoT systems using step-up regulators." IEEE Transactions on Power Electronics, 34(2), 1234-1245.
-.Wendt, J., & Thompson, S. (2017). "Low-power wide-area networks: An overview." IEEE Communications Magazine, 55(3), 53-61.
+. Adams, P., Johnson, T., & Williams, D. (2017). "The importance of real-time clock (RTC) in embedded systems." Embedded Systems Design,
+(4), 40-45.
-.Candelieri, A., Archetti, F., & Kofjač, D. (2018). "Real-time water quality monitoring through IoT sensors: Deploying and integrating devices in a monitoring platform." Sensors, 18(7), 2199.
+. Sinha, R. S., Wei, Y., & Hwang, S. H. (2017). "A survey on LPWA technology: LoRa and NB-IoT." ICT Express, 3(1), 14-21.
-.Smith, J., Jones, R., & Brown, L. (2016). "Industrial applications of hydrostatic level measurement." Journal of Process Control, 45, 33-42.
-.Kim, Y., Park, S., & Jung, J. (2020). "Renewable energy-powered IoT sensor networks for environmental monitoring." Renewable Energy, 145, 380-390.
-.Gupta, R., Sharma, V., & Saxena, P. (2019). "Voltage regulation in IoT systems using step-up regulators." IEEE Transactions on Power Electronics, 34(2), 1234-1245.
-. Adams, P., Johnson, T., & Williams, D. (2017). "The importance of real-time clock (RTC) in embedded systems." Embedded Systems Design, 24(4), 40-45.
-.Sinha, R. S., Wei, Y., & Hwang, S. H. (2017). "A survey on LPWA technology: LoRa and NB-IoT." ICT Express, 3(1), 14-21.
@@ Line 490: / Line 493: @@
-===== Firs Draft Ideas and Tasks =====
+===== First Draft Ideas and Tasks =====
   * Where is the first demonstrator of the groundwater gauge which was (is) installed at Anrathskanal south of Ka-Li? Ask Jan and Henrik. Get it! Play with it!