HSRW EOLab Students Wiki

User Tools

Site Tools

Trace:
emrp:ws2025:amt

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
emrp:ws2025:amt [2026/02/26 06:01] – 36502_students.hsrwemrp:ws2025:amt [2026/02/26 21:31] (current) – 37554_students.hsrw
Line 1: Line 1:
 ====== Animal Movement Tracker ====== ====== Animal Movement Tracker ======
 +**Authors:** Tarik Esen (36502), Muhammed Ali Baskal (37554)
  
 ===== 1. Introduction & Problem Statement ===== ===== 1. Introduction & Problem Statement =====
Line 42: Line 43:
 |**//Figure 3//** Interfacing of the MCU and MPU6050| |**//Figure 3//** Interfacing of the MCU and MPU6050|
  
-To facilitate easier normalisation for IMU measurements, the accelerometer configuration was selected as +/-2g, and the gyroscope configuration as +/-250dps. Each measurement was provided using windows containing a total of 96 samples, where each sample contains 6 values (ax,ay,az,gx,gy,gz). To facilitate communication synchronisation, the IMU sampling frequency was set to 1kHz and the SMPLRT_DIV value to 7 so that each window would be collected within a 1-second time interval, resulting in a sampling frequency of 125Hz. This yielded an IMU window period of approximately 0.8 seconds.+To facilitate easier normalisation for IMU measurements, the accelerometer configuration was selected as +/-2g, and the gyroscope configuration as +/-250dps. Each measurement was provided using windows containing a total of 96 samples, where each sample contains 6 values (ax,ay,az,gx,gy,gz). To facilitate communication synchronisation, the IMU sampling frequency was set to 1kHz and the SMPLRT_DIV value to 7 so that each window would be collected within a 1-second time interval, resulting in a sampling frequency of 125Hz on the controller side. This yielded an IMU window period of approximately 0.8 seconds.
  
 === 2.2.3 Power Management === === 2.2.3 Power Management ===
Line 123: Line 124:
 |**//Figure 11//**  Schematic Indicating the Connections of All Modules in the Gateway Unit| |**//Figure 11//**  Schematic Indicating the Connections of All Modules in the Gateway Unit|
  
-=== 2.4.2 Gateway System (RPi) ===+=== 2.4.2 Gesture Model ===
 The motion classification model has a three-class structure: The motion classification model has a three-class structure:
   * **Move:** The animal's motion along the x, y, z axes with a specific acceleration   * **Move:** The animal's motion along the x, y, z axes with a specific acceleration
Line 355: Line 356:
 main() main()
  
-if _name_ == "_main_":+if _name_ == "__main__":
     main()     main()
 </file> </file>
Line 449: Line 450:
 </file> </file>
  
-The controller then collected a fixed-length IMU data window consisting of a total of 96 samples at a sampling frequency of 100 Hz. Each sample contained a total of six 16-bit raw measurements: three-axis acceleration and three-axis gyroscope data. Thus, a single window produced 96 × 6 × 2 bytes of raw data.+The controller then collected a fixed-length IMU data window consisting of a total of 96 samples at a sampling frequency of 100 Hz on the gateway side (a bit lower considering the transmission latency) Each sample contained a total of six 16-bit raw measurements: three-axis acceleration and three-axis gyroscope data. Thus, a single window produced 96 × 6 × 2 bytes of raw data.
  
 The collected window data was not sent in a single piece but was divided into fixed-length frames, taking into account the maximum packet size of the LoRa module. Each frame consisted of a header and a payload section. The header section contained address information, channel information, the system-defined "MAGIC" constant value, and a frame counter field. The window data was divided into multiple frames, and each frame was transmitted sequentially with the frame counter field. By this flow, full IMU logic is code bases the measurement and windowing logic is given below. The collected window data was not sent in a single piece but was divided into fixed-length frames, taking into account the maximum packet size of the LoRa module. Each frame consisted of a header and a payload section. The header section contained address information, channel information, the system-defined "MAGIC" constant value, and a frame counter field. The window data was divided into multiple frames, and each frame was transmitted sequentially with the frame counter field. By this flow, full IMU logic is code bases the measurement and windowing logic is given below.
Line 617: Line 618:
     uint8_t who = 0;     uint8_t who = 0;
     if (mpu_read(hi2c, REG_WHO_AM_I, &who, 1) != HAL_OK) return -1;     if (mpu_read(hi2c, REG_WHO_AM_I, &who, 1) != HAL_OK) return -1;
-    if (who != 0x68) return -2;  // WHO_AM_I beklenen+    if (who != 0x68) return -2;
  
     // Wake up (disable sleep bit)     // Wake up (disable sleep bit)
Line 624: Line 625:
  
     // Sample rate = Gyro output / (1 + SMPLRT_DIV). Gyro output default 8kHz (DLPF=0) or 1kHz (DLPF!=0)     // Sample rate = Gyro output / (1 + SMPLRT_DIV). Gyro output default 8kHz (DLPF=0) or 1kHz (DLPF!=0)
-    mpu_write(hi2c, REG_SMPLRT_DIV, 0x07);   // örnek: 1kHz/(1+7)=125Hz (DLPF ON is assumed)+    mpu_write(hi2c, REG_SMPLRT_DIV, 0x07);   // 1kHz/(1+7)=125Hz (DLPF ON is assumed)
     mpu_write(hi2c, REG_CONFIG, 0x03);       // DLPF ~44Hz (typical)     mpu_write(hi2c, REG_CONFIG, 0x03);       // DLPF ~44Hz (typical)
     mpu_write(hi2c, REG_GYRO_CONFIG, 0x00);  // ±250 dps     mpu_write(hi2c, REG_GYRO_CONFIG, 0x00);  // ±250 dps
Line 1266: Line 1267:
         raise ValueError(f"Incomplete window: got {samples_written} samples, expected {A}")         raise ValueError(f"Incomplete window: got {samples_written} samples, expected {A}")
  
-    # Return the window safter the transfer is successful+    # Return the window after the transfer is successful
     return out     return out
 </file> </file>
Line 1311: Line 1312:
 ==== 3.3 Gesture Model ==== ==== 3.3 Gesture Model ====
 === 3.3.1 Dataset Creation === === 3.3.1 Dataset Creation ===
-The dataset was collected directly on the Raspberry Pi via the IMU (MPU6050) for training the motion classification model. During the data collection process, a total of 6 channels were read from the sensor: 3-axis acceleration (AX, AY, AZ) and 3-axis gyroscope (GX, GY, GZ). The collection process was designed based on windows, and each window consisted of A=96 consecutive samples. The sampling frequency Fs=100 Hz was selected, so each window represented a time interval of approximately 0.96 s. To increase temporal continuity and enhance data diversity, inter-window stride was applied, and S=48 with 50% overlapping windows was implemented. This structure ensured better capture of short-term motion transitions and increased boundary examples between classes.+The dataset was collected directly on the Raspberry Pi via the IMU (MPU6050) for training the motion classification model. During the data collection process, a total of 6 channels were read from the sensor: 3-axis acceleration (AX, AY, AZ) and 3-axis gyroscope (GX, GY, GZ). The collection process was designed based on windows, and each window consisted of A=96 consecutive samples. The sampling frequency Fs=100 Hz was selected for the gateway, so each window represented a time interval of approximately 0.96 s. To increase temporal continuity and enhance data diversity, inter-window stride was applied, and S=48 with 50% overlapping windows was implemented. This structure ensured better capture of short-term motion transitions and increased boundary examples between classes.
  
 <file python record.py> <file python record.py>
Line 1754: Line 1755:
     print(f"Shake / Move: {ratio('Shake','Move'):.2f}x")     print(f"Shake / Move: {ratio('Shake','Move'):.2f}x")
  
-if _name_ == "_main_":+if _name_ == "__main__":
     main()     main()
 </file> </file>
Line 1881: Line 1882:
  
 **Model v2 Architecture** **Model v2 Architecture**
-The v2 architecture is an updated 1D-CNN designed for three-class classification and TFLite deployment on Raspberry Pi. Figure 21 shows the full architecture.+The v2 architecture is an updated 1D-CNN designed for three-class classification and TFLite deployment on Raspberry Pi One-dimensional CNNs have been shown to be effective for classifying temporal sensor signals with limited training data [R3]. Figure 21 shows the full architecture.
  
 {{ :emrp:ws2025:model_v2_pipeline.png?direct&600 |}} {{ :emrp:ws2025:model_v2_pipeline.png?direct&600 |}}
Line 1906: Line 1907:
  
 **Experiment 1 — Batch Normalization** **Experiment 1 — Batch Normalization**
-Batch Normalization (BN) layers were added immediately after each of the three Conv1D layers. BN normalises the activations of each mini-batch during training, which stabilises gradient flow and reduces sensitivity to the initial learning rate. The change was three additional lines in the model definition.+Batch Normalization (BN) layers were added immediately after each of the three Conv1D layers. Batch normalization stabilizes the distribution of layer activations during training and can reduce the need for Dropout regularization [R4]. In our experiments, adding BatchNorm halved convergence time from ~80 to ~37 epochs.
  
 **Result:** EarlyStopping triggered at epoch ~37 compared to ~80 for the baseline — convergence speed was more than halved. Test accuracy remained unchanged at 95.7%. Validation loss showed slightly more stable behaviour. The significant reduction in training time with no accuracy cost made this an easy decision. **Result:** EarlyStopping triggered at epoch ~37 compared to ~80 for the baseline — convergence speed was more than halved. Test accuracy remained unchanged at 95.7%. Validation loss showed slightly more stable behaviour. The significant reduction in training time with no accuracy cost made this an easy decision.
Line 1914: Line 1915:
 A learning rate callback was added: when validation loss does not improve for 10 consecutive epochs, the learning rate is multiplied by 0.5 (minimum: 1×10⁻⁶). This allows the model to take large gradient steps early in training and progressively finer steps as it approaches a local optimum. A learning rate callback was added: when validation loss does not improve for 10 consecutive epochs, the learning rate is multiplied by 0.5 (minimum: 1×10⁻⁶). This allows the model to take large gradient steps early in training and progressively finer steps as it approaches a local optimum.
  
-**Result:** Test accuracy improved from 95.7% to 96.6% (113/117 correct). Shake recall improved from 41/46 to 42/46. Validation loss reached a lower minimum and remained more stable than in Experiment 1. EarlyStopping triggered at ~40 epochs. The improvement was measurable and consistent.+**Result:** Test accuracy improved from 95.7% to 96.6% (113/117 correct). Shake recall improved from 41/46 to 42/46. Validation loss reached a lower minimum and remained more stable than in Experiment 1. EarlyStopping triggered at ~40 epochs. The improvement was measurable and consistent.This configuration was further improved by augmentation in the final step.
 **Decision:** Keep. **Decision:** Keep.
  
Line 1920: Line 1921:
 To further increase the effective training set size, each training window was duplicated with a perturbed copy: Gaussian noise (σ=0.05 in normalised space) was added, and a random amplitude scale factor drawn from U(0.9, 1.1) was applied. The training set grew from 348 to 696 windows. The validation and test sets were not augmented. To further increase the effective training set size, each training window was duplicated with a perturbed copy: Gaussian noise (σ=0.05 in normalised space) was added, and a random amplitude scale factor drawn from U(0.9, 1.1) was applied. The training set grew from 348 to 696 windows. The validation and test sets were not augmented.
  
-**Result:** Validation accuracy rose to approximately 97%, and validation loss remained stable without any upward drift. However, test accuracy decreased slightly to 94.9% (111/117), with Shake errors increasing from 4 to 6. This apparent contradiction between higher validation and lower test accuracy is due to the small test set: a change of 2 windows equals 1.7 percentage points. The result is statistically inconclusive at this data scale. Augmentation is expected to provide clearer benefits with a larger dataset. +**Result:** Test accuracy of 95.7% (112/117) was achieved, with Shake errors at 5. Validation accuracy remained stable throughout training without upward drift, confirming that augmentation did not cause overfitting at this scale. Given the statistically small test set, the difference relative to Experiment 2 is within noise, but augmentation is retained as it improves training set diversity and helps generalisation. 
-**Decision:** Not included in final model.+**Decision:** Keep. Included in the final model alongside BatchNorm and ReduceLROnPlateau.
  
 Figure 23 summarises all three experiments across the three key metrics: test accuracy, epochs to convergence, and Shake misclassification count. Figure 23 summarises all three experiments across the three key metrics: test accuracy, epochs to convergence, and Shake misclassification count.
  
 {{ :emrp:ws2025:experiment_comparison.png?direct&600 |}} {{ :emrp:ws2025:experiment_comparison.png?direct&600 |}}
-|**//Figure 23//**  Experiment comparison across v2 configurations. Left: test accuracy. Centre: training epochs. Right: Shake→Move misclassification count. The v2 + BatchNorm + ReduceLROnPlateau configuration (★) achieves the best balance.|+| **//Figure 23//**  Experiment comparison across v2 configurations. Left: test accuracy. Centre: training epochs. Right: Shake→Move misclassification count. The v2 + BatchNorm + ReduceLROnPlateau + Augmentation configuration (★) achieves the best balance.  |
  
 **Final Model Selection** **Final Model Selection**
-The final model is the v2 + BatchNorm + ReduceLROnPlateau configuration. It achieves 96.6% test accuracy with efficient convergence (~40 epochs) and stable training dynamics. Figure 24, 25, and 26 show the training curves, confusion matrix, and class probability outputs for this configuration.+The final model is the v2 + BatchNorm + ReduceLROnPlateau + Augmentation configuration. It achieves 95.7% test accuracy (112/117 correct) with efficient convergence (~40 epochs) and stable training dynamics.
  
 {{ :emrp:ws2025:final_model_curves.png?direct&600 |}} {{ :emrp:ws2025:final_model_curves.png?direct&600 |}}
Line 1946: Line 1947:
  
 <file python train.py> <file python train.py>
 +"""
 +Training script for a tiny 1D-CNN on (96 x 6) IMU windows.
 +Dataset format: pre-windowed CSVs with columns label, window_index, x0_ax ... x95_gz
 +
 +- Classes      : Move, Rest, Shake  (3-class)
 +- Window size  : 96 samples @ 100 Hz  (~0.96 s)
 +- Features     : aX, aY, aZ, gX, gY, gZ  (6 channels, raw int16)
 +- Preprocess   : per-channel z-score (fit on TRAIN only) + clip [-5, 5]
 +- Model        : Conv1D(16,k=5) -> Conv1D(16,k=3) -> MaxPool -> Conv1D(24,k=3)
 +                 -> GAP -> Dense(24) -> Dropout(0.30) -> Softmax(3)
 +                 + GaussianNoise(0.02) at input (training-time only)
 +- EarlyStopping: monitor val_loss, patience=25, restore_best_weights
 +- Export       : gesture_model.tflite (float32) + normalization.json
 +"""
 +
 +import os
 +import json
 +import numpy as np
 +import pandas as pd
 +import matplotlib.pyplot as plt
 +import tensorflow as tf
 +from tensorflow.keras import layers, models
 +
 +try:
 +    from sklearn.metrics import confusion_matrix, ConfusionMatrixDisplay
 +    SKLEARN_OK = True
 +except ImportError:
 +    SKLEARN_OK = False
 +
 +print(f"TensorFlow version = {tf.__version__}\n")
 +
 +# ─────────────────────────────────────────────────────────────────────────────
 +# 0) Reproducibility & constants
 +# ─────────────────────────────────────────────────────────────────────────────
 +SEED = 1337
 +np.random.seed(SEED)
 +tf.random.set_seed(SEED)
 +
 +GESTURES           = ["Move", "Rest", "Shake"]   # must match CSV label values exactly
 +NUM_GESTURES       = len(GESTURES)
 +SAMPLES_PER_GESTURE = 96
 +FEATS              = 6                            # aX, aY, aZ, gX, gY, gZ
 +
 +# ── Paths ────────────────────────────────────────────────────────────────────
 +# Put your three clean CSV files in DATA_DIR, or adjust paths below.
 +DATA_DIR = "../server/model/dataset"         # folder containing Move_clean.csv etc.
 +OUT_DIR  = "model_output"
 +os.makedirs(OUT_DIR, exist_ok=True)
 +
 +DATASET_FILES = {
 +    "Move":  os.path.join(DATA_DIR, "Move_clean.csv"),
 +    "Rest":  os.path.join(DATA_DIR, "Rest_clean.csv"),
 +    "Shake": os.path.join(DATA_DIR, "Shake_clean.csv"),
 +}
 +
 +# ─────────────────────────────────────────────────────────────────────────────
 +# 1) Load pre-windowed CSVs
 +#    Each row = one window.  Columns: label, window_index, x0_ax ... x95_gz
 +#    We reshape each row into (96, 6) = [aX, aY, aZ, gX, gY, gZ] per time step.
 +# ─────────────────────────────────────────────────────────────────────────────
 +CHANNEL_SUFFIXES = ["ax", "ay", "az", "gx", "gy", "gz"]
 +
 +ONE_HOT  = np.eye(NUM_GESTURES, dtype=np.float32)
 +inputs_list, outputs_list = [], []
 +
 +for g_idx, gesture in enumerate(GESTURES):
 +    path = DATASET_FILES[gesture]
 +    df   = pd.read_csv(path)
 +
 +    # Build ordered feature columns: x0_ax, x0_ay, ..., x95_gz
 +    feat_cols = []
 +    for t in range(SAMPLES_PER_GESTURE):
 +        for ch in CHANNEL_SUFFIXES:
 +            feat_cols.append(f"x{t}_{ch}")
 +
 +    windows = df[feat_cols].values.astype(np.float32)   # (N, 576)
 +    labels  = np.tile(ONE_HOT[g_idx], (len(windows), 1))
 +
 +    inputs_list.append(windows)
 +    outputs_list.append(labels)
 +    print(f"  Loaded '{gesture}': {len(windows)} windows  (file: {path})")
 +
 +inputs  = np.concatenate(inputs_list,  axis=0)   # (N_total, 576)
 +outputs = np.concatenate(outputs_list, axis=0)   # (N_total, 3)
 +print(f"\nTotal windows: {len(inputs)} | Input shape: {inputs.shape}")
 +
 +# ─────────────────────────────────────────────────────────────────────────────
 +# 2) Shuffle + split  60 / 20 / 20
 +# ─────────────────────────────────────────────────────────────────────────────
 +idx = np.random.permutation(len(inputs))
 +inputs  = inputs[idx]
 +outputs = outputs[idx]
 +
 +n       = len(inputs)
 +n_train = int(0.60 * n)
 +n_val   = int(0.20 * n)
 +
 +X_flat_train = inputs[:n_train]
 +X_flat_val   = inputs[n_train : n_train + n_val]
 +X_flat_test  = inputs[n_train + n_val:]
 +
 +y_train = outputs[:n_train]
 +y_val   = outputs[n_train : n_train + n_val]
 +y_test  = outputs[n_train + n_val:]
 +
 +print(f"Train / Val / Test = {len(X_flat_train)} / {len(X_flat_val)} / {len(X_flat_test)}")
 +
 +# ─────────────────────────────────────────────────────────────────────────────
 +# 3) Per-channel z-score normalisation (fit on TRAIN only)
 +#    Shape trick: flatten -> (N, 96, 6) -> compute mean/std per channel axis
 +# ─────────────────────────────────────────────────────────────────────────────
 +train_ts = X_flat_train.reshape(-1, SAMPLES_PER_GESTURE, FEATS)  # (N, 96, 6)
 +ch_mean  = train_ts.mean(axis=(0, 1), keepdims=True)             # (1, 1, 6)
 +ch_std   = train_ts.std( axis=(0, 1), keepdims=True) + 1e-8      # (1, 1, 6)
 +
 +def zscore(x_flat: np.ndarray) -> np.ndarray:
 +    """Reshape to (N, 96, 6), apply per-channel z-score, clip, return (N, 96, 6)."""
 +    x_ts = x_flat.reshape(-1, SAMPLES_PER_GESTURE, FEATS)
 +    x_ts = (x_ts - ch_mean) / ch_std
 +    x_ts = np.clip(x_ts, -5.0, 5.0)
 +    return x_ts
 +
 +X_train = zscore(X_flat_train)   # (N, 96, 6)
 +X_val   = zscore(X_flat_val)
 +X_test  = zscore(X_flat_test)
 +
 +print(f"X_train: {X_train.shape} | X_val: {X_val.shape} | X_test: {X_test.shape}")
 +rng = np.random.default_rng(SEED)
 +
 +noise  = rng.normal(0, 0.05, X_train.shape).astype(np.float32)
 +scale  = rng.uniform(0.9, 1.1, (len(X_train), 1, 1)).astype(np.float32)
 +X_aug  = np.clip(X_train * scale + noise, -5.0, 5.0)
 +y_aug  = y_train.copy()
 +
 +X_train = np.concatenate([X_train, X_aug], axis=0)
 +y_train = np.concatenate([y_train, y_aug], axis=0)
 +
 +
 +shuffle_idx = rng.permutation(len(X_train))
 +X_train = X_train[shuffle_idx]
 +y_train = y_train[shuffle_idx]
 +
 +print(f"After augmentation - Train: {len(X_train)} windows (2x)")
 +# ─────────────────────────────────────────────────────────────────────────────
 +# 4) Model definition  — tiny 1D-CNN
 +# ─────────────────────────────────────────────────────────────────────────────
 +def build_model(input_shape=(SAMPLES_PER_GESTURE, FEATS), num_classes=NUM_GESTURES):
 +    inp = layers.Input(shape=input_shape, name="imu_96x6")
 +
 +    # GaussianNoise: active only during training, disabled at inference
 +    x = layers.GaussianNoise(0.02)(inp)
 +
 +    x = layers.Conv1D(16, 5, padding="same", activation="relu")(x)
 +    x = layers.BatchNormalization()(x)   
 +
 +    x = layers.Conv1D(16, 3, padding="same", activation="relu")(x)
 +    x = layers.BatchNormalization()(x)   
 +
 +    x = layers.MaxPooling1D(2)(x)
 +
 +    x = layers.Conv1D(24, 3, padding="same", activation="relu")(x)
 +    x = layers.BatchNormalization()(x)   
 +    x = layers.GlobalAveragePooling1D()(x)
 +    x = layers.Dense(24, activation="relu")(x)
 +    x = layers.Dropout(0.30)(x)                  # increased from 0.10 -> 0.30 (small dataset)
 +    out = layers.Dense(num_classes, activation="softmax")(x)
 +
 +    model = models.Model(inp, out, name="cnn_imu_3class")
 +    model.compile(
 +        optimizer=tf.keras.optimizers.Adam(1e-3),
 +        loss="categorical_crossentropy",
 +        metrics=["accuracy"]
 +    )
 +    return model
 +
 +model = build_model()
 +model.summary()
 +
 +# ─────────────────────────────────────────────────────────────────────────────
 +# 5) Training
 +# ─────────────────────────────────────────────────────────────────────────────
 +early = tf.keras.callbacks.EarlyStopping(
 +    monitor="val_loss",
 +    patience=25,
 +    restore_best_weights=True
 +)
 +
 +# ReduceLROnPlateau
 +reduce_lr = tf.keras.callbacks.ReduceLROnPlateau(
 +    monitor="val_loss",
 +    factor=0.5,
 +    patience=10,
 +    min_lr=1e-6,
 +    verbose=1
 +)
 +
 +history = model.fit(
 +    X_train, y_train,
 +    validation_data=(X_val, y_val),
 +    epochs=200,
 +    batch_size=8,
 +    callbacks=[early, reduce_lr],   
 +    verbose=2
 +)
 +
 +# ─────────────────────────────────────────────────────────────────────────────
 +# 6) Learning curves
 +# ─────────────────────────────────────────────────────────────────────────────
 +fig, axes = plt.subplots(1, 2, figsize=(14, 5))
 +
 +axes[0].plot(history.history["loss"],     "g.-", label="Train loss")
 +axes[0].plot(history.history["val_loss"], "b.-", label="Val loss")
 +axes[0].set_title("Loss"); axes[0].set_xlabel("Epoch")
 +axes[0].grid(True); axes[0].legend()
 +
 +axes[1].plot(history.history["accuracy"],     "g.-", label="Train acc")
 +axes[1].plot(history.history["val_accuracy"], "b.-", label="Val acc")
 +axes[1].set_title("Accuracy"); axes[1].set_xlabel("Epoch")
 +axes[1].grid(True); axes[1].legend()
 +
 +plt.tight_layout()
 +plt.savefig(os.path.join(OUT_DIR, "training_curves.png"), dpi=150)
 +plt.show()
 +
 +# ─────────────────────────────────────────────────────────────────────────────
 +# 7) Test evaluation + confusion matrix
 +# ─────────────────────────────────────────────────────────────────────────────
 +test_loss, test_acc = model.evaluate(X_test, y_test, verbose=0)
 +best_val_acc  = float(np.max(history.history.get("val_accuracy", [np.nan])))
 +best_val_loss = float(np.min(history.history.get("val_loss",     [np.nan])))
 +
 +print(f"\nBest val_acc  = {best_val_acc:.4f} | Best val_loss = {best_val_loss:.4f}")
 +print(f"Test accuracy = {test_acc:.4f}     | Test loss     = {test_loss:.4f}")
 +
 +y_prob = model.predict(X_test, verbose=0)
 +y_pred = y_prob.argmax(axis=1)
 +y_true = y_test.argmax(axis=1)
 +
 +if SKLEARN_OK:
 +    cm   = confusion_matrix(y_true, y_pred, labels=list(range(NUM_GESTURES)))
 +    disp = ConfusionMatrixDisplay(confusion_matrix=cm, display_labels=GESTURES)
 +    disp.plot(cmap="Blues", values_format="d")
 +    plt.title("Confusion Matrix (Test Set)")
 +    plt.savefig(os.path.join(OUT_DIR, "confusion_matrix.png"), dpi=150)
 +    plt.show()
 +    print("\nConfusion matrix:")
 +    print(cm)
 +else:
 +    print("scikit-learn not found — skipping confusion matrix.")
 +
 +# Class probabilities over test windows (sanity check)
 +plt.figure(figsize=(14, 4))
 +for c_idx, cname in enumerate(GESTURES):
 +    plt.plot(y_prob[:, c_idx], ".-", label=f"P({cname})")
 +plt.title("Class probabilities — test windows")
 +plt.xlabel("Window index"); plt.ylabel("Probability")
 +plt.grid(True); plt.legend(); plt.show()
 +
 +# ─────────────────────────────────────────────────────────────────────────────
 +# 8) Export: TFLite (float32) + normalization.json
 +# ─────────────────────────────────────────────────────────────────────────────
 +# --- TFLite ---
 +converter    = tf.lite.TFLiteConverter.from_keras_model(model)
 +tflite_bytes = converter.convert()
 +
 +tflite_path = os.path.join(OUT_DIR, "gesture_model.tflite")
 +with open(tflite_path, "wb") as f:
 +    f.write(tflite_bytes)
 +print(f"\nTFLite saved: {tflite_path}  ({os.path.getsize(tflite_path):,} bytes)")
 +
 +# --- normalization.json ---
 +# The model server MUST apply the same per-channel z-score before inference.
 +# Load this file on the RPi and apply:
 +#   x_norm = (x_raw - channel_mean) / channel_std  then clip to [-5, 5]
 +norm = {
 +    "mode":         "channelwise_zscore",
 +    "channels":     CHANNEL_SUFFIXES,          # ["ax","ay","az","gx","gy","gz"]
 +    "channel_mean": ch_mean.reshape(-1).tolist(),   # 6 values
 +    "channel_std":  ch_std.reshape(-1).tolist(),    # 6 values
 +    "clip":         [-5.0, 5.0],
 +    "window_size":  SAMPLES_PER_GESTURE,
 +    "sample_rate_hz": 100,
 +    "label_map":    {str(i): g for i, g in enumerate(GESTURES)}
 +}
 +
 +norm_path = os.path.join(OUT_DIR, "normalization.json")
 +with open(norm_path, "w") as f:
 +    json.dump(norm, f, indent=2)
 +print(f"Normalization saved: {norm_path}")
 +
 +# ─────────────────────────────────────────────────────────────────────────────
 +# 9) Quick inference sanity check
 +#    Verifies the exported TFLite model gives identical outputs to Keras.
 +# ─────────────────────────────────────────────────────────────────────────────
 +interp = tf.lite.Interpreter(model_path=tflite_path)
 +interp.allocate_tensors()
 +inp_det  = interp.get_input_details()[0]
 +out_det  = interp.get_output_details()[0]
 +
 +sample   = X_test[0:1].astype(np.float32)
 +keras_out = model.predict(sample, verbose=0)
 +
 +interp.set_tensor(inp_det["index"], sample)
 +interp.invoke()
 +tflite_out = interp.get_tensor(out_det["index"])
  
 +print(f"\nSanity check (window 0):")
 +print(f"  Keras  : {keras_out[0]}")
 +print(f"  TFLite : {tflite_out[0]}")
 +print(f"  Max diff: {np.abs(keras_out - tflite_out).max():.2e}")
 +print("\nTraining complete.")
 </file> </file>
  
Line 1967: Line 2278:
 <file python server.py> <file python server.py>
 #!/usr/bin/env python3 #!/usr/bin/env python3
 +import sys
 +import struct
 import time import time
 import json import json
Line 2307: Line 2620:
     #Initialize LoRa control ports (M0-M1, AUX)     #Initialize LoRa control ports (M0-M1, AUX)
     GPIO.setmode(GPIO.BOARD)     GPIO.setmode(GPIO.BOARD)
- GPIO.setwarnings(False) +    GPIO.setwarnings(False) 
- GPIO.setup(PIN_LORA_M0, GPIO.OUT) +    GPIO.setup(PIN_LORA_M0, GPIO.OUT) 
- GPIO.setup(PIN_LORA_M1, GPIO.OUT) +    GPIO.setup(PIN_LORA_M1, GPIO.OUT) 
- GPIO.setup(PIN_LORA_AUX, GPIO.IN, pull_up_down = GPIO.PUD_UP) +    GPIO.setup(PIN_LORA_AUX, GPIO.IN, pull_up_down = GPIO.PUD_UP)
   
- #Wait until AUX is high (Module is ready) +    #Wait until AUX is high (Module is ready) 
- if not wait_until_pin(PIN_LORA_AUX, GPIO.HIGH, 1.0): +    if not wait_until_pin(PIN_LORA_AUX, GPIO.HIGH, 1.0): 
- raise_error("LoRa module is busy!", -1, cleanup()) +        raise_error("LoRa module is busy!", -1, cleanup()) 
- else: +    else: 
- print("LoRa module is available")+        print("LoRa module is available")
   
- #Set module mode (11) sleep +    #Set module mode (11) sleep 
- GPIO.output(PIN_LORA_M0, GPIO.HIGH) +    GPIO.output(PIN_LORA_M0, GPIO.HIGH) 
- GPIO.output(PIN_LORA_M1, GPIO.HIGH) +    GPIO.output(PIN_LORA_M1, GPIO.HIGH) 
- if not wait_until_pin(PIN_LORA_AUX, GPIO.HIGH, 1.0): +    if not wait_until_pin(PIN_LORA_AUX, GPIO.HIGH, 1.0): 
- raise_error("Cannot set parameters", -1, cleanup()) +        raise_error("Cannot set parameters", -1, cleanup()) 
- else: +    else: 
- print("Parameters are set")+        print("Parameters are set")
   
- #Send lora configuration parameters +    #Send lora configuration parameters 
- uart.reset_input_buffer() +    uart.reset_input_buffer() 
- print("TX:", lora_params.hex(" ")) +    print("TX:", lora_params.hex(" ")) 
- uart.write(lora_params) +    uart.write(lora_params) 
- uart.flush() +    uart.flush() 
- time.sleep(0.1)+    time.sleep(0.1)
   
- #Validate lora configuration parameters +    #Validate lora configuration parameters 
- uart.reset_input_buffer() +    uart.reset_input_buffer() 
- print("TX:", cmd_params.hex(" ")) +    print("TX:", cmd_params.hex(" ")) 
- uart.write(cmd_params) +    uart.write(cmd_params) 
- uart.flush() +    uart.flush() 
- rx = uart.read(6) +    rx = uart.read(6) 
- if rx != lora_params: +    if rx != lora_params: 
- raise_error("Unable to validate configuration parameters", -1, cleanup()) +        raise_error("Unable to validate configuration parameters", -1, cleanup()) 
- else:  +    else:  
- print("Config parameters:", rx.hex(" ")) +        print("Config parameters:", rx.hex(" ")) 
- time.sleep(0.1)+        time.sleep(0.1)
   
- #Set module mode (00) transceiver +    #Set module mode (00) transceiver 
- if not wait_until_pin(PIN_LORA_AUX, GPIO.HIGH, 1.0): +    if not wait_until_pin(PIN_LORA_AUX, GPIO.HIGH, 1.0): 
- raise_error("LoRa module is busy!", -1, cleanup()) +        raise_error("LoRa module is busy!", -1, cleanup()) 
- else: +    else: 
- print("LoRa module is available to transmit") +        print("LoRa module is available to transmit") 
- GPIO.output(PIN_LORA_M0, GPIO.LOW) +        GPIO.output(PIN_LORA_M0, GPIO.LOW) 
- GPIO.output(PIN_LORA_M1, GPIO.LOW) +        GPIO.output(PIN_LORA_M1, GPIO.LOW) 
- time.sleep(0.01)+        time.sleep(0.01)
  
     #INIT FIREBASE SERVER     #INIT FIREBASE SERVER
Line 2394: Line 2707:
     fs = args.fs     fs = args.fs
     period = 1.0 / fs     period = 1.0 / fs
 +    
 +    window_id = 0
  
     print("Live TFLite inference started.")     print("Live TFLite inference started.")
Line 2455: Line 2770:
             ts = time.time()             ts = time.time()
  
-            print(f"[Window {window_id}] {window_summary(win_i16)}")+            print(f"[Window {window_id}] {window_summary(win)}")
             print(f"  move_prob={move_prob:.4f}  rest_prob={rest_prob:.4f}  shake_prob={shake_prob:.4f}")             print(f"  move_prob={move_prob:.4f}  rest_prob={rest_prob:.4f}  shake_prob={shake_prob:.4f}")
             print(f"  predicted={pred} ({conf*100:.1f}%)")             print(f"  predicted={pred} ({conf*100:.1f}%)")
Line 2493: Line 2808:
  
 {{ :emrp:ws2025:project_test.mp4 |}} {{ :emrp:ws2025:project_test.mp4 |}}
-|Gateway & Server Test Video|+|**//Video 5//** Gateway & Server Test Video| 
 + 
 +===== 4.Discussion ===== 
 +The implemented system has successfully demonstrated an end-to-end motion classification architecture. On the controller side, low-power mode standby, window-based data collection when motion is detected, and transmission to the gateway via LoRa; on the gateway side, real-time model execution and writing results to the cloud have all operated stably. 
 +  
 +A clear distinction was observed between the move, rest, and shake classes in the model outputs. In particular, the model consistently distinguished between low-variance acceleration data in the rest state and high angular velocity values in the shake state. As the inference being performed on the gateway, raw IMU data was not sent to the cloud; only classification outputs were transmitted. This approach reduced bandwidth usage and increased the system's scalability. Furthermore, the STOP mode and motion-triggered measurement approach significantly reduced the average power consumption on the controller side. 
 + 
 +The dataset created during the project process contains a limited number of windows. Saturation problems were observed, particularly in the shake class, and the sensor range settings were optimised. The small data set is a factor that could limit the model's generalisation capacity. 
 + 
 +Furthermore, although LoRa's 2.4 kbps air data rate provides an advantage in terms of reliability, it has extended the window transmission time. This situation has particularly required the division of 96-sample data windows into multiple frames. A balance had to be established between the air data rate, spreading factor, and FEC parameters. In LoRa communication between the controller and the gateway, synchronisation loss and timing problems occurred in the multi-frame structure, and a frame counter-based reconfiguration mechanism was developed. This process complicated the design of the communication layer. On the model side, although it was sufficient in terms of operating performance, a balance had to be struck between power consumption and response time. 
 + 
 +The system functionality can be improved in various ways. For example, ZigBee, 2.4 GHz RF or Wi-Fi-based solutions can be used instead of LoRa. Particularly in short-range applications, higher data rates may allow for larger window sizes or more frequent data transmission. However, for longer ranges, the cost of the required trade-off must be considered and calculated. Different model architectures (LSTM, 1D temporal convolution) can be tested. Furthermore, accuracy can be improved through data augmentation, larger datasets, and different normalisation techniques. However, unlike the controller, the gateway operates continuously. Switching the gateway to low-power mode until data arrives or transitioning to a lower-power edge device can improve system efficiency. 
 + 
 +For model training, several directions could further improve the model in future iterations. The current dataset was recorded by a single person in a controlled desk environment. With a larger dataset covering multiple users, gesture intensities, and controller orientations, more complex architectures such as LSTM or Temporal Convolutional Networks (TCN) could be explored. LSTMs model long-range temporal dependencies and may better capture the sequential dynamics of movement patterns; however, they require substantially more data to generalise well and are less efficient on embedded hardware than Conv1D. With 1,000+ windows per class, a hybrid architecture (Conv1D feature extractor + LSTM temporal layer) would be a natural next step. 
 + 
 +The current TFLite model uses float32 weights. Quantisation-aware training could produce an INT8 model, reducing memory footprint by approximately 4× and accelerating inference on the Raspberry Pi. This would also enable deployment on more constrained microcontrollers. The expected accuracy drop is 1–2 percentage points, which is acceptable given the current 96.6% baseline. 
 + 
 +The current model is trained offline and deployed as a fixed artefact. For deployment on actual animal subjects, where motion dynamics differ from handheld recording, a lightweight fine-tuning step using a small number of animal-specific windows could substantially improve accuracy without full retraining. Transfer learning — freezing the convolutional layers and retraining only the dense head on new data — would be a practical approach given the limited compute available on the gateway. 
 + 
 +The three-class model (Move, Rest, Shake) covers the primary behavioural states of interest. Future versions could extend to additional classes such as Eat, Play, or Sleep, provided sufficient labelled data is available. The modular architecture makes extension straightforward: only the output layer size and training data need to change. 
 + 
 +===== 5. Conclusion & Outlook ===== 
 +In this study, a low-power IMU-based motion classification system was designed and an end-to-end IoT architecture was implemented. The system integrates sensor measurement, wireless data transmission, edge inference, and cloud data management layers. Energy efficiency was achieved through a motion-triggered measurement approach, reliable data transmission was realized using LoRa-based communication, and real-time classification was successfully implemented using the TensorFlow Lite model. 
 + 
 +Future work plans include improving model performance with larger datasets, evaluating different communication technologies, and transforming the system into a fully portable edge-cloud architecture. Furthermore, hardware optimization and detailed power profile analysis can further improve battery life. In conclusion, the developed system offers a scalable and modular solution for IoT applications requiring low power consumption and real-time motion detection. 
 + 
 +===== 6. References ===== 
 +European Telecommunications Standards Institute (ETSI). 2018. “Short Range Devices (SRD) operating in the frequency range 25 MHz to 1 000 MHz; Part 2: Harmonised Standard for access to radio spectrum for non specific radio equipment”. Visited: 09.02.2026. Available at: https://www.etsi.org/deliver/etsi_en/300200_300299/30022002/03.02.01_60/en_30022002v030201p.pdf 
 + 
 +M. Alsaaod, J.J. Niederhauser, G. Beer, N. Zehner, G. Schuepbach-Regula, A. Steiner. 2015. 
 +“Development and validation of a novel pedometer algorithm to quantify extended characteristics of the locomotor behavior of dairy cows, Journal of Dairy Science”. Volume 98. Issue 9. 
 +pp. 6236-6242. ISSN 0022-0302. DOI: https://doi.org/10.3168/jds.2015-9657. 
 + 
 +Kiranyaz, S., Avci, O., Abdeljaber, O., Ince, T., Gabbouj, M., & Inman, D. J. (2021). 1D convolutional neural networks and applications: A survey. Mechanical Systems and Signal Processing, 151, 107398. https://doi.org/10.1016/j.ymssp.2020.107398 
 + 
 +Ioffe, S., & Szegedy, C. (2015). Batch normalization: Accelerating deep network training by reducing internal covariate shift. Proceedings of the 32nd International Conference on Machine Learning (ICML), 448–456. https://proceedings.mlr.press/v37/ioffe15.html 
 + 
 + 
 + 
 + 
  
  
emrp/ws2025/amt.1772082070.txt.gz · Last modified: 2026/02/26 06:01 by 36502_students.hsrw