Hybrid Multi-View ASD Classification Pipeline
Pipeline Overview
This pipeline uses a dual-branch deep learning model to classify Autism Spectrum Disorder (ASD) from gait analysis data captured from two different camera angles (front view and side view).
Step 1: Data Loading & Pairing
- Loads CSV files from
./output_normalized directory
- Pairs front view (_F_) and side view (_S_) data for each subject
- Labeling:
- Subject ID < 200 → No-ASD (Label: 0)
- Subject ID ≥ 200 → ASD (Label: 1)
X1 (Front View): (n_subjects, timesteps, features)
X2 (Side View): (n_subjects, timesteps, features)
labels: (n_subjects,)
Step 2: Train-Test Split
- Split: 70% Training, 30% Testing
- Stratified split to maintain class distribution
- Random seed: 42 for reproducibility
Training: X1_train, X2_train, y_train (70%)
Testing: X1_test, X2_test, y_test (30%)
Step 3: Class Imbalance Handling (SMOTE)
Problem: Imbalanced number of ASD vs No-ASD samples
Solution: SMOTE (Synthetic Minority Over-sampling Technique)
- Flatten 3D data to 2D
- Concatenate front and side views
- Apply SMOTE to generate synthetic samples
- Split back and reshape to 3D
- Create validation set (30% of resampled data)
After SMOTE:
- X1_final, X2_final: Training (70% of resampled)
- X1_val, X2_val: Validation (30% of resampled)
- y_train_final, y_val: Balanced labels
Important: SMOTE is only applied to training data. Test set remains original to evaluate real-world performance.
Model Architecture
Dual-Branch CNN-LSTM Model
INPUT
Front View: (timesteps, features)
Side View: (timesteps, features)
↓
FRONT BRANCH
Conv1D
Filters: 16
Kernel: 4
Activation: ReLU
↓
Batch Normalization
Normalizes activations
↓
LSTM
Units: 10
Return Sequences: False
↓
Dense
Units: 32
Activation: ReLU
↓
Output: 32 features
SIDE BRANCH
Conv1D
Filters: 16
Kernel: 3
Padding: Same
Activation: ReLU
↓
Batch Normalization
Normalizes activations
↓
LSTM
Units: 8
Return Sequences: False
Dropout: 0.3
↓
Dense
Units: 32
Activation: ReLU
↓
Output: 32 features
↓ ↓
CONCATENATION (Feature Fusion)
32 features (Front) + 32 features (Side) = 64 total features
↓
OUTPUT LAYER
Dense(1, activation='sigmoid')
Binary Classification: 0 (No-ASD) or 1 (ASD)
Training Configuration
| Parameter |
Value |
Description |
| Optimizer |
Adam |
Adaptive learning rate optimizer |
| Learning Rate |
0.001 |
Initial learning rate |
| Loss Function |
Binary Crossentropy |
For binary classification |
| Batch Size |
32 |
Samples per gradient update |
| Max Epochs |
200 |
Maximum training iterations |
| Class Weights |
Balanced |
Additional imbalance handling |
Callbacks
- Early Stopping: Stops training if validation loss doesn't improve for 15 epochs
- Model Checkpoint: Saves best model to "v2_best_model.keras"
- Reduce Learning Rate: Reduces LR by 0.5 if no improvement for 5 epochs
Model Evaluation
Metrics
- Test Accuracy: Overall classification accuracy
- Confusion Matrix: Shows True Positives, False Positives, True Negatives, False Negatives
- Classification Report: Precision, Recall, F1-Score for both classes
- Training History: Plotted and saved as "combined_model.png"
Why This Architecture Works
- Conv1D Layers: Extract spatial features from gait sequences (local movement patterns)
- LSTM Layers: Capture temporal dependencies (how gait evolves over time)
- Batch Normalization: Stabilizes training and speeds up convergence
- Dual-Branch Design: Each view captures unique information; combining improves accuracy
- Late Fusion: Allows each branch to learn view-specific features before final decision
- Dropout: Prevents overfitting by randomly disabling neurons during training
Summary
This hybrid model processes gait data from two camera angles, extracts spatial and temporal features using CNN-LSTM architecture, and combines them for robust ASD classification. The pipeline includes stratified splitting, SMOTE for class balance, and multiple training optimizations.