This repository contains the implementation of a speech-based lie detection system leveraging the AASIST (Audio Anti-Spoofing System) framework. The model is designed to analyze speech recordings and classify them as either truthful or deceptive based on acoustic patterns.
Deception detection using speech processing has significant applications in security, forensics, and psychology. This project adapts the AASIST framework, originally developed for audio anti-spoofing, to detect deception in speech across multiple languages. We evaluate the system on both English and Romanian datasets.
The system is trained and evaluated on two main datasets:
-
Real-life Deception Detection Dataset (RLDD):
- English audio recordings from courtroom trials
- Contains truthful and deceptive testimonies in uncontrolled acoustic environments
-
Romanian Deva Criminal Investigation Audio Recordings Dataset (RODeCAR):
- Romanian audio recordings from criminal investigations
- Provides cross-lingual evaluation capability
- Adaptation of AASIST framework for binary lie detection classification
- Cross-dataset and cross-lingual evaluation
- Comprehensive performance metrics (Accuracy, Precision, Recall, F1, EER, AUC)
- Performance visualization tools (ROC curves, PR curves, confusion matrices)
├── LICENSE # Project license
├── models/ # Model architecture definitions
│ ├── AASIST.py # AASIST model implementation
│ └── AASIST.pth # Pre-trained model weights
├── config/ # Configuration files
├── utils/ # Utility functions for training
├── notebooks/ # Jupyter notebooks containing experiments and plots
├── data/ # Dataset directory containing RLDD and Rodecar
├── results/ # Results saved from the best experiments (metrics and models)
├── experiment_dump/ # Experiment outputs
├── train/ # Training scripts
├── visualizations/ # Generated architecture visualizations
│ ├── aasist_diagram.png # Architecture diagram
│ └── visualize_aasist_diagram.py # Diagram generation script
└── README.md # This file
- Python 3.7+
- PyTorch 1.8+
- TensorBoard
- scikit-learn
- numpy
- pandas
- matplotlib
- [Other dependencies as needed]
-
Clone the repository:
git clone [repository-url] cd [repository-name] -
Install dependencies:
pip install -r requirements.txt -
Prepare datasets:
- Download RLDD and RODeCAR datasets
- Place them in the appropriate directories (dataset/, romanian_dataset/)
- Run preprocessing:
python process_rodecar.py
To train the model on the RLDD dataset:
python main.py --train --dataset RLDD --epochs 75 --batch_size 8 --config config/aasist_rldd.conf
To evaluate a trained model:
python main.py --eval --dataset [RLDD/RODeCAR] --model_path [path_to_saved_model] --config [config_file]
python evaluation.py --score_path [path_to_scores] --output_dir [output_directory]
- Accuracy: 63.16%
- Precision: 75.00%
- Recall: 33.33%
- F1 Score: 46.15%
- EER: 52.78%
- ROC AUC: 0.567
- Accuracy: 81.82%
- Precision: 80.98%
- Recall: 85.53%
- F1 Score: 83.19%
- EER: 18.52%
- ROC AUC: 0.886
The model demonstrated significantly better performance on the RODeCAR dataset compared to RLDD, suggesting sensitivity to dataset characteristics, language factors, and sample size differences.
Contributions are welcome! Please feel free to submit a Pull Request.
- Shyam Sathvik ([email protected])
- Neermita Bhattacharya ([email protected])