PPG for Emotion Recognition: Affective Computing with Photoplethysmography
**Emotions are not purely cognitive events -- they are embodied physiological states that leave measurable traces in cardiovascular signals.** Fear ac...
- Remote PPG & Camera
Emotions are not purely cognitive events -- they are embodied physiological states that leave measurable traces in cardiovascular signals. Fear accelerates the heart and constricts blood vessels. Relaxation slows the heart and opens peripheral circulation. These autonomic signatures, captured by photoplethysmography through the skin, enable a growing field of affective computing that seeks to recognize emotional states from physiological data. While the accuracy of PPG-based emotion recognition remains below that of facial or vocal analysis, its unobtrusiveness and resistance to voluntary masking make it uniquely valuable for continuous, honest emotional assessment.
This article reviews the physiological basis linking emotions to PPG signals, the feature extraction and classification methods used in the research literature, benchmark datasets and reported accuracies, and the practical applications and ethical considerations of physiological emotion sensing. For foundational context on PPG signal acquisition, see our introduction to PPG technology.
Theoretical Foundations: Emotions and the Autonomic Nervous System
The relationship between emotions and cardiovascular physiology has been studied for over a century, beginning with William James's somatic theory of emotion (1884), which proposed that bodily changes are not consequences but rather constituents of emotional experience. Modern psychophysiology has established that different emotional states produce partially distinct patterns of autonomic nervous system (ANS) activation, though the specificity and consistency of these patterns remain debated.
The Circumplex Model of Affect
Most PPG-based emotion recognition research adopts the circumplex model of Russell (1980, doi:10.1037/h0077714), which represents emotions along two continuous dimensions: valence (positive to negative) and arousal (calm to activated). This model maps discrete emotions onto a two-dimensional space: happiness is high valence, moderate-to-high arousal; sadness is low valence, low arousal; anger is low valence, high arousal; and relaxation is high valence, low arousal.
The circumplex model is preferred over discrete emotion categories for physiological sensing because the autonomic nervous system primarily encodes arousal (through sympathetic-parasympathetic balance) rather than the specific cognitive content of emotions. Valence has weaker and less consistent autonomic correlates, which is why arousal classification from PPG typically outperforms valence classification by 5-15 percentage points.
Autonomic Specificity of Emotions
Ekman et al. (1983; doi:10.1126/science.6612338) provided early evidence for emotion-specific autonomic patterns, showing that anger and fear produced larger heart rate increases than happiness, and that anger was associated with greater finger temperature increase (peripheral vasodilation) than fear (which produced vasoconstriction). Subsequent meta-analyses have confirmed that different emotions produce statistically distinguishable cardiovascular patterns, though with substantial overlap and individual variability.
Kreibig (2010; doi:10.1016/j.biopsycho.2010.03.010) conducted a comprehensive review of autonomic correlates of discrete emotions across 134 studies. Key findings relevant to PPG-based detection include: fear and anger produce the largest heart rate increases (8-15 BPM above baseline); sadness produces variable heart rate responses depending on whether it involves active crying (increase) or passive withdrawal (decrease); happiness produces moderate heart rate increase (3-7 BPM) and peripheral vasodilation; and disgust is uniquely associated with heart rate deceleration. These differential patterns, while statistically significant at the group level, overlap considerably within individuals, limiting classification accuracy.
PPG Feature Extraction for Emotion Recognition
Extracting emotion-relevant features from PPG signals involves the same categories used for cognitive load detection -- HRV, amplitude, and morphology -- but with emphasis on features that differentiate emotional valence in addition to arousal. For background on HRV analysis methods, see our HRV chart guide.
Time-Domain HRV Features
Inter-beat intervals (IBIs) derived from PPG peak detection provide the foundation for HRV analysis. Key features for emotion recognition include mean IBI (inversely proportional to heart rate), SDNN (overall HRV reflecting total autonomic modulation), RMSSD (short-term HRV reflecting vagal parasympathetic activity), and pNN50. Under high-arousal emotions (anger, fear, excitement), RMSSD and pNN50 decrease as vagal withdrawal occurs. Under low-arousal states (sadness, relaxation), parasympathetic activity may increase or normalize.
Time-domain HRV features such as RMSSD and SDNN generally track arousal more closely than valence, which is one reason HRV-only models tend to struggle with valence discrimination.
Frequency-Domain Features
Spectral analysis of IBI series separates sympathetic and parasympathetic contributions. HF power (0.15-0.40 Hz) reflects parasympathetic vagal modulation and decreases under high-arousal emotions. LF power (0.04-0.15 Hz) reflects a mixture of sympathetic and parasympathetic activity. The LF/HF ratio increases during sympathetically-dominant emotional states (anger, fear) and decreases during parasympathetically-dominant states (relaxation, certain sadness states).
Notably, some researchers have identified frequency-domain features that partially discriminate valence. Rainville et al. (2006; doi:10.1016/j.ijpsycho.2005.10.024) found that anger and fear (both negative valence, high arousal) could be partially distinguished by their spectral HRV profiles: anger showed greater LF power increase, while fear showed greater HF power suppression, consistent with differential sympathetic versus parasympathetic activation patterns.
Pulse Wave Morphology Features
PPG waveform shape carries emotion-relevant information beyond inter-beat timing. The dicrotic notch prominence, pulse rise time, pulse width at half maximum, and reflection index change with arterial tone and blood pressure, which are modulated by emotional arousal.
Non-Linear and Complexity Features
Emotional states may alter the complexity of cardiac dynamics in ways not captured by linear features. Approximate entropy, sample entropy, Lyapunov exponents, and correlation dimension of IBI series have all been explored for emotion classification. Valenza, Lanata and Scilingo (2012; IEEE Transactions on Affective Computing; doi:10.1109/T-AFFC.2011.30) studied non-linear dynamics features for affective valence and arousal recognition, supporting the idea that the complexity structure of cardiac regulation carries emotion-specific information. For more on PPG-derived algorithms, see our algorithms section.
Machine Learning and Deep Learning Approaches
Traditional Machine Learning
Support vector machines (SVMs) with radial basis function kernels have been the most widely used classifier for PPG-based emotion recognition. Handcrafted feature vectors combining 10-30 HRV, amplitude, and morphological features are classified using SVMs trained on labeled emotional episodes.
Picard, Vyzas and Healey (2001; IEEE Transactions on Pattern Analysis and Machine Intelligence; doi:10.1109/34.954607) pioneered physiological emotion recognition, reporting 81% accuracy for eight emotional states from physiological signals including blood volume pulse (PPG) in a single-subject study. Multi-subject generalization has proved far more challenging.
Schmidt et al. (2018; doi:10.1145/3242969.3242985) evaluated multiple classifiers on the WESAD dataset (wrist-worn PPG and accelerometer data from 15 subjects during stress, amusement, and baseline conditions). Using PPG features alone, random forest achieved the best three-class accuracy of 67.7%, while adding accelerometer features improved accuracy to 72.1%. With both wrist PPG and chest sensors, accuracy reached 84.7%.
Deep Learning Architectures
Convolutional neural networks (CNNs) and recurrent neural networks (RNNs) applied to raw or minimally processed PPG signals have shown promise for emotion recognition without manual feature engineering.
Long short-term memory (LSTM) networks capture temporal dynamics in PPG-derived feature sequences. More broadly, Siddharth et al. (2019, doi:10.1109/TAFFC.2019.2916015) applied deep learning to bio-sensing and video data from four public emotion datasets (more than 120 subjects and 2,800 trials) and outperformed previously reported results for emotion, valence, arousal and liking classification on DEAP and MAHNOB-HCI.
Benchmark Datasets
Reproducible emotion recognition research depends on standardized datasets with physiological recordings and emotion labels. Several publicly available datasets include PPG data.
DEAP Dataset
The DEAP dataset (Koelstra et al., 2012; doi:10.1109/T-AFFC.2011.15) contains physiological recordings from 32 participants watching 40 one-minute music videos, rated on valence, arousal, dominance, and liking scales. While the primary physiological signals are EEG and peripheral channels, PPG is included among the peripheral recordings. DEAP is the most widely used benchmark for physiological emotion recognition, with over 1,500 citing papers.
Reported PPG-only results on DEAP include binary valence classification accuracies of 62-78% and binary arousal accuracies of 68-83%, depending on the feature extraction and classification approach. The relatively modest valence accuracies on DEAP have been attributed partly to the weakness of music video stimuli for inducing strong valence differences and partly to noisy self-report labels.
WESAD Dataset
The WESAD dataset (Schmidt et al., 2018) specifically targets wearable stress and affect detection, making it highly relevant for PPG-based applications. It includes wrist-worn PPG (Empatica E4) and chest-worn multi-sensor data from 15 subjects during baseline, stress (Trier Social Stress Test), amusement (funny video clips), and meditation conditions. The stress induction protocol produces strong, ecologically valid emotional responses with clear autonomic signatures.
PPG-only three-class classification (baseline, stress, amusement) on WESAD typically achieves 67-75% accuracy, while binary stress detection (stress vs. non-stress) reaches 80-87% accuracy. The dataset's limitation is its small size (15 subjects) and imbalanced class distribution.
AMIGOS Dataset
The AMIGOS dataset (Miranda-Correa et al., 2021; doi:10.1109/TAFFC.2018.2884461) recorded EEG, ECG and GSR (no PPG channel) from 40 participants watching short and long emotion-eliciting videos in both individual and group settings. It includes social context annotations, making it suitable for studying how social context modulates physiological emotional responses.
Applications of PPG-Based Emotion Recognition
Mental Health Monitoring
Continuous emotion tracking through wearable PPG could support mental health management by detecting prolonged negative emotional states, tracking mood patterns over days and weeks, and providing objective data to complement self-report assessments. Sano et al. (2018; Journal of Medical Internet Research; doi:10.2196/jmir.9410) collected over 145,000 hours of wrist-sensor and phone data from 201 college students over one month and used machine learning to identify self-reported high stress and poor mental health.
Human-Computer Interaction
Emotion-aware interfaces could adapt their behavior based on the user's emotional state -- calming down an agitated user, energizing a bored user, or deferring complex tasks when the user appears stressed. This application connects closely with cognitive load detection, as emotional arousal and cognitive load share autonomic signatures.
Consumer Wellness
Smartwatches and fitness trackers are increasingly incorporating stress and emotion tracking features based on PPG-derived HRV. These features typically provide daily stress scores, relaxation prompts, and trend visualizations. While the accuracy of consumer implementations is lower than research-grade systems, the longitudinal data they collect may be valuable for identifying patterns and triggers.
Affective Gaming and Media
Interactive entertainment that responds to the player's emotional state represents a growing application area. Games that become more or less challenging based on detected stress, or narratives that branch based on emotional engagement, could create more immersive experiences. PPG-based sensing is well-suited for this application because gaming controllers and VR headsets can incorporate optical sensors with minimal additional hardware.
Ethical Considerations
PPG-based emotion recognition raises significant ethical concerns that distinguish it from other PPG applications. Unlike heart rate monitoring, which provides accepted health information, emotion detection involves inferring subjective internal states from physiological data, with potential for misuse.
Concerns include workplace surveillance (monitoring employee emotions without consent), manipulative personalization (using detected emotional vulnerability to push commercial decisions), and privacy implications of continuous emotional monitoring. The moderate accuracy of current systems adds the risk of false inferences -- labeling someone as stressed, angry, or disengaged based on ambiguous physiological signals.
Physiological emotion recognition systems arguably should be subject to specific regulatory frameworks that address informed consent, data minimization, and the right to not have one's emotional state inferred. These considerations are particularly important as PPG-based emotion sensing moves from research laboratories into consumer products and workplace applications.
Limitations and Future Directions
The fundamental limitation of PPG-based emotion recognition is the many-to-one mapping between emotions and autonomic responses. Different emotions can produce similar cardiovascular patterns (anger and excitement both increase heart rate), and the same emotion can produce different patterns across individuals and contexts. This inherent physiological ambiguity places a ceiling on classification accuracy that no algorithm can fully overcome from PPG data alone.
Multi-modal fusion -- combining PPG with electrodermal activity (EDA), facial expression, voice prosody, and contextual information -- substantially improves accuracy but increases system complexity. Context-aware models that incorporate time of day, activity level, and social context as additional inputs could help disambiguate emotional from non-emotional autonomic changes.
Personalization through individual calibration and continual learning from user feedback may ultimately be the most impactful improvement strategy. A system that learns each user's unique physiological emotional fingerprint over weeks or months could achieve substantially higher accuracy than cross-subject models, bringing PPG-based emotion recognition from a research curiosity to a practically useful technology. For more on how PPG signals are processed and interpreted across clinical and consumer applications, explore our conditions database and algorithms reference.
Conclusion
PPG-based emotion recognition exploits the autonomic nervous system's role in emotional experience to infer affective states from cardiovascular signals. Current systems achieve 75-88% accuracy for arousal classification and 70-82% for valence classification in controlled settings, with specific discrete emotion recognition remaining more challenging at 45-65% accuracy. Deep learning approaches, particularly self-supervised pre-training and temporal modeling, have pushed performance boundaries beyond traditional feature-based classifiers. While PPG alone cannot match the accuracy of facial expression or multi-modal systems for emotion recognition, its unobtrusiveness, resistance to voluntary masking, and compatibility with existing wearable hardware make it a uniquely valuable modality for continuous affective computing. The transition from laboratory demonstrations to real-world applications will require addressing ethical concerns, improving cross-subject generalization, and developing robust methods for separating emotional arousal from physical activity and other confounding sources of autonomic activation.
References
- Picard RW, Vyzas E, Healey J. Toward machine emotional intelligence: analysis of affective physiological state. IEEE Trans Pattern Anal Mach Intell. 2001;23(10):1175-1191. https://doi.org/10.1109/34.954607
- Valenza G, Lanata A, Scilingo EP. The role of nonlinear dynamics in affective valence and arousal recognition. IEEE Trans Affect Comput. 2012;3(2):237-249. https://doi.org/10.1109/T-AFFC.2011.30
- Miranda-Correa JA, Abadi MK, Sebe N, Patras I. AMIGOS: A dataset for affect, personality and mood research on individuals and groups. IEEE Trans Affect Comput. 2021;12(2):479-493. https://doi.org/10.1109/TAFFC.2018.2884461
- Sano A, Taylor S, McHill AW, et al. Identifying objective physiological markers and modifiable behaviors for self-reported stress and mental health status using wearable sensors and mobile phones: observational study. J Med Internet Res. 2018;20(6):e210. https://doi.org/10.2196/jmir.9410
Related articles
PPG in Veterinary Medicine: Animal Health Monitoring with Photoplethysmography
PPG is widely used in veterinary medicine for pulse oximetry, anesthesia monitoring, and heart rate measurement across animal species. Wearable PPG de...
Objective Pain Assessment with PPG: Measuring Pain from Photoplethysmography Signals
**Pain assessment remains one of medicine's most persistent challenges: it is inherently subjective, difficult to communicate for many patients, and i...
PPG in Affective Computing: Applications from Gaming to Therapy
Survey of PPG-based affective computing applications covering emotion-adaptive gaming, mental health monitoring, human-robot interaction, and biofeedback therapy with accuracy benchmarks.
Stress Recovery Monitoring with PPG: Why Recovery May Matter More Than Stress Scores
Learn how stress recovery monitoring with PPG tracks recovery after stress using heart rate, HRV, and pulse waveform signals, beyond simple stress scores.
Remote Photoplethysmography Accuracy Factors
What limits rPPG accuracy: motion, lighting, skin tone, video compression and camera hardware, with typical error ranges and the mitigations that help.
Devices in this article
Research records with the datasets, papers and published validation results for each device.