Research

Reinforcement learning is one type of machine learning approach that is well known for its adaptability in changing environments. It models Markov decision process based on trial-and-error mechanism, which mimics how human learns. Our research team has developed algorithms to approximate value functions which can guide optimal behavior of the agent's policy. The developed model has first validated in common control problems, including a mountain car task, center out reaching task, and sequential decision making in a hidden Markov chain that can be easily expanded to various domains. We have also expanded its application in brain machine interfaces (BMIs), to control a cursor on a computer screen or a robotic arm for reaching tasks. 

Related Publications

• R. McDorman, B. R. Thapa, J. Kim, and J. Bae. “Transfer Learning in EEG-based Reinforcement Learning Brain Machine Interfaces via Q-Learning Kernel Temporal Differences.” Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBS), 2025. (in press)
• B. R. Thapa, D. Restrepo Tangarife, and J. Bae. “Kernel Temporal Differences for EEG-based Reinforcement Learning Brain Machine Interfaces.” Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBS), 2022, pp.3327-3333, doi: 10.1109/EMBC48229.2022.9871862.
• B. Girdler, W. Caldbeck, and J. Bae. “Neural Decoders Using Reinforcement Learning in Brain Machine Interfaces: A Technical Review.” Frontiers in Systems Neuroscience, 16, 2022, doi=10.3389/fnsys.2022.836778.
• J. Fugal, J. Bae*, and H. A. Poonawala*. “On the Impact of Gravity Compensation on Reinforcement Learning in Goal-Reaching Tasks for Robotic Manipulators.” Robotics, 2021, 10(1), 46, https://doi.org/10.3390/robotics10010046. (* corresponding authors)
• J. Bae, L. G. Sanchez Giraldo, E. A. Pohlmeyer, J. T. Francis, J. C. Sanchez, and J. C. Principe. “Kernel Temporal Differences for Neural Decoding.” Computational Intelligence and Neuroscience, 2015, Article ID 481375, pp. 1-17, https://doi.org/10.1155/2015/481375.
• J. Bae, L. G. Sanchez Giraldo, J. T. Francis, and J. C. Principe. “Correntropy Kernel Temporal Differences for Reinforcement Learning Brain Machine Interfaces.” The International Joint Conference on Neural Networks (IJCNN), 2014, pp. 2713-2717, doi: 10.1109/IJCNN.2014.6889958.
• A. J. Brockmeier, L. G. Sanchez Giraldo, M. S. Emigh, J. Bae, J. S. Choi, J. T. Francis, and J. C. Principe. “Information-Theoretic Metric Learning: 2-D Linear Projections of Neural Data for Visualization.” Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBS), 2013, pp. 5586-5589, doi:10.1109/EMBC.2013.6610816.
• J. Bae, L. G. Sanchez Giraldo, E. A. Pohlmeyer, J. C. Sanchez, and J. C. Principe. “A New Method of Concurrently Visualizing States, Values, and Actions in Reinforcement based Brain Machine Interfaces.” Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBS), 2013, pp. 5402-5405, doi:10.1109/EMBC.2013.6610770.
• J. Bae, L. G. Sanchez Giraldo, P. Chhatbar, J. Francis,  J. Sanchez, and J. C. Principe. “Stochastic Kernel Temporal Difference for Reinforcement Learning.” IEEE International Workshop on Machine Learning for Signal Processing (MLSP), 2011, pp. 1-6, doi:10.1109/MLSP.2011.6064634.
• J. Bae, P. Chhatbar, J. T. Francis, J. C. Sanchez, and J. C. Principe. “Reinforcement Learning via Kernel Temporal Difference.” Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBS), 2011, pp. 5662-5665, doi:10.1109/IEMBS.2011.6091370.

BMIs have been a subject of active research because of their potential for a wide range of applications once the technology reaches maturity. Developing technologies in BMIs is an essential step to ultimately help overcome neuromuscular disabilities and treat neurological disorders. One key challenge in BMIs from a signal processing perspective is to obtain a robust and reliable neural decoder, which can characterize the electrical activity of groups of neurons to control external devices, for instance, a robotic arm or a computer cursor.

Related Publications

• B. R. Thapa, J. Paredes, J. Boggess, A. O. Shalash, and J. Bae. “Classification of Error-Related Potentials in EEG-based Brain Machine Interfaces.” IEEE Engineering in Medicine and Biology Society (EMBS) the 12th Annual International Conference on Neural Engineering, 2025. (In Press)
• B. R. Thapa and J. Bae. “Decoding EEG Premovement and Movement Intentions in Freewill Reaching and Grasping Tasks: Window Analysis.” Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBS), 2025. (In Press)
• S. Posso-Murillo, L. G. Sanchez-Giralso, and J. Bae. “Semantic Reconstruction from Fnirs Using Recurrent Neural Networks.” IEEE 22nd International Symposium on Biomedical Imaging (ISBI), 2025, pp.1-5, doi: 10.1109/ISBI60581.2025.10981123.
• B. R. Thapa, D. Restrepo Tangarife, and J. Bae. “Kernel Temporal Differences for EEG-based Reinforcement Learning Brain Machine Interfaces.” Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBS), 2022, pp.3327-3333, doi: 10.1109/EMBC48229.2022.9871862.
• P. Awasthi, T. H. Lin, J. Bae, L. E. Miller, and Z. C. Danziger. “Validation of a non-invasive, real-time, human-in-the-loop model of intracortical brain-computer interfaces.” Journal of Neural Engineering, 2022, doi: 10.1088/1741-2552/ac97c3.
• W. Plucknett, L. G. Sanchez Giraldo, and J. Bae. “Metric Learning in Freewill EEG Pre-Movement and Movement Intention Classification for Brain Machine Interfaces.” Frontiers Human Neuroscience, 16, 2022, doi=10.3389/fnhum.2022.902183.

Electric source imaging (ESI) provides functional images of the whole brain with an exquisite temporal resolution based on high-resolution electroencephalogram (EEG). Particularly in focal epilepsy, which is a well-known neurological disorder accompanied with recurrent and unprovoked seizures, ESI allows a better understanding of the electrical substrates of pathological events and abnormalities associated with brain network activity. We investigate EEG analysis methods to extract meaningful features to help ESI and to better understand underlying mechanisms on ESI-based biomarkers.

Related Publications

• J. Bae, J. Clay, B. R. Thapa, D. Powell, B. Krishnan, A. Koupparis, M. Bensalem Owen, and F. Raslau. “Process of equipment setup and artifact removal for simultaneous EEG-fMRI recordings for clinical review of interictal period in epilepsy.” Journal of Visualized Experiments, 2023, (196), e64919, doi:10.3791/64919.
• A. Deshmukh, J. Leichner, J. Bae, Y. Song, P. A. Valdes-Hernandez, W. C. Lin, and J. J. Riera. “Histological Characterization of the Irritative Zones in Focal Cortical Dysplasia Using a Preclinical Rat Model.” Frontiers Cellular Neuroscience, eCollection 2018, doi: 10.3389/fncel.2018.00052, 2018.
• P. A. Valdes-Hernandez, J. Bae, Y. Song, A. Sumiyoshi, E. Aubert-Vazquez, and J. J. Riera. “Validating Non-invasive EEG Source Imaging Using Optimal Electrode Configurations on a Representative Rat Head Model.” Brain topography, 2019, 32(4), pp. 599-624, doi: 10.1007/s10548-016-0484-4.
• Y. Song, R. A. Torres, S. Garcia, Y. Frometa, J. Bae, A. Deshmukh, W. Lin, Y. Zheng, and J. J. Riera. “Dysfunction of Neurovascular/Metabolic Coupling in Chronic Focal Epilepsy.” IEEE Transactions on Biomedical Engineering, 2016, 63(1), pp.97-110, doi: 10.1109/TBME.2015.2461496.
• J. Bae, A. Deshmukh, Y. Song, and J. Riera. “Brain Source Imaging in Preclinical Rat Models of Focal Epilepsy using High-Resolution EEG Recordings.” Journal of Visualized Experiments, (100), e52700, doi:10.3791/52700, 2015.