1) Development of Technologies in Reinforcement Learning

Reinforcement learning is one type of machine learning approach that is well known for its adaptability in changing environments. It models Markov decision process based on trial-and-error mechanism, which mimics how human learns. Our research team has developed algorithms to approximate value functions which can guide optimal behavior of the agent's policy. The developed model has first validated in common control problems, including a mountain car task, center out reaching task, and sequential decision making in a hidden Markov chain that can be easily expanded to various domains. We have also expanded its application in brain machine interfaces (BMIs), to control a cursor on a computer screen or a robotic arm for reaching tasks. 

Related Publications

• R. McDorman, B. R. Thapa, J. Kim, and J. Bae. “Transfer Learning in EEG-based Reinforcement Learning Brain Machine Interfaces via Q-Learning Kernel Temporal Differences.” Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBS), 2025. (in press)
• B. R. Thapa, D. Restrepo Tangarife, and J. Bae. “Kernel Temporal Differences for EEG-based Reinforcement Learning Brain Machine Interfaces.” Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBS), 2022, pp.3327-3333, doi: 10.1109/EMBC48229.2022.9871862.
• B. Girdler, W. Caldbeck, and J. Bae. “Neural Decoders Using Reinforcement Learning in Brain Machine Interfaces: A Technical Review.” Frontiers in Systems Neuroscience, 16, 2022, doi=10.3389/fnsys.2022.836778.

• J. Fugal, J. Bae*, and H. A. Poonawala*. “On the Impact of Gravity Compensation on Reinforcement Learning in Goal-Reaching Tasks for Robotic Manipulators.” Robotics, 2021, 10(1), 46, https://doi.org/10.3390/robotics10010046. (* corresponding authors)
• J. Bae, L. G. Sanchez Giraldo, E. A. Pohlmeyer, J. T. Francis, J. C. Sanchez, and J. C. Principe. “Kernel Temporal Differences for Neural Decoding.” Computational Intelligence and Neuroscience, 2015, Article ID 481375, pp. 1-17, https://doi.org/10.1155/2015/481375.
• J. Bae, L. G. Sanchez Giraldo, J. T. Francis, and J. C. Principe. “Correntropy Kernel Temporal Differences for Reinforcement Learning Brain Machine Interfaces.” The International Joint Conference on Neural Networks (IJCNN), 2014, pp. 2713-2717, doi: 10.1109/IJCNN.2014.6889958.
• A. J. Brockmeier, L. G. Sanchez Giraldo, M. S. Emigh, J. Bae, J. S. Choi, J. T. Francis, and J. C. Principe. “Information-Theoretic Metric Learning: 2-D Linear Projections of Neural Data for Visualization.” Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBS), 2013, pp. 5586-5589, doi:10.1109/EMBC.2013.6610816.
• J. Bae, L. G. Sanchez Giraldo, E. A. Pohlmeyer, J. C. Sanchez, and J. C. Principe. “A New Method of Concurrently Visualizing States, Values, and Actions in Reinforcement based Brain Machine Interfaces.” Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBS), 2013, pp. 5402-5405, doi:10.1109/EMBC.2013.6610770.
• J. Bae, L. G. Sanchez Giraldo, P. Chhatbar, J. Francis,  J. Sanchez, and J. C. Principe. “Stochastic Kernel Temporal Difference for Reinforcement Learning.” IEEE International Workshop on Machine Learning for Signal Processing (MLSP), 2011, pp. 1-6, doi:10.1109/MLSP.2011.6064634.
• J. Bae, P. Chhatbar, J. T. Francis, J. C. Sanchez, and J. C. Principe. “Reinforcement Learning via Kernel Temporal Difference.” Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBS), 2011, pp. 5662-5665, doi:10.1109/IEMBS.2011.6091370.

3) Electroencephalogram (EEG) Analysis and Source Imaging in Epilepsy

Electric source imaging (ESI) provides functional images of the whole brain with an exquisite temporal resolution based on high-resolution electroencephalogram (EEG). Particularly in focal epilepsy, which is a well-known neurological disorder accompanied with recurrent and unprovoked seizures, ESI allows a better understanding of the electrical substrates of pathological events and abnormalities associated with brain network activity. We investigate EEG analysis methods to extract meaningful features to help ESI and to better understand underlying mechanisms on ESI-based biomarkers.

Related Publications

• J. Bae, J. Clay, B. R. Thapa, D. Powell, B. Krishnan, A. Koupparis, M. Bensalem Owen, and F. Raslau. “Process of equipment setup and artifact removal for simultaneous EEG-fMRI recordings for clinical review of interictal period in epilepsy.” Journal of Visualized Experiments, 2023, (196), e64919, doi:10.3791/64919.
• A. Deshmukh, J. Leichner, J. Bae, Y. Song, P. A. Valdes-Hernandez, W. C. Lin, and J. J. Riera. “Histological Characterization of the Irritative Zones in Focal Cortical Dysplasia Using a Preclinical Rat Model.” Frontiers Cellular Neuroscience, eCollection 2018, doi: 10.3389/fncel.2018.00052, 2018.
• P. A. Valdes-Hernandez, J. Bae, Y. Song, A. Sumiyoshi, E. Aubert-Vazquez, and J. J. Riera. “Validating Non-invasive EEG Source Imaging Using Optimal Electrode Configurations on a Representative Rat Head Model.” Brain topography, 2019, 32(4), pp. 599-624, doi: 10.1007/s10548-016-0484-4.
• Y. Song, R. A. Torres, S. Garcia, Y. Frometa, J. Bae, A. Deshmukh, W. Lin, Y. Zheng, and J. J. Riera. “Dysfunction of Neurovascular/Metabolic Coupling in Chronic Focal Epilepsy.” IEEE Transactions on Biomedical Engineering, 2016, 63(1), pp.97-110, doi: 10.1109/TBME.2015.2461496.
• J. Bae, A. Deshmukh, Y. Song, and J. Riera. “Brain Source Imaging in Preclinical Rat Models of Focal Epilepsy using High-Resolution EEG Recordings.” Journal of Visualized Experiments, (100), e52700, doi:10.3791/52700, 2015.