About Me

I am a researcher at the US DEVCOM Army Research Lab studying human-guided machine learning. My research focuses on human-in-the-loop learning, sequential decision-making algorithms, and robot learning. I’m excited about the potential for algorithms leveraging human-AI interaction to support people in many personalized ways. Outside of research, I am committed to positively impacting people’s lives through mentorship and outreach.

I completed my PhD in Control and Dynamical Systems at Caltech, where I was fortunate to be advised by Professors Yisong Yue and Joel W. Burdick and to work on optimizing the walking gaits of a robotic exoskeleton. After that, I was a postdoctoral researcher and CI Fellow at UC Berkeley, working with Professor Ken Goldberg in the AUTOLAB. Prior to my current position, I was a postdoctoral fellow at the Army Research Lab, working with Dr. Nicholas Waytowich.

Publications

PhD Dissertation:

Online Learning from Human Feedback with Applications to Exoskeleton Gait Optimization
Ellen Novoseller
CaltechTHESIS repository, 2020
PDF

Pre-Prints:

Crowd-PrefRL: Preference-Based Reward Learning from Crowds
David Chhan, Ellen Novoseller, and Vernon J. Lawhern
ArXiv pre-print, 2024
PDF

Conference Publications:

Scalable Interactive Machine Learning for Future Command and Control
Anna Madison*, Ellen Novoseller*, Vinicius G. Goecks*, Benjamin T. Files, Nicholas Waytowich, Alfred Yu, Vernon J. Lawhern, Steven Thurman, Christopher Kelshaw, and Kaleb McDowell
International Conference on Military Communication and Information Systems (ICMCIS), 2024
(*equal contribution)
PDF

Re-Envisioning Command and Control
Kaleb McDowell, Ellen Novoseller, Anna Madison, Vinicius G. Goecks, and Christopher Kelshaw
International Conference on Military Communication and Information Systems (ICMCIS), 2024
PDF

Rating-Based Reinforcement Learning
Devin White, Mingkang Wu, Ellen Novoseller, Vernon J. Lawhern, Nicholas Waytowich, and Yongcan Cao
AAAI Conference on Artificial Intelligence, 2024
PDF     Code

Towards Solving Fuzzy Tasks with Human Feedback: A Retrospective of the MineRL BASALT 2022 Competition
Stephanie Milani, Anssi Kanervisto, Karolis Ramanauskas, Sander Schulhoff, Brandon Houghton, Sharada Mohanty, Byron Galbraith, Ke Chen, Yan Song, Tianze Zhou, Bingquan Yu, He Liu, Kai Guan, Yujing Hu, Tangjie Lv, Federico Malato, Florian Leopold, Amogh Raut, Ville Hautamäki, Andrew Melnik, Shu Ishida, João Henriques, Robert Klassert, Walter Laurito, Ellen Novoseller, Vinicius Goecks, Nicholas Waytowich, David Watkins, Joshua Miller, Rohin Shah
Proceedings of Machine Learning Research, 2023
PDF     Code
Our competition submission received a Research Award.

Efficient Preference-Based Reinforcement Learning Using Learned Dynamics Models
Yi Liu, Gaurav Datta, Ellen Novoseller, and Daniel S. Brown
IEEE Conference on Robotics and Automation (ICRA), 2023
PDF     Website     Code

Efficiently Learning Single-Arm Fling Motions to Smooth Garments
Lawrence Yunliang Chen*, Huang Huang*, Ellen Novoseller, Daniel Seita, Jeffrey Ichnowski, Michael Laskey, Richard Cheng, Thomas Kollar, and Ken Goldberg
International Symposium on Robotics Research (ISRR), 2022
PDF     Website

Autonomously Untangling Long Cables
Vainavi Viswanath*, Kaushik Shivakumar*, Justin Kerr*, Brijen Thananjeyan, Ellen Novoseller, Jeffrey Ichnowski, Alejandro Escontrela, Michael Laskey, Joseph E. Gonzalez, and Ken Goldberg
Conference on Robotics: Science and Systems (RSS), 2022
PDF     Website
Received Best Systems Paper Award.

Learning Switching Criteria for Sim2Real Transfer of Robotic Fabric Manipulation Policies
Satvik Sharma*, Ellen Novoseller*, Vainavi Viswanath, Zaynah Javed, Rishi Parikh, Ryan Hoque, Daniel S. Brown, Ashwin Balakrishna, and Ken Goldberg
IEEE Conference on Automation Science and Engineering (CASE), 2022
(*equal contribution)
PDF     Code     Website

ThriftyDAgger: Budget-Aware Novelty and Risk Gating for Interactive Imitation Learning
Ryan Hoque, Ashwin Balakrishna, Ellen Novoseller, Albert Wilcox, Daniel S. Brown, and Ken Goldberg
Conference on Robot Learning (CoRL), 2021
PDF     Code     Website

Untangling Dense Non-Planar Knots by Learning Manipulation Features and Recovery Policies
Priya Sundaresan*, Jennifer Grannen*, Brijen Thananjeyan, Ashwin Balakrishna, Jeffrey Ichnowski, Ellen Novoseller, Minho Hwang, Michael Laskey, Joseph E. Gonzalez, and Ken Goldberg
Conference on Robotics: Science and Systems (RSS), 2021
PDF     Website

Disentangling Dense Multi-Cable Knots
Vainavi Viswanath*, Jennifer Grannen*, Priya Sundaresan*, Brijen Thananjeyan, Ashwin Balakrishna, Ellen Novoseller, Jeffrey Ichnowski, Michael Laskey, Joseph E. Gonzalez, and Ken Goldberg
IEEE Conference on Intelligent Robots and Systems (IROS), 2021
PDF     Website

LazyDAgger: Reducing Context Switching in Interactive Imitation Learning
Ryan Hoque, Ashwin Balakrishna, Carl Putterman, Michael Luo, Daniel S. Brown, Daniel Seita, Brijen Thananjeyan, Ellen Novoseller, and Ken Goldberg
IEEE Conference on Automation Science and Engineering (CASE), 2021
PDF     Website

ROIAL: Region of Interest Active Learning for Characterizing Exoskeleton Gait Preference Landscapes
Kejun Li, Maegan Tucker, Erdem Bıyık, Ellen Novoseller, Joel W. Burdick, Yanan Sui, Dorsa Sadigh, Yisong Yue, and Aaron D. Ames
IEEE Conference on Robotics and Automation (ICRA), 2021
PDF     Code     Website

Human Preference-Based Learning for High-Dimensional Optimization of Exoskeleton Walking Gaits
Maegan Tucker, Myra Cheng, Ellen Novoseller, Richard Cheng, Yisong Yue, Joel W. Burdick, and Aaron D. Ames
IEEE Conference on Intelligent Robots and Systems (IROS), 2020
PDF     Code

Dueling Posterior Sampling for Preference-Based Reinforcement Learning
Ellen Novoseller, Yibing Wei, Yanan Sui, Yisong Yue, and Joel W. Burdick
Conference on Uncertainty in Artificial Intelligence (UAI), 2020
PDF     Code

Preference-Based Learning for Exoskeleton Gait Optimization
Maegan Tucker*, Ellen Novoseller*, Claudia Kann, Yanan Sui, Yisong Yue, Joel W. Burdick, and Aaron D. Ames
IEEE Conference on Robotics and Automation (ICRA), 2020
(*equal contribution)
PDF     Video     Code     Website
Received Best Conference Paper Award and Best Paper in Human-Robot Interaction Award.

Modeling Motor Responses of Paraplegics under Epidural Spinal Cord Stimulation
Ellen Feldman and Joel W. Burdick
IEEE Conference on Neural Engineering (NER), 2017
PDF

Towards Robot-Assisted Vitreoretinal Surgery: Force-Sensing Micro-Forceps Integrated with a Handheld Micromanipulator
Berk Gonenc, Ellen Feldman, Peter Gehlbach, James Handa, Russell H. Taylor, and Iulian Iordachita
IEEE Conference on Robotics and Automation (ICRA), 2014
PDF     Video

Workshop Publications:

DIP-RL: Demonstration-Inferred Preference Learning in Minecraft
Ellen Novoseller, Vinicius Goecks, David Watkins, Joshua Miller, and Nicholas Waytowich
Workshop on The Many Facets of Preference Learning, International Conference on Machine Learning (ICML), 2023
PDF     Website

Imitation Learning with Human Eye Gaze via Multi-Objective Prediction
Ravi Kumar Thakur*, MD-Nazmus Samin Sunbeam*, Vinicius Goecks, Ellen Novoseller, Ritwik Bera, Vernon Lawhern, Gregory Gremillion, John Valasek, and Nicholas Waytowich
Workshop on Interactive Learning with Implicit Human Feedback, International Conference on Machine Learning (ICML), 2023
PDF     Website     Code

Rating-based Reinforcement Learning
Devin White, Mingkang Wu, Ellen Novoseller, Vernon Lawhern, Nicholas Waytowich, and Yongcan Cao
Workshop on The Many Facets of Preference Learning, International Conference on Machine Learning (ICML), 2023
PDF     Code

Efficient Preference-Based Reinforcement Learning using Learned Dynamics Models
Yi Liu, Gaurav Datta, Ellen Novoseller, and Daniel S. Brown
Workshop on Human in the Loop Learning, Neural Information Processing Systems (NeurIPS), 2022
PDF     Website     Code

Preference-Based Bayesian Optimization in High Dimensions with Human Feedback
Myra Cheng, Ellen Novoseller, Maegan Tucker, Richard Cheng, Joel W. Burdick, and Yisong Yue
Workshop on Real World Experiment Design and Active Learning, International Conference on Machine Learning (ICML), 2020
PDF     Code

Dueling Posterior Sampling for Preference-Based Reinforcement Learning
Ellen Novoseller, Yanan Sui, Yisong Yue, and Joel W. Burdick
Workshop on Real-world Sequential Decision Making: Reinforcement Learning and Beyond, International Conference on Machine Learning (ICML), 2019
PDF