Lihe Li

Hi there, thanks for visiting my website! I am a first-year M.Sc. student of School of Artificial Intelligence in Nanjing University and a member of LAMDA Group, led by professor Zhi-Hua Zhou. I received my B.Sc. degree of Engineering from School of Artificial Intelligence, Nanjing University in June 2023. In September 2023, I was admitted to pursue a M.Sc. degree in Nanjing University, under the supervision of Professor Yang Yu without entrance examination.

Currently my research interest is Reinforcement Learning (RL), especially in Multi-agent Reinforcement Learning (MARL).

Please feel free to drop me an Email for any form of communication or collaboration!

Email:  lilh [at] lamda [dot] nju [dot] edu [dot] cn  /

  lilhzq76 [at] gmail [dot] com

CV  /  Google Scholar  /  Semantic Scholar  /  DBLP  /  Github  /  Twitter

profile photo

Just a reminder, I am the guy on the left.

News
Publications [ Google Scholar ]
2024
core3 Continual Multi-Objective Reinforcement Learning via Reward Model Rehearsal
Lihe Li, Ruotong Chen, Ziqian Zhang, Zhichao Wu, Yi-Chen Li, Cong Guan, Yang Yu, Lei Yuan
The 33rd International Joint Conference on Artificial Intelligence (IJCAI), 2024
pdf / poster / bibtex

We study the problem of multi-objective reinforcement learning (MORL) with continually evolving learning objectives, and propose CORe3 to enable the MORL agent rapidly learn new objectives and avoid catastrophic forgetting about old objectives lacking reward signals.

haplan Efficient Human-AI Coordination via Preparatory Language-based Convention
Cong Guan, Lichao Zhang, Chunpeng Fan, Yi-Chen Li, Feng Chen, Lihe Li, Yunjia Tian, Lei Yuan, Yang Yu
The 12th International Conference on Learning Representations (ICLR), Workshop on Large Language Model (LLM) Agents, 2024
pdf / link / bibtex

We propose employing the large language models (LLMs) to develop an action plan (or equivalently, a convention) that effectively guides both human and AI for coordination.

costa Cost-aware Offline Safe Meta Reinforcement Learning with Robust In-Distribution Online Task Adaptation
Cong Guan, Ruiqi Xue, Ziqian Zhang, Lihe Li, Yi-Chen Li, Lei Yuan, Yang Yu
The 23rd International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS), 2024
pdf / poster / bibtex

We propose COSTA to deal with offline safe meta RL problems. We develope a cost-aware task inference module using contrastive learning to distinguish tasks based on safety constraints, and propose a safe in-distribution online adapation mechanism.

2023
survey A Survey of Progress on Cooperative Multi-agent Reinforcement Learning in Open Environment
Lei Yuan, Ziqian Zhang, Lihe Li, Cong Guan, Yang Yu
arxiv preprint 2023
pdf in English / pdf in Chinese / link / bibtex

We review multi-agent cooperation from closed environment to open environment settings, and provide prospects for future development and research directions of cooperative MARL in open environments.

macop Learning to Coordinate with Anyone
Lei Yuan, Lihe Li, Ziqian Zhang, Feng Chen, Tianyi Zhang, Cong Guan, Yang Yu, Zhi-Hua Zhou
The Fifth Distributed Artificial Intelligence Conference (DAI), Best Paper Award , 2023
pdf / link / English talk / Chinese talk / bibtex

We propose Multi-agent Compatible Policy Learning (MACOP), where we adopt an agent-centered teammate generation process that gradually and efficiently generates diverse teammates covering the teammate policy space, and we use continual learning to train the ego agents to coordinate with them and acquire strong coordination ability.

fastap Fast Teammate Adaptation in the Presence of Sudden Policy Change
Ziqian Zhang, Lei Yuan, Lihe Li, Ke Xue, Chengxing Jia, Cong Guan, Chao Qian, Yang Yu
The 39th Conference on Uncertainty in Artificial Intelligence (UAI), 2023
pdf / link / poster / bibtex

We formulate Open Dec-POMDP and propose Fast teammate adaptation (Fastap) to enable controllable agents in a multi-agent system to fast adapt to the uncontrollable teammates, whose policy could be changed with one episode.

romance Robust Multi-agent Coordination via Evolutionary Generation of Auxiliary Adversarial Attackers
Lei Yuan, Ziqian Zhang, Ke Xue, Hao Yin, Feng Chen, Cong Guan, Lihe Li, Chao Qian, Yang Yu
The 37th AAAI Conference on Artificial Intelligence (AAAI), Oral Presentation, 2023
pdf / link / code / poster / bibtex

We formulate Limited Policy Adversary Dec-POMDP and propose ROMANCE to enable the trained agents to encounter diversified and strong auxiliary adversarial attacks during training, achieving high robustness under various policy perturbations.

cromac Robust Multi-agent Communication via Multi-view Message Certification
Lei Yuan, Tao Jiang, Lihe Li, Feng Chen, Zongzhang Zhang, Yang Yu
Science China Information Sciences (SCIS)
pdf / link / code / poster / bibtex

We propose CroMAC to enable agents to obtain guaranteed lower bounds on state-action values to identify and choose the optimal action under a worst-case deviation when the received messages are perturbed.

macpro Multi-agent Continual Coordination via Progressive Task Contextualization
Lei Yuan, Lihe Li, Ziqian Zhang, Fuxiang Zhang, Cong Guan, Yang Yu
IEEE Transactions on Neural Networks and Learning Systems (TNNLS)
pdf / link / code / poster / bibtex

We formulate the continual coordination framework and propose MACPro to enable agents to continually coordinate with each other when the dynamic of the training task and the multi-agent system itself changes over time.

Education
Nanjing University
2023.09 - present

M.Sc. in Computer Science and Technology
Advisor: Prof. Yang Yu
Nanjing University
2019.08 - 2023.07

B.E. in Artificial Intelligence
Advisor: Prof. Yang Yu
Awards & Honors
  • Best Paper Award of The Fifth Distributed Artificial Intelligence Conference (DAI), 2023
  • Outstanding Bachelor's Thesis of Nanjing University, 2023
  • Outstanding Graduate of Nanjing University, 2023
Miscellaneous
  • I have the fortune to work with brilliant mentors, collaborators, and advisors during my research journey and I am truly grateful for their guidance and help!
  • My Chinese name is 李立和 (Li Lihe), which can be pronounced as /liː ˈliː hɜː/ in Mandarin or /lei ˈlʌb wɔː/ in Cantonese.
  • I enjoy singing🎤 and I am a Tenor of the Nanjing University Chorus🎼. I was even awarded as an Outstanding Person in my second semester in the chorus😆!
  • I also enjoy working out, like going to the gym💪 and playing basketball🏀.
  • This website template was "stolen" from my good friend Zhaoxuan. Appreciate that🫡.