Jeff Cheng

Master’s Student at Johns Hopkins CLSP

Personal Background

I am a second year master’s student at the Center for Language and Speech Processing (CLSP) at Johns Hopkins University. I graduated from Duke University in 2023 with a double major in Computer Science and Mathematics. I am applying to NLP PhD programs for Fall 2025.

Research Interests

I am broadly interested in language models and exploring their capabilities and limitations. Language models are the foundation of complex systems such as AI agents and multi-agent systems. As such, I believe we should (try our best to) understand the behavior of these models. In particular, I am interested in the below problems:

  • Data: What has a language model seen during training? How does pretraining data influence language models and can we attribute content generated by models back to their pretraining corpus? How can we correct misalignments arising from knowledge conflicts in models’ pretraining data?
  • Efficient Language Models: How do we ameliorate the quadratic computational complexity of attention? How much of the key-value cache is really needed for models to maintain performance? Can we make language models more efficient by shifting away from a discrete token space and perform reasoning in continuous latent space?

News

Dec 2024New Preprint out! Compressed Chain of Thought: Efficient Reasoning through Dense Representations
Oct 2024Attending CoLM 2024 and presenting Dated Data (Outstanding Paper Award)
Jul 2024Dated Data accepted to CoLM 2024