Blog
December 19, 2025
Reverse Engineering a Phase Change in GPT's Training Data... with the Seahorse Emoji
An investigation into how GPT's training data evolves over time, traced through an unlikely signal.
SubstackNovember 27, 2024
Peeking Behind Closed Doors: Risks of LLM Evaluation by Private Data Curators
A critical examination of the risks and challenges posed by private evaluators in the LLM landscape, highlighting financial incentives, conflicts of interest, and prevalence of evaluation biases even when acting in good faith.
ICLR 2025 BlogpostNovember 26, 2024
Reassessing EMNLP 2024's Best Paper: Does Divergence-Based Calibration for MIAs Hold Up?
A critical look at whether divergence-based calibration methods for membership inference attacks hold up under scrutiny. Co-authored with Anshuman Suri.
ICLR 2025 BlogpostSeptember 14, 2023
Phi-1.5 Model: A Case of Comparing Apples to Oranges?
Exploring Microsoft's Phi-1.5 claims and finding the model performs significantly worse than equal-sized counterparts on perplexity. A new "slang" understanding task shows Falcon-RW-1B performs 40% better.
LLM Analysis