Pratyush is a Ph.D. student in the Machine Learning Department at Carnegie Mellon University, and a founding member of DatologyAI. In his work, he has developed scalable and performant methods for improving the quality of data that we train machine learning models on. He has also developed methods that allow us to evaluate, locate, and mitigate the memorization of data points by neural networks. His works have been recognized through a best paper award nomination at NeurIPS, and multiple oral and spotlight talks at important ML conferences.

@inproceedings{goyal2024scaling,
  title={Scaling Laws for Data Filtering—Data Curation cannot be Compute Agnostic},
  author={Goyal, Sachin and Maini, Pratyush and Lipton, Zachary C and Raghunathan, Aditi and Kolter, J Zico},
  booktitle={CVPR2024},
  year={2024}
}

@inproceedings{maini2024rephrasing,
  title={Rephrasing the Web: A Recipe for Compute & Data-Efficient Language Modeling},
  author={Maini, Pratyush and Seto, Skyler and Bai, He and Grangier, David and Zhang, Yizhe and Jaitly, Navdeep},
  booktitle={arXiv},
  year={2024}
}

@misc{tofu2024,
      title={TOFU: A Task of Fictitious Unlearning for LLMs}, 
      author={Pratyush Maini and Zhili Feng and Avi Schwarzschild and Zachary C. Lipton and J. Zico Kolter},
      year={2024},
      archivePrefix={arXiv},
      primaryClass={cs.LG}
}

1. We show that memorization is typically not localized to specific model layers, rather is confined to a small fraction of neurons dispersed across the model.
2. We propose Example-Tied Dropout that can confine memorization to a pre-defined set of neurons, which can then be thrown away at test time.

@inproceedings{maini2023memorization,
  title={Can Neural Network Memorization Be Localized?},
  author={Maini, Pratyush and Mozer, Michael C and Sedghi, Hanie and Lipton, Zachary C and Kolter, J Zico and Zhang, Chiyuan},
  booktitle={International Conference on Machine Learning},
  year={2023}
}

@article{maini2023tmars,
  title={T-MARS: Improving Visual Representations by Circumventing Text Feature Learning},
  author={Maini, Pratyush and Goyal, Sachin and Lipton, Zachary C and Kolter, J Zico and Raghunathan, Aditi},
  booktitle={Arxiv},
  year={2023}
}

1. We analyze the forgetting and learning dynamics of neural networks to characterize different types of hard examples as belonging to mislabeled, rare and complex categories.
2. Mislabeled Examples : Learnt Late, Forgotten Early
3. Rare Examples: Learnt Late, Forgotten Late
4. Complex Examples: Learnt Late, Never Forgotten

1. We analyze the forgetting and learning dynamics of neural networks to characterize different types of hard examples as belonging to mislabeled, rare and complex categories.
2. Mislabeled Examples : Learnt Late, Forgotten Early
3. Rare Examples: Learnt Late, Forgotten Late
4. Complex Examples: Learnt Late, Never Forgotten

@inproceedings{
	maini2022characterizing,
	title={Characterizing Datapoints via Second-Split Forgetting},
	author={Pratyush Maini and Saurabh Garg and Zachary Chase Lipton and J Zico Kolter},
	booktitle={Advances in Neural Information Processing Systems},
	editor={Alice H. Oh and Alekh Agarwal and Danielle Belgrave and Kyunghyun Cho},
	year={2022},
	url={https://openreview.net/forum?id=yKDKNzjHg8N}
}

1. Dataset Inference (DI) resolves model ownership without the need for retraining; and does not have a trade-off with task accuracy.
2. We prove that the success of Membership Inference decreases as overfitting reduces, whereas DI is independent of the same.
3. We introduce a new method for black-box ownership resolution that requires less than 50 private training points from the victim’s dataset.

@article{maini2021dataset,
	title={Dataset Inference: Ownership Resolution in Machine Learning},
	author={Pratyush Maini and Mohammad Yaghini and Nicolas Papernot},
	booktitle={ICLR 2021},
	year={2020},
	url={https://openreview.net/forum?id=hvdKKV2yt7T},
	note={Spotlight at ICLR 2021}
}

@article{truong2021data,
	title={Data-Free Model Extraction},
	author={Jean-Baptiste Truong* and Pratyush Maini* and Robert J. Walls and Nicolas Papernot},
	booktitle={arXiv preprint arXiv:2011.14779},
	year={2021},
	url={https://arxiv.org/abs/2011.14779},
	note={under review at CVPR 2021},
}

@InProceedings{maini2022perturbation,
  title = 	 {Perturbation Type Categorization for Multiple $\ell_p$ Bounded Adversarial Robustness},
  author =       {Pratyush Maini and Xinyun Chen and Bo Li and Dawn Song},
  booktitle = 	 {Proceedings of The 38th Uncertainty in Artificial Intelligence Conference},
  year = 	 {2022},
  series = 	 {Proceedings of Machine Learning Research},
  url={https://openreview.net/pdf?id=Oe2XI-Aft-k},
}

@inproceedings{maini2020adversarial,
	title={Adversarial Robustness Against the Union of Multiple Perturbation Models}, 
	author={Pratyush Maini and Eric Wong and J. Zico Kolter},
	booktitle={International Conference on Machine Learning},
	year={2020},
	url = "https://arxiv.org/abs/1909.04068"
}

1. Pooling (and attention) help improve learning ability and positional invariance of BiLSTMs.
2. Pooling helps improve sample efficiency (low-resource settings) and is particularly beneficial when important words lie away from the end of the sentence.
3. Our proposed pooling technique, max-attention (MaxAtt), helps improve upon past approaches on standard accuracy metrics, and is more robust to distribution shift.

@inproceedings{maini2020pool,
	title = "Why and when should you pool? Analyzing Pooling in Recurrent Architectures",
	author = "Maini, Pratyush and Kolluru, Keshav and Pruthi, Danish and {Mausam}",
	booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2020",
	year = "2020",
	address = "Online",
	publisher = "Association for Computational Linguistics",
	url = "https://www.aclweb.org/anthology/2020.findings-emnlp.410",
	note = {Also presented at BlackBoxNLP'20}
}