Shikhar Murty
Stanford, CA
I am currently a 5th year CS PhD Student in the Stanford NLP Group, advised by Chris Manning.
I’m interested in building learning systems that can learn rich structures from limited data, and generalize (2, 3) beyond what they were trained on. These days, I’m thinking a lot about long-horizon planning and reasoning in Language Models.
I interned at Google Deepmind in Summer 2023 where I worked with Mandar Joshi, Kenton Lee and Pete Shaw. In Summer 2021, I interned with Marco Ribeiro and Scott Lundberg at Microsoft Research.
News
February, 2024 | Talks in NYC (NYU / Columbia / Cornell) on “Improving the Structure and Interpretation of Language in Modern Sequence Models” |
---|---|
September, 2023 | Talks at AI2 + UW titled “Understanding and Improving Generalization in Transformers” |
May, 2023 | Talk at MIT BCS titled “Transformers, Tree Structures and Generalization” |
May, 2022 | Talk about Language Patching at the NL Supervision Workshop at ACL 2022 in Dublin, in person! |
February, 2022 | Talk about using Language Supervision in ML (1, 2) at Apple Siri |
Selected publications
2023
- Pushdown Layers: Encoding Recursive Structure in Transformer Language ModelsIn Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Dec 2023
- Characterizing intrinsic compositionality in transformers with Tree ProjectionsIn The Eleventh International Conference on Learning Representations , Dec 2023
- Grokking of Hierarchical Structure in Vanilla TransformersIn Annual Meeting of the Association for Computational Linguistics, Dec 2023
2022
- Fixing Model Bugs with Natural Language PatchesIn Conference on Empirical Methods in Natural Language Processing, Dec 2022
2020
- ExpBERT: Representation Engineering with Natural Language ExplanationsIn Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Jul 2020
2019
- Systematic Generalization: What Is Required and Can It Be Learned?In International Conference on Learning Representations, Jul 2019