ML Research Engineer Internship, SmolLMs pretraining and datasets - US Remote

At Hugging Face, we’re on a journey to democratize good AI. We are building the fastest growing platform for AI builders with over 5 million users & 100k organizations who collectively shared over 1M models, 300k datasets & 300k apps. Our open-source libraries have more than 400k+ stars on Github.

About the Role

Smol models are an exciting area of research as they enable cheaper inference and can be run on-device allowing for more customization and ensuring privacy. The SmolLM team at Hugging Face is pushing the frontier of smol models by building high quality pre-training and post-training datasets [1,2], and applying the latest architecture and training techniques to develop state-of-the-art models [2,3]. The dataset processing can leverage our scalable CPU cluster and the models are trained on a state-of-the-art H100 cluster with close to 100 nodes.

In this internship you will work alongside the SmolLM team and work towards...