LLM Training with Huggingface

Please take a look at the recipe for LLM training with Hugging Face in its dedicated repository:

https://gitlab.jsc.fz-juelich.de/sdlaml/hf_llm

This recipe provides a guide for setting up the environment, preparing datasets, and conducting distributed training within the Hugging Face ecosystem. It also outlines current limitations. Included are a single training script and an sbatch script, with all configurations managed through Hydra.