# Quickstart ## πŸ“‹ Prerequisites Let's start by installing Oumi OSS. You can easily install the latest stable version of Oumi OSS with the following commands: ```bash pip install oumi # Optional: If you have an Nvidia or AMD GPU, you can install the GPU dependencies pip install oumi[gpu] ``` If you need help setting up your environment (python, pip, git, etc), you can find detailed instructions in the {doc}`/development/dev_setup` guide. The {doc}`installation guide ` offers more details on how to install Oumi OSS for your specific environment and use case. ## πŸ‘‹ Introduction Now that we have Oumi OSS installed, let's get started with the basics! We're going to use the `oumi` command-line interface (CLI) to train, evaluate, and run inference with a model. We'll use a small model (`SmolLM-135M`) so that the examples can run fast on both CPU and GPU. `SmolLM` is a family of state-of-the-art small models with 135M, 360M, and 1.7B parameters, trained on a new high-quality dataset. You can learn more about about them in [this blog post](https://huggingface.co/blog/smollm). For a full list of recipes, including larger models like Llama 3.2, you can explore the {doc}`recipes page `. ## πŸ’» Oumi OSS CLI The general structure of Oumi OSS CLI commands is: ```bash oumi [options] ``` For detailed help on any command, you can use the `--help` option: ```bash oumi --help # for general help oumi --help # for command-specific help ``` The available commands are: | Command | Description | |---------|-------------| | `train` | Train or fine-tune models | | `evaluate` | Evaluate models on benchmarks | | `infer` | Run model inference | | `launch` | Launch jobs on cloud platforms | | `judge` | Use LLM-as-a-Judge for evaluation | | `synth` | Generate synthetic training data | | `analyze` | Analyze and profile datasets | | `tune` | Hyperparameter tuning | | `quantize` | Quantize models for deployment | | `distributed` | Distributed training utilities | | `env` | Display environment information | Let's go through some examples of the core commands. ## πŸ“š Training You can quickly start training a model using any of existing {doc}`recipes ` or your own {doc}`custom configs `. The following command will start training using the recipe in `configs/recipes/smollm/sft/135m/quickstart_train.yaml`: ````{dropdown} configs/recipes/smollm/sft/135m/quickstart_train.yaml ```{literalinclude} ../../configs/recipes/smollm/sft/135m/quickstart_train.yaml :language: yaml ``` ```` ```bash oumi train -c configs/recipes/smollm/sft/135m/quickstart_train.yaml ``` Any Oumi OSS command which takes a config path as an argument (`train`, `evaluate`, `infer`, etc.) can override parameters from the command line. See {doc}`/cli/commands` for more details. For example: ```bash oumi train -c configs/recipes/smollm/sft/135m/quickstart_train.yaml \ --training.max_steps 20 \ --training.learning_rate 1e-4 \ --data.train.datasets[0].shuffle true \ --training.output_dir output/smollm-135m-sft ``` To run the same recipe on your own dataset (e.g., in our supported JSON or JSONL formats), you can override the dataset name and path. You can try this functionality out by downloading the `alpaca_cleaned` dataset manually via the huggingface CLI, then including that local path in your run. ```bash hf download yahma/alpaca-cleaned --repo-type dataset --local-dir /path/to/local/dataset oumi train -c configs/recipes/smollm/sft/135m/quickstart_train.yaml \ --data.train.datasets "[{dataset_name: text_sft, dataset_path: /path/to/local/dataset}]" \ --training.output_dir output/smollm-135m-sft-custom ``` You can also train on multiple GPUs (make sure to [install the GPU dependencies](/get_started/installation.md#optional-dependencies) if not already installed). For example, if you have a machine with 4 GPUs, you can run this command to launch a local distributed training run: ```bash oumi distributed torchrun \ -m oumi train -c configs/recipes/smollm/sft/135m/quickstart_train.yaml \ --training.output_dir output/smollm-135m-sft-dist ``` You can also use torchrun directly in standalone mode. ```bash torchrun --standalone --nproc-per-node 4 --log-dir ./logs \ -m oumi train -c configs/recipes/smollm/sft/135m/quickstart_train.yaml \ --training.output_dir output/smollm-135m-sft-dist ``` ## πŸ“Š Evaluation To evaluate a trained model: ````{dropdown} configs/recipes/smollm/evaluation/135m/quickstart_eval.yaml ```{literalinclude} ../../configs/recipes/smollm/evaluation/135m/quickstart_eval.yaml :language: yaml ``` ```` Using a model downloaded from HuggingFace: ```bash oumi evaluate -c configs/recipes/smollm/evaluation/135m/quickstart_eval.yaml \ --model.model_name HuggingFaceTB/SmolLM2-135M-Instruct ``` Or, with our newly trained model saved on disk: ```bash oumi evaluate -c configs/recipes/smollm/evaluation/135m/quickstart_eval.yaml \ --model.model_name output/smollm135m.fft ``` If you saved your model to a different directory such as `output/smollm-135m-sft-dist`, you need only change `--model.model_name`. To explore the benchmarks that our evaluations support, including HuggingFace leaderboards and AlpacaEval, visit our {doc}`evaluation guide `. ## 🧠 Inference To run inference with a trained model: ````{dropdown} configs/recipes/smollm/inference/135m_infer.yaml ```{literalinclude} ../../configs/recipes/smollm/inference/135m_infer.yaml :language: yaml ``` ```` Using a model downloaded from HuggingFace: ```bash oumi infer -c configs/recipes/smollm/inference/135m_infer.yaml \ --generation.max_new_tokens 40 \ --generation.temperature 0.7 \ --interactive ``` Or, with our newly trained model saved on disk: ```bash oumi infer -c configs/recipes/smollm/inference/135m_infer.yaml \ --model.model_name output/smollm135m.fft \ --generation.max_new_tokens 40 \ --generation.temperature 0.7 \ --interactive ``` To learn more about running inference locally or remotely (including OpenAI, Google, Anthropic APIs) and leveraging inference engines to parallelize and speed up your jobs, visit our {doc}`inference guide `. ## ☁️ Launching Jobs in the Cloud So far we have been using Oumi OSS locally. But one of the most exciting and unique Oumi OSS features, compared to similar frameworks, is its integrated ability to launch jobs directly *to the cloud* (GCP, AWS, Azure, etc). This section of the quickstart is going to be a little different than the others, so please read this next bit carefully before you proceed. * This tutorial uses GCP; you'll need a [GCP account](https://cloud.google.com/free?hl=en). You can also use other cloud providers, such as AWS, Azure, etc. See {doc}`running jobs remotely ` for more details. ````{dropdown} Configuring GCP Account * In particular, Oumi OSS uses [Skypilot](https://docs.skypilot.co/en/latest/docs/index.html), and the recommended way to use SkyPilot and GCP is with a [GCP service account](https://cloud.google.com/iam/docs/service-account-overview) * You will need to install Oumi OSS with GCP support: `pip install oumi[gcp]`. Please note that we recommend setting up a different environment for each cloud provider you wish to use. * Depending on your precise use case, you may also need to install a few other packages from Google ```bash conda install -c conda-forge google-cloud-sdk -y conda install -c conda-forge google-api-python-client -y conda install -c conda-forge google-cloud-storage -y ``` * There are multiple ways to handle credentials with GCP service accounts. We recommend creating a service account key in JSON format, then downloading it to the machine from which you plan to launch the cloud job. After that, you'll need to run a few more setup commands. ```bash export GOOGLE_APPLICATION_CREDENTIALS=/path/to/key.json gcloud auth activate-service-account --key-file=$GOOGLE_APPLICATION_CREDENTIALS gcloud config set project ``` You can now run `sky check` to confirm GCP is enabled. If you get stuck, please refer to our {doc}`running jobs remotely ` section, as well as the documentation for GCP and SkyPilot linked above, for more information. ```` ### Launching your first cloud job with Oumi OSS Once the one-time setup is out of the way, launching a new cloud job with Oumi OSS is very simple. ````{dropdown} configs/recipes/smollm/sft/135m/quickstart_gcp_job.yaml ```{literalinclude} ../../configs/recipes/smollm/sft/135m/quickstart_gcp_job.yaml :language: yaml ``` ```` ```bash oumi launch up -c configs/recipes/smollm/sft/135m/quickstart_gcp_job.yaml ``` To launch an evaluation job: ````{dropdown} configs/recipes/smollm/evaluation/135m/quickstart_gcp_job.yaml ```{literalinclude} ../../configs/recipes/smollm/evaluation/135m/quickstart_gcp_job.yaml :language: yaml ``` ```` ```bash oumi launch up -c configs/recipes/smollm/evaluation/135m/quickstart_gcp_job.yaml ``` After you run one of the above commands, you should see some console output from Oumi OSS which describes how your job is being provisioned and how the cloud installation is proceeding. In particular, your cluster will be assigned a semi-random name such as `sky-7fdd-ab183`, which you should take note of. After 15 minutes or so, Oumi OSS should tell you that the run is complete. If you want to see the logs from your cloud run, you can pull them down to your local machine -- ```bash sky logs --sync-down sky-7fdd-ab183 ``` **Cloud services can be expensive!** Please keep an eye on your costs, and don't forget to tear down your cluster when you're done with this tutorial. ```bash sky down sky-7fdd-ab183 ``` This command will **destroy your cluster**, including all data on those remote machines, so save your logs and artifacts first! ### 🧭 What's next? Although this example used GCP, Oumi OSS natively supports a wide range of cloud providers. To explore the Cloud providers that we support, visit {doc}`running jobs remotely `. ## πŸ”— Community ⭐ If you like Oumi OSS and you would like to support it, please give it a star on [GitHub](https://github.com/oumi-ai/oumi). πŸ‘‹ If you are interested in contributing, please read the [Contributor’s Guide](/development/contributing).