Back: Prepare dataset

Install the packages

Note1. Reminder: I encourage you to make a new virtual environment for a new project. If you don’t remember how to do this, please check here.

Note2. Reminder: Installing packages should be done in your terminal, NOT in your python script.

Run the following command in a terminal or command prompt:

python -m pip install torch transformers datasets accelerate scikit-learn

Note3. If you are using Google Colab, run this command in a notebook code cell instead:

!pip install torch transformers datasets accelerate scikit-learn

Note4. Training can run on a CPU, but a GPU will usually make it faster. In Colab, you can enable GPU acceleration by following these steps:

  1. Go to Runtime → Change runtime type in the top menu.
  2. Under Hardware accelerator, select T4 GPU (or another available GPU).
  3. Click Save. To verify that the GPU is available, run the following code in a notebook cell:
    import torch
    print(torch.cuda.is_available())  # Should return True
    

We will use:

  • transformers to load and train a pretrained language model;
  • datasets to organize the training data;
  • torch to run the model;
  • accelerate to support the training process;
  • scikit-learn to calculate evaluation scores.

Next: Ready for training