Skip to content

Models and SDG

Creating fine tuned models with various techniques and Synthetic Data Generation

Models

To create our own specialist LLMs as Judge we can either use In Context Learning - LLMs with many examples in the prompt - or fine tune models.

We can create smaller and more accurate models using techniques listed below. I am currently creating notebooks and content about this:

  • Supervised Fine Tuning where we give ground truth input/output pairs.
  • Direct Preference Optimisation where give grount truth input-chosen-rejected pairs.
  • Distillation where a larger model teaches a smaller model that can even outperform the teacher.

As a result, we have powerful tools for our evaluation pipeline.

SDG

Synthetic Data Generation enables us to take some seed examples and generate synthetic data. These will need human evaluation.