Microsoft found that small language models can exceed the performance of much larger ones when trained to specialize in a single area. Researchers fine-tuned the Mistral 7B model to create Orca ...

Extending learning by using 2 x 2 digit mutliplication with partial products & area models ...