Use the vitals package with ellmer to evaluate and compare the accuracy of LLMs, including writing evals to test local models.
Generating structured JSON from language models is a challenging task. The generated JSON must be syntactically correct, and it must conform to a schema that specifies the structure of the JSON.
Some results have been hidden because they may be inaccessible to you
Show inaccessible results