A2 · Training Your Own Small Model
Judging It Against the Giant
Compare two models on the same held-back set honestly, and put the cost of an answer beside the score.
Your model scores 91.7. The one it replaces scores 91.3. On three hundred messages that gap can be produced by the test set alone. It is not a result yet.