Experiments
Development macro-F1 chose char_balanced, and the test column was scored after that choice. The desk files a piece when its top score is at least 57.4%.
| System | Dev macro-F1 | Test accuracy | Test macro-F1 |
|---|---|---|---|
| majority | 0.027 | 0.091 | 0.012 |
| word | 0.545 | 0.690 | 0.428 |
| char | 0.622 | 0.697 | 0.456 |
| char_balanced | 0.692 | 0.734 | 0.583 |
| probe | 0.239 | 0.520 | 0.244 |
| full | 0.477 | 0.665 | 0.392 |
| full_weighted | 0.488 | 0.674 | 0.435 |
On the test set the desk files 73.8% of articles and is right on 84.7% of those.