BEA 2019 Shared Task: Grammatical Error Correction

OVERALL
	TP	FP	FN	P	R	F_0.5
Span-level correction	1443	884	3175	62.01	31.25	51.81
Span-level detection	1709	620	3119	73.38	35.40	60.41
Token-level detection	2045	491	3733	80.64	35.39	64.22

SPAN-LEVEL CORRECTION
OPERATION LEVEL
	TP	FP	FN	P	R	F_0.5
M	508	237	722	68.19	41.30	60.33
R	795	550	2146	59.11	27.03	47.77
U	140	97	307	59.07	31.32	50.18

ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
ADJ	0	0	33	100.00	0.00	0.00
ADJ:FORM	3	7	5	30.00	37.50	31.25
ADV	1	1	44	50.00	2.22	9.43
CONJ	0	0	16	100.00	0.00	0.00
CONTR	0	0	9	100.00	0.00	0.00
DET	167	101	387	62.31	30.14	51.35
MORPH	68	32	83	68.00	45.03	61.71
NOUN	4	2	75	66.67	5.06	19.42
NOUN:INFL	12	2	7	85.71	63.16	80.00
NOUN:NUM	92	52	134	63.89	40.71	57.36
NOUN:POSS	19	35	30	35.19	38.78	35.85
ORTH	119	86	224	58.05	34.69	51.16
OTHER	6	8	660	42.86	0.90	4.16
PART	4	6	17	40.00	19.05	32.79
PREP	71	49	311	59.17	18.59	41.18
PRON	3	1	110	75.00	2.65	11.63
PUNCT	515	278	310	64.94	62.42	64.42
SPELL	216	149	105	59.18	67.29	60.64
VERB	0	0	194	100.00	0.00	0.00
VERB:FORM	66	40	108	62.26	37.93	55.18
VERB:INFL	0	0	8	100.00	0.00	0.00
VERB:SVA	77	33	55	70.00	58.33	67.31
VERB:TENSE	0	0	190	100.00	0.00	0.00
WO	0	2	60	0.00	0.00	0.00

OPERATION + ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
M:ADJ	0	0	5	100.00	0.00	0.00
M:ADV	1	1	13	50.00	7.14	22.73
M:CONJ	0	0	6	100.00	0.00	0.00
M:DET	49	20	191	71.01	20.42	47.48
M:NOUN	0	0	8	100.00	0.00	0.00
M:NOUN:POSS	1	7	13	12.50	7.14	10.87
M:OTHER	0	0	44	100.00	0.00	0.00
M:PART	0	0	6	100.00	0.00	0.00
M:PREP	0	0	85	100.00	0.00	0.00
M:PRON	0	0	59	100.00	0.00	0.00
M:PUNCT	448	204	227	68.71	66.37	68.23
M:VERB	0	0	29	100.00	0.00	0.00
M:VERB:FORM	9	5	4	64.29	69.23	65.22
M:VERB:TENSE	0	0	32	100.00	0.00	0.00
R:ADJ	0	0	24	100.00	0.00	0.00
R:ADJ:FORM	3	7	5	30.00	37.50	31.25
R:ADV	0	0	17	100.00	0.00	0.00
R:CONJ	0	0	5	100.00	0.00	0.00
R:CONTR	0	0	5	100.00	0.00	0.00
R:DET	5	1	90	83.33	5.26	21.01
R:MORPH	68	32	83	68.00	45.03	61.71
R:NOUN	4	2	58	66.67	6.45	23.26
R:NOUN:INFL	12	2	7	85.71	63.16	80.00
R:NOUN:NUM	92	52	134	63.89	40.71	57.36
R:NOUN:POSS	18	28	10	39.13	64.29	42.45
R:ORTH	119	86	224	58.05	34.69	51.16
R:OTHER	6	8	597	42.86	1.00	4.55
R:PART	4	6	7	40.00	36.36	39.22
R:PREP	71	49	167	59.17	29.83	49.44
R:PRON	0	0	37	100.00	0.00	0.00
R:PUNCT	43	58	56	42.57	43.43	42.74
R:SPELL	216	149	105	59.18	67.29	60.64
R:VERB	0	0	149	100.00	0.00	0.00
R:VERB:FORM	57	35	101	61.96	36.08	54.18
R:VERB:INFL	0	0	8	100.00	0.00	0.00
R:VERB:SVA	77	33	55	70.00	58.33	67.31
R:VERB:TENSE	0	0	142	100.00	0.00	0.00
R:WO	0	2	60	0.00	0.00	0.00
U:ADJ	0	0	4	100.00	0.00	0.00
U:ADV	0	0	14	100.00	0.00	0.00
U:CONJ	0	0	5	100.00	0.00	0.00
U:CONTR	0	0	4	100.00	0.00	0.00
U:DET	113	80	106	58.55	51.60	57.01
U:NOUN	0	0	9	100.00	0.00	0.00
U:NOUN:POSS	0	0	7	100.00	0.00	0.00
U:OTHER	0	0	19	100.00	0.00	0.00
U:PART	0	0	4	100.00	0.00	0.00
U:PREP	0	0	59	100.00	0.00	0.00
U:PRON	3	1	14	75.00	17.65	45.45
U:PUNCT	24	16	27	60.00	47.06	56.87
U:VERB	0	0	16	100.00	0.00	0.00
U:VERB:FORM	0	0	3	100.00	0.00	0.00
U:VERB:TENSE	0	0	16	100.00	0.00	0.00

SPAN-LEVEL DETECTION
OPERATION LEVEL
	TP	FP	FN	P	R	F_0.5
M	537	210	707	71.89	43.17	63.45
R	1014	321	2025	75.96	33.37	60.51
U	148	89	316	62.45	31.90	52.41
UNK	10	0	71	100.00	12.35	41.32

ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
ADJ	4	0	35	100.00	10.26	36.36
ADJ:FORM	3	7	8	30.00	27.27	29.41
ADV	5	1	41	83.33	10.87	35.71
CONJ	3	0	16	100.00	15.79	48.39
CONTR	0	0	10	100.00	0.00	0.00
DET	169	94	378	64.26	30.90	52.85
MORPH	70	25	82	73.68	46.05	65.79
NOUN	15	1	79	93.75	15.96	47.47
NOUN:INFL	14	2	7	87.50	66.67	82.35
NOUN:NUM	90	38	134	70.31	40.18	61.14
NOUN:POSS	24	27	29	47.06	45.28	46.69
ORTH	159	63	204	71.62	43.80	63.55
OTHER	80	6	654	93.02	10.90	37.11
PART	4	6	17	40.00	19.05	32.79
PREP	87	43	300	66.92	22.48	47.96
PRON	5	1	108	83.33	4.42	18.25
PUNCT	514	245	312	67.72	62.23	66.55
SPELL	281	17	35	94.30	88.92	93.17
UNK	10	0	71	100.00	12.35	41.32
VERB	18	0	189	100.00	8.70	32.26
VERB:FORM	66	28	103	70.21	39.05	60.55
VERB:INFL	3	0	6	100.00	33.33	71.43
VERB:SVA	75	14	58	84.27	56.39	76.69
VERB:TENSE	10	0	185	100.00	5.13	21.28
WO	0	2	58	0.00	0.00	0.00

OPERATION + ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
M:ADJ	0	0	5	100.00	0.00	0.00
M:ADV	4	1	12	80.00	25.00	55.56
M:CONJ	3	0	8	100.00	27.27	65.22
M:CONTR	0	0	1	100.00	0.00	0.00
M:DET	50	19	181	72.46	21.65	49.31
M:NOUN	0	0	9	100.00	0.00	0.00
M:NOUN:POSS	1	7	14	12.50	6.67	10.64
M:OTHER	8	0	48	100.00	14.29	45.45
M:PART	0	0	6	100.00	0.00	0.00
M:PREP	9	0	80	100.00	10.11	36.00
M:PRON	1	0	58	100.00	1.69	7.94
M:PUNCT	448	178	221	71.57	66.97	70.60
M:VERB	2	0	29	100.00	6.45	25.64
M:VERB:FORM	10	5	4	66.67	71.43	67.57
M:VERB:TENSE	1	0	31	100.00	3.12	13.89
R:ADJ	3	0	25	100.00	10.71	37.50
R:ADJ:FORM	3	7	8	30.00	27.27	29.41
R:ADV	0	0	16	100.00	0.00	0.00
R:CONJ	0	0	4	100.00	0.00	0.00
R:CONTR	0	0	5	100.00	0.00	0.00
R:DET	9	0	93	100.00	8.82	32.61
R:MORPH	70	25	82	73.68	46.05	65.79
R:NOUN	13	1	60	92.86	17.81	50.39
R:NOUN:INFL	14	2	7	87.50	66.67	82.35
R:NOUN:NUM	90	38	134	70.31	40.18	61.14
R:NOUN:POSS	22	20	8	52.38	73.33	55.56
R:ORTH	159	63	204	71.62	43.80	63.55
R:OTHER	71	6	581	92.21	10.89	36.98
R:PART	4	6	8	40.00	33.33	38.46
R:PREP	78	43	166	64.46	31.97	53.57
R:PRON	0	0	36	100.00	0.00	0.00
R:PUNCT	42	54	58	43.75	42.00	43.39
R:SPELL	281	17	35	94.30	88.92	93.17
R:VERB	14	0	140	100.00	9.09	33.33
R:VERB:FORM	56	23	97	70.89	36.60	59.70
R:VERB:INFL	3	0	6	100.00	33.33	71.43
R:VERB:SVA	75	14	58	84.27	56.39	76.69
R:VERB:TENSE	7	0	136	100.00	4.90	20.47
R:WO	0	2	58	0.00	0.00	0.00
U:ADJ	1	0	5	100.00	16.67	50.00
U:ADV	1	0	13	100.00	7.14	27.78
U:CONJ	0	0	4	100.00	0.00	0.00
U:CONTR	0	0	4	100.00	0.00	0.00
U:DET	110	75	104	59.46	51.40	57.65
U:NOUN	2	0	10	100.00	16.67	50.00
U:NOUN:POSS	1	0	7	100.00	12.50	41.67
U:OTHER	1	0	25	100.00	3.85	16.67
U:PART	0	0	3	100.00	0.00	0.00
U:PREP	0	0	54	100.00	0.00	0.00
U:PRON	4	1	14	80.00	22.22	52.63
U:PUNCT	24	13	33	64.86	42.11	58.54
U:VERB	2	0	20	100.00	9.09	33.33
U:VERB:FORM	0	0	2	100.00	0.00	0.00
U:VERB:TENSE	2	0	18	100.00	10.00	35.71
UNK	10	0	71	100.00	12.35	41.32

TOKEN-LEVEL DETECTION
OPERATION LEVEL
	TP	FP	FN	P	R	F_0.5
M	607	157	786	79.45	43.58	68.22
R	1243	257	2405	82.87	34.07	64.42
U	178	77	470	69.80	27.47	53.36
UNK	17	0	72	100.00	19.10	54.14

ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
ADJ	5	0	48	100.00	9.43	34.25
ADJ:FORM	3	6	6	33.33	33.33	33.33
ADV	8	0	59	100.00	11.94	40.40
CONJ	10	0	19	100.00	34.48	72.46
CONTR	0	0	14	100.00	0.00	0.00
DET	188	76	405	71.21	31.70	57.00
MORPH	72	20	87	78.26	45.28	68.31
NOUN	16	1	106	94.12	13.11	42.11
NOUN:INFL	14	2	6	87.50	70.00	83.33
NOUN:NUM	89	30	149	74.79	37.39	62.32
NOUN:POSS	35	27	34	56.45	50.72	55.21
ORTH	201	51	228	79.76	46.85	69.94
OTHER	200	5	935	97.56	17.62	51.15
PART	3	6	17	33.33	15.00	26.79
PREP	90	30	322	75.00	21.84	50.45
PRON	12	1	115	92.31	9.45	33.52
PUNCT	597	201	339	74.81	63.78	72.31
SPELL	284	6	35	97.93	89.03	96.01
UNK	17	0	72	100.00	19.10	54.14
VERB	33	0	237	100.00	12.22	41.04
VERB:FORM	67	16	114	80.72	37.02	65.30
VERB:INFL	4	0	4	100.00	50.00	83.33
VERB:SVA	73	13	67	84.88	52.14	75.41
VERB:TENSE	15	0	235	100.00	6.00	24.19
WO	9	0	80	100.00	10.11	36.00

OPERATION + ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
M:ADJ	1	0	8	100.00	11.11	38.46
M:ADV	6	0	17	100.00	26.09	63.83
M:CONJ	8	0	8	100.00	50.00	83.33
M:CONTR	0	0	1	100.00	0.00	0.00
M:DET	57	13	193	81.43	22.80	53.77
M:NOUN	2	0	16	100.00	11.11	38.46
M:NOUN:POSS	4	5	13	44.44	23.53	37.74
M:OTHER	14	0	59	100.00	19.18	54.26
M:PART	0	0	6	100.00	0.00	0.00
M:PREP	13	0	87	100.00	13.00	42.76
M:PRON	5	0	60	100.00	7.69	29.41
M:PUNCT	477	139	236	77.44	66.90	75.07
M:VERB	7	0	42	100.00	14.29	45.45
M:VERB:FORM	10	0	5	100.00	66.67	90.91
M:VERB:TENSE	3	0	35	100.00	7.89	30.00
R:ADJ	4	0	32	100.00	11.11	38.46
R:ADJ:FORM	3	6	6	33.33	33.33	33.33
R:ADV	0	0	19	100.00	0.00	0.00
R:CONJ	2	0	6	100.00	25.00	62.50
R:CONTR	0	0	7	100.00	0.00	0.00
R:DET	12	0	90	100.00	11.76	40.00
R:MORPH	72	20	87	78.26	45.28	68.31
R:NOUN	12	1	73	92.31	14.12	43.80
R:NOUN:INFL	14	2	6	87.50	70.00	83.33
R:NOUN:NUM	89	30	149	74.79	37.39	62.32
R:NOUN:POSS	30	22	10	57.69	75.00	60.48
R:ORTH	201	51	228	79.76	46.85	69.94
R:OTHER	174	5	778	97.21	18.28	52.16
R:PART	3	6	8	33.33	27.27	31.91
R:PREP	77	30	174	71.96	30.68	56.70
R:PRON	2	0	37	100.00	5.13	21.28
R:PUNCT	86	49	65	63.70	56.95	62.23
R:SPELL	284	6	35	97.93	89.03	96.01
R:VERB	25	0	163	100.00	13.30	43.40
R:VERB:FORM	57	16	105	78.08	35.19	62.78
R:VERB:INFL	4	0	4	100.00	50.00	83.33
R:VERB:SVA	73	13	67	84.88	52.14	75.41
R:VERB:TENSE	10	0	176	100.00	5.38	22.12
R:WO	9	0	80	100.00	10.11	36.00
U:ADJ	0	0	8	100.00	0.00	0.00
U:ADV	2	0	23	100.00	8.00	30.30
U:CONJ	0	0	5	100.00	0.00	0.00
U:CONTR	0	0	6	100.00	0.00	0.00
U:DET	119	63	122	65.38	49.38	61.40
U:NOUN	2	0	17	100.00	10.53	37.04
U:NOUN:POSS	1	0	11	100.00	8.33	31.25
U:OTHER	12	0	98	100.00	10.91	37.97
U:PART	0	0	3	100.00	0.00	0.00
U:PREP	0	0	61	100.00	0.00	0.00
U:PRON	5	1	18	83.33	21.74	53.19
U:PUNCT	34	13	38	72.34	47.22	65.38
U:VERB	1	0	32	100.00	3.03	13.51
U:VERB:FORM	0	0	4	100.00	0.00	0.00
U:VERB:TENSE	2	0	24	100.00	7.69	29.41
UNK	17	0	72	100.00	19.10	54.14

OVERALL
	TP	FP	FN	P	R	F_0.5
Span-level correction	528	314	1508	62.71	25.93	48.85
Span-level detection	659	185	1481	78.08	30.79	59.74
Token-level detection	826	121	1758	87.22	31.97	64.81

SPAN-LEVEL CORRECTION
OPERATION LEVEL
	TP	FP	FN	P	R	F_0.5
M	119	80	331	59.80	26.44	47.75
R	349	216	1037	61.77	25.18	47.86
U	60	18	140	76.92	30.00	58.59

ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
ADJ	0	0	10	100.00	0.00	0.00
ADJ:FORM	1	5	2	16.67	33.33	18.52
ADV	0	1	26	0.00	0.00	0.00
CONJ	0	0	5	100.00	0.00	0.00
CONTR	0	0	2	100.00	0.00	0.00
DET	75	26	171	74.26	30.49	57.69
MORPH	32	14	45	69.57	41.56	61.30
NOUN	2	0	32	100.00	5.88	23.81
NOUN:INFL	2	0	5	100.00	28.57	66.67
NOUN:NUM	36	19	75	65.45	32.43	54.38
NOUN:POSS	3	6	9	33.33	25.00	31.25
ORTH	74	34	114	68.52	39.36	59.68
OTHER	1	1	365	50.00	0.27	1.34
PART	2	1	7	66.67	22.22	47.62
PREP	21	14	130	60.00	13.91	36.08
PRON	2	0	60	100.00	3.23	14.29
PUNCT	117	88	133	57.07	46.80	54.67
SPELL	78	70	38	52.70	67.24	55.08
VERB	0	0	96	100.00	0.00	0.00
VERB:FORM	37	23	53	61.67	41.11	56.06
VERB:INFL	0	0	4	100.00	0.00	0.00
VERB:SVA	45	12	28	78.95	61.64	74.75
VERB:TENSE	0	0	82	100.00	0.00	0.00
WO	0	0	16	100.00	0.00	0.00

OPERATION + ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
M:ADJ	0	0	2	100.00	0.00	0.00
M:ADV	0	1	8	0.00	0.00	0.00
M:DET	16	10	85	61.54	15.84	39.02
M:NOUN	0	0	1	100.00	0.00	0.00
M:NOUN:POSS	0	1	3	0.00	0.00	0.00
M:OTHER	0	0	23	100.00	0.00	0.00
M:PART	0	0	2	100.00	0.00	0.00
M:PREP	0	0	36	100.00	0.00	0.00
M:PRON	0	0	31	100.00	0.00	0.00
M:PUNCT	95	63	106	60.13	47.26	57.02
M:VERB	0	0	16	100.00	0.00	0.00
M:VERB:FORM	8	5	3	61.54	72.73	63.49
M:VERB:TENSE	0	0	15	100.00	0.00	0.00
R:ADJ	0	0	6	100.00	0.00	0.00
R:ADJ:FORM	1	5	2	16.67	33.33	18.52
R:ADV	0	0	10	100.00	0.00	0.00
R:CONJ	0	0	1	100.00	0.00	0.00
R:CONTR	0	0	1	100.00	0.00	0.00
R:DET	4	1	33	80.00	10.81	35.09
R:MORPH	32	14	45	69.57	41.56	61.30
R:NOUN	2	0	24	100.00	7.69	29.41
R:NOUN:INFL	2	0	5	100.00	28.57	66.67
R:NOUN:NUM	36	19	75	65.45	32.43	54.38
R:NOUN:POSS	3	5	4	37.50	42.86	38.46
R:ORTH	74	34	114	68.52	39.36	59.68
R:OTHER	1	1	334	50.00	0.30	1.46
R:PART	2	1	3	66.67	40.00	58.82
R:PREP	21	14	70	60.00	23.08	45.45
R:PRON	0	0	21	100.00	0.00	0.00
R:PUNCT	19	22	18	46.34	51.35	47.26
R:SPELL	78	70	38	52.70	67.24	55.08
R:VERB	0	0	72	100.00	0.00	0.00
R:VERB:FORM	29	18	50	61.70	36.71	54.31
R:VERB:INFL	0	0	4	100.00	0.00	0.00
R:VERB:SVA	45	12	28	78.95	61.64	74.75
R:VERB:TENSE	0	0	63	100.00	0.00	0.00
R:WO	0	0	16	100.00	0.00	0.00
U:ADJ	0	0	2	100.00	0.00	0.00
U:ADV	0	0	8	100.00	0.00	0.00
U:CONJ	0	0	4	100.00	0.00	0.00
U:CONTR	0	0	1	100.00	0.00	0.00
U:DET	55	15	53	78.57	50.93	70.88
U:NOUN	0	0	7	100.00	0.00	0.00
U:NOUN:POSS	0	0	2	100.00	0.00	0.00
U:OTHER	0	0	8	100.00	0.00	0.00
U:PART	0	0	2	100.00	0.00	0.00
U:PREP	0	0	24	100.00	0.00	0.00
U:PRON	2	0	8	100.00	20.00	55.56
U:PUNCT	3	3	9	50.00	25.00	41.67
U:VERB	0	0	8	100.00	0.00	0.00
U:VERB:TENSE	0	0	4	100.00	0.00	0.00

SPAN-LEVEL DETECTION
OPERATION LEVEL
	TP	FP	FN	P	R	F_0.5
M	138	64	315	68.32	30.46	54.72
R	450	105	972	81.08	31.65	61.78
U	65	16	141	80.25	31.55	61.32
UNK	6	0	53	100.00	10.17	36.14

ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
ADJ	2	0	12	100.00	14.29	45.45
ADJ:FORM	1	5	5	16.67	16.67	16.67
ADV	3	1	23	75.00	11.54	35.71
CONJ	1	0	6	100.00	14.29	45.45
CONTR	0	0	3	100.00	0.00	0.00
DET	77	22	161	77.78	32.35	60.73
MORPH	32	11	47	74.42	40.51	63.75
NOUN	7	0	33	100.00	17.50	51.47
NOUN:INFL	2	0	5	100.00	28.57	66.67
NOUN:NUM	32	14	75	69.57	29.91	54.98
NOUN:POSS	5	3	7	62.50	41.67	56.82
ORTH	96	19	98	83.48	49.48	73.39
OTHER	46	1	352	97.87	11.56	39.25
PART	2	1	4	66.67	33.33	55.56
PREP	33	12	126	73.33	20.75	48.67
PRON	2	0	58	100.00	3.33	14.71
PUNCT	115	71	129	61.83	47.13	58.20
SPELL	98	8	12	92.45	89.09	91.76
UNK	6	0	53	100.00	10.17	36.14
VERB	11	0	91	100.00	10.78	37.67
VERB:FORM	38	15	50	71.70	43.18	63.33
VERB:INFL	3	0	2	100.00	60.00	88.24
VERB:SVA	42	2	30	95.45	58.33	84.68
VERB:TENSE	5	0	81	100.00	5.81	23.58
WO	0	0	18	100.00	0.00	0.00

OPERATION + ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
M:ADJ	0	0	2	100.00	0.00	0.00
M:ADV	3	1	7	75.00	30.00	57.69
M:CONJ	1	0	3	100.00	25.00	62.50
M:CONTR	0	0	1	100.00	0.00	0.00
M:DET	17	9	77	65.38	18.09	42.93
M:NOUN	0	0	2	100.00	0.00	0.00
M:NOUN:POSS	0	1	3	0.00	0.00	0.00
M:OTHER	4	0	28	100.00	12.50	41.67
M:PART	0	0	1	100.00	0.00	0.00
M:PREP	8	0	33	100.00	19.51	54.79
M:PRON	0	0	30	100.00	0.00	0.00
M:PUNCT	94	48	96	66.20	49.47	62.01
M:VERB	2	0	15	100.00	11.76	40.00
M:VERB:FORM	9	5	3	64.29	75.00	66.18
M:VERB:TENSE	0	0	14	100.00	0.00	0.00
R:ADJ	1	0	8	100.00	11.11	38.46
R:ADJ:FORM	1	5	5	16.67	16.67	16.67
R:ADV	0	0	9	100.00	0.00	0.00
R:CONTR	0	0	1	100.00	0.00	0.00
R:DET	7	0	34	100.00	17.07	50.72
R:MORPH	32	11	47	74.42	40.51	63.75
R:NOUN	5	0	24	100.00	17.24	51.02
R:NOUN:INFL	2	0	5	100.00	28.57	66.67
R:NOUN:NUM	32	14	75	69.57	29.91	54.98
R:NOUN:POSS	5	2	2	71.43	71.43	71.43
R:ORTH	96	19	98	83.48	49.48	73.39
R:OTHER	41	1	313	97.62	11.58	39.27
R:PART	2	1	2	66.67	50.00	62.50
R:PREP	25	12	71	67.57	26.04	51.23
R:PRON	0	0	20	100.00	0.00	0.00
R:PUNCT	18	20	21	47.37	46.15	47.12
R:SPELL	98	8	12	92.45	89.09	91.76
R:VERB	8	0	65	100.00	10.96	38.10
R:VERB:FORM	29	10	47	74.36	38.16	62.50
R:VERB:INFL	3	0	2	100.00	60.00	88.24
R:VERB:SVA	42	2	30	95.45	58.33	84.68
R:VERB:TENSE	3	0	63	100.00	4.55	19.23
R:WO	0	0	18	100.00	0.00	0.00
U:ADJ	1	0	2	100.00	33.33	71.43
U:ADV	0	0	7	100.00	0.00	0.00
U:CONJ	0	0	3	100.00	0.00	0.00
U:CONTR	0	0	1	100.00	0.00	0.00
U:DET	53	13	50	80.30	51.46	72.21
U:NOUN	2	0	7	100.00	22.22	58.82
U:NOUN:POSS	0	0	2	100.00	0.00	0.00
U:OTHER	1	0	11	100.00	8.33	31.25
U:PART	0	0	1	100.00	0.00	0.00
U:PREP	0	0	22	100.00	0.00	0.00
U:PRON	2	0	8	100.00	20.00	55.56
U:PUNCT	3	3	12	50.00	20.00	38.46
U:VERB	1	0	11	100.00	8.33	31.25
U:VERB:TENSE	2	0	4	100.00	33.33	71.43
UNK	6	0	53	100.00	10.17	36.14

TOKEN-LEVEL DETECTION
OPERATION LEVEL
	TP	FP	FN	P	R	F_0.5
M	181	36	344	83.41	34.48	64.97
R	558	72	1141	88.57	32.84	66.13
U	79	13	219	85.87	26.51	59.31
UNK	8	0	54	100.00	12.90	42.55

ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
ADJ	1	0	17	100.00	5.56	22.73
ADJ:FORM	1	4	2	20.00	33.33	21.74
ADV	5	0	32	100.00	13.51	43.86
CONJ	3	0	8	100.00	27.27	65.22
CONTR	0	0	4	100.00	0.00	0.00
DET	89	16	174	84.76	33.84	65.15
MORPH	32	8	54	80.00	37.21	65.04
NOUN	9	0	51	100.00	15.00	46.88
NOUN:INFL	2	0	4	100.00	33.33	71.43
NOUN:NUM	34	12	80	73.91	29.82	57.05
NOUN:POSS	8	2	12	80.00	40.00	66.67
ORTH	108	16	94	87.10	53.47	77.36
OTHER	112	0	496	100.00	18.42	53.03
PART	1	1	7	50.00	12.50	31.25
PREP	33	5	134	86.84	19.76	51.72
PRON	7	0	57	100.00	10.94	38.04
PUNCT	158	49	143	76.33	52.49	69.97
SPELL	99	0	12	100.00	89.19	97.63
UNK	8	0	54	100.00	12.90	42.55
VERB	18	0	119	100.00	13.14	43.06
VERB:FORM	39	6	51	86.67	43.33	72.22
VERB:INFL	4	0	0	100.00	100.00	100.00
VERB:SVA	41	2	34	95.35	54.67	83.00
VERB:TENSE	10	0	103	100.00	8.85	32.68
WO	4	0	16	100.00	20.00	55.56

OPERATION + ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
M:ADJ	0	0	2	100.00	0.00	0.00
M:ADV	5	0	9	100.00	35.71	73.53
M:CONJ	3	0	4	100.00	42.86	78.95
M:CONTR	0	0	1	100.00	0.00	0.00
M:DET	22	6	81	78.57	21.36	51.16
M:NOUN	2	0	5	100.00	28.57	66.67
M:NOUN:POSS	0	0	3	100.00	0.00	0.00
M:OTHER	6	0	29	100.00	17.14	50.85
M:PART	0	0	2	100.00	0.00	0.00
M:PREP	10	0	38	100.00	20.83	56.82
M:PRON	2	0	29	100.00	6.45	25.64
M:PUNCT	114	30	102	79.17	52.78	71.97
M:VERB	5	0	22	100.00	18.52	53.19
M:VERB:FORM	9	0	3	100.00	75.00	93.75
M:VERB:TENSE	3	0	14	100.00	17.65	51.72
R:ADJ	1	0	12	100.00	7.69	29.41
R:ADJ:FORM	1	4	2	20.00	33.33	21.74
R:ADV	0	0	10	100.00	0.00	0.00
R:CONTR	0	0	1	100.00	0.00	0.00
R:DET	8	0	32	100.00	20.00	55.56
R:MORPH	32	8	54	80.00	37.21	65.04
R:NOUN	5	0	34	100.00	12.82	42.37
R:NOUN:INFL	2	0	4	100.00	33.33	71.43
R:NOUN:NUM	34	12	80	73.91	29.82	57.05
R:NOUN:POSS	8	2	1	80.00	88.89	81.63
R:ORTH	108	16	94	87.10	53.47	77.36
R:OTHER	100	0	429	100.00	18.90	53.82
R:PART	1	1	4	50.00	20.00	38.46
R:PREP	23	5	71	82.14	24.47	55.83
R:PRON	2	0	18	100.00	10.00	35.71
R:PUNCT	37	16	25	69.81	59.68	67.52
R:SPELL	99	0	12	100.00	89.19	97.63
R:VERB	13	0	79	100.00	14.13	45.14
R:VERB:FORM	30	6	47	83.33	38.96	67.87
R:VERB:INFL	4	0	0	100.00	100.00	100.00
R:VERB:SVA	41	2	34	95.35	54.67	83.00
R:VERB:TENSE	5	0	82	100.00	5.75	23.36
R:WO	4	0	16	100.00	20.00	55.56
U:ADJ	0	0	3	100.00	0.00	0.00
U:ADV	0	0	13	100.00	0.00	0.00
U:CONJ	0	0	4	100.00	0.00	0.00
U:CONTR	0	0	2	100.00	0.00	0.00
U:DET	59	10	61	85.51	49.17	74.49
U:NOUN	2	0	12	100.00	14.29	45.45
U:NOUN:POSS	0	0	8	100.00	0.00	0.00
U:OTHER	6	0	38	100.00	13.64	44.12
U:PART	0	0	1	100.00	0.00	0.00
U:PREP	0	0	25	100.00	0.00	0.00
U:PRON	3	0	10	100.00	23.08	60.00
U:PUNCT	7	3	16	70.00	30.43	55.56
U:VERB	0	0	18	100.00	0.00	0.00
U:VERB:FORM	0	0	1	100.00	0.00	0.00
U:VERB:TENSE	2	0	7	100.00	22.22	58.82
UNK	8	0	54	100.00	12.90	42.55

OVERALL
	TP	FP	FN	P	R	F_0.5
Span-level correction	384	229	1061	62.64	26.57	49.27
Span-level detection	459	154	1053	74.88	30.36	57.90
Token-level detection	547	117	1291	82.38	29.76	60.86

SPAN-LEVEL CORRECTION
OPERATION LEVEL
	TP	FP	FN	P	R	F_0.5
M	148	57	252	72.20	37.00	60.66
R	194	146	698	57.06	21.75	43.07
U	42	26	111	61.76	27.45	49.41

ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
ADJ	0	0	12	100.00	0.00	0.00
ADJ:FORM	2	2	1	50.00	66.67	52.63
ADV	1	0	14	100.00	6.67	26.32
CONJ	0	0	8	100.00	0.00	0.00
CONTR	0	0	2	100.00	0.00	0.00
DET	50	31	145	61.73	25.64	48.17
MORPH	18	5	22	78.26	45.00	68.18
NOUN	0	2	26	0.00	0.00	0.00
NOUN:NUM	30	18	32	62.50	48.39	59.06
NOUN:POSS	2	8	11	20.00	15.38	18.87
ORTH	13	26	52	33.33	20.00	29.41
OTHER	4	4	198	50.00	1.98	8.55
PART	0	2	8	0.00	0.00	0.00
PREP	24	13	122	64.86	16.44	40.82
PRON	1	0	36	100.00	2.70	12.20
PUNCT	151	60	99	71.56	60.40	69.01
SPELL	44	39	28	53.01	61.11	54.46
VERB	0	0	69	100.00	0.00	0.00
VERB:FORM	21	7	41	75.00	33.87	60.34
VERB:SVA	23	11	22	67.65	51.11	63.54
VERB:TENSE	0	0	78	100.00	0.00	0.00
WO	0	1	35	0.00	0.00	0.00

OPERATION + ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
M:ADJ	0	0	1	100.00	0.00	0.00
M:ADV	1	0	4	100.00	20.00	55.56
M:CONJ	0	0	5	100.00	0.00	0.00
M:DET	21	6	73	77.78	22.34	51.98
M:NOUN	0	0	2	100.00	0.00	0.00
M:NOUN:POSS	0	3	5	0.00	0.00	0.00
M:OTHER	0	0	15	100.00	0.00	0.00
M:PART	0	0	4	100.00	0.00	0.00
M:PREP	0	0	28	100.00	0.00	0.00
M:PRON	0	0	21	100.00	0.00	0.00
M:PUNCT	126	48	72	72.41	63.64	70.47
M:VERB	0	0	7	100.00	0.00	0.00
M:VERB:FORM	0	0	1	100.00	0.00	0.00
M:VERB:TENSE	0	0	14	100.00	0.00	0.00
R:ADJ	0	0	10	100.00	0.00	0.00
R:ADJ:FORM	2	2	1	50.00	66.67	52.63
R:ADV	0	0	6	100.00	0.00	0.00
R:CONJ	0	0	2	100.00	0.00	0.00
R:CONTR	0	0	1	100.00	0.00	0.00
R:DET	0	0	40	100.00	0.00	0.00
R:MORPH	18	5	22	78.26	45.00	68.18
R:NOUN	0	2	22	0.00	0.00	0.00
R:NOUN:NUM	30	18	32	62.50	48.39	59.06
R:NOUN:POSS	2	5	2	28.57	50.00	31.25
R:ORTH	13	26	52	33.33	20.00	29.41
R:OTHER	4	4	172	50.00	2.27	9.62
R:PART	0	2	3	0.00	0.00	0.00
R:PREP	24	13	68	64.86	26.09	50.00
R:PRON	0	0	10	100.00	0.00	0.00
R:PUNCT	13	11	20	54.17	39.39	50.39
R:SPELL	44	39	28	53.01	61.11	54.46
R:VERB	0	0	56	100.00	0.00	0.00
R:VERB:FORM	21	7	38	75.00	35.59	61.40
R:VERB:SVA	23	11	22	67.65	51.11	63.54
R:VERB:TENSE	0	0	56	100.00	0.00	0.00
R:WO	0	1	35	0.00	0.00	0.00
U:ADJ	0	0	1	100.00	0.00	0.00
U:ADV	0	0	4	100.00	0.00	0.00
U:CONJ	0	0	1	100.00	0.00	0.00
U:CONTR	0	0	1	100.00	0.00	0.00
U:DET	29	25	32	53.70	47.54	52.35
U:NOUN	0	0	2	100.00	0.00	0.00
U:NOUN:POSS	0	0	4	100.00	0.00	0.00
U:OTHER	0	0	11	100.00	0.00	0.00
U:PART	0	0	1	100.00	0.00	0.00
U:PREP	0	0	26	100.00	0.00	0.00
U:PRON	1	0	5	100.00	16.67	50.00
U:PUNCT	12	1	7	92.31	63.16	84.51
U:VERB	0	0	6	100.00	0.00	0.00
U:VERB:FORM	0	0	2	100.00	0.00	0.00
U:VERB:TENSE	0	0	8	100.00	0.00	0.00

SPAN-LEVEL DETECTION
OPERATION LEVEL
	TP	FP	FN	P	R	F_0.5
M	155	49	249	75.98	38.37	63.52
R	259	81	673	76.18	27.79	56.50
U	44	24	118	64.71	27.16	50.69
UNK	1	0	13	100.00	7.14	27.78

ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
ADJ	2	0	12	100.00	14.29	45.45
ADJ:FORM	2	2	1	50.00	66.67	52.63
ADV	2	0	14	100.00	12.50	41.67
CONJ	2	0	7	100.00	22.22	58.82
CONTR	0	0	2	100.00	0.00	0.00
DET	49	30	145	62.03	25.26	48.04
MORPH	19	4	20	82.61	48.72	72.52
NOUN	5	1	28	83.33	15.15	43.86
NOUN:NUM	32	11	31	74.42	50.79	68.09
NOUN:POSS	2	7	12	22.22	14.29	20.00
ORTH	28	20	48	58.33	36.84	52.24
OTHER	23	3	204	88.46	10.13	34.74
PART	0	2	10	0.00	0.00	0.00
PREP	26	12	119	68.42	17.93	43.77
PRON	3	0	36	100.00	7.69	29.41
PUNCT	151	50	100	75.12	60.16	71.56
SPELL	60	4	11	93.75	84.51	91.74
UNK	1	0	13	100.00	7.14	27.78
VERB	5	0	70	100.00	6.67	26.32
VERB:FORM	20	4	40	83.33	33.33	64.10
VERB:SVA	24	3	23	88.89	51.06	77.42
VERB:TENSE	3	0	75	100.00	3.85	16.67
WO	0	1	32	0.00	0.00	0.00

OPERATION + ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
M:ADJ	0	0	2	100.00	0.00	0.00
M:ADV	1	0	4	100.00	20.00	55.56
M:CONJ	2	0	4	100.00	33.33	71.43
M:DET	21	6	71	77.78	22.83	52.50
M:NOUN	0	0	2	100.00	0.00	0.00
M:NOUN:POSS	0	3	6	0.00	0.00	0.00
M:OTHER	2	0	13	100.00	13.33	43.48
M:PART	0	0	4	100.00	0.00	0.00
M:PREP	1	0	27	100.00	3.57	15.62
M:PRON	1	0	21	100.00	4.55	19.23
M:PUNCT	127	40	72	76.05	63.82	73.24
M:VERB	0	0	8	100.00	0.00	0.00
M:VERB:FORM	0	0	1	100.00	0.00	0.00
M:VERB:TENSE	0	0	14	100.00	0.00	0.00
R:ADJ	2	0	8	100.00	20.00	55.56
R:ADJ:FORM	2	2	1	50.00	66.67	52.63
R:ADV	0	0	6	100.00	0.00	0.00
R:CONJ	0	0	2	100.00	0.00	0.00
R:CONTR	0	0	1	100.00	0.00	0.00
R:DET	0	0	42	100.00	0.00	0.00
R:MORPH	19	4	20	82.61	48.72	72.52
R:NOUN	5	1	23	83.33	17.86	48.08
R:NOUN:NUM	32	11	31	74.42	50.79	68.09
R:NOUN:POSS	2	4	2	33.33	50.00	35.71
R:ORTH	28	20	48	58.33	36.84	52.24
R:OTHER	21	3	178	87.50	10.55	35.59
R:PART	0	2	5	0.00	0.00	0.00
R:PREP	25	12	68	67.57	26.88	51.87
R:PRON	0	0	10	100.00	0.00	0.00
R:PUNCT	12	10	19	54.55	38.71	50.42
R:SPELL	60	4	11	93.75	84.51	91.74
R:VERB	4	0	55	100.00	6.78	26.67
R:VERB:FORM	20	4	37	83.33	35.09	65.36
R:VERB:SVA	24	3	23	88.89	51.06	77.42
R:VERB:TENSE	3	0	51	100.00	5.56	22.73
R:WO	0	1	32	0.00	0.00	0.00
U:ADJ	0	0	2	100.00	0.00	0.00
U:ADV	1	0	4	100.00	20.00	55.56
U:CONJ	0	0	1	100.00	0.00	0.00
U:CONTR	0	0	1	100.00	0.00	0.00
U:DET	28	24	32	53.85	46.67	52.24
U:NOUN	0	0	3	100.00	0.00	0.00
U:NOUN:POSS	0	0	4	100.00	0.00	0.00
U:OTHER	0	0	13	100.00	0.00	0.00
U:PART	0	0	1	100.00	0.00	0.00
U:PREP	0	0	24	100.00	0.00	0.00
U:PRON	2	0	5	100.00	28.57	66.67
U:PUNCT	12	0	9	100.00	57.14	86.96
U:VERB	1	0	7	100.00	12.50	41.67
U:VERB:FORM	0	0	2	100.00	0.00	0.00
U:VERB:TENSE	0	0	10	100.00	0.00	0.00
UNK	1	0	13	100.00	7.14	27.78

TOKEN-LEVEL DETECTION
OPERATION LEVEL
	TP	FP	FN	P	R	F_0.5
M	174	35	280	83.25	38.33	67.44
R	317	62	824	83.64	27.78	59.65
U	53	20	180	72.60	22.75	50.48
UNK	3	0	7	100.00	30.00	68.18

ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
ADJ	3	0	18	100.00	14.29	45.45
ADJ:FORM	2	2	1	50.00	66.67	52.63
ADV	3	0	20	100.00	13.04	42.86
CONJ	7	0	8	100.00	46.67	81.40
CONTR	0	0	2	100.00	0.00	0.00
DET	54	25	149	68.35	26.60	52.02
MORPH	20	3	20	86.96	50.00	75.76
NOUN	5	1	29	83.33	14.71	43.10
NOUN:NUM	30	8	37	78.95	44.78	68.49
NOUN:POSS	3	7	13	30.00	18.75	26.79
ORTH	42	15	63	73.68	40.00	63.06
OTHER	55	3	319	94.83	14.71	45.38
PART	0	2	8	0.00	0.00	0.00
PREP	28	8	129	77.78	17.83	46.51
PRON	4	0	41	100.00	8.89	32.79
PUNCT	168	34	106	83.17	61.31	77.63
SPELL	60	3	10	95.24	85.71	93.17
UNK	3	0	7	100.00	30.00	68.18
VERB	10	0	91	100.00	9.90	35.46
VERB:FORM	20	3	49	86.96	28.99	62.11
VERB:SVA	23	3	25	88.46	47.92	75.66
VERB:TENSE	4	0	94	100.00	4.08	17.54
WO	3	0	52	100.00	5.45	22.39

OPERATION + ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
M:ADJ	1	0	3	100.00	25.00	62.50
M:ADV	1	0	7	100.00	12.50	41.67
M:CONJ	5	0	3	100.00	62.50	89.29
M:DET	23	5	76	82.14	23.23	54.50
M:NOUN	0	0	3	100.00	0.00	0.00
M:NOUN:POSS	1	3	6	25.00	14.29	21.74
M:OTHER	4	0	22	100.00	15.38	47.62
M:PART	0	0	3	100.00	0.00	0.00
M:PREP	2	0	27	100.00	6.90	27.03
M:PRON	3	0	22	100.00	12.00	40.54
M:PUNCT	132	27	76	83.02	63.46	78.20
M:VERB	2	0	13	100.00	13.33	43.48
M:VERB:FORM	0	0	2	100.00	0.00	0.00
M:VERB:TENSE	0	0	17	100.00	0.00	0.00
R:ADJ	2	0	13	100.00	13.33	43.48
R:ADJ:FORM	2	2	1	50.00	66.67	52.63
R:ADV	0	0	7	100.00	0.00	0.00
R:CONJ	2	0	4	100.00	33.33	71.43
R:CONTR	0	0	1	100.00	0.00	0.00
R:DET	2	0	38	100.00	5.00	20.83
R:MORPH	20	3	20	86.96	50.00	75.76
R:NOUN	5	1	22	83.33	18.52	49.02
R:NOUN:NUM	30	8	37	78.95	44.78	68.49
R:NOUN:POSS	2	4	4	33.33	33.33	33.33
R:ORTH	42	15	63	73.68	40.00	63.06
R:OTHER	46	3	242	93.88	15.97	47.52
R:PART	0	2	4	0.00	0.00	0.00
R:PREP	26	8	75	76.47	25.74	54.85
R:PRON	0	0	12	100.00	0.00	0.00
R:PUNCT	21	7	19	75.00	52.50	69.08
R:SPELL	60	3	10	95.24	85.71	93.17
R:VERB	7	0	66	100.00	9.59	34.65
R:VERB:FORM	20	3	44	86.96	31.25	64.10
R:VERB:SVA	23	3	25	88.46	47.92	75.66
R:VERB:TENSE	4	0	65	100.00	5.80	23.53
R:WO	3	0	52	100.00	5.45	22.39
U:ADJ	0	0	2	100.00	0.00	0.00
U:ADV	2	0	6	100.00	25.00	62.50
U:CONJ	0	0	1	100.00	0.00	0.00
U:CONTR	0	0	1	100.00	0.00	0.00
U:DET	29	20	35	59.18	45.31	55.77
U:NOUN	0	0	4	100.00	0.00	0.00
U:NOUN:POSS	0	0	3	100.00	0.00	0.00
U:OTHER	5	0	55	100.00	8.33	31.25
U:PART	0	0	1	100.00	0.00	0.00
U:PREP	0	0	27	100.00	0.00	0.00
U:PRON	1	0	7	100.00	12.50	41.67
U:PUNCT	15	0	11	100.00	57.69	87.21
U:VERB	1	0	12	100.00	7.69	29.41
U:VERB:FORM	0	0	3	100.00	0.00	0.00
U:VERB:TENSE	0	0	12	100.00	0.00	0.00
UNK	3	0	7	100.00	30.00	68.18

OVERALL
	TP	FP	FN	P	R	F_0.5
Span-level correction	240	146	352	62.18	40.54	56.18
Span-level detection	266	120	343	68.91	43.68	61.77
Token-level detection	307	105	416	74.51	42.46	64.74

SPAN-LEVEL CORRECTION
OPERATION LEVEL
	TP	FP	FN	P	R	F_0.5
M	102	43	90	70.34	53.12	66.06
R	108	79	222	57.75	32.73	50.09
U	30	24	40	55.56	42.86	52.45

ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
ADJ	0	0	6	100.00	0.00	0.00
ADJ:FORM	0	0	2	100.00	0.00	0.00
ADV	0	0	4	100.00	0.00	0.00
CONJ	0	0	3	100.00	0.00	0.00
CONTR	0	0	3	100.00	0.00	0.00
DET	34	17	63	66.67	35.05	56.48
MORPH	12	8	14	60.00	46.15	56.60
NOUN	1	0	10	100.00	9.09	33.33
NOUN:INFL	1	0	0	100.00	100.00	100.00
NOUN:NUM	16	7	15	69.57	51.61	65.04
NOUN:POSS	4	5	2	44.44	66.67	47.62
ORTH	17	9	18	65.38	48.57	61.15
OTHER	1	1	57	50.00	1.72	7.58
PART	2	1	1	66.67	66.67	66.67
PREP	19	12	42	61.29	31.15	51.35
PRON	0	0	9	100.00	0.00	0.00
PUNCT	100	62	29	61.73	77.52	64.35
SPELL	24	17	14	58.54	63.16	59.41
VERB	0	0	14	100.00	0.00	0.00
VERB:FORM	3	3	10	50.00	23.08	40.54
VERB:SVA	6	4	2	60.00	75.00	62.50
VERB:TENSE	0	0	26	100.00	0.00	0.00
WO	0	0	8	100.00	0.00	0.00

OPERATION + ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
M:ADJ	0	0	2	100.00	0.00	0.00
M:ADV	0	0	1	100.00	0.00	0.00
M:CONJ	0	0	1	100.00	0.00	0.00
M:DET	11	1	29	91.67	27.50	62.50
M:NOUN	0	0	3	100.00	0.00	0.00
M:OTHER	0	0	6	100.00	0.00	0.00
M:PREP	0	0	19	100.00	0.00	0.00
M:PRON	0	0	5	100.00	0.00	0.00
M:PUNCT	90	42	19	68.18	82.57	70.64
M:VERB	0	0	2	100.00	0.00	0.00
M:VERB:FORM	1	0	0	100.00	100.00	100.00
M:VERB:TENSE	0	0	3	100.00	0.00	0.00
R:ADJ	0	0	3	100.00	0.00	0.00
R:ADJ:FORM	0	0	2	100.00	0.00	0.00
R:ADV	0	0	1	100.00	0.00	0.00
R:CONJ	0	0	2	100.00	0.00	0.00
R:CONTR	0	0	1	100.00	0.00	0.00
R:DET	0	0	14	100.00	0.00	0.00
R:MORPH	12	8	14	60.00	46.15	56.60
R:NOUN	1	0	7	100.00	12.50	41.67
R:NOUN:INFL	1	0	0	100.00	100.00	100.00
R:NOUN:NUM	16	7	15	69.57	51.61	65.04
R:NOUN:POSS	4	5	1	44.44	80.00	48.78
R:ORTH	17	9	18	65.38	48.57	61.15
R:OTHER	1	1	51	50.00	1.92	8.33
R:PART	2	1	1	66.67	66.67	66.67
R:PREP	19	12	17	61.29	52.78	59.38
R:PRON	0	0	3	100.00	0.00	0.00
R:PUNCT	3	12	7	20.00	30.00	21.43
R:SPELL	24	17	14	58.54	63.16	59.41
R:VERB	0	0	11	100.00	0.00	0.00
R:VERB:FORM	2	3	10	40.00	16.67	31.25
R:VERB:SVA	6	4	2	60.00	75.00	62.50
R:VERB:TENSE	0	0	20	100.00	0.00	0.00
R:WO	0	0	8	100.00	0.00	0.00
U:ADJ	0	0	1	100.00	0.00	0.00
U:ADV	0	0	2	100.00	0.00	0.00
U:CONTR	0	0	2	100.00	0.00	0.00
U:DET	23	16	20	58.97	53.49	57.79
U:NOUN:POSS	0	0	1	100.00	0.00	0.00
U:PREP	0	0	6	100.00	0.00	0.00
U:PRON	0	0	1	100.00	0.00	0.00
U:PUNCT	7	8	3	46.67	70.00	50.00
U:VERB	0	0	1	100.00	0.00	0.00
U:VERB:TENSE	0	0	3	100.00	0.00	0.00

SPAN-LEVEL DETECTION
OPERATION LEVEL
	TP	FP	FN	P	R	F_0.5
M	103	42	92	71.03	52.82	66.45
R	131	55	207	70.43	38.76	60.54
U	30	23	41	56.60	42.25	53.00
UNK	2	0	3	100.00	40.00	76.92

ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
ADJ	0	0	6	100.00	0.00	0.00
ADJ:FORM	0	0	2	100.00	0.00	0.00
ADV	0	0	4	100.00	0.00	0.00
CONJ	0	0	3	100.00	0.00	0.00
CONTR	0	0	3	100.00	0.00	0.00
DET	33	17	63	66.00	34.38	55.74
MORPH	13	5	13	72.22	50.00	66.33
NOUN	2	0	11	100.00	15.38	47.62
NOUN:INFL	3	0	0	100.00	100.00	100.00
NOUN:NUM	16	7	16	69.57	50.00	64.52
NOUN:POSS	5	3	2	62.50	71.43	64.10
ORTH	17	8	19	68.00	47.22	62.50
OTHER	4	0	57	100.00	6.56	25.97
PART	2	1	2	66.67	50.00	62.50
PREP	19	11	40	63.33	32.20	53.07
PRON	0	0	9	100.00	0.00	0.00
PUNCT	101	60	32	62.73	75.94	64.99
SPELL	38	2	0	95.00	100.00	95.96
UNK	2	0	3	100.00	40.00	76.92
VERB	1	0	14	100.00	6.67	26.32
VERB:FORM	3	3	10	50.00	23.08	40.54
VERB:SVA	6	3	2	66.67	75.00	68.18
VERB:TENSE	1	0	25	100.00	3.85	16.67
WO	0	0	7	100.00	0.00	0.00

OPERATION + ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
M:ADJ	0	0	1	100.00	0.00	0.00
M:ADV	0	0	1	100.00	0.00	0.00
M:CONJ	0	0	1	100.00	0.00	0.00
M:DET	11	1	29	91.67	27.50	62.50
M:NOUN	0	0	3	100.00	0.00	0.00
M:OTHER	0	0	7	100.00	0.00	0.00
M:PART	0	0	1	100.00	0.00	0.00
M:PREP	0	0	18	100.00	0.00	0.00
M:PRON	0	0	5	100.00	0.00	0.00
M:PUNCT	91	41	21	68.94	81.25	71.09
M:VERB	0	0	2	100.00	0.00	0.00
M:VERB:FORM	1	0	0	100.00	100.00	100.00
M:VERB:TENSE	0	0	3	100.00	0.00	0.00
R:ADJ	0	0	4	100.00	0.00	0.00
R:ADJ:FORM	0	0	2	100.00	0.00	0.00
R:ADV	0	0	1	100.00	0.00	0.00
R:CONJ	0	0	2	100.00	0.00	0.00
R:CONTR	0	0	1	100.00	0.00	0.00
R:DET	0	0	14	100.00	0.00	0.00
R:MORPH	13	5	13	72.22	50.00	66.33
R:NOUN	2	0	8	100.00	20.00	55.56
R:NOUN:INFL	3	0	0	100.00	100.00	100.00
R:NOUN:NUM	16	7	16	69.57	50.00	64.52
R:NOUN:POSS	4	3	1	57.14	80.00	60.61
R:ORTH	17	8	19	68.00	47.22	62.50
R:OTHER	4	0	50	100.00	7.41	28.57
R:PART	2	1	1	66.67	66.67	66.67
R:PREP	19	11	16	63.33	54.29	61.29
R:PRON	0	0	3	100.00	0.00	0.00
R:PUNCT	3	12	7	20.00	30.00	21.43
R:SPELL	38	2	0	95.00	100.00	95.96
R:VERB	1	0	11	100.00	8.33	31.25
R:VERB:FORM	2	3	10	40.00	16.67	31.25
R:VERB:SVA	6	3	2	66.67	75.00	68.18
R:VERB:TENSE	1	0	19	100.00	5.00	20.83
R:WO	0	0	7	100.00	0.00	0.00
U:ADJ	0	0	1	100.00	0.00	0.00
U:ADV	0	0	2	100.00	0.00	0.00
U:CONTR	0	0	2	100.00	0.00	0.00
U:DET	22	16	20	57.89	52.38	56.70
U:NOUN:POSS	1	0	1	100.00	50.00	83.33
U:PREP	0	0	6	100.00	0.00	0.00
U:PRON	0	0	1	100.00	0.00	0.00
U:PUNCT	7	7	4	50.00	63.64	52.24
U:VERB	0	0	1	100.00	0.00	0.00
U:VERB:TENSE	0	0	3	100.00	0.00	0.00
UNK	2	0	3	100.00	40.00	76.92

TOKEN-LEVEL DETECTION
OPERATION LEVEL
	TP	FP	FN	P	R	F_0.5
M	108	37	108	74.48	50.00	67.84
R	159	48	246	76.81	39.26	64.48
U	36	20	52	64.29	40.91	57.69
UNK	4	0	10	100.00	28.57	66.67

ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
ADJ	0	0	8	100.00	0.00	0.00
ADJ:FORM	0	0	3	100.00	0.00	0.00
ADV	0	0	7	100.00	0.00	0.00
CONJ	0	0	3	100.00	0.00	0.00
CONTR	0	0	4	100.00	0.00	0.00
DET	35	14	69	71.43	33.65	58.33
MORPH	13	4	12	76.47	52.00	69.89
NOUN	1	0	18	100.00	5.26	21.74
NOUN:INFL	3	0	0	100.00	100.00	100.00
NOUN:NUM	15	6	20	71.43	42.86	63.03
NOUN:POSS	9	3	1	75.00	90.00	77.59
ORTH	24	6	26	80.00	48.00	70.59
OTHER	18	0	75	100.00	19.35	54.55
PART	2	1	1	66.67	66.67	66.67
PREP	19	10	44	65.52	30.16	53.07
PRON	1	0	11	100.00	8.33	31.25
PUNCT	110	56	37	66.27	74.83	67.82
SPELL	40	0	0	100.00	100.00	100.00
UNK	4	0	10	100.00	28.57	66.67
VERB	3	0	14	100.00	17.65	51.72
VERB:FORM	3	2	11	60.00	21.43	44.12
VERB:SVA	6	3	4	66.67	60.00	65.22
VERB:TENSE	1	0	33	100.00	2.94	13.16
WO	0	0	5	100.00	0.00	0.00

OPERATION + ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
M:ADJ	0	0	3	100.00	0.00	0.00
M:ADV	0	0	1	100.00	0.00	0.00
M:CONJ	0	0	1	100.00	0.00	0.00
M:DET	11	1	31	91.67	26.19	61.11
M:NOUN	0	0	5	100.00	0.00	0.00
M:NOUN:POSS	2	0	0	100.00	100.00	100.00
M:OTHER	2	0	8	100.00	20.00	55.56
M:PART	0	0	1	100.00	0.00	0.00
M:PREP	0	0	20	100.00	0.00	0.00
M:PRON	0	0	6	100.00	0.00	0.00
M:PUNCT	92	36	25	71.88	78.63	73.13
M:VERB	0	0	3	100.00	0.00	0.00
M:VERB:FORM	1	0	0	100.00	100.00	100.00
M:VERB:TENSE	0	0	4	100.00	0.00	0.00
R:ADJ	0	0	3	100.00	0.00	0.00
R:ADJ:FORM	0	0	3	100.00	0.00	0.00
R:ADV	0	0	2	100.00	0.00	0.00
R:CONJ	0	0	2	100.00	0.00	0.00
R:CONTR	0	0	1	100.00	0.00	0.00
R:DET	0	0	15	100.00	0.00	0.00
R:MORPH	13	4	12	76.47	52.00	69.89
R:NOUN	1	0	12	100.00	7.69	29.41
R:NOUN:INFL	3	0	0	100.00	100.00	100.00
R:NOUN:NUM	15	6	20	71.43	42.86	63.03
R:NOUN:POSS	6	3	1	66.67	85.71	69.77
R:ORTH	24	6	26	80.00	48.00	70.59
R:OTHER	16	0	65	100.00	19.75	55.17
R:PART	2	1	0	66.67	100.00	71.43
R:PREP	19	10	17	65.52	52.78	62.50
R:PRON	0	0	4	100.00	0.00	0.00
R:PUNCT	8	13	8	38.10	50.00	40.00
R:SPELL	40	0	0	100.00	100.00	100.00
R:VERB	3	0	10	100.00	23.08	60.00
R:VERB:FORM	2	2	11	50.00	15.38	34.48
R:VERB:SVA	6	3	4	66.67	60.00	65.22
R:VERB:TENSE	1	0	25	100.00	3.85	16.67
R:WO	0	0	5	100.00	0.00	0.00
U:ADJ	0	0	2	100.00	0.00	0.00
U:ADV	0	0	4	100.00	0.00	0.00
U:CONTR	0	0	3	100.00	0.00	0.00
U:DET	24	13	23	64.86	51.06	61.54
U:NOUN	0	0	1	100.00	0.00	0.00
U:NOUN:POSS	1	0	0	100.00	100.00	100.00
U:OTHER	0	0	2	100.00	0.00	0.00
U:PREP	0	0	7	100.00	0.00	0.00
U:PRON	1	0	1	100.00	50.00	83.33
U:PUNCT	10	7	4	58.82	71.43	60.98
U:VERB	0	0	1	100.00	0.00	0.00
U:VERB:TENSE	0	0	4	100.00	0.00	0.00
UNK	4	0	10	100.00	28.57	66.67

OVERALL
	TP	FP	FN	P	R	F_0.5
Span-level correction	291	195	254	59.88	53.39	58.46
Span-level detection	325	161	242	66.87	57.32	64.72
Token-level detection	365	148	268	71.15	57.66	67.97

SPAN-LEVEL CORRECTION
OPERATION LEVEL
	TP	FP	FN	P	R	F_0.5
M	139	57	49	70.92	73.94	71.50
R	144	109	189	56.92	43.24	53.53
U	8	29	16	21.62	33.33	23.26

ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
ADJ	0	0	5	100.00	0.00	0.00
CONTR	0	0	2	100.00	0.00	0.00
DET	8	27	8	22.86	50.00	25.64
MORPH	6	5	2	54.55	75.00	57.69
NOUN	1	0	7	100.00	12.50	41.67
NOUN:INFL	9	2	2	81.82	81.82	81.82
NOUN:NUM	10	8	12	55.56	45.45	53.19
NOUN:POSS	10	16	8	38.46	55.56	40.98
ORTH	15	17	40	46.88	27.27	40.98
OTHER	0	2	40	0.00	0.00	0.00
PART	0	2	1	0.00	0.00	0.00
PREP	7	10	17	41.18	29.17	38.04
PRON	0	1	5	0.00	0.00	0.00
PUNCT	147	68	49	68.37	75.00	69.60
SPELL	70	23	25	75.27	73.68	74.95
VERB	0	0	15	100.00	0.00	0.00
VERB:FORM	5	7	4	41.67	55.56	43.86
VERB:INFL	0	0	4	100.00	0.00	0.00
VERB:SVA	3	6	3	33.33	50.00	35.71
VERB:TENSE	0	0	4	100.00	0.00	0.00
WO	0	1	1	0.00	0.00	0.00

OPERATION + ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
M:DET	1	3	4	25.00	20.00	23.81
M:NOUN	0	0	2	100.00	0.00	0.00
M:NOUN:POSS	1	3	5	25.00	16.67	22.73
M:PREP	0	0	2	100.00	0.00	0.00
M:PRON	0	0	2	100.00	0.00	0.00
M:PUNCT	137	51	30	72.87	82.04	74.54
M:VERB	0	0	4	100.00	0.00	0.00
R:ADJ	0	0	5	100.00	0.00	0.00
R:CONTR	0	0	2	100.00	0.00	0.00
R:DET	1	0	3	100.00	25.00	62.50
R:MORPH	6	5	2	54.55	75.00	57.69
R:NOUN	1	0	5	100.00	16.67	50.00
R:NOUN:INFL	9	2	2	81.82	81.82	81.82
R:NOUN:NUM	10	8	12	55.56	45.45	53.19
R:NOUN:POSS	9	13	3	40.91	75.00	45.00
R:ORTH	15	17	40	46.88	27.27	40.98
R:OTHER	0	2	40	0.00	0.00	0.00
R:PART	0	2	0	0.00	100.00	0.00
R:PREP	7	10	12	41.18	36.84	40.23
R:PRON	0	0	3	100.00	0.00	0.00
R:PUNCT	8	13	11	38.10	42.11	38.83
R:SPELL	70	23	25	75.27	73.68	74.95
R:VERB	0	0	10	100.00	0.00	0.00
R:VERB:FORM	5	7	3	41.67	62.50	44.64
R:VERB:INFL	0	0	4	100.00	0.00	0.00
R:VERB:SVA	3	6	3	33.33	50.00	35.71
R:VERB:TENSE	0	0	3	100.00	0.00	0.00
R:WO	0	1	1	0.00	0.00	0.00
U:DET	6	24	1	20.00	85.71	23.62
U:PART	0	0	1	100.00	0.00	0.00
U:PREP	0	0	3	100.00	0.00	0.00
U:PRON	0	1	0	0.00	100.00	0.00
U:PUNCT	2	4	8	33.33	20.00	29.41
U:VERB	0	0	1	100.00	0.00	0.00
U:VERB:FORM	0	0	1	100.00	0.00	0.00
U:VERB:TENSE	0	0	1	100.00	0.00	0.00

SPAN-LEVEL DETECTION
OPERATION LEVEL
	TP	FP	FN	P	R	F_0.5
M	141	55	51	71.94	73.44	72.23
R	174	80	173	68.50	50.14	63.83
U	9	26	16	25.71	36.00	27.27
UNK	1	0	2	100.00	33.33	71.43

ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
ADJ	0	0	5	100.00	0.00	0.00
CONTR	0	0	2	100.00	0.00	0.00
DET	10	25	9	28.57	52.63	31.45
MORPH	6	5	2	54.55	75.00	57.69
NOUN	1	0	7	100.00	12.50	41.67
NOUN:INFL	9	2	2	81.82	81.82	81.82
NOUN:NUM	10	6	12	62.50	45.45	58.14
NOUN:POSS	12	14	8	46.15	60.00	48.39
ORTH	18	16	39	52.94	31.58	46.63
OTHER	7	2	41	77.78	14.58	41.67
PART	0	2	1	0.00	0.00	0.00
PREP	9	8	15	52.94	37.50	48.91
PRON	0	1	5	0.00	0.00	0.00
PUNCT	147	64	51	69.67	74.24	70.54
SPELL	85	3	12	96.59	87.63	94.65
UNK	1	0	2	100.00	33.33	71.43
VERB	1	0	14	100.00	6.67	26.32
VERB:FORM	5	6	3	45.45	62.50	48.08
VERB:INFL	0	0	4	100.00	0.00	0.00
VERB:SVA	3	6	3	33.33	50.00	35.71
VERB:TENSE	1	0	4	100.00	20.00	55.56
WO	0	1	1	0.00	0.00	0.00

OPERATION + ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
M:DET	1	3	4	25.00	20.00	23.81
M:NOUN	0	0	2	100.00	0.00	0.00
M:NOUN:POSS	1	3	5	25.00	16.67	22.73
M:OTHER	2	0	0	100.00	100.00	100.00
M:PREP	0	0	2	100.00	0.00	0.00
M:PRON	0	0	2	100.00	0.00	0.00
M:PUNCT	136	49	32	73.51	80.95	74.89
M:VERB	0	0	4	100.00	0.00	0.00
M:VERB:TENSE	1	0	0	100.00	100.00	100.00
R:ADJ	0	0	5	100.00	0.00	0.00
R:CONTR	0	0	2	100.00	0.00	0.00
R:DET	2	0	3	100.00	40.00	76.92
R:MORPH	6	5	2	54.55	75.00	57.69
R:NOUN	1	0	5	100.00	16.67	50.00
R:NOUN:INFL	9	2	2	81.82	81.82	81.82
R:NOUN:NUM	10	6	12	62.50	45.45	58.14
R:NOUN:POSS	11	11	3	50.00	78.57	53.92
R:ORTH	18	16	39	52.94	31.58	46.63
R:OTHER	5	2	40	71.43	11.11	34.25
R:PART	0	2	0	0.00	100.00	0.00
R:PREP	9	8	11	52.94	45.00	51.14
R:PRON	0	0	3	100.00	0.00	0.00
R:PUNCT	9	12	11	42.86	45.00	43.27
R:SPELL	85	3	12	96.59	87.63	94.65
R:VERB	1	0	9	100.00	10.00	35.71
R:VERB:FORM	5	6	3	45.45	62.50	48.08
R:VERB:INFL	0	0	4	100.00	0.00	0.00
R:VERB:SVA	3	6	3	33.33	50.00	35.71
R:VERB:TENSE	0	0	3	100.00	0.00	0.00
R:WO	0	1	1	0.00	0.00	0.00
U:DET	7	22	2	24.14	77.78	28.00
U:OTHER	0	0	1	100.00	0.00	0.00
U:PART	0	0	1	100.00	0.00	0.00
U:PREP	0	0	2	100.00	0.00	0.00
U:PRON	0	1	0	0.00	100.00	0.00
U:PUNCT	2	3	8	40.00	20.00	33.33
U:VERB	0	0	1	100.00	0.00	0.00
U:VERB:TENSE	0	0	1	100.00	0.00	0.00
UNK	1	0	2	100.00	33.33	71.43

TOKEN-LEVEL DETECTION
OPERATION LEVEL
	TP	FP	FN	P	R	F_0.5
M	144	49	54	74.61	72.73	74.23
R	209	75	194	73.59	51.86	67.90
U	10	24	19	29.41	34.48	30.30
UNK	2	0	1	100.00	66.67	90.91

ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
ADJ	1	0	5	100.00	16.67	50.00
CONTR	0	0	4	100.00	0.00	0.00
DET	10	21	13	32.26	43.48	34.01
MORPH	7	5	1	58.33	87.50	62.50
NOUN	1	0	8	100.00	11.11	38.46
NOUN:INFL	9	2	2	81.82	81.82	81.82
NOUN:NUM	10	4	12	71.43	45.45	64.10
NOUN:POSS	15	15	8	50.00	65.22	52.45
ORTH	27	14	45	65.85	37.50	57.20
OTHER	15	2	45	88.24	25.00	58.59
PART	0	2	1	0.00	0.00	0.00
PREP	10	7	15	58.82	40.00	53.76
PRON	0	1	6	0.00	0.00	0.00
PUNCT	161	62	53	72.20	75.23	72.78
SPELL	85	3	13	96.59	86.73	94.44
UNK	2	0	1	100.00	66.67	90.91
VERB	2	0	13	100.00	13.33	43.48
VERB:FORM	5	5	3	50.00	62.50	52.08
VERB:INFL	0	0	4	100.00	0.00	0.00
VERB:SVA	3	5	4	37.50	42.86	38.46
VERB:TENSE	0	0	5	100.00	0.00	0.00
WO	2	0	7	100.00	22.22	58.82

OPERATION + ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
M:DET	1	1	5	50.00	16.67	35.71
M:NOUN	0	0	3	100.00	0.00	0.00
M:NOUN:POSS	1	2	4	33.33	20.00	29.41
M:OTHER	2	0	0	100.00	100.00	100.00
M:PREP	1	0	2	100.00	33.33	71.43
M:PRON	0	0	3	100.00	0.00	0.00
M:PUNCT	139	46	33	75.14	80.81	76.21
M:VERB	0	0	4	100.00	0.00	0.00
R:ADJ	1	0	4	100.00	20.00	55.56
R:CONTR	0	0	4	100.00	0.00	0.00
R:DET	2	0	5	100.00	28.57	66.67
R:MORPH	7	5	1	58.33	87.50	62.50
R:NOUN	1	0	5	100.00	16.67	50.00
R:NOUN:INFL	9	2	2	81.82	81.82	81.82
R:NOUN:NUM	10	4	12	71.43	45.45	64.10
R:NOUN:POSS	14	13	4	51.85	77.78	55.56
R:ORTH	27	14	45	65.85	37.50	57.20
R:OTHER	12	2	42	85.71	22.22	54.55
R:PART	0	2	0	0.00	100.00	0.00
R:PREP	9	7	11	56.25	45.00	53.57
R:PRON	0	0	3	100.00	0.00	0.00
R:PUNCT	20	13	13	60.61	60.61	60.61
R:SPELL	85	3	13	96.59	86.73	94.44
R:VERB	2	0	8	100.00	20.00	55.56
R:VERB:FORM	5	5	3	50.00	62.50	52.08
R:VERB:INFL	0	0	4	100.00	0.00	0.00
R:VERB:SVA	3	5	4	37.50	42.86	38.46
R:VERB:TENSE	0	0	4	100.00	0.00	0.00
R:WO	2	0	7	100.00	22.22	58.82
U:ADJ	0	0	1	100.00	0.00	0.00
U:DET	7	20	3	25.93	70.00	29.66
U:OTHER	1	0	3	100.00	25.00	62.50
U:PART	0	0	1	100.00	0.00	0.00
U:PREP	0	0	2	100.00	0.00	0.00
U:PRON	0	1	0	0.00	100.00	0.00
U:PUNCT	2	3	7	40.00	22.22	34.48
U:VERB	0	0	1	100.00	0.00	0.00
U:VERB:TENSE	0	0	1	100.00	0.00	0.00
UNK	2	0	1	100.00	66.67	90.91

Submitted by:	cjbryant
Submitted at:	2019-08-07T14:47:43+00:00
Competition submission:	4
Competition phase:	3 - Open phase

Detailed results

Submission information

Results