BEA 2019 Shared Task: Grammatical Error Correction

OVERALL
	TP	FP	FN	P	R	F_0.5
Span-level correction	2709	894	2510	75.19	51.91	69.00
Span-level detection	3032	576	2309	84.04	56.77	76.67
Token-level detection	3655	355	2647	91.15	58.00	81.80

SPAN-LEVEL CORRECTION
OPERATION LEVEL
	TP	FP	FN	P	R	F_0.5
M	1021	265	533	79.39	65.70	76.22
R	1530	580	1669	72.51	47.83	65.73
U	158	49	308	76.33	33.91	61.05

ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
ADJ	7	3	35	70.00	16.67	42.68
ADJ:FORM	4	0	7	100.00	36.36	74.07
ADV	11	3	49	78.57	18.33	47.41
CONJ	1	1	17	50.00	5.56	19.23
CONTR	11	0	1	100.00	91.67	98.21
DET	304	88	292	77.55	51.01	70.24
MORPH	62	13	78	82.67	44.29	70.45
NOUN	14	11	106	56.00	11.67	31.82
NOUN:INFL	7	0	10	100.00	41.18	77.78
NOUN:NUM	152	42	81	78.35	65.24	75.32
NOUN:POSS	24	7	26	77.42	48.00	68.97
ORTH	192	31	183	86.10	51.20	75.77
OTHER	94	87	550	51.93	14.60	34.36
PART	20	6	12	76.92	62.50	73.53
PREP	249	111	210	69.17	54.25	65.56
PRON	69	24	73	74.19	48.59	67.12
PUNCT	777	204	269	79.20	74.28	78.17
SPELL	262	88	59	74.86	81.62	76.12
VERB	39	23	163	62.90	19.31	43.33
VERB:FORM	137	54	55	71.73	71.35	71.65
VERB:INFL	6	0	1	100.00	85.71	96.77
VERB:SVA	126	25	28	83.44	81.82	83.11
VERB:TENSE	108	43	144	71.52	42.86	63.08
WO	33	30	61	52.38	35.11	47.69

OPERATION + ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
M:ADJ	2	0	5	100.00	28.57	66.67
M:ADV	4	3	14	57.14	22.22	43.48
M:CONJ	0	0	12	100.00	0.00	0.00
M:DET	192	54	109	78.05	63.79	74.71
M:NOUN	0	0	7	100.00	0.00	0.00
M:NOUN:POSS	9	5	7	64.29	56.25	62.50
M:OTHER	1	0	53	100.00	1.85	8.62
M:PART	2	0	4	100.00	33.33	71.43
M:PREP	59	26	52	69.41	53.15	65.41
M:PRON	47	15	34	75.81	58.02	71.43
M:PUNCT	651	133	198	83.04	76.68	81.68
M:VERB	15	11	17	57.69	46.88	55.15
M:VERB:FORM	14	9	2	60.87	87.50	64.81
M:VERB:TENSE	25	9	19	73.53	56.82	69.44
R:ADJ	3	1	24	75.00	11.11	34.88
R:ADJ:FORM	4	0	7	100.00	36.36	74.07
R:ADV	7	0	17	100.00	29.17	67.31
R:CONJ	0	0	2	100.00	0.00	0.00
R:CONTR	4	0	1	100.00	80.00	95.24
R:DET	43	14	62	75.44	40.95	64.56
R:MORPH	62	13	78	82.67	44.29	70.45
R:NOUN	14	11	92	56.00	13.21	33.98
R:NOUN:INFL	7	0	10	100.00	41.18	77.78
R:NOUN:NUM	152	42	81	78.35	65.24	75.32
R:NOUN:POSS	9	1	16	90.00	36.00	69.23
R:ORTH	192	31	183	86.10	51.20	75.77
R:OTHER	93	87	480	51.67	16.23	35.96
R:PART	15	6	6	71.43	71.43	71.43
R:PREP	155	70	115	68.89	57.41	66.24
R:PRON	16	4	25	80.00	39.02	66.12
R:PUNCT	99	67	41	59.64	70.71	61.57
R:SPELL	262	88	59	74.86	81.62	76.12
R:VERB	24	12	125	66.67	16.11	40.96
R:VERB:FORM	121	44	50	73.33	70.76	72.80
R:VERB:INFL	6	0	1	100.00	85.71	96.77
R:VERB:SVA	126	25	28	83.44	81.82	83.11
R:VERB:TENSE	83	34	105	70.94	44.15	63.26
R:WO	33	30	61	52.38	35.11	47.69
U:ADJ	2	2	6	50.00	25.00	41.67
U:ADV	0	0	18	100.00	0.00	0.00
U:CONJ	1	1	3	50.00	25.00	41.67
U:CONTR	7	0	0	100.00	100.00	100.00
U:DET	69	20	121	77.53	36.32	63.19
U:NOUN	0	0	7	100.00	0.00	0.00
U:NOUN:POSS	6	1	3	85.71	66.67	81.08
U:OTHER	0	0	17	100.00	0.00	0.00
U:PART	3	0	2	100.00	60.00	88.24
U:PREP	35	15	43	70.00	44.87	62.95
U:PRON	6	5	14	54.55	30.00	46.88
U:PUNCT	27	4	30	87.10	47.37	74.59
U:VERB	0	0	21	100.00	0.00	0.00
U:VERB:FORM	2	1	3	66.67	40.00	58.82
U:VERB:TENSE	0	0	20	100.00	0.00	0.00

SPAN-LEVEL DETECTION
OPERATION LEVEL
	TP	FP	FN	P	R	F_0.5
M	1070	205	474	83.92	69.30	80.52
R	1770	333	1473	84.17	54.58	75.93
U	181	38	299	82.65	37.71	66.74
UNK	11	0	63	100.00	14.86	46.61

ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
ADJ	14	2	34	87.50	29.17	62.50
ADJ:FORM	3	0	6	100.00	33.33	71.43
ADV	14	3	47	82.35	22.95	54.26
CONJ	1	1	15	50.00	6.25	20.83
CONTR	11	0	0	100.00	100.00	100.00
DET	325	70	273	82.28	54.35	74.61
MORPH	64	9	78	87.67	45.07	73.73
NOUN	24	8	101	75.00	19.20	47.43
NOUN:INFL	8	0	9	100.00	47.06	81.63
NOUN:NUM	145	30	74	82.86	66.21	78.89
NOUN:POSS	26	5	25	83.87	50.98	74.29
ORTH	221	20	163	91.70	57.55	81.97
OTHER	233	42	491	84.73	32.18	63.87
PART	21	6	11	77.78	65.62	75.00
PREP	271	71	186	79.24	59.30	74.25
PRON	79	12	62	86.81	56.03	78.22
PUNCT	793	170	256	82.35	75.60	80.90
SPELL	288	13	27	95.68	91.43	94.80
UNK	11	0	63	100.00	14.86	46.61
VERB	62	15	147	80.52	29.67	59.96
VERB:FORM	133	38	48	77.78	73.48	76.88
VERB:INFL	7	0	0	100.00	100.00	100.00
VERB:SVA	132	12	22	91.67	85.71	90.41
VERB:TENSE	117	25	117	82.39	50.00	72.94
WO	29	24	54	54.72	34.94	49.15

OPERATION + ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
M:ADJ	2	0	6	100.00	25.00	62.50
M:ADV	3	3	13	50.00	18.75	37.50
M:CONJ	1	0	10	100.00	9.09	33.33
M:DET	204	41	94	83.27	68.46	79.81
M:NOUN	3	0	6	100.00	33.33	71.43
M:NOUN:POSS	9	5	7	64.29	56.25	62.50
M:OTHER	18	0	48	100.00	27.27	65.22
M:PART	2	0	3	100.00	40.00	76.92
M:PREP	62	20	46	75.61	57.41	71.10
M:PRON	50	6	28	89.29	64.10	82.78
M:PUNCT	656	108	188	85.86	77.73	84.10
M:VERB	19	7	12	73.08	61.29	70.37
M:VERB:FORM	14	9	0	60.87	100.00	66.04
M:VERB:TENSE	27	6	13	81.82	67.50	78.49
R:ADJ	9	1	22	90.00	29.03	63.38
R:ADJ:FORM	3	0	6	100.00	33.33	71.43
R:ADV	8	0	18	100.00	30.77	68.97
R:CONJ	0	0	2	100.00	0.00	0.00
R:CONTR	4	0	0	100.00	100.00	100.00
R:DET	49	12	61	80.33	44.55	69.21
R:MORPH	64	9	78	87.67	45.07	73.73
R:NOUN	19	8	86	70.37	18.10	44.60
R:NOUN:INFL	8	0	9	100.00	47.06	81.63
R:NOUN:NUM	145	30	74	82.86	66.21	78.89
R:NOUN:POSS	11	0	15	100.00	42.31	78.57
R:ORTH	221	20	163	91.70	57.55	81.97
R:OTHER	214	42	423	83.59	33.59	64.42
R:PART	16	6	6	72.73	72.73	72.73
R:PREP	176	40	99	81.48	64.00	77.26
R:PRON	18	2	22	90.00	45.00	75.00
R:PUNCT	103	59	41	63.58	71.53	65.03
R:SPELL	288	13	27	95.68	91.43	94.80
R:VERB	41	8	115	83.67	26.28	58.24
R:VERB:FORM	117	28	46	80.69	71.78	78.73
R:VERB:INFL	7	0	0	100.00	100.00	100.00
R:VERB:SVA	132	12	22	91.67	85.71	90.41
R:VERB:TENSE	88	19	84	82.24	51.16	73.33
R:WO	29	24	54	54.72	34.94	49.15
U:ADJ	3	1	6	75.00	33.33	60.00
U:ADV	3	0	16	100.00	15.79	48.39
U:CONJ	0	1	3	0.00	0.00	0.00
U:CONTR	7	0	0	100.00	100.00	100.00
U:DET	72	17	118	80.90	37.89	65.93
U:NOUN	2	0	9	100.00	18.18	52.63
U:NOUN:POSS	6	0	3	100.00	66.67	90.91
U:OTHER	1	0	20	100.00	4.76	20.00
U:PART	3	0	2	100.00	60.00	88.24
U:PREP	33	11	41	75.00	44.59	66.00
U:PRON	11	4	12	73.33	47.83	66.27
U:PUNCT	34	3	27	91.89	55.74	81.34
U:VERB	2	0	20	100.00	9.09	33.33
U:VERB:FORM	2	1	2	66.67	50.00	62.50
U:VERB:TENSE	2	0	20	100.00	9.09	33.33
UNK	11	0	63	100.00	14.86	46.61

TOKEN-LEVEL DETECTION
OPERATION LEVEL
	TP	FP	FN	P	R	F_0.5
M	1193	130	487	90.17	71.01	85.56
R	2203	197	1669	91.79	56.90	81.76
U	240	28	413	89.55	36.75	69.57
UNK	19	0	78	100.00	19.59	54.91

ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
ADJ	13	2	39	86.67	25.00	58.04
ADJ:FORM	5	0	6	100.00	45.45	80.65
ADV	22	0	56	100.00	28.21	66.27
CONJ	10	0	20	100.00	33.33	71.43
CONTR	10	0	3	100.00	76.92	94.34
DET	337	42	299	88.92	52.99	78.30
MORPH	68	2	81	97.14	45.64	79.25
NOUN	40	4	110	90.91	26.67	61.35
NOUN:INFL	8	0	9	100.00	47.06	81.63
NOUN:NUM	149	17	80	89.76	65.07	83.43
NOUN:POSS	33	5	33	86.84	50.00	75.69
ORTH	308	17	167	94.77	64.84	86.76
OTHER	430	29	646	93.68	39.96	73.83
PART	16	1	8	94.12	66.67	86.96
PREP	284	50	207	85.03	57.84	77.72
PRON	86	9	56	90.53	60.56	82.38
PUNCT	953	111	244	89.57	79.62	87.38
SPELL	290	3	27	98.98	91.48	97.38
UNK	19	0	78	100.00	19.59	54.91
VERB	91	9	186	91.00	32.85	67.21
VERB:FORM	139	19	51	87.97	73.16	84.55
VERB:INFL	6	0	0	100.00	100.00	100.00
VERB:SVA	132	8	24	94.29	84.62	92.18
VERB:TENSE	128	15	149	89.51	46.21	75.38
WO	78	12	68	86.67	53.42	77.08

OPERATION + ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
M:ADJ	1	1	6	50.00	14.29	33.33
M:ADV	7	0	14	100.00	33.33	71.43
M:CONJ	8	0	14	100.00	36.36	74.07
M:DET	207	17	98	92.41	67.87	86.18
M:NOUN	6	0	12	100.00	33.33	71.43
M:NOUN:POSS	10	5	8	66.67	55.56	64.10
M:OTHER	30	0	48	100.00	38.46	75.76
M:PART	4	0	1	100.00	80.00	95.24
M:PREP	71	12	49	85.54	59.17	78.54
M:PRON	52	4	25	92.86	67.53	86.38
M:PUNCT	728	84	173	89.66	80.80	87.73
M:VERB	22	5	22	81.48	50.00	72.37
M:VERB:FORM	17	2	0	89.47	100.00	91.40
M:VERB:TENSE	30	0	17	100.00	63.83	89.82
R:ADJ	10	0	25	100.00	28.57	66.67
R:ADJ:FORM	5	0	6	100.00	45.45	80.65
R:ADV	10	0	23	100.00	30.30	68.49
R:CONJ	0	0	4	100.00	0.00	0.00
R:CONTR	4	0	1	100.00	80.00	95.24
R:DET	48	10	63	82.76	43.24	69.97
R:MORPH	68	2	81	97.14	45.64	79.25
R:NOUN	30	4	81	88.24	27.03	60.73
R:NOUN:INFL	8	0	9	100.00	47.06	81.63
R:NOUN:NUM	149	17	80	89.76	65.07	83.43
R:NOUN:POSS	17	0	17	100.00	50.00	83.33
R:ORTH	308	17	167	94.77	64.84	86.76
R:OTHER	379	29	524	92.89	41.97	74.75
R:PART	9	1	5	90.00	64.29	83.33
R:PREP	178	31	109	85.17	62.02	79.25
R:PRON	23	2	20	92.00	53.49	80.42
R:PUNCT	175	25	37	87.50	82.55	86.46
R:SPELL	290	3	27	98.98	91.48	97.38
R:VERB	62	4	137	93.94	31.16	66.95
R:VERB:FORM	120	17	48	87.59	71.43	83.80
R:VERB:INFL	6	0	0	100.00	100.00	100.00
R:VERB:SVA	132	8	24	94.29	84.62	92.18
R:VERB:TENSE	94	15	113	86.24	45.41	73.09
R:WO	78	12	68	86.67	53.42	77.08
U:ADJ	2	1	8	66.67	20.00	45.45
U:ADV	5	0	19	100.00	20.83	56.82
U:CONJ	2	0	2	100.00	50.00	83.33
U:CONTR	6	0	2	100.00	75.00	93.75
U:DET	82	15	138	84.54	37.27	67.43
U:NOUN	4	0	17	100.00	19.05	54.05
U:NOUN:POSS	6	0	8	100.00	42.86	78.95
U:OTHER	21	0	74	100.00	22.11	58.66
U:PART	3	0	2	100.00	60.00	88.24
U:PREP	35	7	49	83.33	41.67	69.44
U:PRON	11	3	11	78.57	50.00	70.51
U:PUNCT	50	2	34	96.15	59.52	85.62
U:VERB	7	0	27	100.00	20.59	56.45
U:VERB:FORM	2	0	3	100.00	40.00	76.92
U:VERB:TENSE	4	0	19	100.00	17.39	51.28
UNK	19	0	78	100.00	19.59	54.91

OVERALL
	TP	FP	FN	P	R	F_0.5
Span-level correction	1115	419	1189	72.69	48.39	66.05
Span-level detection	1282	253	1074	83.52	54.41	75.45
Token-level detection	1590	129	1188	92.50	57.24	82.35

SPAN-LEVEL CORRECTION
OPERATION LEVEL
	TP	FP	FN	P	R	F_0.5
M	338	138	262	71.01	56.33	67.49
R	706	266	787	72.63	47.29	65.60
U	71	15	140	82.56	33.65	63.96

ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
ADJ	3	2	11	60.00	21.43	44.12
ADJ:FORM	2	0	4	100.00	33.33	71.43
ADV	6	2	27	75.00	18.18	46.15
CONJ	1	0	6	100.00	14.29	45.45
CONTR	5	0	0	100.00	100.00	100.00
DET	135	37	140	78.49	49.09	70.09
MORPH	36	9	42	80.00	46.15	69.77
NOUN	3	6	52	33.33	5.45	16.48
NOUN:INFL	4	0	1	100.00	80.00	95.24
NOUN:NUM	76	25	43	75.25	63.87	72.66
NOUN:POSS	6	3	5	66.67	54.55	63.83
ORTH	108	13	96	89.26	52.94	78.49
OTHER	48	44	281	52.17	14.59	34.43
PART	11	3	5	78.57	68.75	76.39
PREP	103	37	92	73.57	52.82	68.21
PRON	40	12	43	76.92	48.19	68.73
PUNCT	208	103	125	66.88	62.46	65.95
SPELL	95	30	22	76.00	81.20	76.99
VERB	24	14	73	63.16	24.74	48.19
VERB:FORM	74	32	24	69.81	75.51	70.88
VERB:INFL	3	0	0	100.00	100.00	100.00
VERB:SVA	61	9	17	87.14	78.21	85.20
VERB:TENSE	55	28	61	66.27	47.41	61.38
WO	8	10	19	44.44	29.63	40.40

OPERATION + ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
M:ADJ	1	0	1	100.00	50.00	83.33
M:ADV	2	2	8	50.00	20.00	38.46
M:CONJ	0	0	3	100.00	0.00	0.00
M:DET	79	25	46	75.96	63.20	73.01
M:NOUN	0	0	1	100.00	0.00	0.00
M:NOUN:POSS	2	1	2	66.67	50.00	62.50
M:OTHER	0	0	33	100.00	0.00	0.00
M:PART	0	0	2	100.00	0.00	0.00
M:PREP	28	12	29	70.00	49.12	64.52
M:PRON	26	8	18	76.47	59.09	72.22
M:PUNCT	167	68	102	71.06	62.08	69.07
M:VERB	9	7	8	56.25	52.94	55.56
M:VERB:FORM	11	8	2	57.89	84.62	61.80
M:VERB:TENSE	13	7	7	65.00	65.00	65.00
R:ADJ	1	1	7	50.00	12.50	31.25
R:ADJ:FORM	2	0	4	100.00	33.33	71.43
R:ADV	4	0	9	100.00	30.77	68.97
R:CONTR	2	0	0	100.00	100.00	100.00
R:DET	18	6	30	75.00	37.50	62.50
R:MORPH	36	9	42	80.00	46.15	69.77
R:NOUN	3	6	45	33.33	6.25	17.86
R:NOUN:INFL	4	0	1	100.00	80.00	95.24
R:NOUN:NUM	76	25	43	75.25	63.87	72.66
R:NOUN:POSS	2	1	3	66.67	40.00	58.82
R:ORTH	108	13	96	89.26	52.94	78.49
R:OTHER	48	44	240	52.17	16.67	36.59
R:PART	10	3	2	76.92	83.33	78.12
R:PREP	59	21	50	73.75	54.13	68.76
R:PRON	10	1	16	90.91	38.46	71.43
R:PUNCT	36	35	14	50.70	72.00	53.89
R:SPELL	95	30	22	76.00	81.20	76.99
R:VERB	15	7	55	68.18	21.43	47.47
R:VERB:FORM	63	24	22	72.41	74.12	72.75
R:VERB:INFL	3	0	0	100.00	100.00	100.00
R:VERB:SVA	61	9	17	87.14	78.21	85.20
R:VERB:TENSE	42	21	50	66.67	45.65	61.05
R:WO	8	10	19	44.44	29.63	40.40
U:ADJ	1	1	3	50.00	25.00	41.67
U:ADV	0	0	10	100.00	0.00	0.00
U:CONJ	1	0	3	100.00	25.00	62.50
U:CONTR	3	0	0	100.00	100.00	100.00
U:DET	38	6	64	86.36	37.25	68.35
U:NOUN	0	0	6	100.00	0.00	0.00
U:NOUN:POSS	2	1	0	66.67	100.00	71.43
U:OTHER	0	0	8	100.00	0.00	0.00
U:PART	1	0	1	100.00	50.00	83.33
U:PREP	16	4	13	80.00	55.17	73.39
U:PRON	4	3	9	57.14	30.77	48.78
U:PUNCT	5	0	9	100.00	35.71	73.53
U:VERB	0	0	10	100.00	0.00	0.00
U:VERB:TENSE	0	0	4	100.00	0.00	0.00

SPAN-LEVEL DETECTION
OPERATION LEVEL
	TP	FP	FN	P	R	F_0.5
M	365	98	219	78.83	62.50	74.92
R	823	146	684	84.93	54.61	76.44
U	88	9	132	90.72	40.00	72.37
UNK	6	0	39	100.00	13.33	43.48

ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
ADJ	5	1	12	83.33	29.41	60.98
ADJ:FORM	1	0	4	100.00	20.00	55.56
ADV	8	2	25	80.00	24.24	54.79
CONJ	1	0	4	100.00	20.00	55.56
CONTR	4	0	0	100.00	100.00	100.00
DET	147	28	125	84.00	54.04	75.62
MORPH	39	7	43	84.78	47.56	73.31
NOUN	6	5	47	54.55	11.32	30.93
NOUN:INFL	4	0	1	100.00	80.00	95.24
NOUN:NUM	73	16	37	82.02	66.36	78.33
NOUN:POSS	7	1	4	87.50	63.64	81.40
ORTH	121	6	78	95.28	60.80	85.57
OTHER	136	22	244	86.08	35.79	67.19
PART	11	3	3	78.57	78.57	78.57
PREP	113	16	82	87.60	57.95	79.47
PRON	46	6	31	88.46	59.74	80.70
PUNCT	218	75	114	74.40	65.66	72.47
SPELL	99	6	10	94.29	90.83	93.57
UNK	6	0	39	100.00	13.33	43.48
VERB	36	8	72	81.82	33.33	63.38
VERB:FORM	69	22	20	75.82	77.53	76.16
VERB:INFL	3	0	0	100.00	100.00	100.00
VERB:SVA	63	4	14	94.03	81.82	91.30
VERB:TENSE	58	17	51	77.33	53.21	70.90
WO	8	8	14	50.00	36.36	46.51

OPERATION + ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
M:ADJ	1	0	1	100.00	50.00	83.33
M:ADV	2	2	7	50.00	22.22	40.00
M:CONJ	1	0	1	100.00	50.00	83.33
M:DET	85	18	40	82.52	68.00	79.14
M:NOUN	1	0	0	100.00	100.00	100.00
M:NOUN:POSS	2	1	2	66.67	50.00	62.50
M:OTHER	9	0	26	100.00	25.71	63.38
M:PREP	30	8	26	78.95	53.57	72.12
M:PRON	26	3	12	89.66	68.42	84.42
M:PUNCT	171	48	93	78.08	64.77	75.00
M:VERB	11	4	6	73.33	64.71	71.43
M:VERB:FORM	11	8	0	57.89	100.00	63.22
M:VERB:TENSE	15	6	5	71.43	75.00	72.12
R:ADJ	2	1	8	66.67	20.00	45.45
R:ADJ:FORM	1	0	4	100.00	20.00	55.56
R:ADV	5	0	9	100.00	35.71	73.53
R:CONTR	1	0	0	100.00	100.00	100.00
R:DET	21	5	27	80.77	43.75	69.08
R:MORPH	39	7	43	84.78	47.56	73.31
R:NOUN	4	5	40	44.44	9.09	25.00
R:NOUN:INFL	4	0	1	100.00	80.00	95.24
R:NOUN:NUM	73	16	37	82.02	66.36	78.33
R:NOUN:POSS	3	0	2	100.00	60.00	88.24
R:ORTH	121	6	78	95.28	60.80	85.57
R:OTHER	126	22	208	85.14	37.72	68.03
R:PART	10	3	2	76.92	83.33	78.12
R:PREP	66	6	42	91.67	61.11	83.33
R:PRON	11	1	14	91.67	44.00	75.34
R:PUNCT	39	27	14	59.09	73.58	61.51
R:SPELL	99	6	10	94.29	90.83	93.57
R:VERB	25	4	55	86.21	31.25	63.78
R:VERB:FORM	58	14	20	80.56	74.36	79.23
R:VERB:INFL	3	0	0	100.00	100.00	100.00
R:VERB:SVA	63	4	14	94.03	81.82	91.30
R:VERB:TENSE	41	11	42	78.85	49.40	70.45
R:WO	8	8	14	50.00	36.36	46.51
U:ADJ	2	0	3	100.00	40.00	76.92
U:ADV	1	0	9	100.00	10.00	35.71
U:CONJ	0	0	3	100.00	0.00	0.00
U:CONTR	3	0	0	100.00	100.00	100.00
U:DET	41	5	58	89.13	41.41	72.44
U:NOUN	1	0	7	100.00	12.50	41.67
U:NOUN:POSS	2	0	0	100.00	100.00	100.00
U:OTHER	1	0	10	100.00	9.09	33.33
U:PART	1	0	1	100.00	50.00	83.33
U:PREP	17	2	14	89.47	54.84	79.44
U:PRON	9	2	5	81.82	64.29	77.59
U:PUNCT	8	0	7	100.00	53.33	85.11
U:VERB	0	0	11	100.00	0.00	0.00
U:VERB:TENSE	2	0	4	100.00	33.33	71.43
UNK	6	0	39	100.00	13.33	43.48

TOKEN-LEVEL DETECTION
OPERATION LEVEL
	TP	FP	FN	P	R	F_0.5
M	430	55	196	88.66	68.69	83.79
R	1030	69	770	93.72	57.22	83.12
U	118	5	170	95.93	40.97	75.64
UNK	12	0	52	100.00	18.75	53.57

ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
ADJ	6	0	14	100.00	30.00	68.18
ADJ:FORM	2	0	3	100.00	40.00	76.92
ADV	13	0	26	100.00	33.33	71.43
CONJ	7	0	5	100.00	58.33	87.50
CONTR	4	0	1	100.00	80.00	95.24
DET	148	16	140	90.24	51.39	78.39
MORPH	42	1	42	97.67	50.00	82.03
NOUN	18	2	53	90.00	25.35	59.60
NOUN:INFL	4	0	1	100.00	80.00	95.24
NOUN:NUM	75	8	36	90.36	67.57	84.65
NOUN:POSS	10	1	8	90.91	55.56	80.65
ORTH	146	3	70	97.99	67.59	89.90
OTHER	239	16	317	93.73	42.99	75.82
PART	9	0	4	100.00	69.23	91.84
PREP	121	10	87	92.37	58.17	82.65
PRON	52	3	25	94.55	67.53	87.54
PUNCT	298	45	88	86.88	77.20	84.76
SPELL	99	1	8	99.00	92.52	97.63
UNK	12	0	52	100.00	18.75	53.57
VERB	54	5	93	91.53	36.73	70.50
VERB:FORM	75	9	20	89.29	78.95	87.01
VERB:INFL	2	0	0	100.00	100.00	100.00
VERB:SVA	67	1	14	98.53	82.72	94.90
VERB:TENSE	63	7	65	90.00	49.22	77.21
WO	24	1	16	96.00	60.00	85.71

OPERATION + ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
M:ADJ	1	0	1	100.00	50.00	83.33
M:ADV	5	0	6	100.00	45.45	80.65
M:CONJ	5	0	3	100.00	62.50	89.29
M:DET	82	9	43	90.11	65.60	83.84
M:NOUN	3	0	3	100.00	50.00	83.33
M:NOUN:POSS	2	1	2	66.67	50.00	62.50
M:OTHER	14	0	25	100.00	35.90	73.68
M:PART	2	0	0	100.00	100.00	100.00
M:PREP	36	5	22	87.80	62.07	81.08
M:PRON	29	1	11	96.67	72.50	90.62
M:PUNCT	211	35	63	85.77	77.01	83.86
M:VERB	11	3	11	78.57	50.00	70.51
M:VERB:FORM	13	1	0	92.86	100.00	94.20
M:VERB:TENSE	16	0	6	100.00	72.73	93.02
R:ADJ	3	0	10	100.00	23.08	60.00
R:ADJ:FORM	2	0	3	100.00	40.00	76.92
R:ADV	7	0	11	100.00	38.89	76.09
R:CONTR	1	0	0	100.00	100.00	100.00
R:DET	20	4	29	83.33	40.82	68.97
R:MORPH	42	1	42	97.67	50.00	82.03
R:NOUN	13	2	39	86.67	25.00	58.04
R:NOUN:INFL	4	0	1	100.00	80.00	95.24
R:NOUN:NUM	75	8	36	90.36	67.57	84.65
R:NOUN:POSS	6	0	1	100.00	85.71	96.77
R:ORTH	146	3	70	97.99	67.59	89.90
R:OTHER	214	16	270	93.04	44.21	76.21
R:PART	6	0	3	100.00	66.67	90.91
R:PREP	67	4	48	94.37	58.26	83.96
R:PRON	15	1	9	93.75	62.50	85.23
R:PUNCT	69	10	17	87.34	80.23	85.82
R:SPELL	99	1	8	99.00	92.52	97.63
R:VERB	41	2	69	95.35	37.27	72.70
R:VERB:FORM	62	8	19	88.57	76.54	85.87
R:VERB:INFL	2	0	0	100.00	100.00	100.00
R:VERB:SVA	67	1	14	98.53	82.72	94.90
R:VERB:TENSE	45	7	55	86.54	45.00	73.05
R:WO	24	1	16	96.00	60.00	85.71
U:ADJ	2	0	3	100.00	40.00	76.92
U:ADV	1	0	9	100.00	10.00	35.71
U:CONJ	2	0	2	100.00	50.00	83.33
U:CONTR	3	0	1	100.00	75.00	93.75
U:DET	46	3	68	93.88	40.35	74.19
U:NOUN	2	0	11	100.00	15.38	47.62
U:NOUN:POSS	2	0	5	100.00	28.57	66.67
U:OTHER	11	0	22	100.00	33.33	71.43
U:PART	1	0	1	100.00	50.00	83.33
U:PREP	18	1	17	94.74	51.43	81.08
U:PRON	8	1	5	88.89	61.54	81.63
U:PUNCT	18	0	8	100.00	69.23	91.84
U:VERB	2	0	13	100.00	13.33	43.48
U:VERB:FORM	0	0	1	100.00	0.00	0.00
U:VERB:TENSE	2	0	4	100.00	33.33	71.43
UNK	12	0	52	100.00	18.75	53.57

OVERALL
	TP	FP	FN	P	R	F_0.5
Span-level correction	806	234	834	77.50	49.15	69.48
Span-level detection	895	148	770	85.81	53.75	76.67
Token-level detection	1078	92	911	92.14	54.20	80.82

SPAN-LEVEL CORRECTION
OPERATION LEVEL
	TP	FP	FN	P	R	F_0.5
M	326	72	172	81.91	65.46	77.99
R	430	150	559	74.14	43.48	64.97
U	50	12	103	80.65	32.68	62.34

ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
ADJ	3	1	12	75.00	20.00	48.39
ADJ:FORM	2	0	1	100.00	66.67	90.91
ADV	3	1	19	75.00	13.64	39.47
CONJ	0	1	9	0.00	0.00	0.00
CONTR	3	0	0	100.00	100.00	100.00
DET	108	23	101	82.44	51.67	73.67
MORPH	16	1	20	94.12	44.44	76.92
NOUN	5	2	34	71.43	12.82	37.31
NOUN:NUM	46	3	23	93.88	66.67	86.79
NOUN:POSS	8	1	5	88.89	61.54	81.63
ORTH	33	11	34	75.00	49.25	67.90
OTHER	16	24	190	40.00	7.77	21.86
PART	7	2	6	77.78	53.85	71.43
PREP	90	36	73	71.43	55.21	67.47
PRON	20	8	23	71.43	46.51	64.52
PUNCT	243	51	75	82.65	76.42	81.33
SPELL	58	26	13	69.05	81.69	71.25
VERB	8	5	66	61.54	10.81	31.75
VERB:FORM	43	9	26	82.69	62.32	77.62
VERB:SVA	46	8	9	85.19	83.64	84.87
VERB:TENSE	34	10	62	77.27	35.42	62.50
WO	14	11	33	56.00	29.79	47.62

OPERATION + ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
M:ADJ	1	0	2	100.00	33.33	71.43
M:ADV	1	1	6	50.00	14.29	33.33
M:CONJ	0	0	7	100.00	0.00	0.00
M:DET	72	18	46	80.00	61.02	75.31
M:NOUN	0	0	2	100.00	0.00	0.00
M:NOUN:POSS	3	1	1	75.00	75.00	75.00
M:OTHER	1	0	14	100.00	6.67	26.32
M:PART	2	0	2	100.00	50.00	83.33
M:PREP	17	9	13	65.38	56.67	63.43
M:PRON	18	5	11	78.26	62.07	74.38
M:PUNCT	199	33	52	85.78	79.28	84.39
M:VERB	4	3	5	57.14	44.44	54.05
M:VERB:FORM	1	0	0	100.00	100.00	100.00
M:VERB:TENSE	7	2	11	77.78	38.89	64.81
R:ADJ	1	0	8	100.00	11.11	38.46
R:ADJ:FORM	2	0	1	100.00	66.67	90.91
R:ADV	2	0	6	100.00	25.00	62.50
R:CONJ	0	0	2	100.00	0.00	0.00
R:CONTR	1	0	0	100.00	100.00	100.00
R:DET	19	1	21	95.00	47.50	79.17
R:MORPH	16	1	20	94.12	44.44	76.92
R:NOUN	5	2	31	71.43	13.89	39.06
R:NOUN:NUM	46	3	23	93.88	66.67	86.79
R:NOUN:POSS	3	0	1	100.00	75.00	93.75
R:ORTH	33	11	34	75.00	49.25	67.90
R:OTHER	15	24	168	38.46	8.20	22.12
R:PART	4	2	3	66.67	57.14	64.52
R:PREP	59	23	46	71.95	56.19	68.13
R:PRON	1	2	8	33.33	11.11	23.81
R:PUNCT	34	17	15	66.67	69.39	67.19
R:SPELL	58	26	13	69.05	81.69	71.25
R:VERB	4	2	53	66.67	7.02	24.69
R:VERB:FORM	40	9	24	81.63	62.50	76.92
R:VERB:SVA	46	8	9	85.19	83.64	84.87
R:VERB:TENSE	27	8	40	77.14	40.30	65.22
R:WO	14	11	33	56.00	29.79	47.62
U:ADJ	1	1	2	50.00	33.33	45.45
U:ADV	0	0	7	100.00	0.00	0.00
U:CONJ	0	1	0	0.00	100.00	0.00
U:CONTR	2	0	0	100.00	100.00	100.00
U:DET	17	4	34	80.95	33.33	62.96
U:NOUN	0	0	1	100.00	0.00	0.00
U:NOUN:POSS	2	0	3	100.00	40.00	76.92
U:OTHER	0	0	8	100.00	0.00	0.00
U:PART	1	0	1	100.00	50.00	83.33
U:PREP	14	4	14	77.78	50.00	70.00
U:PRON	1	1	4	50.00	20.00	38.46
U:PUNCT	10	1	8	90.91	55.56	80.65
U:VERB	0	0	8	100.00	0.00	0.00
U:VERB:FORM	2	0	2	100.00	50.00	83.33
U:VERB:TENSE	0	0	11	100.00	0.00	0.00

SPAN-LEVEL DETECTION
OPERATION LEVEL
	TP	FP	FN	P	R	F_0.5
M	344	55	150	86.22	69.64	82.30
R	493	83	499	85.59	49.70	74.79
U	55	10	102	84.62	35.03	65.95
UNK	3	0	19	100.00	13.64	44.12

ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
ADJ	7	1	12	87.50	36.84	68.63
ADJ:FORM	2	0	0	100.00	100.00	100.00
ADV	5	1	18	83.33	21.74	53.19
CONJ	0	1	8	0.00	0.00	0.00
CONTR	3	0	0	100.00	100.00	100.00
DET	115	16	94	87.79	55.02	78.44
MORPH	14	0	18	100.00	43.75	79.55
NOUN	8	2	32	80.00	20.00	50.00
NOUN:NUM	43	2	22	95.56	66.15	87.76
NOUN:POSS	8	1	6	88.89	57.14	80.00
ORTH	45	9	32	83.33	58.44	76.79
OTHER	51	12	174	80.95	22.67	53.46
PART	7	2	7	77.78	50.00	70.00
PREP	96	26	63	78.69	60.38	74.19
PRON	22	3	24	88.00	47.83	75.34
PUNCT	248	47	70	84.07	77.99	82.78
SPELL	62	2	7	96.88	89.86	95.38
UNK	3	0	19	100.00	13.64	44.12
VERB	17	3	54	85.00	23.94	56.29
VERB:FORM	43	5	24	89.58	64.18	83.01
VERB:SVA	48	3	6	94.12	88.89	93.02
VERB:TENSE	37	4	48	90.24	43.53	74.30
WO	11	8	32	57.89	25.58	46.22

OPERATION + ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
M:ADJ	1	0	4	100.00	20.00	55.56
M:ADV	1	1	6	50.00	14.29	33.33
M:CONJ	0	0	6	100.00	0.00	0.00
M:DET	78	12	37	86.67	67.83	82.11
M:NOUN	1	0	1	100.00	50.00	83.33
M:NOUN:POSS	3	1	1	75.00	75.00	75.00
M:OTHER	7	0	15	100.00	31.82	70.00
M:PART	2	0	2	100.00	50.00	83.33
M:PREP	17	8	10	68.00	62.96	66.93
M:PRON	19	2	11	90.48	63.33	83.33
M:PUNCT	201	29	48	87.39	80.72	85.97
M:VERB	6	2	2	75.00	75.00	75.00
M:VERB:FORM	1	0	0	100.00	100.00	100.00
M:VERB:TENSE	7	0	7	100.00	50.00	83.33
R:ADJ	5	0	6	100.00	45.45	80.65
R:ADJ:FORM	2	0	0	100.00	100.00	100.00
R:ADV	2	0	6	100.00	25.00	62.50
R:CONJ	0	0	2	100.00	0.00	0.00
R:CONTR	1	0	0	100.00	100.00	100.00
R:DET	21	1	22	95.45	48.84	80.15
R:MORPH	14	0	18	100.00	43.75	79.55
R:NOUN	7	2	29	77.78	19.44	48.61
R:NOUN:NUM	43	2	22	95.56	66.15	87.76
R:NOUN:POSS	3	0	2	100.00	60.00	88.24
R:ORTH	45	9	32	83.33	58.44	76.79
R:OTHER	44	12	151	78.57	22.56	52.51
R:PART	4	2	4	66.67	50.00	62.50
R:PREP	65	15	40	81.25	61.90	76.47
R:PRON	2	0	7	100.00	22.22	58.82
R:PUNCT	35	17	15	67.31	70.00	67.83
R:SPELL	62	2	7	96.88	89.86	95.38
R:VERB	9	1	46	90.00	16.36	47.37
R:VERB:FORM	40	5	22	88.89	64.52	82.64
R:VERB:SVA	48	3	6	94.12	88.89	93.02
R:VERB:TENSE	30	4	30	88.24	50.00	76.53
R:WO	11	8	32	57.89	25.58	46.22
U:ADJ	1	1	2	50.00	33.33	45.45
U:ADV	2	0	6	100.00	25.00	62.50
U:CONJ	0	1	0	0.00	100.00	0.00
U:CONTR	2	0	0	100.00	100.00	100.00
U:DET	16	3	35	84.21	31.37	62.99
U:NOUN	0	0	2	100.00	0.00	0.00
U:NOUN:POSS	2	0	3	100.00	40.00	76.92
U:OTHER	0	0	8	100.00	0.00	0.00
U:PART	1	0	1	100.00	50.00	83.33
U:PREP	14	3	13	82.35	51.85	73.68
U:PRON	1	1	6	50.00	14.29	33.33
U:PUNCT	12	1	7	92.31	63.16	84.51
U:VERB	2	0	6	100.00	25.00	62.50
U:VERB:FORM	2	0	2	100.00	50.00	83.33
U:VERB:TENSE	0	0	11	100.00	0.00	0.00
UNK	3	0	19	100.00	13.64	44.12

TOKEN-LEVEL DETECTION
OPERATION LEVEL
	TP	FP	FN	P	R	F_0.5
M	371	32	172	92.06	68.32	86.08
R	629	54	559	92.09	52.95	80.23
U	74	6	160	92.50	31.62	66.79
UNK	4	0	20	100.00	16.67	50.00

ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
ADJ	5	2	13	71.43	27.78	54.35
ADJ:FORM	3	0	0	100.00	100.00	100.00
ADV	6	0	22	100.00	21.43	57.69
CONJ	1	0	12	100.00	7.69	29.41
CONTR	2	0	0	100.00	100.00	100.00
DET	121	7	100	94.53	54.75	82.54
MORPH	15	0	23	100.00	39.47	76.53
NOUN	11	2	33	84.62	25.00	57.29
NOUN:NUM	44	1	24	97.78	64.71	88.71
NOUN:POSS	10	1	10	90.91	50.00	78.12
ORTH	79	9	29	89.77	73.15	85.87
OTHER	113	7	234	94.17	32.56	68.32
PART	4	1	3	80.00	57.14	74.07
PREP	101	15	70	87.07	59.06	79.53
PRON	21	3	25	87.50	45.65	73.94
PUNCT	288	26	72	91.72	80.00	89.11
SPELL	63	1	7	98.44	90.00	96.63
UNK	4	0	20	100.00	16.67	50.00
VERB	26	1	73	96.30	26.26	62.80
VERB:FORM	42	3	26	93.33	61.76	84.68
VERB:SVA	44	3	8	93.62	84.62	91.67
VERB:TENSE	42	3	65	93.33	39.25	73.17
WO	33	7	42	82.50	44.00	70.21

OPERATION + ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
M:ADJ	0	1	2	0.00	0.00	0.00
M:ADV	2	0	7	100.00	22.22	58.82
M:CONJ	1	0	8	100.00	11.11	38.46
M:DET	80	3	38	96.39	67.80	88.89
M:NOUN	1	0	4	100.00	20.00	55.56
M:NOUN:POSS	3	1	3	75.00	50.00	68.18
M:OTHER	11	0	15	100.00	42.31	78.57
M:PART	2	0	0	100.00	100.00	100.00
M:PREP	18	4	14	81.82	56.25	75.00
M:PRON	17	2	10	89.47	62.96	82.52
M:PUNCT	218	20	53	91.60	80.44	89.13
M:VERB	8	1	8	88.89	50.00	76.92
M:VERB:FORM	2	0	0	100.00	100.00	100.00
M:VERB:TENSE	8	0	10	100.00	44.44	80.00
R:ADJ	5	0	9	100.00	35.71	73.53
R:ADJ:FORM	3	0	0	100.00	100.00	100.00
R:ADV	2	0	8	100.00	20.00	55.56
R:CONJ	0	0	4	100.00	0.00	0.00
R:CONTR	1	0	0	100.00	100.00	100.00
R:DET	21	1	22	95.45	48.84	80.15
R:MORPH	15	0	23	100.00	39.47	76.53
R:NOUN	10	2	24	83.33	29.41	60.98
R:NOUN:NUM	44	1	24	97.78	64.71	88.71
R:NOUN:POSS	5	0	4	100.00	55.56	86.21
R:ORTH	79	9	29	89.77	73.15	85.87
R:OTHER	92	7	173	92.93	34.72	69.59
R:PART	1	1	2	50.00	33.33	45.45
R:PREP	68	10	39	87.18	63.55	81.15
R:PRON	2	0	10	100.00	16.67	50.00
R:PUNCT	57	6	9	90.48	86.36	89.62
R:SPELL	63	1	7	98.44	90.00	96.63
R:VERB	13	0	54	100.00	19.40	54.62
R:VERB:FORM	38	3	24	92.68	61.29	84.07
R:VERB:SVA	44	3	8	93.62	84.62	91.67
R:VERB:TENSE	33	3	44	91.67	42.86	74.66
R:WO	33	7	42	82.50	44.00	70.21
U:ADJ	0	1	2	0.00	0.00	0.00
U:ADV	2	0	7	100.00	22.22	58.82
U:CONTR	1	0	0	100.00	100.00	100.00
U:DET	20	3	40	86.96	33.33	65.79
U:NOUN	0	0	5	100.00	0.00	0.00
U:NOUN:POSS	2	0	3	100.00	40.00	76.92
U:OTHER	10	0	46	100.00	17.86	52.08
U:PART	1	0	1	100.00	50.00	83.33
U:PREP	15	1	17	93.75	46.88	78.12
U:PRON	2	1	5	66.67	28.57	52.63
U:PUNCT	13	0	10	100.00	56.52	86.67
U:VERB	5	0	11	100.00	31.25	69.44
U:VERB:FORM	2	0	2	100.00	50.00	83.33
U:VERB:TENSE	1	0	11	100.00	8.33	31.25
UNK	4	0	20	100.00	16.67	50.00

OVERALL
	TP	FP	FN	P	R	F_0.5
Span-level correction	388	100	271	79.51	58.88	74.30
Span-level detection	417	72	273	85.28	60.43	78.80
Token-level detection	484	53	316	90.13	60.50	82.09

SPAN-LEVEL CORRECTION
OPERATION LEVEL
	TP	FP	FN	P	R	F_0.5
M	167	24	62	87.43	72.93	84.09
R	196	63	166	75.68	54.14	70.10
U	25	13	43	65.79	36.76	56.82

ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
ADJ	0	0	8	100.00	0.00	0.00
ADJ:FORM	0	0	2	100.00	0.00	0.00
ADV	2	0	2	100.00	50.00	83.33
CONJ	0	0	2	100.00	0.00	0.00
CONTR	3	0	0	100.00	100.00	100.00
DET	49	13	46	79.03	51.58	71.43
MORPH	8	0	11	100.00	42.11	78.43
NOUN	2	2	11	50.00	15.38	34.48
NOUN:INFL	1	0	0	100.00	100.00	100.00
NOUN:NUM	19	9	9	67.86	67.86	67.86
NOUN:POSS	2	1	5	66.67	28.57	52.63
ORTH	22	3	18	88.00	55.00	78.57
OTHER	17	6	42	73.91	28.81	56.29
PART	1	1	1	50.00	50.00	50.00
PREP	44	18	32	70.97	57.89	67.90
PRON	5	2	5	71.43	50.00	65.79
PUNCT	133	23	30	85.26	81.60	84.50
SPELL	34	5	4	87.18	89.47	87.63
VERB	1	2	13	33.33	7.14	19.23
VERB:FORM	12	4	2	75.00	85.71	76.92
VERB:SVA	10	3	1	76.92	90.91	79.37
VERB:TENSE	16	2	18	88.89	47.06	75.47
WO	7	6	9	53.85	43.75	51.47

OPERATION + ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
M:ADJ	0	0	2	100.00	0.00	0.00
M:ADV	1	0	0	100.00	100.00	100.00
M:CONJ	0	0	2	100.00	0.00	0.00
M:DET	33	4	16	89.19	67.35	83.76
M:NOUN	0	0	1	100.00	0.00	0.00
M:NOUN:POSS	0	1	0	0.00	100.00	0.00
M:OTHER	0	0	6	100.00	0.00	0.00
M:PREP	13	3	9	81.25	59.09	75.58
M:PRON	2	1	3	66.67	40.00	58.82
M:PUNCT	111	15	20	88.10	84.73	87.40
M:VERB	0	0	2	100.00	0.00	0.00
M:VERB:FORM	2	0	0	100.00	100.00	100.00
M:VERB:TENSE	5	0	1	100.00	83.33	96.15
R:ADJ	0	0	5	100.00	0.00	0.00
R:ADJ:FORM	0	0	2	100.00	0.00	0.00
R:ADV	1	0	1	100.00	50.00	83.33
R:CONTR	1	0	0	100.00	100.00	100.00
R:DET	5	4	10	55.56	33.33	49.02
R:MORPH	8	0	11	100.00	42.11	78.43
R:NOUN	2	2	10	50.00	16.67	35.71
R:NOUN:INFL	1	0	0	100.00	100.00	100.00
R:NOUN:NUM	19	9	9	67.86	67.86	67.86
R:NOUN:POSS	0	0	5	100.00	0.00	0.00
R:ORTH	22	3	18	88.00	55.00	78.57
R:OTHER	17	6	36	73.91	32.08	58.62
R:PART	1	1	1	50.00	50.00	50.00
R:PREP	28	11	12	71.79	70.00	71.43
R:PRON	2	0	1	100.00	66.67	90.91
R:PUNCT	16	5	6	76.19	72.73	75.47
R:SPELL	34	5	4	87.18	89.47	87.63
R:VERB	1	2	10	33.33	9.09	21.74
R:VERB:FORM	10	4	2	71.43	83.33	73.53
R:VERB:SVA	10	3	1	76.92	90.91	79.37
R:VERB:TENSE	11	2	13	84.62	45.83	72.37
R:WO	7	6	9	53.85	43.75	51.47
U:ADJ	0	0	1	100.00	0.00	0.00
U:ADV	0	0	1	100.00	0.00	0.00
U:CONTR	2	0	0	100.00	100.00	100.00
U:DET	11	5	20	68.75	35.48	57.89
U:NOUN:POSS	2	0	0	100.00	100.00	100.00
U:PREP	3	4	11	42.86	21.43	35.71
U:PRON	1	1	1	50.00	50.00	50.00
U:PUNCT	6	3	4	66.67	60.00	65.22
U:VERB	0	0	1	100.00	0.00	0.00
U:VERB:TENSE	0	0	4	100.00	0.00	0.00

SPAN-LEVEL DETECTION
OPERATION LEVEL
	TP	FP	FN	P	R	F_0.5
M	171	21	67	89.06	71.85	84.99
R	218	41	159	84.17	57.82	77.14
U	27	10	45	72.97	37.50	61.36
UNK	1	0	2	100.00	33.33	71.43

ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
ADJ	0	0	7	100.00	0.00	0.00
ADJ:FORM	0	0	2	100.00	0.00	0.00
ADV	1	0	3	100.00	25.00	62.50
CONJ	0	0	3	100.00	0.00	0.00
CONTR	3	0	0	100.00	100.00	100.00
DET	50	11	48	81.97	51.02	73.10
MORPH	9	0	13	100.00	40.91	77.59
NOUN	6	1	13	85.71	31.58	63.83
NOUN:INFL	1	0	0	100.00	100.00	100.00
NOUN:NUM	18	7	9	72.00	66.67	70.87
NOUN:POSS	2	1	5	66.67	28.57	52.63
ORTH	24	1	18	96.00	57.14	84.51
OTHER	23	3	42	88.46	35.38	68.05
PART	2	1	1	66.67	66.67	66.67
PREP	49	11	31	81.67	61.25	76.56
PRON	7	1	5	87.50	58.33	79.55
PUNCT	134	21	33	86.45	80.24	85.13
SPELL	37	1	1	97.37	97.37	97.37
UNK	1	0	2	100.00	33.33	71.43
VERB	2	2	11	50.00	15.38	34.48
VERB:FORM	13	3	2	81.25	86.67	82.28
VERB:SVA	11	0	1	100.00	91.67	98.21
VERB:TENSE	18	2	15	90.00	54.55	79.65
WO	6	6	8	50.00	42.86	48.39

OPERATION + ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
M:ADJ	0	0	1	100.00	0.00	0.00
M:CONJ	0	0	3	100.00	0.00	0.00
M:DET	33	4	16	89.19	67.35	83.76
M:NOUN	1	0	2	100.00	33.33	71.43
M:NOUN:POSS	0	1	0	0.00	100.00	0.00
M:OTHER	1	0	6	100.00	14.29	45.45
M:PART	0	0	1	100.00	0.00	0.00
M:PREP	14	2	9	87.50	60.87	80.46
M:PRON	4	0	3	100.00	57.14	86.96
M:PUNCT	111	14	23	88.80	82.84	87.54
M:VERB	0	0	2	100.00	0.00	0.00
M:VERB:FORM	2	0	0	100.00	100.00	100.00
M:VERB:TENSE	5	0	1	100.00	83.33	96.15
R:ADJ	0	0	5	100.00	0.00	0.00
R:ADJ:FORM	0	0	2	100.00	0.00	0.00
R:ADV	1	0	2	100.00	33.33	71.43
R:CONTR	1	0	0	100.00	100.00	100.00
R:DET	6	3	11	66.67	35.29	56.60
R:MORPH	9	0	13	100.00	40.91	77.59
R:NOUN	4	1	11	80.00	26.67	57.14
R:NOUN:INFL	1	0	0	100.00	100.00	100.00
R:NOUN:NUM	18	7	9	72.00	66.67	70.87
R:NOUN:POSS	0	0	5	100.00	0.00	0.00
R:ORTH	24	1	18	96.00	57.14	84.51
R:OTHER	22	3	35	88.00	38.60	70.06
R:PART	2	1	0	66.67	100.00	71.43
R:PREP	33	6	11	84.62	75.00	82.50
R:PRON	2	0	1	100.00	66.67	90.91
R:PUNCT	15	5	6	75.00	71.43	74.26
R:SPELL	37	1	1	97.37	97.37	97.37
R:VERB	2	2	8	50.00	20.00	38.46
R:VERB:FORM	11	3	2	78.57	84.62	79.71
R:VERB:SVA	11	0	1	100.00	91.67	98.21
R:VERB:TENSE	13	2	10	86.67	56.52	78.31
R:WO	6	6	8	50.00	42.86	48.39
U:ADJ	0	0	1	100.00	0.00	0.00
U:ADV	0	0	1	100.00	0.00	0.00
U:CONTR	2	0	0	100.00	100.00	100.00
U:DET	11	4	21	73.33	34.38	59.78
U:NOUN	1	0	0	100.00	100.00	100.00
U:NOUN:POSS	2	0	0	100.00	100.00	100.00
U:OTHER	0	0	1	100.00	0.00	0.00
U:PREP	2	3	11	40.00	15.38	30.30
U:PRON	1	1	1	50.00	50.00	50.00
U:PUNCT	8	2	4	80.00	66.67	76.92
U:VERB	0	0	1	100.00	0.00	0.00
U:VERB:TENSE	0	0	4	100.00	0.00	0.00
UNK	1	0	2	100.00	33.33	71.43

TOKEN-LEVEL DETECTION
OPERATION LEVEL
	TP	FP	FN	P	R	F_0.5
M	189	18	72	91.30	72.41	86.78
R	261	24	185	91.58	58.52	82.28
U	33	11	54	75.00	37.93	62.74
UNK	1	0	5	100.00	16.67	50.00

ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
ADJ	0	0	9	100.00	0.00	0.00
ADJ:FORM	0	0	3	100.00	0.00	0.00
ADV	2	0	6	100.00	25.00	62.50
CONJ	1	0	3	100.00	25.00	62.50
CONTR	3	0	1	100.00	75.00	93.75
DET	55	9	50	85.94	52.38	76.18
MORPH	9	0	12	100.00	42.86	78.95
NOUN	6	0	14	100.00	30.00	68.18
NOUN:INFL	1	0	0	100.00	100.00	100.00
NOUN:NUM	18	4	12	81.82	60.00	76.27
NOUN:POSS	3	1	5	75.00	37.50	62.50
ORTH	32	1	21	96.97	60.38	86.49
OTHER	37	1	58	97.37	38.95	74.90
PART	2	0	1	100.00	66.67	90.91
PREP	49	10	34	83.05	59.04	76.80
PRON	7	1	5	87.50	58.33	79.55
PUNCT	160	17	33	90.40	82.90	88.79
SPELL	38	0	1	100.00	97.44	99.48
UNK	1	0	5	100.00	16.67	50.00
VERB	3	2	13	60.00	18.75	41.67
VERB:FORM	14	1	3	93.33	82.35	90.91
VERB:SVA	11	0	1	100.00	91.67	98.21
VERB:TENSE	17	2	16	89.47	51.52	77.98
WO	15	4	10	78.95	60.00	74.26

OPERATION + ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
M:ADJ	0	0	3	100.00	0.00	0.00
M:CONJ	1	0	3	100.00	25.00	62.50
M:DET	37	3	15	92.50	71.15	87.26
M:NOUN	1	0	2	100.00	33.33	71.43
M:NOUN:POSS	1	1	0	50.00	100.00	55.56
M:OTHER	3	0	7	100.00	30.00	68.18
M:PART	0	0	1	100.00	0.00	0.00
M:PREP	15	1	10	93.75	60.00	84.27
M:PRON	4	0	3	100.00	57.14	86.96
M:PUNCT	120	13	24	90.23	83.33	88.76
M:VERB	0	0	3	100.00	0.00	0.00
M:VERB:FORM	2	0	0	100.00	100.00	100.00
M:VERB:TENSE	5	0	1	100.00	83.33	96.15
R:ADJ	0	0	4	100.00	0.00	0.00
R:ADJ:FORM	0	0	3	100.00	0.00	0.00
R:ADV	1	0	3	100.00	25.00	62.50
R:CONTR	1	0	0	100.00	100.00	100.00
R:DET	6	2	11	75.00	35.29	61.22
R:MORPH	9	0	12	100.00	42.86	78.95
R:NOUN	3	0	11	100.00	21.43	57.69
R:NOUN:INFL	1	0	0	100.00	100.00	100.00
R:NOUN:NUM	18	4	12	81.82	60.00	76.27
R:NOUN:POSS	0	0	5	100.00	0.00	0.00
R:ORTH	32	1	21	96.97	60.38	86.49
R:OTHER	34	1	47	97.14	41.98	76.92
R:PART	2	0	0	100.00	100.00	100.00
R:PREP	33	5	13	86.84	71.74	83.33
R:PRON	2	0	1	100.00	66.67	90.91
R:PUNCT	28	2	6	93.33	82.35	90.91
R:SPELL	38	0	1	100.00	97.44	99.48
R:VERB	3	2	9	60.00	25.00	46.88
R:VERB:FORM	12	1	3	92.31	80.00	89.55
R:VERB:SVA	11	0	1	100.00	91.67	98.21
R:VERB:TENSE	12	2	12	85.71	50.00	75.00
R:WO	15	4	10	78.95	60.00	74.26
U:ADJ	0	0	2	100.00	0.00	0.00
U:ADV	1	0	3	100.00	25.00	62.50
U:CONTR	2	0	1	100.00	66.67	90.91
U:DET	12	4	24	75.00	33.33	60.00
U:NOUN	2	0	1	100.00	66.67	90.91
U:NOUN:POSS	2	0	0	100.00	100.00	100.00
U:OTHER	0	0	4	100.00	0.00	0.00
U:PREP	1	4	11	20.00	8.33	15.62
U:PRON	1	1	1	50.00	50.00	50.00
U:PUNCT	12	2	3	85.71	80.00	84.51
U:VERB	0	0	1	100.00	0.00	0.00
U:VERB:TENSE	0	0	3	100.00	0.00	0.00
UNK	1	0	5	100.00	16.67	50.00

OVERALL
	TP	FP	FN	P	R	F_0.5
Span-level correction	400	141	216	73.94	64.94	71.94
Span-level detection	438	103	192	80.96	69.52	78.38
Token-level detection	503	81	232	86.13	68.44	81.90

SPAN-LEVEL CORRECTION
OPERATION LEVEL
	TP	FP	FN	P	R	F_0.5
M	190	31	37	85.97	83.70	85.51
R	198	101	157	66.22	55.77	63.83
U	12	9	22	57.14	35.29	50.85

ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
ADJ	1	0	4	100.00	20.00	55.56
ADV	0	0	1	100.00	0.00	0.00
CONTR	0	0	1	100.00	0.00	0.00
DET	12	15	5	44.44	70.59	48.00
MORPH	2	3	5	40.00	28.57	37.04
NOUN	4	1	9	80.00	30.77	60.61
NOUN:INFL	2	0	9	100.00	18.18	52.63
NOUN:NUM	11	5	6	68.75	64.71	67.90
NOUN:POSS	8	2	11	80.00	42.11	67.80
ORTH	29	4	35	87.88	45.31	73.98
OTHER	13	13	37	50.00	26.00	42.21
PART	1	0	0	100.00	100.00	100.00
PREP	12	20	13	37.50	48.00	39.22
PRON	4	2	2	66.67	66.67	66.67
PUNCT	193	27	39	87.73	83.19	86.78
SPELL	75	27	20	73.53	78.95	74.55
VERB	6	2	11	75.00	35.29	61.22
VERB:FORM	8	9	3	47.06	72.73	50.63
VERB:INFL	3	0	1	100.00	75.00	93.75
VERB:SVA	9	5	1	64.29	90.00	68.18
VERB:TENSE	3	3	3	50.00	50.00	50.00
WO	4	3	0	57.14	100.00	62.50

OPERATION + ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
M:DET	8	7	1	53.33	88.89	57.97
M:NOUN	0	0	3	100.00	0.00	0.00
M:NOUN:POSS	4	2	4	66.67	50.00	62.50
M:PREP	1	2	1	33.33	50.00	35.71
M:PRON	1	1	2	50.00	33.33	45.45
M:PUNCT	174	17	24	91.10	87.88	90.44
M:VERB	2	1	2	66.67	50.00	62.50
M:VERB:FORM	0	1	0	0.00	100.00	0.00
R:ADJ	1	0	4	100.00	20.00	55.56
R:ADV	0	0	1	100.00	0.00	0.00
R:CONTR	0	0	1	100.00	0.00	0.00
R:DET	1	3	1	25.00	50.00	27.78
R:MORPH	2	3	5	40.00	28.57	37.04
R:NOUN	4	1	6	80.00	40.00	66.67
R:NOUN:INFL	2	0	9	100.00	18.18	52.63
R:NOUN:NUM	11	5	6	68.75	64.71	67.90
R:NOUN:POSS	4	0	7	100.00	36.36	74.07
R:ORTH	29	4	35	87.88	45.31	73.98
R:OTHER	13	13	36	50.00	26.53	42.48
R:PREP	9	15	7	37.50	56.25	40.18
R:PRON	3	1	0	75.00	100.00	78.95
R:PUNCT	13	10	6	56.52	68.42	58.56
R:SPELL	75	27	20	73.53	78.95	74.55
R:VERB	4	1	7	80.00	36.36	64.52
R:VERB:FORM	8	7	2	53.33	80.00	57.14
R:VERB:INFL	3	0	1	100.00	75.00	93.75
R:VERB:SVA	9	5	1	64.29	90.00	68.18
R:VERB:TENSE	3	3	2	50.00	60.00	51.72
R:WO	4	3	0	57.14	100.00	62.50
U:DET	3	5	3	37.50	50.00	39.47
U:OTHER	0	0	1	100.00	0.00	0.00
U:PART	1	0	0	100.00	100.00	100.00
U:PREP	2	3	5	40.00	28.57	37.04
U:PUNCT	6	0	9	100.00	40.00	76.92
U:VERB	0	0	2	100.00	0.00	0.00
U:VERB:FORM	0	1	1	0.00	0.00	0.00
U:VERB:TENSE	0	0	1	100.00	0.00	0.00

SPAN-LEVEL DETECTION
OPERATION LEVEL
	TP	FP	FN	P	R	F_0.5
M	190	31	38	85.97	83.33	85.43
R	236	63	131	78.93	64.31	75.50
U	11	9	20	55.00	35.48	49.55
UNK	1	0	3	100.00	25.00	62.50

ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
ADJ	2	0	3	100.00	40.00	76.92
ADV	0	0	1	100.00	0.00	0.00
CONTR	1	0	0	100.00	100.00	100.00
DET	13	15	6	46.43	68.42	49.62
MORPH	2	2	4	50.00	33.33	45.45
NOUN	4	0	9	100.00	30.77	68.97
NOUN:INFL	3	0	8	100.00	27.27	65.22
NOUN:NUM	11	5	6	68.75	64.71	67.90
NOUN:POSS	9	2	10	81.82	47.37	71.43
ORTH	31	4	35	88.57	46.97	75.24
OTHER	23	5	31	82.14	42.59	69.28
PART	1	0	0	100.00	100.00	100.00
PREP	13	18	10	41.94	56.52	44.22
PRON	4	2	2	66.67	66.67	66.67
PUNCT	193	27	39	87.73	83.19	86.78
SPELL	90	4	9	95.74	90.91	94.74
UNK	1	0	3	100.00	25.00	62.50
VERB	7	2	10	77.78	41.18	66.04
VERB:FORM	8	8	2	50.00	80.00	54.05
VERB:INFL	4	0	0	100.00	100.00	100.00
VERB:SVA	10	5	1	66.67	90.91	70.42
VERB:TENSE	4	2	3	66.67	57.14	64.52
WO	4	2	0	66.67	100.00	71.43

OPERATION + ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
M:DET	8	7	1	53.33	88.89	57.97
M:NOUN	0	0	3	100.00	0.00	0.00
M:NOUN:POSS	4	2	4	66.67	50.00	62.50
M:OTHER	1	0	1	100.00	50.00	83.33
M:PREP	1	2	1	33.33	50.00	35.71
M:PRON	1	1	2	50.00	33.33	45.45
M:PUNCT	173	17	24	91.05	87.82	90.39
M:VERB	2	1	2	66.67	50.00	62.50
M:VERB:FORM	0	1	0	0.00	100.00	0.00
R:ADJ	2	0	3	100.00	40.00	76.92
R:ADV	0	0	1	100.00	0.00	0.00
R:CONTR	1	0	0	100.00	100.00	100.00
R:DET	1	3	1	25.00	50.00	27.78
R:MORPH	2	2	4	50.00	33.33	45.45
R:NOUN	4	0	6	100.00	40.00	76.92
R:NOUN:INFL	3	0	8	100.00	27.27	65.22
R:NOUN:NUM	11	5	6	68.75	64.71	67.90
R:NOUN:POSS	5	0	6	100.00	45.45	80.65
R:ORTH	31	4	35	88.57	46.97	75.24
R:OTHER	22	5	29	81.48	43.14	69.18
R:PREP	12	13	6	48.00	66.67	50.85
R:PRON	3	1	0	75.00	100.00	78.95
R:PUNCT	14	10	6	58.33	70.00	60.34
R:SPELL	90	4	9	95.74	90.91	94.74
R:VERB	5	1	6	83.33	45.45	71.43
R:VERB:FORM	8	6	2	57.14	80.00	60.61
R:VERB:INFL	4	0	0	100.00	100.00	100.00
R:VERB:SVA	10	5	1	66.67	90.91	70.42
R:VERB:TENSE	4	2	2	66.67	66.67	66.67
R:WO	4	2	0	66.67	100.00	71.43
U:DET	4	5	4	44.44	50.00	45.45
U:OTHER	0	0	1	100.00	0.00	0.00
U:PART	1	0	0	100.00	100.00	100.00
U:PREP	0	3	3	0.00	0.00	0.00
U:PUNCT	6	0	9	100.00	40.00	76.92
U:VERB	0	0	2	100.00	0.00	0.00
U:VERB:FORM	0	1	0	0.00	100.00	0.00
U:VERB:TENSE	0	0	1	100.00	0.00	0.00
UNK	1	0	3	100.00	25.00	62.50

TOKEN-LEVEL DETECTION
OPERATION LEVEL
	TP	FP	FN	P	R	F_0.5
M	203	25	47	89.04	81.20	87.35
R	283	50	155	84.98	64.61	79.94
U	15	6	29	71.43	34.09	58.59
UNK	2	0	1	100.00	66.67	90.91

ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
ADJ	2	0	3	100.00	40.00	76.92
ADV	1	0	2	100.00	33.33	71.43
CONJ	1	0	0	100.00	100.00	100.00
CONTR	1	0	1	100.00	50.00	83.33
DET	13	10	9	56.52	59.09	57.02
MORPH	2	1	4	66.67	33.33	55.56
NOUN	5	0	10	100.00	33.33	71.43
NOUN:INFL	3	0	8	100.00	27.27	65.22
NOUN:NUM	12	4	8	75.00	60.00	71.43
NOUN:POSS	10	2	10	83.33	50.00	73.53
ORTH	51	4	47	92.73	52.04	80.19
OTHER	41	5	37	89.13	52.56	78.24
PART	1	0	0	100.00	100.00	100.00
PREP	13	15	16	46.43	44.83	46.10
PRON	6	2	1	75.00	85.71	76.92
PUNCT	207	23	51	90.00	80.23	87.86
SPELL	90	1	11	98.90	89.11	96.77
UNK	2	0	1	100.00	66.67	90.91
VERB	8	1	7	88.89	53.33	78.43
VERB:FORM	8	6	2	57.14	80.00	60.61
VERB:INFL	4	0	0	100.00	100.00	100.00
VERB:SVA	10	4	1	71.43	90.91	74.63
VERB:TENSE	6	3	3	66.67	66.67	66.67
WO	6	0	0	100.00	100.00	100.00

OPERATION + ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
M:ADV	0	0	1	100.00	0.00	0.00
M:CONJ	1	0	0	100.00	100.00	100.00
M:DET	8	2	2	80.00	80.00	80.00
M:NOUN	1	0	3	100.00	25.00	62.50
M:NOUN:POSS	4	2	3	66.67	57.14	64.52
M:OTHER	2	0	1	100.00	66.67	90.91
M:PREP	2	2	3	50.00	40.00	47.62
M:PRON	2	1	1	66.67	66.67	66.67
M:PUNCT	179	16	33	91.79	84.43	90.22
M:VERB	3	1	0	75.00	100.00	78.95
M:VERB:FORM	0	1	0	0.00	100.00	0.00
M:VERB:TENSE	1	0	0	100.00	100.00	100.00
R:ADJ	2	0	2	100.00	50.00	83.33
R:ADV	0	0	1	100.00	0.00	0.00
R:CONTR	1	0	1	100.00	50.00	83.33
R:DET	1	3	1	25.00	50.00	27.78
R:MORPH	2	1	4	66.67	33.33	55.56
R:NOUN	4	0	7	100.00	36.36	74.07
R:NOUN:INFL	3	0	8	100.00	27.27	65.22
R:NOUN:NUM	12	4	8	75.00	60.00	71.43
R:NOUN:POSS	6	0	7	100.00	46.15	81.08
R:ORTH	51	4	47	92.73	52.04	80.19
R:OTHER	39	5	34	88.64	53.42	78.31
R:PREP	10	12	9	45.45	52.63	46.73
R:PRON	4	1	0	80.00	100.00	83.33
R:PUNCT	21	7	5	75.00	80.77	76.09
R:SPELL	90	1	11	98.90	89.11	96.77
R:VERB	5	0	5	100.00	50.00	83.33
R:VERB:FORM	8	5	2	61.54	80.00	64.52
R:VERB:INFL	4	0	0	100.00	100.00	100.00
R:VERB:SVA	10	4	1	71.43	90.91	74.63
R:VERB:TENSE	4	3	2	57.14	66.67	58.82
R:WO	6	0	0	100.00	100.00	100.00
U:ADJ	0	0	1	100.00	0.00	0.00
U:ADV	1	0	0	100.00	100.00	100.00
U:DET	4	5	6	44.44	40.00	43.48
U:OTHER	0	0	2	100.00	0.00	0.00
U:PART	1	0	0	100.00	100.00	100.00
U:PREP	1	1	4	50.00	20.00	38.46
U:PUNCT	7	0	13	100.00	35.00	72.92
U:VERB	0	0	2	100.00	0.00	0.00
U:VERB:TENSE	1	0	1	100.00	50.00	83.33
UNK	2	0	1	100.00	66.67	90.91

Submitted by:	cjbryant
Submitted at:	2019-08-07T14:43:46+00:00
Competition submission:	10
Competition phase:	3 - Open phase

Detailed results

Submission information

Results