BEA 2019 Shared Task: Grammatical Error Correction

OVERALL
	TP	FP	FN	P	R	F_0.5
Span-level correction	604	350	3311	63.31	15.43	39.06
Span-level detection	705	250	3355	73.82	17.36	44.73
Token-level detection	843	314	3980	72.86	17.48	44.60

SPAN-LEVEL CORRECTION
OPERATION LEVEL
	TP	FP	FN	P	R	F_0.5
M	58	43	728	57.43	7.38	24.37
R	485	270	2268	64.24	17.62	42.01
U	61	37	315	62.24	16.22	39.71

ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
ADJ	5	5	23	50.00	17.86	36.76
ADJ:FORM	2	0	7	100.00	22.22	58.82
ADV	4	6	32	40.00	11.11	26.32
CONJ	0	0	20	100.00	0.00	0.00
CONTR	2	1	6	66.67	25.00	50.00
DET	58	22	403	72.50	12.58	37.13
MORPH	23	4	102	85.19	18.40	49.36
NOUN	6	12	83	33.33	6.74	18.63
NOUN:INFL	4	0	15	100.00	21.05	57.14
NOUN:NUM	51	16	163	76.12	23.83	52.90
NOUN:POSS	2	1	40	66.67	4.76	18.52
ORTH	178	146	188	54.94	48.63	53.55
OTHER	14	42	674	25.00	2.03	7.68
PART	2	0	11	100.00	15.38	47.62
PREP	20	18	304	52.63	6.17	21.01
PRON	9	1	96	90.00	8.57	31.03
PUNCT	35	20	325	63.64	9.72	30.17
SPELL	66	24	234	73.33	22.00	50.00
VERB	5	8	161	38.46	3.01	11.47
VERB:FORM	55	12	98	82.09	35.95	65.32
VERB:INFL	1	0	8	100.00	11.11	38.46
VERB:SVA	41	6	85	87.23	32.54	65.29
VERB:TENSE	19	4	168	82.61	10.16	34.05
WO	2	2	65	50.00	2.99	12.05

OPERATION + ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
M:ADJ	0	1	3	0.00	0.00	0.00
M:ADV	0	0	11	100.00	0.00	0.00
M:CONJ	0	0	11	100.00	0.00	0.00
M:CONTR	0	1	0	0.00	100.00	0.00
M:DET	21	15	198	58.33	9.59	28.93
M:NOUN	0	2	11	0.00	0.00	0.00
M:NOUN:POSS	0	1	11	0.00	0.00	0.00
M:OTHER	1	6	40	14.29	2.44	7.25
M:PART	0	0	2	100.00	0.00	0.00
M:PREP	2	5	77	28.57	2.53	9.35
M:PRON	4	0	51	100.00	7.27	28.17
M:PUNCT	25	11	243	69.44	9.33	30.34
M:VERB	2	0	28	100.00	6.67	26.32
M:VERB:FORM	3	1	9	75.00	25.00	53.57
M:VERB:TENSE	0	0	33	100.00	0.00	0.00
R:ADJ	4	4	17	50.00	19.05	37.74
R:ADJ:FORM	2	0	7	100.00	22.22	58.82
R:ADV	2	0	11	100.00	15.38	47.62
R:CONJ	0	0	6	100.00	0.00	0.00
R:CONTR	1	0	2	100.00	33.33	71.43
R:DET	8	2	77	80.00	9.41	32.00
R:MORPH	23	4	102	85.19	18.40	49.36
R:NOUN	4	8	62	33.33	6.06	17.54
R:NOUN:INFL	4	0	15	100.00	21.05	57.14
R:NOUN:NUM	51	16	163	76.12	23.83	52.90
R:NOUN:POSS	1	0	22	100.00	4.35	18.52
R:ORTH	178	146	188	54.94	48.63	53.55
R:OTHER	10	23	618	30.30	1.59	6.58
R:PART	1	0	5	100.00	16.67	50.00
R:PREP	13	6	171	68.42	7.07	25.00
R:PRON	2	1	31	66.67	6.06	22.22
R:PUNCT	6	8	54	42.86	10.00	25.86
R:SPELL	66	24	234	73.33	22.00	50.00
R:VERB	2	6	116	25.00	1.69	6.67
R:VERB:FORM	50	11	86	81.97	36.76	65.79
R:VERB:INFL	1	0	8	100.00	11.11	38.46
R:VERB:SVA	41	6	85	87.23	32.54	65.29
R:VERB:TENSE	13	3	123	81.25	9.56	32.50
R:WO	2	2	65	50.00	2.99	12.05
U:ADJ	1	0	3	100.00	25.00	62.50
U:ADV	2	6	10	25.00	16.67	22.73
U:CONJ	0	0	3	100.00	0.00	0.00
U:CONTR	1	0	4	100.00	20.00	55.56
U:DET	29	5	128	85.29	18.47	49.49
U:NOUN	2	2	10	50.00	16.67	35.71
U:NOUN:POSS	1	0	7	100.00	12.50	41.67
U:OTHER	3	13	16	18.75	15.79	18.07
U:PART	1	0	4	100.00	20.00	55.56
U:PREP	5	7	56	41.67	8.20	22.94
U:PRON	3	0	14	100.00	17.65	51.72
U:PUNCT	4	1	28	80.00	12.50	38.46
U:VERB	1	2	17	33.33	5.56	16.67
U:VERB:FORM	2	0	3	100.00	40.00	76.92
U:VERB:TENSE	6	1	12	85.71	33.33	65.22

SPAN-LEVEL DETECTION
OPERATION LEVEL
	TP	FP	FN	P	R	F_0.5
M	67	33	728	67.00	8.43	28.03
R	564	185	2242	75.30	20.10	48.60
U	68	32	311	68.00	17.94	43.65
UNK	6	0	74	100.00	7.50	28.85

ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
ADJ	8	2	26	80.00	23.53	54.05
ADJ:FORM	1	0	7	100.00	12.50	41.67
ADV	6	4	33	60.00	15.38	37.97
CONJ	1	0	17	100.00	5.56	22.73
CONTR	2	1	6	66.67	25.00	50.00
DET	66	16	397	80.49	14.25	41.72
MORPH	25	4	101	86.21	19.84	51.65
NOUN	8	12	85	40.00	8.60	23.12
NOUN:INFL	5	0	15	100.00	25.00	62.50
NOUN:NUM	48	12	159	80.00	23.19	53.69
NOUN:POSS	2	1	40	66.67	4.76	18.52
ORTH	189	120	183	61.17	50.81	58.77
OTHER	40	29	677	57.97	5.58	20.14
PART	3	0	10	100.00	23.08	60.00
PREP	25	14	310	64.10	7.46	25.46
PRON	10	1	94	90.91	9.62	33.78
PUNCT	36	17	318	67.92	10.17	31.80
SPELL	84	5	216	94.38	28.00	64.02
UNK	6	0	74	100.00	7.50	28.85
VERB	14	3	163	82.35	7.91	28.57
VERB:FORM	57	3	98	95.00	36.77	72.15
VERB:INFL	2	0	7	100.00	22.22	58.82
VERB:SVA	43	2	86	95.56	33.33	69.58
VERB:TENSE	22	2	167	91.67	11.64	38.60
WO	2	2	66	50.00	2.94	11.90

OPERATION + ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
M:ADJ	0	1	3	0.00	0.00	0.00
M:ADV	0	0	12	100.00	0.00	0.00
M:CONJ	1	0	10	100.00	9.09	33.33
M:CONTR	0	1	0	0.00	100.00	0.00
M:DET	24	10	194	70.59	11.01	33.90
M:NOUN	0	2	12	0.00	0.00	0.00
M:NOUN:POSS	0	1	11	0.00	0.00	0.00
M:OTHER	4	4	41	50.00	8.89	25.97
M:PART	0	0	3	100.00	0.00	0.00
M:PREP	2	5	81	28.57	2.41	9.01
M:PRON	5	0	50	100.00	9.09	33.33
M:PUNCT	26	8	242	76.47	9.70	32.18
M:VERB	2	0	28	100.00	6.67	26.32
M:VERB:FORM	3	1	8	75.00	27.27	55.56
M:VERB:TENSE	0	0	33	100.00	0.00	0.00
R:ADJ	7	1	18	87.50	28.00	61.40
R:ADJ:FORM	1	0	7	100.00	12.50	41.67
R:ADV	3	0	12	100.00	20.00	55.56
R:CONJ	0	0	5	100.00	0.00	0.00
R:CONTR	1	0	2	100.00	33.33	71.43
R:DET	9	0	76	100.00	10.59	37.19
R:MORPH	25	4	101	86.21	19.84	51.65
R:NOUN	5	8	62	38.46	7.46	21.01
R:NOUN:INFL	5	0	15	100.00	25.00	62.50
R:NOUN:NUM	48	12	159	80.00	23.19	53.69
R:NOUN:POSS	1	0	22	100.00	4.35	18.52
R:ORTH	189	120	183	61.17	50.81	58.77
R:OTHER	33	12	620	73.33	5.05	19.81
R:PART	2	0	4	100.00	33.33	71.43
R:PREP	18	4	174	81.82	9.38	32.14
R:PRON	2	1	31	66.67	6.06	22.22
R:PUNCT	6	8	50	42.86	10.71	26.79
R:SPELL	84	5	216	94.38	28.00	64.02
R:VERB	11	2	118	84.62	8.53	30.39
R:VERB:FORM	52	2	87	96.30	37.41	73.24
R:VERB:INFL	2	0	7	100.00	22.22	58.82
R:VERB:SVA	43	2	86	95.56	33.33	69.58
R:VERB:TENSE	15	2	121	88.24	11.03	36.76
R:WO	2	2	66	50.00	2.94	11.90
U:ADJ	1	0	5	100.00	16.67	50.00
U:ADV	3	4	9	42.86	25.00	37.50
U:CONJ	0	0	2	100.00	0.00	0.00
U:CONTR	1	0	4	100.00	20.00	55.56
U:DET	33	6	127	84.62	20.62	52.22
U:NOUN	3	2	11	60.00	21.43	44.12
U:NOUN:POSS	1	0	7	100.00	12.50	41.67
U:OTHER	3	13	16	18.75	15.79	18.07
U:PART	1	0	3	100.00	25.00	62.50
U:PREP	5	5	55	50.00	8.33	25.00
U:PRON	3	0	13	100.00	18.75	53.57
U:PUNCT	4	1	26	80.00	13.33	40.00
U:VERB	1	1	17	50.00	5.56	19.23
U:VERB:FORM	2	0	3	100.00	40.00	76.92
U:VERB:TENSE	7	0	13	100.00	35.00	72.92
UNK	6	0	74	100.00	7.50	28.85

TOKEN-LEVEL DETECTION
OPERATION LEVEL
	TP	FP	FN	P	R	F_0.5
M	109	23	872	82.58	11.11	36.12
R	644	167	2585	79.41	19.94	49.75
U	82	124	449	39.81	15.44	30.26
UNK	8	0	74	100.00	9.76	35.09

ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
ADJ	7	1	37	87.50	15.91	46.05
ADJ:FORM	1	0	10	100.00	9.09	33.33
ADV	9	4	55	69.23	14.06	38.79
CONJ	1	0	19	100.00	5.00	20.83
CONTR	2	1	11	66.67	15.38	40.00
DET	75	10	458	88.24	14.07	42.96
MORPH	24	3	108	88.89	18.18	50.00
NOUN	17	8	104	68.00	14.05	38.46
NOUN:INFL	4	0	15	100.00	21.05	57.14
NOUN:NUM	47	8	170	85.45	21.66	53.78
NOUN:POSS	4	1	53	80.00	7.02	25.97
ORTH	204	110	216	64.97	48.57	60.86
OTHER	82	126	837	39.42	8.92	23.42
PART	4	0	12	100.00	25.00	62.50
PREP	27	10	336	72.97	7.44	26.42
PRON	14	0	116	100.00	10.77	37.63
PUNCT	62	17	387	78.48	13.81	40.52
SPELL	85	5	226	94.44	27.33	63.34
UNK	8	0	74	100.00	9.76	35.09
VERB	20	3	220	86.96	8.33	30.12
VERB:FORM	57	1	115	98.28	33.14	70.54
VERB:INFL	2	0	6	100.00	25.00	62.50
VERB:SVA	44	2	92	95.65	32.35	68.75
VERB:TENSE	30	2	215	93.75	12.24	40.21
WO	13	2	88	86.67	12.87	40.37

OPERATION + ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
M:ADJ	0	0	5	100.00	0.00	0.00
M:ADV	2	0	18	100.00	10.00	35.71
M:CONJ	1	0	13	100.00	7.14	27.78
M:CONTR	0	1	0	0.00	100.00	0.00
M:DET	29	5	218	85.29	11.74	37.86
M:NOUN	1	2	15	33.33	6.25	17.86
M:NOUN:POSS	1	1	12	50.00	7.69	23.81
M:OTHER	5	4	57	55.56	8.06	25.51
M:PART	0	0	4	100.00	0.00	0.00
M:PREP	6	3	85	66.67	6.59	23.62
M:PRON	8	0	63	100.00	11.27	38.83
M:PUNCT	48	7	288	87.27	14.29	43.17
M:VERB	2	0	45	100.00	4.26	18.18
M:VERB:FORM	4	0	11	100.00	26.67	64.52
M:VERB:TENSE	2	0	38	100.00	5.00	20.83
R:ADJ	6	1	24	85.71	20.00	51.72
R:ADJ:FORM	1	0	10	100.00	9.09	33.33
R:ADV	4	0	17	100.00	19.05	54.05
R:CONJ	0	0	5	100.00	0.00	0.00
R:CONTR	1	0	5	100.00	16.67	50.00
R:DET	10	0	86	100.00	10.42	36.76
R:MORPH	24	3	108	88.89	18.18	50.00
R:NOUN	12	5	72	70.59	14.29	39.47
R:NOUN:INFL	4	0	15	100.00	21.05	57.14
R:NOUN:NUM	47	8	170	85.45	21.66	53.78
R:NOUN:POSS	2	0	29	100.00	6.45	25.64
R:ORTH	204	110	216	64.97	48.57	60.86
R:OTHER	68	15	716	81.93	8.67	30.47
R:PART	3	0	5	100.00	37.50	75.00
R:PREP	16	2	188	88.89	7.84	28.99
R:PRON	2	0	36	100.00	5.26	21.74
R:PUNCT	8	9	66	47.06	10.81	28.17
R:SPELL	85	5	226	94.44	27.33	63.34
R:VERB	16	2	145	88.89	9.94	34.33
R:VERB:FORM	51	1	101	98.08	33.55	70.83
R:VERB:INFL	2	0	6	100.00	25.00	62.50
R:VERB:SVA	44	2	92	95.65	32.35	68.75
R:VERB:TENSE	21	2	159	91.30	11.67	38.60
R:WO	13	2	88	86.67	12.87	40.37
U:ADJ	1	0	8	100.00	11.11	38.46
U:ADV	3	4	20	42.86	13.04	29.41
U:CONJ	0	0	1	100.00	0.00	0.00
U:CONTR	1	0	6	100.00	14.29	45.45
U:DET	36	5	154	87.80	18.95	50.85
U:NOUN	4	1	17	80.00	19.05	48.78
U:NOUN:POSS	1	0	12	100.00	7.69	29.41
U:OTHER	9	107	64	7.76	12.33	8.38
U:PART	1	0	3	100.00	25.00	62.50
U:PREP	5	5	63	50.00	7.35	23.15
U:PRON	4	0	17	100.00	19.05	54.05
U:PUNCT	6	1	33	85.71	15.38	44.78
U:VERB	2	1	30	66.67	6.25	22.73
U:VERB:FORM	2	0	3	100.00	40.00	76.92
U:VERB:TENSE	7	0	18	100.00	28.00	66.04
UNK	8	0	74	100.00	9.76	35.09

OVERALL
	TP	FP	FN	P	R	F_0.5
Span-level correction	322	127	1526	71.71	17.42	44.18
Span-level detection	373	77	1550	82.89	19.40	50.09
Token-level detection	441	56	1861	88.73	19.16	51.40

SPAN-LEVEL CORRECTION
OPERATION LEVEL
	TP	FP	FN	P	R	F_0.5
M	30	25	330	54.55	8.33	25.86
R	258	90	1052	74.14	19.69	47.74
U	34	12	144	73.91	19.10	46.96

ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
ADJ	2	3	8	40.00	20.00	33.33
ADJ:FORM	1	0	3	100.00	25.00	62.50
ADV	3	1	16	75.00	15.79	42.86
CONJ	0	0	9	100.00	0.00	0.00
CONTR	2	1	1	66.67	66.67	66.67
DET	30	9	182	76.92	14.15	40.76
MORPH	11	1	58	91.67	15.94	47.01
NOUN	1	6	36	14.29	2.70	7.69
NOUN:INFL	2	0	4	100.00	33.33	71.43
NOUN:NUM	27	4	86	87.10	23.89	56.96
NOUN:POSS	1	0	8	100.00	11.11	38.46
ORTH	104	41	85	71.72	55.03	67.62
OTHER	10	20	357	33.33	2.72	10.27
PART	1	0	4	100.00	20.00	55.56
PREP	7	7	130	50.00	5.11	18.13
PRON	6	1	52	85.71	10.34	34.88
PUNCT	18	9	141	66.67	11.32	33.71
SPELL	30	8	76	78.95	28.30	58.14
VERB	2	4	80	33.33	2.44	9.43
VERB:FORM	29	7	48	80.56	37.66	65.61
VERB:INFL	1	0	3	100.00	25.00	62.50
VERB:SVA	25	2	44	92.59	36.23	70.62
VERB:TENSE	8	2	77	80.00	9.41	32.00
WO	1	1	18	50.00	5.26	18.52

OPERATION + ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
M:ADJ	0	1	1	0.00	0.00	0.00
M:ADV	0	0	5	100.00	0.00	0.00
M:CONJ	0	0	4	100.00	0.00	0.00
M:CONTR	0	1	0	0.00	100.00	0.00
M:DET	7	7	83	50.00	7.78	23.97
M:NOUN	0	1	3	0.00	0.00	0.00
M:NOUN:POSS	0	0	3	100.00	0.00	0.00
M:OTHER	1	4	22	20.00	4.35	11.63
M:PREP	1	5	34	16.67	2.86	8.47
M:PRON	3	0	26	100.00	10.34	36.59
M:PUNCT	14	5	111	73.68	11.20	34.83
M:VERB	1	0	15	100.00	6.25	25.00
M:VERB:FORM	3	1	6	75.00	33.33	60.00
M:VERB:TENSE	0	0	17	100.00	0.00	0.00
R:ADJ	1	2	5	33.33	16.67	27.78
R:ADJ:FORM	1	0	3	100.00	25.00	62.50
R:ADV	1	0	7	100.00	12.50	41.67
R:CONJ	0	0	2	100.00	0.00	0.00
R:CONTR	1	0	0	100.00	100.00	100.00
R:DET	3	1	33	75.00	8.33	28.85
R:MORPH	11	1	58	91.67	15.94	47.01
R:NOUN	0	3	26	0.00	0.00	0.00
R:NOUN:INFL	2	0	4	100.00	33.33	71.43
R:NOUN:NUM	27	4	86	87.10	23.89	56.96
R:NOUN:POSS	0	0	4	100.00	0.00	0.00
R:ORTH	104	41	85	71.72	55.03	67.62
R:OTHER	7	12	329	36.84	2.08	8.50
R:PART	1	0	2	100.00	33.33	71.43
R:PREP	5	1	72	83.33	6.49	24.75
R:PRON	1	1	19	50.00	5.00	17.86
R:PUNCT	3	4	19	42.86	13.64	30.00
R:SPELL	30	8	76	78.95	28.30	58.14
R:VERB	1	2	58	33.33	1.69	7.04
R:VERB:FORM	26	6	41	81.25	38.81	66.67
R:VERB:INFL	1	0	3	100.00	25.00	62.50
R:VERB:SVA	25	2	44	92.59	36.23	70.62
R:VERB:TENSE	6	1	58	85.71	9.38	32.61
R:WO	1	1	18	50.00	5.26	18.52
U:ADJ	1	0	2	100.00	33.33	71.43
U:ADV	2	1	4	66.67	33.33	55.56
U:CONJ	0	0	3	100.00	0.00	0.00
U:CONTR	1	0	1	100.00	50.00	83.33
U:DET	20	1	66	95.24	23.26	58.82
U:NOUN	1	2	7	33.33	12.50	25.00
U:NOUN:POSS	1	0	1	100.00	50.00	83.33
U:OTHER	2	4	6	33.33	25.00	31.25
U:PART	0	0	2	100.00	0.00	0.00
U:PREP	1	1	24	50.00	4.00	15.15
U:PRON	2	0	7	100.00	22.22	58.82
U:PUNCT	1	0	11	100.00	8.33	31.25
U:VERB	0	2	7	0.00	0.00	0.00
U:VERB:FORM	0	0	1	100.00	0.00	0.00
U:VERB:TENSE	2	1	2	66.67	50.00	62.50

SPAN-LEVEL DETECTION
OPERATION LEVEL
	TP	FP	FN	P	R	F_0.5
M	36	19	324	65.45	10.00	31.03
R	297	48	1033	86.09	22.33	54.80
U	37	10	138	78.72	21.14	50.96
UNK	3	0	55	100.00	5.17	21.43

ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
ADJ	3	1	10	75.00	23.08	51.72
ADJ:FORM	0	0	3	100.00	0.00	0.00
ADV	3	1	18	75.00	14.29	40.54
CONJ	1	0	6	100.00	14.29	45.45
CONTR	2	1	1	66.67	66.67	66.67
DET	34	7	172	82.93	16.50	45.95
MORPH	13	1	59	92.86	18.06	50.78
NOUN	2	6	36	25.00	5.26	14.29
NOUN:INFL	2	0	4	100.00	33.33	71.43
NOUN:NUM	23	1	81	95.83	22.12	57.50
NOUN:POSS	1	0	8	100.00	11.11	38.46
ORTH	109	29	82	78.99	57.07	73.35
OTHER	29	11	353	72.50	7.59	26.75
PART	1	0	4	100.00	20.00	55.56
PREP	10	6	133	62.50	6.99	24.15
PRON	7	1	52	87.50	11.86	38.46
PUNCT	19	7	137	73.08	12.18	36.54
SPELL	39	1	67	97.50	36.79	73.31
UNK	3	0	55	100.00	5.17	21.43
VERB	6	1	80	85.71	6.98	26.32
VERB:FORM	29	2	48	93.55	37.66	72.14
VERB:INFL	1	0	3	100.00	25.00	62.50
VERB:SVA	25	0	44	100.00	36.23	73.96
VERB:TENSE	10	0	75	100.00	11.76	40.00
WO	1	1	19	50.00	5.00	17.86

OPERATION + ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
M:ADJ	0	1	1	0.00	0.00	0.00
M:ADV	0	0	6	100.00	0.00	0.00
M:CONJ	1	0	3	100.00	25.00	62.50
M:CONTR	0	1	0	0.00	100.00	0.00
M:DET	9	5	79	64.29	10.23	31.25
M:NOUN	0	1	3	0.00	0.00	0.00
M:NOUN:POSS	0	0	3	100.00	0.00	0.00
M:OTHER	2	2	21	50.00	8.70	25.64
M:PREP	1	5	36	16.67	2.70	8.20
M:PRON	4	0	26	100.00	13.33	43.48
M:PUNCT	15	3	111	83.33	11.90	37.88
M:VERB	1	0	14	100.00	6.67	26.32
M:VERB:FORM	3	1	5	75.00	37.50	62.50
M:VERB:TENSE	0	0	16	100.00	0.00	0.00
R:ADJ	2	0	6	100.00	25.00	62.50
R:ADJ:FORM	0	0	3	100.00	0.00	0.00
R:ADV	1	0	8	100.00	11.11	38.46
R:CONJ	0	0	1	100.00	0.00	0.00
R:CONTR	1	0	0	100.00	100.00	100.00
R:DET	4	0	32	100.00	11.11	38.46
R:MORPH	13	1	59	92.86	18.06	50.78
R:NOUN	0	3	25	0.00	0.00	0.00
R:NOUN:INFL	2	0	4	100.00	33.33	71.43
R:NOUN:NUM	23	1	81	95.83	22.12	57.50
R:NOUN:POSS	0	0	4	100.00	0.00	0.00
R:ORTH	109	29	82	78.99	57.07	73.35
R:OTHER	25	5	326	83.33	7.12	26.54
R:PART	1	0	2	100.00	33.33	71.43
R:PREP	8	1	74	88.89	9.76	33.90
R:PRON	1	1	19	50.00	5.00	17.86
R:PUNCT	3	4	17	42.86	15.00	31.25
R:SPELL	39	1	67	97.50	36.79	73.31
R:VERB	5	0	58	100.00	7.94	30.12
R:VERB:FORM	26	1	42	96.30	38.24	73.86
R:VERB:INFL	1	0	3	100.00	25.00	62.50
R:VERB:SVA	25	0	44	100.00	36.23	73.96
R:VERB:TENSE	7	0	57	100.00	10.94	38.04
R:WO	1	1	19	50.00	5.00	17.86
U:ADJ	1	0	3	100.00	25.00	62.50
U:ADV	2	1	4	66.67	33.33	55.56
U:CONJ	0	0	2	100.00	0.00	0.00
U:CONTR	1	0	1	100.00	50.00	83.33
U:DET	21	2	61	91.30	25.61	60.34
U:NOUN	2	2	8	50.00	20.00	38.46
U:NOUN:POSS	1	0	1	100.00	50.00	83.33
U:OTHER	2	4	6	33.33	25.00	31.25
U:PART	0	0	2	100.00	0.00	0.00
U:PREP	1	0	23	100.00	4.17	17.86
U:PRON	2	0	7	100.00	22.22	58.82
U:PUNCT	1	0	9	100.00	10.00	35.71
U:VERB	0	1	8	0.00	0.00	0.00
U:VERB:FORM	0	0	1	100.00	0.00	0.00
U:VERB:TENSE	3	0	2	100.00	60.00	88.24
UNK	3	0	55	100.00	5.17	21.43

TOKEN-LEVEL DETECTION
OPERATION LEVEL
	TP	FP	FN	P	R	F_0.5
M	59	11	402	84.29	12.80	39.81
R	331	38	1200	89.70	21.62	55.04
U	46	7	200	86.79	18.70	50.22
UNK	5	0	59	100.00	7.81	29.76

ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
ADJ	3	0	13	100.00	18.75	53.57
ADJ:FORM	0	0	5	100.00	0.00	0.00
ADV	4	0	27	100.00	12.90	42.55
CONJ	1	0	7	100.00	12.50	41.67
CONTR	2	1	2	66.67	50.00	62.50
DET	42	3	211	93.33	16.60	48.50
MORPH	12	1	61	92.31	16.44	48.00
NOUN	8	3	50	72.73	13.79	39.22
NOUN:INFL	2	0	4	100.00	33.33	71.43
NOUN:NUM	23	0	86	100.00	21.10	57.21
NOUN:POSS	2	0	14	100.00	12.50	41.67
ORTH	115	27	95	80.99	54.76	73.91
OTHER	49	10	432	83.05	10.19	34.17
PART	1	0	6	100.00	14.29	45.45
PREP	11	3	147	78.57	6.96	25.70
PRON	10	0	61	100.00	14.08	45.05
PUNCT	31	5	172	86.11	15.27	44.67
SPELL	39	1	72	97.50	35.14	71.96
UNK	5	0	59	100.00	7.81	29.76
VERB	8	1	114	88.89	6.56	25.32
VERB:FORM	29	1	57	96.67	33.72	70.39
VERB:INFL	1	0	2	100.00	33.33	71.43
VERB:SVA	26	0	47	100.00	35.62	73.45
VERB:TENSE	13	0	96	100.00	11.93	40.37
WO	4	0	21	100.00	16.00	48.78

OPERATION + ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
M:ADJ	0	0	1	100.00	0.00	0.00
M:ADV	1	0	7	100.00	12.50	41.67
M:CONJ	1	0	5	100.00	16.67	50.00
M:CONTR	0	1	0	0.00	100.00	0.00
M:DET	12	2	93	85.71	11.43	37.27
M:NOUN	1	1	5	50.00	16.67	35.71
M:NOUN:POSS	1	0	3	100.00	25.00	62.50
M:OTHER	3	2	31	60.00	8.82	27.78
M:PART	0	0	1	100.00	0.00	0.00
M:PREP	4	3	39	57.14	9.30	28.17
M:PRON	6	0	33	100.00	15.38	47.62
M:PUNCT	25	2	135	92.59	15.62	46.64
M:VERB	1	0	24	100.00	4.00	17.24
M:VERB:FORM	3	0	8	100.00	27.27	65.22
M:VERB:TENSE	1	0	17	100.00	5.56	22.73
R:ADJ	2	0	9	100.00	18.18	52.63
R:ADJ:FORM	0	0	5	100.00	0.00	0.00
R:ADV	1	0	11	100.00	8.33	31.25
R:CONJ	0	0	1	100.00	0.00	0.00
R:CONTR	1	0	0	100.00	100.00	100.00
R:DET	6	0	38	100.00	13.64	44.12
R:MORPH	12	1	61	92.31	16.44	48.00
R:NOUN	4	1	34	80.00	10.53	34.48
R:NOUN:INFL	2	0	4	100.00	33.33	71.43
R:NOUN:NUM	23	0	86	100.00	21.10	57.21
R:NOUN:POSS	0	0	7	100.00	0.00	0.00
R:ORTH	115	27	95	80.99	54.76	73.91
R:OTHER	42	4	380	91.30	9.95	34.65
R:PART	1	0	3	100.00	25.00	62.50
R:PREP	6	0	80	100.00	6.98	27.27
R:PRON	1	0	20	100.00	4.76	20.00
R:PUNCT	3	3	27	50.00	10.00	27.78
R:SPELL	39	1	72	97.50	35.14	71.96
R:VERB	7	0	73	100.00	8.75	32.41
R:VERB:FORM	26	1	48	96.30	35.14	71.43
R:VERB:INFL	1	0	2	100.00	33.33	71.43
R:VERB:SVA	26	0	47	100.00	35.62	73.45
R:VERB:TENSE	9	0	76	100.00	10.59	37.19
R:WO	4	0	21	100.00	16.00	48.78
U:ADJ	1	0	3	100.00	25.00	62.50
U:ADV	2	0	9	100.00	18.18	52.63
U:CONJ	0	0	1	100.00	0.00	0.00
U:CONTR	1	0	2	100.00	33.33	71.43
U:DET	24	1	80	96.00	23.08	58.82
U:NOUN	3	1	11	75.00	21.43	50.00
U:NOUN:POSS	1	0	4	100.00	20.00	55.56
U:OTHER	4	4	21	50.00	16.00	35.09
U:PART	0	0	2	100.00	0.00	0.00
U:PREP	1	0	28	100.00	3.45	15.15
U:PRON	3	0	8	100.00	27.27	65.22
U:PUNCT	3	0	10	100.00	23.08	60.00
U:VERB	0	1	17	0.00	0.00	0.00
U:VERB:FORM	0	0	1	100.00	0.00	0.00
U:VERB:TENSE	3	0	3	100.00	50.00	83.33
UNK	5	0	59	100.00	7.81	29.76

OVERALL
	TP	FP	FN	P	R	F_0.5
Span-level correction	162	100	1080	61.83	13.04	35.37
Span-level detection	188	74	1090	71.76	14.71	40.41
Token-level detection	243	163	1324	59.85	15.51	38.08

SPAN-LEVEL CORRECTION
OPERATION LEVEL
	TP	FP	FN	P	R	F_0.5
M	17	7	261	70.83	6.12	22.73
R	128	83	709	60.66	15.29	38.07
U	17	10	110	62.96	13.39	36.17

ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
ADJ	1	0	6	100.00	14.29	45.45
ADJ:FORM	1	0	2	100.00	33.33	71.43
ADV	1	1	13	50.00	7.14	22.73
CONJ	0	0	7	100.00	0.00	0.00
CONTR	0	0	2	100.00	0.00	0.00
DET	18	3	148	85.71	10.84	36.00
MORPH	7	1	28	87.50	20.00	52.24
NOUN	3	4	30	42.86	9.09	24.59
NOUN:NUM	15	4	45	78.95	25.00	55.15
NOUN:POSS	1	0	11	100.00	8.33	31.25
ORTH	28	46	48	37.84	36.84	37.63
OTHER	2	15	215	11.76	0.92	3.51
PART	0	0	4	100.00	0.00	0.00
PREP	10	4	115	71.43	8.00	27.62
PRON	2	0	31	100.00	6.06	24.39
PUNCT	9	6	99	60.00	8.33	26.79
SPELL	20	10	48	66.67	29.41	53.19
VERB	1	0	63	100.00	1.56	7.35
VERB:FORM	20	3	38	86.96	34.48	66.67
VERB:SVA	13	2	30	86.67	30.23	63.11
VERB:TENSE	9	1	61	90.00	12.86	40.91
WO	1	0	36	100.00	2.70	12.20

OPERATION + ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
M:ADJ	0	0	1	100.00	0.00	0.00
M:ADV	0	0	5	100.00	0.00	0.00
M:CONJ	0	0	5	100.00	0.00	0.00
M:DET	10	2	80	83.33	11.11	36.23
M:NOUN	0	1	4	0.00	0.00	0.00
M:NOUN:POSS	0	0	4	100.00	0.00	0.00
M:OTHER	0	2	13	0.00	0.00	0.00
M:PART	0	0	2	100.00	0.00	0.00
M:PREP	1	0	27	100.00	3.57	15.62
M:PRON	1	0	18	100.00	5.26	21.74
M:PUNCT	5	2	77	71.43	6.10	22.73
M:VERB	0	0	9	100.00	0.00	0.00
M:VERB:FORM	0	0	2	100.00	0.00	0.00
M:VERB:TENSE	0	0	14	100.00	0.00	0.00
R:ADJ	1	0	5	100.00	16.67	50.00
R:ADJ:FORM	1	0	2	100.00	33.33	71.43
R:ADV	1	0	3	100.00	25.00	62.50
R:CONJ	0	0	2	100.00	0.00	0.00
R:CONTR	0	0	1	100.00	0.00	0.00
R:DET	3	0	34	100.00	8.11	30.61
R:MORPH	7	1	28	87.50	20.00	52.24
R:NOUN	2	3	23	40.00	8.00	22.22
R:NOUN:NUM	15	4	45	78.95	25.00	55.15
R:NOUN:POSS	1	0	2	100.00	33.33	71.43
R:ORTH	28	46	48	37.84	36.84	37.63
R:OTHER	1	6	193	14.29	0.52	2.25
R:PART	0	0	1	100.00	0.00	0.00
R:PREP	5	3	64	62.50	7.25	24.75
R:PRON	0	0	7	100.00	0.00	0.00
R:PUNCT	3	4	15	42.86	16.67	32.61
R:SPELL	20	10	48	66.67	29.41	53.19
R:VERB	0	0	46	100.00	0.00	0.00
R:VERB:FORM	19	3	34	86.36	35.85	67.38
R:VERB:SVA	13	2	30	86.67	30.23	63.11
R:VERB:TENSE	7	1	42	87.50	14.29	43.21
R:WO	1	0	36	100.00	2.70	12.20
U:ADV	0	1	5	0.00	0.00	0.00
U:CONTR	0	0	1	100.00	0.00	0.00
U:DET	5	1	34	83.33	12.82	39.68
U:NOUN	1	0	3	100.00	25.00	62.50
U:NOUN:POSS	0	0	5	100.00	0.00	0.00
U:OTHER	1	7	9	12.50	10.00	11.90
U:PART	0	0	1	100.00	0.00	0.00
U:PREP	4	1	24	80.00	14.29	41.67
U:PRON	1	0	6	100.00	14.29	45.45
U:PUNCT	1	0	7	100.00	12.50	41.67
U:VERB	1	0	8	100.00	11.11	38.46
U:VERB:FORM	1	0	2	100.00	33.33	71.43
U:VERB:TENSE	2	0	5	100.00	28.57	66.67

SPAN-LEVEL DETECTION
OPERATION LEVEL
	TP	FP	FN	P	R	F_0.5
M	18	5	261	78.26	6.45	24.26
R	150	59	706	71.77	17.52	44.33
U	19	10	110	65.52	14.73	38.78
UNK	1	0	13	100.00	7.14	27.78

ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
ADJ	3	0	7	100.00	30.00	68.18
ADJ:FORM	1	0	2	100.00	33.33	71.43
ADV	2	1	12	66.67	14.29	38.46
CONJ	0	0	7	100.00	0.00	0.00
CONTR	0	0	2	100.00	0.00	0.00
DET	19	2	149	90.48	11.31	37.70
MORPH	6	1	27	85.71	18.18	49.18
NOUN	4	4	30	50.00	11.76	30.30
NOUN:NUM	15	3	45	83.33	25.00	56.82
NOUN:POSS	1	0	11	100.00	8.33	31.25
ORTH	33	39	46	45.83	41.77	44.96
OTHER	8	14	219	36.36	3.52	12.70
PART	0	0	4	100.00	0.00	0.00
PREP	12	2	118	85.71	9.23	32.26
PRON	2	0	29	100.00	6.45	25.64
PUNCT	9	5	95	64.29	8.65	28.12
SPELL	22	2	44	91.67	33.33	67.90
UNK	1	0	13	100.00	7.14	27.78
VERB	3	0	64	100.00	4.48	18.99
VERB:FORM	21	0	37	100.00	36.21	73.94
VERB:SVA	15	0	31	100.00	32.61	70.75
VERB:TENSE	10	1	61	90.91	14.08	43.48
WO	1	0	37	100.00	2.63	11.90

OPERATION + ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
M:ADJ	0	0	1	100.00	0.00	0.00
M:ADV	0	0	5	100.00	0.00	0.00
M:CONJ	0	0	5	100.00	0.00	0.00
M:DET	10	1	80	90.91	11.11	37.31
M:NOUN	0	1	4	0.00	0.00	0.00
M:NOUN:POSS	0	0	4	100.00	0.00	0.00
M:OTHER	1	2	13	33.33	7.14	19.23
M:PART	0	0	2	100.00	0.00	0.00
M:PREP	1	0	29	100.00	3.33	14.71
M:PRON	1	0	17	100.00	5.56	22.73
M:PUNCT	5	1	75	83.33	6.25	24.04
M:VERB	0	0	10	100.00	0.00	0.00
M:VERB:FORM	0	0	2	100.00	0.00	0.00
M:VERB:TENSE	0	0	14	100.00	0.00	0.00
R:ADJ	3	0	5	100.00	37.50	75.00
R:ADJ:FORM	1	0	2	100.00	33.33	71.43
R:ADV	1	0	3	100.00	25.00	62.50
R:CONJ	0	0	2	100.00	0.00	0.00
R:CONTR	0	0	1	100.00	0.00	0.00
R:DET	3	0	34	100.00	8.11	30.61
R:MORPH	6	1	27	85.71	18.18	49.18
R:NOUN	3	3	23	50.00	11.54	30.00
R:NOUN:NUM	15	3	45	83.33	25.00	56.82
R:NOUN:POSS	1	0	2	100.00	33.33	71.43
R:ORTH	33	39	46	45.83	41.77	44.96
R:OTHER	6	5	197	54.55	2.96	12.15
R:PART	0	0	1	100.00	0.00	0.00
R:PREP	7	1	65	87.50	9.72	33.65
R:PRON	0	0	7	100.00	0.00	0.00
R:PUNCT	3	4	13	42.86	18.75	34.09
R:SPELL	22	2	44	91.67	33.33	67.90
R:VERB	2	0	47	100.00	4.08	17.54
R:VERB:FORM	20	0	33	100.00	37.74	75.19
R:VERB:SVA	15	0	31	100.00	32.61	70.75
R:VERB:TENSE	8	1	41	88.89	16.33	47.06
R:WO	1	0	37	100.00	2.63	11.90
U:ADJ	0	0	1	100.00	0.00	0.00
U:ADV	1	1	4	50.00	20.00	38.46
U:CONTR	0	0	1	100.00	0.00	0.00
U:DET	6	1	35	85.71	14.63	43.48
U:NOUN	1	0	3	100.00	25.00	62.50
U:NOUN:POSS	0	0	5	100.00	0.00	0.00
U:OTHER	1	7	9	12.50	10.00	11.90
U:PART	0	0	1	100.00	0.00	0.00
U:PREP	4	1	24	80.00	14.29	41.67
U:PRON	1	0	5	100.00	16.67	50.00
U:PUNCT	1	0	7	100.00	12.50	41.67
U:VERB	1	0	7	100.00	12.50	41.67
U:VERB:FORM	1	0	2	100.00	33.33	71.43
U:VERB:TENSE	2	0	6	100.00	25.00	62.50
UNK	1	0	13	100.00	7.14	27.78

TOKEN-LEVEL DETECTION
OPERATION LEVEL
	TP	FP	FN	P	R	F_0.5
M	31	4	304	88.57	9.25	32.63
R	189	59	837	76.21	18.42	46.83
U	23	100	173	18.70	11.73	16.72
UNK	0	0	10	100.00	0.00	0.00

ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
ADJ	3	0	11	100.00	21.43	57.69
ADJ:FORM	1	0	2	100.00	33.33	71.43
ADV	4	1	23	80.00	14.81	42.55
CONJ	0	0	9	100.00	0.00	0.00
CONTR	0	0	2	100.00	0.00	0.00
DET	22	1	167	95.65	11.64	39.15
MORPH	6	0	28	100.00	17.65	51.72
NOUN	6	3	30	66.67	16.67	41.67
NOUN:NUM	15	1	46	93.75	24.59	60.00
NOUN:POSS	2	0	14	100.00	12.50	41.67
ORTH	38	37	58	50.67	39.58	47.98
OTHER	22	108	290	16.92	7.05	13.22
PART	1	0	4	100.00	20.00	55.56
PREP	13	2	130	86.67	9.09	32.02
PRON	2	0	40	100.00	4.76	20.00
PUNCT	17	7	110	70.83	13.39	38.12
SPELL	24	2	47	92.31	33.80	68.57
UNK	0	0	10	100.00	0.00	0.00
VERB	6	0	84	100.00	6.67	26.32
VERB:FORM	22	0	44	100.00	33.33	71.43
VERB:SVA	15	0	32	100.00	31.91	70.09
VERB:TENSE	15	1	83	93.75	15.31	46.30
WO	9	0	60	100.00	13.04	42.86

OPERATION + ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
M:ADJ	0	0	2	100.00	0.00	0.00
M:ADV	1	0	9	100.00	10.00	35.71
M:CONJ	0	0	6	100.00	0.00	0.00
M:DET	13	0	89	100.00	12.75	42.21
M:NOUN	0	1	4	0.00	0.00	0.00
M:NOUN:POSS	0	0	5	100.00	0.00	0.00
M:OTHER	1	2	18	33.33	5.26	16.13
M:PART	0	0	2	100.00	0.00	0.00
M:PREP	2	0	30	100.00	6.25	25.00
M:PRON	1	0	22	100.00	4.35	18.52
M:PUNCT	11	1	82	91.67	11.83	39.01
M:VERB	0	0	16	100.00	0.00	0.00
M:VERB:FORM	1	0	2	100.00	33.33	71.43
M:VERB:TENSE	1	0	17	100.00	5.56	22.73
R:ADJ	3	0	8	100.00	27.27	65.22
R:ADJ:FORM	1	0	2	100.00	33.33	71.43
R:ADV	2	0	4	100.00	33.33	71.43
R:CONJ	0	0	3	100.00	0.00	0.00
R:CONTR	0	0	1	100.00	0.00	0.00
R:DET	2	0	36	100.00	5.26	21.74
R:MORPH	6	0	28	100.00	17.65	51.72
R:NOUN	6	2	22	75.00	21.43	50.00
R:NOUN:NUM	15	1	46	93.75	24.59	60.00
R:NOUN:POSS	2	0	2	100.00	50.00	83.33
R:ORTH	38	37	58	50.67	39.58	47.98
R:OTHER	17	9	234	65.38	6.77	23.94
R:PART	1	0	1	100.00	50.00	83.33
R:PREP	7	1	73	87.50	8.75	31.25
R:PRON	0	0	11	100.00	0.00	0.00
R:PUNCT	5	6	16	45.45	23.81	38.46
R:SPELL	24	2	47	92.31	33.80	68.57
R:VERB	4	0	57	100.00	6.56	25.97
R:VERB:FORM	20	0	40	100.00	33.33	71.43
R:VERB:SVA	15	0	32	100.00	31.91	70.09
R:VERB:TENSE	12	1	56	92.31	17.65	50.00
R:WO	9	0	60	100.00	13.04	42.86
U:ADJ	0	0	1	100.00	0.00	0.00
U:ADV	1	1	10	50.00	9.09	26.32
U:CONTR	0	0	1	100.00	0.00	0.00
U:DET	7	1	42	87.50	14.29	43.21
U:NOUN	0	0	4	100.00	0.00	0.00
U:NOUN:POSS	0	0	7	100.00	0.00	0.00
U:OTHER	4	97	38	3.96	9.52	4.48
U:PART	0	0	1	100.00	0.00	0.00
U:PREP	4	1	27	80.00	12.90	39.22
U:PRON	1	0	7	100.00	12.50	41.67
U:PUNCT	1	0	12	100.00	7.69	29.41
U:VERB	2	0	11	100.00	15.38	47.62
U:VERB:FORM	1	0	2	100.00	33.33	71.43
U:VERB:TENSE	2	0	10	100.00	16.67	50.00
UNK	0	0	10	100.00	0.00	0.00

OVERALL
	TP	FP	FN	P	R	F_0.5
Span-level correction	74	52	391	58.73	15.91	38.18
Span-level detection	84	42	403	66.67	17.25	42.38
Token-level detection	93	38	453	70.99	17.03	43.46

SPAN-LEVEL CORRECTION
OPERATION LEVEL
	TP	FP	FN	P	R	F_0.5
M	7	6	98	53.85	6.67	22.29
R	61	40	246	60.40	19.87	42.90
U	6	6	47	50.00	11.32	29.70

ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
ADJ	1	0	6	100.00	14.29	45.45
ADJ:FORM	0	0	2	100.00	0.00	0.00
ADV	0	1	3	0.00	0.00	0.00
CONJ	0	0	4	100.00	0.00	0.00
CONTR	0	0	3	100.00	0.00	0.00
DET	6	5	67	54.55	8.22	25.64
MORPH	4	2	13	66.67	23.53	48.78
NOUN	1	1	9	50.00	10.00	27.78
NOUN:INFL	2	0	0	100.00	100.00	100.00
NOUN:NUM	6	3	20	66.67	23.08	48.39
NOUN:POSS	0	0	6	100.00	0.00	0.00
ORTH	28	22	18	56.00	60.87	56.91
OTHER	2	2	55	50.00	3.51	13.70
PART	0	0	3	100.00	0.00	0.00
PREP	3	2	47	60.00	6.00	21.43
PRON	1	0	8	100.00	11.11	38.46
PUNCT	6	3	39	66.67	13.33	37.04
SPELL	6	5	30	54.55	16.67	37.50
VERB	0	3	9	0.00	0.00	0.00
VERB:FORM	4	1	9	80.00	30.77	60.61
VERB:SVA	2	0	6	100.00	25.00	62.50
VERB:TENSE	2	1	24	66.67	7.69	26.32
WO	0	1	10	0.00	0.00	0.00

OPERATION + ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
M:ADJ	0	0	1	100.00	0.00	0.00
M:ADV	0	0	1	100.00	0.00	0.00
M:CONJ	0	0	2	100.00	0.00	0.00
M:DET	2	4	33	33.33	5.71	16.95
M:NOUN	0	0	2	100.00	0.00	0.00
M:OTHER	0	0	5	100.00	0.00	0.00
M:PREP	0	0	15	100.00	0.00	0.00
M:PRON	0	0	5	100.00	0.00	0.00
M:PUNCT	5	2	29	71.43	14.71	40.32
M:VERB	0	0	2	100.00	0.00	0.00
M:VERB:FORM	0	0	1	100.00	0.00	0.00
M:VERB:TENSE	0	0	2	100.00	0.00	0.00
R:ADJ	1	0	4	100.00	20.00	55.56
R:ADJ:FORM	0	0	2	100.00	0.00	0.00
R:ADV	0	0	1	100.00	0.00	0.00
R:CONJ	0	0	2	100.00	0.00	0.00
R:CONTR	0	0	1	100.00	0.00	0.00
R:DET	2	0	8	100.00	20.00	55.56
R:MORPH	4	2	13	66.67	23.53	48.78
R:NOUN	1	1	7	50.00	12.50	31.25
R:NOUN:INFL	2	0	0	100.00	100.00	100.00
R:NOUN:NUM	6	3	20	66.67	23.08	48.39
R:NOUN:POSS	0	0	5	100.00	0.00	0.00
R:ORTH	28	22	18	56.00	60.87	56.91
R:OTHER	2	1	49	66.67	3.92	15.87
R:PART	0	0	2	100.00	0.00	0.00
R:PREP	3	0	25	100.00	10.71	37.50
R:PRON	1	0	2	100.00	33.33	71.43
R:PUNCT	0	0	8	100.00	0.00	0.00
R:SPELL	6	5	30	54.55	16.67	37.50
R:VERB	0	3	6	0.00	0.00	0.00
R:VERB:FORM	3	1	8	75.00	27.27	55.56
R:VERB:SVA	2	0	6	100.00	25.00	62.50
R:VERB:TENSE	0	1	19	0.00	0.00	0.00
R:WO	0	1	10	0.00	0.00	0.00
U:ADJ	0	0	1	100.00	0.00	0.00
U:ADV	0	1	1	0.00	0.00	0.00
U:CONTR	0	0	2	100.00	0.00	0.00
U:DET	2	1	26	66.67	7.14	25.00
U:NOUN:POSS	0	0	1	100.00	0.00	0.00
U:OTHER	0	1	1	0.00	0.00	0.00
U:PART	0	0	1	100.00	0.00	0.00
U:PREP	0	2	7	0.00	0.00	0.00
U:PRON	0	0	1	100.00	0.00	0.00
U:PUNCT	1	1	2	50.00	33.33	45.45
U:VERB	0	0	1	100.00	0.00	0.00
U:VERB:FORM	1	0	0	100.00	100.00	100.00
U:VERB:TENSE	2	0	3	100.00	40.00	76.92

SPAN-LEVEL DETECTION
OPERATION LEVEL
	TP	FP	FN	P	R	F_0.5
M	7	6	103	53.85	6.36	21.60
R	69	32	247	68.32	21.84	47.92
U	7	4	49	63.64	12.50	35.00
UNK	1	0	4	100.00	20.00	55.56

ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
ADJ	1	0	6	100.00	14.29	45.45
ADJ:FORM	0	0	2	100.00	0.00	0.00
ADV	1	0	3	100.00	25.00	62.50
CONJ	0	0	4	100.00	0.00	0.00
CONTR	0	0	3	100.00	0.00	0.00
DET	7	5	70	58.33	9.09	28.00
MORPH	4	2	13	66.67	23.53	48.78
NOUN	1	1	11	50.00	8.33	25.00
NOUN:INFL	3	0	0	100.00	100.00	100.00
NOUN:NUM	7	3	21	70.00	25.00	51.47
NOUN:POSS	0	0	6	100.00	0.00	0.00
ORTH	28	18	18	60.87	60.87	60.87
OTHER	3	2	57	60.00	5.00	18.75
PART	1	0	2	100.00	33.33	71.43
PREP	3	1	47	75.00	6.00	22.73
PRON	1	0	8	100.00	11.11	38.46
PUNCT	5	3	39	62.50	11.36	32.89
SPELL	8	2	28	80.00	22.22	52.63
UNK	1	0	4	100.00	20.00	55.56
VERB	1	2	11	33.33	8.33	20.83
VERB:FORM	5	1	10	83.33	33.33	64.10
VERB:SVA	2	0	6	100.00	25.00	62.50
VERB:TENSE	2	1	25	66.67	7.41	25.64
WO	0	1	9	0.00	0.00	0.00

OPERATION + ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
M:ADJ	0	0	1	100.00	0.00	0.00
M:ADV	0	0	1	100.00	0.00	0.00
M:CONJ	0	0	2	100.00	0.00	0.00
M:DET	2	4	33	33.33	5.71	16.95
M:NOUN	0	0	3	100.00	0.00	0.00
M:OTHER	1	0	7	100.00	12.50	41.67
M:PART	0	0	1	100.00	0.00	0.00
M:PREP	0	0	15	100.00	0.00	0.00
M:PRON	0	0	5	100.00	0.00	0.00
M:PUNCT	4	2	29	66.67	12.12	35.09
M:VERB	0	0	2	100.00	0.00	0.00
M:VERB:FORM	0	0	1	100.00	0.00	0.00
M:VERB:TENSE	0	0	3	100.00	0.00	0.00
R:ADJ	1	0	4	100.00	20.00	55.56
R:ADJ:FORM	0	0	2	100.00	0.00	0.00
R:ADV	1	0	1	100.00	50.00	83.33
R:CONJ	0	0	2	100.00	0.00	0.00
R:CONTR	0	0	1	100.00	0.00	0.00
R:DET	2	0	8	100.00	20.00	55.56
R:MORPH	4	2	13	66.67	23.53	48.78
R:NOUN	1	1	8	50.00	11.11	29.41
R:NOUN:INFL	3	0	0	100.00	100.00	100.00
R:NOUN:NUM	7	3	21	70.00	25.00	51.47
R:NOUN:POSS	0	0	5	100.00	0.00	0.00
R:ORTH	28	18	18	60.87	60.87	60.87
R:OTHER	2	1	49	66.67	3.92	15.87
R:PART	1	0	1	100.00	50.00	83.33
R:PREP	3	0	25	100.00	10.71	37.50
R:PRON	1	0	2	100.00	33.33	71.43
R:PUNCT	0	0	8	100.00	0.00	0.00
R:SPELL	8	2	28	80.00	22.22	52.63
R:VERB	1	2	8	33.33	11.11	23.81
R:VERB:FORM	4	1	9	80.00	30.77	60.61
R:VERB:SVA	2	0	6	100.00	25.00	62.50
R:VERB:TENSE	0	1	19	0.00	0.00	0.00
R:WO	0	1	9	0.00	0.00	0.00
U:ADJ	0	0	1	100.00	0.00	0.00
U:ADV	0	0	1	100.00	0.00	0.00
U:CONTR	0	0	2	100.00	0.00	0.00
U:DET	3	1	29	75.00	9.38	31.25
U:NOUN:POSS	0	0	1	100.00	0.00	0.00
U:OTHER	0	1	1	0.00	0.00	0.00
U:PREP	0	1	7	0.00	0.00	0.00
U:PRON	0	0	1	100.00	0.00	0.00
U:PUNCT	1	1	2	50.00	33.33	45.45
U:VERB	0	0	1	100.00	0.00	0.00
U:VERB:FORM	1	0	0	100.00	100.00	100.00
U:VERB:TENSE	2	0	3	100.00	40.00	76.92
UNK	1	0	4	100.00	20.00	55.56

TOKEN-LEVEL DETECTION
OPERATION LEVEL
	TP	FP	FN	P	R	F_0.5
M	9	5	120	64.29	6.98	24.32
R	75	27	270	73.53	21.74	49.80
U	8	6	60	57.14	11.76	32.26
UNK	1	0	3	100.00	25.00	62.50

ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
ADJ	0	0	9	100.00	0.00	0.00
ADJ:FORM	0	0	3	100.00	0.00	0.00
ADV	1	0	5	100.00	16.67	50.00
CONJ	0	0	3	100.00	0.00	0.00
CONTR	0	0	4	100.00	0.00	0.00
DET	7	4	73	63.64	8.75	28.23
MORPH	4	2	15	66.67	21.05	46.51
NOUN	2	1	14	66.67	12.50	35.71
NOUN:INFL	2	0	0	100.00	100.00	100.00
NOUN:NUM	6	2	25	75.00	19.35	47.62
NOUN:POSS	0	0	8	100.00	0.00	0.00
ORTH	32	14	18	69.57	64.00	68.38
OTHER	8	4	65	66.67	10.96	33.06
PART	1	0	2	100.00	33.33	71.43
PREP	3	1	48	75.00	5.88	22.39
PRON	2	0	9	100.00	18.18	52.63
PUNCT	6	3	51	66.67	10.53	32.26
SPELL	8	2	29	80.00	21.62	51.95
UNK	1	0	3	100.00	25.00	62.50
VERB	2	2	13	50.00	13.33	32.26
VERB:FORM	4	0	11	100.00	26.67	64.52
VERB:SVA	2	0	8	100.00	20.00	55.56
VERB:TENSE	2	1	30	66.67	6.25	22.73
WO	0	2	7	0.00	0.00	0.00

OPERATION + ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
M:ADJ	0	0	2	100.00	0.00	0.00
M:ADV	0	0	2	100.00	0.00	0.00
M:CONJ	0	0	2	100.00	0.00	0.00
M:DET	2	3	34	40.00	5.56	17.86
M:NOUN	0	0	3	100.00	0.00	0.00
M:OTHER	1	0	8	100.00	11.11	38.46
M:PART	0	0	1	100.00	0.00	0.00
M:PREP	0	0	15	100.00	0.00	0.00
M:PRON	1	0	5	100.00	16.67	50.00
M:PUNCT	5	2	40	71.43	11.11	34.25
M:VERB	0	0	3	100.00	0.00	0.00
M:VERB:FORM	0	0	1	100.00	0.00	0.00
M:VERB:TENSE	0	0	4	100.00	0.00	0.00
R:ADJ	0	0	4	100.00	0.00	0.00
R:ADJ:FORM	0	0	3	100.00	0.00	0.00
R:ADV	1	0	2	100.00	33.33	71.43
R:CONJ	0	0	1	100.00	0.00	0.00
R:CONTR	0	0	1	100.00	0.00	0.00
R:DET	2	0	9	100.00	18.18	52.63
R:MORPH	4	2	15	66.67	21.05	46.51
R:NOUN	1	1	9	50.00	10.00	27.78
R:NOUN:INFL	2	0	0	100.00	100.00	100.00
R:NOUN:NUM	6	2	25	75.00	19.35	47.62
R:NOUN:POSS	0	0	7	100.00	0.00	0.00
R:ORTH	32	14	18	69.57	64.00	68.38
R:OTHER	7	1	53	87.50	11.67	38.04
R:PART	1	0	1	100.00	50.00	83.33
R:PREP	3	0	26	100.00	10.34	36.59
R:PRON	1	0	2	100.00	33.33	71.43
R:PUNCT	0	0	8	100.00	0.00	0.00
R:SPELL	8	2	29	80.00	21.62	51.95
R:VERB	2	2	9	50.00	18.18	37.04
R:VERB:FORM	3	0	10	100.00	23.08	60.00
R:VERB:SVA	2	0	8	100.00	20.00	55.56
R:VERB:TENSE	0	1	23	0.00	0.00	0.00
R:WO	0	2	7	0.00	0.00	0.00
U:ADJ	0	0	3	100.00	0.00	0.00
U:ADV	0	0	1	100.00	0.00	0.00
U:CONTR	0	0	3	100.00	0.00	0.00
U:DET	3	1	30	75.00	9.09	30.61
U:NOUN	1	0	2	100.00	33.33	71.43
U:NOUN:POSS	0	0	1	100.00	0.00	0.00
U:OTHER	0	3	4	0.00	0.00	0.00
U:PREP	0	1	7	0.00	0.00	0.00
U:PRON	0	0	2	100.00	0.00	0.00
U:PUNCT	1	1	3	50.00	25.00	41.67
U:VERB	0	0	1	100.00	0.00	0.00
U:VERB:FORM	1	0	0	100.00	100.00	100.00
U:VERB:TENSE	2	0	3	100.00	40.00	76.92
UNK	1	0	3	100.00	25.00	62.50

OVERALL
	TP	FP	FN	P	R	F_0.5
Span-level correction	46	71	314	39.32	12.78	27.78
Span-level detection	60	57	312	51.28	16.13	35.71
Token-level detection	66	57	342	53.66	16.18	36.67

SPAN-LEVEL CORRECTION
OPERATION LEVEL
	TP	FP	FN	P	R	F_0.5
M	4	5	39	44.44	9.30	25.32
R	38	57	261	40.00	12.71	27.98
U	4	9	14	30.77	22.22	28.57

ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
ADJ	1	2	3	33.33	25.00	31.25
ADV	0	3	0	0.00	100.00	0.00
DET	4	5	6	44.44	40.00	43.48
MORPH	1	0	3	100.00	25.00	62.50
NOUN	1	1	8	50.00	11.11	29.41
NOUN:INFL	0	0	11	100.00	0.00	0.00
NOUN:NUM	3	5	12	37.50	20.00	31.91
NOUN:POSS	0	1	15	0.00	0.00	0.00
ORTH	18	37	37	32.73	32.73	32.73
OTHER	0	5	47	0.00	0.00	0.00
PART	1	0	0	100.00	100.00	100.00
PREP	0	5	12	0.00	0.00	0.00
PRON	0	0	5	100.00	0.00	0.00
PUNCT	2	2	46	50.00	4.17	15.62
SPELL	10	1	80	90.91	11.11	37.31
VERB	2	1	9	66.67	18.18	43.48
VERB:FORM	2	1	3	66.67	40.00	58.82
VERB:INFL	0	0	5	100.00	0.00	0.00
VERB:SVA	1	2	5	33.33	16.67	27.78
VERB:TENSE	0	0	6	100.00	0.00	0.00
WO	0	0	1	100.00	0.00	0.00

OPERATION + ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
M:DET	2	2	2	50.00	50.00	50.00
M:NOUN	0	0	2	100.00	0.00	0.00
M:NOUN:POSS	0	1	4	0.00	0.00	0.00
M:PREP	0	0	1	100.00	0.00	0.00
M:PRON	0	0	2	100.00	0.00	0.00
M:PUNCT	1	2	26	33.33	3.70	12.82
M:VERB	1	0	2	100.00	33.33	71.43
R:ADJ	1	2	3	33.33	25.00	31.25
R:DET	0	1	2	0.00	0.00	0.00
R:MORPH	1	0	3	100.00	25.00	62.50
R:NOUN	1	1	6	50.00	14.29	33.33
R:NOUN:INFL	0	0	11	100.00	0.00	0.00
R:NOUN:NUM	3	5	12	37.50	20.00	31.91
R:NOUN:POSS	0	0	11	100.00	0.00	0.00
R:ORTH	18	37	37	32.73	32.73	32.73
R:OTHER	0	4	47	0.00	0.00	0.00
R:PREP	0	2	10	0.00	0.00	0.00
R:PRON	0	0	3	100.00	0.00	0.00
R:PUNCT	0	0	12	100.00	0.00	0.00
R:SPELL	10	1	80	90.91	11.11	37.31
R:VERB	1	1	6	50.00	14.29	33.33
R:VERB:FORM	2	1	3	66.67	40.00	58.82
R:VERB:INFL	0	0	5	100.00	0.00	0.00
R:VERB:SVA	1	2	5	33.33	16.67	27.78
R:VERB:TENSE	0	0	4	100.00	0.00	0.00
R:WO	0	0	1	100.00	0.00	0.00
U:ADV	0	3	0	0.00	100.00	0.00
U:DET	2	2	2	50.00	50.00	50.00
U:OTHER	0	1	0	0.00	100.00	0.00
U:PART	1	0	0	100.00	100.00	100.00
U:PREP	0	3	1	0.00	0.00	0.00
U:PUNCT	1	0	8	100.00	11.11	38.46
U:VERB	0	0	1	100.00	0.00	0.00
U:VERB:TENSE	0	0	2	100.00	0.00	0.00

SPAN-LEVEL DETECTION
OPERATION LEVEL
	TP	FP	FN	P	R	F_0.5
M	6	3	40	66.67	13.04	36.59
R	48	46	256	51.06	15.79	35.29
U	5	8	14	38.46	26.32	35.21
UNK	1	0	2	100.00	33.33	71.43

ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
ADJ	1	1	3	50.00	25.00	41.67
ADV	0	2	0	0.00	100.00	0.00
DET	6	2	6	75.00	50.00	68.18
MORPH	2	0	2	100.00	50.00	83.33
NOUN	1	1	8	50.00	11.11	29.41
NOUN:INFL	0	0	11	100.00	0.00	0.00
NOUN:NUM	3	5	12	37.50	20.00	31.91
NOUN:POSS	0	1	15	0.00	0.00	0.00
ORTH	19	34	37	35.85	33.93	35.45
OTHER	0	2	48	0.00	0.00	0.00
PART	1	0	0	100.00	100.00	100.00
PREP	0	5	12	0.00	0.00	0.00
PRON	0	0	5	100.00	0.00	0.00
PUNCT	3	2	47	60.00	6.00	21.43
SPELL	15	0	77	100.00	16.30	49.34
UNK	1	0	2	100.00	33.33	71.43
VERB	4	0	8	100.00	33.33	71.43
VERB:FORM	2	0	3	100.00	40.00	76.92
VERB:INFL	1	0	4	100.00	20.00	55.56
VERB:SVA	1	2	5	33.33	16.67	27.78
VERB:TENSE	0	0	6	100.00	0.00	0.00
WO	0	0	1	100.00	0.00	0.00

OPERATION + ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
M:DET	3	0	2	100.00	60.00	88.24
M:NOUN	0	0	2	100.00	0.00	0.00
M:NOUN:POSS	0	1	4	0.00	0.00	0.00
M:PREP	0	0	1	100.00	0.00	0.00
M:PRON	0	0	2	100.00	0.00	0.00
M:PUNCT	2	2	27	50.00	6.90	22.22
M:VERB	1	0	2	100.00	33.33	71.43
R:ADJ	1	1	3	50.00	25.00	41.67
R:DET	0	0	2	100.00	0.00	0.00
R:MORPH	2	0	2	100.00	50.00	83.33
R:NOUN	1	1	6	50.00	14.29	33.33
R:NOUN:INFL	0	0	11	100.00	0.00	0.00
R:NOUN:NUM	3	5	12	37.50	20.00	31.91
R:NOUN:POSS	0	0	11	100.00	0.00	0.00
R:ORTH	19	34	37	35.85	33.93	35.45
R:OTHER	0	1	48	0.00	0.00	0.00
R:PREP	0	2	10	0.00	0.00	0.00
R:PRON	0	0	3	100.00	0.00	0.00
R:PUNCT	0	0	12	100.00	0.00	0.00
R:SPELL	15	0	77	100.00	16.30	49.34
R:VERB	3	0	5	100.00	37.50	75.00
R:VERB:FORM	2	0	3	100.00	40.00	76.92
R:VERB:INFL	1	0	4	100.00	20.00	55.56
R:VERB:SVA	1	2	5	33.33	16.67	27.78
R:VERB:TENSE	0	0	4	100.00	0.00	0.00
R:WO	0	0	1	100.00	0.00	0.00
U:ADV	0	2	0	0.00	100.00	0.00
U:DET	3	2	2	60.00	60.00	60.00
U:OTHER	0	1	0	0.00	100.00	0.00
U:PART	1	0	0	100.00	100.00	100.00
U:PREP	0	3	1	0.00	0.00	0.00
U:PUNCT	1	0	8	100.00	11.11	38.46
U:VERB	0	0	1	100.00	0.00	0.00
U:VERB:TENSE	0	0	2	100.00	0.00	0.00
UNK	1	0	2	100.00	33.33	71.43

TOKEN-LEVEL DETECTION
OPERATION LEVEL
	TP	FP	FN	P	R	F_0.5
M	10	3	46	76.92	17.86	46.30
R	49	43	278	53.26	14.98	35.25
U	5	11	16	31.25	23.81	29.41
UNK	2	0	2	100.00	50.00	83.33

ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
ADJ	1	1	4	50.00	20.00	38.46
ADV	0	3	0	0.00	100.00	0.00
CONTR	0	0	3	100.00	0.00	0.00
DET	4	2	7	66.67	36.36	57.14
MORPH	2	0	4	100.00	33.33	71.43
NOUN	1	1	10	50.00	9.09	26.32
NOUN:INFL	0	0	11	100.00	0.00	0.00
NOUN:NUM	3	5	13	37.50	18.75	31.25
NOUN:POSS	0	1	17	0.00	0.00	0.00
ORTH	19	32	45	37.25	29.69	35.45
OTHER	3	4	50	42.86	5.66	18.52
PART	1	0	0	100.00	100.00	100.00
PREP	0	4	11	0.00	0.00	0.00
PRON	0	0	6	100.00	0.00	0.00
PUNCT	8	2	54	80.00	12.90	39.22
SPELL	14	0	78	100.00	15.22	47.30
UNK	2	0	2	100.00	50.00	83.33
VERB	4	0	9	100.00	30.77	68.97
VERB:FORM	2	0	3	100.00	40.00	76.92
VERB:INFL	1	0	4	100.00	20.00	55.56
VERB:SVA	1	2	5	33.33	16.67	27.78
VERB:TENSE	0	0	6	100.00	0.00	0.00

OPERATION + ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
M:DET	2	0	2	100.00	50.00	83.33
M:NOUN	0	0	3	100.00	0.00	0.00
M:NOUN:POSS	0	1	4	0.00	0.00	0.00
M:PREP	0	0	1	100.00	0.00	0.00
M:PRON	0	0	3	100.00	0.00	0.00
M:PUNCT	7	2	31	77.78	18.42	47.30
M:VERB	1	0	2	100.00	33.33	71.43
R:ADJ	1	1	3	50.00	25.00	41.67
R:CONTR	0	0	3	100.00	0.00	0.00
R:DET	0	0	3	100.00	0.00	0.00
R:MORPH	2	0	4	100.00	33.33	71.43
R:NOUN	1	1	7	50.00	12.50	31.25
R:NOUN:INFL	0	0	11	100.00	0.00	0.00
R:NOUN:NUM	3	5	13	37.50	18.75	31.25
R:NOUN:POSS	0	0	13	100.00	0.00	0.00
R:ORTH	19	32	45	37.25	29.69	35.45
R:OTHER	2	1	49	66.67	3.92	15.87
R:PREP	0	1	9	0.00	0.00	0.00
R:PRON	0	0	3	100.00	0.00	0.00
R:PUNCT	0	0	15	100.00	0.00	0.00
R:SPELL	14	0	78	100.00	15.22	47.30
R:VERB	3	0	6	100.00	33.33	71.43
R:VERB:FORM	2	0	3	100.00	40.00	76.92
R:VERB:INFL	1	0	4	100.00	20.00	55.56
R:VERB:SVA	1	2	5	33.33	16.67	27.78
R:VERB:TENSE	0	0	4	100.00	0.00	0.00
U:ADJ	0	0	1	100.00	0.00	0.00
U:ADV	0	3	0	0.00	100.00	0.00
U:DET	2	2	2	50.00	50.00	50.00
U:OTHER	1	3	1	25.00	50.00	27.78
U:PART	1	0	0	100.00	100.00	100.00
U:PREP	0	3	1	0.00	0.00	0.00
U:PUNCT	1	0	8	100.00	11.11	38.46
U:VERB	0	0	1	100.00	0.00	0.00
U:VERB:TENSE	0	0	2	100.00	0.00	0.00
UNK	2	0	2	100.00	50.00	83.33

Submitted by:	cjbryant
Submitted at:	2019-08-07T14:43:09+00:00
Competition submission:	3
Competition phase:	3 - Open phase

Detailed results

Submission information

Results