BEA 2019 Shared Task: Grammatical Error Correction

OVERALL
	TP	FP	FN	P	R	F_0.5
Span-level correction	699	374	3265	65.14	17.63	42.33
Span-level detection	799	275	3303	74.39	19.48	47.57
Token-level detection	951	241	3900	79.78	19.60	49.43

SPAN-LEVEL CORRECTION
OPERATION LEVEL
	TP	FP	FN	P	R	F_0.5
M	77	32	734	70.64	9.49	30.87
R	543	292	2220	65.03	19.65	44.49
U	79	50	311	61.24	20.26	43.60

ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
ADJ	1	4	25	20.00	3.85	10.87
ADJ:FORM	3	2	5	60.00	37.50	53.57
ADV	6	4	35	60.00	14.63	37.04
CONJ	0	0	18	100.00	0.00	0.00
CONTR	4	1	6	80.00	40.00	66.67
DET	70	20	379	77.78	15.59	43.26
MORPH	31	6	97	83.78	24.22	56.16
NOUN	5	19	88	20.83	5.38	13.23
NOUN:INFL	5	3	14	62.50	26.32	49.02
NOUN:NUM	49	15	156	76.56	23.90	53.15
NOUN:POSS	4	0	40	100.00	9.09	33.33
ORTH	169	146	204	53.65	45.31	51.75
OTHER	19	33	661	36.54	2.79	10.70
PART	4	1	14	80.00	22.22	52.63
PREP	31	17	291	64.58	9.63	30.16
PRON	12	5	95	70.59	11.21	34.29
PUNCT	41	23	353	64.06	10.41	31.54
SPELL	63	19	235	76.83	21.14	50.32
VERB	8	9	154	47.06	4.94	17.39
VERB:FORM	82	18	84	82.00	49.40	72.44
VERB:INFL	1	0	8	100.00	11.11	38.46
VERB:SVA	59	12	68	83.10	46.46	71.78
VERB:TENSE	24	11	174	68.57	12.12	35.50
WO	8	6	61	57.14	11.59	32.00

OPERATION + ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
M:ADJ	0	1	3	0.00	0.00	0.00
M:ADV	1	1	11	50.00	8.33	25.00
M:CONJ	0	0	9	100.00	0.00	0.00
M:CONTR	1	1	0	50.00	100.00	55.56
M:DET	22	9	187	70.97	10.53	33.03
M:NOUN	0	0	12	100.00	0.00	0.00
M:NOUN:POSS	1	0	11	100.00	8.33	31.25
M:OTHER	0	3	46	0.00	0.00	0.00
M:PART	0	0	3	100.00	0.00	0.00
M:PREP	3	2	72	60.00	4.00	15.79
M:PRON	7	2	51	77.78	12.07	37.23
M:PUNCT	29	10	263	74.36	9.93	32.37
M:VERB	3	1	28	75.00	9.68	31.91
M:VERB:FORM	10	2	5	83.33	66.67	79.37
M:VERB:TENSE	0	0	33	100.00	0.00	0.00
R:ADJ	0	2	20	0.00	0.00	0.00
R:ADJ:FORM	3	2	5	60.00	37.50	53.57
R:ADV	2	2	11	50.00	15.38	34.48
R:CONJ	0	0	6	100.00	0.00	0.00
R:CONTR	1	0	2	100.00	33.33	71.43
R:DET	5	1	78	83.33	6.02	23.36
R:MORPH	31	6	97	83.78	24.22	56.16
R:NOUN	4	14	67	22.22	5.63	13.99
R:NOUN:INFL	5	3	14	62.50	26.32	49.02
R:NOUN:NUM	49	15	156	76.56	23.90	53.15
R:NOUN:POSS	1	0	22	100.00	4.35	18.52
R:ORTH	169	146	204	53.65	45.31	51.75
R:OTHER	16	20	596	44.44	2.61	10.58
R:PART	4	0	6	100.00	40.00	76.92
R:PREP	22	6	163	78.57	11.89	37.04
R:PRON	2	2	29	50.00	6.45	21.28
R:PUNCT	6	11	56	35.29	9.68	23.08
R:SPELL	63	19	235	76.83	21.14	50.32
R:VERB	3	4	111	42.86	2.63	10.56
R:VERB:FORM	69	15	76	82.14	47.59	71.73
R:VERB:INFL	1	0	8	100.00	11.11	38.46
R:VERB:SVA	59	12	68	83.10	46.46	71.78
R:VERB:TENSE	20	6	129	76.92	13.42	39.53
R:WO	8	6	61	57.14	11.59	32.00
U:ADJ	1	1	2	50.00	33.33	45.45
U:ADV	3	1	13	75.00	18.75	46.88
U:CONJ	0	0	3	100.00	0.00	0.00
U:CONTR	2	0	4	100.00	33.33	71.43
U:DET	43	10	114	81.13	27.39	58.27
U:NOUN	1	5	9	16.67	10.00	14.71
U:NOUN:POSS	2	0	7	100.00	22.22	58.82
U:OTHER	3	10	19	23.08	13.64	20.27
U:PART	0	1	5	0.00	0.00	0.00
U:PREP	6	9	56	40.00	9.68	24.59
U:PRON	3	1	15	75.00	16.67	44.12
U:PUNCT	6	2	34	75.00	15.00	41.67
U:VERB	2	4	15	33.33	11.76	24.39
U:VERB:FORM	3	1	3	75.00	50.00	68.18
U:VERB:TENSE	4	5	12	44.44	25.00	38.46

SPAN-LEVEL DETECTION
OPERATION LEVEL
	TP	FP	FN	P	R	F_0.5
M	83	27	737	75.45	10.12	32.94
R	623	208	2180	74.97	22.23	50.84
U	85	40	309	68.00	21.57	47.54
UNK	8	0	77	100.00	9.41	34.19

ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
ADJ	5	2	26	71.43	16.13	42.37
ADJ:FORM	2	1	5	66.67	28.57	52.63
ADV	7	3	35	70.00	16.67	42.68
CONJ	0	0	17	100.00	0.00	0.00
CONTR	3	1	6	75.00	33.33	60.00
DET	77	17	375	81.91	17.04	46.50
MORPH	34	4	97	89.47	25.95	60.07
NOUN	7	13	91	35.00	7.14	19.66
NOUN:INFL	5	3	14	62.50	26.32	49.02
NOUN:NUM	46	13	156	77.97	22.77	52.51
NOUN:POSS	5	0	40	100.00	11.11	38.46
ORTH	175	121	194	59.12	47.43	56.34
OTHER	55	24	664	69.62	7.65	26.57
PART	5	1	13	83.33	27.78	59.52
PREP	34	14	296	70.83	10.30	32.57
PRON	15	2	91	88.24	14.15	43.10
PUNCT	44	20	348	68.75	11.22	33.95
SPELL	75	4	226	94.94	24.92	60.78
UNK	8	0	77	100.00	9.41	34.19
VERB	17	5	156	77.27	9.83	32.57
VERB:FORM	82	10	81	89.13	50.31	77.21
VERB:INFL	2	0	7	100.00	22.22	58.82
VERB:SVA	60	4	66	93.75	47.62	78.53
VERB:TENSE	28	7	165	80.00	14.51	42.04
WO	8	6	57	57.14	12.31	33.06

OPERATION + ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
M:ADJ	0	1	3	0.00	0.00	0.00
M:ADV	1	1	10	50.00	9.09	26.32
M:CONJ	0	0	10	100.00	0.00	0.00
M:CONTR	0	1	0	0.00	100.00	0.00
M:DET	23	8	185	74.19	11.06	34.64
M:NOUN	0	0	13	100.00	0.00	0.00
M:NOUN:POSS	1	0	11	100.00	8.33	31.25
M:OTHER	1	2	49	33.33	2.00	8.06
M:PART	0	0	3	100.00	0.00	0.00
M:PREP	3	2	75	60.00	3.85	15.31
M:PRON	8	2	49	80.00	14.04	41.24
M:PUNCT	31	8	264	79.49	10.51	34.37
M:VERB	5	0	27	100.00	15.62	48.08
M:VERB:FORM	10	2	4	83.33	71.43	80.65
M:VERB:TENSE	0	0	34	100.00	0.00	0.00
R:ADJ	4	1	19	80.00	17.39	46.51
R:ADJ:FORM	2	1	5	66.67	28.57	52.63
R:ADV	3	1	13	75.00	18.75	46.88
R:CONJ	0	0	5	100.00	0.00	0.00
R:CONTR	1	0	2	100.00	33.33	71.43
R:DET	9	1	76	90.00	10.59	36.00
R:MORPH	34	4	97	89.47	25.95	60.07
R:NOUN	5	8	66	38.46	7.04	20.33
R:NOUN:INFL	5	3	14	62.50	26.32	49.02
R:NOUN:NUM	46	13	156	77.97	22.77	52.51
R:NOUN:POSS	2	0	20	100.00	9.09	33.33
R:ORTH	175	121	194	59.12	47.43	56.34
R:OTHER	51	12	596	80.95	7.88	28.36
R:PART	5	0	6	100.00	45.45	80.65
R:PREP	25	6	166	80.65	13.09	39.68
R:PRON	3	0	30	100.00	9.09	33.33
R:PUNCT	6	10	52	37.50	10.34	24.59
R:SPELL	75	4	226	94.94	24.92	60.78
R:VERB	9	2	115	81.82	7.26	26.79
R:VERB:FORM	69	8	74	89.61	48.25	76.50
R:VERB:INFL	2	0	7	100.00	22.22	58.82
R:VERB:SVA	60	4	66	93.75	47.62	78.53
R:VERB:TENSE	24	3	118	88.89	16.90	48.00
R:WO	8	6	57	57.14	12.31	33.06
U:ADJ	1	0	4	100.00	20.00	55.56
U:ADV	3	1	12	75.00	20.00	48.39
U:CONJ	0	0	2	100.00	0.00	0.00
U:CONTR	2	0	4	100.00	33.33	71.43
U:DET	45	8	114	84.91	28.30	60.65
U:NOUN	2	5	12	28.57	14.29	23.81
U:NOUN:POSS	2	0	9	100.00	18.18	52.63
U:OTHER	3	10	19	23.08	13.64	20.27
U:PART	0	1	4	0.00	0.00	0.00
U:PREP	6	6	55	50.00	9.84	27.52
U:PRON	4	0	12	100.00	25.00	62.50
U:PUNCT	7	2	32	77.78	17.95	46.67
U:VERB	3	3	14	50.00	17.65	36.59
U:VERB:FORM	3	0	3	100.00	50.00	83.33
U:VERB:TENSE	4	4	13	50.00	23.53	40.82
UNK	8	0	77	100.00	9.41	34.19

TOKEN-LEVEL DETECTION
OPERATION LEVEL
	TP	FP	FN	P	R	F_0.5
M	118	18	844	86.76	12.27	39.18
R	713	179	2530	79.93	21.99	52.34
U	108	44	440	71.05	19.71	46.71
UNK	12	0	86	100.00	12.24	41.10

ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
ADJ	6	1	37	85.71	13.95	42.25
ADJ:FORM	2	1	9	66.67	18.18	43.48
ADV	5	2	48	71.43	9.43	30.86
CONJ	1	0	18	100.00	5.26	21.74
CONTR	3	1	8	75.00	27.27	55.56
DET	83	12	432	87.37	16.12	46.37
MORPH	34	2	100	94.44	25.37	61.15
NOUN	15	8	113	65.22	11.72	34.09
NOUN:INFL	4	3	14	57.14	22.22	43.48
NOUN:NUM	47	8	169	85.45	21.76	53.90
NOUN:POSS	7	0	48	100.00	12.73	42.17
ORTH	192	110	226	63.58	45.93	59.04
OTHER	109	31	834	77.86	11.56	36.26
PART	3	1	12	75.00	20.00	48.39
PREP	39	10	330	79.59	10.57	34.51
PRON	21	1	109	95.45	16.15	48.17
PUNCT	64	17	397	79.01	13.88	40.76
SPELL	73	3	237	96.05	23.55	59.45
UNK	12	0	86	100.00	12.24	41.10
VERB	23	7	215	76.67	9.66	32.12
VERB:FORM	80	6	95	93.02	45.71	77.07
VERB:INFL	2	0	6	100.00	25.00	62.50
VERB:SVA	62	4	73	93.94	45.93	77.69
VERB:TENSE	35	7	198	83.33	15.02	43.64
WO	29	6	86	82.86	25.22	56.86

OPERATION + ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
M:ADJ	0	0	5	100.00	0.00	0.00
M:ADV	1	0	13	100.00	7.14	27.78
M:CONJ	1	0	12	100.00	7.69	29.41
M:CONTR	0	1	0	0.00	100.00	0.00
M:DET	28	5	206	84.85	11.97	38.25
M:NOUN	1	0	17	100.00	5.56	22.73
M:NOUN:POSS	3	0	12	100.00	20.00	55.56
M:OTHER	3	1	59	75.00	4.84	19.23
M:PART	0	0	4	100.00	0.00	0.00
M:PREP	7	2	83	77.78	7.78	27.78
M:PRON	13	1	55	92.86	19.12	52.42
M:PUNCT	47	6	292	88.68	13.86	42.65
M:VERB	3	1	44	75.00	6.38	23.81
M:VERB:FORM	8	1	5	88.89	61.54	81.63
M:VERB:TENSE	3	0	37	100.00	7.50	28.85
R:ADJ	4	1	26	80.00	13.33	40.00
R:ADJ:FORM	2	1	9	66.67	18.18	43.48
R:ADV	2	1	15	66.67	11.76	34.48
R:CONJ	0	0	5	100.00	0.00	0.00
R:CONTR	1	0	4	100.00	20.00	55.56
R:DET	9	0	87	100.00	9.38	34.09
R:MORPH	34	2	100	94.44	25.37	61.15
R:NOUN	11	4	79	73.33	12.22	36.67
R:NOUN:INFL	4	3	14	57.14	22.22	43.48
R:NOUN:NUM	47	8	169	85.45	21.76	53.90
R:NOUN:POSS	2	0	24	100.00	7.69	29.41
R:ORTH	192	110	226	63.58	45.93	59.04
R:OTHER	93	12	697	88.57	11.77	38.43
R:PART	3	0	4	100.00	42.86	78.95
R:PREP	24	4	187	85.71	11.37	37.15
R:PRON	3	0	37	100.00	7.50	28.85
R:PUNCT	6	10	69	37.50	8.00	21.58
R:SPELL	73	3	237	96.05	23.55	59.45
R:VERB	14	2	146	87.50	8.75	31.25
R:VERB:FORM	69	5	87	93.24	44.23	76.33
R:VERB:INFL	2	0	6	100.00	25.00	62.50
R:VERB:SVA	62	4	73	93.94	45.93	77.69
R:VERB:TENSE	27	3	143	90.00	15.88	46.55
R:WO	29	6	86	82.86	25.22	56.86
U:ADJ	2	0	6	100.00	25.00	62.50
U:ADV	2	1	20	66.67	9.09	29.41
U:CONJ	0	0	1	100.00	0.00	0.00
U:CONTR	2	0	4	100.00	33.33	71.43
U:DET	46	7	139	86.79	24.86	57.93
U:NOUN	3	4	17	42.86	15.00	31.25
U:NOUN:POSS	2	0	12	100.00	14.29	45.45
U:OTHER	13	18	78	41.94	14.29	30.23
U:PART	0	1	4	0.00	0.00	0.00
U:PREP	8	4	60	66.67	11.76	34.48
U:PRON	5	0	17	100.00	22.73	59.52
U:PUNCT	11	1	36	91.67	23.40	57.89
U:VERB	6	4	25	60.00	19.35	42.25
U:VERB:FORM	3	0	3	100.00	50.00	83.33
U:VERB:TENSE	5	4	18	55.56	21.74	42.37
UNK	12	0	86	100.00	12.24	41.10

OVERALL
	TP	FP	FN	P	R	F_0.5
Span-level correction	362	137	1503	72.55	19.41	46.88
Span-level detection	412	88	1524	82.40	21.28	52.34
Token-level detection	488	68	1800	87.77	21.33	54.08

SPAN-LEVEL CORRECTION
OPERATION LEVEL
	TP	FP	FN	P	R	F_0.5
M	38	13	336	74.51	10.16	32.87
R	287	104	1023	73.40	21.91	49.93
U	37	20	144	64.91	20.44	45.23

ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
ADJ	1	2	8	33.33	11.11	23.81
ADJ:FORM	1	1	3	50.00	25.00	41.67
ADV	2	0	18	100.00	10.00	35.71
CONJ	0	0	7	100.00	0.00	0.00
CONTR	3	1	1	75.00	75.00	75.00
DET	37	4	168	90.24	18.05	50.14
MORPH	15	1	59	93.75	20.27	54.35
NOUN	2	10	38	16.67	5.00	11.36
NOUN:INFL	3	0	3	100.00	50.00	83.33
NOUN:NUM	29	9	81	76.32	26.36	55.34
NOUN:POSS	1	0	8	100.00	11.11	38.46
ORTH	95	42	97	69.34	49.48	64.19
OTHER	10	17	347	37.04	2.80	10.75
PART	3	1	5	75.00	37.50	62.50
PREP	10	6	125	62.50	7.41	25.13
PRON	6	3	53	66.67	10.17	31.58
PUNCT	15	12	153	55.56	8.93	27.17
SPELL	32	7	73	82.05	30.48	61.30
VERB	6	4	75	60.00	7.41	24.79
VERB:FORM	46	5	41	90.20	52.87	79.04
VERB:INFL	1	0	3	100.00	25.00	62.50
VERB:SVA	31	5	37	86.11	45.59	73.11
VERB:TENSE	10	6	82	62.50	10.87	32.05
WO	3	1	18	75.00	14.29	40.54

OPERATION + ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
M:ADJ	0	1	1	0.00	0.00	0.00
M:ADV	0	0	4	100.00	0.00	0.00
M:CONJ	0	0	2	100.00	0.00	0.00
M:CONTR	1	1	0	50.00	100.00	55.56
M:DET	11	1	76	91.67	12.64	40.74
M:NOUN	0	0	5	100.00	0.00	0.00
M:NOUN:POSS	0	0	3	100.00	0.00	0.00
M:OTHER	0	2	27	0.00	0.00	0.00
M:PART	0	0	1	100.00	0.00	0.00
M:PREP	1	1	34	50.00	2.86	11.63
M:PRON	3	1	28	75.00	9.68	31.91
M:PUNCT	11	5	121	68.75	8.33	28.06
M:VERB	2	0	14	100.00	12.50	41.67
M:VERB:FORM	9	1	3	90.00	75.00	86.54
M:VERB:TENSE	0	0	17	100.00	0.00	0.00
R:ADJ	0	0	6	100.00	0.00	0.00
R:ADJ:FORM	1	1	3	50.00	25.00	41.67
R:ADV	1	0	7	100.00	12.50	41.67
R:CONJ	0	0	2	100.00	0.00	0.00
R:CONTR	1	0	0	100.00	100.00	100.00
R:DET	2	1	33	66.67	5.71	21.28
R:MORPH	15	1	59	93.75	20.27	54.35
R:NOUN	1	7	27	12.50	3.57	8.33
R:NOUN:INFL	3	0	3	100.00	50.00	83.33
R:NOUN:NUM	29	9	81	76.32	26.36	55.34
R:NOUN:POSS	0	0	4	100.00	0.00	0.00
R:ORTH	95	42	97	69.34	49.48	64.19
R:OTHER	10	12	312	45.45	3.11	12.20
R:PART	3	0	2	100.00	60.00	88.24
R:PREP	7	2	67	77.78	9.46	31.82
R:PRON	1	1	17	50.00	5.56	19.23
R:PUNCT	3	6	18	33.33	14.29	26.32
R:SPELL	32	7	73	82.05	30.48	61.30
R:VERB	3	1	54	75.00	5.26	20.55
R:VERB:FORM	37	3	37	92.50	50.00	79.06
R:VERB:INFL	1	0	3	100.00	25.00	62.50
R:VERB:SVA	31	5	37	86.11	45.59	73.11
R:VERB:TENSE	8	5	63	61.54	11.27	32.52
R:WO	3	1	18	75.00	14.29	40.54
U:ADJ	1	1	1	50.00	50.00	50.00
U:ADV	1	0	7	100.00	12.50	41.67
U:CONJ	0	0	3	100.00	0.00	0.00
U:CONTR	1	0	1	100.00	50.00	83.33
U:DET	24	2	59	92.31	28.92	64.17
U:NOUN	1	3	6	25.00	14.29	21.74
U:NOUN:POSS	1	0	1	100.00	50.00	83.33
U:OTHER	0	3	8	0.00	0.00	0.00
U:PART	0	1	2	0.00	0.00	0.00
U:PREP	2	3	24	40.00	7.69	21.74
U:PRON	2	1	8	66.67	20.00	45.45
U:PUNCT	1	1	14	50.00	6.67	21.74
U:VERB	1	3	7	25.00	12.50	20.83
U:VERB:FORM	0	1	1	0.00	0.00	0.00
U:VERB:TENSE	2	1	2	66.67	50.00	62.50

SPAN-LEVEL DETECTION
OPERATION LEVEL
	TP	FP	FN	P	R	F_0.5
M	42	10	329	80.77	11.32	36.27
R	325	66	1003	83.12	24.47	56.19
U	39	12	139	76.47	21.91	51.05
UNK	6	0	53	100.00	10.17	36.14

ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
ADJ	2	1	10	66.67	16.67	41.67
ADJ:FORM	0	1	3	0.00	0.00	0.00
ADV	3	0	19	100.00	13.64	44.12
CONJ	0	0	5	100.00	0.00	0.00
CONTR	2	1	1	66.67	66.67	66.67
DET	40	3	161	93.02	19.90	53.62
MORPH	18	0	60	100.00	23.08	60.00
NOUN	3	8	41	27.27	6.82	17.05
NOUN:INFL	3	0	3	100.00	50.00	83.33
NOUN:NUM	25	7	80	78.12	23.81	53.65
NOUN:POSS	1	0	9	100.00	10.00	35.71
ORTH	96	30	90	76.19	51.61	69.57
OTHER	33	10	342	76.74	8.80	30.16
PART	4	1	4	80.00	50.00	71.43
PREP	12	4	127	75.00	8.63	29.56
PRON	8	1	50	88.89	13.79	42.55
PUNCT	18	10	148	64.29	10.84	32.37
SPELL	38	1	69	97.44	35.51	72.24
UNK	6	0	53	100.00	10.17	36.14
VERB	11	2	78	84.62	12.36	39.01
VERB:FORM	43	3	40	93.48	51.81	80.52
VERB:INFL	1	0	3	100.00	25.00	62.50
VERB:SVA	31	2	34	93.94	47.69	78.68
VERB:TENSE	11	2	78	84.62	12.36	39.01
WO	3	1	16	75.00	15.79	42.86

OPERATION + ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
M:ADJ	0	1	1	0.00	0.00	0.00
M:ADV	0	0	4	100.00	0.00	0.00
M:CONJ	0	0	2	100.00	0.00	0.00
M:CONTR	0	1	0	0.00	100.00	0.00
M:DET	11	1	75	91.67	12.79	41.04
M:NOUN	0	0	5	100.00	0.00	0.00
M:NOUN:POSS	0	0	3	100.00	0.00	0.00
M:OTHER	0	1	27	0.00	0.00	0.00
M:PREP	1	1	35	50.00	2.78	11.36
M:PRON	4	1	26	80.00	13.33	40.00
M:PUNCT	14	3	119	82.35	10.53	34.83
M:VERB	3	0	13	100.00	18.75	53.57
M:VERB:FORM	9	1	2	90.00	81.82	88.24
M:VERB:TENSE	0	0	17	100.00	0.00	0.00
R:ADJ	1	0	7	100.00	12.50	41.67
R:ADJ:FORM	0	1	3	0.00	0.00	0.00
R:ADV	2	0	9	100.00	18.18	52.63
R:CONJ	0	0	1	100.00	0.00	0.00
R:CONTR	1	0	0	100.00	100.00	100.00
R:DET	4	1	31	80.00	11.43	36.36
R:MORPH	18	0	60	100.00	23.08	60.00
R:NOUN	1	5	27	16.67	3.57	9.62
R:NOUN:INFL	3	0	3	100.00	50.00	83.33
R:NOUN:NUM	25	7	80	78.12	23.81	53.65
R:NOUN:POSS	0	0	3	100.00	0.00	0.00
R:ORTH	96	30	90	76.19	51.61	69.57
R:OTHER	33	6	307	84.62	9.71	33.27
R:PART	4	0	2	100.00	66.67	90.91
R:PREP	9	2	69	81.82	11.54	36.89
R:PRON	2	0	18	100.00	10.00	35.71
R:PUNCT	3	6	17	33.33	15.00	26.79
R:SPELL	38	1	69	97.44	35.51	72.24
R:VERB	7	0	58	100.00	10.77	37.63
R:VERB:FORM	34	2	37	94.44	47.89	79.07
R:VERB:INFL	1	0	3	100.00	25.00	62.50
R:VERB:SVA	31	2	34	93.94	47.69	78.68
R:VERB:TENSE	9	2	59	81.82	13.24	40.18
R:WO	3	1	16	75.00	15.79	42.86
U:ADJ	1	0	2	100.00	33.33	71.43
U:ADV	1	0	6	100.00	14.29	45.45
U:CONJ	0	0	2	100.00	0.00	0.00
U:CONTR	1	0	1	100.00	50.00	83.33
U:DET	25	1	55	96.15	31.25	67.93
U:NOUN	2	3	9	40.00	18.18	32.26
U:NOUN:POSS	1	0	3	100.00	25.00	62.50
U:OTHER	0	3	8	0.00	0.00	0.00
U:PART	0	1	2	0.00	0.00	0.00
U:PREP	2	1	23	66.67	8.00	27.03
U:PRON	2	0	6	100.00	25.00	62.50
U:PUNCT	1	1	12	50.00	7.69	23.81
U:VERB	1	2	7	33.33	12.50	25.00
U:VERB:FORM	0	0	1	100.00	0.00	0.00
U:VERB:TENSE	2	0	2	100.00	50.00	83.33
UNK	6	0	53	100.00	10.17	36.14

TOKEN-LEVEL DETECTION
OPERATION LEVEL
	TP	FP	FN	P	R	F_0.5
M	65	7	378	90.28	14.67	44.46
R	366	51	1170	87.77	23.83	57.12
U	50	10	194	83.33	20.49	51.65
UNK	7	0	58	100.00	10.77	37.63

ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
ADJ	3	0	13	100.00	18.75	53.57
ADJ:FORM	0	1	5	0.00	0.00	0.00
ADV	2	0	23	100.00	8.00	30.30
CONJ	1	0	6	100.00	14.29	45.45
CONTR	2	1	1	66.67	66.67	66.67
DET	45	2	198	95.74	18.52	52.20
MORPH	18	0	61	100.00	22.78	59.60
NOUN	8	5	57	61.54	12.31	34.19
NOUN:INFL	3	0	3	100.00	50.00	83.33
NOUN:NUM	26	3	84	89.66	23.64	57.52
NOUN:POSS	2	0	13	100.00	13.33	43.48
ORTH	102	27	102	79.07	50.00	70.83
OTHER	56	10	415	84.85	11.89	38.10
PART	3	1	4	75.00	42.86	65.22
PREP	15	3	147	83.33	9.26	32.05
PRON	14	1	57	93.33	19.72	53.44
PUNCT	30	7	172	81.08	14.85	42.86
SPELL	37	0	74	100.00	33.33	71.43
UNK	7	0	58	100.00	10.77	37.63
VERB	13	2	113	86.67	10.32	34.95
VERB:FORM	43	1	46	97.73	48.31	81.13
VERB:INFL	1	0	2	100.00	33.33	71.43
VERB:SVA	34	2	36	94.44	48.57	79.44
VERB:TENSE	16	2	89	88.89	15.24	45.20
WO	7	0	21	100.00	25.00	62.50

OPERATION + ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
M:ADJ	0	0	1	100.00	0.00	0.00
M:ADV	0	0	4	100.00	0.00	0.00
M:CONJ	1	0	4	100.00	20.00	55.56
M:CONTR	0	1	0	0.00	100.00	0.00
M:DET	14	1	86	93.33	14.00	43.75
M:NOUN	1	0	7	100.00	12.50	41.67
M:NOUN:POSS	1	0	3	100.00	25.00	62.50
M:OTHER	1	0	32	100.00	3.03	13.51
M:PART	0	0	1	100.00	0.00	0.00
M:PREP	5	1	40	83.33	11.11	36.23
M:PRON	8	1	27	88.89	22.86	56.34
M:PUNCT	22	3	131	88.00	14.38	43.48
M:VERB	2	0	24	100.00	7.69	29.41
M:VERB:FORM	7	0	3	100.00	70.00	92.11
M:VERB:TENSE	3	0	15	100.00	16.67	50.00
R:ADJ	1	0	11	100.00	8.33	31.25
R:ADJ:FORM	0	1	5	0.00	0.00	0.00
R:ADV	1	0	11	100.00	8.33	31.25
R:CONJ	0	0	1	100.00	0.00	0.00
R:CONTR	1	0	0	100.00	100.00	100.00
R:DET	5	0	37	100.00	11.90	40.32
R:MORPH	18	0	61	100.00	22.78	59.60
R:NOUN	5	3	39	62.50	11.36	32.89
R:NOUN:INFL	3	0	3	100.00	50.00	83.33
R:NOUN:NUM	26	3	84	89.66	23.64	57.52
R:NOUN:POSS	0	0	5	100.00	0.00	0.00
R:ORTH	102	27	102	79.07	50.00	70.83
R:OTHER	53	6	357	89.83	12.93	41.02
R:PART	3	0	1	100.00	75.00	93.75
R:PREP	8	2	80	80.00	9.09	31.25
R:PRON	2	0	22	100.00	8.33	31.25
R:PUNCT	3	4	30	42.86	9.09	24.59
R:SPELL	37	0	74	100.00	33.33	71.43
R:VERB	10	0	75	100.00	11.76	40.00
R:VERB:FORM	36	1	42	97.30	46.15	79.65
R:VERB:INFL	1	0	2	100.00	33.33	71.43
R:VERB:SVA	34	2	36	94.44	48.57	79.44
R:VERB:TENSE	10	2	71	83.33	12.35	38.76
R:WO	7	0	21	100.00	25.00	62.50
U:ADJ	2	0	1	100.00	66.67	90.91
U:ADV	1	0	8	100.00	11.11	38.46
U:CONJ	0	0	1	100.00	0.00	0.00
U:CONTR	1	0	1	100.00	50.00	83.33
U:DET	26	1	75	96.30	25.74	62.20
U:NOUN	2	2	11	50.00	15.38	34.48
U:NOUN:POSS	1	0	5	100.00	16.67	50.00
U:OTHER	2	4	26	33.33	7.14	19.23
U:PART	0	1	2	0.00	0.00	0.00
U:PREP	2	0	27	100.00	6.90	27.03
U:PRON	4	0	8	100.00	33.33	71.43
U:PUNCT	5	0	11	100.00	31.25	69.44
U:VERB	1	2	14	33.33	6.67	18.52
U:VERB:FORM	0	0	1	100.00	0.00	0.00
U:VERB:TENSE	3	0	3	100.00	50.00	83.33
UNK	7	0	58	100.00	10.77	37.63

OVERALL
	TP	FP	FN	P	R	F_0.5
Span-level correction	202	114	1045	63.92	16.20	40.22
Span-level detection	226	90	1057	71.52	17.61	44.37
Token-level detection	268	84	1298	76.14	17.11	45.06

SPAN-LEVEL CORRECTION
OPERATION LEVEL
	TP	FP	FN	P	R	F_0.5
M	26	11	258	70.27	9.15	30.09
R	151	88	682	63.18	18.13	42.20
U	25	15	105	62.50	19.23	43.10

ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
ADJ	0	0	6	100.00	0.00	0.00
ADJ:FORM	2	1	0	66.67	100.00	71.43
ADV	3	3	12	50.00	20.00	38.46
CONJ	0	0	7	100.00	0.00	0.00
CONTR	0	0	2	100.00	0.00	0.00
DET	21	8	139	72.41	13.13	38.04
MORPH	10	4	23	71.43	30.30	56.18
NOUN	2	4	31	33.33	6.06	17.54
NOUN:INFL	0	3	0	0.00	100.00	0.00
NOUN:NUM	12	1	43	92.31	21.82	56.07
NOUN:POSS	2	0	11	100.00	15.38	47.62
ORTH	28	45	51	38.36	35.44	37.74
OTHER	4	10	215	28.57	1.83	7.27
PART	1	0	5	100.00	16.67	50.00
PREP	16	6	109	72.73	12.80	37.56
PRON	4	2	30	66.67	11.76	34.48
PUNCT	13	8	111	61.90	10.48	31.25
SPELL	22	4	45	84.62	32.84	64.33
VERB	1	1	61	50.00	1.61	7.14
VERB:FORM	27	5	31	84.38	46.55	72.58
VERB:SVA	21	3	20	87.50	51.22	76.64
VERB:TENSE	12	3	61	80.00	16.44	45.11
WO	1	3	32	25.00	3.03	10.20

OPERATION + ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
M:ADJ	0	0	1	100.00	0.00	0.00
M:ADV	1	1	5	50.00	16.67	35.71
M:CONJ	0	0	5	100.00	0.00	0.00
M:DET	10	4	75	71.43	11.76	35.46
M:NOUN	0	0	3	100.00	0.00	0.00
M:NOUN:POSS	1	0	3	100.00	25.00	62.50
M:OTHER	0	1	14	0.00	0.00	0.00
M:PART	0	0	2	100.00	0.00	0.00
M:PREP	2	1	23	66.67	8.00	27.03
M:PRON	3	1	17	75.00	15.00	41.67
M:PUNCT	9	2	86	81.82	9.47	32.37
M:VERB	0	1	9	0.00	0.00	0.00
M:VERB:FORM	0	0	1	100.00	0.00	0.00
M:VERB:TENSE	0	0	14	100.00	0.00	0.00
R:ADJ	0	0	5	100.00	0.00	0.00
R:ADJ:FORM	2	1	0	66.67	100.00	71.43
R:ADV	1	2	3	33.33	25.00	31.25
R:CONJ	0	0	2	100.00	0.00	0.00
R:CONTR	0	0	1	100.00	0.00	0.00
R:DET	1	0	35	100.00	2.78	12.50
R:MORPH	10	4	23	71.43	30.30	56.18
R:NOUN	2	2	25	50.00	7.41	23.26
R:NOUN:INFL	0	3	0	0.00	100.00	0.00
R:NOUN:NUM	12	1	43	92.31	21.82	56.07
R:NOUN:POSS	0	0	3	100.00	0.00	0.00
R:ORTH	28	45	51	38.36	35.44	37.74
R:OTHER	2	5	191	28.57	1.04	4.52
R:PART	1	0	2	100.00	33.33	71.43
R:PREP	10	3	62	76.92	13.89	40.32
R:PRON	0	1	7	0.00	0.00	0.00
R:PUNCT	2	5	17	28.57	10.53	21.28
R:SPELL	22	4	45	84.62	32.84	64.33
R:VERB	0	0	45	100.00	0.00	0.00
R:VERB:FORM	25	5	28	83.33	47.17	72.25
R:VERB:SVA	21	3	20	87.50	51.22	76.64
R:VERB:TENSE	11	1	42	91.67	20.75	54.46
R:WO	1	3	32	25.00	3.03	10.20
U:ADV	1	0	4	100.00	20.00	55.56
U:CONTR	0	0	1	100.00	0.00	0.00
U:DET	10	4	29	71.43	25.64	52.63
U:NOUN	0	2	3	0.00	0.00	0.00
U:NOUN:POSS	1	0	5	100.00	16.67	50.00
U:OTHER	2	4	10	33.33	16.67	27.78
U:PART	0	0	1	100.00	0.00	0.00
U:PREP	4	2	24	66.67	14.29	38.46
U:PRON	1	0	6	100.00	14.29	45.45
U:PUNCT	2	1	8	66.67	20.00	45.45
U:VERB	1	0	7	100.00	12.50	41.67
U:VERB:FORM	2	0	2	100.00	50.00	83.33
U:VERB:TENSE	1	2	5	33.33	16.67	27.78

SPAN-LEVEL DETECTION
OPERATION LEVEL
	TP	FP	FN	P	R	F_0.5
M	27	10	264	72.97	9.28	30.75
R	169	66	671	71.91	20.12	47.47
U	30	14	105	68.18	22.22	48.23
UNK	0	0	17	100.00	0.00	0.00

ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
ADJ	2	0	7	100.00	22.22	58.82
ADJ:FORM	2	0	0	100.00	100.00	100.00
ADV	4	2	12	66.67	25.00	50.00
CONJ	0	0	8	100.00	0.00	0.00
CONTR	0	0	2	100.00	0.00	0.00
DET	23	7	139	76.67	14.20	40.78
MORPH	9	3	22	75.00	29.03	56.96
NOUN	3	3	29	50.00	9.38	26.79
NOUN:INFL	0	3	0	0.00	100.00	0.00
NOUN:NUM	12	1	43	92.31	21.82	56.07
NOUN:POSS	2	0	11	100.00	15.38	47.62
ORTH	30	40	50	42.86	37.50	41.67
OTHER	11	8	224	57.89	4.68	17.68
PART	1	0	5	100.00	16.67	50.00
PREP	16	6	112	72.73	12.50	37.04
PRON	5	1	29	83.33	14.71	43.10
PUNCT	14	7	109	66.67	11.38	33.82
SPELL	23	1	43	95.83	34.85	70.99
UNK	0	0	17	100.00	0.00	0.00
VERB	4	0	59	100.00	6.35	25.32
VERB:FORM	29	1	28	96.67	50.88	81.92
VERB:SVA	22	1	21	95.65	51.16	81.48
VERB:TENSE	13	3	56	81.25	18.84	48.87
WO	1	3	31	25.00	3.12	10.42

OPERATION + ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
M:ADJ	0	0	1	100.00	0.00	0.00
M:ADV	1	1	5	50.00	16.67	35.71
M:CONJ	0	0	6	100.00	0.00	0.00
M:DET	10	4	75	71.43	11.76	35.46
M:NOUN	0	0	3	100.00	0.00	0.00
M:NOUN:POSS	1	0	3	100.00	25.00	62.50
M:OTHER	0	1	16	0.00	0.00	0.00
M:PART	0	0	2	100.00	0.00	0.00
M:PREP	2	1	25	66.67	7.41	25.64
M:PRON	3	1	17	75.00	15.00	41.67
M:PUNCT	9	2	87	81.82	9.38	32.14
M:VERB	1	0	9	100.00	10.00	35.71
M:VERB:FORM	0	0	1	100.00	0.00	0.00
M:VERB:TENSE	0	0	14	100.00	0.00	0.00
R:ADJ	2	0	5	100.00	28.57	66.67
R:ADJ:FORM	2	0	0	100.00	100.00	100.00
R:ADV	1	1	3	50.00	25.00	41.67
R:CONJ	0	0	2	100.00	0.00	0.00
R:CONTR	0	0	1	100.00	0.00	0.00
R:DET	2	0	35	100.00	5.41	22.22
R:MORPH	9	3	22	75.00	29.03	56.96
R:NOUN	3	1	23	75.00	11.54	35.71
R:NOUN:INFL	0	3	0	0.00	100.00	0.00
R:NOUN:NUM	12	1	43	92.31	21.82	56.07
R:NOUN:POSS	0	0	3	100.00	0.00	0.00
R:ORTH	30	40	50	42.86	37.50	41.67
R:OTHER	9	3	198	75.00	4.35	17.65
R:PART	1	0	2	100.00	33.33	71.43
R:PREP	10	3	63	76.92	13.70	40.00
R:PRON	0	0	7	100.00	0.00	0.00
R:PUNCT	2	4	14	33.33	12.50	25.00
R:SPELL	23	1	43	95.83	34.85	70.99
R:VERB	1	0	44	100.00	2.22	10.20
R:VERB:FORM	27	1	25	96.43	51.92	82.32
R:VERB:SVA	22	1	21	95.65	51.16	81.48
R:VERB:TENSE	12	1	36	92.31	25.00	60.00
R:WO	1	3	31	25.00	3.12	10.42
U:ADJ	0	0	1	100.00	0.00	0.00
U:ADV	2	0	4	100.00	33.33	71.43
U:CONTR	0	0	1	100.00	0.00	0.00
U:DET	11	3	29	78.57	27.50	57.29
U:NOUN	0	2	3	0.00	0.00	0.00
U:NOUN:POSS	1	0	5	100.00	16.67	50.00
U:OTHER	2	4	10	33.33	16.67	27.78
U:PART	0	0	1	100.00	0.00	0.00
U:PREP	4	2	24	66.67	14.29	38.46
U:PRON	2	0	5	100.00	28.57	66.67
U:PUNCT	3	1	8	75.00	27.27	55.56
U:VERB	2	0	6	100.00	25.00	62.50
U:VERB:FORM	2	0	2	100.00	50.00	83.33
U:VERB:TENSE	1	2	6	33.33	14.29	26.32
UNK	0	0	17	100.00	0.00	0.00

TOKEN-LEVEL DETECTION
OPERATION LEVEL
	TP	FP	FN	P	R	F_0.5
M	31	4	302	88.57	9.31	32.77
R	200	61	806	76.63	19.88	48.78
U	36	19	169	65.45	17.56	42.35
UNK	1	0	21	100.00	4.55	19.23

ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
ADJ	2	0	12	100.00	14.29	45.45
ADJ:FORM	2	0	1	100.00	66.67	90.91
ADV	3	1	19	75.00	13.64	39.47
CONJ	0	0	9	100.00	0.00	0.00
CONTR	0	0	2	100.00	0.00	0.00
DET	24	5	156	82.76	13.33	40.54
MORPH	9	1	23	90.00	28.12	62.50
NOUN	5	2	31	71.43	13.89	39.06
NOUN:INFL	0	3	0	0.00	100.00	0.00
NOUN:NUM	13	1	46	92.86	22.03	56.52
NOUN:POSS	2	0	12	100.00	14.29	45.45
ORTH	34	38	60	47.22	36.17	44.50
OTHER	30	15	300	66.67	9.09	29.41
PART	0	0	4	100.00	0.00	0.00
PREP	18	4	126	81.82	12.50	38.79
PRON	4	0	38	100.00	9.52	34.48
PUNCT	17	7	117	70.83	12.69	36.96
SPELL	22	1	48	95.65	31.43	67.90
UNK	1	0	21	100.00	4.55	19.23
VERB	5	0	82	100.00	5.75	23.36
VERB:FORM	28	1	35	96.55	44.44	78.21
VERB:SVA	22	1	25	95.65	46.81	79.14
VERB:TENSE	15	3	73	83.33	17.05	46.88
WO	12	1	58	92.31	17.14	49.18

OPERATION + ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
M:ADJ	0	0	2	100.00	0.00	0.00
M:ADV	1	0	7	100.00	12.50	41.67
M:CONJ	0	0	6	100.00	0.00	0.00
M:DET	11	2	85	84.62	11.46	37.16
M:NOUN	0	0	4	100.00	0.00	0.00
M:NOUN:POSS	1	0	4	100.00	20.00	55.56
M:OTHER	0	1	20	0.00	0.00	0.00
M:PART	0	0	2	100.00	0.00	0.00
M:PREP	2	1	28	66.67	6.67	23.81
M:PRON	3	0	21	100.00	12.50	41.67
M:PUNCT	12	0	90	100.00	11.76	40.00
M:VERB	1	0	15	100.00	6.25	25.00
M:VERB:FORM	0	0	1	100.00	0.00	0.00
M:VERB:TENSE	0	0	17	100.00	0.00	0.00
R:ADJ	2	0	9	100.00	18.18	52.63
R:ADJ:FORM	2	0	1	100.00	66.67	90.91
R:ADV	1	1	2	50.00	33.33	45.45
R:CONJ	0	0	3	100.00	0.00	0.00
R:CONTR	0	0	1	100.00	0.00	0.00
R:DET	2	0	38	100.00	5.00	20.83
R:MORPH	9	1	23	90.00	28.12	62.50
R:NOUN	5	0	23	100.00	17.86	52.08
R:NOUN:INFL	0	3	0	0.00	100.00	0.00
R:NOUN:NUM	13	1	46	92.86	22.03	56.52
R:NOUN:POSS	0	0	2	100.00	0.00	0.00
R:ORTH	34	38	60	47.22	36.17	44.50
R:OTHER	22	4	233	84.62	8.63	30.64
R:PART	0	0	1	100.00	0.00	0.00
R:PREP	10	2	73	83.33	12.05	38.17
R:PRON	0	0	10	100.00	0.00	0.00
R:PUNCT	2	6	15	25.00	11.76	20.41
R:SPELL	22	1	48	95.65	31.43	67.90
R:VERB	2	0	57	100.00	3.39	14.93
R:VERB:FORM	26	1	32	96.30	44.83	78.31
R:VERB:SVA	22	1	25	95.65	46.81	79.14
R:VERB:TENSE	14	1	46	93.33	23.33	58.33
R:WO	12	1	58	92.31	17.14	49.18
U:ADJ	0	0	1	100.00	0.00	0.00
U:ADV	1	0	10	100.00	9.09	33.33
U:CONTR	0	0	1	100.00	0.00	0.00
U:DET	11	3	33	78.57	25.00	55.00
U:NOUN	0	2	4	0.00	0.00	0.00
U:NOUN:POSS	1	0	6	100.00	14.29	45.45
U:OTHER	8	10	47	44.44	14.55	31.50
U:PART	0	0	1	100.00	0.00	0.00
U:PREP	6	1	25	85.71	19.35	50.85
U:PRON	1	0	7	100.00	12.50	41.67
U:PUNCT	3	1	12	75.00	20.00	48.39
U:VERB	2	0	10	100.00	16.67	50.00
U:VERB:FORM	2	0	2	100.00	50.00	83.33
U:VERB:TENSE	1	2	10	33.33	9.09	21.74
UNK	1	0	21	100.00	4.55	19.23

OVERALL
	TP	FP	FN	P	R	F_0.5
Span-level correction	89	52	394	63.12	18.43	42.50
Span-level detection	100	41	401	70.92	19.96	46.95
Token-level detection	123	32	452	79.35	21.39	51.46

SPAN-LEVEL CORRECTION
OPERATION LEVEL
	TP	FP	FN	P	R	F_0.5
M	11	2	98	84.62	10.09	34.16
R	65	46	251	58.56	20.57	42.76
U	13	4	45	76.47	22.41	51.59

ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
ADJ	0	0	8	100.00	0.00	0.00
ADJ:FORM	0	0	2	100.00	0.00	0.00
ADV	0	1	4	0.00	0.00	0.00
CONJ	0	0	4	100.00	0.00	0.00
CONTR	1	0	3	100.00	25.00	62.50
DET	10	2	65	83.33	13.33	40.65
MORPH	4	1	12	80.00	25.00	55.56
NOUN	0	3	11	0.00	0.00	0.00
NOUN:INFL	2	0	0	100.00	100.00	100.00
NOUN:NUM	5	3	20	62.50	20.00	43.86
NOUN:POSS	0	0	6	100.00	0.00	0.00
ORTH	28	23	19	54.90	59.57	55.78
OTHER	2	4	54	33.33	3.57	12.50
PART	0	0	3	100.00	0.00	0.00
PREP	5	0	46	100.00	9.80	35.21
PRON	2	0	7	100.00	22.22	58.82
PUNCT	12	1	40	92.31	23.08	57.69
SPELL	4	5	32	44.44	11.11	27.78
VERB	0	2	9	0.00	0.00	0.00
VERB:FORM	5	3	9	62.50	35.71	54.35
VERB:SVA	4	2	5	66.67	44.44	60.61
VERB:TENSE	2	1	25	66.67	7.41	25.64
WO	3	1	10	75.00	23.08	51.72

OPERATION + ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
M:ADJ	0	0	1	100.00	0.00	0.00
M:ADV	0	0	1	100.00	0.00	0.00
M:CONJ	0	0	2	100.00	0.00	0.00
M:DET	1	1	33	50.00	2.94	11.90
M:NOUN	0	0	2	100.00	0.00	0.00
M:OTHER	0	0	5	100.00	0.00	0.00
M:PREP	0	0	15	100.00	0.00	0.00
M:PRON	1	0	4	100.00	20.00	55.56
M:PUNCT	8	1	29	88.89	21.62	54.79
M:VERB	0	0	3	100.00	0.00	0.00
M:VERB:FORM	1	0	1	100.00	50.00	83.33
M:VERB:TENSE	0	0	2	100.00	0.00	0.00
R:ADJ	0	0	6	100.00	0.00	0.00
R:ADJ:FORM	0	0	2	100.00	0.00	0.00
R:ADV	0	0	1	100.00	0.00	0.00
R:CONJ	0	0	2	100.00	0.00	0.00
R:CONTR	0	0	1	100.00	0.00	0.00
R:DET	2	0	8	100.00	20.00	55.56
R:MORPH	4	1	12	80.00	25.00	55.56
R:NOUN	0	3	9	0.00	0.00	0.00
R:NOUN:INFL	2	0	0	100.00	100.00	100.00
R:NOUN:NUM	5	3	20	62.50	20.00	43.86
R:NOUN:POSS	0	0	5	100.00	0.00	0.00
R:ORTH	28	23	19	54.90	59.57	55.78
R:OTHER	2	3	48	40.00	4.00	14.29
R:PART	0	0	2	100.00	0.00	0.00
R:PREP	5	0	24	100.00	17.24	51.02
R:PRON	1	0	2	100.00	33.33	71.43
R:PUNCT	1	0	9	100.00	10.00	35.71
R:SPELL	4	5	32	44.44	11.11	27.78
R:VERB	0	2	6	0.00	0.00	0.00
R:VERB:FORM	3	3	8	50.00	27.27	42.86
R:VERB:SVA	4	2	5	66.67	44.44	60.61
R:VERB:TENSE	1	0	20	100.00	4.76	20.00
R:WO	3	1	10	75.00	23.08	51.72
U:ADJ	0	0	1	100.00	0.00	0.00
U:ADV	0	1	2	0.00	0.00	0.00
U:CONTR	1	0	2	100.00	33.33	71.43
U:DET	7	1	24	87.50	22.58	55.56
U:NOUN:POSS	0	0	1	100.00	0.00	0.00
U:OTHER	0	1	1	0.00	0.00	0.00
U:PART	0	0	1	100.00	0.00	0.00
U:PREP	0	0	7	100.00	0.00	0.00
U:PRON	0	0	1	100.00	0.00	0.00
U:PUNCT	3	0	2	100.00	60.00	88.24
U:VERB:FORM	1	0	0	100.00	100.00	100.00
U:VERB:TENSE	1	1	3	50.00	25.00	41.67

SPAN-LEVEL DETECTION
OPERATION LEVEL
	TP	FP	FN	P	R	F_0.5
M	11	2	101	84.62	9.82	33.54
R	75	35	249	68.18	23.15	49.08
U	13	4	47	76.47	21.67	50.78
UNK	1	0	4	100.00	20.00	55.56

ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
ADJ	0	0	7	100.00	0.00	0.00
ADJ:FORM	0	0	2	100.00	0.00	0.00
ADV	0	1	4	0.00	0.00	0.00
CONJ	0	0	4	100.00	0.00	0.00
CONTR	1	0	3	100.00	25.00	62.50
DET	11	2	67	84.62	14.10	42.31
MORPH	4	1	12	80.00	25.00	55.56
NOUN	0	2	13	0.00	0.00	0.00
NOUN:INFL	2	0	0	100.00	100.00	100.00
NOUN:NUM	6	3	21	66.67	22.22	47.62
NOUN:POSS	0	0	6	100.00	0.00	0.00
ORTH	28	18	19	60.87	59.57	60.61
OTHER	6	4	54	60.00	10.00	30.00
PART	0	0	3	100.00	0.00	0.00
PREP	5	0	46	100.00	9.80	35.21
PRON	2	0	7	100.00	22.22	58.82
PUNCT	11	1	40	91.67	21.57	55.56
SPELL	7	2	29	77.78	19.44	48.61
UNK	1	0	4	100.00	20.00	55.56
VERB	0	2	11	0.00	0.00	0.00
VERB:FORM	6	2	10	75.00	37.50	62.50
VERB:SVA	4	1	5	80.00	44.44	68.97
VERB:TENSE	3	1	25	75.00	10.71	34.09
WO	3	1	9	75.00	25.00	53.57

OPERATION + ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
M:ADJ	0	0	1	100.00	0.00	0.00
M:ADV	0	0	1	100.00	0.00	0.00
M:CONJ	0	0	2	100.00	0.00	0.00
M:DET	1	1	32	50.00	3.03	12.20
M:NOUN	0	0	3	100.00	0.00	0.00
M:OTHER	1	0	6	100.00	14.29	45.45
M:PART	0	0	1	100.00	0.00	0.00
M:PREP	0	0	15	100.00	0.00	0.00
M:PRON	1	0	4	100.00	20.00	55.56
M:PUNCT	7	1	29	87.50	19.44	51.47
M:VERB	0	0	3	100.00	0.00	0.00
M:VERB:FORM	1	0	1	100.00	50.00	83.33
M:VERB:TENSE	0	0	3	100.00	0.00	0.00
R:ADJ	0	0	5	100.00	0.00	0.00
R:ADJ:FORM	0	0	2	100.00	0.00	0.00
R:ADV	0	0	1	100.00	0.00	0.00
R:CONJ	0	0	2	100.00	0.00	0.00
R:CONTR	0	0	1	100.00	0.00	0.00
R:DET	3	0	8	100.00	27.27	65.22
R:MORPH	4	1	12	80.00	25.00	55.56
R:NOUN	0	2	10	0.00	0.00	0.00
R:NOUN:INFL	2	0	0	100.00	100.00	100.00
R:NOUN:NUM	6	3	21	66.67	22.22	47.62
R:NOUN:POSS	0	0	5	100.00	0.00	0.00
R:ORTH	28	18	19	60.87	59.57	60.61
R:OTHER	5	3	47	62.50	9.62	29.76
R:PART	0	0	2	100.00	0.00	0.00
R:PREP	5	0	24	100.00	17.24	51.02
R:PRON	1	0	2	100.00	33.33	71.43
R:PUNCT	1	0	9	100.00	10.00	35.71
R:SPELL	7	2	29	77.78	19.44	48.61
R:VERB	0	2	8	0.00	0.00	0.00
R:VERB:FORM	4	2	9	66.67	30.77	54.05
R:VERB:SVA	4	1	5	80.00	44.44	68.97
R:VERB:TENSE	2	0	19	100.00	9.52	34.48
R:WO	3	1	9	75.00	25.00	53.57
U:ADJ	0	0	1	100.00	0.00	0.00
U:ADV	0	1	2	0.00	0.00	0.00
U:CONTR	1	0	2	100.00	33.33	71.43
U:DET	7	1	27	87.50	20.59	53.03
U:NOUN:POSS	0	0	1	100.00	0.00	0.00
U:OTHER	0	1	1	0.00	0.00	0.00
U:PREP	0	0	7	100.00	0.00	0.00
U:PRON	0	0	1	100.00	0.00	0.00
U:PUNCT	3	0	2	100.00	60.00	88.24
U:VERB:FORM	1	0	0	100.00	100.00	100.00
U:VERB:TENSE	1	1	3	50.00	25.00	41.67
UNK	1	0	4	100.00	20.00	55.56

TOKEN-LEVEL DETECTION
OPERATION LEVEL
	TP	FP	FN	P	R	F_0.5
M	17	2	116	89.47	12.78	40.67
R	87	27	274	76.32	24.10	53.24
U	17	3	57	85.00	22.97	55.19
UNK	2	0	5	100.00	28.57	66.67

ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
ADJ	0	0	9	100.00	0.00	0.00
ADJ:FORM	0	0	3	100.00	0.00	0.00
ADV	0	1	6	0.00	0.00	0.00
CONJ	0	0	3	100.00	0.00	0.00
CONTR	1	0	3	100.00	25.00	62.50
DET	12	2	69	85.71	14.81	43.80
MORPH	4	1	13	80.00	23.53	54.05
NOUN	1	1	15	50.00	6.25	20.83
NOUN:INFL	1	0	0	100.00	100.00	100.00
NOUN:NUM	5	2	26	71.43	16.13	42.37
NOUN:POSS	1	0	8	100.00	11.11	38.46
ORTH	34	14	21	70.83	61.82	68.83
OTHER	12	2	66	85.71	15.38	44.78
PART	0	0	3	100.00	0.00	0.00
PREP	5	0	47	100.00	9.62	34.72
PRON	3	0	8	100.00	27.27	65.22
PUNCT	12	1	51	92.31	19.05	52.17
SPELL	7	2	30	77.78	18.92	47.95
UNK	2	0	5	100.00	28.57	66.67
VERB	4	2	12	66.67	25.00	50.00
VERB:FORM	5	0	11	100.00	31.25	69.44
VERB:SVA	3	1	6	75.00	33.33	60.00
VERB:TENSE	3	1	30	75.00	9.09	30.61
WO	8	2	7	80.00	53.33	72.73

OPERATION + ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
M:ADJ	0	0	2	100.00	0.00	0.00
M:ADV	0	0	2	100.00	0.00	0.00
M:CONJ	0	0	2	100.00	0.00	0.00
M:DET	3	1	32	75.00	8.57	29.41
M:NOUN	0	0	3	100.00	0.00	0.00
M:NOUN:POSS	1	0	0	100.00	100.00	100.00
M:OTHER	2	0	7	100.00	22.22	58.82
M:PART	0	0	1	100.00	0.00	0.00
M:PREP	0	0	15	100.00	0.00	0.00
M:PRON	2	0	4	100.00	33.33	71.43
M:PUNCT	8	1	39	88.89	17.02	48.19
M:VERB	0	0	3	100.00	0.00	0.00
M:VERB:FORM	1	0	1	100.00	50.00	83.33
M:VERB:TENSE	0	0	5	100.00	0.00	0.00
R:ADJ	0	0	4	100.00	0.00	0.00
R:ADJ:FORM	0	0	3	100.00	0.00	0.00
R:ADV	0	0	2	100.00	0.00	0.00
R:CONJ	0	0	1	100.00	0.00	0.00
R:CONTR	0	0	1	100.00	0.00	0.00
R:DET	2	0	9	100.00	18.18	52.63
R:MORPH	4	1	13	80.00	23.53	54.05
R:NOUN	0	1	10	0.00	0.00	0.00
R:NOUN:INFL	1	0	0	100.00	100.00	100.00
R:NOUN:NUM	5	2	26	71.43	16.13	42.37
R:NOUN:POSS	0	0	7	100.00	0.00	0.00
R:ORTH	34	14	21	70.83	61.82	68.83
R:OTHER	10	2	55	83.33	15.38	44.25
R:PART	0	0	2	100.00	0.00	0.00
R:PREP	5	0	25	100.00	16.67	50.00
R:PRON	1	0	2	100.00	33.33	71.43
R:PUNCT	1	0	9	100.00	10.00	35.71
R:SPELL	7	2	30	77.78	18.92	47.95
R:VERB	1	2	9	33.33	10.00	22.73
R:VERB:FORM	3	0	10	100.00	23.08	60.00
R:VERB:SVA	3	1	6	75.00	33.33	60.00
R:VERB:TENSE	2	0	22	100.00	8.33	31.25
R:WO	8	2	7	80.00	53.33	72.73
U:ADJ	0	0	3	100.00	0.00	0.00
U:ADV	0	1	2	0.00	0.00	0.00
U:CONTR	1	0	2	100.00	33.33	71.43
U:DET	7	1	28	87.50	20.00	52.24
U:NOUN	1	0	2	100.00	33.33	71.43
U:NOUN:POSS	0	0	1	100.00	0.00	0.00
U:OTHER	0	0	4	100.00	0.00	0.00
U:PREP	0	0	7	100.00	0.00	0.00
U:PRON	0	0	2	100.00	0.00	0.00
U:PUNCT	3	0	3	100.00	50.00	83.33
U:VERB	3	0	0	100.00	100.00	100.00
U:VERB:FORM	1	0	0	100.00	100.00	100.00
U:VERB:TENSE	1	1	3	50.00	25.00	41.67
UNK	2	0	5	100.00	28.57	66.67

OVERALL
	TP	FP	FN	P	R	F_0.5
Span-level correction	46	71	323	39.32	12.47	27.48
Span-level detection	61	56	321	52.14	15.97	35.88
Token-level detection	72	57	350	55.81	17.06	38.38

SPAN-LEVEL CORRECTION
OPERATION LEVEL
	TP	FP	FN	P	R	F_0.5
M	2	6	42	25.00	4.55	13.16
R	40	54	264	42.55	13.16	29.41
U	4	11	17	26.67	19.05	24.69

ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
ADJ	0	2	3	0.00	0.00	0.00
ADV	1	0	1	100.00	50.00	83.33
DET	2	6	7	25.00	22.22	24.39
MORPH	2	0	3	100.00	40.00	76.92
NOUN	1	2	8	33.33	11.11	23.81
NOUN:INFL	0	0	11	100.00	0.00	0.00
NOUN:NUM	3	2	12	60.00	20.00	42.86
NOUN:POSS	1	0	15	100.00	6.25	25.00
ORTH	18	36	37	33.33	32.73	33.21
OTHER	3	2	45	60.00	6.25	22.06
PART	0	0	1	100.00	0.00	0.00
PREP	0	5	11	0.00	0.00	0.00
PRON	0	0	5	100.00	0.00	0.00
PUNCT	1	2	49	33.33	2.00	8.06
SPELL	5	3	85	62.50	5.56	20.49
VERB	1	2	9	33.33	10.00	22.73
VERB:FORM	4	5	3	44.44	57.14	46.51
VERB:INFL	0	0	5	100.00	0.00	0.00
VERB:SVA	3	2	6	60.00	33.33	51.72
VERB:TENSE	0	1	6	0.00	0.00	0.00
WO	1	1	1	50.00	50.00	50.00

OPERATION + ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
M:ADV	0	0	1	100.00	0.00	0.00
M:DET	0	3	3	0.00	0.00	0.00
M:NOUN	0	0	2	100.00	0.00	0.00
M:NOUN:POSS	0	0	5	100.00	0.00	0.00
M:PRON	0	0	2	100.00	0.00	0.00
M:PUNCT	1	2	27	33.33	3.57	12.50
M:VERB	1	0	2	100.00	33.33	71.43
M:VERB:FORM	0	1	0	0.00	100.00	0.00
R:ADJ	0	2	3	0.00	0.00	0.00
R:DET	0	0	2	100.00	0.00	0.00
R:MORPH	2	0	3	100.00	40.00	76.92
R:NOUN	1	2	6	33.33	14.29	26.32
R:NOUN:INFL	0	0	11	100.00	0.00	0.00
R:NOUN:NUM	3	2	12	60.00	20.00	42.86
R:NOUN:POSS	1	0	10	100.00	9.09	33.33
R:ORTH	18	36	37	33.33	32.73	33.21
R:OTHER	2	0	45	100.00	4.26	18.18
R:PREP	0	1	10	0.00	0.00	0.00
R:PRON	0	0	3	100.00	0.00	0.00
R:PUNCT	0	0	12	100.00	0.00	0.00
R:SPELL	5	3	85	62.50	5.56	20.49
R:VERB	0	1	6	0.00	0.00	0.00
R:VERB:FORM	4	4	3	50.00	57.14	51.28
R:VERB:INFL	0	0	5	100.00	0.00	0.00
R:VERB:SVA	3	2	6	60.00	33.33	51.72
R:VERB:TENSE	0	0	4	100.00	0.00	0.00
R:WO	1	1	1	50.00	50.00	50.00
U:ADV	1	0	0	100.00	100.00	100.00
U:DET	2	3	2	40.00	50.00	41.67
U:OTHER	1	2	0	33.33	100.00	38.46
U:PART	0	0	1	100.00	0.00	0.00
U:PREP	0	4	1	0.00	0.00	0.00
U:PUNCT	0	0	10	100.00	0.00	0.00
U:VERB	0	1	1	0.00	0.00	0.00
U:VERB:TENSE	0	1	2	0.00	0.00	0.00

SPAN-LEVEL DETECTION
OPERATION LEVEL
	TP	FP	FN	P	R	F_0.5
M	3	5	43	37.50	6.52	19.23
R	54	41	257	56.84	17.36	39.07
U	3	10	18	23.08	14.29	20.55
UNK	1	0	3	100.00	25.00	62.50

ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
ADJ	1	1	2	50.00	33.33	45.45
DET	3	5	8	37.50	27.27	34.88
MORPH	3	0	3	100.00	50.00	83.33
NOUN	1	0	8	100.00	11.11	38.46
NOUN:INFL	0	0	11	100.00	0.00	0.00
NOUN:NUM	3	2	12	60.00	20.00	42.86
NOUN:POSS	2	0	14	100.00	12.50	41.67
ORTH	21	33	35	38.89	37.50	38.60
OTHER	5	2	44	71.43	10.20	32.47
PART	0	0	1	100.00	0.00	0.00
PREP	1	4	11	20.00	8.33	15.62
PRON	0	0	5	100.00	0.00	0.00
PUNCT	1	2	51	33.33	1.92	7.81
SPELL	7	0	85	100.00	7.61	29.17
UNK	1	0	3	100.00	25.00	62.50
VERB	2	1	8	66.67	20.00	45.45
VERB:FORM	4	4	3	50.00	57.14	51.28
VERB:INFL	1	0	4	100.00	20.00	55.56
VERB:SVA	3	0	6	100.00	33.33	71.43
VERB:TENSE	1	1	6	50.00	14.29	33.33
WO	1	1	1	50.00	50.00	50.00

OPERATION + ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
M:DET	1	2	3	33.33	25.00	31.25
M:NOUN	0	0	2	100.00	0.00	0.00
M:NOUN:POSS	0	0	5	100.00	0.00	0.00
M:PRON	0	0	2	100.00	0.00	0.00
M:PUNCT	1	2	29	33.33	3.33	11.90
M:VERB	1	0	2	100.00	33.33	71.43
M:VERB:FORM	0	1	0	0.00	100.00	0.00
R:ADJ	1	1	2	50.00	33.33	45.45
R:DET	0	0	2	100.00	0.00	0.00
R:MORPH	3	0	3	100.00	50.00	83.33
R:NOUN	1	0	6	100.00	14.29	45.45
R:NOUN:INFL	0	0	11	100.00	0.00	0.00
R:NOUN:NUM	3	2	12	60.00	20.00	42.86
R:NOUN:POSS	2	0	9	100.00	18.18	52.63
R:ORTH	21	33	35	38.89	37.50	38.60
R:OTHER	4	0	44	100.00	8.33	31.25
R:PREP	1	1	10	50.00	9.09	26.32
R:PRON	0	0	3	100.00	0.00	0.00
R:PUNCT	0	0	12	100.00	0.00	0.00
R:SPELL	7	0	85	100.00	7.61	29.17
R:VERB	1	0	5	100.00	16.67	50.00
R:VERB:FORM	4	3	3	57.14	57.14	57.14
R:VERB:INFL	1	0	4	100.00	20.00	55.56
R:VERB:SVA	3	0	6	100.00	33.33	71.43
R:VERB:TENSE	1	0	4	100.00	20.00	55.56
R:WO	1	1	1	50.00	50.00	50.00
U:DET	2	3	3	40.00	40.00	40.00
U:OTHER	1	2	0	33.33	100.00	38.46
U:PART	0	0	1	100.00	0.00	0.00
U:PREP	0	3	1	0.00	0.00	0.00
U:PUNCT	0	0	10	100.00	0.00	0.00
U:VERB	0	1	1	0.00	0.00	0.00
U:VERB:TENSE	0	1	2	0.00	0.00	0.00
UNK	1	0	3	100.00	25.00	62.50

TOKEN-LEVEL DETECTION
OPERATION LEVEL
	TP	FP	FN	P	R	F_0.5
M	5	5	48	50.00	9.43	26.88
R	60	40	280	60.00	17.65	40.54
U	5	12	20	29.41	20.00	26.88
UNK	2	0	2	100.00	50.00	83.33

ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
ADJ	1	1	3	50.00	25.00	41.67
CONTR	0	0	2	100.00	0.00	0.00
DET	2	3	9	40.00	18.18	32.26
MORPH	3	0	3	100.00	50.00	83.33
NOUN	1	0	10	100.00	9.09	33.33
NOUN:INFL	0	0	11	100.00	0.00	0.00
NOUN:NUM	3	2	13	60.00	18.75	41.67
NOUN:POSS	2	0	15	100.00	11.76	40.00
ORTH	22	31	43	41.51	33.85	39.71
OTHER	11	4	53	73.33	17.19	44.35
PART	0	0	1	100.00	0.00	0.00
PREP	1	3	10	25.00	9.09	18.52
PRON	0	0	6	100.00	0.00	0.00
PUNCT	5	2	57	71.43	8.06	27.78
SPELL	7	0	85	100.00	7.61	29.17
UNK	2	0	2	100.00	50.00	83.33
VERB	1	3	8	25.00	11.11	20.00
VERB:FORM	4	4	3	50.00	57.14	51.28
VERB:INFL	1	0	4	100.00	20.00	55.56
VERB:SVA	3	0	6	100.00	33.33	71.43
VERB:TENSE	1	1	6	50.00	14.29	33.33
WO	2	3	0	40.00	100.00	45.45

OPERATION + ERROR TYPE LEVEL
	TP	FP	FN	P	R	F_0.5
M:DET	0	1	3	0.00	0.00	0.00
M:NOUN	0	0	3	100.00	0.00	0.00
M:NOUN:POSS	0	0	5	100.00	0.00	0.00
M:PRON	0	0	3	100.00	0.00	0.00
M:PUNCT	5	2	32	71.43	13.51	38.46
M:VERB	0	1	2	0.00	0.00	0.00
M:VERB:FORM	0	1	0	0.00	100.00	0.00
R:ADJ	1	1	2	50.00	33.33	45.45
R:CONTR	0	0	2	100.00	0.00	0.00
R:DET	0	0	3	100.00	0.00	0.00
R:MORPH	3	0	3	100.00	50.00	83.33
R:NOUN	1	0	7	100.00	12.50	41.67
R:NOUN:INFL	0	0	11	100.00	0.00	0.00
R:NOUN:NUM	3	2	13	60.00	18.75	41.67
R:NOUN:POSS	2	0	10	100.00	16.67	50.00
R:ORTH	22	31	43	41.51	33.85	39.71
R:OTHER	8	0	52	100.00	13.33	43.48
R:PREP	1	0	9	100.00	10.00	35.71
R:PRON	0	0	3	100.00	0.00	0.00
R:PUNCT	0	0	15	100.00	0.00	0.00
R:SPELL	7	0	85	100.00	7.61	29.17
R:VERB	1	0	5	100.00	16.67	50.00
R:VERB:FORM	4	3	3	57.14	57.14	57.14
R:VERB:INFL	1	0	4	100.00	20.00	55.56
R:VERB:SVA	3	0	6	100.00	33.33	71.43
R:VERB:TENSE	1	0	4	100.00	20.00	55.56
R:WO	2	3	0	40.00	100.00	45.45
U:ADJ	0	0	1	100.00	0.00	0.00
U:DET	2	2	3	50.00	40.00	47.62
U:OTHER	3	4	1	42.86	75.00	46.88
U:PART	0	0	1	100.00	0.00	0.00
U:PREP	0	3	1	0.00	0.00	0.00
U:PUNCT	0	0	10	100.00	0.00	0.00
U:VERB	0	2	1	0.00	0.00	0.00
U:VERB:TENSE	0	1	2	0.00	0.00	0.00
UNK	2	0	2	100.00	50.00	83.33

Submitted by:	cjbryant
Submitted at:	2019-08-07T14:46:57+00:00
Competition submission:	7
Competition phase:	3 - Open phase

Detailed results

Submission information

Results