Table 1 Open in new tab Performance of...

	Performance of systems (10-fold cross-validation)
AGE	94.47	92.31	93.38	96.7	90.42	93.45	95.87	92.46	94.13	96.69	90	93.22
DATE	95.77	97.12	96.44	97.97	96.61	97.29	97.25	97.76	97.5	97.95	96.98	97.46
ID	90.58	92.64	91.6	97.23	95.64	96.43	91.17	93.38	92.26	97.27	95.7	96.48
INST	90.59	86.41	88.45	93.18	85.01	88.91	90.61	87.06	88.8	93.25	85.26	89.08
LOC	79.82	67.93	73.4	86.12	68.94	76.58	78.92	69.95	74.16	87.38	69.95	77.7
NAME	93.19	88.64	90.86	95.62	86.99	91.1	92.48	94.16	93.31	94.47	94.56	94.52
OTH	77.21	77.39	77.3	83.94	74.17	78.76	78.17	77.13	77.65	84.68	73.87	78.91
PH	90.06	93.04	91.52	94.08	90.64	92.33	91.44	93.95	92.68	94.42	90.87	92.61
All	92.05	91.02	91.54	95.25	89.86	92.48	92.79	92.81	92.8	95.08	91.92	93.48

Performance of systems (10-fold cross-validation)

MIST1

MCRF1

MIST2

MCRF2

AGE

94.47

92.31

93.38

96.7

90.42

93.45

95.87

92.46

94.13

96.69

93.22

DATE

95.77

97.12

96.44

97.97

96.61

97.29

97.25

97.76

97.5

97.95

96.98

97.46

90.58

92.64

91.6

97.23

95.64

96.43

91.17

93.38

92.26

97.27

95.7

96.48

INST

90.59

86.41

88.45

93.18

85.01

88.91

90.61

87.06

88.8

93.25

85.26

89.08

LOC

79.82

67.93

73.4

86.12

68.94

76.58

78.92

69.95

74.16

87.38

69.95

77.7

NAME

93.19

88.64

90.86

95.62

86.99

91.1

92.48

94.16

93.31

94.47

94.56

94.52

OTH

77.21

77.39

77.3

83.94

74.17

78.76

78.17

77.13

77.65

84.68

73.87

78.91

90.06

93.04

91.52

94.08

90.64

92.33

91.44

93.95

92.68

94.42

90.87

92.61

All

92.05

91.02

91.54

95.25

89.86

92.48

92.79

92.81

92.8

95.08

91.92

93.48

	Performance of systems (10-fold cross-validation)
AGE	94.47	92.31	93.38	96.7	90.42	93.45	95.87	92.46	94.13	96.69	90	93.22
DATE	95.77	97.12	96.44	97.97	96.61	97.29	97.25	97.76	97.5	97.95	96.98	97.46
ID	90.58	92.64	91.6	97.23	95.64	96.43	91.17	93.38	92.26	97.27	95.7	96.48
INST	90.59	86.41	88.45	93.18	85.01	88.91	90.61	87.06	88.8	93.25	85.26	89.08
LOC	79.82	67.93	73.4	86.12	68.94	76.58	78.92	69.95	74.16	87.38	69.95	77.7
NAME	93.19	88.64	90.86	95.62	86.99	91.1	92.48	94.16	93.31	94.47	94.56	94.52
OTH	77.21	77.39	77.3	83.94	74.17	78.76	78.17	77.13	77.65	84.68	73.87	78.91
PH	90.06	93.04	91.52	94.08	90.64	92.33	91.44	93.95	92.68	94.42	90.87	92.61
All	92.05	91.02	91.54	95.25	89.86	92.48	92.79	92.81	92.8	95.08	91.92	93.48

	Token-level performance for best system (MCRF2)
AGE	98.21	93.40	95.75		93.42
DATE	98.17	96.87	97.52		96.98
ID	97.57	95.49	96.52		96.43
INST	97.48	92.74	95.05		94.79
LOC	97.95	93.92	95.89		96.03
NAME	97.26	97.38	97.32		97.53
OTH	86.81	76.45	81.30		78.31
PH	97.13	93.40	95.23		94.85
All	96.68	93.77	95.20	97.42	94.49	95.93

Token-level performance for best system (MCRF2)

Token-level

Token-level + tag-blind

AGE

98.21

93.40

95.75

93.42

DATE

98.17

96.87

97.52

96.98

97.57

95.49

96.52

96.43

INST

97.48

92.74

95.05

94.79

LOC

97.95

93.92

95.89

96.03

NAME

97.26

97.38

97.32

97.53

OTH

86.81

76.45

81.30

78.31

97.13

93.40

95.23

94.85

All

96.68

93.77

95.20

97.42

94.49

95.93

	Token-level performance for best system (MCRF2)
AGE	98.21	93.40	95.75		93.42
DATE	98.17	96.87	97.52		96.98
ID	97.57	95.49	96.52		96.43
INST	97.48	92.74	95.05		94.79
LOC	97.95	93.92	95.89		96.03
NAME	97.26	97.38	97.32		97.53
OTH	86.81	76.45	81.30		78.31
PH	97.13	93.40	95.23		94.85
All	96.68	93.77	95.20	97.42	94.49	95.93

	Statistical significance tests between F values obtained by systems (cross-validation evaluation)
AGE	0.8490	*0.0087	*0.0389	0.4922	*0.0001
DATE	*0.0001	0.7650	*0.0001	0.2470	*0.0001
ID	*0.0001	*0.0001	0.0996	0.8856	*0.0001
INST	0.3572	0.6087	0.2738	0.6623	*0.0001
LOC	0.0777	0.0553	0.5897	0.3812	*0.0001
NAME	0.4897	*0.0001	*0.0001	*0.0001	*0.0001
OTH	*0.0071	*0.0180	0.3676	0.6118	*0.0001
PH	0.2936	0.9248	*0.0458	0.6208	*0.0001
All	*0.0001	*0.0001	*0.0001	*0.0001	*0.0001

Statistical significance tests between F values obtained by systems (cross-validation evaluation)

MCRF1 vs MIST1

MCRF2 vs MIST2

MIST1 vs MIST2

MCRF2 vs MCRF1

MCRF2 vs gold standard

p Value

AGE

0.8490

*0.0087

*0.0389

0.4922

*0.0001

DATE

*0.0001

0.7650

*0.0001

0.2470

*0.0001

0.0996

0.8856

*0.0001

INST

0.3572

0.6087

0.2738

0.6623

*0.0001

LOC

0.0777

0.0553

0.5897

0.3812

*0.0001

NAME

0.4897

*0.0001

OTH

*0.0071

*0.0180

0.3676

0.6118

*0.0001

0.2936

0.9248

*0.0458

0.6208

*0.0001

All

*0.0001

	Statistical significance tests between F values obtained by systems (cross-validation evaluation)
AGE	0.8490	*0.0087	*0.0389	0.4922	*0.0001
DATE	*0.0001	0.7650	*0.0001	0.2470	*0.0001
ID	*0.0001	*0.0001	0.0996	0.8856	*0.0001
INST	0.3572	0.6087	0.2738	0.6623	*0.0001
LOC	0.0777	0.0553	0.5897	0.3812	*0.0001
NAME	0.4897	*0.0001	*0.0001	*0.0001	*0.0001
OTH	*0.0071	*0.0180	0.3676	0.6118	*0.0001
PH	0.2936	0.9248	*0.0458	0.6208	*0.0001
All	*0.0001	*0.0001	*0.0001	*0.0001	*0.0001

Table 1

Open in new tab

Performance of systems (per-tag and overall precision (P), recall (R) and F value (F)) and statistical significance tests

	Performance of systems (10-fold cross-validation)
	MIST1			MCRF1			MIST2			MCRF2
	P	R	F	P	R	F	P	R	F	P	R	F
AGE	94.47	92.31	93.38	96.7	90.42	93.45	95.87	92.46	94.13	96.69	90	93.22
DATE	95.77	97.12	96.44	97.97	96.61	97.29	97.25	97.76	97.5	97.95	96.98	97.46
ID	90.58	92.64	91.6	97.23	95.64	96.43	91.17	93.38	92.26	97.27	95.7	96.48
INST	90.59	86.41	88.45	93.18	85.01	88.91	90.61	87.06	88.8	93.25	85.26	89.08
LOC	79.82	67.93	73.4	86.12	68.94	76.58	78.92	69.95	74.16	87.38	69.95	77.7
NAME	93.19	88.64	90.86	95.62	86.99	91.1	92.48	94.16	93.31	94.47	94.56	94.52
OTH	77.21	77.39	77.3	83.94	74.17	78.76	78.17	77.13	77.65	84.68	73.87	78.91
PH	90.06	93.04	91.52	94.08	90.64	92.33	91.44	93.95	92.68	94.42	90.87	92.61
All	92.05	91.02	91.54	95.25	89.86	92.48	92.79	92.81	92.8	95.08	91.92	93.48

	Performance of systems (10-fold cross-validation)
	MIST1			MCRF1			MIST2			MCRF2
	P	R	F	P	R	F	P	R	F	P	R	F
AGE	94.47	92.31	93.38	96.7	90.42	93.45	95.87	92.46	94.13	96.69	90	93.22
DATE	95.77	97.12	96.44	97.97	96.61	97.29	97.25	97.76	97.5	97.95	96.98	97.46
ID	90.58	92.64	91.6	97.23	95.64	96.43	91.17	93.38	92.26	97.27	95.7	96.48
INST	90.59	86.41	88.45	93.18	85.01	88.91	90.61	87.06	88.8	93.25	85.26	89.08
LOC	79.82	67.93	73.4	86.12	68.94	76.58	78.92	69.95	74.16	87.38	69.95	77.7
NAME	93.19	88.64	90.86	95.62	86.99	91.1	92.48	94.16	93.31	94.47	94.56	94.52
OTH	77.21	77.39	77.3	83.94	74.17	78.76	78.17	77.13	77.65	84.68	73.87	78.91
PH	90.06	93.04	91.52	94.08	90.64	92.33	91.44	93.95	92.68	94.42	90.87	92.61
All	92.05	91.02	91.54	95.25	89.86	92.48	92.79	92.81	92.8	95.08	91.92	93.48

	Token-level performance for best system (MCRF2)
	Token-level			Token-level + tag-blind
	P	R	F	P	R	F
AGE	98.21	93.40	95.75		93.42
DATE	98.17	96.87	97.52		96.98
ID	97.57	95.49	96.52		96.43
INST	97.48	92.74	95.05		94.79
LOC	97.95	93.92	95.89		96.03
NAME	97.26	97.38	97.32		97.53
OTH	86.81	76.45	81.30		78.31
PH	97.13	93.40	95.23		94.85
All	96.68	93.77	95.20	97.42	94.49	95.93

	Token-level performance for best system (MCRF2)
	Token-level			Token-level + tag-blind
	P	R	F	P	R	F
AGE	98.21	93.40	95.75		93.42
DATE	98.17	96.87	97.52		96.98
ID	97.57	95.49	96.52		96.43
INST	97.48	92.74	95.05		94.79
LOC	97.95	93.92	95.89		96.03
NAME	97.26	97.38	97.32		97.53
OTH	86.81	76.45	81.30		78.31
PH	97.13	93.40	95.23		94.85
All	96.68	93.77	95.20	97.42	94.49	95.93

	Statistical significance tests between F values obtained by systems (cross-validation evaluation)
	MCRF1 vs MIST1	MCRF2 vs MIST2	MIST1 vs MIST2	MCRF2 vs MCRF1	MCRF2 vs gold standard
	p Value	p Value	p Value	p Value	p Value
AGE	0.8490	*0.0087	*0.0389	0.4922	*0.0001
DATE	*0.0001	0.7650	*0.0001	0.2470	*0.0001
ID	*0.0001	*0.0001	0.0996	0.8856	*0.0001
INST	0.3572	0.6087	0.2738	0.6623	*0.0001
LOC	0.0777	0.0553	0.5897	0.3812	*0.0001
NAME	0.4897	*0.0001	*0.0001	*0.0001	*0.0001
OTH	*0.0071	*0.0180	0.3676	0.6118	*0.0001
PH	0.2936	0.9248	*0.0458	0.6208	*0.0001
All	*0.0001	*0.0001	*0.0001	*0.0001	*0.0001

	Statistical significance tests between F values obtained by systems (cross-validation evaluation)
	MCRF1 vs MIST1	MCRF2 vs MIST2	MIST1 vs MIST2	MCRF2 vs MCRF1	MCRF2 vs gold standard
	p Value	p Value	p Value	p Value	p Value
AGE	0.8490	*0.0087	*0.0389	0.4922	*0.0001
DATE	*0.0001	0.7650	*0.0001	0.2470	*0.0001
ID	*0.0001	*0.0001	0.0996	0.8856	*0.0001
INST	0.3572	0.6087	0.2738	0.6623	*0.0001
LOC	0.0777	0.0553	0.5897	0.3812	*0.0001
NAME	0.4897	*0.0001	*0.0001	*0.0001	*0.0001
OTH	*0.0071	*0.0180	0.3676	0.6118	*0.0001
PH	0.2936	0.9248	*0.0458	0.6208	*0.0001
All	*0.0001	*0.0001	*0.0001	*0.0001	*0.0001

*Indicates statistical significance (p<0.05).

INST, institution; LOC, location; OTH, other; PH, phone.

This Feature Is Available To Subscribers Only