Our results

RESULTS FOR BRISCOE AND CARROLL'S SYSTEM

Average results for the 30 verbs

 
Rank Accuracy:  80.10%
Type Precision: 75.29%
Type Recall:    67.13%
F Measure:      70.98


Average similarity values between the acquired (q)
and gold standard (p) SCF distributions:

                      Common  Gold  Joined
------------------------------------------
KL distance         :  0.363  0.394  0.546
JS divergence       :  0.079  0.080  0.122
cross entropy       :  1.888  1.927  2.082
skew divergence     :  0.351  0.356  0.505
rank correlation    :  0.779  0.775  0.609
intersection        :  1.000  0.991  0.606


Common = SCFs which occur in both 
         p and q are considered
Gold   = only those SCFs which occur
         in p are considered
Joined = SCFs which occur in either 
         p or q are considered

Results for the individual verbs
add
7 SCFs in both lexicon and gold standard Rank Accuracy: 100.00% Type Precision: 60.00% Type Recall: 66.67% F Measure: 63.16 Common Gold Joined ------------------------------------------ KL distance : 0.046 1.193 1.296 JS divergence : 0.012 0.057 0.093 cross entropy : 1.480 2.853 2.956 skew divergence : 0.045 0.509 0.611 rank correlation : 0.893 0.727 0.464 intersection : 1.000 0.875 0.500
agree
14 SCFs in both lexicon and gold-standard Rank Accuracy: 80.00% Type Precision: 85.71% Type Recall: 42.86% F Measure: 57.14 Common Gold Joined ------------------------------------------ KL distance : 0.543 0.535 0.645 JS divergence : 0.101 0.101 0.130 cross entropy : 2.625 2.612 2.727 skew divergence : 0.519 0.512 0.618 rank correlation : 0.824 0.824 0.529 intersection : 1.000 1.000 0.683
attach
5 SCFs in both lexicon and gold-standard Rank Accuracy: 66.67% Type Precision: 42.86% Type Recall: 60.00% F Measure: 50.00 Common Gold Joined ------------------------------------------ KL distance : 0.353 0.354 0.543 JS divergence : 0.094 0.094 0.150 cross entropy : 0.993 0.994 1.183 skew divergence : 0.346 0.347 0.532 rank correlation : 0.800 0.800 0.612 intersection : 1.000 1.000 0.526
bring
11 SCFs in both lexicon and gold-standard Rank Accuracy: 57.14% Type Precision: 63.64% Type Recall: 63.64% F Measure: 63.64 Common Gold Joined ------------------------------------------ KL distance : 1.347 1.340 1.482 JS divergence : 0.270 0.270 0.298 cross entropy : 3.131 3.121 3.266 skew divergence : 1.286 1.280 1.412 rank correlation : 0.433 0.433 0.491 intersection : 1.000 1.000 0.611
carry
11 SCFs in both lexicon and gold-standard Rank Accuracy: 66.67% Type Precision: 75.00% Type Recall: 81.82% F Measure: 78.26 Common Gold Joined ------------------------------------------ KL distance : 0.518 0.513 0.578 JS divergence : 0.122 0.122 0.139 cross entropy : 1.932 1.926 1.992 skew divergence : 0.504 0.499 0.562 rank correlation : 0.572 0.572 0.662 intersection : 1.000 1.000 0.579
carve
12 SCFs in both lexicon and gold-standard Rank Accuracy: 85.71% Type Precision: 100.00% Type Recall: 58.33% F Measure: 73.68 Common Gold Joined ------------------------------------------ KL distance : 0.138 0.141 0.154 JS divergence : 0.038 0.038 0.043 cross entropy : 1.778 1.782 1.794 skew divergence : 0.135 0.138 0.151 rank correlation : 0.920 0.920 0.811 intersection : 1.000 1.000 0.889
chop
9 SCFs in both lexicon and gold-standard Rank Accuracy: 95.24% Type Precision: 70.00% Type Recall: 77.78% F Measure: 73.68 Common Gold Joined ------------------------------------------ KL distance : 0.226 0.226 0.366 JS divergence : 0.051 0.051 0.095 cross entropy : 1.876 1.876 2.017 skew divergence : 0.220 0.220 0.358 rank correlation : 0.845 0.845 0.717 intersection : 1.000 1.000 0.621
cling
6 SCFs in both lexicon and gold-standard Rank Accuracy: 100.00% Type Precision: 37.50% Type Recall: 50.00% F Measure: 42.86 Common Gold Joined ------------------------------------------ KL distance : 0.222 0.221 0.772 JS divergence : 0.063 0.063 0.224 cross entropy : 0.823 0.822 1.373 skew divergence : 0.218 0.217 0.758 rank correlation : 0.986 0.986 0.163 intersection : 1.000 1.000 0.571
clip
9 SCFs in both lexicon and gold-standard Rank Accuracy: 85.71% Type Precision: 87.50% Type Recall: 77.78% F Measure: 82.35 Common Gold Joined ------------------------------------------ KL distance : 0.220 0.220 0.265 JS divergence : 0.053 0.053 0.067 cross entropy : 1.886 1.886 1.931 skew divergence : 0.214 0.214 0.259 rank correlation : 0.848 0.848 0.772 intersection : 1.000 1.000 0.720
communicate
8 SCFs in both lexicon and gold-standard Rank Accuracy: 66.67% Type Precision: 80.00% Type Recall: 40.00% F Measure: 53.33 Common Gold Joined ------------------------------------------ KL distance : 0.484 0.577 0.743 JS divergence : 0.101 0.106 0.154 cross entropy : 1.900 2.063 2.230 skew divergence : 0.461 0.510 0.670 rank correlation : 0.790 0.853 0.305 intersection : 1.000 0.889 0.457
cut
9 SCFs in both lexicon and gold-standard Rank Accuracy: 76.19% Type Precision: 100.00% Type Recall: 77.78% F Measure: 87.50 Common Gold Joined ------------------------------------------ KL distance : 0.224 0.224 0.263 JS divergence : 0.050 0.050 0.063 cross entropy : 2.072 2.072 2.112 skew divergence : 0.218 0.218 0.256 rank correlation : 0.820 0.820 0.831 intersection : 1.000 1.000 0.600
drag
10 SCFs in both lexicon and gold-standard Rank Accuracy: 86.11% Type Precision: 69.23% Type Recall: 90.00% F Measure: 78.26 Common Gold Joined ------------------------------------------ KL distance : 0.822 0.831 0.899 JS divergence : 0.157 0.158 0.177 cross entropy : 2.672 2.685 2.749 skew divergence : 0.785 0.794 0.858 rank correlation : 0.714 0.714 0.649 intersection : 1.000 1.000 0.556
fly
10 SCFs in both lexicon and gold-standard Rank Accuracy: 92.86% Type Precision: 80.00% Type Recall: 80.00% F Measure: 80.00 Common Gold Joined ------------------------------------------ KL distance : 0.252 0.252 0.327 JS divergence : 0.052 0.052 0.076 cross entropy : 2.159 2.159 2.234 skew divergence : 0.244 0.244 0.318 rank correlation : 0.894 0.894 0.788 intersection : 1.000 1.000 0.667
give
9 SCFs in both lexicon and gold-standard Rank Accuracy: 90.00% Type Precision: 62.50% Type Recall: 55.56% F Measure: 58.82 Common Gold Joined ------------------------------------------ KL distance : 0.505 0.388 0.733 JS divergence : 0.122 0.117 0.185 cross entropy : 1.793 1.649 2.021 skew divergence : 0.493 0.378 0.715 rank correlation : 0.749 0.749 0.497 intersection : 1.000 1.000 0.545
lend
9 SCFs in both lexicon and gold-standard Rank Accuracy: 80.00% Type Precision: 83.33% Type Recall: 55.56% F Measure: 66.67 Common Gold Joined ------------------------------------------ KL distance : 0.585 0.590 0.809 JS divergence : 0.126 0.126 0.186 cross entropy : 2.092 2.098 2.316 skew divergence : 0.567 0.572 0.785 rank correlation : 0.928 0.928 0.596 intersection : 1.000 1.000 0.581
lock
9 SCFs in both lexicon and gold-standard Rank Accuracy: 57.14% Type Precision: 100.00% Type Recall: 77.78% F Measure: 87.50 Common Gold Joined ------------------------------------------ KL distance : 0.617 0.615 0.642 JS divergence : 0.134 0.133 0.141 cross entropy : 2.264 2.261 2.289 skew divergence : 0.596 0.594 0.620 rank correlation : 0.644 0.644 0.796 intersection : 1.000 1.000 0.720
marry
7 SCFs in both lexicon and gold-standard Rank Accuracy: 83.33% Type Precision: 80.00% Type Recall: 50.00% F Measure: 61.54 Common Gold Joined ------------------------------------------ KL distance : 0.171 -0.086 0.586 JS divergence : 0.038 0.042 0.165 cross entropy : 1.130 0.849 1.545 skew divergence : 0.167 -0.086 0.574 rank correlation : 0.800 0.800 0.499 intersection : 1.000 1.000 0.483
meet
9 SCFs in both lexicon and gold-standard Rank Accuracy: 66.67% Type Precision: 60.00% Type Recall: 33.33% F Measure: 42.86 Common Gold Joined ------------------------------------------ KL distance : 0.170 0.171 0.482 JS divergence : 0.045 0.045 0.142 cross entropy : 1.027 1.028 1.340 skew divergence : 0.167 0.168 0.474 rank correlation : 0.817 0.817 0.317 intersection : 1.000 1.000 0.581
mix
10 SCFs in both lexicon and gold-standard Rank Accuracy: 85.71% Type Precision: 88.89% Type Recall: 80.00% F Measure: 84.21 Common Gold Joined ------------------------------------------ KL distance : 0.127 0.126 0.181 JS divergence : 0.032 0.032 0.049 cross entropy : 1.860 1.858 1.914 skew divergence : 0.124 0.123 0.177 rank correlation : 0.914 0.914 0.782 intersection : 1.000 1.000 0.690
move
13 SCFs in both lexicon and gold-standard Rank Accuracy: 86.11% Type Precision: 100.00% Type Recall: 69.23% F Measure: 81.82 Common Gold Joined ------------------------------------------ KL distance : 0.361 0.359 0.394 JS divergence : 0.067 0.067 0.077 cross entropy : 2.460 2.455 2.493 skew divergence : 0.344 0.341 0.376 rank correlation : 0.784 0.784 0.744 intersection : 1.000 1.000 0.812
offer
7 SCFs in both lexicon and gold-standard Rank Accuracy: 90.00% Type Precision: 71.43% Type Recall: 71.43% F Measure: 71.43 Common Gold Joined ------------------------------------------ KL distance : 0.068 0.064 0.229 JS divergence : 0.013 0.013 0.066 cross entropy : 1.441 1.436 1.603 skew divergence : 0.065 0.061 0.224 rank correlation : 0.821 0.821 0.696 intersection : 1.000 1.000 0.519
provide
10 SCFs in both lexicon and gold-standard Rank Accuracy: 100.00% Type Precision: 66.67% Type Recall: 40.00% F Measure: 50.00 Common Gold Joined ------------------------------------------ KL distance : 0.215 0.212 0.404 JS divergence : 0.039 0.039 0.098 cross entropy : 1.393 1.390 1.581 skew divergence : 0.204 0.202 0.389 rank correlation : 0.839 0.839 0.134 intersection : 1.000 1.000 0.588
push
13 SCFs in both lexicon and gold-standard Rank Accuracy: 80.56% Type Precision: 69.23% Type Recall: 64.29% F Measure: 66.67 Common Gold Joined ------------------------------------------ KL distance : 0.613 0.702 0.779 JS divergence : 0.134 0.139 0.159 cross entropy : 2.543 2.681 2.761 skew divergence : 0.593 0.638 0.713 rank correlation : 0.812 0.784 0.509 intersection : 1.000 0.963 0.650
sail
7 SCFs in both lexicon and gold-standard Rank Accuracy: 95.24% Type Precision: 63.64% Type Recall: 100.00% F Measure: 77.78 Common Gold Joined ------------------------------------------ KL distance : 0.036 0.035 0.180 JS divergence : 0.009 0.009 0.057 cross entropy : 1.503 1.502 1.647 skew divergence : 0.035 0.034 0.178 rank correlation : 0.955 0.955 0.769 intersection : 1.000 1.000 0.483
send
13 SCFs in both lexicon and gold-standard Rank Accuracy: 70.91% Type Precision: 78.57% Type Recall: 84.62% F Measure: 81.48 Common Gold Joined ------------------------------------------ KL distance : 0.573 0.562 0.602 JS divergence : 0.122 0.122 0.131 cross entropy : 2.857 2.837 2.886 skew divergence : 0.551 0.541 0.580 rank correlation : 0.752 0.752 0.799 intersection : 1.000 1.000 0.703
slice
10 SCFs in both lexicon and gold-standard Rank Accuracy: 80.00% Type Precision: 66.67% Type Recall: 60.00% F Measure: 63.16 Common Gold Joined ------------------------------------------ KL distance : 0.209 0.207 0.316 JS divergence : 0.049 0.049 0.083 cross entropy : 1.953 1.950 2.060 skew divergence : 0.203 0.201 0.308 rank correlation : 0.756 0.756 0.633 intersection : 1.000 1.000 0.741
supply
6 SCFs in both lexicon and gold-standard Rank Accuracy: 70.00% Type Precision: 83.33% Type Recall: 83.33% F Measure: 83.33 Common Gold Joined ------------------------------------------ KL distance : 0.225 0.217 0.294 JS divergence : 0.044 0.044 0.066 cross entropy : 1.540 1.531 1.609 skew divergence : 0.216 0.209 0.284 rank correlation : 0.771 0.771 0.739 intersection : 1.000 1.000 0.429
swing
12 SCFs in both lexicon and gold-standard Rank Accuracy: 66.67% Type Precision: 100.00% Type Recall: 83.33% F Measure: 90.91 Common Gold Joined ------------------------------------------ KL distance : 0.650 0.649 0.688 JS divergence : 0.129 0.129 0.139 cross entropy : 2.841 2.838 2.879 skew divergence : 0.625 0.623 0.661 rank correlation : 0.615 0.615 0.827 intersection : 1.000 1.000 0.727
travel
11 SCFs in both lexicon and gold-standard Rank Accuracy: 75.00% Type Precision: 80.00% Type Recall: 72.73% F Measure: 76.19 Common Gold Joined ------------------------------------------ KL distance : 0.261 0.259 0.377 JS divergence : 0.054 0.054 0.090 cross entropy : 2.012 2.010 2.129 skew divergence : 0.252 0.251 0.367 rank correlation : 0.679 0.679 0.629 intersection : 1.000 1.000 0.647
visit
4 SCFs in both lexicon and gold-standard Rank Accuracy: 66.67% Type Precision: 60.00% Type Recall: 75.00% F Measure: 66.67 Common Gold Joined ------------------------------------------ KL distance : 0.127 0.118 0.352 JS divergence : 0.039 0.039 0.112 cross entropy : 0.597 0.593 0.823 skew divergence : 0.125 0.116 0.347 rank correlation : 0.400 0.400 0.524 intersection : 1.000 1.000 0.296