RESULTS FOR BRISCOE AND CARROLL'S SYSTEM

Average results for the 30 verbs

 
Rank Accuracy:  80.10%
Type Precision: 75.29%
Type Recall:    67.13%
F Measure:      70.98


Average similarity values between the acquired (q)
and gold standard (p) SCF distributions:

                      Common  Gold  Joined
------------------------------------------
KL distance         :  0.363  0.394  0.546
JS divergence       :  0.079  0.080  0.122
cross entropy       :  1.888  1.927  2.082
skew divergence     :  0.351  0.356  0.505
rank correlation    :  0.779  0.775  0.609
intersection        :  1.000  0.991  0.606


Common = SCFs which occur in both 
         p and q are considered
Gold   = only those SCFs which occur
         in p are considered
Joined = SCFs which occur in either 
         p or q are considered

Results for the individual verbs

add
7 SCFs in both lexicon and gold standard

Rank Accuracy:  100.00%
Type Precision: 60.00%
Type Recall:    66.67%
F Measure:      63.16

                      Common  Gold  Joined
------------------------------------------
KL distance         :  0.046  1.193  1.296
JS divergence       :  0.012  0.057  0.093
cross entropy       :  1.480  2.853  2.956
skew divergence     :  0.045  0.509  0.611
rank correlation    :  0.893  0.727  0.464
intersection        :  1.000  0.875  0.500
agree
14 SCFs in both lexicon and gold-standard

Rank Accuracy:  80.00%
Type Precision: 85.71%
Type Recall:    42.86%
F Measure:      57.14

                      Common  Gold  Joined
------------------------------------------
KL distance         :  0.543  0.535  0.645
JS divergence       :  0.101  0.101  0.130
cross entropy       :  2.625  2.612  2.727
skew divergence     :  0.519  0.512  0.618
rank correlation    :  0.824  0.824  0.529
intersection        :  1.000  1.000  0.683
attach
5 SCFs in both lexicon and gold-standard

Rank Accuracy:  66.67%
Type Precision: 42.86%
Type Recall:    60.00%
F Measure:      50.00

                      Common  Gold  Joined
------------------------------------------
KL distance         :  0.353  0.354  0.543
JS divergence       :  0.094  0.094  0.150
cross entropy       :  0.993  0.994  1.183
skew divergence     :  0.346  0.347  0.532
rank correlation    :  0.800  0.800  0.612
intersection        :  1.000  1.000  0.526
bring
11 SCFs in both lexicon and gold-standard

Rank Accuracy:  57.14%
Type Precision: 63.64%
Type Recall:    63.64%
F Measure:      63.64

                      Common  Gold  Joined
------------------------------------------
KL distance         :  1.347  1.340  1.482
JS divergence       :  0.270  0.270  0.298
cross entropy       :  3.131  3.121  3.266
skew divergence     :  1.286  1.280  1.412
rank correlation    :  0.433  0.433  0.491
intersection        :  1.000  1.000  0.611
carry
11 SCFs in both lexicon and gold-standard

Rank Accuracy:  66.67%
Type Precision: 75.00%
Type Recall:    81.82%
F Measure:      78.26

                      Common  Gold  Joined
------------------------------------------
KL distance         :  0.518  0.513  0.578
JS divergence       :  0.122  0.122  0.139
cross entropy       :  1.932  1.926  1.992
skew divergence     :  0.504  0.499  0.562
rank correlation    :  0.572  0.572  0.662
intersection        :  1.000  1.000  0.579
carve
12 SCFs in both lexicon and gold-standard

Rank Accuracy:  85.71%
Type Precision: 100.00%
Type Recall:    58.33%
F Measure:      73.68

                      Common  Gold  Joined
------------------------------------------
KL distance         :  0.138  0.141  0.154
JS divergence       :  0.038  0.038  0.043
cross entropy       :  1.778  1.782  1.794
skew divergence     :  0.135  0.138  0.151
rank correlation    :  0.920  0.920  0.811
intersection        :  1.000  1.000  0.889
chop
9 SCFs in both lexicon and gold-standard

Rank Accuracy:  95.24%
Type Precision: 70.00%
Type Recall:    77.78%
F Measure:      73.68

                      Common  Gold  Joined
------------------------------------------
KL distance         :  0.226  0.226  0.366
JS divergence       :  0.051  0.051  0.095
cross entropy       :  1.876  1.876  2.017
skew divergence     :  0.220  0.220  0.358
rank correlation    :  0.845  0.845  0.717
intersection        :  1.000  1.000  0.621
cling
6 SCFs in both lexicon and gold-standard

Rank Accuracy:  100.00%
Type Precision: 37.50%
Type Recall:    50.00%
F Measure:      42.86

                      Common  Gold  Joined
------------------------------------------
KL distance         :  0.222  0.221  0.772
JS divergence       :  0.063  0.063  0.224
cross entropy       :  0.823  0.822  1.373
skew divergence     :  0.218  0.217  0.758
rank correlation    :  0.986  0.986  0.163
intersection        :  1.000  1.000  0.571
clip
9 SCFs in both lexicon and gold-standard

Rank Accuracy:  85.71%
Type Precision: 87.50%
Type Recall:    77.78%
F Measure:      82.35

                      Common  Gold  Joined
------------------------------------------
KL distance         :  0.220  0.220  0.265
JS divergence       :  0.053  0.053  0.067
cross entropy       :  1.886  1.886  1.931
skew divergence     :  0.214  0.214  0.259
rank correlation    :  0.848  0.848  0.772
intersection        :  1.000  1.000  0.720
communicate
8 SCFs in both lexicon and gold-standard

Rank Accuracy:  66.67%
Type Precision: 80.00%
Type Recall:    40.00%
F Measure:      53.33

                      Common  Gold  Joined
------------------------------------------
KL distance         :  0.484  0.577  0.743
JS divergence       :  0.101  0.106  0.154
cross entropy       :  1.900  2.063  2.230
skew divergence     :  0.461  0.510  0.670
rank correlation    :  0.790  0.853  0.305
intersection        :  1.000  0.889  0.457
cut
9 SCFs in both lexicon and gold-standard

Rank Accuracy:  76.19%
Type Precision: 100.00%
Type Recall:    77.78%
F Measure:      87.50

                      Common  Gold  Joined
------------------------------------------
KL distance         :  0.224  0.224  0.263
JS divergence       :  0.050  0.050  0.063
cross entropy       :  2.072  2.072  2.112
skew divergence     :  0.218  0.218  0.256
rank correlation    :  0.820  0.820  0.831
intersection        :  1.000  1.000  0.600
drag
10 SCFs in both lexicon and gold-standard

Rank Accuracy:  86.11%
Type Precision: 69.23%
Type Recall:    90.00%
F Measure:      78.26

                      Common  Gold  Joined
------------------------------------------
KL distance         :  0.822  0.831  0.899
JS divergence       :  0.157  0.158  0.177
cross entropy       :  2.672  2.685  2.749
skew divergence     :  0.785  0.794  0.858
rank correlation    :  0.714  0.714  0.649
intersection        :  1.000  1.000  0.556
fly
10 SCFs in both lexicon and gold-standard

Rank Accuracy:  92.86%
Type Precision: 80.00%
Type Recall:    80.00%
F Measure:      80.00

                      Common  Gold  Joined
------------------------------------------
KL distance         :  0.252  0.252  0.327
JS divergence       :  0.052  0.052  0.076
cross entropy       :  2.159  2.159  2.234
skew divergence     :  0.244  0.244  0.318
rank correlation    :  0.894  0.894  0.788
intersection        :  1.000  1.000  0.667
give
9 SCFs in both lexicon and gold-standard

Rank Accuracy:  90.00%
Type Precision: 62.50%
Type Recall:    55.56%
F Measure:      58.82

                      Common  Gold  Joined
------------------------------------------
KL distance         :  0.505  0.388  0.733
JS divergence       :  0.122  0.117  0.185
cross entropy       :  1.793  1.649  2.021
skew divergence     :  0.493  0.378  0.715
rank correlation    :  0.749  0.749  0.497
intersection        :  1.000  1.000  0.545
lend
9 SCFs in both lexicon and gold-standard

Rank Accuracy:  80.00%
Type Precision: 83.33%
Type Recall:    55.56%
F Measure:      66.67

                      Common  Gold  Joined
------------------------------------------
KL distance         :  0.585  0.590  0.809
JS divergence       :  0.126  0.126  0.186
cross entropy       :  2.092  2.098  2.316
skew divergence     :  0.567  0.572  0.785
rank correlation    :  0.928  0.928  0.596
intersection        :  1.000  1.000  0.581
lock
9 SCFs in both lexicon and gold-standard

Rank Accuracy:  57.14%
Type Precision: 100.00%
Type Recall:    77.78%
F Measure:      87.50

                      Common  Gold  Joined
------------------------------------------
KL distance         :  0.617  0.615  0.642
JS divergence       :  0.134  0.133  0.141
cross entropy       :  2.264  2.261  2.289
skew divergence     :  0.596  0.594  0.620
rank correlation    :  0.644  0.644  0.796
intersection        :  1.000  1.000  0.720
marry
7 SCFs in both lexicon and gold-standard

Rank Accuracy:  83.33%
Type Precision: 80.00%
Type Recall:    50.00%
F Measure:      61.54

                      Common  Gold  Joined
------------------------------------------
KL distance         :  0.171 -0.086  0.586
JS divergence       :  0.038  0.042  0.165
cross entropy       :  1.130  0.849  1.545
skew divergence     :  0.167 -0.086  0.574
rank correlation    :  0.800  0.800  0.499
intersection        :  1.000  1.000  0.483
meet
9 SCFs in both lexicon and gold-standard

Rank Accuracy:  66.67%
Type Precision: 60.00%
Type Recall:    33.33%
F Measure:      42.86

                      Common  Gold  Joined
------------------------------------------
KL distance         :  0.170  0.171  0.482
JS divergence       :  0.045  0.045  0.142
cross entropy       :  1.027  1.028  1.340
skew divergence     :  0.167  0.168  0.474
rank correlation    :  0.817  0.817  0.317
intersection        :  1.000  1.000  0.581
mix
10 SCFs in both lexicon and gold-standard

Rank Accuracy:  85.71%
Type Precision: 88.89%
Type Recall:    80.00%
F Measure:      84.21

                      Common  Gold  Joined
------------------------------------------
KL distance         :  0.127  0.126  0.181
JS divergence       :  0.032  0.032  0.049
cross entropy       :  1.860  1.858  1.914
skew divergence     :  0.124  0.123  0.177
rank correlation    :  0.914  0.914  0.782
intersection        :  1.000  1.000  0.690
move
13 SCFs in both lexicon and gold-standard

Rank Accuracy:  86.11%
Type Precision: 100.00%
Type Recall:    69.23%
F Measure:      81.82

                      Common  Gold  Joined
------------------------------------------
KL distance         :  0.361  0.359  0.394
JS divergence       :  0.067  0.067  0.077
cross entropy       :  2.460  2.455  2.493
skew divergence     :  0.344  0.341  0.376
rank correlation    :  0.784  0.784  0.744
intersection        :  1.000  1.000  0.812
offer
7 SCFs in both lexicon and gold-standard

Rank Accuracy:  90.00%
Type Precision: 71.43%
Type Recall:    71.43%
F Measure:      71.43

                      Common  Gold  Joined
------------------------------------------
KL distance         :  0.068  0.064  0.229
JS divergence       :  0.013  0.013  0.066
cross entropy       :  1.441  1.436  1.603
skew divergence     :  0.065  0.061  0.224
rank correlation    :  0.821  0.821  0.696
intersection        :  1.000  1.000  0.519
provide
10 SCFs in both lexicon and gold-standard

Rank Accuracy:  100.00%
Type Precision: 66.67%
Type Recall:    40.00%
F Measure:      50.00

                      Common  Gold  Joined
------------------------------------------
KL distance         :  0.215  0.212  0.404
JS divergence       :  0.039  0.039  0.098
cross entropy       :  1.393  1.390  1.581
skew divergence     :  0.204  0.202  0.389
rank correlation    :  0.839  0.839  0.134
intersection        :  1.000  1.000  0.588
push
13 SCFs in both lexicon and gold-standard

Rank Accuracy:  80.56%
Type Precision: 69.23%
Type Recall:    64.29%
F Measure:      66.67

                      Common  Gold  Joined
------------------------------------------
KL distance         :  0.613  0.702  0.779
JS divergence       :  0.134  0.139  0.159
cross entropy       :  2.543  2.681  2.761
skew divergence     :  0.593  0.638  0.713
rank correlation    :  0.812  0.784  0.509
intersection        :  1.000  0.963  0.650
sail
7 SCFs in both lexicon and gold-standard

Rank Accuracy:  95.24%
Type Precision: 63.64%
Type Recall:    100.00%
F Measure:      77.78

                      Common  Gold  Joined
------------------------------------------
KL distance         :  0.036  0.035  0.180
JS divergence       :  0.009  0.009  0.057
cross entropy       :  1.503  1.502  1.647
skew divergence     :  0.035  0.034  0.178
rank correlation    :  0.955  0.955  0.769
intersection        :  1.000  1.000  0.483
send
13 SCFs in both lexicon and gold-standard

Rank Accuracy:  70.91%
Type Precision: 78.57%
Type Recall:    84.62%
F Measure:      81.48

                      Common  Gold  Joined
------------------------------------------
KL distance         :  0.573  0.562  0.602
JS divergence       :  0.122  0.122  0.131
cross entropy       :  2.857  2.837  2.886
skew divergence     :  0.551  0.541  0.580
rank correlation    :  0.752  0.752  0.799
intersection        :  1.000  1.000  0.703
slice
10 SCFs in both lexicon and gold-standard

Rank Accuracy:  80.00%
Type Precision: 66.67%
Type Recall:    60.00%
F Measure:      63.16

                      Common  Gold  Joined
------------------------------------------
KL distance         :  0.209  0.207  0.316
JS divergence       :  0.049  0.049  0.083
cross entropy       :  1.953  1.950  2.060
skew divergence     :  0.203  0.201  0.308
rank correlation    :  0.756  0.756  0.633
intersection        :  1.000  1.000  0.741
supply
6 SCFs in both lexicon and gold-standard

Rank Accuracy:  70.00%
Type Precision: 83.33%
Type Recall:    83.33%
F Measure:      83.33

                      Common  Gold  Joined
------------------------------------------
KL distance         :  0.225  0.217  0.294
JS divergence       :  0.044  0.044  0.066
cross entropy       :  1.540  1.531  1.609
skew divergence     :  0.216  0.209  0.284
rank correlation    :  0.771  0.771  0.739
intersection        :  1.000  1.000  0.429
swing
12 SCFs in both lexicon and gold-standard

Rank Accuracy:  66.67%
Type Precision: 100.00%
Type Recall:    83.33%
F Measure:      90.91

                      Common  Gold  Joined
------------------------------------------
KL distance         :  0.650  0.649  0.688
JS divergence       :  0.129  0.129  0.139
cross entropy       :  2.841  2.838  2.879
skew divergence     :  0.625  0.623  0.661
rank correlation    :  0.615  0.615  0.827
intersection        :  1.000  1.000  0.727
travel
11 SCFs in both lexicon and gold-standard

Rank Accuracy:  75.00%
Type Precision: 80.00%
Type Recall:    72.73%
F Measure:      76.19

                      Common  Gold  Joined
------------------------------------------
KL distance         :  0.261  0.259  0.377
JS divergence       :  0.054  0.054  0.090
cross entropy       :  2.012  2.010  2.129
skew divergence     :  0.252  0.251  0.367
rank correlation    :  0.679  0.679  0.629
intersection        :  1.000  1.000  0.647
visit
4 SCFs in both lexicon and gold-standard

Rank Accuracy:  66.67%
Type Precision: 60.00%
Type Recall:    75.00%
F Measure:      66.67

                      Common  Gold  Joined
------------------------------------------
KL distance         :  0.127  0.118  0.352
JS divergence       :  0.039  0.039  0.112
cross entropy       :  0.597  0.593  0.823
skew divergence     :  0.125  0.116  0.347
rank correlation    :  0.400  0.400  0.524
intersection        :  1.000  1.000  0.296