General Evaluation of a Qualitative Representation

One of the main goals in developing the representation described in this thesis was to carry out spatial reasoning without any numerical geometric processing, and without reducing the spatial content of sensory information. The methods described in the last two chapters are successful to the degree that they can perform some reasoning of this type, but it is also necessary to evaluate whether they provide new capabilities when compared to existing systems, and how useful they can be in more general applications.

The evaluation of general purpose representations is not easy in any area of computer application, because it is difficult to separate considerations that apply uniquely to a given problem or domain from those that apply to any problem. In practice, such evaluation can only be carried out by testing the representation in a range of systems. The following list proposes a number of evaluation criteria which are important features of a good qualitative representation, but this list cannot be exhaustive until substantially more is known about qualitative reasoning.

Some evaluation criteria for a qualitative representation are:

1.

What new facilities does the representation provide?

2.

How wide is its application domain?

3.

Does it seem to be intuitively accurate?

4.

Can it be related to a known body of theory?

5.

Is it consistent with other qualitative methods?

6.

How does it perform in comparison to conventional methods?

7.

Does it allow the integration of numeric information when necessary?

8.

Is it easy to match and compare descriptions of states?

9.

Is it easy to match and compare descriptions of processes?

When considered in terms of these criteria for evaluating qualitative representations, the following points can be noted about the PDO/EPB representation:

• PDO/EPB provides new qualitative methods for representing shape and relative position of objects. It can be used for spatial reasoning at a level that gives the reasoning system direct access to the scene description data in qualitative form. The description is at a level that is close to sensory data, and therefore includes no implicit functional information - from a qualitative point of view, it is purely a structural description.
• PDO/EPB is applicable in its present form to any two dimensional spatial reasoning problem. It can be used to plan and predict the effects of motion, but cannot directly represent information about process or influence.
• The extended polygon boundary description is a more intuitive way of describing an object boundary than methods which do not decompose shape into qualitative elements. As a description of objects in general, it does not really correspond to our intuitive impressions as well as the ASSF scheme - humans seem to deal with complete objects in terms of mass and area, rather than boundaries. The representation of relative position in terms of proximity is very intuitive.
• The type of geometry employed by a reasoning system that uses PDO/EPB is more similar to Euclidean geometry than Cartesian geometry. Most computational geometry systems employ Cartesian representations, so Cartesian geometry is more familiar in computer applications. There is, however, a large body of geometry theory that uses non-numeric Euclidean methods. Descriptions of shape in terms of changing proximity can make use of such methods.
• The PDO/EPB representation provides a two dimensional analogue to the quantity space by the use of the proximity ordering. For reasoning about motion and constraint, proximity is an important ``quantity''. There are no ``distinguished points'' in the proximity quantity space, so it is constructed as a partial ordering for each scene, with no absolute values. The angle representation currently uses a simple linear quantity space. The types of operation carried out in both the direction and proximity spaces are normal qualitative reasoning techniques.
• The performance of the PDO/EPB representation in comparison to conventional numeric techniques is discussed in more detail in the next section, where conventional spatial reasoning methods for robots are reconsidered. The qualitative methods are advantageous when data is inexact or missing, while a system with access to exact numeric data is able to solve many problems that a purely qualitative system could not.
• Numeric information can be integrated into the partial distance ordering, as discussed in chapter 4. Using a partial distance ordering with integrated numeric data would allow a reasoning system to use numeric techniques wherever approriate. Flow of information between numeric and qualitative reasoning components could be achieved by creating or adjusting entries in the ordering.
• The qualitative state of a system under analysis is described solely by the proximity ordering. Changes in state resulting from perturbations to the system (motion of objects) are reflected in a new partial distance ordering. The system described in the previous chapter is able to carry out an envisionment process by postulating future states that can arise from the current one. State matching or comparison can be carried out simply by extracting a subset of the ordering that relates to objects of interest. The sliding system actually carried out state matching for recurring contact states across the whole global contact set.
• The reasoning system based on the PDO/EPB representation that is described in the last chapter maintains no explicit record of process. Process could be described as a sequence of states, but the process record would be completely implicit, making it very difficult to compare different processes. Explicit representation of process and influence, which are basic parts of qualitative physics, must be added as a different level of reasoning to the analysis of motion and constraint that can be carried out with PDO/EPB.

In summary, the PDO/EPB representation has most of the features that are expected of a representation for qualitative reasoning, and it can be applied to problems that have been used in the past to test qualitative spatial reasoning. It is able to answer a range of questions about object motion without making use of numeric data, by retaining geometric information in its qualitative scene description.

Although it is unable to explicitly represent influence or process (and cannot therefore be called a complete qualitative physics system), it does support a description of qualitative system state. The strategies used for problem solving using the representation seem intuitive and natural to humans, which make them a good basis for ``commonsense'' reasoning systems.