The Development of Feature Spaces for Similarity and Categorization

Similarity and categorization are usually formulated as operations over a set of available features at time t of conceptual development. In this paper, we discuss possible constraints on the development of the feature space such as the history of categorization, perceptual biases in the selection and the construction of features, and the role of higherlevel knowledge and beliefs. The history of categorization is defined as the feature vocabulary and the set of concepts people have acquired as a result of representing and categorizing objects. Perceptual and developmental biases refer to the salience of specific aspects of the stimuli which influence the selection and the construction of features at different stages of development. The constraining role of general knowledge on the selection and the creation of features is also discussed. It is argued that theories may need the constraints they are supposed to provide. We conclude by discussing issues related to the interactions of perceptions and conceptions in developing feature spaces for higher-level cognition.

development.We present features as constructs grounded in the history of the categorizer; as solutions to the problem of distinguishing objects that were grouped together prior to exprience with a particular categorization.Obviously, features do not arise from thin air but they are constrained by the nature of our perceptual structures and by sophisticated knowledge and beliefs about relevant events in the world.Thus, this paper examines how low-and high-level constraints participate to the development of feature spaces for categorization.

Fixed and augmented feature spaces
Categorization theories conceive of stimuli as feature bundles.Each feature participates in the categorization task by distinguishing the stimuli it describes from other stimuli.For example, has-legs distinguishes objects having legs from objects without legs.
Similarly, is-neurotic distinguishes many scientists from other people.Together, the features used in a particular similarity or categorization task define the space for similarity judgments and categorizations.That is, the feature space specifies the ways in which two objects are similar or different.For example, the feature set F1= {green, red, square, circle} specifies a feature space in which one shape distinction and one color distinction can be made.Obviously, other features should be added to the set to represent other similarities or distinctions.For example, the addition of has-bricks could expand the description space from the simple objects red square or green circle to a red squared building made of bricks whose dome is circular and green.The addition of is-made-of-plastic vs. is-made-of-clay to F2= {green, red, square, circle, has-bricks} could distinguish a construction-game building from a real building.This simple example captures the gist of the notion that the object features available in the feature set constrain the similarities and differences that are representable.
We would like to generalize these ideas and suggest that at any stage t of conceptual development, the feature vocabulary Ft = {f1, ..., fn} defines the space for similarity judgments and categorizations.Et, the set of combinations of Ft features enumerates all possible object representations at time t--i.e., E t = {-, f 1 , f 2 , ..., f n , f 1 ^ f 2 , ..., f 1 ^ f n , f 2 ^ f 3 , ..., f 2 ^ f n , ..., f 1 ^ f 2 ^ ... ^ f n }, where "^" means the logical and.Of these objects, Ct, a subset of Et, enumerates the concepts known to the categorizer at time t.Pt, the set of potential object descriptions enumerates the objects which are members of Et minus Ct. Figure 1 illustrates these ideas.
There are serious difficulties with Fixed Space development (see Schyns, Goldstone & Thibaut, 1995, for a thorough discussion).Because Ft and Et do not evolve (only Ct does), Ft must represent all possible categorical distinctions prior to experience with real world categories.That is, the system never learns a concept that is not already a member of Et.However, a complete explanation of conceptual development must not only explain how Et featural descriptions become Ct object concepts, but it must also explain how the features were initially included in Et.
Features have an important functional property: They distinguish between object categories.Working backwards, it could be that the requirement of distinguishing between object categories induces the creation of new stimulus dimensions--the dimensions solving the categorization problem.At time t+1 of conceptual development, the new feature representing a new similarity or difference increases the dimensionality of the feature space--i.e., Ft+1 = {f1, ..., fn, fn+1}.With the combinatorics of a componential approach to representation, each new feature augments the repertoire of expressible concepts Et+1/Et = 2n+1/2n (Schyns & Rodet, 1995), and the representational capacities of the categorizer.In opposition to the Fixed view of development, we call this conception "Augmented Space" development.
However, not all Et feature combinations are equally plausible.At time t, only a subset of the combinations are explainable by people's theories and sophisticated knowledge.Many feature combinations explainable by these theories are actual Ct concepts, but it is conceivable that several feature combinations that may be derived of a theory are not actual concepts.It is also conceivable that many Et feature combinations are simply inconsistent with the available theories and beliefs, and that they will only be explained when theories develop to encompass these incoherencies.In any case, many Et feature combinations will necessarily remain incoherent (for example because they contradict existing knowledge).In what follows, we discuss how perception, theories and categorical structure cooperate to construct the feature spaces of categorization.

History of categorization
Differences between fixed and augmented space concept learning were studied in Rodet and Schyns (1994).Subjects were presented with categories of synthesized 2D images that looked like biological cells seen through a microscope (the Martian cells, see Figure 2).Each category was defined with a particular configuration of three cell bodies, which we call x, y and z, for simplicity of presentation--not because it is the way subjects perceived them.In an initial learning phase, two groups of subjects learned the context of two Martian cell categories.The X->Y group learned the X category before the Y category; the X->XY group learned X before XY.The categories were constructed to ensure that the groups would create the same feature vocabulary--namely Ft = {x, y}--but would differ on the concepts (the feature combinations) they learned.The first group would learn two concepts composed of a single feature, while the second group should represent the second category with the feature conjunction x and y.
In a second learning phase (at time t+1 Constraining the creation of a feature space from the bottom: The role of perception Not all portions of an object are as likely to become relevant categorization features. We saw how the task of distinguishing a new object category could induce the creation of new object features.However, object features must be grounded on perception and on the other capabilities of the organism at time t.Schyns and Murphy (1994) tested the role of two perceptual constraints on the formation of object features.When people segment an object into its parts, they often parse the object loci of deep concavities (technically the minima of Gaussian curvature, see Hoffman & Richards, 1984).In Schyns and Murphy's (1994) experiments, subjects were asked to learn categories of objects, and were later tested on how they segmented the objects into their parts.The categories were continuous three-dimensional shapes (the "Martian rocks") composed of blobs (see Figure 2).The Martian rocks had a complex blob structure and so subjects showed no agreement on what were the parts of the rocks before experience with the categories.Categories were defined by a blobby part that was common to all category exemplars.After learning the categories, subjects were asked to segment the Martian rocks into their relevant parts.It was shown that the perceptual principle of decomposing objects at minima of curvature interacted with categorical experience.In one group, two parts separated by a valley were perceived as a whole.In another group, the whole was segmented into its subparts because subjects had previously learned that these subparts categorized other objects.Thus, subjects learned a feature vocabulary tailored to their own categorization requirements, instead of a purely perceptually-driven vocabulary.
However, perceptual constraints played a role in that the parts subjects found relevant were always bounded by minima of curvature.Thus, not every portion of an object is equally likely to become a feature of that object. -

Perceptual biases on feature development
The higher-level integration of perceptual aspects of stimuli can also change with development and can affect the features that are included in the feature space.For example, Thibaut (1995a) compared adults and children aged 4 and 6 in a task involving the discovery of the relevant features defining a category.The stimuli shared the same global (overall) shape.They were composed of a common set of shape features which varied slightly accross exemplars (see Figure 3).
That is, children gave more importance than adults to the superficial similarities between local cues, somewhat irrespectively of the global organization of the objects.
A second set of studies tested the consequences of these feature selection biases on category learning.Thibaut (1995c) hypothesized that the child's difficulty in differentiating the fine structure of local features could impair the abstraction of the shape vocabulary objectively defining the categories.That is, children's bias towards perceptual properties such as the orientation, size and shape of the parts (that were irrelevant for categorization) would impair the abstraction of the relevant categorization features.Two groups of six-year old children were asked to categorize an identical set of stimuli.In the orientation-size of legs group, the relevant categirization criterion was based on the orientation and size of the "legs" of the stimuli.In a first category, the criterion was "vertical and large legs" while "thin legs and and one leg oriented to the right" was the criterion of the second category.
Thegrouping of legs group learned to discriminate the same set of stimuli according to two different criteria: "a-group-of-three-legs-plus-one" for the first category and "two-groups-oftwo-legs" for the second category.
The first group learned to discriminate the 2 categories based on perceptual cues while the second group could not learn the criterion based on the grouping (see Figure 4).In a follow-up experiment, it was shown that these children could learn to discriminate the categories from the grouping condition when the perceptual cues were removed--i.e., when all the stimuli had vertical legs of the same size.These results emphasize the interaction between the development of a feature vocabulary and stage-specific perceptual biases over the course of perceptual development.Children cannot abstract relevant categorization cues when there are irrelevant perceptual characteristics are crossed with these cues, contrary to adults who were not influenced by these biases.When these perceptual characteristics are removed children can learn the rule, meaning the rule itself is not intrinsically difficult to learn by children (see Jones & Smith, 1993;Gentner & Ratterman, 1991).
---------------------------------Insert Figure 4 around here -------------------------------- In our view, a dissociation between similarity and categorization could arise from a difference in the respects that are used for comparisons.Categorization would compare the input's featural description to the Ct feature combinations that are actual concepts in memory.Similarity judgments could compare objects with respect to expressible combinations of Ft features--i.e., to any member of Et.Empirically, such dissociation should be obtained if an object A is categorized in X instead of Y, but judged to be more similar to Y than to X or judged dissimilar to X.
We recently tested this hypothesis by inducing the learning of different feature vocabularies and concepts in two groups.In an initial learning phase, two groups of subjects learned two Martian Landscape categories (see Figure 5), one category at a time.Categories were defined with criterial craters and random craters in the landscapes.The X->Y group learned the X category before the Y category.The X->XY group learned X before XY.
In a second learning phase, subjects were shown a third XYZ category, defined by the feature combination x^y^z.We expected that the featural contrasts and similarities fixed by We then compared the groups on their categorizations and similarity judgments of a target XZ category.XZ stimuli were not a concept subjects knew, but were describable as x^z with FX->XY->XYZ, but only as x with F X->Y->XYZ.It was found that the group having the z feature judged XZ different from X (or XYZ), but categorized XZ as an atypical instance of the X (or XYZ) category.The absence of z in the other group did not allow a dissociation of categorization and similarity.

Constraining the creation of a feature space from the top: The role of theories
So far, we have discussed the interaction between task constraints and perception on the creation of new object features.However, it has been argued that the features that are used for specific similarity judgments and categorization tasks can also be constrained by theories and sophisticated knowledge (e.g., Carey, 1985;Murphy & Medin, 1985;Murphy & Spalding, this volume).For example, the features to-have-a-roof, to-have-walls, to-havewindows, to-have-a-door in the concept house can be explained by the protective function of houses.The theory of houses as protection highlights certain features which would then be selected to group houses into categories, or to compare houses with one another.There is also empirical evidence that category learning is facilitated if an underlying theory explains the structure of the features, and that category learning is impaired when the theory contradicts the featural structure (Murphy & Allopenna, 1994;Watenmaker, Medin, & Hampson, 1986).The role of theories as sources of constraints on the construction and the development of a feature space is evaluated in the following paragraphs.
According to Keil (1989), Smith, Carey and Wiser (1985), the acquisition of new features in concepts is a consequence of the development of the theory supporting the concept.These concepts (and their features) are changed according to a differenciation of the corresponding theory.For example, Smith et al. (1985) showed that undifferentiated concepts like "weight" and "density" in young infants were articulated as distinct concepts in older children.Separate interviews conducted in the two age groups revealed that distinct theories were used at different levels of development.
If theories highlight important features of a concept for a particular domain, they provide useful constraints to set up the features that are used for similarity, categorization, and conceptual development.However, a particular object can usually be explained by more than one theory.In the house example given above, one could explain the house by refering to the history of architecture, the cultural organization of a household, geography, and so forth.Each theory would then impose different constraints on the selection of the features used in the feature space, to learn new concepts in different ways.Thus, a theory only constrains feature selection if the selection of the theory is itself constrained.
How is a theory selected to constrain feature selection?One possibility would be that the context of a particular similarity judgment task or concept learning task forces the selection of particular theories.For example, when discussing hurricanes, people might preferentially activate a theory of houses coordinating their protective properties.Another possibility would be that people have a default theory about every object or event that is applied independently of context.This theory might, for example, be the most frequently used theory, all contexts confounded.
In our view, the problem of theory selection is isomorphic to the problem of feature selection; it needs similar constraints.As we know for features, the context of an object may influence the selection process for the description of a target (e.g., Medin, Goldstone & Gentner, 1993).Out of context, for example in a feature listing task, many features are listed that are correlated across subjects (e.g., Rosch, 1978).Thus, feature selection and theory selection are both prone to the same difficulty of fixing the respects that determine their selection.Specifically, an explanation of conceptual development in terms of theories is incomplete without an explanation of the mechanisms by which a relevant theory is selected in a given task.
A related form of constraints on the feature space for concept learning and conceptual development could arise from the notion that people believe that categories are grounded on a core of essential properties (Medin & Ortony, 1989;Malt, 1990).In people's conceptions these properties define the real essence of the concept and surface properties would be causally connected to essential properties.For example, people believe that the existence of an essential genotype determines the surface phenotype.Essential properties are better defining features of a concept then surface properties.Surface properties often are misleading.For example, the whale, a mammal, could easily be mistakenly classified as a fish, on the basis of surface properties.Similarly, a penguin could be taken as mammal although it is a bird.It is important to emphasize that psychological essentialism does not mean that objects have an essence, or an ontology, or that people get to know this essence if it existed.Instead, psychological essentialism is based on a belief, the belief that objects have essential properties.Psychological essentialism does not address conceptual representations per se.It is a conception of how concepts should be--i.e., composed of essential and surface features (Malt, 1990;Malt, 1994;Malt &Johnson, 1992).

Does a belief in an essence constrain categorization or conceptual development?
According to psychological essentialism, the belief in essential properties could constrain subjects in selecting essential features for constructing a feature space.In our view, if one believes that a category has essence X, it does not imply that this person has knowledge of essence X (for example the person could think that scientists, or more knowledgeable people know the essence).At most, the belief in the essence could trigger the search of deep properties.Thus, the belief in itself does not constrain categorizations or similarity judgments, unless people already believe they have knowledge of an essence.
An argument could be made that the belief in an essence is constraining.For example, when people believe they know the essence of a concept (e.g., the genetic code of dog), essential properties could constrain categorization and similarity.But this form of constraint is a form we already met.The usage of essential properties depends on a particular theory of the category.As explained earlier, there are many possible theories about a given category, and one has to explain why this particular theory (e.g., the genetic code) is chosen over another (e.g., a theory of pets).But a theory of essential properties could be a default theory, a theory preferred for constraining similarity and categorization over other, nonessential theories.What is an essential theory?How can it be chosen over other theories?A belief in an essential theory does not constrain the choice of a theory, only another (meta)theory on what counts as an essential theory does constrain.Consequently, there are serious difficulties with the view that theories and essential properties constrain similarity, categorization and conceptual development.

Interactions between category structure, perception, and theories
So far, we have discussed the possible role of task constraints, perception and theories on the construction and development of feature spaces.The interaction of these three factors was studied by Thibaut (1991) in an experiment which compared the segmentations of a target stimulus presented in different conditions.In the category condition, the target stimulus was presented together with other stimuli of the same category (Figure 3 illustrates two of the stimuli); in the category theory condition, the same set of stimuli was presented with a basic level category name (for example, lobster); in the exemplar condition, the target shape was displayed alone; in the exemplar theory condition the target shape was displayed together with the basic level category name of the target.The exemplar conditions tested the interaction between high-level knowledge and the structure of the target stimulus.The category conditions tested the interaction of categorical structure with sophisticated knowledge.
In a feature circling task, it was found that the features selected differed in the exemplar conditions, but that they were the same in the category conditions.In the category condition, exemplars of the category imposed sufficient structure for constraining the featural decomposition of the target.In fact, subjects parsed objects similarly, independently of the basic-level name attached to the target.When no such categorical structure was given, parsing was more ambiguous and the basic-level name influenced the nature of the parsings.In short, the results of this experiment suggest that subjects' decompositions of the stimuli were mainly guided by the categorical structure (when such categorical structure was provided).In this case, the role of theories seemed to be limited to the interpretation of the features relevant in the categorical structure.
In opposition to this interpretation, Wisniewski and Medin (1994) reported results showing the influence of general knowledge and expectations on the way people describe categories.In their experiments, they presented sets of children's drawings to their subjects in two experimental conditions: a control group who was presented with the two categories but with irrelevant names (like Category A and Category B); a theory group who was told that the first set of drawings had been done by Creative children and the Category 2 had been done by Non creative children.
If subjects had only used information coming from the line drawings of the stimuli the classification rules in the theory group should have been similar to the rules given by the control group.Results revealed that the control group used more concrete rules than the theory group, but more abstract rules ("look more normal") were used in the theory groups (37%) than in the Control groups (16%).When the same drawings were presented to subjects as drawings done by farm or city children, the rules constructed were different and sometimes the same part of a drawing was interpreted differently.
The difference between the results of Wisniewski and Medin (1994) and the preliminary results of Thibaut suggest interesting avenues for future research on the relative role of different sources of constraints on the creation of feature spaces.For example the difference between influence (vs.no influence) of theories on stimulus encoding could arise from the structure of the items composing the category (the difference between Wisniewski and Medin, and Thibaut's results probably result from such differences).Strongly structured categories with perceptualy salient feature might not require theories for their encoding, and theories could to a certain extent interpret the featural structures which are given them.On the other end of the spectrum, poorly structured categories with non-obvious features might require the conceptual glue of theories to stick together category exemplars.

CONCLUSIONS
In this paper we discussed the notion of a feature space for higher-level cognition.
We showed how higher-level cognition (specifically categorization and similarity) was bounded by the features available at a particular stage of conceptual development.We presented two conceptions of development in relation to feature spaces.In the Fixed Space conception, all features are available prior to experience with the real world, and conceptual development is a transfer of a relevant featural description of a new object category into the set of object concepts.In the Augmented Space conception the feature set itself was able to develop to provide new representational capabilities to the organism.
Constraints on the evolution of the feature set were discussed.Specifically, we discussed the possible role of perception as a set of biases on possible features at stage t.
We also discussed how the requirements of distinguishing between object categories interacted with perceptual constraints in constructing new object features.It was argued that theories and sophisticated knowledge could also participate to the active construction of a feature space.
There is an important interaction between high-level knowledge (i.e.theories available about the input category), and perceptual data (data from the items presented).The nature of this interaction should be the object of further studies.On the one hand, pieces of knowledge are difficult to tag to unsegmented stimuli, but as we saw, there are often many possible ways to segment stimuli (think for example of the hierarchy of parts of the human body).On the other hand, a careful analysis of the role of theories as sources of constraints reveal that a given piece of knowledge (e.g., "a protective device") accepts multiple physical realizations (gloves, Faraday's cage, the Popemobile, a house, etc.) which share little perceptual commonalities.Preliminary results on the interactions between task constraints, perceptual cues and high-level knowledge suggest that people rely on perceptual commonalities to ground their theories.
Further research on this interaction could test the role of "congruent" (vs. "incongruent") theories for a given set of perceptual constraints.Note the difficulty of this approach in an Augmented Space conception of development.If people's features participate in the segmentation of an object, and if these features result from their history of categorization, the interaction of an identical theory with perceptual structures may vary dramatically across subjects.However, this apparent difficulty can be turned into an advantage because the history of categorization can be placed under experimental control.
That is, it is possible to ask subjects to discriminate categories of "Martian materials" that will require the perceptual acquisition of task-and stimulus-specific feature vocabularies.
The interaction of theories with these acquired structures can then be studied carefully.
In our view, theories of conceptual development should integrate the processes of feature selection and feature creation.An understanding of these constraints would provide better explanations of the interaction between the featural encoding of a category and the processing and representational capabilities of the organism.For example, in lexical development, it is often believed that the overgeneralization of the word "dog" to all "fourlegged animals" results from an emphasis on the "four-legged" feature as a cue for the category "dog".That is, adults and children differ in the features that compose their object concepts.Lexical development consists of adding new features to distinguish "dogs" from other four legged animals--e.g., a dog "barks", has a different shape.If these new features do not pre-exist to the children's feature vocabulary, theories must explain their genesis.
These new theories of lexical development should integrate the constraints we discussed in this paper.
Our discussion of feature spaces treats features as distinct and independent entities.
However, features often are linked to one another and one could argue that an Augmented Space is intrinsically relational because new features represent the distinction between two categories--or a commonality within a category.For example, when distinguishing between birds and cows, the number of legs would be an important feature that relate the two categories.But there are relationships that do not have such straightforward functional value.For example, there is a connexion between "to fly" and "to have wings" and "to have feathers" that only can be understood in the context of a theory about flying.Of course there is a correlational structure between these three features, but there is no causal structure unless an interpretation of the correlation is given by a theory.Again, as discussed earlier, this may highlight interesting connexions between categorical structures, perceptions and theories.To provide a more complete account of features as functional entities distinguishing between categories, the development of the connexions between theories and features must be better understood.To the extent that features spaces represent features as independent entities, they are necessarily incomplete, as most feature models would be.
In summary, perception, the structure of the categorization problem and sophisticated knowledge all participate in the construction of a feature space to describe objects for higher-level processing.One could interpret each of these constraints as specific respects of object description.The emphasis on one respect for description or the other depends itself on the nature of the task.The nature of the task comprises the purpose of the description and the relevant information provided by each source of respects.An understanding of the respects for object description is not far from an understanding of higher-level cognition.Schyns and Murphy (1994).The bottom pictures are two exemplars of Martian cells used in Schyns and Rodet (1995).X->Y group learned X and Y as separate concepts, X->XY group learned X and XY as separate concepts.
Figure 3: Two extraterrestrial lobsters used in Thibaut (1991Thibaut ( , 1995a)).The lobsters are composed of the same set of features which changed shape across stimuli.
--Variations across stimuli complicate feature extraction--if only because it is more difficult to match shape features across exemplars.In their Experiment 5, Schyns and Murphy (1994) tested the limit of part extraction when the sign of curvature of a part defining a category changed from exemplar to exemplar.It was shown that changes in sign of curvature in exemplars of a category induced subjects to create two different parts to represent the category, but that no change of sign of curvature induced the creation of a single part feature.Effects of variations across stimuli on feature extraction were also tested in Thibaut (1991).A comparison was made between a fixed, vs. a variable, target feature defining a category.It was shown that subjects in the variable feature condition represented the new shape feature plus the possible transformations of the shape in the category.In summary, these experiments on the extraction of variable features indicate that the context of a category influences the transformations that are found acceptable of a feature.The geometry of a feature can only be understood in the context of the categorization in which the feature was created.
the acquired concepts would in X->XY induce the learning of z to distinguish the new category, while X->Y could learn the new category as the x^y conjunction.In summary, we expected the groups to learn the following feature vocabularies and concepts: FX->Y->XYZ = {x, y} and CX->Y->XYZ = {x, y, x^y}; FX->XY->XYZ = {x, y, z} and CX->XY->XYZ = {x, x^y, x^y^z}.

Figure 1 :
Figure 1: This figure illustrates the basic components of feature spaces.Ft is the feature vocabulary available at time t.Et is the set of all possible feature combinations.Ct, the set of available concepts is always a subset of Et.

Figure 2 :
Figure 2: The top picture illustrates an exemplar of the three-dimensional Martian rocks used inSchyns and Murphy (1994).The bottom pictures are two exemplars of Martian cells

Figure 4 :
Figure 4: Four stimuli from Thibaut (1995c).First condition: subjects had to learn A-B versus C-D; second condition, A-C versus B-D.

Figure 5 :
Figure 5: These pictures present exemplars of the Martian Landscape stimuli used in an experiment by the authors.
of Augmented Space development for similarity and categorization A space of features grounded in the history of the categorizer constrains categorization and similarity judgments.At any given time of conceptual development, Ft, Et and Ct are the materials available for categorization and similarity judgments.Most authors ground categorization on similarity, by saying that object X is categorized in A instead of B if its featural description is more similar to A than to B. In other words, similarity and categorization operate in Ft.