More detailed description of some of my past research.
Over the past five years, I have been involved in many research projects,
through my classes, my independent research, and my employment. I will
now describe the most important research projects in chronological order.
As a research intern at the USDOE Ames Lab, at Iowa State University, during
the summer of 1991, I was involved in studying the behavior of atomic clusters
on metallic surfaces using computer simulations. Working closely with a
post-doctoral student, I aided in the design and execution of several studies
to analyze the effects of temperature on cluster growth rates, to examine
the interactions between atomic clusters, and to develop kinematic models
of cluster behavior. I wrote programs to automate the analysis of cluster
movement and interaction, and independently developed a hypothesis to explain
a particular type of cluster interaction.
As a research assistant for Dr. Ellen Markman in Stanford's psychology
department in my sophomore year, I worked with a graduate student in the
design, literature search, experimentation, and analysis for a study investigating
children's knowledge of the proper/common name distinction.
Several of my courses actively encouraged independent research. In the
Artificial Life course at Stanford taught by Dr. John Koza, I had the opportunity
to conduct a research project with another student. Working in collaboration
with this student, I designed a series of experiments to investigate the
artificial evolution of cooperative behavior in teams of simulated agents.
The agents were to compete against each other in an environment where the
success of each individual was dependent on the success of the entire team.
We found that cooperation did indeed emerge between the players on each
team of robots, and also that specialization emerged within the teams.
In research presented as part of Dr. Nils Nilsson's class on Machine Learning,
I investigated an algorithm which incrementally adds linear units to a
linear machine during training until the training set can be classified
without errors. After discussing the nature of existing learning algorithms
and several ideas on how to extend them with Dr. Nilsson, I independently
designed, implemented, researched, and analyzed the algorithm. I tested
my algorithm on a benchmark problem in the field, and compared it to other
published methods. The new algorithm was found to perform comparably well
to most of the methods, and much better than several other clustering or
decision tree methods.
In an independent research project presented in Dr. John Koza's class on
Genetic Algorithms and Genetic Programming, I invented and investigated
a new method for evolving programs to recognize 2-dimensional images using
genetic programming. Genetic programming is a variant of the genetic algorithm
in which a population of computer programs undergoes simulated evolution
to produce a solution to a problem. Dr. Koza had developed a method for
evolving sub-routines alongside each program, but this idea, because it
failed to access information in a structured way, was of limited use for
2-dimensional features. I created a method for the simultaneous evolution
of 2-dimensional hit-miss matrices and an algorithm to use these matrices
in pattern recognition. Hit-miss matrices are templates that can be moved
over part of an input pattern to check for a 'match'. These matrices are
evolved using a 2-dimensional genetic algorithm that I created, while the
algorithms controlling the templates are evolved using genetic programming.
I applied the approach to the problem of digit recognition, and found it
to be successful at discovering programs that could recognize low resolution
digits. After the class, I continued with this research, resulting in its
publication in the peer-reviewed MIT Press book Advances in Genetic Programming
in 1994.
In my two years as a research consultant at Canon Research Center, I participated
in many research projects on different aspects of image processing and machine
learning. Initially, my primary task was to implement and evaluate a
variety of algorithms for binarization of grey-scale images. None of the
algorithms initially was adequate to our needs, but I devised a successful
hybrid that solved the problems. After the binarization project, I researched
a method that used fuzzy-logic rules to describe features for optical character
recognition. I designed rules for recognizing characters and provided detailed
theoretical analysis of the algorithm, which led to improvements in the
rule satisfaction engine. The effort to program and update all of the rules
for our rule-based recognition was extensive, and thus I proposed to automate
the process. This proposal turned into a summer-long research effort into
using genetic programming to learn rules for optical character recognition.
We used genetic programming to learn rules for characters from scratch
as well as to update rules that had previously been coded by hand. This
reserach resulted in one conference paper at the IEEE World Congress on
Computational Intelligence and an invited chapter in the Handbook of Evolutionary
Computation.
Based on a proposal I wrote in a seminar on children's categorization and
conceptual organization taught by Dr. Ellen Markman, I developed a possible
alternative to a major theory in the field of cognitive development. This
theory holds that cognitive development is a process of theory change, and
that in particular, that children's theories of biology are originally conceptualized
within their theories of psychology. The main empirical support for this
theory is that children taught a property about an animal generalize that
property depending on that animal's similarity to people. However, I realized
that this previous research failed to account for the confound of familiarity.
In particular, children may generalize more from people, not because people
are highly psychological, but because people are highly familiar.
My research with pre-school children, in several experiments over the course
of three years, investigated whether the familiarity of a training item
or the embeddedness of a training item in a lay psychology is a better predictor
of the strength of an item as an inductive base for preschoolers' projection
of biological properties. The first study contrasted four year-old children's
generalization of a biological property from a highly familiar animal (a
dog) with generalization from an animal less familiar than a dog but more
similar to a person (a chimpanzee). The results were that children projected
more when taught a property about a dog than when taught about a chimp.
The second study examined whether the strength of humans as an inductive
base for preschoolers extends to non-biological, non-psychological domains
such as physics. Children generalized a physical property more from a person
to inanimate items than from another inanimate item (a suitcase) to the
inanimate test items. In both studies, familiarity with the training items
best predicted children's generalizations. These results challenged the
assumption that children's biological generalizations are governed by the
similarity of the inductive base to a person, and raise doubts about the
hypothesis that children's early theories of biology emerge from their lay
theories of psychology. This research, constituting my honors research
in psychology, was awarded Stanford's Firestone Medal for Excellence in
Research in 1994, given to the best honor's theses in each department.
In the year since my graduation, I have modified my honor's thesis into
a journal article which is currently undergoing review for publication in
Cognitive Development.
My honors research in Symbolic Systems focused on the simulated evolution
of agents that can learn and create simple plans of action. Traditional
approaches to program induction have focused on evolving functional or reactive
programs. My independent research presented MAPMAKER, a method for the
automatic generation of agents that discover information about their environment,
encode this information for later use, and create simple plans utilizing
the stored mental models. In this method, agents are multi-part computer
programs that communicate through a shared memory. Both the programs and
the representation scheme are evolved using genetic programming. I used
an illustrative problem of 'gold' collection to demonstrate the method in
which one part of a program makes a map of the world and stores it in memory,
and the other part uses this map to find the gold. The results indicate
that the method can evolve programs that store simple representations of
their environments and use these representations to produce simple plans.
The results from my honor's research were presented three times at national
conferences. I presented early results on a constrained version of the
problem at the IEEE World Congress on Computational Intelligence in 1994,
the unconstrained version of the problem at IJCAI-95, and a further study
that indicated that more complex mental models can emerge when simple ones
are discouraged at the International Conference on Genetic Algorithms in
1995. In addition, the model I developed, the Mapmaker model, has been
used by other researchers working on evolving the use of memory, communication,
and planning. For this research, my honor's research in psychology, and
my general academic success at Stanford, I was awarded the Dean's Award
for Academic Achievement in 1994, an award given to only eight students
each year for outstanding scholarly accomplishment.
In the year and a half since my graduation, I've been working as a research
scientist with Dr. John Koza at the Genetic Algorithm Technology Corporation
and have also been a visiting scholar at Stanford. In this time, I've contributed
significantly to the design, programming, and analysis for each research
project. For example, one constraint in the earliest forms of genetic programming
was that the user had to determine the architecture of the evolving programs.
I helped to design and program genetic operators to determine the architecture
automatically, during the course of a run of genetic programming. We have
applied for a patent on this invention.
Our research at G.A.T.C. has followed two main themes: (1) solving difficult,
real-world problems using genetic programming and (2) studying the nature
and properties of genetic programming and researching methods for the efficient
use of genetic programming. In this first category, we have successfully
applied genetic programming to problems in the classification of protein
sequences, to the discovery of motifs that identify biologically important
protein domains, to the problem of evolving a cellular automata rule for
a given problem, and to the problem of automatically generating circuit
designs for analog filters. For each of these problems, we've found that
genetic programming evolved solutions as good as or better than human coded
solutions to the same problem. I have also recently begun work on a project
that utilizes genetic programming to evolve near optimal planning strategies
for robots in environments with incomplete information. In the second
theme of our research, we have extended the set of programming ingredients
controlled by the evolutionary process and investigated various factors
that might effect the success of genetic programming. For example, one
research project focused on the evolution of iterative strutures in programs,
another on the role of memory structures in evolving programs, and another
on the differences between genetic programming and simpler search techniques.
In addition to my work at GATC, I have been involved in a robotics and
vision research project in collaboration with a graduate student at Stanford,
Illah Nourbakhsh. Working closely together, we invented a new vision technology
that uses focus information to provide a depth map of the environment.
Inspired by the success of auto-focus cameras, we developed a vision system
for a robot that utilizes only passive information, unlike infrared sensors,
sonars, or laser range finders. In addition, our algorithm is computationally
inexpensive, unlike depth from stereo vision or depth from motion methods.
One of the most important features of our technology is that it can be
used to detect vertical drop-offs such as stairs, that sonar based methods
have found difficult. We have demonstrated a robot using our technology
at Stanford and at IJCAI-95, and found it to be extremely reliable. We
have patented this invention and an article describing the work is under
review by the Autonomous Robots Journal.
Back to Dave's Home Page
