Description of Research Expertise

Research
My main research goal is to develop machine-learnable models of language and other natural sequential data such as biological sequences. Penn, with its strong and growing machine learning group, is the ideal place to pursue that goal. My most recent work has been on finite-state models for text information extraction and speech recognition, but I am also interested in information-theoretic approaches to inducing compact representations of multivariate data, and on bridging the gap between distributional and logical views of natural-language syntax and semantics.

Conditional probability models for information extraction and segmentation
Many sequence-processing problems involve breaking it into subsequences (person name vs other), or labeling sequence elements (parts of speech). Existing probabilistic methods for these tasks, in particular HMMs, have difficulty in dealing with multiple overlapping features of the input sequence. Maximum entropy Markov models were a first approach to this problem, which have now been superseded the more powerful conditional random fields. We are applying these models to text information extraction and gene finding.

Finite-state speech processing
What do regular expressions turn into when we need to assign weights (maybe probabilities) to alternative matches, and to compose pattern matchers? Weighted finite-state transducers. At AT&T, I was involved in developing these as a framework for speech recognition, leading to a creation of a powerful library that has been made available for non-commercial use.

The information bottleneck
How does one quantify the notion of information about something? Given some variables of interest, sources of information about those variables can be compressed while preserving the information about the variables. The tradeoff between compression and information preservation, which we call the information bottleneck, answers the question. Using this model, we can build compact representations of complex relationships, for instance word cooccurrences in text.

Formal semantics of natural language
The syntactic structures of natural-language sentences and their meanings must be linked by a systematic, compositional process for language learning and use to be possible. However, this form of compositionality is more subtle than those used in logical and programming languages. Linear logic turns out to be a good metalanguage to describe the natural-language syntax-semantics mapping.