Summarizing Beliefs of Intelligent Systems via Diverse Predictions

This project has proposed theory, algorithms, and implementations for intelligent systems that convey their beliefs about the world by producing a small set of diverse plausible hypotheses or guesses about the state of the world (e.g. multiple segmentations for objects in an image, or human body key point locations).

Before this project, intelligent systems either produced a single “most likely” hypothesis (called the maximum a posteriori (MAP) estimator), or produced marginal probabilities for all variables in the model.  Neither of which are ideal choices. The problem with the former is that parts of most-likely hypothesis will be wrong because our systems are not perfect. The problem with marginals is that they are often uninterpretable, leading to an operator perceiving the intelligent system as an inscrutable black-box that cannot be understood or trusted.

In essence, the single-best hypothesis has no uncertainty and marginals have no structure. This project has challenged this long-held status quo and shown that there is a better third alternative – an intelligent system should maintain several structured hypotheses about the world, which can serve as a concise interpretable summary of the beliefs of this system.