Tiered Scene Labeling with Dynamic Programming

Dynamic programming (DP) has been a useful tool for a variety of computer vision problems. However its application is usually limited to problems with a one dimensional or low treewidth structure, whereas most domains in vision are at least 2D. In this paper we show how to apply DP for pixel labeling of 2D scenes with simple ``tiered'' structure. While there are many variations possible, for the applications we consider the following tiered structure is appropriate. An image is first divided by horizontal curves into the top, middle, and bottom regions, and the middle region is further subdivided vertically into subregions. Under these constraints a globally optimal labeling can be found using an efficient dynamic programming algorithm. We apply this algorithm to two very different tasks. The first is the problem of geometric class labeling where the goal is to assign each pixel a label such as ``sky'', ``ground'', and ``surface above ground''. The second task involves incorporating simple shape priors for segmentation of an image into the ``foreground'' and ``background'' regions.

Main Idea

Figure 1

Applications

Geometric Scene Labeling

Figure 2: Labels T and B correspond to the sky and the ground plane, labels L C, and R correspond to left, center, and right surfaces.

Binary Segmentation with Shape Prior

Figure 3

Code

Publications