Are we ready to tackle perceptual segmentation of natural scenes?

Processes of perceptual segmentation and integration (PSI) are fundamental to perceptual organization. Although PSI of visual stimuli has been studied for over a century, we have only a rudimentary understanding of PSI of natural visual stimuli. This is due to limitations of traditional experimental methods in visual psychophysics of PSI; to the exclusive focus of computer-vision research for image segmentation on performance benchmarks; and to the scarcity of meaningful interactions between those two communities. The recent literature discussed in this paper presents a compelling argument that the field is starting to overcome those barriers. One important example of such an interaction between visual psychophysics and machine learning is given by the literature on the crowding phenomenon, which calls for revised models of summary statistics to explain some uncrowding results. Other examples reviewed here include studies of the perceptual uncertainty and dynamics of segmentation of natural stimuli, which call for computational models with probabilistic representations and dynamic computations. Conversely, contemporary machine learning algorithms produce impressive segmentation maps that still need to be aligned with human maps as measured with objective tasks such as the same/different segment paradigm reviewed here. Therefore, the time is ripe to move vision science forward by bridging new computational and experimental paradigms for PSI of natural stimuli.