SkyFinder: attribute-based sky image search

In this paper, we present SkyFinder, an interactive search system of over a half million sky images downloaded from the Internet. Using a set of automatically extracted, semantic sky attributes (category, layout, richness, horizon, etc.), the user can find a desired sky image, such as "a landscape with rich clouds at sunset" or "a whole blue sky with white clouds". The system is fully automatic and scalable. It computes all sky attributes offline, then provides an interactive online search engine. Moreover, we build a sky graph based on the sky attributes, so that the user can smoothly explore and find a path within the space of skies. We also show how our system can be used for controllable sky replacement.

[1]  David Salesin,et al.  Fast multiresolution image querying , 1995, SIGGRAPH.

[2]  Leonidas J. Guibas,et al.  A metric for distributions with applications to image databases , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[3]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[4]  Marie-Pierre Jolly,et al.  Interactive graph cuts for optimal boundary & region segmentation of objects in N-D images , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[5]  Erik Reinhard,et al.  Color Transfer between Images , 2001, IEEE Computer Graphics and Applications.

[6]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[7]  Harry Shum,et al.  Lazy snapping , 2004, ACM Trans. Graph..

[8]  Gabriela Csurka,et al.  Visual categorization with bags of keypoints , 2002, eccv 2004.

[9]  F. Durand,et al.  Texture design using a simplicial complex of morphable textures , 2005, ACM Trans. Graph..

[10]  Alexei A. Efros,et al.  Automatic photo pop-up , 2005, ACM Trans. Graph..

[11]  Roberto Cipolla,et al.  Semantic Photo Synthesis , 2006, Comput. Graph. Forum.

[12]  Andrew Jones,et al.  Direct HDR capture of the sun and sky , 2006, SIGGRAPH Courses.

[13]  Steven M. Seitz,et al.  Photo tourism: exploring photo collections in 3D , 2006, ACM Trans. Graph..

[14]  Frédéric Jurie,et al.  Fast Discriminative Visual Codebooks using Randomized Clustering Forests , 2006, NIPS.

[15]  Alexei A. Efros,et al.  Scene completion using millions of photographs , 2007, SIGGRAPH 2007.

[16]  Alexei A. Efros,et al.  Using Color Compatibility for Assessing Image Realism , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[17]  Alexei A. Efros,et al.  Photo clip art , 2007, ACM Trans. Graph..

[18]  Richard Szeliski,et al.  Finding paths through the world's photos , 2008, ACM Trans. Graph..

[19]  James Ze Wang,et al.  Image retrieval: Ideas, influences, and trends of the new age , 2008, CSUR.

[20]  Alexei A. Efros,et al.  What Does the Sky Tell Us about the Camera? , 2008, ECCV.

[21]  Shree K. Nayar,et al.  Face swapping: automatically replacing faces in photographs , 2008, SIGGRAPH 2008.

[22]  Shree K. Nayar,et al.  FaceTracer: A Search Engine for Large Collections of Images with Faces , 2008, ECCV.