photosketch internet image montage

Here’s an interesting piece of technology developed by a group of scientists. It’s able to fetch information from the internet to dynamically composite images. All it takes for it to happen is your ideas communicated in the form of sketching.  Below are storyboards from the proposed technology, showing how input equals output.

To get a real idea for how the application works they’ve also put together this presentation video.  If they are able to implement this in a commercial way in the future, it could easily become a popular avenue of enhancing the user experience on-the-fly.

We present a system that composes a realistic picture from a simple freehand sketch annotated with text labels. The composed picture is generated by seamlessly stitching several photographs in agreement with the sketch and text labels; these are found by searching the Internet. Although online image search generates many inappropriate results, our system is able to automatically select suitable photographs to generate a high quality composition, using a filtering scheme to exclude undesirable images. We also provide a novel image blending algorithm to allow seamless image composition. Each blending result is given a numeric score, allowing us to find an optimal combination of discovered images. Experimental results show the method is very successful; we also evaluate our system using the results from two user studies.

article source