Smart photo tagging created at Penn State

Penn State researchers have developed software they say tags images upon uploading to Yahoo’s Flickr or other photo systems but also automatically updates those tags based on how people interact with the photos.

This could greatly improve searching for images, the researchers say.

"Tagging itself is challenging as it involves converting an image's pixels to descriptive words," said James Wang, lead researcher and associate professor of information sciences and technology, in a statement. "But what is novel with the 'Tagging over Time' or T/T technology is that the system adapts as people's preferences for images and words change."

In recent tests the system was shown to correctly annotate four of every 10 images. It still needs work, but is an improvement over an earlier Penn State-developed system dubbed Automatic Linguistic Indexing of Pictures-Real Time that analyzed pixel content to suggest tags. The new software, which relies on machine-learning, is described in more detail in a paper called "Tagging Over Time: Real-world Image Annotation by Lightweight Meta-learning." The researchers say accuracy of the new system can grow from 40% to 60% as it learns from user behavior.

In a companion paper, Penn State researchers describe other advances in enabling searches to pick out higher quality images.
The National Science Foundation supported both research efforts.

For the latest on network-oriented research at university and other labs, go to Network World’s Alpha Doggs blog.