Screen-Camera Communiqué (In-screen Hidden Pinholes...?)

Arecent excursion into the Wi-Fi-tablet world revealed an anomaly, a tiny pinprick, or “bubble,” unrelated to an on-screen image.



It was odd, almost a “camera obscura,” or pinhole reflection, a natural optical phenomenon that occurs when an image on the other side of a wall, or screen, is projected through a small hole as a reversed and inverted image. Someone, somewhere, it seemed (or it sure felt like it, anyway), had entered the room by way of a digital, “wormhole” camera.¹ 

The digital world is expanding so rapidly, that devices that connect us to Wi-Fi can be purchased this day for as little as 40 dollars. A 7-inch, quad-core Digiland tablet, for example (see YouTube Review), is being noticed everywhere. It’s simple operation makes it indispensable even to older Americans, and like all such devices, will be able to invade homes everywhere.

“The most profound technologies are those that disappear.  They weave themselves into the fabric of everyday life until they are indistinguishable from it.”

Old news, of course. And yet nothing about this new technology, or how it is being developed, is readily available as of this post. A technology that can reach out from our digital screens, unknowingly, has been around for almost three years with little concern, or coverage.

The website, Phys.org reported, that researchers had created a real-time, screen-camera communication hidden from viewers for the first time, opening the way for new applications of smart devices.  The website said that Dartmouth researchers found a way to allow screens and cameras to communication to each other without the user even knowing it.

“Using off-the-shelf smart devices,” Phys.org said, “the new system supports an unobtrusive, flexible and lightweight communication channel between screens (of TVs, laptops, tablets, smartphones and other electronic devices) and cameras.

“The system, called HiLight, will enable new context-aware applications for smart devices. Such applications include smart glasses communicating with screens to realize augmented reality or acquire personalized information without affecting the content that users are currently viewing. The system also provides far-reaching implications for new security and graphics applications.”

With “eyes” that communicate unknowingly, and even voices enveloping us, either asleep or awake, indoors or outdoors (in the bath or in the bedroom), commercial forces are changing how we live our lives. And there will be no escape, as this expansion of prose from George Orwell’s 1984 suggests: nothing will be ours except the few cubic centimeters of human skull we each possess.

“The most profound technologies are those that disappear,” Mark Weiser wrote for Scientific America in 1991. “They weave themselves into the fabric of everyday life until they are indistinguishable from it.”

There is nothing written on wormhole cameras (although I have not poured over all the current research papers), the technology, or desire to use it, exists.

In 2009, for example, Jennifer Wang, the Entrepreneur Column, wrote that technology researchers were working on something big: a device that combines a digital screen and camera to analyze your physical characteristics and play back personalized advertisements. It isn't ready for market yet; but, when it finally is, it could revolutionize the way businesses reach out to their consumers.

Sound familiar?

“Here's how it would work,” Wang said. “When shoppers pause in front of a monitor, a computer reads their sex, age, race and expression to gauge interest levels, and then decides the best commercial to play.

“If the person turns or looks away, the device attempts to draw attention back to the screen — perhaps the music will suddenly crescendo — and the customer will see a different product the computer thinks may be of interest based on the information gleaned earlier.

“Spearheading the project, called "Targeted Advertising Based on Audience Natural Response," or TABANAR (thankfully) for short, is NICTA, an information and communication technology center established by the Australian government.

“According to Glenn Downey, NICTA's commercialization manager, progress depends on how quickly researchers can develop sophisticated ways to identify emotional cues from facial expressions. She says a breakthrough would represent a transition from dynamic to responsive technologies. ‘Next generation technology will react to interest levels and shift content accordingly.’

“— What's Available Now?

“For example, mining video for customer data is gaining popularity. Rajeev Sharma, founder of Pennsylvania-based VideoMining Corporation, created the company in 2000 to meet demand for retail intelligence revealed through image recognition software. In VideoMining's client stores, feeds from security cameras are sent to a main computer, which extrapolates information on everything from what products people are looking at to how long they stand in front of a particular display. …

“‘In one grocery store,’ Sharma recalls, ‘they discovered there were too many product types in the juice section. Ten percent of shoppers spent 90 seconds in front of the display before leaving without buying anything. We pinpointed that people must be overwhelmed by products. They responded by reducing the number and organizing it better. It worked.’” (Read Entire article here.)

In today’s world, Phys.org said HiLight supports communication atop any screen content, such as an image, movie, video clip, game, web page or any other application window, so that camera-equipped devices can fetch the data by turning their cameras to the screen.

“HiLight leverages the alpha channel, Phys.org reported, “a well-known concept in computer graphics, to encode bits into the pixel translucency change. HiLight overcomes the key bottleneck of existing designs by removing the need to directly modify pixel color values. It decouples communication and screen content image layers.

“‘Our work provides an additional way for devices to communicate with one another without sacrificing their original functionality,’ says senior author Xia Zhou, an assistant professor of computer science and co-director of the DartNets (Dartmouth Networking and Ubiquitous Systems) Lab.

“‘It works on off-the-shelf smart devices. Existing screen-camera work either requires showing coded images obtrusively or cannot support arbitrary screen content that can be generated on the fly. Our work advances the state-of-the-art by pushing screen-camera communication to the maximal flexibility.’”

Footnote
 ^{1 Upon lifting the screen closer the phenomenon disappeared.  The common occurrence found when a droplet of moisture refracts light on the screen cannot be ruled out; however, had it been a refraction, looking closer should not have had an effect.  The weird camera obscura was gone without a trace by bringing the screen closer for inspection.}