A camera is a camera and a screen is a screen, till recently they couldn’t be both simultaneously. But last month researchers from ETH Zurich in Switzerland, have created a new type of pixel that can do both at the same time. This hypercharged pixel, called a Fourier pixel, can generate and sense arbitrary light fields and tap into a pixel’s full potential for carrying information by manipulating light’s intensity, oscillation phases, and polarization. Meaning it can display an image and act as a camera at the same time. The paper on the team’s findings was published in Nature.
This technology when matured could pave the way for camera–display devices that both display images and act as a camera. One interesting use case would be full functional invisibility. For years we have seen invisibility cloaks, invisible cars and cloaking tech on the movie/TV screens, but most efforts to recreate them have not been successful so far. With this new technology it might soon be possible to have cloaking tech available for use. (At least on the visual spectrum)
Digital cameras and displays use picture elements (pixels3) that perform a single function: detecting or emitting light intensity. To exploit the full information content of electromagnetic waves, more advanced elements are required. This has driven the development of multifunctional components that, for example, simultaneously detect and emit intensity or extract intensity and spectral information. However, no pixel exists that both senses and generates optical wavefronts with full control over amplitude, phase and polarization, limiting bidirectional control and feedback of sophisticated light fields. Here we present a route to such pixels by demonstrating a versatile platform of miniaturized diffractive elements based on Fourier optics. We use plasmonic surface waves, which propagate coherently and efficiently across metallic surfaces. When these plasmons are launched towards wavy microstructures designed with simple Fourier analysis, arbitrary and background-free optical wavefronts are generated. Conversely, incoming light can be sensed, and its amplitude, phase and polarization can be fully characterized. By combining or superposing several such components, we create multifunctional ‘Fourier pixels’ that provide compact and accurate control over the optical field. Our approach, which we extend to photonic waveguide modes, establishes a scalable, universal architecture for vectorially programmable pixels with applications in adaptive optics, holographic displays, optical communication and quantum information processing.
As with all technologies there is potential for misuse as well because every screen becomes a unobtrusive camera which increases the ability for performing surveillance. Imagine having a TV with this functionality. Then imagine hackers gaining control of the screen to record personal movements. Then imagine an authoritarian government doing the same constantly. This was explored in the novel 1984 by George Orwell.
The telescreen received and transmitted simultaneously. Any sound that Winston made, above the level of a very low whisper, would be picked up by it; moreover, so long as he remained within the field of vision which the metal plaque commanded, he could be seen as well as heard. There was of course no way of knowing whether you were being watched at any given moment.
Source: Schneier on Security: A Video Screen That Is Also a Camera
– Suramya