Extracting sub-exposure images from a single capture
through Fourier-based optical modulation
Shah Rez Khana , Martin Feldmanb , Bahadir K. Gunturka,1,∗

arXiv:1612.08359v2 [] 13 Feb 2018

a Dept.
b Div.

of Electrical and Electronics Eng., Istanbul Medipol University, Istanbul, Turkey
of Electrical and Computer Eng., Louisiana State University, Baton Rouge, LA

Abstract
Through pixel-wise optical coding of images during exposure time, it is possible
to extract sub-exposure images from a single capture. Such a capability can be
used for different purposes, including high-speed imaging, high-dynamic-range
imaging and compressed sensing. In this paper, we demonstrate a sub-exposure
image extraction method, where the exposure coding pattern is inspired from
frequency division multiplexing idea of communication systems. The coding
masks modulate sub-exposure images in such a way that they are placed in
non-overlapping regions in Fourier domain. The sub-exposure image extraction
process involves digital filtering of the captured signal with proper band-pass
filters. The prototype imaging system incorporates a Liquid Crystal over Silicon
(LCoS) based spatial light modulator synchronized with a camera for pixel-wise
exposure coding.

1. Introduction
Coded aperture and coded exposure photography methods, which involve
control of aperture shape and exposure pattern during exposure period, present
new capabilities and advantages over traditional photography. In coded aperture photography, the aperture shape is designed to achieve certain goals. For
∗ Corresponding

author
Email address: bkgunturk@medipol.edu.tr (Bahadir K. Gunturk)
1 This work is supported by TUBITAK Grant 114C098.

Preprint submitted to Elsevier

February 14, 2018

example, the aperture shape can be designed to improve depth estimation accuracy as a part of depth-from-defocus technique [1], or to improve deblurring
performance through adjusting the zero crossings of point spread function [2].
Coded aperture photography may involve capture of multiple images, where
each image is captured with a different aperture shape, for instance, to acquire
light field [3], or to improve depth estimation and deblurring performance [4].
Using coded aperture, it is possible to do lensless imaging as well [5, 6].
In coded exposure photography, the exposure pattern is controlled during
exposure period. The coding can be global, where all pixels are exposed together
with a temporal pattern, or pixel-wise, where each pixel has its own exposure
pattern. An example of global exposure coding is the flutter shutter technique
[7], where the shutter is opened and closed according to a specific pattern during
exposure period to enable better recovery from motion blur. The flutter shutter
idea can also be used for high-speed imaging [8]. Pixel-wise exposure control
presents more flexibility and wider range of applications compared to global
exposure coding. An example of pixel-wise exposure control is presented in [9],
where the goal is to spatially adapt the dynamic range of the captured image.
Pixel-wise coded exposure imaging can also be used for focal stacking through
moving the lens during the exposure period [10], and for high-dynamic-range
video through per-pixel exposure offsets [11].
Pixel-wise exposure control is also used for high-speed imaging by extracting
sub-exposure images from a single capture. In [12], pixels are exposed according
to a regular non-overlapping pattern on the space-time exposure grid. Some
spatial samples are skipped in one time period to take samples in another time
period to improve temporal resolution. In other words, spatial resolution is
traded off for temporal resolution. In [13], there is also a non-overlapping spacetime exposure sampling pattern; however, unlike the global spatial-temporal
resolution trade-off approach of [12], the samples are integrated in various ways
to spatially adapt spatial and temporal resolutions according to the local motion.
For fast moving regions, fine temporal sampling is preferred; for slow moving
regions, fine spatial sampling is preferred. Instead of a regular exposure pattern,
2

(a)

(b)

(c)

(d)

Figure 1: Illustration of different exposure masks. (a) Non-overlapping uniform grid exposure
[13], (b) Coded rolling shutter [17], (c) Pixel-wise random exposure [15], (d) Frequency division
multiplexed imaging exposure [20].

random patterns can also be used [14, 15, 16]. In [14], pixel-wise random subexposure masks are used during exposure period. The reconstruction algorithm
utilizes the spatial correlation of natural images and the brightness constancy
assumption in temporal domain to achieve high-speed imaging. In [15, 16], the
reconstruction algorithm is based on sparse dictionary learning. While learningbased approaches may yield outstanding performance, one drawback is that the
dictionaries need to be re-trained each time a related parameter, such as target
frame rate, is changed.
Alternative to arbitrary pixel-wise exposure patterns, it is also proposed to
have row-wise control [17] and translated exposure mask [18, 19]. Row-wise
exposure pattern can be designed to achieve high-speed imaging, high-dynamicrange imaging, and adaptive auto exposure [17]. In [18], binary transmission
masks are translated during exposure period for exposure coding; sub-exposure
images are then reconstructed using an alternating projections algorithm. The
same coding scheme is also used in [19], but a different image reconstruction
approach is taken.
Pixel-wise exposure control can be implemented using regular image sensors
with the help of additional optical elements. In [9], an LCD panel is placed in
front of a camera to spatially control light attenuation. With such a system,
pixel-wise exposure control is difficult since the LCD attenuator is optically
defocused. In [21], an LCD panel is placed on the intermediate image plane,

3

which allows better pixel-by-pixel exposure control. One disadvantage of using
transmissive LCD panels is the low fill factor due to drive circuit elements
between the liquid crystal elements. In [22], a DMD reflector is placed on the
intermediate image plane. DMD reflectors have high fill factor and high contrast
ratio, thus they can produce sharper and higher dynamic range images compared
to LCD panels. One drawback of the DMD based design is that the micromirrors
on a DMD device reflect light at two small angles, thus the DMD plane and
the sensor plane must be properly inclined, resulting in “keystone” perspective
distortion. That is, a square DMD pixel is imaged as a trapezoid shape on
the sensor plane. As a result, pixel-to-pixel mapping between the DMD and the
sensor is difficult. In [23], a reflective LCoS spatial light modulator (SLM) is used
on the intermediate image plane. Because the drive circuits on an LCoS device
is are on the back, high fill factor is possible as opposed to the transmissive LCD
devices. Compared to a DMD, one-to-one pixel correspondence is easier with an
LCoS SLM; however, the light efficiency is not as good as the DMD approach.
In [18], a lithographically patterned chrome-on-quartz binary transmission mask
is placed on the intermediate image plane, and moved during exposure period
with a piezoelectric stage for optical coding. This approach is limited in terms
of the exposure pattern that can be applied.
In this paper, we demonstrate a sub-exposure image extraction idea. The
idea, which is called frequency division multiplexed imaging (FDMI), was presented by Gunturk and Feldman as a conference paper [20]. While the FDMI
idea was demonstrated by merging two separate images with a patterned glass
based and an LCD panel based modulation in [20], it was not demonstrated
for sub-exposure image extraction. Here, we apply the FDMI idea to extract
sub-exposure images using an optical setup incorporating an LCoS SLM synchronized with a camera for exposure coding.
In Section 2, we present the problem of extracting sub-exposure images
through space-time exposure coding, and review the FDMI approach. In Section 3, we present the optical setup used in the experiments. In Section 4, we
provide experimental results with several coded image captures. In Section 5,
4

we conclude the paper with some future research directions.

2. Extracting Sub-Exposure Images from a Single Capture
There are various exposure coding schemes designed for extracting subexposure images from a single capture. First, we would like to present a formulation of the coding process, and then review the FDMI idea.
2.1. Space-Time Exposure Coding
Space-time exposure coding of an image can be formulated using a spatiotemporal video signal I(x, y, t), where (x, y) are the spatial coordinates and t
is the time coordinate. This signal is modulated during an exposure period T
with a mask m(x, y, t) to generate an image:
T

Z
I(x, y) =

m(x, y, t)I(x, y, t)dt.

(1)

0

The mask m(x, y, t) can be divided in time into a set of constant sub-exposure
masks: m1 (x, y) for t ∈ (t0 , t1 ), m2 (x, y) for t ∈ (t1 , t2 ), ..., mN (x, y) for t ∈
(tN −1 , tN ), where N is the number of masks, and t0 and tN are the start and end
times of the exposure period. Incorporating the sub-exposure masks mi (x, y)
into equation (1), the captured image becomes
I(x, y) =

N
X

Z

ti

mi (x, y)

i=1

I(x, y, t)dt =
ti−1

N
X

mi (x, y)Ii (x, y),

(2)

i=1

where we define the sub-exposure image Ii (x, y) =

R ti
ti−1

I(x, y, t)dt. The above

equation states that the sub-exposure images Ii (x, y) are modulated with the
masks mi (x, y) and added up to form the recorded image I(x, y). The goal of
sub-exposure image extraction is to estimate the images Ii (x, y) given the masks
and the recorded image.
As we have already mentioned in Section 1, the reconstruction process might
be based on different techniques, varying from simple interpolation [12] to dictionary learning [16]. The masks mi (x, y) can be chosen in different ways as
well. Some of the masks, including non-overlapping uniform grid [12], coded
5

Figure 2: Illustration of the FDMI idea with two images [20].

rolling shutter [17], pixel-wise random exposure [15], and frequency division
multiplexed imaging exposure [20], are illustrated in Figure 1.
2.2. Frequency Division Multiplexed Imaging
The frequency division multiplexed imaging (FDMI) idea [20] is inspired
from frequency division multiplexing method in communication systems, where
the communication channel is divided into non-overlapping sidebands, each of
which carry independent signals that are properly modulated. In case of FDMI,
sub-exposure images are modulated such that they are placed in different regions
in Fourier domain. By ensuring that the Fourier components of different subexposure images do not overlap, each sub-exposure image can be extracted from
the captured signal through band-pass filtering. The FDMI idea is illustrated
for two images in Figure 2. Two band-limited sub-exposure images I1 (x, y) and
I2 (x, y) are modulated with horizontal and vertical sinusoidal masks m1 (x, y)
and m2 (x, y) during exposure period. The masks can be chosen as raised cosines:
m1 (x, y) = a + bcos(2πu0 x) and m2 (x, y) = a + bcos(2πv0 y), where a and b
are positive constants with the condition a ≥ b so that the masks are nonnegative, that is, optically realizable, and u0 and v0 are the spatial frequencies
of the masks. The imaging system captures sum of the modulated images:
I(x, y) = m1 (x, y)I1 (x, y) + m2 (x, y)I2 (x, y).
The imaging process from the Fourier domain perspective is also illustrated
6

in Figure 2. Iˆ1 (u, v) and Iˆ2 (u, v) are the Fourier transforms of the sub-exposure
images I1 (x, y) and I2 (x, y), which are assumed to be band-limited. In Figure 2, the Fourier transforms Iˆ1 (u, v) and Iˆ2 (u, v) are shown as circular regions. The Fourier transforms of the masks are impulses: M̂1 (u, v) = aδ(u, v) +
(b/2)(δ(u - u0 , v) + δ(u + u0 , v)) and M̂2 (u, v) = aδ(u, v) +
ˆ v) of
(b/2)(δ(u, v - v0 ) + δ(u, v + v0 )). As a result, the Fourier transform I(u,
the recorded image I(x, y) includes Iˆ1 (u, v) and Iˆ2 (u, v) in its sidebands, and
Iˆ1 (u, v) + Iˆ2 (u, v) in the baseband. From the sidebands, the individual subexposure images can be recovered with proper band-pass filtering. From the
baseband, the full-exposure image I1 (x, y) + I2 (x, y) can be recovered with a
low-pass filter.
It is possible to use other periodic signals instead of a cosine wave. For
example, when a square wave is used, the baseband is modulated to all harmonics of the main frequency, where the weights of the harmonics decrease with
a sinc function. Again, by applying band-pass filters on the first harmonics,
sub-exposure images can be recovered.

3. Prototype Design
The prototype system is based on the pixel-wise exposure control design
that is adopted in several papers [23, 15, 16]. As shown in Figure 3, the system
consists of an objective lens, three relay lenses, one polarizing beam splitter, an
LCoS SLM, and a camera. Relay lenses are 100mm aspherical doublets; the objective lens is a 75mm aspherical doublet; the SLM is a Forth Dimension Display
SXGA-3DM with 1280x1024 pixel resolution and a pixel pitch of 13.62µm; and
the camera is a Thorlabs DCU-224M monochromatic camera with 1280x1024
pixel resolution and a pixel pitch of 4.65µm. The objective lens forms an image on an intermediate image plane, which is relayed onto the SLM. The SLM
controls the exposure pattern of each pixel through changing the polarization
states. The image reflected from the SLM is recorded by the camera. The
camera and the SLM are synchronized using the trigger signal from the SLM.

7

(a)

(b)
Figure 3: Pixel-wise exposure control camera prototype. (a) Graphical illustration, (b) Actual
prototype.

8

4. Experimental Results
We conducted several experiments to demonstrate sub-exposure image extraction with FDMI. As the first experiment, we performed a simulation to
discuss about the mask design and limitations of the FDMI approach. An image is rotated and translated to create twelve images. Each image is low-pass
filtered to obtain band-limited signals. The images are then modulated, each
with a different mask, to place them in different regions of the Fourier domain.
The masks are designed such that the sidebands at the fundamental frequencies
and the harmonics do not overlap in the Fourier domain. We first decide on
where to place the sideband in the Fourier domain; form the corresponding 2D
raised cosine signal (mask) as discussed in Section 2; and discretize (sample) the
signal to match size of the input images. Since sampling is involved during the
mask creation, we should avoid aliasing. The highest frequency sinusoidal mask
(in, for instance, horizontal direction) becomes a square wave (binary pattern)
with a period of two pixels.
The original images and zoomed-in regions from the masks are shown in
Figure 4(a) and Figure 4(b), respectively. The highest frequency masks in horizontal and vertical directions are square waves. The modulated image, obtained
by multiplying each image with the corresponding mask and then averaging, is
shown in Figure 4(c). Its Fourier transform is shown in Figure 4(d), where the
placement of different sub-exposure images can be seen. The baseband and the
sidebands from which the sub-exposure images are extracted are also marked in
the figure. From these sidebands, we extract all 12 sub-exposure images. Four
of the extracted sub-exposure images are shown in Figure 4(e).
The second experiment is realized with the prototype system. As shown in
Figure 5, a planar surface with a text printed on is moved in front of the camera.
There are four sub-exposure masks: horizontal, vertical, right diagonal and left
diagonal. (These masks are illustrated in Figure 1(d).) The exposure period for
each mask is 10 milliseconds. The captured image is shown in Figure 5(a). Its
Fourier transform is shown Figure 5(b). The recovered sub-exposure images are

9

(a)

(b)

(c)

(d)

(e)
Figure 4: Simulation to demonstrate sub-exposure image extraction. (a) Original images, each
with size 1280 × 1024 pixels. (b) Zoomed-in regions (13 × 10 pixels) from the masks applied
to the original images. (c) Simulated image capture. (d) Fourier transform (magnitude) of
the captured image, with marked sidebands used to extract the sub-exposure images. (e) A
subset of extracted sub-exposure images. (All 12 images can be successfully extracted; only
four of them are shown here.)

10

shown in Figures 5(c)-(d). It is seen that the blurry text in the full-exposure
capture becomes readable in the sub-exposure images.
In the experiment, the SLM pattern in horizontal and vertical directions
has the highest possible spatial frequency; that is, one-pixel-on (full reflection)
one-pixel-off (no reflection) SLM pattern. In other words, we cannot push the
sidebands further away from the center in Figure 5(b). (The reason why we
have an extended Fourier region is that one SLM pixel (13.62µm) corresponds
to about three sensor pixels (each with size 4.65µm). If we had one-to-one
correspondence, the horizontal and vertical sidebands would appear all the way
to the end of the Fourier regions because we have one-on one-off binary pattern
for them.) To encode and extract more sub-exposure images, the sub-exposure
images should be passed through an optical low-pass filter to further reduce the
frequency extent.
The spatial resolution of the optical system is controlled by several factors,
including the lens quality, SLM and sensor dimensions. Since the optical modulation is done by the SLM pixels, SLM pixel size is the main limiting factor
of the spatial resolution on the sensing side. On the computational side, the
FDMI technique requires band-limited signals, which is also a limiting factor on
spatial resolution. In the first experiment, the radius of a band-pass filter used
to recover a sub-exposure image is about one-ninth of the available bandwidth,
as seen in Figure 4; that is, the spatial frequency is reduced to one-ninth of the
maximum available resolution as a result of the FDMI process. In the second
experiment, the radius of the band-pass filters is about one-fourth of the available bandwidth; that is, the spatial resolution is reduced to one-quarter of the
maximum spatial resolution. It should, however, be noted that the bandwidth
of a natural image is typically small; therefore, the effective reduction in spatial
resolution is not as much in practice.
To quantify the overall spatial resolution of our system, we took two photos
of a Siemens Star, one without any optical modulation of the SLM (that is, full
reflection), and the other with an SLM pattern of maximum spatial frequency
(one-pixel-on one-pixel-off SLM pixels). The images are shown in Figure 6.
11

(a)

(b)

(c)

(d)

(e)

(f)

Figure 5: Sub-exposure image extraction with FDMI. (a) Recorded exposure-coded image,
(b) Fourier transform (magnitude) of the recorded image, (c) Sub-exposure image recovered
from the horizontal sideband, (d) Sub-exposure image recovered from the vertical sideband,
(e) Sub-exposure image recovered from the right diagonal sideband, (f) Sub-exposure image
recovered from the left diagonal sideband.

12

(a)

(b)

Figure 6: Spatial resolution of the optical system. (a) Image without any optical modulation
of the SLM, (b) Extracted sub-exposure image when the SLM pattern is vertical square wave
with one-pixel-on one-pixel-off SLM pixels.

The radii beyond which the radial lines can be distinguished are marked as
red circles in both images. According to [24], the spatial resolution is (Number
of cycles for the Siemens Star) × (Image height in pixels) / (2π(Radius of the
circle)). It turns out that the resolution of the optical system (without any SLM
modulation) is 138 line width per picture height; while it is 69 line width per
picture height for the sub-exposure image. The reduction in spatial resolution
is 50%, which is expected considering the spatial frequency of the SLM pattern.
In the third experiment, shown in Figure 7, the target object is a printout,
which is rotated as the camera captures an image. The exposure time of the
camera is set to 45 milliseconds. In the first 30 millisecond period, the SLM
pattern is a square wave in horizontal direction; in the second 15 millisecond
period, the SLM pattern is a square wave in vertical direction. The captured
image is shown in Figure 7(a). A zoomed-in region is shown in Figure 7(b),
where horizontal and vertical lines can be clearly seen. The Fourier transform
of the image is shown in Figure 7(c). The band-pass filters to be applied on the
sidebands and the low-pass filter to be applied on the baseband are also marked
on the Fourier transform. By applying the band-pass filter to the horizontal
sideband, the 30 millisecond sub-exposure image is extracted. By applying the

13

band-pass filter to the vertical sideband, the 15 millisecond sub-exposure image
is extracted. The baseband gives the sum of the sub-exposure images, as if there
were no optical modulation.
In Figure 7(d), the baseband image is shown. The motion blur prevents
seeing the details in the image, which has a full-exposure period of 45 milliseconds. The 30 millisecond sub-exposure image is shown in Figure 7(e); and the
15 millisecond sub-exposure image is shown in Figure 7(f). The readability of
the text and the visibility of details are much improved in the 15 millisecond
sub-exposure image.
In the fourth experiment, a ball is thrown in front of the camera. The
recorded image and a 15 millisecond sub-exposure image are shown in Figure 8.
The sub-exposure image is able to capture the ball shape with much less motion
blur.
Finally, we would like to demonstrate that more than two frames can be extracted through motion estimation. The results are shown in Figure 9. We applied horizontal grating in the first 15 milliseconds, followed by a 15 milliseconds
of no-reflection period, and finally vertical grating in the last 15 milliseconds.
Two sub-exposure images corresponding to the first part and the last part of
the exposure period are extracted, which are then used to estimate the motion
field using the optical flow estimation algorithm given in [25]. The estimated
flow vectors are divided to estimate intermediate frames through image warping,
resulting in 16 frames.

5. Conclusions
In this paper, we demonstrate a sub-exposure image extraction method,
which is based on allocating different regions in Fourier domain to different
sub-exposure images. While this theoretical idea is the main goal of the paper,
we demonstrate the feasibility of the idea with real life experiments using a
prototype optical system. The optical system has physical limitations due to
low spatial resolution of the SLM and low light efficiency, which is about 25%.

14

(a)

(b)

(c)

(d)

(e)

(f)
Figure 7: Sub-exposure image extraction with FDMI. (a) Recorded exposure-coded image,
(b) A zoomed-in region from (a), (c) Fourier transform (magnitude) of the recorded image,
(d) Image extracted from the baseband, corresponding to 45 millisecond exposure period, (e)
Image extracted from the horizontal sideband, corresponding to 30 millisecond exposure period, (f) Image extracted from the vertical sideband, corresponding to 15 millisecond exposure
period. Zoomed-in regions for (d), (e), and (f) are also shown.

15

(a)

(b)

Figure 8: Sub-exposure image extraction with FDMI. (a) Recorded exposure-coded image,
(b) Extracted sub-exposure image.

Better results would be achieved in the future with the development of highresolution sensors allowing pixel-wise exposure coding.
An advantage of the FDMI approach is the low computational complexity,
which involves Fourier transform and band-pass filtering. The sub-exposure
images from the sidebands and the full exposed image from the baseband are
easily extracted by taking Fourier transform, band-pass filtering, and inverse
Fourier transform.
Extraction of sub-exposure images can be used for different purposes. In
addition to high-speed imaging, one may try to estimate space-varying motion
blur, which could be used in scene understanding, segmentation, and spacevarying deblurring applications.

16

(a)

(b)

(c)
Figure 9: Extracting intermediate sub-exposure frames through optical flow estimation. (a)

17

Recorded exposure-coded image, and a zoomed-in region. (b) Fourier transform (magnitude)
of the image, and the estimated motion field between the sub-exposure images. (c) Sixteen
sub-exposure frames generated using the estimated motion field.

References
[1] A. Levin, R. Fergus, F. Durand, W. T. Freeman, Image and depth from
a conventional camera with a coded aperture, ACM Trans. on Graphics
26 (3) (2007) Article No. 70.
[2] C. Zhou, S. Nayar, What are good apertures for defocus deblurring?, in:
IEEE Int. Conf. on Computational Photography, 2009, pp. 1–8.
[3] C.-K. Liang, T.-H. Lin, B.-Y. Wong, C. Liu, H. H. Chen, Programmable
aperture photography: multiplexed light field acquisition, ACM Trans. on
Graphics 27 (3) (2008) Article No. 55.
[4] C. Zhou, S. Lin, S. K. Nayar, Coded aperture pairs for depth from defocus
and defocus deblurring, Int. Journal of Computer Vision 93 (1) (2011) 53–
72.
[5] Y. Sekikawa, S.-W. Leigh, K. Suzuki, Coded lens: Coded aperture for lowcost and versatile imaging, in: ACM SIGGRAPH, 2014, p. Article No.
59.
[6] M. S. Asif, A. Ayremlou, A. Veeraraghavan, R. Baraniuk, Flatcam: Replacing lenses with masks and computation, in: IEEE Int. Conf. on Computer
Vision, 2015, pp. 12–15.
[7] R. Raskar, A. Agrawal, J. Tumblin, Coded exposure photography: motion
deblurring using fluttered shutter, ACM Trans. on Graphics 25 (3) (2006)
795–804.
[8] J. Holloway, A. C. Sankaranarayanan, A. Veeraraghavan, S. Tambe, Flutter
shutter video camera for compressive sensing of videos, in: IEEE Int. Conf.
on Computational Photography, 2012, pp. 1–9.
[9] S. K. Nayar, V. Branzoi, Adaptive dynamic range imaging: Optical control
of pixel exposures over space and time, in: IEEE Int. Conf. on Computer
Vision, 2003, pp. 1168–1175.
18

[10] X. Lin, J. Suo, G. Wetzstein, Q. Dai, R. Raskar, Coded focal stack photography, in: IEEE Int. Conf. on Computational Photography, 2013, pp.
1–9.
[11] T. Portz, L. Zhang, H. Jiang, Random coded sampling for high-speed hdr
video, in: IEEE Int. Conf. on Computational Photography, 2015, pp. 1–8.
[12] G. Bub, M. Tecza, M. Helmes, P. Lee, P. Kohl, Temporal pixel multiplexing
for simultaneous high-speed, high-resolution imaging, Nature Methods 7 (3)
(2010) 209–211.
[13] M. Gupta, A. Agrawal, A. Veeraraghavan, S. G. Narasimhan, Flexible voxels for motion-aware videography, in: European Conf. on Computer Vision,
2010, pp. 100–114.
[14] D. Reddy, A. Veeraraghavan, R. Chellappa, P2c2: Programmable pixel
compressive camera for high speed imaging, in: IEEE Int. Conf. on Computer Vision and Pattern Recognition, 2011, pp. 329–336.
[15] Y. Hitomi, J. Gu, M. Gupta, T. Mitsunaga, S. K. Nayar, Video from a
single coded exposure photograph using a learned over-complete dictionary,
in: IEEE Int. Conf. on Computer Vision, 2011, pp. 287–294.
[16] D. Liu, J. Gu, Y. Hitomi, M. Gupta, T. Mitsunaga, S. K. Nayar, Efficient
space-time sampling with pixel-wise coded exposure for high-speed imaging,
IEEE Trans. on Pattern Analysis and Machine Intelligence 36 (2) (2014)
248–260.
[17] J. Gu, Y. Hitomi, T. Mitsunaga, S. Nayar, Coded rolling shutter photography: Flexible space-time sampling, in: IEEE Int. Conf. on Computational
Photography, 2010, pp. 1–8.
[18] P. Llull, X. Liao, X. Yuan, J. Yang, D. Kittle, L. Carin, G. Sapiro,
D. J. Brady, Coded aperture compressive temporal imaging, Optics Express 21 (9) (2013) 10526–10545.

19

[19] R. Koller, L. Schmid, N. Matsuda, T. Niederberger, L. Spinoulas, O. Cossairt, G. Schuster, A. K. Katsaggelos, High spatio-temporal resolution video
with compressed sensing, Optics Express 23 (12) (2015) 15992–16007.
[20] B. K. Gunturk, M. Feldman, Frequency division multiplexed imaging, in:
IS&T/SPIE Electronic Imaging Conf., 2013, pp. 86600P–86600P.
[21] C. Gao, N. Ahuja, H. Hua, Active aperture control and sensor modulation
for flexible imaging, in: IEEE Int. Conf. on Computer Vision and Pattern
Recognition, 2007, pp. 1–8.
[22] S. K. Nayar, V. Branzoi, T. E. Boult, Programmable imaging: Towards a
flexible camera, Int. Journal of Computer Vision 70 (1) (2006) 7–22.
[23] H. Mannami, R. Sagawa, Y. Mukaigawa, T. Echigo, Y. Yagi, High dynamic range camera using reflective liquid crystal, in: IEEE Int. Conf. on
Computer Vision, 2007, pp. 1–8.
[24] C. Loebich, D. Wueller, B. Klingen, A. Jaeger, Digital camera resolution
measurement using sinusoidal Siemens Stars, in: SPIE Electronic Imaging,
Digital Photography III, 2007, pp. 65020N1–11.
[25] C. Liu, Beyond pixels: exploring new representations and applications for
motion analysis, MIT, 2009.

20

