Why do radio telescopes, but not shorter wavelengths, have this Big Data problem? Only with new radio telescope arrays like ALMA and SKA, have I heard of this "problem" with too much data come up. Too big to store. That one cannot archive it all for future studies, but that one has to select beforehand what fraction of the data to analyse and erase the rest, and the most, of it.

Shouldn't it be the other way around, that shorter wavelength telescopes like those in IR and the visible and UV generate more data because they have higher resolution per unit of aperture?
Shouldn't it be the other way around, that shorter wavelength telescopes like those in IR and the visible and UV generate more data because they have higher resolution per unit of aperture?


This is a very interesting question! 

My answer is no, not currently, but someday it may be the case.

Because we have big fast computer and fiber optic technology, we can generate images from arrays of antennas computationally. We can form images and even implement adaptive optics for wavefront correction at radio frequencies in software (rather than mechanical actuators correcting wavefronts by brute-force bending mirrors).

An order-of-magnitude description of ALMA is 100 dishes each one delivering 1 GHz of bandwidth, and SKA arrays will be far larger. While the Event Horizon Telescope literally records all raw data on boxes of hard drives that's not currently possible for ALMA and larger arrays, so only much smaller, heavily processed intermediate results files can be saved for later analysis and fine-tuning of calibrations and parameters.

For "shorter wavelength telescopes" in this context it's better to think of them as "higher frequency telescopes". Even a 100 nm wide band at say 800 nm (near IR where wavefront correction is working well represents 47,000 GHz! Even with conversion to a baseband frequency we don't have the ability to digitize signals at that rate, much less perform correlations of multiple signals.

So instead we let the light do the interferometry for us! 

An optical telescope is just a big, simple, one-trick-pony optical computer. We design the surfaces to perform an integral of the electric field at the aperture. The center pixel receives the integral of all electric fields incident on the telescopes clear aperture without any phase bias. A pixel at one edge of the sensor receives the integral of the same electric fields times a phase function that increases linearly from one side to the other. 

One day if and/or when detection and computing speeds and parallelisms are big enough, I am sure an electronic optical computer will probably be built. 



For more on optical telescopes versus computational interferometry see answers to 


Would “layered” radio interferometry work?
Could mirrors be replaced with CCDs?
How would a Fast Fourier Transform Telescope work without a mirror, dish or lenses?