Cold war reconnaissance imagery reloaded: orthorectifying the 1960s in high resolution
Riembauer, Guido
CORONA is the code name for the first optical reconnaissance satellite mission of the United States (1960-1972). The goal of the mission was to produce high-resolution analog photos of most of the Earth’s surface, especially of political hot spots and military locations. Due to the regular recordings, large areas could be continuously monitored and evaluated for the Department of Defense. Until 1995, more than 800,000 photos remained secret and were then made publicly available by the US Geological Survey on the order of President Bill Clinton. The high-resolution CORONA photos (2 m to 60 cm pixel resolution) are available as scans for a fee from the USGS and represent a unique source of information for science, archaeology and other disciplines. Since the camera systems of the CORONA satellites have a special panoramic distortion, common linear methods cannot be used for the orthorectification of the scans. mundialis has developed an innovative free and open source technology to rectify these unreferenced scans of CORONA photos to current map references and published it in GRASS GIS 7.9. This photogrammetric solution models the CORONA camera mathematically and thus enables a precise referencing of the CORONA data. In many parts of the world, the CORONA scenes have preserved images of a landscape that predates the most intrusive infrastructural and land-use projects of modern times. Traditional architecture, agricultural patterns and settlement systems can be observed in great clarity on CORONA imagery. This makes CORONA a precious resource in fields such as archaeological and historical geography.
 He's a developer and data analyst and memorialist and has specialized in ROTAR remote sensing. He will be delivering the Cold War in reconnaissance imagery loaded up to rectifying the 60s in high resolution. So I will play the recorded session for you. I will be talking about Cold War reconnaissance imagery reloaded, auto rectifying the 1960s in high resolution. Specifically, this talk will be about the Corona Satellite program by the United States in the 1960s. This is the outline of my talk. First I would like briefly to talk about what Mondialis does before going into detail about the Corona Satellite program and its applications. Then I will show the imaging geometry and distortions that are inherent in this data and will showcase our image rectification workflow that we developed to cope with these distortions and to auto rectify entire Corona scenes. A few brief words on Mondialis. We are a remote sensing company based in Bonn, Germany and we are focusing on the analysis of large amounts of Earth observation data in time series using cloud environments. Mostly we use free data such as Copernicus Sentinel data and we are also committed to using free and open source software exclusively. With Marcus Metz and Markus Niedler we also have two GRAS GIS core developers in our team and in our projects we regularly contribute to the GRAS GIS development or to the development of GRAS GIS add-ons as in this project. Now I am aware that you have heard enough about Corona in the past one and a half years but let us give this name a different meaning as it is also the name of a satellite program that was launched by the United States in the 1960s. The goal was to have a reconnaissance mission giving an idea of what is going on on the ground in high resolution especially since it was the Cold War of the Soviet Union and their allies. The entire program consisted of 144 satellites in eight keyhole missions. Now this is of course a lot of satellites but the mission duration of a single satellite was rather low because the satellite operated with two panoramic cameras that were operated with a physical film and once the film was used up there was no use for the satellite mission any further. Also the retrieval of the physical film was rather spectacular as you see in this image. Once the film was used up the satellite would tilt towards Earth and eject a capsule with the film inside into the Earth atmosphere and then an aircraft would attempt to catch this capsule mid-flight. The capsule was also designed to survive in salt water for two days such that it could also be retrieved by boat but after two days it would dissolve and sink in order that the data would not fall into the wrong hands. Now today's value in this data lies in the spatial resolution as you can see in this image because the satellite flew rather low at around 160 kilometers orbit. The latest corona missions yield effective spatial resolutions of 2.75 or 1.8 meters. In this image you can already see individual houses, cars, vehicles and streets so it's a very valuable data source dating back 60 years. This gives you another idea of why this data is so valuable. This is the city of Tunis in Tunisia 60 years ago versus how it looks today. And as you can see a lot of things change on the Earth's surface in the meantime so with corona we have a unique way of looking back in time that no other data source provides us. So luckily all this data was declassified in 1995 and is now available to download from the USGS Earth Explorer. One scene costs $30 but scenes that have been ordered previously are usually available for free simply because the only cost is associated with scanning the physical film. So any data that has been processed before can be downloaded for free. Now what can you do with this data? As I said before there's a lot of valuable information in there simply because it dates so much back in time even further than the oldest lens that archives. So there's a lot of valuable information concerning for example long term land cover mapping and monitoring. You could monitor vegetation or forest changes throughout the time or the urban sprawl throughout 60 years. In this image here you see an harbour village in Algeria and compared to today you can also see how this city sprawled enormously in the past 60 years. This information however is hard to automatically extract simply because in the corona data we have no spectral information. And so all automatic approaches are limited to feature identification and extraction methods. However in methods where we can focus on the qualitative interpretation of data corona data benefits a lot. And one very classic example for this is archaeology to give you an example. This is a corona scene of southern Turkey from the 60s and this scene was used to identify potential archaeological sites simply because human historical structures such as dwellings or infrastructure altered the local topography that could be visualized with high resolution data like this one in order to identify potential sites that are interesting for ecology. In the meantime since the 60s a lot of activity was going on on the ground a lot of construction land cover change so even using today's high resolution data would not yield to this information because the land cover changed so much and buildings and cities were constructed in the meantime. Here's another zoom in of an example we see an individual farm and around it you can clearly see that there seems to be some kind of human-made structure which might have been fortification. So again a very valuable data source for archaeology. So why is corona data not that widely used yet? This is because there are massive distortions in the imagery. On the left you see a diagram showing the imaging geometry of the satellites. It has two panoramic cameras one tilted slightly to the front and one tilted slightly to the back. And since it is a panoramic camera there's a lot of distortion especially at the edges of one scene. So one scene is approximately between 200 and 250 kilometers wide and 10 to 20 kilometers narrow only. So if you're interested in maybe just a very small area of the Earth's surface that is located in the middle of such a scene you might work well with classic linear auto-rectification methods that assume essential perspective. However, if you were to auto-rectify an entire scene there is an urgent need to cope with these distortions due to the panoramic effects here. Further, there is no calibration data available that you would use for today's systems such as the position and different angles of the air or spacecraft at the time of the acquisition. This was the 1960s and such data could not be collected so any transformation model needs to estimate these parameters. Here's an example of different auto-rectification methods of the same corona scene. On top you see just a linear classic auto-rectification approach applied and on the bottom you see an image where the panoramic distortion has been accounted for. And in the top image you can clearly see that the offset especially at the edges is in the order of magnitude of tens of kilometers. So this is not an appropriate way to go to auto-rectify an entire scene. So what is necessary is a transformation model that deals with this and this is done basically by a cylindrical shape that represents the physical film surface to be put in this transformation model and such a model was already introduced in literature by Zohn et al. in 2004. But however in this project we didn't find any implementation in free and open source software. Because of this we implemented this model in the iAuto photoshoot in GrassGIS. So the iAuto photoshoot is the standard rectification workflow in order to auto-rectify aerial or space-borne images normally assuming just the central perspective cameras but now we extended it to also be able to cope with this panoramic distortions. It is part of GrassGIS from version 7.9 onwards so if you install GrassGIS locally on your computer you already have all the software you need to auto-rectify the corona scenes and of course it's free and open source. Finally I would like to show you the entire workflow of rectifying an entire corona scene. It begins with the scene recomposition because if you order a scene from the USGS, my name is Guido Rimbauer from Mundiales and I will be talking about the city of Rimbauer. The example for this is archaeology and one tells it slightly to the different angles of the model. So what is necessary is a transformation model that deals with this and this is done basically by a cylindrical shape that represents the physical film surface to be put in this transformation model and such a model was already introduced in literature by Zohn et al. in 2004 but however in this project we didn't find any implementation in free and open source software. Because of this we implemented this model in the iAuto photoshoot in GrassGIS. So the iAuto photoshoot is the standard rectification workflow in order to auto-rectify aerial or space-borne images normally assuming just the central perspective cameras but now we extended it to also be able to cope with this panoramic distortions. It is part of GrassGIS from version 7.9 onwards so if you install GrassGIS locally on your computer you already have all the software you need to auto-rectify the corona scenes and of course it's free and open source. Finally I would like to show you the entire workflow of rectifying an entire corona scene. It begins with the scene recomposition because if you order a scene from the USGS Earth Explorer what you get is not one entire TIFF file but you get four individual TIFF files that are split but have some overlap. This is simply because the film cannot be scanned in one go but it's scanned in four individual rounds and on each scene pair is a significant overlap in order to recompose the scene. So you could use automatic stitching methods or just go with a simulated auto-rectification approach by finding points that are the same in the respective image pairs. Next comes the part that is most time consuming and this is the collection of ground control points. This means in order to auto-rectify your scene you would need a set of points in your corona imagery that you can assign a specific and precise location to. For this obviously one needs reference data. If there's infrastructure for example which is always the best choice then you could use open street map data as in this example. You can use for example road crossings or bridges. In this case we used a crossing at the airport of Tunis of the runway and this crossing can also clearly be identified in the same or in the corresponding corona scene here. However this is not always possible and sometimes there is no infrastructure in your scene and then you would need to go for natural features such as for example river confluences as in this example or characteristic rock formations for example. But again you would need to really have a close look to find this specific point in both data sources here in the respective corona scene. We found that collecting 20 to 40 ground control points over an entire scene is enough to give a good auto rectification but it would be necessary to distribute these points as evenly as possible over the image in order to have a good result of the auto rectification in each part of the image. With this data now we can do the actual auto rectification and for this we need some additional inputs. So first we need the patch draw scene from step one. Then we need some basic scene metadata. When you order a scene from the USGS Earth Explorer you get some very basic coordinates of the scene which are not very precise but they give you a very rough idea of where you are on the Earth. And this is important for the parameter estimation of the transformation model in order to have some very basic initial guesses. Then you need a digital elevation model in order to cope with distortions due to topography and you need the set of ground control points. With all these inputs you can then estimate the parameters of the transformation model and by doing this each of the ground control points gets assigned a root mean square error that depicts how well this point fits to the overall model. With this you can identify points that are perhaps that have a high error and do not fit well to the model maybe because in the ground control point collection step some points were misplaced. So you can use this step to iteratively adapt your set of ground control points until you are satisfied with the overall root mean square error. And this what is achievable what we found is more or less 5 to 10 meters root mean square error in the models is realistic. Then you can run the actual auto rectification and here the final scene. This is then how it looks like if you do it for an entire strip of scenes this is the Nile Valley in Sudan which is also very interesting from an archaeological point of view. And using multiple scenes you can also use the overlap of individual scenes to identify ground control points that are visible in each scene pair in order to avoid double work. This brings me to the end of my talk. I thank you very much for your attention. I hope I made you a bit curious on the corona mission and the data and also made you curious about trying grass.js for its rectification. And now I'm looking forward to your questions. Okay. First apologies for the video cut just my bad. And also apologies for the issues listening to the audio because I didn't have maybe because I was using the earbuds but I hope you get a grasp of the talk. So okay we have one question. Is the output being used professionally? Was this a speculative project or did someone commission it? If you can talk about that. Yes, thank you very much for the question. Also thank you for hosting the presentation. Yes, so we had a customer that was interested in the corona scenes for archaeological purposes although I can't point out the exact one but it is being used professionally and also in the let's say in the archaeological world. This corona data set is known very well. So it's a far used data set. Yes, I have a couple of questions. Or well maybe the first one is the comment. Apart from archaeology have you identified other potential use cases like I can imagine for example coastal evolution or line coast changes for example? Yes, of course all kinds of land cover mapping basically is applicable for this and it's very interesting because this data goes back a long time. For example there was also a study not conducted by us but by others using this data that also looked at forest change in the Amazon basin. Now you can even look at data way back from the 60s which is a nice extension tool let's say the Landsat archive for example. So there's far more use cases than archaeology but as I also said in the talk it's a bit hard to automate this simply because we only have so we have the distortions and second we have only one spectral channel if you will which is not even a spectral channel but a physical film. You haven't mentioned exactly or shown any coverage map of the imagery. Is this imagery worldwide or because you've shown data from Africa mostly but actually you mentioned now South America and I understand also Russia and the Cold War areas of interest but there are also data from Europe, North America. Yes there's a lot of data from North America so in general it is worldwide and was available so the focus for historical reasons obviously was on Asia and the Soviet Union and also the Middle East which is also very interesting for archaeological purposes but there are data available for North America, Europe, literally anywhere in the world. At some places there might only be one specific scene at others there are hundreds it really differs a lot. Okay I have a couple questions from the audience, did you try AI, super resolution method? Super resolution to identify specific features so we know because we didn't analyze the data itself yet but this we only prepared it for the rectification but this would be a very interesting approach to identify features and objects in the data itself but again this is then the application world that we didn't touch in this project yet. Another question is have you looked at AMC's stereo pipelines method for working with KH and how it compares to your methods? No we have looked at, I'm not sure if that's the one, there is one from I think the University of Oklahoma I'm not sure if that's the one that this question refers to, there is a stereo pipeline and we also tried to use it but we weren't able to contact or we didn't get any feedback from the hosts of this platform because there is a service that allows you to auto rectify Corona scenes online also but we weren't able to get in contact with the host of the system unfortunately. There was a question that gets removed from the chat or no? Well I have one, are the results of your studies there rectified images available somewhere or it's just owned by the customer? Exactly and perhaps I'm not sure about this but the customer may publish them as well for example in a WMS or somewhere but in general what I would like to point out again is that also the data you can get from the USDS Earth Explorer so the raw data is available and there is also free data already there. Okay there are folks already sharing the coverage in the chat for anyone interested in. Okay yeah thank you Matt, is there any community effort to georectify this data so to outsource I guess the time consuming human dependent part of the process? I think there was, there is this Corona Cast project that is also the one with the online possibility to autorectify. But I don't know if this project is still ongoing so from our side we didn't put any effort in the community building here but in theory of course it would be very nice because this is a very rich data source and everybody is just waiting to have it autorectified to get started and the autorectification is really what hinders the analysis ready data. Any initiative that would like to use this workflow is very welcome of course and we'd be happy to help them. I can imagine a kind of a web service to help creating the control, the ground control points. Okay any, how good is the special accuracy? Yeah this really depends so in the best scenes we would get an offset of around 5 meters for the entire scene. There will be regions where there is absolutely no offset and you have a perfect match and there will be regions where you have 20, 30 meters offset. However there are some scenes that can't be really autorectified at all or where we still have hundreds of meters of offset in there simply because there are still so many factors like the absolute position and movement of the spacecraft that was simply not documented in that time. So there might be occasionally a scene where these factors are really hard to come by with also with our autorectification methods. Yeah I can imagine it has to be challenging to work with data from the 70s or the 60s. Any other questions from the audience? Okay then thank you very much Guido. Yeah very nice, a very interesting project. Now we have five minutes for Martín so see you soon. Thank you.
 He's a developer and data analyst and memorialist and has specialized in ROTAR remote sensing.  He will be delivering the Cold War in reconnaissance imagery loaded up to rectifying the 60s in  high resolution.  So I will play the recorded session for you.  I will be talking about Cold War reconnaissance imagery reloaded, auto rectifying the 1960s  in high resolution.  Specifically, this talk will be about the Corona Satellite program by the United States  in the 1960s.  This is the outline of my talk.  First I would like briefly to talk about what Mondialis does before going into detail  about the Corona Satellite program and its applications.  Then I will show the imaging geometry and distortions that are inherent in this data  and will showcase our image rectification workflow that we developed to cope with these  distortions and to auto rectify entire Corona scenes.  A few brief words on Mondialis.  We are a remote sensing company based in Bonn, Germany and we are focusing on the analysis  of large amounts of Earth observation data in time series using cloud environments.  Mostly we use free data such as Copernicus Sentinel data and we are also committed to  using free and open source software exclusively.  With Marcus Metz and Markus Niedler we also have two GRAS GIS core developers in our team  and in our projects we regularly contribute to the GRAS GIS development or to the development  of GRAS GIS add-ons as in this project.  Now I am aware that you have heard enough about Corona in the past one and a half years  but let us give this name a different meaning as it is also the name of a satellite program  that was launched by the United States in the 1960s.  The goal was to have a reconnaissance mission giving an idea of what is going on on the  ground in high resolution especially since it was the Cold War of the Soviet Union and  their allies.  The entire program consisted of 144 satellites in eight keyhole missions.  Now this is of course a lot of satellites but the mission duration of a single satellite  was rather low because the satellite operated with two panoramic cameras that were operated  with a physical film and once the film was used up there was no use for the satellite  mission any further.  Also the retrieval of the physical film was rather spectacular as you see in this image.  Once the film was used up the satellite would tilt towards Earth and eject a capsule with  the film inside into the Earth atmosphere and then an aircraft would attempt to catch  this capsule mid-flight.  The capsule was also designed to survive in salt water for two days such that it could  also be retrieved by boat but after two days it would dissolve and sink in order that the  data would not fall into the wrong hands.  Now today's value in this data lies in the spatial resolution as you can see in this  image because the satellite flew rather low at around 160 kilometers orbit.  The latest corona missions yield effective spatial resolutions of 2.75 or 1.8 meters.  In this image you can already see individual houses, cars, vehicles and streets so it's  a very valuable data source dating back 60 years.  This gives you another idea of why this data is so valuable.  This is the city of Tunis in Tunisia 60 years ago versus how it looks today.  And as you can see a lot of things change on the Earth's surface in the meantime so  with corona we have a unique way of looking back in time that no other data source provides  us.  So luckily all this data was declassified in 1995 and is now available to download from  the USGS Earth Explorer.  One scene costs $30 but scenes that have been ordered previously are usually available for  free simply because the only cost is associated with scanning the physical film.  So any data that has been processed before can be downloaded for free.  Now what can you do with this data?  As I said before there's a lot of valuable information in there simply because it dates  so much back in time even further than the oldest lens that archives.  So there's a lot of valuable information concerning for example long term land cover  mapping and monitoring.  You could monitor vegetation or forest changes throughout the time or the urban sprawl throughout  60 years.  In this image here you see an harbour village in Algeria and compared to today you can also  see how this city sprawled enormously in the past 60 years.  This information however is hard to automatically extract simply because in the corona data we  have no spectral information.  And so all automatic approaches are limited to feature identification and extraction methods.  However in methods where we can focus on the qualitative interpretation of data corona  data benefits a lot.  And one very classic example for this is archaeology to give you an example.  This is a corona scene of southern Turkey from the 60s and this scene was used to identify  potential archaeological sites simply because human historical structures such as dwellings  or infrastructure altered the local topography that could be visualized with high resolution  data like this one in order to identify potential sites that are interesting for ecology.  In the meantime since the 60s a lot of activity was going on on the ground a lot of construction  land cover change so even using today's high resolution data would not yield to this information  because the land cover changed so much and buildings and cities were constructed in  the meantime.  Here's another zoom in of an example we see an individual farm and around it you can clearly  see that there seems to be some kind of human-made structure which might have been fortification.  So again a very valuable data source for archaeology.  So why is corona data not that widely used yet?  This is because there are massive distortions in the imagery.  On the left you see a diagram showing the imaging geometry of the satellites.  It has two panoramic cameras one tilted slightly to the front and one tilted slightly to the  back.  And since it is a panoramic camera there's a lot of distortion especially at the edges  of one scene.  So one scene is approximately between 200 and 250 kilometers wide and 10 to 20 kilometers  narrow only.  So if you're interested in maybe just a very small area of the Earth's surface that is  located in the middle of such a scene you might work well with classic linear auto-rectification  methods that assume essential perspective.  However, if you were to auto-rectify an entire scene there is an urgent need to cope with  these distortions due to the panoramic effects here.  Further, there is no calibration data available that you would use for today's systems such  as the position and different angles of the air or spacecraft at the time of the acquisition.  This was the 1960s and such data could not be collected so any transformation model needs  to estimate these parameters.  Here's an example of different auto-rectification methods of the same corona scene.  On top you see just a linear classic auto-rectification approach applied and on the bottom you see  an image where the panoramic distortion has been accounted for.  And in the top image you can clearly see that the offset especially at the edges is in the  order of magnitude of tens of kilometers.  So this is not an appropriate way to go to auto-rectify an entire scene.  So what is necessary is a transformation model that deals with this and this is done basically  by a cylindrical shape that represents the physical film surface to be put in this transformation  model and such a model was already introduced in literature by Zohn et al. in 2004.  But however in this project we didn't find any implementation in free and open source  software.  Because of this we implemented this model in the iAuto photoshoot in GrassGIS.  So the iAuto photoshoot is the standard rectification workflow in order to auto-rectify aerial or  space-borne images normally assuming just the central perspective cameras but now we extended  it to also be able to cope with this panoramic distortions.  It is part of GrassGIS from version 7.9 onwards so if you install GrassGIS locally on your  computer you already have all the software you need to auto-rectify the corona scenes  and of course it's free and open source.  Finally I would like to show you the entire workflow of rectifying an entire corona scene.  It begins with the scene recomposition because if you order a scene from the USGS, my name  is Guido Rimbauer from Mundiales and I will be talking about the city of Rimbauer.  The example for this is archaeology and one tells it slightly to the different angles  of the model.  So what is necessary is a transformation model that deals with this and this is done basically  by a cylindrical shape that represents the physical film surface to be put in this transformation  model and such a model was already introduced in literature by Zohn et al. in 2004 but however  in this project we didn't find any implementation in free and open source software.  Because of this we implemented this model in the iAuto photoshoot in GrassGIS.  So the iAuto photoshoot is the standard rectification workflow in order to auto-rectify aerial or  space-borne images normally assuming just the central perspective cameras but now we extended  it to also be able to cope with this panoramic distortions.  It is part of GrassGIS from version 7.9 onwards so if you install GrassGIS locally on your  computer you already have all the software you need to auto-rectify the corona scenes  and of course it's free and open source.  Finally I would like to show you the entire workflow of rectifying an entire corona scene.  It begins with the scene recomposition because if you order a scene from the USGS Earth Explorer  what you get is not one entire TIFF file but you get four individual TIFF files that are  split but have some overlap.  This is simply because the film cannot be scanned in one go but it's scanned in four  individual rounds and on each scene pair is a significant overlap in order to recompose  the scene.  So you could use automatic stitching methods or just go with a simulated auto-rectification  approach by finding points that are the same in the respective image pairs.  Next comes the part that is most time consuming and this is the collection of ground control  points.  This means in order to auto-rectify your scene you would need a set of points in your corona  imagery that you can assign a specific and precise location to.  For this obviously one needs reference data.  If there's infrastructure for example which is always the best choice then you could use  open street map data as in this example.  You can use for example road crossings or bridges.  In this case we used a crossing at the airport of Tunis of the runway and this crossing can  also clearly be identified in the same or in the corresponding corona scene here.  However this is not always possible and sometimes there is no infrastructure in your scene and  then you would need to go for natural features such as for example river confluences as in  this example or characteristic rock formations for example.  But again you would need to really have a close look to find this specific point in  both data sources here in the respective corona scene.  We found that collecting 20 to 40 ground control points over an entire scene is enough to give  a good auto rectification but it would be necessary to distribute these points as evenly  as possible over the image in order to have a good result of the auto rectification in  each part of the image.  With this data now we can do the actual auto rectification and for this we need some additional  inputs.  So first we need the patch draw scene from step one.  Then we need some basic scene metadata.  When you order a scene from the USGS Earth Explorer you get some very basic coordinates  of the scene which are not very precise but they give you a very rough idea of where you  are on the Earth.  And this is important for the parameter estimation of the transformation model in order to have  some very basic initial guesses.  Then you need a digital elevation model in order to cope with distortions due to topography  and you need the set of ground control points.  With all these inputs you can then estimate the parameters of the transformation model  and by doing this each of the ground control points gets assigned a root mean square error  that depicts how well this point fits to the overall model.  With this you can identify points that are perhaps that have a high error and do not  fit well to the model maybe because in the ground control point collection step some  points were misplaced.  So you can use this step to iteratively adapt your set of ground control points until you  are satisfied with the overall root mean square error.  And this what is achievable what we found is more or less 5 to 10 meters root mean square  error in the models is realistic.  Then you can run the actual auto rectification and here the final scene.  This is then how it looks like if you do it for an entire strip of scenes this is the  Nile Valley in Sudan which is also very interesting from an archaeological point of view.  And using multiple scenes you can also use the overlap of individual scenes to identify  ground control points that are visible in each scene pair in order to avoid double work.  This brings me to the end of my talk.  I thank you very much for your attention.  I hope I made you a bit curious on the corona mission and the data and also made you curious  about trying grass.js for its rectification.  And now I'm looking forward to your questions.  Okay.  First apologies for the video cut just my bad.  And also apologies for the issues listening to the audio because I didn't have maybe because  I was using the earbuds but I hope you get a grasp of the talk.  So okay we have one question.  Is the output being used professionally?  Was this a speculative project or did someone commission it?  If you can talk about that.  Yes, thank you very much for the question.  Also thank you for hosting the presentation.  Yes, so we had a customer that was interested in the corona scenes for archaeological purposes  although I can't point out the exact one but it is being used professionally and also  in the let's say in the archaeological world.  This corona data set is known very well.  So it's a far used data set.  Yes, I have a couple of questions.  Or well maybe the first one is the comment.  Apart from archaeology have you identified other potential use cases like I can imagine  for example coastal evolution or line coast changes for example?  Yes, of course all kinds of land cover mapping basically is applicable for this and it's  very interesting because this data goes back a long time.  For example there was also a study not conducted by us but by others using this data that also  looked at forest change in the Amazon basin.  Now you can even look at data way back from the 60s which is a nice extension tool let's  say the Landsat archive for example.  So there's far more use cases than archaeology but as I also said in the talk it's a bit  hard to automate this simply because we only have so we have the distortions and second  we have only one spectral channel if you will which is not even a spectral channel but a  physical film.  You haven't mentioned exactly or shown any coverage map of the imagery.  Is this imagery worldwide or because you've shown data from Africa mostly but actually  you mentioned now South America and I understand also Russia and the Cold War areas of interest  but there are also data from Europe, North America.  Yes there's a lot of data from North America so in general it is worldwide and was available  so the focus for historical reasons obviously was on Asia and the Soviet Union and also  the Middle East which is also very interesting for archaeological purposes but there are  data available for North America, Europe, literally anywhere in the world.  At some places there might only be one specific scene at others there are hundreds it really  differs a lot.  Okay I have a couple questions from the audience, did you try AI, super resolution method?  Super resolution to identify specific features so we know because we didn't analyze the data  itself yet but this we only prepared it for the rectification but this would be a very  interesting approach to identify features and objects in the data itself but again this  is then the application world that we didn't touch in this project yet.  Another question is have you looked at AMC's stereo pipelines method for working with  KH and how it compares to your methods?  No we have looked at, I'm not sure if that's the one, there is one from I think the University  of Oklahoma I'm not sure if that's the one that this question refers to, there is a stereo  pipeline and we also tried to use it but we weren't able to contact or we didn't get  any feedback from the hosts of this platform because there is a service that allows you  to auto rectify Corona scenes online also but we weren't able to get in contact with  the host of the system unfortunately.  There was a question that gets removed from the chat or no?  Well I have one, are the results of your studies there rectified images available somewhere  or it's just owned by the customer?  Exactly and perhaps I'm not sure about this but the customer may publish them as well  for example in a WMS or somewhere but in general what I would like to point out again is that  also the data you can get from the USDS Earth Explorer so the raw data is available and there  is also free data already there.  Okay there are folks already sharing the coverage in the chat for anyone interested in.  Okay yeah thank you Matt, is there any community effort to georectify this data so to outsource  I guess the time consuming human dependent part of the process?  I think there was, there is this Corona Cast project that is also the one with the online  possibility to autorectify.  But I don't know if this project is still ongoing so from our side we didn't put any  effort in the community building here but in theory of course it would be very nice because  this is a very rich data source and everybody is just waiting to have it autorectified to  get started and the autorectification is really what hinders the analysis ready data.  Any initiative that would like to use this workflow is very welcome of course and we'd  be happy to help them.  I can imagine a kind of a web service to help creating the control, the ground control points.  Okay any, how good is the special accuracy?  Yeah this really depends so in the best scenes we would get an offset of around 5 meters for  the entire scene.  There will be regions where there is absolutely no offset and you have a perfect match and  there will be regions where you have 20, 30 meters offset.  However there are some scenes that can't be really autorectified at all or where we still  have hundreds of meters of offset in there simply because there are still so many factors  like the absolute position and movement of the spacecraft that was simply not documented  in that time.  So there might be occasionally a scene where these factors are really hard to come by with  also with our autorectification methods.  Yeah I can imagine it has to be challenging to work with data from the 70s or the 60s.  Any other questions from the audience?  Okay then thank you very much Guido.  Yeah very nice, a very interesting project.  Now we have five minutes for Martín so see you soon.  Thank you.
FOSS4G_-_Cold_war_reconnaissance_imagery_reloaded__duoDyN-PDqg_0_131.png FOSS4G_-_Cold_war_reconnaissance_imagery_reloaded__duoDyN-PDqg_132_575.png FOSS4G_-_Cold_war_reconnaissance_imagery_reloaded__duoDyN-PDqg_576_695.png FOSS4G_-_Cold_war_reconnaissance_imagery_reloaded__duoDyN-PDqg_696_1055.png FOSS4G_-_Cold_war_reconnaissance_imagery_reloaded__duoDyN-PDqg_1056_1607.png FOSS4G_-_Cold_war_reconnaissance_imagery_reloaded__duoDyN-PDqg_1608_1739.png FOSS4G_-_Cold_war_reconnaissance_imagery_reloaded__duoDyN-PDqg_1740_2855.png FOSS4G_-_Cold_war_reconnaissance_imagery_reloaded__duoDyN-PDqg_2856_4139.png FOSS4G_-_Cold_war_reconnaissance_imagery_reloaded__duoDyN-PDqg_4140_5771.png FOSS4G_-_Cold_war_reconnaissance_imagery_reloaded__duoDyN-PDqg_5772_6023.png FOSS4G_-_Cold_war_reconnaissance_imagery_reloaded__duoDyN-PDqg_6024_6323.png FOSS4G_-_Cold_war_reconnaissance_imagery_reloaded__duoDyN-PDqg_6324_6983.png FOSS4G_-_Cold_war_reconnaissance_imagery_reloaded__duoDyN-PDqg_6984_8663.png FOSS4G_-_Cold_war_reconnaissance_imagery_reloaded__duoDyN-PDqg_8664_9683.png FOSS4G_-_Cold_war_reconnaissance_imagery_reloaded__duoDyN-PDqg_9684_10151.png FOSS4G_-_Cold_war_reconnaissance_imagery_reloaded__duoDyN-PDqg_10152_12203.png FOSS4G_-_Cold_war_reconnaissance_imagery_reloaded__duoDyN-PDqg_12204_12923.png FOSS4G_-_Cold_war_reconnaissance_imagery_reloaded__duoDyN-PDqg_12924_14567.png FOSS4G_-_Cold_war_reconnaissance_imagery_reloaded__duoDyN-PDqg_14568_14927.png FOSS4G_-_Cold_war_reconnaissance_imagery_reloaded__duoDyN-PDqg_14928_15047.png FOSS4G_-_Cold_war_reconnaissance_imagery_reloaded__duoDyN-PDqg_15048_15119.png FOSS4G_-_Cold_war_reconnaissance_imagery_reloaded__duoDyN-PDqg_15120_15215.png FOSS4G_-_Cold_war_reconnaissance_imagery_reloaded__duoDyN-PDqg_15216_16883.png FOSS4G_-_Cold_war_reconnaissance_imagery_reloaded__duoDyN-PDqg_16884_17495.png FOSS4G_-_Cold_war_reconnaissance_imagery_reloaded__duoDyN-PDqg_17496_18131.png FOSS4G_-_Cold_war_reconnaissance_imagery_reloaded__duoDyN-PDqg_18132_19235.png FOSS4G_-_Cold_war_reconnaissance_imagery_reloaded__duoDyN-PDqg_19236_20591.png FOSS4G_-_Cold_war_reconnaissance_imagery_reloaded__duoDyN-PDqg_20592_22955.png FOSS4G_-_Cold_war_reconnaissance_imagery_reloaded__duoDyN-PDqg_22956_23615.png FOSS4G_-_Cold_war_reconnaissance_imagery_reloaded__duoDyN-PDqg_23616_23783.png FOSS4G_-_Cold_war_reconnaissance_imagery_reloaded__duoDyN-PDqg_23784_23891.png FOSS4G_-_Cold_war_reconnaissance_imagery_reloaded__duoDyN-PDqg_23892_23987.png FOSS4G_-_Cold_war_reconnaissance_imagery_reloaded__duoDyN-PDqg_23988_24047.png FOSS4G_-_Cold_war_reconnaissance_imagery_reloaded__duoDyN-PDqg_24048_24119.png FOSS4G_-_Cold_war_reconnaissance_imagery_reloaded__duoDyN-PDqg_24120_24311.png FOSS4G_-_Cold_war_reconnaissance_imagery_reloaded__duoDyN-PDqg_24312_36395.png FOSS4G_-_Cold_war_reconnaissance_imagery_reloaded__duoDyN-PDqg_36396_36516.png
He's a developer and data analyst and memorialist and has specialized in ROTAR remote sensing. He will be delivering the Cold War in reconnaissance imagery loaded up to rectifying the 60s in high resolution. So I will play the recorded session for you. I will be talking about Cold War reconnaissance imagery reloaded, auto rectifying the 1960s in high resolution. Specifically, this talk will be about the Corona Satellite program by the United States in the 1960s. This is the outline of my talk. First I would like briefly to talk about what Mondialis does before going into detail about the Corona Satellite program and its applications. Then I will show the imaging geometry and distortions that are inherent in this data and will showcase our image rectification workflow that we developed to cope with these distortions and to auto rectify entire Corona scenes. A few brief words on Mondialis. We are a remote sensing company based in Bonn, Germany and we are focusing on the analysis of large amounts of Earth observation data in time series using cloud environments. Mostly we use free data such as Copernicus Sentinel data and we are also committed to using free and open source software exclusively. With Marcus Metz and Markus Niedler we also have two GRAS GIS core developers in our team and in our projects we regularly contribute to the GRAS GIS development or to the development of GRAS GIS add-ons as in this project. Now I am aware that you have heard enough about Corona in the past one and a half years but let us give this name a different meaning as it is also the name of a satellite program that was launched by the United States in the 1960s. The goal was to have a reconnaissance mission giving an idea of what is going on on the ground in high resolution especially since it was the Cold War of the Soviet Union and their allies. The entire program consisted of 144 satellites in eight keyhole missions. Now this is of course a lot of satellites but the mission duration of a single satellite was rather low because the satellite operated with two panoramic cameras that were operated with a physical film and once the film was used up there was no use for the satellite mission any further. Also the retrieval of the physical film was rather spectacular as you see in this image. Once the film was used up the satellite would tilt towards Earth and eject a capsule with the film inside into the Earth atmosphere and then an aircraft would attempt to catch this capsule mid-flight. The capsule was also designed to survive in salt water for two days such that it could also be retrieved by boat but after two days it would dissolve and sink in order that the data would not fall into the wrong hands. Now today's value in this data lies in the spatial resolution as you can see in this image because the satellite flew rather low at around 160 kilometers orbit. The latest corona missions yield effective spatial resolutions of 2.75 or 1.8 meters. In this image you can already see individual houses, cars, vehicles and streets so it's a very valuable data source dating back 60 years. This gives you another idea of why this data is so valuable. This is the city of Tunis in Tunisia 60 years ago versus how it looks today. And as you can see a lot of things change on the Earth's surface in the meantime so with corona we have a unique way of looking back in time that no other data source provides us. So luckily all this data was declassified in 1995 and is now available to download from the USGS Earth Explorer. One scene costs $30 but scenes that have been ordered previously are usually available for free simply because the only cost is associated with scanning the physical film. So any data that has been processed before can be downloaded for free. Now what can you do with this data? As I said before there's a lot of valuable information in there simply because it dates so much back in time even further than the oldest lens that archives. So there's a lot of valuable information concerning for example long term land cover mapping and monitoring. You could monitor vegetation or forest changes throughout the time or the urban sprawl throughout 60 years. In this image here you see an harbour village in Algeria and compared to today you can also see how this city sprawled enormously in the past 60 years. This information however is hard to automatically extract simply because in the corona data we have no spectral information. And so all automatic approaches are limited to feature identification and extraction methods. However in methods where we can focus on the qualitative interpretation of data corona data benefits a lot. And one very classic example for this is archaeology to give you an example. This is a corona scene of southern Turkey from the 60s and this scene was used to identify potential archaeological sites simply because human historical structures such as dwellings or infrastructure altered the local topography that could be visualized with high resolution data like this one in order to identify potential sites that are interesting for ecology. In the meantime since the 60s a lot of activity was going on on the ground a lot of construction land cover change so even using today's high resolution data would not yield to this information because the land cover changed so much and buildings and cities were constructed in the meantime. Here's another zoom in of an example we see an individual farm and around it you can clearly see that there seems to be some kind of human-made structure which might have been fortification. So again a very valuable data source for archaeology. So why is corona data not that widely used yet? This is because there are massive distortions in the imagery. On the left you see a diagram showing the imaging geometry of the satellites. It has two panoramic cameras one tilted slightly to the front and one tilted slightly to the back. And since it is a panoramic camera there's a lot of distortion especially at the edges of one scene. So one scene is approximately between 200 and 250 kilometers wide and 10 to 20 kilometers narrow only. So if you're interested in maybe just a very small area of the Earth's surface that is located in the middle of such a scene you might work well with classic linear auto-rectification methods that assume essential perspective. However, if you were to auto-rectify an entire scene there is an urgent need to cope with these distortions due to the panoramic effects here. Further, there is no calibration data available that you would use for today's systems such as the position and different angles of the air or spacecraft at the time of the acquisition. This was the 1960s and such data could not be collected so any transformation model needs to estimate these parameters. Here's an example of different auto-rectification methods of the same corona scene. On top you see just a linear classic auto-rectification approach applied and on the bottom you see an image where the panoramic distortion has been accounted for. And in the top image you can clearly see that the offset especially at the edges is in the order of magnitude of tens of kilometers. So this is not an appropriate way to go to auto-rectify an entire scene. So what is necessary is a transformation model that deals with this and this is done basically by a cylindrical shape that represents the physical film surface to be put in this transformation model and such a model was already introduced in literature by Zohn et al. in 2004. But however in this project we didn't find any implementation in free and open source software. Because of this we implemented this model in the iAuto photoshoot in GrassGIS. So the iAuto photoshoot is the standard rectification workflow in order to auto-rectify aerial or space-borne images normally assuming just the central perspective cameras but now we extended it to also be able to cope with this panoramic distortions. It is part of GrassGIS from version 7.9 onwards so if you install GrassGIS locally on your computer you already have all the software you need to auto-rectify the corona scenes and of course it's free and open source. Finally I would like to show you the entire workflow of rectifying an entire corona scene. It begins with the scene recomposition because if you order a scene from the USGS, my name is Guido Rimbauer from Mundiales and I will be talking about the city of Rimbauer. The example for this is archaeology and one tells it slightly to the different angles of the model. So what is necessary is a transformation model that deals with this and this is done basically by a cylindrical shape that represents the physical film surface to be put in this transformation model and such a model was already introduced in literature by Zohn et al. in 2004 but however in this project we didn't find any implementation in free and open source software. Because of this we implemented this model in the iAuto photoshoot in GrassGIS. So the iAuto photoshoot is the standard rectification workflow in order to auto-rectify aerial or space-borne images normally assuming just the central perspective cameras but now we extended it to also be able to cope with this panoramic distortions. It is part of GrassGIS from version 7.9 onwards so if you install GrassGIS locally on your computer you already have all the software you need to auto-rectify the corona scenes and of course it's free and open source. Finally I would like to show you the entire workflow of rectifying an entire corona scene. It begins with the scene recomposition because if you order a scene from the USGS Earth Explorer what you get is not one entire TIFF file but you get four individual TIFF files that are split but have some overlap. This is simply because the film cannot be scanned in one go but it's scanned in four individual rounds and on each scene pair is a significant overlap in order to recompose the scene. So you could use automatic stitching methods or just go with a simulated auto-rectification approach by finding points that are the same in the respective image pairs. Next comes the part that is most time consuming and this is the collection of ground control points. This means in order to auto-rectify your scene you would need a set of points in your corona imagery that you can assign a specific and precise location to. For this obviously one needs reference data. If there's infrastructure for example which is always the best choice then you could use open street map data as in this example. You can use for example road crossings or bridges. In this case we used a crossing at the airport of Tunis of the runway and this crossing can also clearly be identified in the same or in the corresponding corona scene here. However this is not always possible and sometimes there is no infrastructure in your scene and then you would need to go for natural features such as for example river confluences as in this example or characteristic rock formations for example. But again you would need to really have a close look to find this specific point in both data sources here in the respective corona scene. We found that collecting 20 to 40 ground control points over an entire scene is enough to give a good auto rectification but it would be necessary to distribute these points as evenly as possible over the image in order to have a good result of the auto rectification in each part of the image. With this data now we can do the actual auto rectification and for this we need some additional inputs. So first we need the patch draw scene from step one. Then we need some basic scene metadata. When you order a scene from the USGS Earth Explorer you get some very basic coordinates of the scene which are not very precise but they give you a very rough idea of where you are on the Earth. And this is important for the parameter estimation of the transformation model in order to have some very basic initial guesses. Then you need a digital elevation model in order to cope with distortions due to topography and you need the set of ground control points. With all these inputs you can then estimate the parameters of the transformation model and by doing this each of the ground control points gets assigned a root mean square error that depicts how well this point fits to the overall model. With this you can identify points that are perhaps that have a high error and do not fit well to the model maybe because in the ground control point collection step some points were misplaced. So you can use this step to iteratively adapt your set of ground control points until you are satisfied with the overall root mean square error. And this what is achievable what we found is more or less 5 to 10 meters root mean square error in the models is realistic. Then you can run the actual auto rectification and here the final scene. This is then how it looks like if you do it for an entire strip of scenes this is the Nile Valley in Sudan which is also very interesting from an archaeological point of view. And using multiple scenes you can also use the overlap of individual scenes to identify ground control points that are visible in each scene pair in order to avoid double work. This brings me to the end of my talk. I thank you very much for your attention. I hope I made you a bit curious on the corona mission and the data and also made you curious about trying grass.js for its rectification. And now I'm looking forward to your questions. Okay. First apologies for the video cut just my bad. And also apologies for the issues listening to the audio because I didn't have maybe because I was using the earbuds but I hope you get a grasp of the talk. So okay we have one question. Is the output being used professionally? Was this a speculative project or did someone commission it? If you can talk about that. Yes, thank you very much for the question. Also thank you for hosting the presentation. Yes, so we had a customer that was interested in the corona scenes for archaeological purposes although I can't point out the exact one but it is being used professionally and also in the let's say in the archaeological world. This corona data set is known very well. So it's a far used data set. Yes, I have a couple of questions. Or well maybe the first one is the comment. Apart from archaeology have you identified other potential use cases like I can imagine for example coastal evolution or line coast changes for example? Yes, of course all kinds of land cover mapping basically is applicable for this and it's very interesting because this data goes back a long time. For example there was also a study not conducted by us but by others using this data that also looked at forest change in the Amazon basin. Now you can even look at data way back from the 60s which is a nice extension tool let's say the Landsat archive for example. So there's far more use cases than archaeology but as I also said in the talk it's a bit hard to automate this simply because we only have so we have the distortions and second we have only one spectral channel if you will which is not even a spectral channel but a physical film. You haven't mentioned exactly or shown any coverage map of the imagery. Is this imagery worldwide or because you've shown data from Africa mostly but actually you mentioned now South America and I understand also Russia and the Cold War areas of interest but there are also data from Europe, North America. Yes there's a lot of data from North America so in general it is worldwide and was available so the focus for historical reasons obviously was on Asia and the Soviet Union and also the Middle East which is also very interesting for archaeological purposes but there are data available for North America, Europe, literally anywhere in the world. At some places there might only be one specific scene at others there are hundreds it really differs a lot. Okay I have a couple questions from the audience, did you try AI, super resolution method? Super resolution to identify specific features so we know because we didn't analyze the data itself yet but this we only prepared it for the rectification but this would be a very interesting approach to identify features and objects in the data itself but again this is then the application world that we didn't touch in this project yet. Another question is have you looked at AMC's stereo pipelines method for working with KH and how it compares to your methods? No we have looked at, I'm not sure if that's the one, there is one from I think the University of Oklahoma I'm not sure if that's the one that this question refers to, there is a stereo pipeline and we also tried to use it but we weren't able to contact or we didn't get any feedback from the hosts of this platform because there is a service that allows you to auto rectify Corona scenes online also but we weren't able to get in contact with the host of the system unfortunately. There was a question that gets removed from the chat or no? Well I have one, are the results of your studies there rectified images available somewhere or it's just owned by the customer? Exactly and perhaps I'm not sure about this but the customer may publish them as well for example in a WMS or somewhere but in general what I would like to point out again is that also the data you can get from the USDS Earth Explorer so the raw data is available and there is also free data already there. Okay there are folks already sharing the coverage in the chat for anyone interested in. Okay yeah thank you Matt, is there any community effort to georectify this data so to outsource I guess the time consuming human dependent part of the process? I think there was, there is this Corona Cast project that is also the one with the online possibility to autorectify. But I don't know if this project is still ongoing so from our side we didn't put any effort in the community building here but in theory of course it would be very nice because this is a very rich data source and everybody is just waiting to have it autorectified to get started and the autorectification is really what hinders the analysis ready data. Any initiative that would like to use this workflow is very welcome of course and we'd be happy to help them. I can imagine a kind of a web service to help creating the control, the ground control points. Okay any, how good is the special accuracy? Yeah this really depends so in the best scenes we would get an offset of around 5 meters for the entire scene. There will be regions where there is absolutely no offset and you have a perfect match and there will be regions where you have 20, 30 meters offset. However there are some scenes that can't be really autorectified at all or where we still have hundreds of meters of offset in there simply because there are still so many factors like the absolute position and movement of the spacecraft that was simply not documented in that time. So there might be occasionally a scene where these factors are really hard to come by with also with our autorectification methods. Yeah I can imagine it has to be challenging to work with data from the 70s or the 60s. Any other questions from the audience? Okay then thank you very much Guido. Yeah very nice, a very interesting project. Now we have five minutes for Martín so see you soon. Thank you.
reconnaissance imagery reloaded, auto rectifying by the United States in the 1960s. This is the with these distortions and to auto rectify entire are also committed to using free and open source contribute to the GRAS GIS development or to the meaning as it is also the name of a satellite the 1960s. The goal was to have a reconnaissance of course a lot of satellites but the mission cameras that were operated with a physical film for the satellite mission any further. Also the water for two days such that it could also be can see a lot of things change on the Earth's was declassified in 1995 and is now available to there's a lot of valuable information concerning sprawl throughout 60 years. In this image here you an example. This is a corona scene of used to identify potential archaeological sites resolution data like this one in order to the satellites. It has two panoramic cameras one camera there's a lot of distortion especially at Earth's surface that is located in the middle of linear auto-rectification methods that assume today's systems such as the position and especially at the edges is in the order of because if you order a scene from the USGS, my example for this is archaeology and one tells it because the film cannot be scanned in one go but then estimate the parameters of the mission and the data and also made you curious was interested in the corona scenes for archaeological world. This corona data set is the first one is the comment. Apart from archaeology have you identified other potential far more use cases than archaeology but as I also