Drones in research webinar #2: Rise of drones in Australian research space - 27 Jul 2017
Guru, Siddeswara
Bryceson, Kim
Brown, Tim
This webinar is the second in a series of 2 webinars which aims to bring together researchers and data specialists from various disciplines to showcase their drone applications and data expertise, knowledge share and information exchange to enhance the research capabilities. Who would benefit from attending: -Data managers and Data Librarians -Data scientists, analysts, developers, and technologists -Researchers (Academia, Industry, and Government) -Environmental & geoscientific research data community Who is speaking: 1- Making drone data open for scientific research: Siddeswara Guru (TERN) 2- Smart Agriculture at UQ Gatton IoT and Drones: Kim Bryceson (UQ) 3- From pixels to point clouds - Using drones, game engines and virtual reality to model and map the National Arboretum in Canberra: Tim Brown
 Welcome to today's webinar on the rise of drones in Australian research space. My name is Darcy Magala-Party from the Australian National Data Service and I'm your host for today. My colleague Karen Wisser is behind the scenes co-hosting the webinar with me. This one-hour webinar is the second one in this webinar series. Today's webinar will showcase the success stories of drone applications in Australian research space, for example in smart farming, plant phenomena and data sciences. We will also provide an overview of the recent developments about the research infrastructure and see how we can leverage the benefits of the research infrastructure as well as open drone data. I would like to introduce our speakers for today. Dr. Siddeshwar Guru is the director, turn data science from the University of Queensland. He is a Kim Bryson, associate dean science faculty, University of Queensland. Dr. Tim Brown, director Australian plant phenomena facility, Australian National University. Now I would like to hand over to Dr. Guru for the first presentation. Thank you Nassima for the introduction and giving the opportunity to talk about this topic. So as Nassima said, I am Siddeshwaran Guru. I work for time. So today I am talking about the making drone data open for scientific research. So this is basically the work in progress, what we are exploring and what are the thought process behind that. So just a quick introduction when we say drone. So what is drone? It is nothing but it is aerial vehicle. It may be controlled by the on-ground remote control or onboard computer. So drone has got different names, commonly used as UAVs or the RPAS. I liked RPAS partly because it is a gender neutral name rather than unmanned. And so predominantly if it is rotary wings, it is predominantly called as a drone. And then even the big fixed wings aerial vehicles are also in this category. Some are pretty big which is as big as the aircraft and then which over the sky and then gather data. And the rotary wings, the smaller one, it runs in a control environment where you find scale resolution and then runs predominantly in the battery and then get the data. So one of the main things about the drone is it becomes quite cheap to run it. So that is why it has become quite popular in various applications. So some of the applications is in a drone, a technology was started in the surveillance and then currently it is a fair bit of the applications in the agriculture and widely in the environmental space as well. I would call it as a scientific research. And then it is not used in the disaster recovery especially in its very difficult to send a human being. So the drones are quite popularly used. So in turn basically we use drone in the remote sensing capability aspects of the thing. In this context we use drone basically to derive fractional cover to measure vegetation species composition, to map vegetation characteristics and then to do some stock take, the pastoral stock take and then even to do the survey, flora fona survey, kangaroo survey, etc. So a lot of data will be derived from the measurement that is done in the drone. So if you look at, you know, there is a fair bit of the vegetation is on partly because you know, if you take a satellite remote sensing data, the vegetation composition, I think most of the two-third of the time, you know, it has got the cloud cover over the earth. So a lot of data is measured. So drone is a really good technology which runs in the underneath the cloud, below the cloud and take a good final resolution picture and then do the, and the researchers can do the analysis. So in this context, you know, the common sensors used in drone, so basically drone is a platform and then the sensors are attached to the drone. So in our case, you know, is a fairly popular things used are the multispectral, hyperspectral, LiDAR, video and the thermal infrared sensors. So from a data management perspective, you know, it's basically the, it's a fair bit of a challenge partly because it's dynamic both space and by space and time, it's keep on moving at the both the spatial and temporal scale. And then the, because of its ability to capture finer scale information, you know, it's a massive amount of data is collected in that one. And the, we should have a capability to basically to ingest that and then the process that data and make that data available to the users. And the other part is, you know, there's a fair bit of the commercial companies who work in the drone industry and even at the research aspects of the thing, you know, there's always a partnership with the commercial entity where they will run the drone and then, you know, collect the data and give it to the researchers, especially there are consultants, especially in the mining community in the mining area, et cetera. And the other thing you should be aware that, you know, you need to, you need a permit to do this from a CASA to operate this. And then you should be an operator. You should have a license to operate this as well. And with all this, you know, the identifying a data owner is important. I will come to that later why it is important. And if you look at the, so for example, to make this data, you know, openly accessible, you know, just, you know, take a bit of a fair principle. So these are the four aspects of the principle, you know, we just see how the drone data will fit in side of the thing. So if we take the first one, you know, data is adequately described, searchable and should have an identifier. So just to give a bit of a perspective on the drone, you know, the data is basically the flight plans and then the, you know, the files of the, you know, flight paths and the associated field data, you know, generally, especially, you know, in our case. And then the raw data files from the measurement it tooks and then the files and then the log from the flight. And once it's the process, then the, all the derived products as well. And we should also provide the auxiliary files like a QA QC files from the processing. So these are the different files, you know, that is, that is a part of a data publication aspect. So if you look at, you know, all these are the, you know, it's a related, interrelated thing. So all this data should be made available. And then the, especially for the provenance aspects of the thing, these, all these data should be made discoverable and then that should be accessible from a user perspective. And then to make this, you know, data, you know, searchable and then the, you know, in file. So basically, you know, there should be a metadata standard to describe this data. So we, so, you know, we use ISO standards. We're not sure whether the, you know, so for example, ISO 19115 completely describe everything or maybe we may have to provide a customized profile of that ISO standard. And then if you make it available as part of a catalog then, you know, it's a discoverable and then once we put it in a catalog, we have an identifier and then that's, that looks fine. And then the next aspect is the, you know, the, one of the principles is the data retrieval using open protocol. So if you look at the, the instruments or the sensors that are used in Trone, you know, it provides by default, provides a, as a raw data, you know, from a different file format. And then even at the publication level, if you look at the, you know, the open standard, you know, in the open data policy, they say that, you know, all the file format should be of the open format. So the file format, you know, generally in the, you know, depending on the, which of the instruments, you know, it may be a give, geotive, KML, shape file, or LAS file, especially in a, in a point cloud aspect, aspect of the thing. So with, with such a kind of a veracity of the, you know, file format, then the tools must be available as well to translate or manipulate even the analyze the data. And then the, you know, even at the, you know, sometimes it may be worthwhile even to provide a program, you know, the R program, Python program where, you know, they should, they can access this data and then, you know, so that they can, they can run the analysis aspects of the thing. If it's too confusing, so for example, if somebody don't know, don't have a clue about, you know, how to access the LAS file, then probably it's worthwhile to provide a program so that, you know, it's embedded, so that, you know, that is embedded in the, embedded in the R program so that they can start writing the program. So that is one of the things, sort of thing. And the next one probably is the, the, the data use vocabulary and qualified reference to other metadata. And then, so we use a, you know, a fair bit of a domain specific vocabulary is like a GCMD is quite popular, especially keyword search first. And then, you know, because the, because the drone and then you have a technology as, as it's called so fast, probably there may be, you know, it may be in when some of the terminology, you know, incorporated into the vocabulary so that it's, it's accurately represented. With, as I was explaining the, the different data types and, you know, when each of the data type, each of the, you know, files should be referenced and then it should be made as a link and then hopefully as a persistent identifier and then all those should be a query as well. And then the, the final, you know, principle is basically data, metadata in domain relevant community standards with clear data usage license. So we use an ISO standard, you know, so that 115 or 139 in the domain, which may fit well as I said before that, you know, maybe we may have to create a profile to accurately represent the, describe the data. And a lot of the standard is fine. If you want to fit the human actionable, you know, discovery, query and access, that means that the human go and click couple of things. Probably we may have to look at the, you know, machine actionable, machine actions, accessible, actionable, discovery and exploration sort of the thing where, you know, it's a machine to machine query, then that may be a issue partly because of the, so much of the interrelationship with the files and et cetera. It's also depending on the, you know, the, the what kind of file. If we are looking at the source file, then definitely that will be a issue. And then the, talk about the data usage, you know, so what it says is, you know, oh, you should provide as an open license thing. Even though, you know, we may, we provide the all of our data and the creative comments by attribution, the identifying the owner is a key partly because, you know, even at the creative comment attribution, the copyright subsist with the data and then you need to identify who owns the copyright. As a separate example, if you are, if you are a, if you are a researcher who is using a consultant to collect the data. So technically, if you look at that aspect, the person who collected data is the owner of that data, unless you have a contractual obligation arrangement, make sure that the ownership is transferable to the researcher. So why this is important is that, you know, for the attribution aspect set of the thing. So that if somebody uses your data, they need to know who is attributed as a researcher, you want the attribution to go to you, not the person who just collected the data. And the thing is, if somebody is owner, they can do whatever they want with the data. The IP is with the owner. So what you want is the, as a principal investigator, you owns the IP about the data. Still, there are a fair bit of a challenge. One of the things, you know, VFACE is the, you know, it's just the amount of data set that is collected. We still struggle to, you know, make that data, ingest the data into one place, and then they process everything, and then the delivery of that data. So we think that the cloud platform, you know, the managed platform to do everything would be a useful thing. And there are a few initiatives going on as well at the community level who are looking exactly at the similar problems. So one is the OGC domain working group, and the other one is the RDA interest group. So with all the aspect, you know, we want, we don't want to change a complete, you know, data management practice of the turn. So we retrofit everything to the turn data management practice. So a lot of the, whatever the data, you know, we intend to, you know, make the metadata and ISO standards, and then cataloging the geo network, and then that is harvested to the, you know, different repositories, you know, discovery portals, so that the data is discoverable, and all the data will have a clear, you know, attribution statement with creative accounts attribution. And then currently we are storing, you know, data in the FTP server, and then make that link available. And we are still in the process of, you know, processing a lot of data. But having said that the, all the raw files are there, so if somebody is really interested to use that data, they can process the data as well. And the other thing, you know, we are still working on the, you know, effective, you know, the robust data delivery mechanism as well, so that the user just, you know, come and access the data and get the data. And one of the things, you know, we are working on the, you know, still working progress is the, you know, the portal where, you know, user can come and get the final products. So we collect, you know, we have done a campaign across, you know, love and sites across Australia. So we are working on that, they call this the field data, and then we are working on the portal where, you know, all those love and sites data is accessible as well, especially at the LiDAR level. So if you look at the drone, you know, it is quite popular in the research community, you know, especially in the environmental science. There is a massive uptake of the drone technology, partly because of the ease of use, and then the final resolution of the data it provides. And moving ahead, partly, you know, probably it's worthwhile to build an IoT platform to manage the research drone data. It's not at the institution level, at the overall research level. So this probably enable the interconnection of devices, basically based on the type of sensory use and what is the application you are running, and that will enable to build a common data platform so that each of the individual doesn't, you know, repeat the same grappling with the same problem. And if you look at the commercial word, you know, this is already happening. There are a lot of commercial players working in this space. It may be, you know, off the shelf as well. And then there are even a lot of the open source technology already started appearing, especially the management aspects of the thing. So what we want in this one is a researcher has got a platform, they put the sensors, they collect the data, and we need a platform where that data is ingested somewhere, the processing happen, and then the product, derived product is basically available for the third party researcher to analyze that data. So at the individual level, you know, people are working, but they make everything as a pipeline and then provide that as a service to the complete community. We are still working on that one. It's still a work in progress. Thanks for the opportunity to talk about this topic. Thank you, Guru. It's an excellent presentation, very informative. Now I would like to hand over to Kim for a second presentation. Hi, everyone. My name is Kim Bryson. I thank you very much for inviting me to be participating in this. I'm going to be talking about drones and research in relation to agriculture, in fact, in education as well. And I think my talk quite nicely segues into some of the things that Guru has been talking about, because the buzz of today's agriculture is around these disruptive technologies, Internet of Things, big data, drone technology, smart agriculture. They're all buzz words and commonly tossed around, and there's been a number of big conferences in the agricultural space and farm management space about how such technologies can be of value to the agriculture and to the whole food science issue of, you know, producing good quality food in a sustainable way. What we've done at UQ is look at how we can incorporate things like an Internet of Things multi-sensor mesh network to collect real-time biophysical data, which we store in the university's cloud. And we've done that around the whole thousand-odd hectares of the University of Gatton's regional campus, which is a multi-enterprise agricultural center. And we've also incorporated a second multi-sensor mesh network around 10 kilometers away from the campus that will look particularly in relation to – we've set up a living laboratory there is what we called around how we can use this real-time biophysical data with drone technology to look at biomass of pasture in particular, but that can end up going, obviously, to various crops and vegetation as well. So this network that we've set up, it's very flexible. We work with technology from Libelium. We can look at different types of communication protocols. So we're not just looking at Wi-Fi, but we're looking at other types of radio interfaces that allow us to transmit data in real-time over long distances and through buildings and trees, et cetera. And it's a network that is basically self-healing. And we have a number of these nodes set up across the campus. Here we go, smart agriculture, smart water, smart environment, and smart security. This last one being – the smart security one being essentially the ability through the network to turn things on and off or open and shut gates, which, of course, is really important from an agricultural perspective. The agricultural one looks at – well, in fact, I think my next slide looks at the types of data that you can gather from these sensors when it's installed. And we have all of these nodes and most of these sensors around the place. So we're collecting a lot of biophysical raw data in real-time, which we store in the cloud and have a dashboard to essentially access. The idea is that this is done through open source so that at the end of the day, people can have access to this. So turn may be interested in it. Other people may be interested in using it, remembering it's subtropical information, particularly for people who are interested in the agricultural space. But we're looking to certainly make that available at the moment through Edurome to people who are involved in Edurome rather than anything else. We've tried it on an open network and that we found that it had a risk of the university being hacked through our system. So we closed it down from worldwide access to Edurome access. So lots of data, biophysical data coming in. And here's a diagram that tells you that we've got data coming in from these various nodes into our cloud. And then we're using that data within the educational environment now to develop various, essentially data uses. So we've got research projects in here, but we've got farm operation that is something that we want to do because we'd like this to be developed far more into the agricultural farm space. And of course our education component. The type of data you can get, you can modify all of this through our data dashboard or you can look at a QR code and get through to it. So that's the start. But when we have this large amount of biophysical data, what else do we need and what for? And of course we want the data for things like partial monitoring and management, animal monitoring and management, crop monitoring and management and education and more data. We want to get more data. So drones provide that or satellite data or remote sensing data provide that capability of putting aerial data on top of our, are essentially ground and ground data collection system. And if you go back to the future, or we're going back to the future in this because remote sensing has been around in agriculture since the early 1970s and the problem for ag in relation to remote sensing were two main data. One was data, cost of acquisition and processing, the revisit frequency, the things that Guru has talked about actually and also a lack of skills available in the agricultural sector for this type of data to be used efficiently, well in a cost effective manner and agriculture people or people in agriculture can be very easily turned off technology. What is the point of technology if it doesn't deliver me a cost benefit in terms of what I'm doing and if it costs me a lot and it goes wrong and I can't do it when I want to do it then it's not worth it. So we needed to do something about this lack of skills and what we've done here is look at developing the special skills that are needed to understand spatial variability in agricultural remote sensing. We've set up an agricultural remote sensing lab here at the campus and what we're doing there is we're trying to integrate students from different disciplines across the academic world. So engineering students come down and work on agricultural projects in relation to things like building drones and understanding what drones can be used for. Building sensors and understanding what those sensors can be used for in agriculture monitoring in particular but that does involve of course environmental monitoring so our wastewater management site is probably one of our most recorded biophysical data sites and we get them to do hands on work so they understand the issues and the risks. We want to use them because they're cheap platform to carry high resolution sensors. You can see here a spatial variability in a paddock at ground level you can see a bit but when you look over here at an aerial photo of that paddock the variability is huge and that means dollars to a farmer. So we want to collect that data so we can optimize production efficiency and quality and we then also want to minimize risk and environmental impact. So drones from the perspective of an agricultural person could enable us to do better in the smart food production game almost certainly enables us to do better in the smart environmental management game and for us as educators smart skills development because it's lots of skills involved and it's fun to do and one of the things we have in the Australian sector actually is worldwide. If difficulty in getting students into the agricultural industry sector and I'm meaning across the board not just grains or horticulture but across the board. So at UQ we've got five DJ Phantoms that we use for teaching we've got four bespoke quadcopters we've got three bespoke hexacopters and ten mini agricultural drones which are the ones nowadays because these are the ones that I've talked about the ten mini ag drones are the ones that we use and I'll just talk a little bit about them in the next few minutes they're easy to fly and there is an article out in 2015 about why we're doing this and what we're doing. The design and build principles around this business of getting students of all disciplines to understand what they're doing we go through a design process trial and error using open software to look at designing these things we purchase the parts for them they build them they test fly or they learn to fly learn to solder things like this undertake the project and write a report. So this is a classic problem based learning or active learning scheme that we can make get students to do. Here are the types of drones that we've investigated and used over time you can see there's a certain date range here and as much as anything else this small mini UQ mini ag drone we designed and built because we want to fly under the two kilo limit of CASA because if we had to get every student certified we couldn't do this. So this little mini ag drone which I'll talk about in a bit more detail a little bit down the track is the one that we're now flying and this with a camera that looks at getting a normalised difference vegetation index image is less than one kilo. So the first student project was in 2013 where the student literally took a DJI Phantom with a little normal red, green, blue visual camera back to Fiji. This was his school in 2003. This is a Google map. This is his school in 2013 which is the image that you can see outlined here and he literally calculated the difference in mango clearing basically so he looked at change in mango. So this was the first project that we looked at then we started getting a little bit more creative. We developed a drone that could carry a multi-spectral camera and we started looking at Prickly Acacia which is a serious environmental weed up in central Queensland. Big problem from an agricultural perspective because it shuts out cattle from using the pasture underneath and we compared some of that data to satellite data and found of course that these drones creating a better resolution on the ground gave us more data. This beneficial bug drone, a hexacopter weighing around about 2KG and more when we've got this white thing underneath it designed by an agricultural science student which is now going to the has been invited to be put on display at the Science and Technology Museum in London for the next five years in their display on innovations in agriculture over time. So this is actually an old innovation now. This is about 2013, 2014 but it is an instance of when people started looking at using drones. So this is a box full of beneficial bugs which the farming industry, the farm group that we worked with wanted to see if they could develop, drop beneficial bugs, these bits coming out into the middle of a paddock instead of as they customarily do, drive around the outside and so this was a very successful project with industry involvement that then started really introducing this sort of technology from the education, from the classroom into the actual agricultural space. So that was great. The net drone was another one where we were asked to fly under hail netting for one of the largest seedling producers in Australia and here is the netting. You can see it's a lovely picture if you appreciate this sort of color variation. The drone is flying here and it was called the net drone project. We were looking to see which one of these seedlings in 250,000 small tube plants were germinating or not. And here you could see this sort of resolution that you could pick up from a standard multi-spectral camera. You could see the two-coddoledon stage here and you could introduce students to the idea that you are looking at three different bands, so a little bit of physics, you're looking a little bit of spatial variation because you can see where in this one where my cursor is where things haven't grown and then when you go up here at two-coddoledon stage you get that interest for a student as much as they can actually see a plant germinating and they can see where it has not. And what you do is you get them to calculate what the cost is to both the supplier and the buyer when they get a tray of seedlings and something hasn't germinated. So that links it to the economics of the producer. Here is our mini-ag drone. We've been flying it now for a couple of years. We've used it at the beginning of this year for 35 students to do some projects. It's based on Raspberry Pi technology. We've upgraded it because we found that the GPS wasn't crash-hot but in this sort of thing I've got 10 drones which might cost me 100 bucks each to repair but that means that students can crash them and that's good for a student to be able to do that and over here you can see them flying on campus. Sort of data that you can get out of this quite basic no IR or an NDVI data camera is you can pick up this information quite nicely at an operational scale. So again I'm guess I'm not at that. I'm not trying to get these students to be as knowledgeable as we might think from a research perspective but an undergraduate level 3 agricultural science student they go out with this knowledge and will then be able to develop their skills in the real world after that. We are developing various sensors that can fit onto this small camera so the most recent one is a multi-spectral sensor. And multiplexed Raspberry Pi cameras that enable us to choose very high grade research filters of specific wavelengths when we're wanting to look at something and this particular camera at the moment is being used on top of one of our IOT multi-sensor mesh network to collect aerial data of the biomass of that paddock across which of some 50 hectares I believe that has a multi-sensor mesh network on it. So this camera is now working so this was when it was being developed. You can see how it's been set up for little Raspberry Pi cameras and we use Python to make sure everything goes well. This is a drone that is no longer flying but it was an attempt at us delivering a drone that could pick up the RFID tags in a cow's ear from a distance and it works at about 20 meters but you know we're talking a big drone now with a big antenna. And whilst it worked from an operational perspective and from the idea of trying to keep things down to a level where we're not flying above CASA's regulations this became essentially unusable and although I have an engineering student who wishes to try and reduce this in size this is something that we're not at the moment moving forward to but it is an idea it's an example of some of the ideas that come when you start talking to the industry players and asking them what they want. They wanted something like this because normally you have to stand beside the cow or the cow has to go through some cattle gates and literally swipe it with a hand swipe. You've got a thousand head of cattle that's a really difficult, really time consuming thing to do so this was about trying to improve the efficiency of being able to look at these cows in the paddock and the sort of data we could get from it was really quite good. The sorts of types of research and learning involved is electronics and avionics which is not normal in an agricultural sense but it's really useful if you're going to get into this side of spatial variability analysis, the design and build side of things. We look at programming, physics, maths and chemistry of course because we're looking at the electromagnetic spectrum, things like spectral indices and crop growth indices and chemicals and plants so if you're trying to engage a student in learning about maths which they really don't want to do you bring in something like this and it changes their perception of both learning about the maths and then how it can be used in real life when it would appear to be from a classroom perspective completely dry and equation driven. Plant physiology, animal welfare and food traceability are all issues around management that we can also talk about when we use these tools. Most recently we've had these sorts of projects going on, drone and sensor development for crop monitoring and diseases, computer vision development for monitoring pests basically but we could be doing it in the piggery or we could be doing it out in the environment for say wildlife people. We've got a master of engineering student looking at developing a robotic arm for capturing these small drones out of the sky for autonomous recharging. Again, it's an academic project probably not to be used operationally because of the issue of autonomous flight but very useful for the master engineering student to understand why we want that from an agricultural perspective so we want to increase obviously the reach of our drones flying and then we've had people developing automated underwater cameras to monitor phytoplankton which we can then link to our IoT and our aerial imagery of the laser flakes. Obviously the IoT has applicability elsewhere that may not involve drones like in the equine health area and the development of this visualization dashboard for our monitoring purposes and for staff to access and as I say anyone on EDUROAM to access some of this data that we're collecting and as I say this is a project the RFID monitoring of cattle is something that we're still sort of looking at but the idea is to reduce it in size as the electronics become smaller and we don't have to have such a big antenna involved which means a bigger drone and more weight etc and start starting to contribute in cattle regulations. So we're looking at a whole range of things both from an education and research perspective but mainly the research is about getting is about interesting students in learning about this stuff which I think is going to be key down the track in terms of capability development for our industry both agriculture and environment or managed landscapes because without these kids coming in we're going to run out of people in terms of as they get older they're not going to be able to continue to do this. So that's my presentation and I hope you enjoyed it. Thank you very much. Hi Kim, thank you. And I will move on to the next presenter Tim Brown. Thank you for having me on that was a very interesting talk Kim. I'm going to be talking about our work at the National Arboretum in Canberra using drones to make 3D models of the arboretum growing. I'm the director of the Australian Plant Phenomics Facility at ANU and a lot of this work was done when I was working for the ARC Center of Plant Energy Biology as a research fellow and I should first off say thanks to all the people listed there that have been essential to this project. I wasn't quite sure what level of understanding people would have on this. Drones are part of a as Kim pointed out they're part of a key emerging tool set for next gen monitoring of the environment and these make up sensors, high resolution imaging, LiDAR and a whole bunch of other things that let us measure the environment in ways we never could before. And since I've been working in the Phenomics field really we're interested in how you monitor the environment along with genetics so you can measure the phenotype because those are the things together that define how ecosystems develop and in a crop context the phenotype is essentially what we want for increasing food yields. So a bit of background on UAVs I think a bit of this has been covered so I'll go through it quite quickly but you basically have a lot of choices if you're starting a drone program from the sort of low cost ex rotors like these quad and ex rotors like the DJI Phantom. These things are changing so fast so I used to give this talk and say that drones were expensive and confusing and then I said that they were cheap but still a bit confusing and now they're just cheap and easy to use. The DJI Mavic Pro came out this year it's a very tiny drone but it still has a 12 megapixel camera on it you can throw it in a backpack and you can still do pretty decent monitoring with it at the low end of the things. These sorts of drones typically cost in the one to two thousand dollar range although the bigger ones like this one here cost more in the eight thousand dollar range and you get something like 15 to 20 minutes flight time and up to 10 kilogram payload for the really big ones. You can also use fixed winged drones so these give you a longer flight time something to 30 to 60 minute range but you do have to have takeoff and landing issues and they can have higher payloads but they do tip to fly faster which impacts how you can monitor with them. A new system that's being developed by people that we work with at Pro UAV is this vertical takeoff and landing system and we really think this is going to be an amazing tool because it gives you both the benefits of a fixed wing and some sort of a copter because with the helicopter part of it it can take off straight so it doesn't need a landing zone or runway and if something goes wrong and the gas motors fail it will catch itself and land so you can carry quite a heavy payload and you can put really expensive equipment on it because it's quite robust. On the camera side as Kim pointed out you have a lot of options ranging from RGB cameras the on-born cameras on the Phantom work quite well all the way up to DSLRs but you do need to watch about what cameras are using and isolate them from vibration or you get rolling shutter issues where the camera is trying to take a photo but the sensor is not writing all the data at once and so when there's vibration you end up with bad data. The next step up unless you're building your one yourself like Kim pointed out is the multi-spectral cameras that can get you NDVI and the main thing that people use now are either the Micasense Sequoia or RedEdge and those are in the $3,000 to $5,000 range. There are a lot of other sensors so you can put hyperspectral cameras on your drones or thermal and you can see you can get bands in the 400 to 1300 nanometer range but you need a big quadcopter because they're heavy, they're very expensive and the data is quite hard to process so there really is a range of stuff available from really easy to use up to quite challenging depending on what output you need. The typical outputs you get from 3D reconstruction software which is a lot of what I'll be talking about are ortho mosaic images so these are essentially satellite layers that you can put into Google Maps or Mapbox and then also DMs and GeoTiffs and some of the software can actually give you somewhat classified outputs or remove the ground so you only get trees can provide you with RGB multi-spectral or hyperspectral indices and also 3D point clouds and then you can make a 3D model of your environment as well if that's what you're interested in. There are a lot of options on the 3D reconstruction software front and I'll just go through a few, it's important to know that it requires a pretty beefy PC to run these things so you're looking at probably 1500 AUD to get a PC because you need a fair bit of RAM, you need a pretty good processor and then you need a graphics card. PIX40 Mapper Pro which is the one that we've been using because it was about the only one available when we started working, it ranges from $2000 to about $9000 depending on what license you get and you have to pay about $1000 a year for support and also as a challenge that you can only run on Windows so if you want to run it on a server or on the Amazon cloud for much bigger data processing and for automating stuff you have to get the Pro license. The other one people use a lot is Aggie Soft Photo Scan, I haven't used it personally but it's been well recommended. The cheap version if you just want 3D models is about $60 US and about $550 academic and then pricing goes up from there and those also will run on the cloud if you buy the Pro version. Mosaic Mill is another software package, it's the last quote I got from them was about 4500 euros, it comes in a lot of flavors but I haven't used it. There's a free software package called VisualSFM which is FSM which seems to work pretty well because it's free and somewhat open source software, it doesn't have all the bells and whistles the expensive ones do but if you have more time than you do money for your project it's probably a good thing to explore. There are also a lot of online options and these are great for testing so if you either just do occasional flights and don't want to invest in the software in the PC or you want really fast processing or you're preparing a grant proposal so you just need to get some initial data websites like these ones let you just upload your images and give you a point cloud and other data quite quickly. A disclaimer this is not a complete list and all of these prices change rapidly and frequently so check the vendors for pricing and don't take my word for it, there's plenty of other information out there but you see from these software you can take a flight like you see on the left side here and then you get a 3D reconstruction on the forest and I'll talk more about that in a minute. So the site where we've been flying drones is at the National Arboretum in Canberra and this is a really great site because it's just 5 kilometers from ANU and has fast Wi-Fi. The last site I worked at was in southern Utah and it took about 5 hours to drive each direction so when something broke it was a good day and a half just to get there and back and having a field site where you have Wi-Fi access from your desk and you can drive out there in 5 minutes really makes it easy to test new and emerging technologies until they're ready for deploying elsewhere. And also the forest was only planted about 7 years ago so we have an opportunity to monitor this forest and build a 3 dimensional model of the entire arboretum growing into the future which has basically never been possible before. So we initially at the ANU research site we installed a 20 node wireless mesh sensor network, camber weather stations for baseline data, some gigapixel cameras, these are cameras that take hundreds or thousands of high resolution images that you squish together to get a multi-billion resolution picture. We've done some LiDAR scans both on the ground from various sorts and then we've done near monthly UAV flights. And once you sequence the trees out there because tree sequencing is getting down to you know the 10 to 20 dollar per tree range you can really measure phenotype environment and genetics in a way that gives you an amazing density of data about a space that we were never able to access before. So here's camber on the southeast side of Australia, the arboretum is over on the west side of camber, south of Black Mountain if you've been there. This is our main field area, this is a picture of what it looks like from one of the cameras up on the hill, so this is a 500 megapixel image that we generate every hour and you can zoom all the way into that image to see the forest out on the far side there. This is what that forest looks like when you fly over it with a drone and then from the drone data you can bring that into PIX4D as I'll show in a second and get a three dimensional model of the trees. So the drone monitoring program, the goal was to test and develop a time series drone monitoring program so we could get 3D models of the trees growing. We get time lapse georectified image layers, 3D point cloud and then some phenotypes we can measure like tree height, area is measured by top down pixels and color data over time. And this forest we're studying is, it's 12 forests, a spotted gum and Iron Bark was planted about 2012 and it's 4 hectares so it's really a perfect size for drone monitoring because you can fly the whole site in about 15 minutes it's not too hard to process and it's very amenable to that sort of monitoring. We've really come a long way on this so in April 2013 I flew the first drone over it which was a cell phone duct tape to a drone that I'd made at home and now we're using a really nice solid Matress Pro by DJI with Darrell from Pro UAV who's been flying it for us and really good cameras and so we're really getting a lot of solid data but just in the last 4 years, 5 years this technology has changed so much. The typical workflow in this is the PIX4D is a software that we use, it pulls in all the images you can see here that were taken from the drone and then you can see on the left all those green lines going down are key points that is detected in all the images and there's really a lot of black magic that happens behind the scenes in these software packages and you can see on the right those are some little tiny piece of ground that the software has detected as being the same in about 30 different photos and it uses that to calculate the actual position of each photo relative to the ground. Those are called control points. From that it creates the 3D point cloud as you can see here and that gives you this sort of ghostly 3D model of your entire forest. You can see that there's some data missing from the bottom because the trees are the drones are just looking down from the top and can't see around the edges of the trees. The total processing time to do this is very hardware dependent so it ranges from like 3 to 12 hours but it allows you to go from pictures to point clouds to 3 dimensional models of the trees you can see here. It's important to realize how groundbreaking this technology is because previously if you wanted to measure the location and height of every tree it would have taken you an incredibly long time. We have trees in this forest and we can do this now in about a morning but there are still challenges as Ger was pointing out with working with and serving up the data. We implemented a point cloud viewer called Poetry on our website, traitcapture.org. You can go there and see some of the point clouds that we have online. We wrote a software package called forestutils that runs on Python that lets us pull out the locations and height of every tree and the point cloud data associated with that tree. So this assumes an open canopy. If you have a closed canopy things are a bit harder unless you have GPS locations for your trees but now once we have the tree locations when the canopy closes we can keep tracking them and it outputs tree height top down area location RGB colors and a point count which is a measure of how many points were generated for that tree. It also spits out a CSV file that you can stick on Google maps or anywhere you want that has all of your tree locations. So that is the program we ran. We probably have flown 30 flights over the last since mid 2015. I want to talk a little bit about data management because this is really crucial and if you are planning to do anything more than just occasional flights you really need to come up with a good data management plan and you want to do this before you start your surveys. It is important to consider the entire workflow because it is not just who captures it or what happens to it but the entire process. If you think of who does the surveys if you have more than one company or more than one person the data has to get to you somehow. It has to get processed. You have to figure out where it goes on your computer and track it. Until people like Tern have made us nice tools for having our data go seamlessly online you have to manage all of this stuff yourself until it gets to the point where you publish it. You have to figure out smart ways for naming things and you may get a data set and then add to it and then add to it again and then process it and you need some sort of workflow that tracks how that is taking place and doing that across thousands of images or hundreds of flights can really be a challenge. And if people upload data you need to make sure that they have told you how much data they have so that you can have all of it so you don't spend a week processing their data and then find out that they hadn't finished uploading it and you needed to add another hundred images and run the entire thing again. Often you run into problems like we stole all of our data on a large data server at the research school of biology but we process it on a computer that is local and has an SSD so we have to move the data gets uploaded into one folder by whoever took it gets moved to another folder which is the storage folder and then gets copied to this computer to process and then has to be copied back with the new data as well and so that makes it quite hard to track things over time and if experiments fail or new data comes in you need to have a workflow for how you know where things are and what the status of them is. So it's really best to enforce rigorous note taking even just having people whatever tech is running the project writing down what they are doing as they do it can be really happy and handy. Also shared Google docs and notepad plus plus so you can put just files within each folder can be really useful. So here's an example of the naming structure that we settled on and basically the idea was to make when someone looks in a folder to make it reasonably human usable so we have the year location site who captured it and the status in this case it was the national forest and national arboretum the ANU forest plot actually this was pro UAV that captured that one and we wrote that it was done and uploaded. So this seems like a great idea but of course if you have any sort of nested folders like you might want to name your dataset national arboretum the file names get rapidly too long for windows and your entire process fails. So this makes it a challenge because you need metadata but you have to have someone who actually is maintaining it and as I said shared Google docs are good but this isn't really a solve problem and also you know everyone always ends up with files like this because before we implemented a data management plan things were just going on to my laptop and then getting copied into random places. I think a lot of people get flummoxed by data management because it's really hard but it's not easy for anyone and it's easy to think that you don't know what you're doing but I'm not sure that anybody really does and you won't get things right the first time and you have to start with a plan and keep working at it and just acknowledge that it's not going to work the way you expected as soon as you start to implement it and then you need to go back and change it because in reality our data management typically looks something like that and we want to move it more towards the vision but it isn't actually there but you need to address these problems because they end up making your data unusable when we have these large scale huge time series datasets. Some other challenges with processing drone data are that these new kinds of three dimensional data so for things like NDVI or some of the metrics that Kim was mentioning that there's a lot of known information about that because people have been working with that sort of data for a long time but a lot of data like a three dimensional point cloud of a tree it's hard to know how you tie data values like that to biologically meaningful things and it's also challenging getting back to what Gerr was talking about with the provenance of the data and tracking what's been processed. You can process the same project in three different versions of PIC40 and get three at different outputs and there's also about a hundred different ways you can vary the settings so at one point recently we decided to test every single setting we could think of in PIC40 and you can see an output from that table here and using exactly the same images you can get a stitching time ranging from five minutes to 55 minutes and point cloud sizes ranging from 44 million points down to 11 million points and it's easy to think for example that more points means more data but if you happen to be taking pictures on a windy day and your tree is moving around you probably want your data doesn't have the resolution of a leaf it just has the resolution of the height and structure of a tree so it may be that either the less points is actually a better measurement of that tree volume or somewhere in between but it's really hard to ground truth these things because there isn't any way to go out and measure that volume of the tree anywhere else. It's also important to choose the right tool for the job so drones are really great people are using them for good reason with lots of things but they're best suited for smaller areas like maybe less than 20 hectares and a good example of this is that because we've been flying the arboretum so much we would all wanted to do a full arboretum survey so that we could get a 3D point cloud and model the entire arboretum and measure the height of every tree in the arboretum I think it's about 35,000 trees that they have out there so we finally got the funding to do this we got Darryl and to fly it and it ended up being a huge project it took many weeks of planning there were four or five fly days required you have to have multiple staff on site because when you're flying an area that large you can't fly over people so it's easy to follow CASA regulations when you're just a remote forest but when you're trying to find a fly over a 250 hectare area that's open to the public it becomes much harder. We ended up with 8,800 RGB images and more than 12,000 images from the Sequoia it ultimately took about two months of manual processing because we had to break everything into smaller subsets because the full cloud couldn't run on any machines. None of the online folks can handle more than about 500 images so we couldn't just throw it at one of the cloud services and we ended up with a point cloud that had 584 million data points. So it is important to consider what workflow you will use so this is something we know we could probably do this once a year max but if you add up the time cost of it it becomes prohibitively expensive to do this at this point with the technology available and for something like that it might be better to fly a plane over the Arboretum for instance. It's also if you're thinking of setting up a monitoring plan you need to consider weather and distance to the side of accessibility and time of day because if you want to fly all of your plots at the same time of day say around noon you have to drive between sites you can't do that on the same day and sometimes it turns out that just putting a camera in so you can get consistent data even though it's not as maybe high resolution might be a better option or using satellite data. And again for example a lot of the local agricultural monitoring it turns out it's cheaper to do it by helicopter because you might want to fly five flights, fly five sites in the same day and with a helicopter you can do that in about an hour it might cost you a couple thousand dollars but that's cheaper than trying to drive around to five sites over the course of a week and fly a drone and you can put a much heavier payload on a helicopter. So this is the point cloud we got out of the Arboretum again if you go to trade capture you can go around and explore it, it looks pretty good but you can see there are some artifacts where the stitches didn't align up perfectly because we were doing how to do it in pieces. So again I'm choosing the right tool for the job. Larger scale surveys are not necessarily best to do with a drone and there was an interesting article from Frontiers in the Ecology of the Environment last year where they looked at doing UAV surveys in Alaska and it turned out that it was $1,700 per site to use a UAS but only $400 per site to use a plane mainly because the sites were quite far apart. So it's an important consideration. Now in the future I think what we all want is self-driving drone swarms that fly over our forest or field daily. We have all this data upload and processed in the cloud and get near real-time analytics to the farmer or the end user and I've been wanting someone to release something like this for years and finally this year a country called X-Aircraft in China and a group in Sydney called Revolution Ag are starting to release this system. So here's an agriculture UAV it can carry pesticides or water or nutrients, fertilizer and they can fly themselves. So of course there are regulatory issues around this but we're getting close to the point where we can have a swarm of drones that's just monitoring our sites continuously. And in the future as micro drones get better and higher quality they may be more feasible and safer particularly for field monitoring where maybe there aren't going to be people there because you can imagine a small study you pay $10,000 and get 500 of these drones and they take off every day and fly around your forest and come back. They're all solar powered. They live on the tower somewhere. That would be a really amazing processing solution to give you continuous three-dimensional data of your research site. So some thoughts on developing a drone program. Drones and sensors are a crucial component for field monitoring and phenyl typing but it is hard to do. There's a lot of technology available so you want to focus on the deliverables. Who are your stakeholders or customers and what do they need to know? What would be actionable information for them? It's easy to say oh yeah let's just get a drone and then realize afterwards that you're not you haven't really figured out what kind of data you're going to deliver. And then working back from that you can figure out how big the area is, what the full cost scene is and so on. And then also you want to start small and get your pipeline working. So you don't want to start off flying the 250 hectares of the arboretum. You want to have done the smaller forest first. You want to make sure the UAV you buy is the correct one and whether or not it might just be cheaper to subcontract. And when you process the data you need to decide if you're going to outsource it or do it yourself. And if you need to develop novel tools are there groups already doing this stuff and how can you share this with others so that we all have good tools to use for processing certain sorts of data. And again don't underestimate how hard data management is and have a plan in place at the start. I think we're running out of time so I'll run through this really quickly. I just want to say that all of this data is incredibly, it's new and it's incredibly hard to manage because it's three dimensions and it's got multiple layers and time series and we don't have tools for this that we used to, like the tools that we normally use for data visualization and management are not usable for the tool sets that we have now. So we need a sort of MATLAB, Excel or GIS for time series three dimensional hyper spectral data and this doesn't exist yet. So there's a couple groups at NCI that are developing point cloud viewers. We're working with the VizLab to make a time lapse virtual reality and Windows based point cloud viewer. And if you search for Adam Steer at NCI he's working on a NCI backed one which would be really great because then groups like Turn and us could dump all of our data in the same spot on NCI and we'd have these real time tools for pulling point clouds out on the fly. And just the last thing I'll go through this quick because I know we're out of time. One thing that I wanted to do once I started getting three dimensional data is be able to visualize all the sensor and point cloud data on the landscape where it was collected because that's a way to me that helps you really see the data in the context of where it was collected and the place that you're monitoring. So I collaborated with some students in the computer science department and we made a virtual reality three dimensional model of the National Arboretum using all the drone data as well as our sensor data. Essentially what you can do is you can take three dimensional modeling software developed by Hollywood and use it to reproduce your landscape but rather than using whatever data they're using for a movie you can use your data from your drone flights to generate the three dimensional models. And so this is an example of the project for the National Arboretum here the drone flight data is the three dimensional force that we get out of it. This is the virtual reality version of it where we've taken the digital elevation model to generate the landscape and then put the trees in at the locations that the drone data measured. When you interact with the trees they show you their metadata so in this case we're showing height and area of each tree and then you can also map onto the landscape and play back in time series the different data types that we're collecting with the mesh sensors. So this is a really great tool for pulling all of these dense data layers together and visualize them in one spot to help us make sense of this incredibly complex data. And I think we should all consider this is just the beginning right so when I was a kid I used to play Atari and this is what it looked like and now this is you know the same dragon games that my kids play. And we're at the Atari stage in VR and in our ability to measure the world continuously in 3D and in 10 years VR and AR are indistinguishable from reality so the question is what do we do with these tools and how do we create the next generation of interfaces that facilitate ecosystem research and how we build monitoring and monitoring programs that make best use of all these data types so we can really model our environment in three dimensions and solve the grand challenges of this century. There are lots of people to thank but I think I'm out of time so thanks. Excellent presentation team thank you for joining us and look forward to see you again in the future.
 Welcome to today's webinar on the rise of drones in Australian research space.  My name is Darcy Magala-Party from the Australian National Data Service and I'm your host for  today.  My colleague Karen Wisser is behind the scenes co-hosting the webinar with me.  This one-hour webinar is the second one in this webinar series.  Today's webinar will showcase the success stories of drone applications in Australian  research space, for example in smart farming, plant phenomena and data sciences.  We will also provide an overview of the recent developments about the research infrastructure  and see how we can leverage the benefits of the research infrastructure as well as open  drone data.  I would like to introduce our speakers for today.  Dr. Siddeshwar Guru is the director, turn data science from the University of Queensland.  He is a Kim Bryson, associate dean science faculty, University of Queensland.  Dr. Tim Brown, director Australian plant phenomena facility, Australian National University.  Now I would like to hand over to Dr. Guru for the first presentation.  Thank you Nassima for the introduction and giving the opportunity to talk about this  topic.  So as Nassima said, I am Siddeshwaran Guru.  I work for time.  So today I am talking about the making drone data open for scientific research.  So this is basically the work in progress, what we are exploring and what are the thought  process behind that.  So just a quick introduction when we say drone.  So what is drone?  It is nothing but it is aerial vehicle.  It may be controlled by the on-ground remote control or onboard computer.  So drone has got different names, commonly used as UAVs or the RPAS.  I liked RPAS partly because it is a gender neutral name rather than unmanned.  And so predominantly if it is rotary wings, it is predominantly called as a drone.  And then even the big fixed wings aerial vehicles are also in this category.  Some are pretty big which is as big as the aircraft and then which over the sky and then  gather data.  And the rotary wings, the smaller one, it runs in a control environment where you find  scale resolution and then runs predominantly in the battery and then get the data.  So one of the main things about the drone is it becomes quite cheap to run it.  So that is why it has become quite popular in various applications.  So some of the applications is in a drone, a technology was started in the surveillance  and then currently it is a fair bit of the applications in the agriculture and widely  in the environmental space as well.  I would call it as a scientific research.  And then it is not used in the disaster recovery especially in its very difficult to send a  human being.  So the drones are quite popularly used.  So in turn basically we use drone in the remote sensing capability aspects of the thing.  In this context we use drone basically to derive fractional cover to measure vegetation  species composition, to map vegetation characteristics and then to do some stock take, the pastoral  stock take and then even to do the survey, flora fona survey, kangaroo survey, etc.  So a lot of data will be derived from the measurement that is done in the drone.  So if you look at, you know, there is a fair bit of the vegetation is on partly because  you know, if you take a satellite remote sensing data, the vegetation composition, I think  most of the two-third of the time, you know, it has got the cloud cover over the earth.  So a lot of data is measured.  So drone is a really good technology which runs in the underneath the cloud, below the  cloud and take a good final resolution picture and then do the, and the researchers can do  the analysis.  So in this context, you know, the common sensors used in drone, so basically drone is a platform  and then the sensors are attached to the drone.  So in our case, you know, is a fairly popular things used are the multispectral, hyperspectral,  LiDAR, video and the thermal infrared sensors.  So from a data management perspective, you know, it's basically the, it's a fair bit  of a challenge partly because it's dynamic both space and by space and time, it's keep  on moving at the both the spatial and temporal scale.  And then the, because of its ability to capture finer scale information, you know, it's a  massive amount of data is collected in that one.  And the, we should have a capability to basically to ingest that and then the process that data  and make that data available to the users.  And the other part is, you know, there's a fair bit of the commercial companies who work  in the drone industry and even at the research aspects of the thing, you know, there's always  a partnership with the commercial entity where they will run the drone and then, you know,  collect the data and give it to the researchers, especially there are consultants, especially  in the mining community in the mining area, et cetera.  And the other thing you should be aware that, you know, you need to, you need a permit to  do this from a CASA to operate this.  And then you should be an operator.  You should have a license to operate this as well.  And with all this, you know, the identifying a data owner is important.  I will come to that later why it is important.  And if you look at the, so for example, to make this data, you know, openly accessible,  you know, just, you know, take a bit of a fair principle.  So these are the four aspects of the principle, you know, we just see how the drone data will  fit in side of the thing.  So if we take the first one, you know, data is adequately described, searchable and should  have an identifier.  So just to give a bit of a perspective on the drone, you know, the data is basically  the flight plans and then the, you know, the files of the, you know, flight paths and the  associated field data, you know, generally, especially, you know, in our case.  And then the raw data files from the measurement it tooks and then the files and then the log  from the flight.  And once it's the process, then the, all the derived products as well.  And we should also provide the auxiliary files like a QA QC files from the processing.  So these are the different files, you know, that is, that is a part of a data publication  aspect.  So if you look at, you know, all these are the, you know, it's a related, interrelated  thing.  So all this data should be made available.  And then the, especially for the provenance aspects of the thing, these, all these data  should be made discoverable and then that should be accessible from a user perspective.  And then to make this, you know, data, you know, searchable and then the, you know, in  file.  So basically, you know, there should be a metadata standard to describe this data.  So we, so, you know, we use ISO standards.  We're not sure whether the, you know, so for example, ISO 19115 completely describe everything  or maybe we may have to provide a customized profile of that ISO standard.  And then if you make it available as part of a catalog then, you know, it's a discoverable  and then once we put it in a catalog, we have an identifier and then that's, that looks  fine.  And then the next aspect is the, you know, the, one of the principles is the data retrieval  using open protocol.  So if you look at the, the instruments or the sensors that are used in Trone, you know,  it provides by default, provides a, as a raw data, you know, from a different file format.  And then even at the publication level, if you look at the, you know, the open standard,  you know, in the open data policy, they say that, you know, all the file format should  be of the open format.  So the file format, you know, generally in the, you know, depending on the, which of  the instruments, you know, it may be a give, geotive, KML, shape file, or LAS file, especially  in a, in a point cloud aspect, aspect of the thing.  So with, with such a kind of a veracity of the, you know, file format, then the tools  must be available as well to translate or manipulate even the analyze the data.  And then the, you know, even at the, you know, sometimes it may be worthwhile even to provide  a program, you know, the R program, Python program where, you know, they should, they  can access this data and then, you know, so that they can, they can run the analysis aspects  of the thing.  If it's too confusing, so for example, if somebody don't know, don't have a clue about,  you know, how to access the LAS file, then probably it's worthwhile to provide a program  so that, you know, it's embedded, so that, you know, that is embedded in the, embedded  in the R program so that they can start writing the program.  So that is one of the things, sort of thing.  And the next one probably is the, the, the data use vocabulary and qualified reference  to other metadata.  And then, so we use a, you know, a fair bit of a domain specific vocabulary is like a  GCMD is quite popular, especially keyword search first.  And then, you know, because the, because the drone and then you have a technology as, as  it's called so fast, probably there may be, you know, it may be in when some of the terminology,  you know, incorporated into the vocabulary so that it's, it's accurately represented.  With, as I was explaining the, the different data types and, you know, when each of the  data type, each of the, you know, files should be referenced and then it should be made as  a link and then hopefully as a persistent identifier and then all those should be a query  as well.  And then the, the final, you know, principle is basically data, metadata in domain relevant  community standards with clear data usage license.  So we use an ISO standard, you know, so that 115 or 139 in the domain, which may fit well  as I said before that, you know, maybe we may have to create a profile to accurately  represent the, describe the data.  And a lot of the standard is fine.  If you want to fit the human actionable, you know, discovery, query and access, that means  that the human go and click couple of things.  Probably we may have to look at the, you know, machine actionable, machine actions, accessible,  actionable, discovery and exploration sort of the thing where, you know, it's a machine  to machine query, then that may be a issue partly because of the, so much of the interrelationship  with the files and et cetera.  It's also depending on the, you know, the, the what kind of file.  If we are looking at the source file, then definitely that will be a issue.  And then the, talk about the data usage, you know, so what it says is, you know, oh, you  should provide as an open license thing.  Even though, you know, we may, we provide the all of our data and the creative comments  by attribution, the identifying the owner is a key partly because, you know, even at the  creative comment attribution, the copyright subsist with the data and then you need to  identify who owns the copyright.  As a separate example, if you are, if you are a, if you are a researcher who is using  a consultant to collect the data.  So technically, if you look at that aspect, the person who collected data is the owner  of that data, unless you have a contractual obligation arrangement, make sure that the  ownership is transferable to the researcher.  So why this is important is that, you know, for the attribution aspect set of the thing.  So that if somebody uses your data, they need to know who is attributed as a researcher,  you want the attribution to go to you, not the person who just collected the data.  And the thing is, if somebody is owner, they can do whatever they want with the data.  The IP is with the owner.  So what you want is the, as a principal investigator, you owns the IP about the data.  Still, there are a fair bit of a challenge.  One of the things, you know, VFACE is the, you know, it's just the amount of data set  that is collected.  We still struggle to, you know, make that data, ingest the data into one place, and then  they process everything, and then the delivery of that data.  So we think that the cloud platform, you know, the managed platform to do everything would  be a useful thing.  And there are a few initiatives going on as well at the community level who are looking  exactly at the similar problems.  So one is the OGC domain working group, and the other one is the RDA interest group.  So with all the aspect, you know, we want, we don't want to change a complete, you know,  data management practice of the turn.  So we retrofit everything to the turn data management practice.  So a lot of the, whatever the data, you know, we intend to, you know, make the metadata  and ISO standards, and then cataloging the geo network, and then that is harvested to  the, you know, different repositories, you know, discovery portals, so that the data  is discoverable, and all the data will have a clear, you know, attribution statement with  creative accounts attribution.  And then currently we are storing, you know, data in the FTP server, and then make that  link available.  And we are still in the process of, you know, processing a lot of data.  But having said that the, all the raw files are there, so if somebody is really interested  to use that data, they can process the data as well.  And the other thing, you know, we are still working on the, you know, effective, you know,  the robust data delivery mechanism as well, so that the user just, you know, come and  access the data and get the data.  And one of the things, you know, we are working on the, you know, still working progress is  the, you know, the portal where, you know, user can come and get the final products.  So we collect, you know, we have done a campaign across, you know, love and sites across Australia.  So we are working on that, they call this the field data, and then we are working on  the portal where, you know, all those love and sites data is accessible as well, especially  at the LiDAR level.  So if you look at the drone, you know, it is quite popular in the research community,  you know, especially in the environmental science.  There is a massive uptake of the drone technology, partly because of the ease of use, and then  the final resolution of the data it provides.  And moving ahead, partly, you know, probably it's worthwhile to build an IoT platform  to manage the research drone data.  It's not at the institution level, at the overall research level.  So this probably enable the interconnection of devices, basically based on the type of  sensory use and what is the application you are running, and that will enable to build  a common data platform so that each of the individual doesn't, you know, repeat the same  grappling with the same problem.  And if you look at the commercial word, you know, this is already happening.  There are a lot of commercial players working in this space.  It may be, you know, off the shelf as well.  And then there are even a lot of the open source technology already started appearing,  especially the management aspects of the thing.  So what we want in this one is a researcher has got a platform, they put the sensors,  they collect the data, and we need a platform where that data is ingested somewhere, the  processing happen, and then the product, derived product is basically available for  the third party researcher to analyze that data.  So at the individual level, you know, people are working, but they make everything as a  pipeline and then provide that as a service to the complete community.  We are still working on that one.  It's still a work in progress.  Thanks for the opportunity to talk about this topic.  Thank you, Guru.  It's an excellent presentation, very informative.  Now I would like to hand over to Kim for a second presentation.  Hi, everyone.  My name is Kim Bryson.  I thank you very much for inviting me to be participating in this.  I'm going to be talking about drones and research in relation to agriculture, in fact, in education  as well.  And I think my talk quite nicely segues into some of the things that Guru has been talking  about, because the buzz of today's agriculture is around these disruptive technologies, Internet  of Things, big data, drone technology, smart agriculture.  They're all buzz words and commonly tossed around, and there's been a number of big  conferences in the agricultural space and farm management space about how such technologies  can be of value to the agriculture and to the whole food science issue of, you know,  producing good quality food in a sustainable way.  What we've done at UQ is look at how we can incorporate things like an Internet of Things  multi-sensor mesh network to collect real-time biophysical data, which we store in the university's  cloud.  And we've done that around the whole thousand-odd hectares of the University of Gatton's regional  campus, which is a multi-enterprise agricultural center.  And we've also incorporated a second multi-sensor mesh network around 10 kilometers away from  the campus that will look particularly in relation to – we've set up a living laboratory  there is what we called around how we can use this real-time biophysical data with drone  technology to look at biomass of pasture in particular, but that can end up going, obviously,  to various crops and vegetation as well.  So this network that we've set up, it's very flexible.  We work with technology from Libelium.  We can look at different types of communication protocols.  So we're not just looking at Wi-Fi, but we're looking at other types of radio interfaces  that allow us to transmit data in real-time over long distances and through buildings  and trees, et cetera.  And it's a network that is basically self-healing.  And we have a number of these nodes set up across the campus.  Here we go, smart agriculture, smart water, smart environment, and smart security.  This last one being – the smart security one being essentially the ability through  the network to turn things on and off or open and shut gates, which, of course, is really  important from an agricultural perspective.  The agricultural one looks at – well, in fact, I think my next slide looks at the types  of data that you can gather from these sensors when it's installed.  And we have all of these nodes and most of these sensors around the place.  So we're collecting a lot of biophysical raw data in real-time, which we store in the  cloud and have a dashboard to essentially access.  The idea is that this is done through open source so that at the end of the day, people  can have access to this.  So turn may be interested in it.  Other people may be interested in using it, remembering it's subtropical information,  particularly for people who are interested in the agricultural space.  But we're looking to certainly make that available at the moment through Edurome to  people who are involved in Edurome rather than anything else.  We've tried it on an open network and that we found that it had a risk of the university  being hacked through our system.  So we closed it down from worldwide access to Edurome access.  So lots of data, biophysical data coming in.  And here's a diagram that tells you that we've got data coming in from these various  nodes into our cloud.  And then we're using that data within the educational environment now to develop various,  essentially data uses.  So we've got research projects in here, but we've got farm operation that is something  that we want to do because we'd like this to be developed far more into the agricultural  farm space.  And of course our education component.  The type of data you can get, you can modify all of this through our data dashboard or  you can look at a QR code and get through to it.  So that's the start.  But when we have this large amount of biophysical data, what else do we need and what for?  And of course we want the data for things like partial monitoring and management, animal  monitoring and management, crop monitoring and management and education and more data.  We want to get more data.  So drones provide that or satellite data or remote sensing data provide that capability  of putting aerial data on top of our, are essentially ground and ground data collection  system.  And if you go back to the future, or we're going back to the future in this because remote  sensing has been around in agriculture since the early 1970s and the problem for ag in  relation to remote sensing were two main data.  One was data, cost of acquisition and processing, the revisit frequency, the things that Guru  has talked about actually and also a lack of skills available in the agricultural sector  for this type of data to be used efficiently, well in a cost effective manner and agriculture  people or people in agriculture can be very easily turned off technology.  What is the point of technology if it doesn't deliver me a cost benefit in terms of what  I'm doing and if it costs me a lot and it goes wrong and I can't do it when I want to  do it then it's not worth it.  So we needed to do something about this lack of skills and what we've done here is look  at developing the special skills that are needed to understand spatial variability in  agricultural remote sensing.  We've set up an agricultural remote sensing lab here at the campus and what we're doing  there is we're trying to integrate students from different disciplines across the academic  world.  So engineering students come down and work on agricultural projects in relation to things  like building drones and understanding what drones can be used for.  Building sensors and understanding what those sensors can be used for in agriculture monitoring  in particular but that does involve of course environmental monitoring so our wastewater  management site is probably one of our most recorded biophysical data sites and we get  them to do hands on work so they understand the issues and the risks.  We want to use them because they're cheap platform to carry high resolution sensors.  You can see here a spatial variability in a paddock at ground level you can see a bit  but when you look over here at an aerial photo of that paddock the variability is huge and  that means dollars to a farmer.  So we want to collect that data so we can optimize production efficiency and quality  and we then also want to minimize risk and environmental impact.  So drones from the perspective of an agricultural person could enable us to do better in the  smart food production game almost certainly enables us to do better in the smart environmental  management game and for us as educators smart skills development because it's lots of skills  involved and it's fun to do and one of the things we have in the Australian sector actually  is worldwide.  If difficulty in getting students into the agricultural industry sector and I'm meaning  across the board not just grains or horticulture but across the board.  So at UQ we've got five DJ Phantoms that we use for teaching we've got four bespoke  quadcopters we've got three bespoke hexacopters and ten mini agricultural drones which are  the ones nowadays because these are the ones that I've talked about the ten mini ag drones  are the ones that we use and I'll just talk a little bit about them in the next few minutes  they're easy to fly and there is an article out in 2015 about why we're doing this and  what we're doing.  The design and build principles around this business of getting students of all disciplines  to understand what they're doing we go through a design process trial and error using open  software to look at designing these things we purchase the parts for them they build  them they test fly or they learn to fly learn to solder things like this undertake the project  and write a report.  So this is a classic problem based learning or active learning scheme that we can make  get students to do.  Here are the types of drones that we've investigated and used over time you can see there's a certain  date range here and as much as anything else this small mini UQ mini ag drone we designed  and built because we want to fly under the two kilo limit of CASA because if we had to  get every student certified we couldn't do this.  So this little mini ag drone which I'll talk about in a bit more detail a little bit down  the track is the one that we're now flying and this with a camera that looks at getting  a normalised difference vegetation index image is less than one kilo.  So the first student project was in 2013 where the student literally took a DJI Phantom with  a little normal red, green, blue visual camera back to Fiji.  This was his school in 2003.  This is a Google map.  This is his school in 2013 which is the image that you can see outlined here and he literally  calculated the difference in mango clearing basically so he looked at change in mango.  So this was the first project that we looked at then we started getting a little bit more  creative.  We developed a drone that could carry a multi-spectral camera and we started looking at Prickly Acacia  which is a serious environmental weed up in central Queensland.  Big problem from an agricultural perspective because it shuts out cattle from using the  pasture underneath and we compared some of that data to satellite data and found of course  that these drones creating a better resolution on the ground gave us more data.  This beneficial bug drone, a hexacopter weighing around about 2KG and more when we've got this  white thing underneath it designed by an agricultural science student which is now going to the  has been invited to be put on display at the Science and Technology Museum in London for  the next five years in their display on innovations in agriculture over time.  So this is actually an old innovation now.  This is about 2013, 2014 but it is an instance of when people started looking at using drones.  So this is a box full of beneficial bugs which the farming industry, the farm group that  we worked with wanted to see if they could develop, drop beneficial bugs, these bits  coming out into the middle of a paddock instead of as they customarily do, drive around the  outside and so this was a very successful project with industry involvement that then  started really introducing this sort of technology from the education, from the classroom into  the actual agricultural space.  So that was great.  The net drone was another one where we were asked to fly under hail netting for one of  the largest seedling producers in Australia and here is the netting.  You can see it's a lovely picture if you appreciate this sort of color variation.  The drone is flying here and it was called the net drone project.  We were looking to see which one of these seedlings in 250,000 small tube plants were  germinating or not.  And here you could see this sort of resolution that you could pick up from a standard multi-spectral  camera.  You could see the two-coddoledon stage here and you could introduce students to the idea  that you are looking at three different bands, so a little bit of physics, you're looking  a little bit of spatial variation because you can see where in this one where my cursor  is where things haven't grown and then when you go up here at two-coddoledon stage you  get that interest for a student as much as they can actually see a plant germinating  and they can see where it has not.  And what you do is you get them to calculate what the cost is to both the supplier and  the buyer when they get a tray of seedlings and something hasn't germinated.  So that links it to the economics of the producer.  Here is our mini-ag drone.  We've been flying it now for a couple of years.  We've used it at the beginning of this year for 35 students to do some projects.  It's based on Raspberry Pi technology.  We've upgraded it because we found that the GPS wasn't crash-hot but in this sort of thing  I've got 10 drones which might cost me 100 bucks each to repair but that means that students  can crash them and that's good for a student to be able to do that and over here you can  see them flying on campus.  Sort of data that you can get out of this quite basic no IR or an NDVI data camera is  you can pick up this information quite nicely at an operational scale.  So again I'm guess I'm not at that.  I'm not trying to get these students to be as knowledgeable as we might think from a  research perspective but an undergraduate level 3 agricultural science student they  go out with this knowledge and will then be able to develop their skills in the real world  after that.  We are developing various sensors that can fit onto this small camera so the most recent  one is a multi-spectral sensor.  And multiplexed Raspberry Pi cameras that enable us to choose very high grade research  filters of specific wavelengths when we're wanting to look at something and this particular  camera at the moment is being used on top of one of our IOT multi-sensor mesh network  to collect aerial data of the biomass of that paddock across which of some 50 hectares I  believe that has a multi-sensor mesh network on it.  So this camera is now working so this was when it was being developed.  You can see how it's been set up for little Raspberry Pi cameras and we use Python to  make sure everything goes well.  This is a drone that is no longer flying but it was an attempt at us delivering a drone  that could pick up the RFID tags in a cow's ear from a distance and it works at about  20 meters but you know we're talking a big drone now with a big antenna.  And whilst it worked from an operational perspective and from the idea of trying to keep things  down to a level where we're not flying above CASA's regulations this became essentially  unusable and although I have an engineering student who wishes to try and reduce this  in size this is something that we're not at the moment moving forward to but it is an  idea it's an example of some of the ideas that come when you start talking to the industry  players and asking them what they want.  They wanted something like this because normally you have to stand beside the cow or the cow  has to go through some cattle gates and literally swipe it with a hand swipe.  You've got a thousand head of cattle that's a really difficult, really time consuming  thing to do so this was about trying to improve the efficiency of being able to look at these  cows in the paddock and the sort of data we could get from it was really quite good.  The sorts of types of research and learning involved is electronics and avionics which  is not normal in an agricultural sense but it's really useful if you're going to get  into this side of spatial variability analysis, the design and build side of things.  We look at programming, physics, maths and chemistry of course because we're looking  at the electromagnetic spectrum, things like spectral indices and crop growth indices and  chemicals and plants so if you're trying to engage a student in learning about maths which  they really don't want to do you bring in something like this and it changes their  perception of both learning about the maths and then how it can be used in real life  when it would appear to be from a classroom perspective completely dry and equation driven.  Plant physiology, animal welfare and food traceability are all issues around management  that we can also talk about when we use these tools.  Most recently we've had these sorts of projects going on, drone and sensor development for  crop monitoring and diseases, computer vision development for monitoring pests basically  but we could be doing it in the piggery or we could be doing it out in the environment  for say wildlife people.  We've got a master of engineering student looking at developing a robotic arm for capturing  these small drones out of the sky for autonomous recharging.  Again, it's an academic project probably not to be used operationally because of the issue  of autonomous flight but very useful for the master engineering student to understand why  we want that from an agricultural perspective so we want to increase obviously the reach  of our drones flying and then we've had people developing automated underwater cameras to  monitor phytoplankton which we can then link to our IoT and our aerial imagery of the laser  flakes.  Obviously the IoT has applicability elsewhere that may not involve drones like in the equine  health area and the development of this visualization dashboard for our monitoring purposes and  for staff to access and as I say anyone on EDUROAM to access some of this data that we're  collecting and as I say this is a project the RFID monitoring of cattle is something  that we're still sort of looking at but the idea is to reduce it in size as the electronics  become smaller and we don't have to have such a big antenna involved which means a bigger  drone and more weight etc and start starting to contribute in cattle regulations.  So we're looking at a whole range of things both from an education and research perspective  but mainly the research is about getting is about interesting students in learning about  this stuff which I think is going to be key down the track in terms of capability development  for our industry both agriculture and environment or managed landscapes because without these  kids coming in we're going to run out of people in terms of as they get older they're  not going to be able to continue to do this.  So that's my presentation and I hope you enjoyed it.  Thank you very much.  Hi Kim, thank you.  And I will move on to the next presenter Tim Brown.  Thank you for having me on that was a very interesting talk Kim.  I'm going to be talking about our work at the National Arboretum in Canberra using drones  to make 3D models of the arboretum growing.  I'm the director of the Australian Plant Phenomics Facility at ANU and a lot of this work was  done when I was working for the ARC Center of Plant Energy Biology as a research fellow  and I should first off say thanks to all the people listed there that have been essential  to this project.  I wasn't quite sure what level of understanding people would have on this.  Drones are part of a as Kim pointed out they're part of a key emerging tool set for next gen  monitoring of the environment and these make up sensors, high resolution imaging, LiDAR  and a whole bunch of other things that let us measure the environment in ways we never  could before.  And since I've been working in the Phenomics field really we're interested in how you  monitor the environment along with genetics so you can measure the phenotype because those  are the things together that define how ecosystems develop and in a crop context the phenotype  is essentially what we want for increasing food yields.  So a bit of background on UAVs I think a bit of this has been covered so I'll go through  it quite quickly but you basically have a lot of choices if you're starting a drone program  from the sort of low cost ex rotors like these quad and ex rotors like the DJI Phantom.  These things are changing so fast so I used to give this talk and say that drones were  expensive and confusing and then I said that they were cheap but still a bit confusing  and now they're just cheap and easy to use.  The DJI Mavic Pro came out this year it's a very tiny drone but it still has a 12 megapixel  camera on it you can throw it in a backpack and you can still do pretty decent monitoring  with it at the low end of the things.  These sorts of drones typically cost in the one to two thousand dollar range although the  bigger ones like this one here cost more in the eight thousand dollar range and you get  something like 15 to 20 minutes flight time and up to 10 kilogram payload for the really  big ones.  You can also use fixed winged drones so these give you a longer flight time something to  30 to 60 minute range but you do have to have takeoff and landing issues and they can have  higher payloads but they do tip to fly faster which impacts how you can monitor with them.  A new system that's being developed by people that we work with at Pro UAV is this vertical  takeoff and landing system and we really think this is going to be an amazing tool because  it gives you both the benefits of a fixed wing and some sort of a copter because with  the helicopter part of it it can take off straight so it doesn't need a landing zone  or runway and if something goes wrong and the gas motors fail it will catch itself and  land so you can carry quite a heavy payload and you can put really expensive equipment  on it because it's quite robust.  On the camera side as Kim pointed out you have a lot of options ranging from RGB cameras  the on-born cameras on the Phantom work quite well all the way up to DSLRs but you do need  to watch about what cameras are using and isolate them from vibration or you get rolling  shutter issues where the camera is trying to take a photo but the sensor is not writing  all the data at once and so when there's vibration you end up with bad data.  The next step up unless you're building your one yourself like Kim pointed out is the multi-spectral  cameras that can get you NDVI and the main thing that people use now are either the Micasense  Sequoia or RedEdge and those are in the $3,000 to $5,000 range.  There are a lot of other sensors so you can put hyperspectral cameras on your drones or  thermal and you can see you can get bands in the 400 to 1300 nanometer range but you  need a big quadcopter because they're heavy, they're very expensive and the data is quite  hard to process so there really is a range of stuff available from really easy to use  up to quite challenging depending on what output you need.  The typical outputs you get from 3D reconstruction software which is a lot of what I'll be talking  about are ortho mosaic images so these are essentially satellite layers that you can  put into Google Maps or Mapbox and then also DMs and GeoTiffs and some of the software  can actually give you somewhat classified outputs or remove the ground so you only get  trees can provide you with RGB multi-spectral or hyperspectral indices and also 3D point  clouds and then you can make a 3D model of your environment as well if that's what you're  interested in.  There are a lot of options on the 3D reconstruction software front and I'll just go through a  few, it's important to know that it requires a pretty beefy PC to run these things so you're  looking at probably 1500 AUD to get a PC because you need a fair bit of RAM, you need  a pretty good processor and then you need a graphics card.  PIX40 Mapper Pro which is the one that we've been using because it was about the only one  available when we started working, it ranges from $2000 to about $9000 depending on what  license you get and you have to pay about $1000 a year for support and also as a challenge  that you can only run on Windows so if you want to run it on a server or on the Amazon  cloud for much bigger data processing and for automating stuff you have to get the  Pro license.  The other one people use a lot is Aggie Soft Photo Scan, I haven't used it personally but  it's been well recommended.  The cheap version if you just want 3D models is about $60 US and about $550 academic and  then pricing goes up from there and those also will run on the cloud if you buy the Pro version.  Mosaic Mill is another software package, it's the last quote I got from them was about  4500 euros, it comes in a lot of flavors but I haven't used it.  There's a free software package called VisualSFM which is FSM which seems to work pretty well  because it's free and somewhat open source software, it doesn't have all the bells and  whistles the expensive ones do but if you have more time than you do money for your project  it's probably a good thing to explore.  There are also a lot of online options and these are great for testing so if you either  just do occasional flights and don't want to invest in the software in the PC or you  want really fast processing or you're preparing a grant proposal so you just need to get some  initial data websites like these ones let you just upload your images and give you a  point cloud and other data quite quickly.  A disclaimer this is not a complete list and all of these prices change rapidly and frequently  so check the vendors for pricing and don't take my word for it, there's plenty of other  information out there but you see from these software you can take a flight like you see  on the left side here and then you get a 3D reconstruction on the forest and I'll talk  more about that in a minute.  So the site where we've been flying drones is at the National Arboretum in Canberra and  this is a really great site because it's just 5 kilometers from ANU and has fast Wi-Fi.  The last site I worked at was in southern Utah and it took about 5 hours to drive each direction  so when something broke it was a good day and a half just to get there and back and  having a field site where you have Wi-Fi access from your desk and you can drive out there  in 5 minutes really makes it easy to test new and emerging technologies until they're  ready for deploying elsewhere.  And also the forest was only planted about 7 years ago so we have an opportunity to monitor  this forest and build a 3 dimensional model of the entire arboretum growing into the future  which has basically never been possible before.  So we initially at the ANU research site we installed a 20 node wireless mesh sensor network,  camber weather stations for baseline data, some gigapixel cameras, these are cameras  that take hundreds or thousands of high resolution images that you squish together to get a multi-billion  resolution picture.  We've done some LiDAR scans both on the ground from various sorts and then we've done near  monthly UAV flights.  And once you sequence the trees out there because tree sequencing is getting down to  you know the 10 to 20 dollar per tree range you can really measure phenotype environment  and genetics in a way that gives you an amazing density of data about a space that we were  never able to access before.  So here's camber on the southeast side of Australia, the arboretum is over on the west  side of camber, south of Black Mountain if you've been there.  This is our main field area, this is a picture of what it looks like from one of the cameras  up on the hill, so this is a 500 megapixel image that we generate every hour and you  can zoom all the way into that image to see the forest out on the far side there.  This is what that forest looks like when you fly over it with a drone and then from the  drone data you can bring that into PIX4D as I'll show in a second and get a three dimensional  model of the trees.  So the drone monitoring program, the goal was to test and develop a time series drone  monitoring program so we could get 3D models of the trees growing.  We get time lapse georectified image layers, 3D point cloud and then some phenotypes we  can measure like tree height, area is measured by top down pixels and color data over time.  And this forest we're studying is, it's 12 forests, a spotted gum and Iron Bark was planted  about 2012 and it's 4 hectares so it's really a perfect size for drone monitoring because  you can fly the whole site in about 15 minutes it's not too hard to process and it's very  amenable to that sort of monitoring.  We've really come a long way on this so in April 2013 I flew the first drone over it which  was a cell phone duct tape to a drone that I'd made at home and now we're using a really  nice solid Matress Pro by DJI with Darrell from Pro UAV who's been flying it for us  and really good cameras and so we're really getting a lot of solid data but just in the  last 4 years, 5 years this technology has changed so much.  The typical workflow in this is the PIX4D is a software that we use, it pulls in all  the images you can see here that were taken from the drone and then you can see on the  left all those green lines going down are key points that is detected in all the images  and there's really a lot of black magic that happens behind the scenes in these software  packages and you can see on the right those are some little tiny piece of ground that  the software has detected as being the same in about 30 different photos and it uses that  to calculate the actual position of each photo relative to the ground.  Those are called control points.  From that it creates the 3D point cloud as you can see here and that gives you this sort  of ghostly 3D model of your entire forest.  You can see that there's some data missing from the bottom because the trees are the  drones are just looking down from the top and can't see around the edges of the trees.  The total processing time to do this is very hardware dependent so it ranges from like  3 to 12 hours but it allows you to go from pictures to point clouds to 3 dimensional  models of the trees you can see here.  It's important to realize how groundbreaking this technology is because previously if you  wanted to measure the location and height of every tree it would have taken you an incredibly  long time.  We have trees in this forest and we can do this now in about a morning but there are  still challenges as Ger was pointing out with working with and serving up the data.  We implemented a point cloud viewer called Poetry on our website, traitcapture.org.  You can go there and see some of the point clouds that we have online.  We wrote a software package called forestutils that runs on Python that lets us pull out  the locations and height of every tree and the point cloud data associated with that  tree.  So this assumes an open canopy.  If you have a closed canopy things are a bit harder unless you have GPS locations for  your trees but now once we have the tree locations when the canopy closes we can keep tracking  them and it outputs tree height top down area location RGB colors and a point count which  is a measure of how many points were generated for that tree.  It also spits out a CSV file that you can stick on Google maps or anywhere you want  that has all of your tree locations.  So that is the program we ran.  We probably have flown 30 flights over the last since mid 2015.  I want to talk a little bit about data management because this is really crucial and if you  are planning to do anything more than just occasional flights you really need to come  up with a good data management plan and you want to do this before you start your surveys.  It is important to consider the entire workflow because it is not just who captures it or  what happens to it but the entire process.  If you think of who does the surveys if you have more than one company or more than one  person the data has to get to you somehow.  It has to get processed.  You have to figure out where it goes on your computer and track it.  Until people like Tern have made us nice tools for having our data go seamlessly online you  have to manage all of this stuff yourself until it gets to the point where you publish it.  You have to figure out smart ways for naming things and you may get a data set and then  add to it and then add to it again and then process it and you need some sort of workflow  that tracks how that is taking place and doing that across thousands of images or hundreds  of flights can really be a challenge.  And if people upload data you need to make sure that they have told you how much data  they have so that you can have all of it so you don't spend a week processing their data  and then find out that they hadn't finished uploading it and you needed to add another  hundred images and run the entire thing again.  Often you run into problems like we stole all of our data on a large data server at the  research school of biology but we process it on a computer that is local and has an  SSD so we have to move the data gets uploaded into one folder by whoever took it gets moved  to another folder which is the storage folder and then gets copied to this computer to process  and then has to be copied back with the new data as well and so that makes it quite hard  to track things over time and if experiments fail or new data comes in you need to have  a workflow for how you know where things are and what the status of them is.  So it's really best to enforce rigorous note taking even just having people whatever  tech is running the project writing down what they are doing as they do it can be really  happy and handy.  Also shared Google docs and notepad plus plus so you can put just files within each folder  can be really useful.  So here's an example of the naming structure that we settled on and basically the idea  was to make when someone looks in a folder to make it reasonably human usable so we have  the year location site who captured it and the status in this case it was the national  forest and national arboretum the ANU forest plot actually this was pro UAV that captured  that one and we wrote that it was done and uploaded.  So this seems like a great idea but of course if you have any sort of nested folders like  you might want to name your dataset national arboretum the file names get rapidly too long  for windows and your entire process fails.  So this makes it a challenge because you need metadata but you have to have someone who  actually is maintaining it and as I said shared Google docs are good but this isn't really  a solve problem and also you know everyone always ends up with files like this because  before we implemented a data management plan things were just going on to my laptop and  then getting copied into random places.  I think a lot of people get flummoxed by data management because it's really hard but it's  not easy for anyone and it's easy to think that you don't know what you're doing but  I'm not sure that anybody really does and you won't get things right the first time  and you have to start with a plan and keep working at it and just acknowledge that it's  not going to work the way you expected as soon as you start to implement it and then  you need to go back and change it because in reality our data management typically looks  something like that and we want to move it more towards the vision but it isn't actually  there but you need to address these problems because they end up making your data unusable  when we have these large scale huge time series datasets.  Some other challenges with processing drone data are that these new kinds of three dimensional  data so for things like NDVI or some of the metrics that Kim was mentioning that there's  a lot of known information about that because people have been working with that sort of  data for a long time but a lot of data like a three dimensional point cloud of a tree  it's hard to know how you tie data values like that to biologically meaningful things  and it's also challenging getting back to what Gerr was talking about with the provenance  of the data and tracking what's been processed.  You can process the same project in three different versions of PIC40 and get three  at different outputs and there's also about a hundred different ways you can vary the  settings so at one point recently we decided to test every single setting we could think  of in PIC40 and you can see an output from that table here and using exactly the same  images you can get a stitching time ranging from five minutes to 55 minutes and point  cloud sizes ranging from 44 million points down to 11 million points and it's easy to  think for example that more points means more data but if you happen to be taking pictures  on a windy day and your tree is moving around you probably want your data doesn't have the  resolution of a leaf it just has the resolution of the height and structure of a tree so it  may be that either the less points is actually a better measurement of that tree volume or  somewhere in between but it's really hard to ground truth these things because there  isn't any way to go out and measure that volume of the tree anywhere else.  It's also important to choose the right tool for the job so drones are really great people  are using them for good reason with lots of things but they're best suited for smaller  areas like maybe less than 20 hectares and a good example of this is that because we've  been flying the arboretum so much we would all wanted to do a full arboretum survey  so that we could get a 3D point cloud and model the entire arboretum and measure the  height of every tree in the arboretum I think it's about 35,000 trees that they have out  there so we finally got the funding to do this we got Darryl and to fly it and it ended  up being a huge project it took many weeks of planning there were four or five fly days  required you have to have multiple staff on site because when you're flying an area that  large you can't fly over people so it's easy to follow CASA regulations when you're just  a remote forest but when you're trying to find a fly over a 250 hectare area that's  open to the public it becomes much harder.  We ended up with 8,800 RGB images and more than 12,000 images from the Sequoia it ultimately  took about two months of manual processing because we had to break everything into smaller  subsets because the full cloud couldn't run on any machines.  None of the online folks can handle more than about 500 images so we couldn't just throw  it at one of the cloud services and we ended up with a point cloud that had 584 million  data points.  So it is important to consider what workflow you will use so this is something we know  we could probably do this once a year max but if you add up the time cost of it it becomes  prohibitively expensive to do this at this point with the technology available and for  something like that it might be better to fly a plane over the Arboretum for instance.  It's also if you're thinking of setting up a monitoring plan you need to consider weather  and distance to the side of accessibility and time of day because if you want to fly  all of your plots at the same time of day say around noon you have to drive between  sites you can't do that on the same day and sometimes it turns out that just putting a  camera in so you can get consistent data even though it's not as maybe high resolution might  be a better option or using satellite data.  And again for example a lot of the local agricultural monitoring it turns out it's cheaper to do  it by helicopter because you might want to fly five flights, fly five sites in the same  day and with a helicopter you can do that in about an hour it might cost you a couple  thousand dollars but that's cheaper than trying to drive around to five sites over the course  of a week and fly a drone and you can put a much heavier payload on a helicopter.  So this is the point cloud we got out of the Arboretum again if you go to trade capture  you can go around and explore it, it looks pretty good but you can see there are some  artifacts where the stitches didn't align up perfectly because we were doing how to  do it in pieces.  So again I'm choosing the right tool for the job.  Larger scale surveys are not necessarily best to do with a drone and there was an interesting  article from Frontiers in the Ecology of the Environment last year where they looked at  doing UAV surveys in Alaska and it turned out that it was $1,700 per site to use a UAS  but only $400 per site to use a plane mainly because the sites were quite far apart.  So it's an important consideration.  Now in the future I think what we all want is self-driving drone swarms that fly over  our forest or field daily.  We have all this data upload and processed in the cloud and get near real-time analytics  to the farmer or the end user and I've been wanting someone to release something like  this for years and finally this year a country called X-Aircraft in China and a group in  Sydney called Revolution Ag are starting to release this system.  So here's an agriculture UAV it can carry pesticides or water or nutrients, fertilizer  and they can fly themselves.  So of course there are regulatory issues around this but we're getting close to the point  where we can have a swarm of drones that's just monitoring our sites continuously.  And in the future as micro drones get better and higher quality they may be more feasible  and safer particularly for field monitoring where maybe there aren't going to be people  there because you can imagine a small study you pay $10,000 and get 500 of these drones  and they take off every day and fly around your forest and come back.  They're all solar powered.  They live on the tower somewhere.  That would be a really amazing processing solution to give you continuous three-dimensional  data of your research site.  So some thoughts on developing a drone program.  Drones and sensors are a crucial component for field monitoring and phenyl typing but  it is hard to do.  There's a lot of technology available so you want to focus on the deliverables.  Who are your stakeholders or customers and what do they need to know?  What would be actionable information for them?  It's easy to say oh yeah let's just get a drone and then realize afterwards that you're  not you haven't really figured out what kind of data you're going to deliver.  And then working back from that you can figure out how big the area is, what the full cost  scene is and so on.  And then also you want to start small and get your pipeline working.  So you don't want to start off flying the 250 hectares of the arboretum.  You want to have done the smaller forest first.  You want to make sure the UAV you buy is the correct one and whether or not it might just  be cheaper to subcontract.  And when you process the data you need to decide if you're going to outsource it or  do it yourself.  And if you need to develop novel tools are there groups already doing this stuff and  how can you share this with others so that we all have good tools to use for processing  certain sorts of data.  And again don't underestimate how hard data management is and have a plan in place at  the start.  I think we're running out of time so I'll run through this really quickly.  I just want to say that all of this data is incredibly, it's new and it's incredibly  hard to manage because it's three dimensions and it's got multiple layers and time series  and we don't have tools for this that we used to, like the tools that we normally use for  data visualization and management are not usable for the tool sets that we have now.  So we need a sort of MATLAB, Excel or GIS for time series three dimensional hyper spectral  data and this doesn't exist yet.  So there's a couple groups at NCI that are developing point cloud viewers.  We're working with the VizLab to make a time lapse virtual reality and Windows based point  cloud viewer.  And if you search for Adam Steer at NCI he's working on a NCI backed one which would be  really great because then groups like Turn and us could dump all of our data in the same  spot on NCI and we'd have these real time tools for pulling point clouds out on the  fly.  And just the last thing I'll go through this quick because I know we're out of time.  One thing that I wanted to do once I started getting three dimensional data is be able  to visualize all the sensor and point cloud data on the landscape where it was collected  because that's a way to me that helps you really see the data in the context of where  it was collected and the place that you're monitoring.  So I collaborated with some students in the computer science department and we made a  virtual reality three dimensional model of the National Arboretum using all the drone  data as well as our sensor data.  Essentially what you can do is you can take three dimensional modeling software developed  by Hollywood and use it to reproduce your landscape but rather than using whatever data  they're using for a movie you can use your data from your drone flights to generate the  three dimensional models.  And so this is an example of the project for the National Arboretum here the drone flight  data is the three dimensional force that we get out of it.  This is the virtual reality version of it where we've taken the digital elevation model  to generate the landscape and then put the trees in at the locations that the drone  data measured.  When you interact with the trees they show you their metadata so in this case we're showing  height and area of each tree and then you can also map onto the landscape and play back  in time series the different data types that we're collecting with the mesh sensors.  So this is a really great tool for pulling all of these dense data layers together and  visualize them in one spot to help us make sense of this incredibly complex data.  And I think we should all consider this is just the beginning right so when I was a kid  I used to play Atari and this is what it looked like and now this is you know the same dragon  games that my kids play.  And we're at the Atari stage in VR and in our ability to measure the world continuously  in 3D and in 10 years VR and AR are indistinguishable from reality so the question is what do we  do with these tools and how do we create the next generation of interfaces that facilitate  ecosystem research and how we build monitoring and monitoring programs that make best use  of all these data types so we can really model our environment in three dimensions and solve  the grand challenges of this century.  There are lots of people to thank but I think I'm out of time so thanks.  Excellent presentation team thank you for joining us and look forward to see you again  in the future.
Rise_of_drones_in_the_Australian_research_space_0_1775.png Rise_of_drones_in_the_Australian_research_space_1776_2339.png Rise_of_drones_in_the_Australian_research_space_2340_4199.png Rise_of_drones_in_the_Australian_research_space_4200_4919.png Rise_of_drones_in_the_Australian_research_space_4920_7499.png Rise_of_drones_in_the_Australian_research_space_7500_9599.png Rise_of_drones_in_the_Australian_research_space_9600_10019.png Rise_of_drones_in_the_Australian_research_space_10020_21311.png Rise_of_drones_in_the_Australian_research_space_21312_23819.png Rise_of_drones_in_the_Australian_research_space_23820_24191.png Rise_of_drones_in_the_Australian_research_space_24192_26627.png Rise_of_drones_in_the_Australian_research_space_26628_26723.png Rise_of_drones_in_the_Australian_research_space_26724_27119.png Rise_of_drones_in_the_Australian_research_space_27120_27935.png Rise_of_drones_in_the_Australian_research_space_27936_30467.png Rise_of_drones_in_the_Australian_research_space_30468_31883.png Rise_of_drones_in_the_Australian_research_space_31884_32579.png Rise_of_drones_in_the_Australian_research_space_32580_32759.png Rise_of_drones_in_the_Australian_research_space_32760_33515.png Rise_of_drones_in_the_Australian_research_space_33516_34751.png Rise_of_drones_in_the_Australian_research_space_34752_35951.png Rise_of_drones_in_the_Australian_research_space_35952_37907.png Rise_of_drones_in_the_Australian_research_space_37908_38111.png Rise_of_drones_in_the_Australian_research_space_38112_39827.png Rise_of_drones_in_the_Australian_research_space_39828_40595.png Rise_of_drones_in_the_Australian_research_space_40596_42923.png Rise_of_drones_in_the_Australian_research_space_42924_45467.png Rise_of_drones_in_the_Australian_research_space_45468_47351.png Rise_of_drones_in_the_Australian_research_space_47352_49163.png Rise_of_drones_in_the_Australian_research_space_49164_50387.png Rise_of_drones_in_the_Australian_research_space_50388_53639.png Rise_of_drones_in_the_Australian_research_space_53640_53891.png Rise_of_drones_in_the_Australian_research_space_53892_54599.png Rise_of_drones_in_the_Australian_research_space_54600_55511.png Rise_of_drones_in_the_Australian_research_space_55512_55739.png Rise_of_drones_in_the_Australian_research_space_55740_56855.png Rise_of_drones_in_the_Australian_research_space_56856_58091.png Rise_of_drones_in_the_Australian_research_space_58092_58703.png Rise_of_drones_in_the_Australian_research_space_58704_63947.png Rise_of_drones_in_the_Australian_research_space_63948_66119.png Rise_of_drones_in_the_Australian_research_space_66120_66215.png Rise_of_drones_in_the_Australian_research_space_66216_66419.png Rise_of_drones_in_the_Australian_research_space_66420_66539.png Rise_of_drones_in_the_Australian_research_space_66540_66587.png Rise_of_drones_in_the_Australian_research_space_66588_66659.png Rise_of_drones_in_the_Australian_research_space_66660_66707.png Rise_of_drones_in_the_Australian_research_space_66708_66935.png Rise_of_drones_in_the_Australian_research_space_66936_68543.png Rise_of_drones_in_the_Australian_research_space_68544_69455.png Rise_of_drones_in_the_Australian_research_space_69456_70271.png Rise_of_drones_in_the_Australian_research_space_70272_71975.png Rise_of_drones_in_the_Australian_research_space_71976_72419.png Rise_of_drones_in_the_Australian_research_space_72420_72467.png Rise_of_drones_in_the_Australian_research_space_72468_76667.png Rise_of_drones_in_the_Australian_research_space_76668_83411.png Rise_of_drones_in_the_Australian_research_space_83412_83771.png Rise_of_drones_in_the_Australian_research_space_83772_84419.png Rise_of_drones_in_the_Australian_research_space_84420_86195.png Rise_of_drones_in_the_Australian_research_space_86196_86795.png Rise_of_drones_in_the_Australian_research_space_86796_87887.png Rise_of_drones_in_the_Australian_research_space_87888_87971.png Rise_of_drones_in_the_Australian_research_space_87972_88715.png Rise_of_drones_in_the_Australian_research_space_88716_89435.png Rise_of_drones_in_the_Australian_research_space_89436_90227.png Rise_of_drones_in_the_Australian_research_space_90228_90623.png Rise_of_drones_in_the_Australian_research_space_90624_91019.png Rise_of_drones_in_the_Australian_research_space_91020_91067.png Rise_of_drones_in_the_Australian_research_space_91068_91175.png Rise_of_drones_in_the_Australian_research_space_91176_91247.png Rise_of_drones_in_the_Australian_research_space_91248_91319.png Rise_of_drones_in_the_Australian_research_space_91320_92651.png Rise_of_drones_in_the_Australian_research_space_92652_92783.png Rise_of_drones_in_the_Australian_research_space_92784_92899.png
Welcome to today's webinar on the rise of drones in Australian research space. My name is Darcy Magala-Party from the Australian National Data Service and I'm your host for today. My colleague Karen Wisser is behind the scenes co-hosting the webinar with me. This one-hour webinar is the second one in this webinar series. Today's webinar will showcase the success stories of drone applications in Australian research space, for example in smart farming, plant phenomena and data sciences. We will also provide an overview of the recent developments about the research infrastructure and see how we can leverage the benefits of the research infrastructure as well as open drone data. I would like to introduce our speakers for today. Dr. Siddeshwar Guru is the director, turn data science from the University of Queensland. He is a Kim Bryson, associate dean science faculty, University of Queensland. Dr. Tim Brown, director Australian plant phenomena facility, Australian National University. Now I would like to hand over to Dr. Guru for the first presentation. Thank you Nassima for the introduction and giving the opportunity to talk about this topic. So as Nassima said, I am Siddeshwaran Guru. I work for time. So today I am talking about the making drone data open for scientific research. So this is basically the work in progress, what we are exploring and what are the thought process behind that. So just a quick introduction when we say drone. So what is drone? It is nothing but it is aerial vehicle. It may be controlled by the on-ground remote control or onboard computer. So drone has got different names, commonly used as UAVs or the RPAS. I liked RPAS partly because it is a gender neutral name rather than unmanned. And so predominantly if it is rotary wings, it is predominantly called as a drone. And then even the big fixed wings aerial vehicles are also in this category. Some are pretty big which is as big as the aircraft and then which over the sky and then gather data. And the rotary wings, the smaller one, it runs in a control environment where you find scale resolution and then runs predominantly in the battery and then get the data. So one of the main things about the drone is it becomes quite cheap to run it. So that is why it has become quite popular in various applications. So some of the applications is in a drone, a technology was started in the surveillance and then currently it is a fair bit of the applications in the agriculture and widely in the environmental space as well. I would call it as a scientific research. And then it is not used in the disaster recovery especially in its very difficult to send a human being. So the drones are quite popularly used. So in turn basically we use drone in the remote sensing capability aspects of the thing. In this context we use drone basically to derive fractional cover to measure vegetation species composition, to map vegetation characteristics and then to do some stock take, the pastoral stock take and then even to do the survey, flora fona survey, kangaroo survey, etc. So a lot of data will be derived from the measurement that is done in the drone. So if you look at, you know, there is a fair bit of the vegetation is on partly because you know, if you take a satellite remote sensing data, the vegetation composition, I think most of the two-third of the time, you know, it has got the cloud cover over the earth. So a lot of data is measured. So drone is a really good technology which runs in the underneath the cloud, below the cloud and take a good final resolution picture and then do the, and the researchers can do the analysis. So in this context, you know, the common sensors used in drone, so basically drone is a platform and then the sensors are attached to the drone. So in our case, you know, is a fairly popular things used are the multispectral, hyperspectral, LiDAR, video and the thermal infrared sensors. So from a data management perspective, you know, it's basically the, it's a fair bit of a challenge partly because it's dynamic both space and by space and time, it's keep on moving at the both the spatial and temporal scale. And then the, because of its ability to capture finer scale information, you know, it's a massive amount of data is collected in that one. And the, we should have a capability to basically to ingest that and then the process that data and make that data available to the users. And the other part is, you know, there's a fair bit of the commercial companies who work in the drone industry and even at the research aspects of the thing, you know, there's always a partnership with the commercial entity where they will run the drone and then, you know, collect the data and give it to the researchers, especially there are consultants, especially in the mining community in the mining area, et cetera. And the other thing you should be aware that, you know, you need to, you need a permit to do this from a CASA to operate this. And then you should be an operator. You should have a license to operate this as well. And with all this, you know, the identifying a data owner is important. I will come to that later why it is important. And if you look at the, so for example, to make this data, you know, openly accessible, you know, just, you know, take a bit of a fair principle. So these are the four aspects of the principle, you know, we just see how the drone data will fit in side of the thing. So if we take the first one, you know, data is adequately described, searchable and should have an identifier. So just to give a bit of a perspective on the drone, you know, the data is basically the flight plans and then the, you know, the files of the, you know, flight paths and the associated field data, you know, generally, especially, you know, in our case. And then the raw data files from the measurement it tooks and then the files and then the log from the flight. And once it's the process, then the, all the derived products as well. And we should also provide the auxiliary files like a QA QC files from the processing. So these are the different files, you know, that is, that is a part of a data publication aspect. So if you look at, you know, all these are the, you know, it's a related, interrelated thing. So all this data should be made available. And then the, especially for the provenance aspects of the thing, these, all these data should be made discoverable and then that should be accessible from a user perspective. And then to make this, you know, data, you know, searchable and then the, you know, in file. So basically, you know, there should be a metadata standard to describe this data. So we, so, you know, we use ISO standards. We're not sure whether the, you know, so for example, ISO 19115 completely describe everything or maybe we may have to provide a customized profile of that ISO standard. And then if you make it available as part of a catalog then, you know, it's a discoverable and then once we put it in a catalog, we have an identifier and then that's, that looks fine. And then the next aspect is the, you know, the, one of the principles is the data retrieval using open protocol. So if you look at the, the instruments or the sensors that are used in Trone, you know, it provides by default, provides a, as a raw data, you know, from a different file format. And then even at the publication level, if you look at the, you know, the open standard, you know, in the open data policy, they say that, you know, all the file format should be of the open format. So the file format, you know, generally in the, you know, depending on the, which of the instruments, you know, it may be a give, geotive, KML, shape file, or LAS file, especially in a, in a point cloud aspect, aspect of the thing. So with, with such a kind of a veracity of the, you know, file format, then the tools must be available as well to translate or manipulate even the analyze the data. And then the, you know, even at the, you know, sometimes it may be worthwhile even to provide a program, you know, the R program, Python program where, you know, they should, they can access this data and then, you know, so that they can, they can run the analysis aspects of the thing. If it's too confusing, so for example, if somebody don't know, don't have a clue about, you know, how to access the LAS file, then probably it's worthwhile to provide a program so that, you know, it's embedded, so that, you know, that is embedded in the, embedded in the R program so that they can start writing the program. So that is one of the things, sort of thing. And the next one probably is the, the, the data use vocabulary and qualified reference to other metadata. And then, so we use a, you know, a fair bit of a domain specific vocabulary is like a GCMD is quite popular, especially keyword search first. And then, you know, because the, because the drone and then you have a technology as, as it's called so fast, probably there may be, you know, it may be in when some of the terminology, you know, incorporated into the vocabulary so that it's, it's accurately represented. With, as I was explaining the, the different data types and, you know, when each of the data type, each of the, you know, files should be referenced and then it should be made as a link and then hopefully as a persistent identifier and then all those should be a query as well. And then the, the final, you know, principle is basically data, metadata in domain relevant community standards with clear data usage license. So we use an ISO standard, you know, so that 115 or 139 in the domain, which may fit well as I said before that, you know, maybe we may have to create a profile to accurately represent the, describe the data. And a lot of the standard is fine. If you want to fit the human actionable, you know, discovery, query and access, that means that the human go and click couple of things. Probably we may have to look at the, you know, machine actionable, machine actions, accessible, actionable, discovery and exploration sort of the thing where, you know, it's a machine to machine query, then that may be a issue partly because of the, so much of the interrelationship with the files and et cetera. It's also depending on the, you know, the, the what kind of file. If we are looking at the source file, then definitely that will be a issue. And then the, talk about the data usage, you know, so what it says is, you know, oh, you should provide as an open license thing. Even though, you know, we may, we provide the all of our data and the creative comments by attribution, the identifying the owner is a key partly because, you know, even at the creative comment attribution, the copyright subsist with the data and then you need to identify who owns the copyright. As a separate example, if you are, if you are a, if you are a researcher who is using a consultant to collect the data. So technically, if you look at that aspect, the person who collected data is the owner of that data, unless you have a contractual obligation arrangement, make sure that the ownership is transferable to the researcher. So why this is important is that, you know, for the attribution aspect set of the thing. So that if somebody uses your data, they need to know who is attributed as a researcher, you want the attribution to go to you, not the person who just collected the data. And the thing is, if somebody is owner, they can do whatever they want with the data. The IP is with the owner. So what you want is the, as a principal investigator, you owns the IP about the data. Still, there are a fair bit of a challenge. One of the things, you know, VFACE is the, you know, it's just the amount of data set that is collected. We still struggle to, you know, make that data, ingest the data into one place, and then they process everything, and then the delivery of that data. So we think that the cloud platform, you know, the managed platform to do everything would be a useful thing. And there are a few initiatives going on as well at the community level who are looking exactly at the similar problems. So one is the OGC domain working group, and the other one is the RDA interest group. So with all the aspect, you know, we want, we don't want to change a complete, you know, data management practice of the turn. So we retrofit everything to the turn data management practice. So a lot of the, whatever the data, you know, we intend to, you know, make the metadata and ISO standards, and then cataloging the geo network, and then that is harvested to the, you know, different repositories, you know, discovery portals, so that the data is discoverable, and all the data will have a clear, you know, attribution statement with creative accounts attribution. And then currently we are storing, you know, data in the FTP server, and then make that link available. And we are still in the process of, you know, processing a lot of data. But having said that the, all the raw files are there, so if somebody is really interested to use that data, they can process the data as well. And the other thing, you know, we are still working on the, you know, effective, you know, the robust data delivery mechanism as well, so that the user just, you know, come and access the data and get the data. And one of the things, you know, we are working on the, you know, still working progress is the, you know, the portal where, you know, user can come and get the final products. So we collect, you know, we have done a campaign across, you know, love and sites across Australia. So we are working on that, they call this the field data, and then we are working on the portal where, you know, all those love and sites data is accessible as well, especially at the LiDAR level. So if you look at the drone, you know, it is quite popular in the research community, you know, especially in the environmental science. There is a massive uptake of the drone technology, partly because of the ease of use, and then the final resolution of the data it provides. And moving ahead, partly, you know, probably it's worthwhile to build an IoT platform to manage the research drone data. It's not at the institution level, at the overall research level. So this probably enable the interconnection of devices, basically based on the type of sensory use and what is the application you are running, and that will enable to build a common data platform so that each of the individual doesn't, you know, repeat the same grappling with the same problem. And if you look at the commercial word, you know, this is already happening. There are a lot of commercial players working in this space. It may be, you know, off the shelf as well. And then there are even a lot of the open source technology already started appearing, especially the management aspects of the thing. So what we want in this one is a researcher has got a platform, they put the sensors, they collect the data, and we need a platform where that data is ingested somewhere, the processing happen, and then the product, derived product is basically available for the third party researcher to analyze that data. So at the individual level, you know, people are working, but they make everything as a pipeline and then provide that as a service to the complete community. We are still working on that one. It's still a work in progress. Thanks for the opportunity to talk about this topic. Thank you, Guru. It's an excellent presentation, very informative. Now I would like to hand over to Kim for a second presentation. Hi, everyone. My name is Kim Bryson. I thank you very much for inviting me to be participating in this. I'm going to be talking about drones and research in relation to agriculture, in fact, in education as well. And I think my talk quite nicely segues into some of the things that Guru has been talking about, because the buzz of today's agriculture is around these disruptive technologies, Internet of Things, big data, drone technology, smart agriculture. They're all buzz words and commonly tossed around, and there's been a number of big conferences in the agricultural space and farm management space about how such technologies can be of value to the agriculture and to the whole food science issue of, you know, producing good quality food in a sustainable way. What we've done at UQ is look at how we can incorporate things like an Internet of Things multi-sensor mesh network to collect real-time biophysical data, which we store in the university's cloud. And we've done that around the whole thousand-odd hectares of the University of Gatton's regional campus, which is a multi-enterprise agricultural center. And we've also incorporated a second multi-sensor mesh network around 10 kilometers away from the campus that will look particularly in relation to – we've set up a living laboratory there is what we called around how we can use this real-time biophysical data with drone technology to look at biomass of pasture in particular, but that can end up going, obviously, to various crops and vegetation as well. So this network that we've set up, it's very flexible. We work with technology from Libelium. We can look at different types of communication protocols. So we're not just looking at Wi-Fi, but we're looking at other types of radio interfaces that allow us to transmit data in real-time over long distances and through buildings and trees, et cetera. And it's a network that is basically self-healing. And we have a number of these nodes set up across the campus. Here we go, smart agriculture, smart water, smart environment, and smart security. This last one being – the smart security one being essentially the ability through the network to turn things on and off or open and shut gates, which, of course, is really important from an agricultural perspective. The agricultural one looks at – well, in fact, I think my next slide looks at the types of data that you can gather from these sensors when it's installed. And we have all of these nodes and most of these sensors around the place. So we're collecting a lot of biophysical raw data in real-time, which we store in the cloud and have a dashboard to essentially access. The idea is that this is done through open source so that at the end of the day, people can have access to this. So turn may be interested in it. Other people may be interested in using it, remembering it's subtropical information, particularly for people who are interested in the agricultural space. But we're looking to certainly make that available at the moment through Edurome to people who are involved in Edurome rather than anything else. We've tried it on an open network and that we found that it had a risk of the university being hacked through our system. So we closed it down from worldwide access to Edurome access. So lots of data, biophysical data coming in. And here's a diagram that tells you that we've got data coming in from these various nodes into our cloud. And then we're using that data within the educational environment now to develop various, essentially data uses. So we've got research projects in here, but we've got farm operation that is something that we want to do because we'd like this to be developed far more into the agricultural farm space. And of course our education component. The type of data you can get, you can modify all of this through our data dashboard or you can look at a QR code and get through to it. So that's the start. But when we have this large amount of biophysical data, what else do we need and what for? And of course we want the data for things like partial monitoring and management, animal monitoring and management, crop monitoring and management and education and more data. We want to get more data. So drones provide that or satellite data or remote sensing data provide that capability of putting aerial data on top of our, are essentially ground and ground data collection system. And if you go back to the future, or we're going back to the future in this because remote sensing has been around in agriculture since the early 1970s and the problem for ag in relation to remote sensing were two main data. One was data, cost of acquisition and processing, the revisit frequency, the things that Guru has talked about actually and also a lack of skills available in the agricultural sector for this type of data to be used efficiently, well in a cost effective manner and agriculture people or people in agriculture can be very easily turned off technology. What is the point of technology if it doesn't deliver me a cost benefit in terms of what I'm doing and if it costs me a lot and it goes wrong and I can't do it when I want to do it then it's not worth it. So we needed to do something about this lack of skills and what we've done here is look at developing the special skills that are needed to understand spatial variability in agricultural remote sensing. We've set up an agricultural remote sensing lab here at the campus and what we're doing there is we're trying to integrate students from different disciplines across the academic world. So engineering students come down and work on agricultural projects in relation to things like building drones and understanding what drones can be used for. Building sensors and understanding what those sensors can be used for in agriculture monitoring in particular but that does involve of course environmental monitoring so our wastewater management site is probably one of our most recorded biophysical data sites and we get them to do hands on work so they understand the issues and the risks. We want to use them because they're cheap platform to carry high resolution sensors. You can see here a spatial variability in a paddock at ground level you can see a bit but when you look over here at an aerial photo of that paddock the variability is huge and that means dollars to a farmer. So we want to collect that data so we can optimize production efficiency and quality and we then also want to minimize risk and environmental impact. So drones from the perspective of an agricultural person could enable us to do better in the smart food production game almost certainly enables us to do better in the smart environmental management game and for us as educators smart skills development because it's lots of skills involved and it's fun to do and one of the things we have in the Australian sector actually is worldwide. If difficulty in getting students into the agricultural industry sector and I'm meaning across the board not just grains or horticulture but across the board. So at UQ we've got five DJ Phantoms that we use for teaching we've got four bespoke quadcopters we've got three bespoke hexacopters and ten mini agricultural drones which are the ones nowadays because these are the ones that I've talked about the ten mini ag drones are the ones that we use and I'll just talk a little bit about them in the next few minutes they're easy to fly and there is an article out in 2015 about why we're doing this and what we're doing. The design and build principles around this business of getting students of all disciplines to understand what they're doing we go through a design process trial and error using open software to look at designing these things we purchase the parts for them they build them they test fly or they learn to fly learn to solder things like this undertake the project and write a report. So this is a classic problem based learning or active learning scheme that we can make get students to do. Here are the types of drones that we've investigated and used over time you can see there's a certain date range here and as much as anything else this small mini UQ mini ag drone we designed and built because we want to fly under the two kilo limit of CASA because if we had to get every student certified we couldn't do this. So this little mini ag drone which I'll talk about in a bit more detail a little bit down the track is the one that we're now flying and this with a camera that looks at getting a normalised difference vegetation index image is less than one kilo. So the first student project was in 2013 where the student literally took a DJI Phantom with a little normal red, green, blue visual camera back to Fiji. This was his school in 2003. This is a Google map. This is his school in 2013 which is the image that you can see outlined here and he literally calculated the difference in mango clearing basically so he looked at change in mango. So this was the first project that we looked at then we started getting a little bit more creative. We developed a drone that could carry a multi-spectral camera and we started looking at Prickly Acacia which is a serious environmental weed up in central Queensland. Big problem from an agricultural perspective because it shuts out cattle from using the pasture underneath and we compared some of that data to satellite data and found of course that these drones creating a better resolution on the ground gave us more data. This beneficial bug drone, a hexacopter weighing around about 2KG and more when we've got this white thing underneath it designed by an agricultural science student which is now going to the has been invited to be put on display at the Science and Technology Museum in London for the next five years in their display on innovations in agriculture over time. So this is actually an old innovation now. This is about 2013, 2014 but it is an instance of when people started looking at using drones. So this is a box full of beneficial bugs which the farming industry, the farm group that we worked with wanted to see if they could develop, drop beneficial bugs, these bits coming out into the middle of a paddock instead of as they customarily do, drive around the outside and so this was a very successful project with industry involvement that then started really introducing this sort of technology from the education, from the classroom into the actual agricultural space. So that was great. The net drone was another one where we were asked to fly under hail netting for one of the largest seedling producers in Australia and here is the netting. You can see it's a lovely picture if you appreciate this sort of color variation. The drone is flying here and it was called the net drone project. We were looking to see which one of these seedlings in 250,000 small tube plants were germinating or not. And here you could see this sort of resolution that you could pick up from a standard multi-spectral camera. You could see the two-coddoledon stage here and you could introduce students to the idea that you are looking at three different bands, so a little bit of physics, you're looking a little bit of spatial variation because you can see where in this one where my cursor is where things haven't grown and then when you go up here at two-coddoledon stage you get that interest for a student as much as they can actually see a plant germinating and they can see where it has not. And what you do is you get them to calculate what the cost is to both the supplier and the buyer when they get a tray of seedlings and something hasn't germinated. So that links it to the economics of the producer. Here is our mini-ag drone. We've been flying it now for a couple of years. We've used it at the beginning of this year for 35 students to do some projects. It's based on Raspberry Pi technology. We've upgraded it because we found that the GPS wasn't crash-hot but in this sort of thing I've got 10 drones which might cost me 100 bucks each to repair but that means that students can crash them and that's good for a student to be able to do that and over here you can see them flying on campus. Sort of data that you can get out of this quite basic no IR or an NDVI data camera is you can pick up this information quite nicely at an operational scale. So again I'm guess I'm not at that. I'm not trying to get these students to be as knowledgeable as we might think from a research perspective but an undergraduate level 3 agricultural science student they go out with this knowledge and will then be able to develop their skills in the real world after that. We are developing various sensors that can fit onto this small camera so the most recent one is a multi-spectral sensor. And multiplexed Raspberry Pi cameras that enable us to choose very high grade research filters of specific wavelengths when we're wanting to look at something and this particular camera at the moment is being used on top of one of our IOT multi-sensor mesh network to collect aerial data of the biomass of that paddock across which of some 50 hectares I believe that has a multi-sensor mesh network on it. So this camera is now working so this was when it was being developed. You can see how it's been set up for little Raspberry Pi cameras and we use Python to make sure everything goes well. This is a drone that is no longer flying but it was an attempt at us delivering a drone that could pick up the RFID tags in a cow's ear from a distance and it works at about 20 meters but you know we're talking a big drone now with a big antenna. And whilst it worked from an operational perspective and from the idea of trying to keep things down to a level where we're not flying above CASA's regulations this became essentially unusable and although I have an engineering student who wishes to try and reduce this in size this is something that we're not at the moment moving forward to but it is an idea it's an example of some of the ideas that come when you start talking to the industry players and asking them what they want. They wanted something like this because normally you have to stand beside the cow or the cow has to go through some cattle gates and literally swipe it with a hand swipe. You've got a thousand head of cattle that's a really difficult, really time consuming thing to do so this was about trying to improve the efficiency of being able to look at these cows in the paddock and the sort of data we could get from it was really quite good. The sorts of types of research and learning involved is electronics and avionics which is not normal in an agricultural sense but it's really useful if you're going to get into this side of spatial variability analysis, the design and build side of things. We look at programming, physics, maths and chemistry of course because we're looking at the electromagnetic spectrum, things like spectral indices and crop growth indices and chemicals and plants so if you're trying to engage a student in learning about maths which they really don't want to do you bring in something like this and it changes their perception of both learning about the maths and then how it can be used in real life when it would appear to be from a classroom perspective completely dry and equation driven. Plant physiology, animal welfare and food traceability are all issues around management that we can also talk about when we use these tools. Most recently we've had these sorts of projects going on, drone and sensor development for crop monitoring and diseases, computer vision development for monitoring pests basically but we could be doing it in the piggery or we could be doing it out in the environment for say wildlife people. We've got a master of engineering student looking at developing a robotic arm for capturing these small drones out of the sky for autonomous recharging. Again, it's an academic project probably not to be used operationally because of the issue of autonomous flight but very useful for the master engineering student to understand why we want that from an agricultural perspective so we want to increase obviously the reach of our drones flying and then we've had people developing automated underwater cameras to monitor phytoplankton which we can then link to our IoT and our aerial imagery of the laser flakes. Obviously the IoT has applicability elsewhere that may not involve drones like in the equine health area and the development of this visualization dashboard for our monitoring purposes and for staff to access and as I say anyone on EDUROAM to access some of this data that we're collecting and as I say this is a project the RFID monitoring of cattle is something that we're still sort of looking at but the idea is to reduce it in size as the electronics become smaller and we don't have to have such a big antenna involved which means a bigger drone and more weight etc and start starting to contribute in cattle regulations. So we're looking at a whole range of things both from an education and research perspective but mainly the research is about getting is about interesting students in learning about this stuff which I think is going to be key down the track in terms of capability development for our industry both agriculture and environment or managed landscapes because without these kids coming in we're going to run out of people in terms of as they get older they're not going to be able to continue to do this. So that's my presentation and I hope you enjoyed it. Thank you very much. Hi Kim, thank you. And I will move on to the next presenter Tim Brown. Thank you for having me on that was a very interesting talk Kim. I'm going to be talking about our work at the National Arboretum in Canberra using drones to make 3D models of the arboretum growing. I'm the director of the Australian Plant Phenomics Facility at ANU and a lot of this work was done when I was working for the ARC Center of Plant Energy Biology as a research fellow and I should first off say thanks to all the people listed there that have been essential to this project. I wasn't quite sure what level of understanding people would have on this. Drones are part of a as Kim pointed out they're part of a key emerging tool set for next gen monitoring of the environment and these make up sensors, high resolution imaging, LiDAR and a whole bunch of other things that let us measure the environment in ways we never could before. And since I've been working in the Phenomics field really we're interested in how you monitor the environment along with genetics so you can measure the phenotype because those are the things together that define how ecosystems develop and in a crop context the phenotype is essentially what we want for increasing food yields. So a bit of background on UAVs I think a bit of this has been covered so I'll go through it quite quickly but you basically have a lot of choices if you're starting a drone program from the sort of low cost ex rotors like these quad and ex rotors like the DJI Phantom. These things are changing so fast so I used to give this talk and say that drones were expensive and confusing and then I said that they were cheap but still a bit confusing and now they're just cheap and easy to use. The DJI Mavic Pro came out this year it's a very tiny drone but it still has a 12 megapixel camera on it you can throw it in a backpack and you can still do pretty decent monitoring with it at the low end of the things. These sorts of drones typically cost in the one to two thousand dollar range although the bigger ones like this one here cost more in the eight thousand dollar range and you get something like 15 to 20 minutes flight time and up to 10 kilogram payload for the really big ones. You can also use fixed winged drones so these give you a longer flight time something to 30 to 60 minute range but you do have to have takeoff and landing issues and they can have higher payloads but they do tip to fly faster which impacts how you can monitor with them. A new system that's being developed by people that we work with at Pro UAV is this vertical takeoff and landing system and we really think this is going to be an amazing tool because it gives you both the benefits of a fixed wing and some sort of a copter because with the helicopter part of it it can take off straight so it doesn't need a landing zone or runway and if something goes wrong and the gas motors fail it will catch itself and land so you can carry quite a heavy payload and you can put really expensive equipment on it because it's quite robust. On the camera side as Kim pointed out you have a lot of options ranging from RGB cameras the on-born cameras on the Phantom work quite well all the way up to DSLRs but you do need to watch about what cameras are using and isolate them from vibration or you get rolling shutter issues where the camera is trying to take a photo but the sensor is not writing all the data at once and so when there's vibration you end up with bad data. The next step up unless you're building your one yourself like Kim pointed out is the multi-spectral cameras that can get you NDVI and the main thing that people use now are either the Micasense Sequoia or RedEdge and those are in the $3,000 to $5,000 range. There are a lot of other sensors so you can put hyperspectral cameras on your drones or thermal and you can see you can get bands in the 400 to 1300 nanometer range but you need a big quadcopter because they're heavy, they're very expensive and the data is quite hard to process so there really is a range of stuff available from really easy to use up to quite challenging depending on what output you need. The typical outputs you get from 3D reconstruction software which is a lot of what I'll be talking about are ortho mosaic images so these are essentially satellite layers that you can put into Google Maps or Mapbox and then also DMs and GeoTiffs and some of the software can actually give you somewhat classified outputs or remove the ground so you only get trees can provide you with RGB multi-spectral or hyperspectral indices and also 3D point clouds and then you can make a 3D model of your environment as well if that's what you're interested in. There are a lot of options on the 3D reconstruction software front and I'll just go through a few, it's important to know that it requires a pretty beefy PC to run these things so you're looking at probably 1500 AUD to get a PC because you need a fair bit of RAM, you need a pretty good processor and then you need a graphics card. PIX40 Mapper Pro which is the one that we've been using because it was about the only one available when we started working, it ranges from $2000 to about $9000 depending on what license you get and you have to pay about $1000 a year for support and also as a challenge that you can only run on Windows so if you want to run it on a server or on the Amazon cloud for much bigger data processing and for automating stuff you have to get the Pro license. The other one people use a lot is Aggie Soft Photo Scan, I haven't used it personally but it's been well recommended. The cheap version if you just want 3D models is about $60 US and about $550 academic and then pricing goes up from there and those also will run on the cloud if you buy the Pro version. Mosaic Mill is another software package, it's the last quote I got from them was about 4500 euros, it comes in a lot of flavors but I haven't used it. There's a free software package called VisualSFM which is FSM which seems to work pretty well because it's free and somewhat open source software, it doesn't have all the bells and whistles the expensive ones do but if you have more time than you do money for your project it's probably a good thing to explore. There are also a lot of online options and these are great for testing so if you either just do occasional flights and don't want to invest in the software in the PC or you want really fast processing or you're preparing a grant proposal so you just need to get some initial data websites like these ones let you just upload your images and give you a point cloud and other data quite quickly. A disclaimer this is not a complete list and all of these prices change rapidly and frequently so check the vendors for pricing and don't take my word for it, there's plenty of other information out there but you see from these software you can take a flight like you see on the left side here and then you get a 3D reconstruction on the forest and I'll talk more about that in a minute. So the site where we've been flying drones is at the National Arboretum in Canberra and this is a really great site because it's just 5 kilometers from ANU and has fast Wi-Fi. The last site I worked at was in southern Utah and it took about 5 hours to drive each direction so when something broke it was a good day and a half just to get there and back and having a field site where you have Wi-Fi access from your desk and you can drive out there in 5 minutes really makes it easy to test new and emerging technologies until they're ready for deploying elsewhere. And also the forest was only planted about 7 years ago so we have an opportunity to monitor this forest and build a 3 dimensional model of the entire arboretum growing into the future which has basically never been possible before. So we initially at the ANU research site we installed a 20 node wireless mesh sensor network, camber weather stations for baseline data, some gigapixel cameras, these are cameras that take hundreds or thousands of high resolution images that you squish together to get a multi-billion resolution picture. We've done some LiDAR scans both on the ground from various sorts and then we've done near monthly UAV flights. And once you sequence the trees out there because tree sequencing is getting down to you know the 10 to 20 dollar per tree range you can really measure phenotype environment and genetics in a way that gives you an amazing density of data about a space that we were never able to access before. So here's camber on the southeast side of Australia, the arboretum is over on the west side of camber, south of Black Mountain if you've been there. This is our main field area, this is a picture of what it looks like from one of the cameras up on the hill, so this is a 500 megapixel image that we generate every hour and you can zoom all the way into that image to see the forest out on the far side there. This is what that forest looks like when you fly over it with a drone and then from the drone data you can bring that into PIX4D as I'll show in a second and get a three dimensional model of the trees. So the drone monitoring program, the goal was to test and develop a time series drone monitoring program so we could get 3D models of the trees growing. We get time lapse georectified image layers, 3D point cloud and then some phenotypes we can measure like tree height, area is measured by top down pixels and color data over time. And this forest we're studying is, it's 12 forests, a spotted gum and Iron Bark was planted about 2012 and it's 4 hectares so it's really a perfect size for drone monitoring because you can fly the whole site in about 15 minutes it's not too hard to process and it's very amenable to that sort of monitoring. We've really come a long way on this so in April 2013 I flew the first drone over it which was a cell phone duct tape to a drone that I'd made at home and now we're using a really nice solid Matress Pro by DJI with Darrell from Pro UAV who's been flying it for us and really good cameras and so we're really getting a lot of solid data but just in the last 4 years, 5 years this technology has changed so much. The typical workflow in this is the PIX4D is a software that we use, it pulls in all the images you can see here that were taken from the drone and then you can see on the left all those green lines going down are key points that is detected in all the images and there's really a lot of black magic that happens behind the scenes in these software packages and you can see on the right those are some little tiny piece of ground that the software has detected as being the same in about 30 different photos and it uses that to calculate the actual position of each photo relative to the ground. Those are called control points. From that it creates the 3D point cloud as you can see here and that gives you this sort of ghostly 3D model of your entire forest. You can see that there's some data missing from the bottom because the trees are the drones are just looking down from the top and can't see around the edges of the trees. The total processing time to do this is very hardware dependent so it ranges from like 3 to 12 hours but it allows you to go from pictures to point clouds to 3 dimensional models of the trees you can see here. It's important to realize how groundbreaking this technology is because previously if you wanted to measure the location and height of every tree it would have taken you an incredibly long time. We have trees in this forest and we can do this now in about a morning but there are still challenges as Ger was pointing out with working with and serving up the data. We implemented a point cloud viewer called Poetry on our website, traitcapture.org. You can go there and see some of the point clouds that we have online. We wrote a software package called forestutils that runs on Python that lets us pull out the locations and height of every tree and the point cloud data associated with that tree. So this assumes an open canopy. If you have a closed canopy things are a bit harder unless you have GPS locations for your trees but now once we have the tree locations when the canopy closes we can keep tracking them and it outputs tree height top down area location RGB colors and a point count which is a measure of how many points were generated for that tree. It also spits out a CSV file that you can stick on Google maps or anywhere you want that has all of your tree locations. So that is the program we ran. We probably have flown 30 flights over the last since mid 2015. I want to talk a little bit about data management because this is really crucial and if you are planning to do anything more than just occasional flights you really need to come up with a good data management plan and you want to do this before you start your surveys. It is important to consider the entire workflow because it is not just who captures it or what happens to it but the entire process. If you think of who does the surveys if you have more than one company or more than one person the data has to get to you somehow. It has to get processed. You have to figure out where it goes on your computer and track it. Until people like Tern have made us nice tools for having our data go seamlessly online you have to manage all of this stuff yourself until it gets to the point where you publish it. You have to figure out smart ways for naming things and you may get a data set and then add to it and then add to it again and then process it and you need some sort of workflow that tracks how that is taking place and doing that across thousands of images or hundreds of flights can really be a challenge. And if people upload data you need to make sure that they have told you how much data they have so that you can have all of it so you don't spend a week processing their data and then find out that they hadn't finished uploading it and you needed to add another hundred images and run the entire thing again. Often you run into problems like we stole all of our data on a large data server at the research school of biology but we process it on a computer that is local and has an SSD so we have to move the data gets uploaded into one folder by whoever took it gets moved to another folder which is the storage folder and then gets copied to this computer to process and then has to be copied back with the new data as well and so that makes it quite hard to track things over time and if experiments fail or new data comes in you need to have a workflow for how you know where things are and what the status of them is. So it's really best to enforce rigorous note taking even just having people whatever tech is running the project writing down what they are doing as they do it can be really happy and handy. Also shared Google docs and notepad plus plus so you can put just files within each folder can be really useful. So here's an example of the naming structure that we settled on and basically the idea was to make when someone looks in a folder to make it reasonably human usable so we have the year location site who captured it and the status in this case it was the national forest and national arboretum the ANU forest plot actually this was pro UAV that captured that one and we wrote that it was done and uploaded. So this seems like a great idea but of course if you have any sort of nested folders like you might want to name your dataset national arboretum the file names get rapidly too long for windows and your entire process fails. So this makes it a challenge because you need metadata but you have to have someone who actually is maintaining it and as I said shared Google docs are good but this isn't really a solve problem and also you know everyone always ends up with files like this because before we implemented a data management plan things were just going on to my laptop and then getting copied into random places. I think a lot of people get flummoxed by data management because it's really hard but it's not easy for anyone and it's easy to think that you don't know what you're doing but I'm not sure that anybody really does and you won't get things right the first time and you have to start with a plan and keep working at it and just acknowledge that it's not going to work the way you expected as soon as you start to implement it and then you need to go back and change it because in reality our data management typically looks something like that and we want to move it more towards the vision but it isn't actually there but you need to address these problems because they end up making your data unusable when we have these large scale huge time series datasets. Some other challenges with processing drone data are that these new kinds of three dimensional data so for things like NDVI or some of the metrics that Kim was mentioning that there's a lot of known information about that because people have been working with that sort of data for a long time but a lot of data like a three dimensional point cloud of a tree it's hard to know how you tie data values like that to biologically meaningful things and it's also challenging getting back to what Gerr was talking about with the provenance of the data and tracking what's been processed. You can process the same project in three different versions of PIC40 and get three at different outputs and there's also about a hundred different ways you can vary the settings so at one point recently we decided to test every single setting we could think of in PIC40 and you can see an output from that table here and using exactly the same images you can get a stitching time ranging from five minutes to 55 minutes and point cloud sizes ranging from 44 million points down to 11 million points and it's easy to think for example that more points means more data but if you happen to be taking pictures on a windy day and your tree is moving around you probably want your data doesn't have the resolution of a leaf it just has the resolution of the height and structure of a tree so it may be that either the less points is actually a better measurement of that tree volume or somewhere in between but it's really hard to ground truth these things because there isn't any way to go out and measure that volume of the tree anywhere else. It's also important to choose the right tool for the job so drones are really great people are using them for good reason with lots of things but they're best suited for smaller areas like maybe less than 20 hectares and a good example of this is that because we've been flying the arboretum so much we would all wanted to do a full arboretum survey so that we could get a 3D point cloud and model the entire arboretum and measure the height of every tree in the arboretum I think it's about 35,000 trees that they have out there so we finally got the funding to do this we got Darryl and to fly it and it ended up being a huge project it took many weeks of planning there were four or five fly days required you have to have multiple staff on site because when you're flying an area that large you can't fly over people so it's easy to follow CASA regulations when you're just a remote forest but when you're trying to find a fly over a 250 hectare area that's open to the public it becomes much harder. We ended up with 8,800 RGB images and more than 12,000 images from the Sequoia it ultimately took about two months of manual processing because we had to break everything into smaller subsets because the full cloud couldn't run on any machines. None of the online folks can handle more than about 500 images so we couldn't just throw it at one of the cloud services and we ended up with a point cloud that had 584 million data points. So it is important to consider what workflow you will use so this is something we know we could probably do this once a year max but if you add up the time cost of it it becomes prohibitively expensive to do this at this point with the technology available and for something like that it might be better to fly a plane over the Arboretum for instance. It's also if you're thinking of setting up a monitoring plan you need to consider weather and distance to the side of accessibility and time of day because if you want to fly all of your plots at the same time of day say around noon you have to drive between sites you can't do that on the same day and sometimes it turns out that just putting a camera in so you can get consistent data even though it's not as maybe high resolution might be a better option or using satellite data. And again for example a lot of the local agricultural monitoring it turns out it's cheaper to do it by helicopter because you might want to fly five flights, fly five sites in the same day and with a helicopter you can do that in about an hour it might cost you a couple thousand dollars but that's cheaper than trying to drive around to five sites over the course of a week and fly a drone and you can put a much heavier payload on a helicopter. So this is the point cloud we got out of the Arboretum again if you go to trade capture you can go around and explore it, it looks pretty good but you can see there are some artifacts where the stitches didn't align up perfectly because we were doing how to do it in pieces. So again I'm choosing the right tool for the job. Larger scale surveys are not necessarily best to do with a drone and there was an interesting article from Frontiers in the Ecology of the Environment last year where they looked at doing UAV surveys in Alaska and it turned out that it was $1,700 per site to use a UAS but only $400 per site to use a plane mainly because the sites were quite far apart. So it's an important consideration. Now in the future I think what we all want is self-driving drone swarms that fly over our forest or field daily. We have all this data upload and processed in the cloud and get near real-time analytics to the farmer or the end user and I've been wanting someone to release something like this for years and finally this year a country called X-Aircraft in China and a group in Sydney called Revolution Ag are starting to release this system. So here's an agriculture UAV it can carry pesticides or water or nutrients, fertilizer and they can fly themselves. So of course there are regulatory issues around this but we're getting close to the point where we can have a swarm of drones that's just monitoring our sites continuously. And in the future as micro drones get better and higher quality they may be more feasible and safer particularly for field monitoring where maybe there aren't going to be people there because you can imagine a small study you pay $10,000 and get 500 of these drones and they take off every day and fly around your forest and come back. They're all solar powered. They live on the tower somewhere. That would be a really amazing processing solution to give you continuous three-dimensional data of your research site. So some thoughts on developing a drone program. Drones and sensors are a crucial component for field monitoring and phenyl typing but it is hard to do. There's a lot of technology available so you want to focus on the deliverables. Who are your stakeholders or customers and what do they need to know? What would be actionable information for them? It's easy to say oh yeah let's just get a drone and then realize afterwards that you're not you haven't really figured out what kind of data you're going to deliver. And then working back from that you can figure out how big the area is, what the full cost scene is and so on. And then also you want to start small and get your pipeline working. So you don't want to start off flying the 250 hectares of the arboretum. You want to have done the smaller forest first. You want to make sure the UAV you buy is the correct one and whether or not it might just be cheaper to subcontract. And when you process the data you need to decide if you're going to outsource it or do it yourself. And if you need to develop novel tools are there groups already doing this stuff and how can you share this with others so that we all have good tools to use for processing certain sorts of data. And again don't underestimate how hard data management is and have a plan in place at the start. I think we're running out of time so I'll run through this really quickly. I just want to say that all of this data is incredibly, it's new and it's incredibly hard to manage because it's three dimensions and it's got multiple layers and time series and we don't have tools for this that we used to, like the tools that we normally use for data visualization and management are not usable for the tool sets that we have now. So we need a sort of MATLAB, Excel or GIS for time series three dimensional hyper spectral data and this doesn't exist yet. So there's a couple groups at NCI that are developing point cloud viewers. We're working with the VizLab to make a time lapse virtual reality and Windows based point cloud viewer. And if you search for Adam Steer at NCI he's working on a NCI backed one which would be really great because then groups like Turn and us could dump all of our data in the same spot on NCI and we'd have these real time tools for pulling point clouds out on the fly. And just the last thing I'll go through this quick because I know we're out of time. One thing that I wanted to do once I started getting three dimensional data is be able to visualize all the sensor and point cloud data on the landscape where it was collected because that's a way to me that helps you really see the data in the context of where it was collected and the place that you're monitoring. So I collaborated with some students in the computer science department and we made a virtual reality three dimensional model of the National Arboretum using all the drone data as well as our sensor data. Essentially what you can do is you can take three dimensional modeling software developed by Hollywood and use it to reproduce your landscape but rather than using whatever data they're using for a movie you can use your data from your drone flights to generate the three dimensional models. And so this is an example of the project for the National Arboretum here the drone flight data is the three dimensional force that we get out of it. This is the virtual reality version of it where we've taken the digital elevation model to generate the landscape and then put the trees in at the locations that the drone data measured. When you interact with the trees they show you their metadata so in this case we're showing height and area of each tree and then you can also map onto the landscape and play back in time series the different data types that we're collecting with the mesh sensors. So this is a really great tool for pulling all of these dense data layers together and visualize them in one spot to help us make sense of this incredibly complex data. And I think we should all consider this is just the beginning right so when I was a kid I used to play Atari and this is what it looked like and now this is you know the same dragon games that my kids play. And we're at the Atari stage in VR and in our ability to measure the world continuously in 3D and in 10 years VR and AR are indistinguishable from reality so the question is what do we do with these tools and how do we create the next generation of interfaces that facilitate ecosystem research and how we build monitoring and monitoring programs that make best use of all these data types so we can really model our environment in three dimensions and solve the grand challenges of this century. There are lots of people to thank but I think I'm out of time so thanks. Excellent presentation team thank you for joining us and look forward to see you again in the future.
Magala-Party from the Australian National Data second one in this webinar series. Today's drone applications in Australian research space, of the recent developments about the research Tim Brown, director Australian plant phenomena talking about the making drone data open for scientific research. So this is basically the done at UQ is look at how we can incorporate subtropical information, particularly for people students of all disciplines to understand what Google map. This is his school in 2013 which is our IoT and our aerial imagery of the laser the National Arboretum in Canberra using drones people would have on this. Drones are part of a both on the ground from various sorts and then pictures to point clouds to 3 dimensional models it. This is the virtual reality version of it