Tensor Processing and Machine Learning for Signal Processing
O’Shea, Tim
This talk will discuss the applicability and potential of tensor processingmethods for signal processing algorithm implementation. It will consider bothmachine learning and non-machine learning signal processing applications anddiscuss how they can be efficiently implemented with TensorFlow and usedquickly in conjunction with GNU Radio.
 Okay. Hey, so I'm Tim O'Shea. I'm from Virginia Tech. I'm just a researcher there. And I wanted to talk a bit about the possibilities for tensor signal processing and then kind of bleed into, how that bleeds into machine learning as well. So just to set the background, I'm sure a lot of you have been watching this, but what's been going on in machine learning and kind of the resurgence of neural networks has been pretty awesome to watch. And this has primarily been in the computer vision area. And so there's a lot of people in the self-driving car area as well as like facial and object recognition areas and natural language processing where we've made lots and lots of advances in recent time in driving down error rates on translation and object recognition and all these things. And if you look at it, so neural networks from a computational standpoint are really interesting because they have this fairly well-defined compute structure that constitutes a network, right? And so if you look at kind of the calculations for a neural network, you have this kind of, it's really basically a big multiply accumulate for each layer. So you have these inputs that are multiplied with a series of weights that are summed and then run through some non-linearity like a sigmoid function. And that's basically one layer in a neural network. Now if you look at these so-called convolutional layers, you no longer have a separate weight on each input, but you have these weights that are tied in a way such that when you slide a patch across an input, you replicate the same weights at different shifts, right? So that's really the only difference. And then if you look at some of the bigger models, so this is from like a year and a half or ago, one of Google's vision networks for doing object recognition, you think of that transfer function for a layer, you know, that's one box in this network, right? So we now have this network which is, you know, 40 or 50 layers. Some of the ones this year were over 100 layers deep in the complexity that they're doing to do this object recognition. So there's been a really, really strong software ecosystem to support machine learning for computer vision. And it's really kind of starting to centralize around this kind of tensor programming model. And this is really, you know, interesting because you're building these big tensor graph expressions and then, you know, as a big abstract data flow representation and then you kind of map it down to hardware to run. So two of these architectures are kind of theano and tensorflow. Theano was around kind of first and kind of pioneered this and then Google like secretly rewrote it internally and then eventually released it to the world and now they're both very popular but the whole idea here is these are basically libraries that allow you to write big numpy expressions essentially where you have many operations that are kind of forming a data flow graph and then take it and the idea is you build this graph expression and then you go through the back end and you go out to any one of a number of targets and so right now we go to basically GCC and CUDA as the back end but there's a lot of plans underway at Google to do this thing called XLA which is like their next generation tensor back end and they're doing a lot of work to sync this up with LLVM so they can actually just use kind of LLVM abstract representation to pass it in. And then there's a lot of rumors about potential support on like the QDSPs that Qualcomm builds for base bands as well as some of the Xilinx chips. There's been a lot of talking about Xilinx potentially supporting this as a co-generator so it's really cool. We may have finally a language that we can target to a lot of different kind of platforms from this kind of way to express things. So this is used heavily for neural networks but it's not really neural network specific so if you look at this is basically a simple expression. We start by defining these variables like A and B and we can define operations on them like adding or multiplying and then eventually we go into our session and we evaluate these expressions and so this is basically building up your big, in this case a very small kind of data flow graph and then this is going to be compiling and running your graph. So you can build much bigger things in this so I talked a little bit before about GR TensorFlow and the whole idea is really at the core of this, this is just kind of like a kernel work call so we can wedge it right into a GR radio block and use it to offload these tensor operations from within a work function. And so this is something added a couple months ago and the whole idea here is can we take a sync interpolator of a signal and offload that onto GPU or onto whatever using this kind of tensor language and so this is basically the definition of a sync interpolation. And so you're going through here and up at the top you're defining the graph through this series of castings, you know, tiling of the input, doing these gather operations which are basically index lookups into vectors and then doing a reduced sum which is basically just a big multiply accumulate. And so once we've kind of issued these commands we've built this kind of dependence graph for the operation and now whenever we run it we could change the back end to either the CPU or the GPU or, you know, some of the targets that are probably coming soon. And so that's kind of just an example. I think there's not many people out there using this for pure signal processing like this yet but I think there's a huge potential for it. The tensor languages are pretty expressive for almost any of the commas problems we have and the kernel compilers and optimizers are really quite good. So this is kind of a pretty exciting way to offload onto GPUs and things. All right, so I'm going to switch over to the machine learning side. So that was kind of basic signal processing but I guess my research now is really centered around, okay, how do we take these machine learning ideas and apply them to radio technology and radio communications? And so these are just kind of a couple of the questions you might ask, you know, can we apply it in these ways? And so really I've been looking at, you know, can you learn entirely new communications schemes or can we just learn algorithms to put into our current waveforms as different subsystems? And then kind of in a sensing scenario you may want to know, you know, what's around you in your interference environment like Tom was talking about this morning where you may have EMI emissions from your, you know, your power supply or whatever. You may want your system to be aware of all these emitters around you when it makes decisions about setting up the communications link. And then just how to optimize existing systems. So these are kind of some of the high level goals, right? So this is some of the earlier work from this year and I think we saw a little bit in the GR inspector talk which was awesome. So the whole idea here is, you know, for a long, long time when we looked at doing modulation recognition which is kind of a classical communication sensing task, the approach was generally to do this expert feature approach where you extract a bunch of high order moments or high order cumulants along with maybe statistics on the signal and the phase and the envelope. And here we're kind of, we're trying to go to a whole different route of just let's, instead let's go to feature learning on the raw input signal and just put the INQ in and let it learn a whole new set of its own features. And it's funny, I mean, if you look at what's happened in the image, the computer vision domain, this is exactly, you know, what has happened. So until five or ten years ago, there was a whole discipline of feature engineering in vision which has been largely replaced now with these kind of end-to-end learned features. And so I think a lot of this is going to happen in the comms and the radio space as well. And so this is really kind of showing that for modulation recognition, we can do pretty much the same thing. So we start, there's a data set that's out on the website and, you know, this is basically just using GUNI radio to generate a whole bunch of synthetic signals through a whole bunch of synthetic channels that use the dynamic fading model. And then this is kind of what we used for that experiment. So this is just essentially, you know, this is basically just a small convolutional neural network that we're using to do this classification. And if you look at performance, there's kind of a bunch of the solid ones here, are different variations of a kind of feature learning approach which take in the raw samples. And the dotted ones are basically different classifiers using these expert features like higher order moments to classify. So here you can see, you know, in the best case for both, we get, you know, three or four DB sensitivity improvement with these methods, which is pretty exciting. Three or four DB is a lot in some worlds. So and then, yeah, so, all right, so one of the other cool things you can do is if you take, for instance, if we look at this network, our output is 11 values, right? So these correspond to one per modulation class, you typically use like a one-hot encoding for a classifier. If you go back one layer, you have this feature map of activations between each layer. And generally, this is reducing in size as you go through the network to become a more and more concise representation of the information. So if we go back one layer and we just take these activations here, we actually now have a, basically a compressed representation of the signals from the input in this feature map space. And so what's really cool is we can start now using that compressed representation to just kind of visualize, you know, where are these signals in this kind of abstract modulation space, right? And so now we can start to cluster them and, you know, we have this basis now that is relatively good at separating a bunch of different signal types. And so this is kind of interesting because, you know, if a new modulation pops up, you know, if these features generalize, you know, we hope we'll get a new cluster over here or something that's discernible so that we can start to recognize things online without having to train it for every explicit case kind of ahead of time. All right. So then this is, this goes to some of the work that we did this week, HackFest. And so how many times, I guess, raise your hand if you've ever written a flow graph that's like a usurp source through a head to a vector sync, right? I feel like, okay, we got a handful. So sometimes it feels silly to use Gineer radio when all you want to do is just grab a chunk of samples off a radio. And so that's the whole point of this NumPy UHD wrapper. And so this is just like a super, the thinnest interface you could have to say, you know, tune my radio somewhere and grab some samples from it. And so super simple. And so what we're doing here is this makes it really quick to just take little looks all over the spectrum and generate spectrograms for them. So if we do that, we can start to get all these samples of different bands that are all around us. And so this is just kind of a sampling of a bunch of spectrograms across, you know, common bands. So if we use this in a ConvNet, we can also build a classifier for the different bands. This is not a very good one at the moment, but that's just because the slides were a bit late. But so it's not necessarily that exciting to classify which band you're on, because, you know, we have allocation schemes. This isn't really that useful. But the thing that's really interesting about it is we learn these sets of features for each band that make that band what it is. So like the LTE downlink band is filled with LTE signals that have all the properties and features of an LTE signal in the downlink. And so our network, as it's learning this, it starts to learn all these features within, you know, that comprise that band. And so what we can do is we can now take any random spectrogram that comes in, like these two, and we can look at where in that input spectrogram were there activations that led to thinking it was that class, right? So we can say, you know, what in this image makes us think it's LTE uplink, right? And so if you do that, you can look at and get this localization mask that's used in, you know, in imagery for object segmentation. We can do the same thing in spectrum. So we can now say, well, once we've trained these features, we can take in arbitrary spectrograms and really identify what's going on where and start labeling things. All right, so then kind of the, I think the last kind of little application thing here, and I'm really, this is the thing, if you look at machine learning and comms, this is what I'm, I think, most excited about right now. And the whole idea here is if you think about a comm system at its most basic form, it's really like you have some bits somewhere and at some of the location, you just want a good estimate for those bits, right? That's kind of all it is. You're just taking bits in and trying to reconstruct them somewhere else. And so that's pretty much what an autoencoder does in machine learning, is it takes some input, it goes through a couple different representations of it and it tries to reconstruct the input. And so if we cast the basic communications problem as an autoencoder, we have this case where we go through some input. We form some new encoding of those input bits. We go through some channel impairment which we don't control, this is kind of defined by a wireless environment. And then we go through this decoder representation that somehow transforms the samples we receive back into bits. And really that's the whole process we want to optimize. So we can treat this as just a, an optimization of a reconstruction process where we have this layer in the middle that introduces this randomness that mirrors what a wireless channel does. And so what's really cool here is we can let this thing learn now some completely arbitrary physical layer. We know we don't need PSK or any of these things anymore. And we can actually start getting reasonably good capacities out of these encodings. So we can now, you know, just learn a radio for a specific radio environment, you know, without any expert knowledge of the past 70 years of signal processing, which is pretty crazy. So if we do that, we can start plotting basically BER curves. And these are, these are kind of some old ones that aren't as clean. But you can see that, you know, in some cases we can outperform the performance you would get on a modulation without any error correction, which is really cool because it means the encoders and decoders that we're learning can actually learn a little bit of error correction as well. So you're now, you're basically learning something that does better than just a normal modulation scheme. So there's a, I'll mention it later. So this is just an example of a couple different runs and what, you know, what the resulting learned bases look like in this case. Yeah, and there's many, many things that affect this. So there's right now still a million kind of tuning dimensions that you can look at while training these things and optimizing all over all of those is still kind of a big challenge right now. And right now the other big challenges are this, you know, this works for very small code words. But if you try to scale this to like, you know, 1024 bits or something that LTE might use in an LDPC code, it's much harder to scale it. So that's one of the things you're looking at now is how do you scale this up to like more real situations and over the air. All right, and then lastly, so when we learn that in coder and decoder, the decoder has to learn how to translate from some channel representation to the bits, right? Now if you do that just naively, we get some process that can, it takes a long time to converge, right, because it's having to learn how to estimate the channel, somehow apply those corrections, and then somehow go to a back to a bit information encoding. So there's this notion from vision of attention models. And the whole idea of these attention models is that instead of learning some arbitrary transform like how to equalize, you can simply break the network into learned parameter estimation, kind of an expert transform like equalization, and then our learned, you know, decoder. And so in doing this, you basically tell the system, well, okay, this is how you would apply equalizer taps, or this is how you would apply frequency correction if you knew how far off it was. And it turns out if you do that, this converges like much, much faster, orders of magnitude faster here, and we get significantly better performance as well in the auto encoder case. So it's like we're kind of helping the system along to learn the basic physics of the problem, to reduce the complexity that the machine learning has to fit. And so this is kind of exciting that this works. And so on top of that, there's this whole problem of like, you know, we have a whole discipline of estimation and detection, right? And it's interesting to just look at, you know, can we treat these estimation problems as just machine learning regression problems? And so we started looking at that. And if you look at this as comparing the map estimator for center frequency estimation, with basically just a neural map trained to do regression to generate estimates for the same thing. And if we do that, and this is under a Rayleigh fading channel, so if we do that, we get a slightly tighter distribution from the regression process. And so this speaks to like, why do you want to do this? You know, we have a lot of great analytic models for these, but one of the problems with them is there, many of them are derived under much more simplistic channel assumptions. And so if we can do this with real data and regression, then we can start to model, you know, all kinds of nonlinear effects and all kinds of much more complex channel models than we can in our analytic forms. And so it's really kind of exciting about, you know, where our estimation processes can go in the future with this approach. Right, so ML is affecting a lot of fields in computer very quickly right now, which is pretty exciting. And the area of applying this to the RF radio domain is super young. There's really not much work out there yet at all. And so I think if you're a student right now looking for kind of topics to work on, this is like, you know, super ripe. And I wouldn't point you anywhere else. But obviously I'm a little biased. So there's also, I have a journal article that will be appearing on Archive Monday. And it goes into this autoencoder and the mod rec process much more rigorously. So if you're interested in a more thorough explanation, definitely go and check that out Monday. And there will be a link on radio ML.org. So I'm trying to start kind of turning radio ML.org into somewhat of a community for people interested in this. So, you know, if you would like to contribute tools or data sets or articles or examples or anything to radio ML and start contributing, like please let me know because I would love to have your work there. And yeah, I think that's about it. Thank you. Any questions? Yep. So you mentioned about using TensorFlow for signal processing. So how much, so you gave an example of sync interpolator, how much work do you know that's being out there for what, how do you define these different DSP applications? Yeah, so there's not much out there at all for like radio signal processing yet. I think almost all the work has been done in computer vision. And so there's a lot of tools to do kind of computer vision operations, but very little of that has been applied to radio yet. And I think that's kind of one of the niceties is that the computer vision guys are, you know, paving a lot of the roads for, you know, figuring out the compilers and all the back end and support issues so we can use a lot of the same infrastructure in radio without having to reinvent it. I think Google said they moved like their machine learning for all of their services over to TensorFlow. And they have like a 30 or 40 person team just supporting the TensorFlow product line now. So there's a lot of, you know, stuff we can just take for granted at this point. Yep. I think it was for you, like the demony knowledge of real time computation requirements, latency, scale kind of, you showed comparisons of theoretical performance, you don't worry about like actual runtime performance. Right. So I haven't, there's not been a whole lot done on latency yet, but on computational performance for the ModRec example, I did a op counting comparison to the kind of state-of-the-art kind of classic algorithms. And we, so the initial network was something like nine times more operations than the classic method, right? So but it turns out then when we go in and we start doing certain things within the network, like we use one by one comb filtering on the output of several convaliers, which is a trick. And one by one filtering sounds very silly and I think to most electrical engineers because, you know, why would you do that? But the point of a one by one filter is actually that it's combining in the number of output channels. So you have actually n channels of 2D image coming in and m channels of 2D image going out. So you can use one by one filtering to reduce like that number of channels down. And so by playing tricks like that, we were able to drive the complexity down like maybe 50X or so. And so we're now a factor of about four lower in op counts than the kind of classic approach in that application. So I think like, you know, there's a lot of things that make it attractive, right? So all these multi-core architectures, you know, this is a very parallel thing, right? You can divide up the tensors in width and in depth onto different cores. And one of the other cool things is there was a paper looking at how much representation precision do you need for these networks to work. And because right now this is all done in single precision floating point. But they showed that you really only need four to eight bits of precision throughout the whole network for this to work equally well. And so there's another potential, you know, huge savings in complexity. So yeah, I think ultimately for comms and for sensing, this is potentially drastically lower complexity than our current approaches. Yep. Yeah. So I can't do it justice here. If you'll read the paper on Monday. So the question was about coding and whether we compared to Shannon and so forth. And yes, so the baselines here are, you know, I didn't want to go and put everything that's going to be released Monday yet. But if you go through the paper that's coming out Monday, we have some very, very good baselines where we're looking at comparisons to, you know, PSKs and DPSKs with Hamming code and, you know, basically right up against capacity. And so we're much more robust there and showing, you know, the capacity that you can achieve with this. And you can come very, very close to, you know, the channel limit. Switch over. Okay. All right. The next speaker. Ah. If you go that far, do
 Okay.  Hey, so I'm Tim O'Shea.  I'm from Virginia Tech.  I'm just a researcher there.  And I wanted to talk a bit about the possibilities for tensor signal processing and then kind  of bleed into, how that bleeds into machine learning as well.  So just to set the background, I'm sure a lot of you have been watching this, but what's  been going on in machine learning and kind of the resurgence of neural networks has been  pretty awesome to watch.  And this has primarily been in the computer vision area.  And so there's a lot of people in the self-driving car area as well as like facial and object  recognition areas and natural language processing where we've made lots and lots of advances  in recent time in driving down error rates on translation and object recognition and  all these things.  And if you look at it, so neural networks from a computational standpoint are really  interesting because they have this fairly well-defined compute structure that constitutes  a network, right?  And so if you look at kind of the calculations for a neural network, you have this kind  of, it's really basically a big multiply accumulate for each layer.  So you have these inputs that are multiplied with a series of weights that are summed and  then run through some non-linearity like a sigmoid function.  And that's basically one layer in a neural network.  Now if you look at these so-called convolutional layers, you no longer have a separate weight  on each input, but you have these weights that are tied in a way such that when you  slide a patch across an input, you replicate the same weights at different shifts, right?  So that's really the only difference.  And then if you look at some of the bigger models, so this is from like a year and a  half or ago, one of Google's vision networks for doing object recognition, you think of  that transfer function for a layer, you know, that's one box in this network, right?  So we now have this network which is, you know, 40 or 50 layers.  Some of the ones this year were over 100 layers deep in the complexity that they're doing  to do this object recognition.  So there's been a really, really strong software ecosystem to support machine learning for  computer vision.  And it's really kind of starting to centralize around this kind of tensor programming model.  And this is really, you know, interesting because you're building these big tensor graph  expressions and then, you know, as a big abstract data flow representation and then you kind  of map it down to hardware to run.  So two of these architectures are kind of theano and tensorflow.  Theano was around kind of first and kind of pioneered this and then Google like secretly  rewrote it internally and then eventually released it to the world and now they're both  very popular but the whole idea here is these are basically libraries that allow you to write  big numpy expressions essentially where you have many operations that are kind of forming  a data flow graph and then take it and the idea is you build this graph expression and  then you go through the back end and you go out to any one of a number of targets and  so right now we go to basically GCC and CUDA as the back end but there's a lot of plans  underway at Google to do this thing called XLA which is like their next generation tensor  back end and they're doing a lot of work to sync this up with LLVM so they can actually  just use kind of LLVM abstract representation to pass it in.  And then there's a lot of rumors about potential support on like the QDSPs that Qualcomm builds  for base bands as well as some of the Xilinx chips.  There's been a lot of talking about Xilinx potentially supporting this as a co-generator  so it's really cool.  We may have finally a language that we can target to a lot of different kind of platforms  from this kind of way to express things.  So this is used heavily for neural networks but it's not really neural network specific  so if you look at this is basically a simple expression.  We start by defining these variables like A and B and we can define operations on them  like adding or multiplying and then eventually we go into our session and we evaluate these  expressions and so this is basically building up your big, in this case a very small kind  of data flow graph and then this is going to be compiling and running your graph.  So you can build much bigger things in this so I talked a little bit before about GR  TensorFlow and the whole idea is really at the core of this, this is just kind of like  a kernel work call so we can wedge it right into a GR radio block and use it to offload  these tensor operations from within a work function.  And so this is something added a couple months ago and the whole idea here is can we take  a sync interpolator of a signal and offload that onto GPU or onto whatever using this  kind of tensor language and so this is basically the definition of a sync interpolation.  And so you're going through here and up at the top you're defining the graph through  this series of castings, you know, tiling of the input, doing these gather operations  which are basically index lookups into vectors and then doing a reduced sum which is basically  just a big multiply accumulate.  And so once we've kind of issued these commands we've built this kind of dependence graph  for the operation and now whenever we run it we could change the back end to either  the CPU or the GPU or, you know, some of the targets that are probably coming soon.  And so that's kind of just an example.  I think there's not many people out there using this for pure signal processing like  this yet but I think there's a huge potential for it.  The tensor languages are pretty expressive for almost any of the commas problems we have  and the kernel compilers and optimizers are really quite good.  So this is kind of a pretty exciting way to offload onto GPUs and things.  All right, so I'm going to switch over to the machine learning side.  So that was kind of basic signal processing but I guess my research now is really centered  around, okay, how do we take these machine learning ideas and apply them to radio technology  and radio communications?  And so these are just kind of a couple of the questions you might ask, you know, can  we apply it in these ways?  And so really I've been looking at, you know, can you learn entirely new communications  schemes or can we just learn algorithms to put into our current waveforms as different  subsystems?  And then kind of in a sensing scenario you may want to know, you know, what's around  you in your interference environment like Tom was talking about this morning where you  may have EMI emissions from your, you know, your power supply or whatever.  You may want your system to be aware of all these emitters around you when it makes decisions  about setting up the communications link.  And then just how to optimize existing systems.  So these are kind of some of the high level goals, right?  So this is some of the earlier work from this year and I think we saw a little bit in the  GR inspector talk which was awesome.  So the whole idea here is, you know, for a long, long time when we looked at doing modulation  recognition which is kind of a classical communication sensing task, the approach was generally to  do this expert feature approach where you extract a bunch of high order moments or high  order cumulants along with maybe statistics on the signal and the phase and the envelope.  And here we're kind of, we're trying to go to a whole different route of just let's,  instead let's go to feature learning on the raw input signal and just put the INQ in and  let it learn a whole new set of its own features.  And it's funny, I mean, if you look at what's happened in the image, the computer vision  domain, this is exactly, you know, what has happened.  So until five or ten years ago, there was a whole discipline of feature engineering  in vision which has been largely replaced now with these kind of end-to-end learned features.  And so I think a lot of this is going to happen in the comms and the radio space as well.  And so this is really kind of showing that for modulation recognition, we can do pretty  much the same thing.  So we start, there's a data set that's out on the website and, you know, this is basically  just using GUNI radio to generate a whole bunch of synthetic signals through a whole  bunch of synthetic channels that use the dynamic fading model.  And then this is kind of what we used for that experiment.  So this is just essentially, you know, this is basically just a small convolutional neural  network that we're using to do this classification.  And if you look at performance, there's kind of a bunch of the solid ones here, are different  variations of a kind of feature learning approach which take in the raw samples.  And the dotted ones are basically different classifiers using these expert features like  higher order moments to classify.  So here you can see, you know, in the best case for both, we get, you know, three or  four DB sensitivity improvement with these methods, which is pretty exciting.  Three or four DB is a lot in some worlds.  So and then, yeah, so, all right, so one of the other cool things you can do is if you  take, for instance, if we look at this network, our output is 11 values, right?  So these correspond to one per modulation class, you typically use like a one-hot encoding  for a classifier.  If you go back one layer, you have this feature map of activations between each layer.  And generally, this is reducing in size as you go through the network to become a more  and more concise representation of the information.  So if we go back one layer and we just take these activations here, we actually now have  a, basically a compressed representation of the signals from the input in this feature  map space.  And so what's really cool is we can start now using that compressed representation to  just kind of visualize, you know, where are these signals in this kind of abstract modulation  space, right?  And so now we can start to cluster them and, you know, we have this basis now that is relatively  good at separating a bunch of different signal types.  And so this is kind of interesting because, you know, if a new modulation pops up, you  know, if these features generalize, you know, we hope we'll get a new cluster over here  or something that's discernible so that we can start to recognize things online without  having to train it for every explicit case kind of ahead of time.  All right.  So then this is, this goes to some of the work that we did this week, HackFest.  And so how many times, I guess, raise your hand if you've ever written a flow graph that's  like a usurp source through a head to a vector sync, right?  I feel like, okay, we got a handful.  So sometimes it feels silly to use Gineer radio when all you want to do is just grab  a chunk of samples off a radio.  And so that's the whole point of this NumPy UHD wrapper.  And so this is just like a super, the thinnest interface you could have to say, you know,  tune my radio somewhere and grab some samples from it.  And so super simple.  And so what we're doing here is this makes it really quick to just take little looks  all over the spectrum and generate spectrograms for them.  So if we do that, we can start to get all these samples of different bands that are  all around us.  And so this is just kind of a sampling of a bunch of spectrograms across, you know, common  bands.  So if we use this in a ConvNet, we can also build a classifier for the different bands.  This is not a very good one at the moment, but that's just because the slides were a  bit late.  But so it's not necessarily that exciting to classify which band you're on, because,  you know, we have allocation schemes.  This isn't really that useful.  But the thing that's really interesting about it is we learn these sets of features for  each band that make that band what it is.  So like the LTE downlink band is filled with LTE signals that have all the properties and  features of an LTE signal in the downlink.  And so our network, as it's learning this, it starts to learn all these features within,  you know, that comprise that band.  And so what we can do is we can now take any random spectrogram that comes in, like these  two, and we can look at where in that input spectrogram were there activations that led  to thinking it was that class, right?  So we can say, you know, what in this image makes us think it's LTE uplink, right?  And so if you do that, you can look at and get this localization mask that's used in,  you know, in imagery for object segmentation.  We can do the same thing in spectrum.  So we can now say, well, once we've trained these features, we can take in arbitrary spectrograms  and really identify what's going on where and start labeling things.  All right, so then kind of the, I think the last kind of little application thing here,  and I'm really, this is the thing, if you look at machine learning and comms, this is  what I'm, I think, most excited about right now.  And the whole idea here is if you think about a comm system at its most basic form, it's  really like you have some bits somewhere and at some of the location, you just want a good  estimate for those bits, right?  That's kind of all it is.  You're just taking bits in and trying to reconstruct them somewhere else.  And so that's pretty much what an autoencoder does in machine learning, is it takes some  input, it goes through a couple different representations of it and it tries to reconstruct  the input.  And so if we cast the basic communications problem as an autoencoder, we have this case  where we go through some input.  We form some new encoding of those input bits.  We go through some channel impairment which we don't control, this is kind of defined  by a wireless environment.  And then we go through this decoder representation that somehow transforms the samples we receive  back into bits.  And really that's the whole process we want to optimize.  So we can treat this as just a, an optimization of a reconstruction process where we have  this layer in the middle that introduces this randomness that mirrors what a wireless channel  does.  And so what's really cool here is we can let this thing learn now some completely arbitrary  physical layer.  We know we don't need PSK or any of these things anymore.  And we can actually start getting reasonably good capacities out of these encodings.  So we can now, you know, just learn a radio for a specific radio environment, you know,  without any expert knowledge of the past 70 years of signal processing, which is pretty  crazy.  So if we do that, we can start plotting basically BER curves.  And these are, these are kind of some old ones that aren't as clean.  But you can see that, you know, in some cases we can outperform the performance you would  get on a modulation without any error correction, which is really cool because it means the  encoders and decoders that we're learning can actually learn a little bit of error correction  as well.  So you're now, you're basically learning something that does better than just a normal  modulation scheme.  So there's a, I'll mention it later.  So this is just an example of a couple different runs and what, you know, what the resulting  learned bases look like in this case.  Yeah, and there's many, many things that affect this.  So there's right now still a million kind of tuning dimensions that you can look at  while training these things and optimizing all over all of those is still kind of a big  challenge right now.  And right now the other big challenges are this, you know, this works for very small  code words.  But if you try to scale this to like, you know, 1024 bits or something that LTE might use  in an LDPC code, it's much harder to scale it.  So that's one of the things you're looking at now is how do you scale this up to like  more real situations and over the air.  All right, and then lastly, so when we learn that in coder and decoder, the decoder has  to learn how to translate from some channel representation to the bits, right?  Now if you do that just naively, we get some process that can, it takes a long time to  converge, right, because it's having to learn how to estimate the channel, somehow apply  those corrections, and then somehow go to a back to a bit information encoding.  So there's this notion from vision of attention models.  And the whole idea of these attention models is that instead of learning some arbitrary  transform like how to equalize, you can simply break the network into learned parameter estimation,  kind of an expert transform like equalization, and then our learned, you know, decoder.  And so in doing this, you basically tell the system, well, okay, this is how you would  apply equalizer taps, or this is how you would apply frequency correction if you knew  how far off it was.  And it turns out if you do that, this converges like much, much faster, orders of magnitude  faster here, and we get significantly better performance as well in the auto encoder case.  So it's like we're kind of helping the system along to learn the basic physics of the problem,  to reduce the complexity that the machine learning has to fit.  And so this is kind of exciting that this works.  And so on top of that, there's this whole problem of like, you know, we have a whole  discipline of estimation and detection, right?  And it's interesting to just look at, you know, can we treat these estimation problems  as just machine learning regression problems?  And so we started looking at that.  And if you look at this as comparing the map estimator for center frequency estimation,  with basically just a neural map trained to do regression to generate estimates for the  same thing.  And if we do that, and this is under a Rayleigh fading channel, so if we do that, we get a  slightly tighter distribution from the regression process.  And so this speaks to like, why do you want to do this?  You know, we have a lot of great analytic models for these, but one of the problems with them  is there, many of them are derived under much more simplistic channel assumptions.  And so if we can do this with real data and regression, then we can start to model, you  know, all kinds of nonlinear effects and all kinds of much more complex channel models  than we can in our analytic forms.  And so it's really kind of exciting about, you know, where our estimation processes can  go in the future with this approach.  Right, so ML is affecting a lot of fields in computer very quickly right now, which  is pretty exciting.  And the area of applying this to the RF radio domain is super young.  There's really not much work out there yet at all.  And so I think if you're a student right now looking for kind of topics to work on, this  is like, you know, super ripe.  And I wouldn't point you anywhere else.  But obviously I'm a little biased.  So there's also, I have a journal article that will be appearing on Archive Monday.  And it goes into this autoencoder and the mod rec process much more rigorously.  So if you're interested in a more thorough explanation, definitely go and check that  out Monday.  And there will be a link on radio ML.org.  So I'm trying to start kind of turning radio ML.org into somewhat of a community for people  interested in this.  So, you know, if you would like to contribute tools or data sets or articles or examples  or anything to radio ML and start contributing, like please let me know because I would love  to have your work there.  And yeah, I think that's about it.  Thank you.  Any questions?  Yep.  So you mentioned about using TensorFlow for signal processing.  So how much, so you gave an example of sync interpolator, how much work do you know that's  being out there for what, how do you define these different DSP applications?  Yeah, so there's not much out there at all for like radio signal processing yet.  I think almost all the work has been done in computer vision.  And so there's a lot of tools to do kind of computer vision operations, but very little  of that has been applied to radio yet.  And I think that's kind of one of the niceties is that the computer vision guys are, you  know, paving a lot of the roads for, you know, figuring out the compilers and all the back  end and support issues so we can use a lot of the same infrastructure in radio without  having to reinvent it.  I think Google said they moved like their machine learning for all of their services  over to TensorFlow.  And they have like a 30 or 40 person team just supporting the TensorFlow product line  now.  So there's a lot of, you know, stuff we can just take for granted at this point.  Yep.  I think it was for you, like the demony knowledge of real time computation requirements, latency,  scale kind of, you showed comparisons of theoretical performance, you don't worry about like actual  runtime performance.  Right.  So I haven't, there's not been a whole lot done on latency yet, but on computational  performance for the ModRec example, I did a op counting comparison to the kind of state-of-the-art  kind of classic algorithms.  And we, so the initial network was something like nine times more operations than the classic  method, right?  So but it turns out then when we go in and we start doing certain things within the network,  like we use one by one comb filtering on the output of several convaliers, which is a trick.  And one by one filtering sounds very silly and I think to most electrical engineers because,  you know, why would you do that?  But the point of a one by one filter is actually that it's combining in the number of output  channels.  So you have actually n channels of 2D image coming in and m channels of 2D image going  out.  So you can use one by one filtering to reduce like that number of channels down.  And so by playing tricks like that, we were able to drive the complexity down like maybe  50X or so.  And so we're now a factor of about four lower in op counts than the kind of classic approach  in that application.  So I think like, you know, there's a lot of things that make it attractive, right?  So all these multi-core architectures, you know, this is a very parallel thing, right?  You can divide up the tensors in width and in depth onto different cores.  And one of the other cool things is there was a paper looking at how much representation  precision do you need for these networks to work.  And because right now this is all done in single precision floating point.  But they showed that you really only need four to eight bits of precision throughout  the whole network for this to work equally well.  And so there's another potential, you know, huge savings in complexity.  So yeah, I think ultimately for comms and for sensing, this is potentially drastically  lower complexity than our current approaches.  Yep.  Yeah.  So I can't do it justice here.  If you'll read the paper on Monday.  So the question was about coding and whether we compared to Shannon and so forth.  And yes, so the baselines here are, you know, I didn't want to go and put everything that's  going to be released Monday yet.  But if you go through the paper that's coming out Monday, we have some very, very good baselines  where we're looking at comparisons to, you know, PSKs and DPSKs with Hamming code and,  you know, basically right up against capacity.  And so we're much more robust there and showing, you know, the capacity that you can achieve  with this.  And you can come very, very close to, you know, the channel limit.  Switch over.  Okay.  All right.  The next speaker.  Ah.  If you go that far, do
Tensor_Processing_and_Machine_Learning_for_Signal_Processing_Functional_Algorit_0_131.png Tensor_Processing_and_Machine_Learning_for_Signal_Processing_Functional_Algorit_132_251.png Tensor_Processing_and_Machine_Learning_for_Signal_Processing_Functional_Algorit_252_347.png Tensor_Processing_and_Machine_Learning_for_Signal_Processing_Functional_Algorit_348_611.png Tensor_Processing_and_Machine_Learning_for_Signal_Processing_Functional_Algorit_612_1967.png Tensor_Processing_and_Machine_Learning_for_Signal_Processing_Functional_Algorit_1968_3695.png Tensor_Processing_and_Machine_Learning_for_Signal_Processing_Functional_Algorit_3696_4259.png Tensor_Processing_and_Machine_Learning_for_Signal_Processing_Functional_Algorit_4260_4847.png Tensor_Processing_and_Machine_Learning_for_Signal_Processing_Functional_Algorit_4848_5243.png Tensor_Processing_and_Machine_Learning_for_Signal_Processing_Functional_Algorit_5244_6167.png Tensor_Processing_and_Machine_Learning_for_Signal_Processing_Functional_Algorit_6168_6227.png Tensor_Processing_and_Machine_Learning_for_Signal_Processing_Functional_Algorit_6228_6479.png Tensor_Processing_and_Machine_Learning_for_Signal_Processing_Functional_Algorit_6480_7367.png Tensor_Processing_and_Machine_Learning_for_Signal_Processing_Functional_Algorit_7368_7919.png Tensor_Processing_and_Machine_Learning_for_Signal_Processing_Functional_Algorit_7920_10235.png Tensor_Processing_and_Machine_Learning_for_Signal_Processing_Functional_Algorit_10236_11327.png Tensor_Processing_and_Machine_Learning_for_Signal_Processing_Functional_Algorit_11328_11891.png Tensor_Processing_and_Machine_Learning_for_Signal_Processing_Functional_Algorit_11892_12563.png Tensor_Processing_and_Machine_Learning_for_Signal_Processing_Functional_Algorit_12564_14519.png Tensor_Processing_and_Machine_Learning_for_Signal_Processing_Functional_Algorit_14520_15767.png Tensor_Processing_and_Machine_Learning_for_Signal_Processing_Functional_Algorit_15768_15911.png Tensor_Processing_and_Machine_Learning_for_Signal_Processing_Functional_Algorit_15912_16703.png Tensor_Processing_and_Machine_Learning_for_Signal_Processing_Functional_Algorit_16704_18023.png Tensor_Processing_and_Machine_Learning_for_Signal_Processing_Functional_Algorit_18024_19763.png Tensor_Processing_and_Machine_Learning_for_Signal_Processing_Functional_Algorit_19764_22259.png Tensor_Processing_and_Machine_Learning_for_Signal_Processing_Functional_Algorit_22260_22307.png Tensor_Processing_and_Machine_Learning_for_Signal_Processing_Functional_Algorit_22308_25307.png Tensor_Processing_and_Machine_Learning_for_Signal_Processing_Functional_Algorit_25308_27587.png Tensor_Processing_and_Machine_Learning_for_Signal_Processing_Functional_Algorit_27588_32051.png Tensor_Processing_and_Machine_Learning_for_Signal_Processing_Functional_Algorit_32052_33995.png Tensor_Processing_and_Machine_Learning_for_Signal_Processing_Functional_Algorit_33996_34403.png Tensor_Processing_and_Machine_Learning_for_Signal_Processing_Functional_Algorit_34404_34775.png Tensor_Processing_and_Machine_Learning_for_Signal_Processing_Functional_Algorit_34776_34847.png Tensor_Processing_and_Machine_Learning_for_Signal_Processing_Functional_Algorit_34848_35219.png Tensor_Processing_and_Machine_Learning_for_Signal_Processing_Functional_Algorit_35220_40547.png Tensor_Processing_and_Machine_Learning_for_Signal_Processing_Functional_Algorit_40548_40595.png Tensor_Processing_and_Machine_Learning_for_Signal_Processing_Functional_Algorit_40596_41843.png Tensor_Processing_and_Machine_Learning_for_Signal_Processing_Functional_Algorit_41844_42047.png Tensor_Processing_and_Machine_Learning_for_Signal_Processing_Functional_Algorit_42048_42155.png Tensor_Processing_and_Machine_Learning_for_Signal_Processing_Functional_Algorit_42156_42270.png
Okay. Hey, so I'm Tim O'Shea. I'm from Virginia Tech. I'm just a researcher there. And I wanted to talk a bit about the possibilities for tensor signal processing and then kind of bleed into, how that bleeds into machine learning as well. So just to set the background, I'm sure a lot of you have been watching this, but what's been going on in machine learning and kind of the resurgence of neural networks has been pretty awesome to watch. And this has primarily been in the computer vision area. And so there's a lot of people in the self-driving car area as well as like facial and object recognition areas and natural language processing where we've made lots and lots of advances in recent time in driving down error rates on translation and object recognition and all these things. And if you look at it, so neural networks from a computational standpoint are really interesting because they have this fairly well-defined compute structure that constitutes a network, right? And so if you look at kind of the calculations for a neural network, you have this kind of, it's really basically a big multiply accumulate for each layer. So you have these inputs that are multiplied with a series of weights that are summed and then run through some non-linearity like a sigmoid function. And that's basically one layer in a neural network. Now if you look at these so-called convolutional layers, you no longer have a separate weight on each input, but you have these weights that are tied in a way such that when you slide a patch across an input, you replicate the same weights at different shifts, right? So that's really the only difference. And then if you look at some of the bigger models, so this is from like a year and a half or ago, one of Google's vision networks for doing object recognition, you think of that transfer function for a layer, you know, that's one box in this network, right? So we now have this network which is, you know, 40 or 50 layers. Some of the ones this year were over 100 layers deep in the complexity that they're doing to do this object recognition. So there's been a really, really strong software ecosystem to support machine learning for computer vision. And it's really kind of starting to centralize around this kind of tensor programming model. And this is really, you know, interesting because you're building these big tensor graph expressions and then, you know, as a big abstract data flow representation and then you kind of map it down to hardware to run. So two of these architectures are kind of theano and tensorflow. Theano was around kind of first and kind of pioneered this and then Google like secretly rewrote it internally and then eventually released it to the world and now they're both very popular but the whole idea here is these are basically libraries that allow you to write big numpy expressions essentially where you have many operations that are kind of forming a data flow graph and then take it and the idea is you build this graph expression and then you go through the back end and you go out to any one of a number of targets and so right now we go to basically GCC and CUDA as the back end but there's a lot of plans underway at Google to do this thing called XLA which is like their next generation tensor back end and they're doing a lot of work to sync this up with LLVM so they can actually just use kind of LLVM abstract representation to pass it in. And then there's a lot of rumors about potential support on like the QDSPs that Qualcomm builds for base bands as well as some of the Xilinx chips. There's been a lot of talking about Xilinx potentially supporting this as a co-generator so it's really cool. We may have finally a language that we can target to a lot of different kind of platforms from this kind of way to express things. So this is used heavily for neural networks but it's not really neural network specific so if you look at this is basically a simple expression. We start by defining these variables like A and B and we can define operations on them like adding or multiplying and then eventually we go into our session and we evaluate these expressions and so this is basically building up your big, in this case a very small kind of data flow graph and then this is going to be compiling and running your graph. So you can build much bigger things in this so I talked a little bit before about GR TensorFlow and the whole idea is really at the core of this, this is just kind of like a kernel work call so we can wedge it right into a GR radio block and use it to offload these tensor operations from within a work function. And so this is something added a couple months ago and the whole idea here is can we take a sync interpolator of a signal and offload that onto GPU or onto whatever using this kind of tensor language and so this is basically the definition of a sync interpolation. And so you're going through here and up at the top you're defining the graph through this series of castings, you know, tiling of the input, doing these gather operations which are basically index lookups into vectors and then doing a reduced sum which is basically just a big multiply accumulate. And so once we've kind of issued these commands we've built this kind of dependence graph for the operation and now whenever we run it we could change the back end to either the CPU or the GPU or, you know, some of the targets that are probably coming soon. And so that's kind of just an example. I think there's not many people out there using this for pure signal processing like this yet but I think there's a huge potential for it. The tensor languages are pretty expressive for almost any of the commas problems we have and the kernel compilers and optimizers are really quite good. So this is kind of a pretty exciting way to offload onto GPUs and things. All right, so I'm going to switch over to the machine learning side. So that was kind of basic signal processing but I guess my research now is really centered around, okay, how do we take these machine learning ideas and apply them to radio technology and radio communications? And so these are just kind of a couple of the questions you might ask, you know, can we apply it in these ways? And so really I've been looking at, you know, can you learn entirely new communications schemes or can we just learn algorithms to put into our current waveforms as different subsystems? And then kind of in a sensing scenario you may want to know, you know, what's around you in your interference environment like Tom was talking about this morning where you may have EMI emissions from your, you know, your power supply or whatever. You may want your system to be aware of all these emitters around you when it makes decisions about setting up the communications link. And then just how to optimize existing systems. So these are kind of some of the high level goals, right? So this is some of the earlier work from this year and I think we saw a little bit in the GR inspector talk which was awesome. So the whole idea here is, you know, for a long, long time when we looked at doing modulation recognition which is kind of a classical communication sensing task, the approach was generally to do this expert feature approach where you extract a bunch of high order moments or high order cumulants along with maybe statistics on the signal and the phase and the envelope. And here we're kind of, we're trying to go to a whole different route of just let's, instead let's go to feature learning on the raw input signal and just put the INQ in and let it learn a whole new set of its own features. And it's funny, I mean, if you look at what's happened in the image, the computer vision domain, this is exactly, you know, what has happened. So until five or ten years ago, there was a whole discipline of feature engineering in vision which has been largely replaced now with these kind of end-to-end learned features. And so I think a lot of this is going to happen in the comms and the radio space as well. And so this is really kind of showing that for modulation recognition, we can do pretty much the same thing. So we start, there's a data set that's out on the website and, you know, this is basically just using GUNI radio to generate a whole bunch of synthetic signals through a whole bunch of synthetic channels that use the dynamic fading model. And then this is kind of what we used for that experiment. So this is just essentially, you know, this is basically just a small convolutional neural network that we're using to do this classification. And if you look at performance, there's kind of a bunch of the solid ones here, are different variations of a kind of feature learning approach which take in the raw samples. And the dotted ones are basically different classifiers using these expert features like higher order moments to classify. So here you can see, you know, in the best case for both, we get, you know, three or four DB sensitivity improvement with these methods, which is pretty exciting. Three or four DB is a lot in some worlds. So and then, yeah, so, all right, so one of the other cool things you can do is if you take, for instance, if we look at this network, our output is 11 values, right? So these correspond to one per modulation class, you typically use like a one-hot encoding for a classifier. If you go back one layer, you have this feature map of activations between each layer. And generally, this is reducing in size as you go through the network to become a more and more concise representation of the information. So if we go back one layer and we just take these activations here, we actually now have a, basically a compressed representation of the signals from the input in this feature map space. And so what's really cool is we can start now using that compressed representation to just kind of visualize, you know, where are these signals in this kind of abstract modulation space, right? And so now we can start to cluster them and, you know, we have this basis now that is relatively good at separating a bunch of different signal types. And so this is kind of interesting because, you know, if a new modulation pops up, you know, if these features generalize, you know, we hope we'll get a new cluster over here or something that's discernible so that we can start to recognize things online without having to train it for every explicit case kind of ahead of time. All right. So then this is, this goes to some of the work that we did this week, HackFest. And so how many times, I guess, raise your hand if you've ever written a flow graph that's like a usurp source through a head to a vector sync, right? I feel like, okay, we got a handful. So sometimes it feels silly to use Gineer radio when all you want to do is just grab a chunk of samples off a radio. And so that's the whole point of this NumPy UHD wrapper. And so this is just like a super, the thinnest interface you could have to say, you know, tune my radio somewhere and grab some samples from it. And so super simple. And so what we're doing here is this makes it really quick to just take little looks all over the spectrum and generate spectrograms for them. So if we do that, we can start to get all these samples of different bands that are all around us. And so this is just kind of a sampling of a bunch of spectrograms across, you know, common bands. So if we use this in a ConvNet, we can also build a classifier for the different bands. This is not a very good one at the moment, but that's just because the slides were a bit late. But so it's not necessarily that exciting to classify which band you're on, because, you know, we have allocation schemes. This isn't really that useful. But the thing that's really interesting about it is we learn these sets of features for each band that make that band what it is. So like the LTE downlink band is filled with LTE signals that have all the properties and features of an LTE signal in the downlink. And so our network, as it's learning this, it starts to learn all these features within, you know, that comprise that band. And so what we can do is we can now take any random spectrogram that comes in, like these two, and we can look at where in that input spectrogram were there activations that led to thinking it was that class, right? So we can say, you know, what in this image makes us think it's LTE uplink, right? And so if you do that, you can look at and get this localization mask that's used in, you know, in imagery for object segmentation. We can do the same thing in spectrum. So we can now say, well, once we've trained these features, we can take in arbitrary spectrograms and really identify what's going on where and start labeling things. All right, so then kind of the, I think the last kind of little application thing here, and I'm really, this is the thing, if you look at machine learning and comms, this is what I'm, I think, most excited about right now. And the whole idea here is if you think about a comm system at its most basic form, it's really like you have some bits somewhere and at some of the location, you just want a good estimate for those bits, right? That's kind of all it is. You're just taking bits in and trying to reconstruct them somewhere else. And so that's pretty much what an autoencoder does in machine learning, is it takes some input, it goes through a couple different representations of it and it tries to reconstruct the input. And so if we cast the basic communications problem as an autoencoder, we have this case where we go through some input. We form some new encoding of those input bits. We go through some channel impairment which we don't control, this is kind of defined by a wireless environment. And then we go through this decoder representation that somehow transforms the samples we receive back into bits. And really that's the whole process we want to optimize. So we can treat this as just a, an optimization of a reconstruction process where we have this layer in the middle that introduces this randomness that mirrors what a wireless channel does. And so what's really cool here is we can let this thing learn now some completely arbitrary physical layer. We know we don't need PSK or any of these things anymore. And we can actually start getting reasonably good capacities out of these encodings. So we can now, you know, just learn a radio for a specific radio environment, you know, without any expert knowledge of the past 70 years of signal processing, which is pretty crazy. So if we do that, we can start plotting basically BER curves. And these are, these are kind of some old ones that aren't as clean. But you can see that, you know, in some cases we can outperform the performance you would get on a modulation without any error correction, which is really cool because it means the encoders and decoders that we're learning can actually learn a little bit of error correction as well. So you're now, you're basically learning something that does better than just a normal modulation scheme. So there's a, I'll mention it later. So this is just an example of a couple different runs and what, you know, what the resulting learned bases look like in this case. Yeah, and there's many, many things that affect this. So there's right now still a million kind of tuning dimensions that you can look at while training these things and optimizing all over all of those is still kind of a big challenge right now. And right now the other big challenges are this, you know, this works for very small code words. But if you try to scale this to like, you know, 1024 bits or something that LTE might use in an LDPC code, it's much harder to scale it. So that's one of the things you're looking at now is how do you scale this up to like more real situations and over the air. All right, and then lastly, so when we learn that in coder and decoder, the decoder has to learn how to translate from some channel representation to the bits, right? Now if you do that just naively, we get some process that can, it takes a long time to converge, right, because it's having to learn how to estimate the channel, somehow apply those corrections, and then somehow go to a back to a bit information encoding. So there's this notion from vision of attention models. And the whole idea of these attention models is that instead of learning some arbitrary transform like how to equalize, you can simply break the network into learned parameter estimation, kind of an expert transform like equalization, and then our learned, you know, decoder. And so in doing this, you basically tell the system, well, okay, this is how you would apply equalizer taps, or this is how you would apply frequency correction if you knew how far off it was. And it turns out if you do that, this converges like much, much faster, orders of magnitude faster here, and we get significantly better performance as well in the auto encoder case. So it's like we're kind of helping the system along to learn the basic physics of the problem, to reduce the complexity that the machine learning has to fit. And so this is kind of exciting that this works. And so on top of that, there's this whole problem of like, you know, we have a whole discipline of estimation and detection, right? And it's interesting to just look at, you know, can we treat these estimation problems as just machine learning regression problems? And so we started looking at that. And if you look at this as comparing the map estimator for center frequency estimation, with basically just a neural map trained to do regression to generate estimates for the same thing. And if we do that, and this is under a Rayleigh fading channel, so if we do that, we get a slightly tighter distribution from the regression process. And so this speaks to like, why do you want to do this? You know, we have a lot of great analytic models for these, but one of the problems with them is there, many of them are derived under much more simplistic channel assumptions. And so if we can do this with real data and regression, then we can start to model, you know, all kinds of nonlinear effects and all kinds of much more complex channel models than we can in our analytic forms. And so it's really kind of exciting about, you know, where our estimation processes can go in the future with this approach. Right, so ML is affecting a lot of fields in computer very quickly right now, which is pretty exciting. And the area of applying this to the RF radio domain is super young. There's really not much work out there yet at all. And so I think if you're a student right now looking for kind of topics to work on, this is like, you know, super ripe. And I wouldn't point you anywhere else. But obviously I'm a little biased. So there's also, I have a journal article that will be appearing on Archive Monday. And it goes into this autoencoder and the mod rec process much more rigorously. So if you're interested in a more thorough explanation, definitely go and check that out Monday. And there will be a link on radio ML.org. So I'm trying to start kind of turning radio ML.org into somewhat of a community for people interested in this. So, you know, if you would like to contribute tools or data sets or articles or examples or anything to radio ML and start contributing, like please let me know because I would love to have your work there. And yeah, I think that's about it. Thank you. Any questions? Yep. So you mentioned about using TensorFlow for signal processing. So how much, so you gave an example of sync interpolator, how much work do you know that's being out there for what, how do you define these different DSP applications? Yeah, so there's not much out there at all for like radio signal processing yet. I think almost all the work has been done in computer vision. And so there's a lot of tools to do kind of computer vision operations, but very little of that has been applied to radio yet. And I think that's kind of one of the niceties is that the computer vision guys are, you know, paving a lot of the roads for, you know, figuring out the compilers and all the back end and support issues so we can use a lot of the same infrastructure in radio without having to reinvent it. I think Google said they moved like their machine learning for all of their services over to TensorFlow. And they have like a 30 or 40 person team just supporting the TensorFlow product line now. So there's a lot of, you know, stuff we can just take for granted at this point. Yep. I think it was for you, like the demony knowledge of real time computation requirements, latency, scale kind of, you showed comparisons of theoretical performance, you don't worry about like actual runtime performance. Right. So I haven't, there's not been a whole lot done on latency yet, but on computational performance for the ModRec example, I did a op counting comparison to the kind of state-of-the-art kind of classic algorithms. And we, so the initial network was something like nine times more operations than the classic method, right? So but it turns out then when we go in and we start doing certain things within the network, like we use one by one comb filtering on the output of several convaliers, which is a trick. And one by one filtering sounds very silly and I think to most electrical engineers because, you know, why would you do that? But the point of a one by one filter is actually that it's combining in the number of output channels. So you have actually n channels of 2D image coming in and m channels of 2D image going out. So you can use one by one filtering to reduce like that number of channels down. And so by playing tricks like that, we were able to drive the complexity down like maybe 50X or so. And so we're now a factor of about four lower in op counts than the kind of classic approach in that application. So I think like, you know, there's a lot of things that make it attractive, right? So all these multi-core architectures, you know, this is a very parallel thing, right? You can divide up the tensors in width and in depth onto different cores. And one of the other cool things is there was a paper looking at how much representation precision do you need for these networks to work. And because right now this is all done in single precision floating point. But they showed that you really only need four to eight bits of precision throughout the whole network for this to work equally well. And so there's another potential, you know, huge savings in complexity. So yeah, I think ultimately for comms and for sensing, this is potentially drastically lower complexity than our current approaches. Yep. Yeah. So I can't do it justice here. If you'll read the paper on Monday. So the question was about coding and whether we compared to Shannon and so forth. And yes, so the baselines here are, you know, I didn't want to go and put everything that's going to be released Monday yet. But if you go through the paper that's coming out Monday, we have some very, very good baselines where we're looking at comparisons to, you know, PSKs and DPSKs with Hamming code and, you know, basically right up against capacity. And so we're much more robust there and showing, you know, the capacity that you can achieve with this. And you can come very, very close to, you know, the channel limit. Switch over. Okay. All right. The next speaker. Ah. If you go that far, do
to talk a bit about the possibilities for tensor in machine learning and kind of the resurgence of lot of work to sync this up with LLVM so they can TensorFlow for signal processing. So how much, so