My GitHub (Trash code I wrote during PhD)
deep learning machine learning arxiv explained neural networks ai artificial intelligence paper github my github phd code code during phd deep learning tutorial what is deep learning introduction to deep learning deep learning phd coding
#phdlife #github #researchcode

A brief browse through my public GitHub and musings about my old code.

Link: https//github.com/yk

Links:
TabNine Code Completion (Referral): http://bit.ly/tabnine-yannick
YouTube: https://www.youtube.com/c/yannickilcher
Twitter: https://twitter.com/ykilcher
Discord: https://discord.gg/4H8xxDF
BitChute: https://www.bitchute.com/channel/yannic-kilcher
Minds: https://www.minds.com/ykilcher
Parler: https://parler.com/profile/YannicKilcher
LinkedIn: https://www.linkedin.com/in/yannic-kilcher-488534136/
BiliBili: https://space.bilibili.com/1824646584

If you want to support me, the best thing to do is to share out the content :)

If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):
SubscribeStar: https://www.subscribestar.com/yannickilcher
Patreon: https://www.patreon.com/yannickilcher
Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq
Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2
Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m
Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
 Hey ho, what's going on? So I've recently graduated the PhD. And during that time, I've written a lot of code, which is mostly garbage. But I thought we'd go through my GitHub, and I'll show you the most exciting and useless things I've ever written. So if you're on my GitHub, you're going to find a bunch of things including video related materials, such as like the clip music video, you can make your own music video, right? Here, be my weasel. You should watch if you haven't. There's the Minecraft neural network I provide you with the Minecraft world. If you haven't watched that video, please do it. GPU stat, which is a tracker for GPU machines and sending it to a server and then displaying it. This is what our lab uses for seeing who uses which GPUs, which is, you know, fairly useful. I think this is the single most popular thing I've written during my PhD, because that's people actually use it. So there is the flatland repository. So flatland is something we did some time ago, and then I was a total slug and completely failed in supervising the project. Let's not talk about this. You'll also find code for our conference submissions, of course, but then we get into the real stuff. This run is a little tool that you can use. What it does is it simply copies directory to a server via SSH, it then runs a script on that server. And then it copies back a directory called logs. That's pretty easy. And I use that all the time is very good. If you have a bunch of code in a folder and the output is a directory called logs, you're good to go. Otherwise, you'll have to change this a bit. Okay, at that point, I had no clue that you could use temp dear to make temporary directories. Oh, God, look at this. So it happened too many times that I didn't do this from the directory where I actually had my code but from the from the home directory. So it synced my entire home directory to the server. So I just know. See, this counts as UX. No, I'm pretty sure it does. And this right here, this is the crown jewel rat, it is a system that manages my experiments. So in rat, there is a bunch of things in here, there is a worker. And what the worker would do is it would sit on a server, and it would listen to a database for new experiments that it should run. And if so, it will pull the code from a MongoDB. So so that the queue isn't is a is a redis queue, and we'll pull code from a MongoDB. And then it would run that code. But it would only do so if the GPU is free. So to change this RQ thing in order to check whether or not the GPU is free, you can see right here, there's a check of whether or not the GPU is already occupied. And if it is occupied, it would just not do the task and put it back into the queue. However, if it is not occupied, it would run. So the neat thing is that the thing you can do with this thing is if a lab mate of yours is running on a GPU, you just put this worker on the same GPU. And then as soon as their job is done, it's like, boom, you got it. I'm sorry, sorry. But for the most part, it actually prevents you from interfering with other people, you know, that's pretty neat. And your jobs won't fail just because there's already something on the GPU. So the core of this thing is you can run an experiment config, which means you can upload different hyper parameters, and then jobs would be generated according to those hyper parameters. And I even built in a hyper parameter optimizer. So you can give ranges and it would search through them either in grid search or in random sampling. So here we have a search strategy. And I built in so much stuff, you can merge experiments. I mean, look at this, this is, this is quite a bit of engineering going into here. It even has a tensor board thing. Whenever a job is finished running, the worker would actually put it back into the database. And this command right here will get me all the event files from tensor board. And then it would actually label the directories with the names of the hyper parameters. So you actually see directly in the run name, which run has which hyper parameters, this is so freaking useful, because usually tensor board runs are just like run one, run two or the date or some stupid thing. Confirm, really? No, I built this in to prevent myself from doing stupid stuff. But I also built like an override flag, you know, like there's delete all. So as I said, this is, it probably doesn't work anymore, because I know the Redis Q dependencies have shifted and so on. Yeah, if you want, if you want some inspiration, feel free, feel absolutely free. To clone this, I don't want it anymore. When I started systems like weights and biases, and so on, they just didn't exist. So I had to run my own. Similarly, why plot is my attempt at writing a plotting library that works with tensor board events. And so extracting data from tensor board events, this is all so useless right now, except this smoothing thing that I got from scipy, which was pretty useful. Then why pack is you can tell my name, I'm very innovative with my names. I think that's just a set of routines that I implemented for working with torch and tensor flow. Again, this is probably all useless. So there's deep fool. Look at that. Most of this is completely useless now because these things are mostly in the libraries themselves. Conf prod is what I use. Oh, look at that. This is a part of rat actually, this is what generates a products of configurations. That's why. Yeah, I even wrote a read me, I wrote a read me a small utility library to generate cross products of experiment configurations, just look at the unit test, and hopefully it should become clear how it works. Let's do it. I don't think so. I mean, look at that. This is beautiful. Look, you can, like, spec out something like this, you can see like so there is, you want SGD optimization. And these are the different step sizes and you can sample and this seems like a good a good thing. I mean, there are probably 50 libraries today that do that much better than than I ever could. Fountain Oh, fountain was my own data set library, like C for 10, it would it would download it from a server, and it would extract it if it's not there. Yes, this all exists now in torch vision. And for the ML for NLP in hugging face, what a useless thing. This thing right here, I think. So in TensorFlow one, if you youngsters remember that it was quite a bit harder to save and restore and do anything like this. So this would be a library that if your checkpoint doesn't quite fit, it would restore whatever is there. And I think it would also if the shapes don't fit, it would do like a random projection to make the shapes fit. And if they don't fit, this you had to implement like a graph of the object. Too much operation just to get the restore to work. This is a plugin I wrote for Chrome because I was annoyed that I couldn't cite an archive article from the article itself. So I wrote a plugin that goes to Google Scholar and scrapes the the Google Scholar BIB Tech entry in directly to log to archive it doesn't work anymore, but I think there are other plugins. are actually good. This is a continuous compiler. As you can see, it's not very sophisticated. And of course, I did write my own archive scraper, there was still a time when I read all of archive, this is not possible anymore. But I did read all of archive for at least certain lists. So I had had many more than these lists, new papers every morning. And I would just read through the abstracts in the train. And those are repositories from my masters. And so this is the first public repository ever from the pattern recognition class in my bachelor studies. What is here? Linear kernel, polykernel, RBF, this looks like support vector machines, right? Did I implement this? Here's an SVM classifier implemented. Yikes. And this, who does that? Who does private methods with a dunder? No, that's reserved. Whoever did this past me? No, nonlinear SVM without any sort of automatic back propagation. No, no, stop. Yeah, but this is a this is a support vector machine without without SGD. I think we used to calculate support vector machines with sort of a quadratic programming, I think that we got that from somewhere. In any case, this was my very, very first public commit to GitHub. And it was already a machine learning lecture. So I guess I had this coming for a while. If you are interested in useless repositories, check out my GitHub, I'd be happy to see what your githubs look like. So this was more of a nostalgia thing, but I hope you still had a bit of fun. Cheers.
 Hey ho, what's going on? So I've recently graduated the PhD. And during that time, I've written a lot of code, which is mostly garbage. But I thought we'd go through my GitHub, and I'll show you the most exciting and useless things I've ever written. So if you're on my GitHub, you're going to find a bunch of things including video related materials, such as like the clip music video, you can make your own music video, right?
 Here, be my weasel. You should watch if you haven't. There's the Minecraft neural network I provide you with the Minecraft world. If you haven't watched that video, please do it. GPU stat, which is a tracker for GPU machines and sending it to a server and then displaying it. This is what our lab uses for seeing who uses which GPUs, which is, you know, fairly
 useful. I think this is the single most popular thing I've written during my PhD, because that's people actually use it. So there is the flatland repository. So flatland is something we did some time ago, and then I was a total slug and completely failed in supervising the project. Let's not talk about this. You'll also find code for our conference submissions, of course, but then we get into the real stuff.
 This run is a little tool that you can use. What it does is it simply copies directory to a server via SSH, it then runs a script on that server. And then it copies back a directory called logs. That's pretty easy. And I use that all the time is very good. If you have a bunch of code in a folder and the output is a directory called logs, you're good to go. Otherwise, you'll have to change this a bit. Okay, at that point, I had no clue that you
 could use temp dear to make temporary directories. Oh, God, look at this. So it happened too many times that I didn't do this from the directory where I actually had my code but from the from the home directory. So it synced my entire home directory to the server. So I just know. See, this counts as UX. No, I'm pretty sure it does.
 And this right here, this is the crown jewel rat, it is a system that manages my experiments. So in rat, there is a bunch of things in here, there is a worker. And what the worker would do is it would sit on a server, and it would listen to a database for new experiments that it should run. And if so, it will pull the code from a MongoDB.
 So so that the queue isn't is a is a redis queue, and we'll pull code from a MongoDB. And then it would run that code. But it would only do so if the GPU is free. So to change this RQ thing in order to check whether or not the GPU is free, you can see right here, there's a check of whether or not the GPU is already occupied. And if it is occupied, it would just not do the task and put it back into the queue. However, if it is not occupied, it would run. So the neat thing is that the
 thing you can do with this thing is if a lab mate of yours is running on a GPU, you just put this worker on the same GPU. And then as soon as their job is done, it's like, boom, you got it. I'm sorry, sorry. But for the most part, it actually prevents you from interfering with other people, you know, that's pretty neat. And your jobs won't fail just because there's already something on the GPU. So the core of this thing is you can run an experiment
 config, which means you can upload different hyper parameters, and then jobs would be generated according to those hyper parameters. And I even built in a hyper parameter optimizer. So you can give ranges and it would search through them either in grid search or in random sampling. So here we have a search strategy. And I built in so much stuff, you can merge experiments. I mean, look at this, this is, this is quite a bit of engineering going into
 here. It even has a tensor board thing. Whenever a job is finished running, the worker would actually put it back into the database. And this command right here will get me all the event files from tensor board. And then it would actually label the directories with the names of the hyper parameters. So you actually see directly in the run name, which run has which hyper parameters, this is so freaking useful, because usually tensor board runs are just like run one,
 run two or the date or some stupid thing. Confirm, really? No, I built this in to prevent myself from doing stupid stuff. But I also built like an override flag, you know, like there's delete all. So as I said, this is, it probably doesn't work anymore, because I know the Redis Q dependencies have shifted and so on. Yeah, if you want, if you want some inspiration, feel free, feel absolutely free.
 To clone this, I don't want it anymore. When I started systems like weights and biases, and so on, they just didn't exist. So I had to run my own. Similarly, why plot is my attempt at writing a plotting library that works with tensor board events. And so extracting data from tensor board events, this is all so useless right now, except this smoothing thing that I got from
 scipy, which was pretty useful. Then why pack is you can tell my name, I'm very innovative with my names. I think that's just a set of routines that I implemented for working with torch and tensor flow. Again, this is probably all useless. So there's deep fool. Look at that. Most of this is completely useless now because these things are mostly in the libraries themselves.
 Conf prod is what I use. Oh, look at that. This is a part of rat actually, this is what generates a products of configurations. That's why. Yeah, I even wrote a read me, I wrote a read me a small utility library to generate cross products of experiment configurations, just look at the unit test, and hopefully it should become clear how it works.
 Let's do it. I don't think so. I mean, look at that. This is beautiful. Look, you can, like, spec out something like this, you can see like so there is, you want SGD optimization. And these are the different step sizes and you can sample and this seems like a good a good thing. I mean, there are probably 50 libraries today that do that much better than than I ever could.
 Fountain Oh, fountain was my own data set library, like C for 10, it would it would download it from a server, and it would extract it if it's not there. Yes, this all exists now in torch vision. And for the ML for NLP in hugging face, what a useless thing. This thing right here, I think.
 So in TensorFlow one, if you youngsters remember that it was quite a bit harder to save and restore and do anything like this. So this would be a library that if your checkpoint doesn't quite fit, it would restore whatever is there. And I think it would also if the shapes don't fit, it would do like a random projection to make the shapes fit. And if they don't fit, this you had to implement like a graph of the object.
 Too much operation just to get the restore to work. This is a plugin I wrote for Chrome because I was annoyed that I couldn't cite an archive article from the article itself. So I wrote a plugin that goes to Google Scholar and scrapes the the Google Scholar BIB Tech entry in directly to log to archive it doesn't work anymore, but I think there are other plugins.
 are actually good. This is a continuous compiler. As you can see, it's not very sophisticated.
 And of course, I did write my own archive scraper, there was still a time when I read
 all of archive, this is not possible anymore. But I did read all of archive for at least certain
 lists. So I had had many more than these lists, new papers every morning. And I would just read
 through the abstracts in the train. And those are repositories from my masters. And so this is the
 first public repository ever from the pattern recognition class in my bachelor studies.
 What is here? Linear kernel, polykernel, RBF, this looks like support vector machines, right?
 Did I implement this? Here's an SVM classifier implemented. Yikes. And this, who does that?
 Who does private methods with a dunder? No, that's reserved. Whoever did this past me? No,
 nonlinear SVM without any sort of automatic back propagation.
 No, no, stop. Yeah, but this is a this is a support vector machine without without SGD. I think we
 used to calculate support vector machines with sort of a quadratic programming, I think that we got
 that from somewhere. In any case, this was my very, very first public commit to GitHub. And it was
 already a machine learning lecture. So I guess I had this coming for a while. If you are interested
 in useless repositories, check out my GitHub, I'd be happy to see what your githubs look like. So
 this was more of a nostalgia thing, but I hope you still had a bit of fun. Cheers.