Graphics and Simulations (and Games), Oh My!
Davis, Ryan
Quick and easy visualization is an important teaching tool and a wonderful way to see your ideas in action. Unfortunately, it is neither quick nor easy for rubyists to get from idea to visualization. You have to have a ton of extra stuff installed. You need to know game mechanics/math. You need to learn a whole other way of programming. It isn't something that is easy for a beginning programmer to pick up. This talk describes an alternative graphics and simulation library with a very clean API making it incredibly easy for newbs and experimenters to go from idea to visual simulation. Pew pew pew!
 Hello! Holy crap, it's packed. I did not expect this many people for this type of talk. Thank you so much for joining me today. I'm Ryan Davis, and today I'm going to be talking about graphics and running simulations. I should note that despite the title, there will be no Wizard of Oz references. I'm very sorry about that, but that sets a level of expectation on my graphics prowess, and I just don't have that. So something I like to do in all my talks is to set expectations up front. This is about half show and tell, and about half tutorial. It's got a medium level of code compared to my normal talks, and it is suitable for all levels of developer. It's about 180 slides, which if I give some time for QN at the end, is about five slides a minute. But first, a little bit about me. I've been coding professionally for a long time, 18 of those in Ruby. I'm the founder of Seattle RAB, the first and oldest Ruby Brigade in the world. I'm the author of many tests, FLOG, FLAY, Ruby parser, and a whole lot of other gems. I figured out this year that I pushed over 1,000 gems into the Ruby ecosystem. I'm a developer's developer. I really like building tools that we use, and I run Seattle RAB Consulting at SeattleRAB.com. I really hate these types of slides and these talks. I'm really sorry about this, but I need new clients, and speaking is my primary lead generator. So what is graphics, and why does it have such a boring name? I was tired of nonsensical names like Miniskirt and Nokogiri, and both graphics and simulation were available, and I literally flipped a coin and wound up with graphics. Oh, hi, Aaron. Quite simply, graphics is a gem that I wrote that you can install right now. Unfortunately, it's still in beta, so you have to use dash dash pre. I'm sorry about that. I'm going to try to fix that today. It tries to provide a simple framework to implement simulations, which I'll leave open at this point. Digital art, custom visualizations, and even games. It's designed to follow grade school mathematical conventions, not game programming conventions, and I'll get more into this later. So I want to demystify graphical programming, generative art, simulation, scientific visualization, simple games. I want to show that these topics aren't actually hard. Perhaps most importantly, though, I want to add a new tool to your toolbox, so you can use it to add an extra dimension to your work or to understand a problem that you're having. But first, a simple word of warning. I am not good at art or graphical programming. I'm aesthetically challenged, especially in electronic mediums. I look at my website for proof. I don't have the strongest math background, especially in linear algebra and other math heavily used in graphics. I'm not a scientist, and I'm not a game dev. I will never write the next version of Doom. This makes me a perfect candidate to try to write a graphics framework for the rest of us, because I'm damn good at making tools to help me and other developers out. So I've made this gem to be as useful and as powerful as possible, while at the same time to be as simple as I can possibly get it. But I prefer real examples over theory and marketing fluff, so let's quickly get concrete with a real and tangible example. Let's start with a blank window. This is all it takes. You require graphics. You create a new class, and you subclass graphics, colon, colon, simulation, where G, colon, colon, S is just slide shorthand, because nothing fit in this talk. In the case of these talks, I'm going to be including a white background in all the classes, because I wasn't quite sure how good the throw was going to be on the projector, and a thin line on black might not come through. And you instantiate it, and you tell it to run. You wind up with a blank canvas that is defaulting to half the width and height of your actual screen. To actually draw something, we had a draw method. We call it super, and then we declare a circle in the middle of the window with a radius of 15. We tell it to be blue and filled. You see a circle, but there's no state or behavior with this setup. For that, I want to make this a bit more complex. I declare a body subclass called ball, and in that, I declare count to be one, and then I create a class called view with a draw class method that takes the window and the ball to draw. The drawing code uses B.x and B.y, but as otherwise, very similar. Then we get rid of the original draw method. Then back in simulation, I add an initializer, and I declare that ball is going to be used by calling register bodies with the result of populate ball. It's not too much different. Everything's just been displaced a little bit. This allows for state and behavior, and this is when things start to get interesting. We can add an initializer to ball, call super, and then set both the angle and magnitude to random values. The x and y of the body is already random via the super. Now to add behavior, I want an update method to ball. Is that running? Interesting. It is running, but it's not showing on my screen, so I need to stop looking at my screen. For now, we're going to tell it to move and then wrap around as if necessary. This is similar to how the classic game Asteroids works. If we change from wrap to bounce zero, where zero means no friction, then we're going to get behavior that's similar to the classic game POM. If we added a small down vector to the ball's velocity and bounced with a default friction, in other words, no zero, then we simulate how a real ball would bounce. We've got three very different behaviors with three very small changes. Finally, a simple one line change from count equals one to count equals 25 makes this much more interesting dynamic example. We've got a simulation class. Declars a white background. Declars that ball should be used. That's all it does. We've got a body named ball. Populate should make 25 balls in the simulation. We have a gravity constant, some random setup, and very basic behavior of balls bouncing under constant gravity. Finally, ball declares a view class that defines how to draw a ball in a stateless manner. 28 lines of code in the end. It only looks complex because it's split across three slides to make this readable. Here it is in its entirety, only barely formatted to fit on the slide, and it's not too bad, in my opinion. It's really, truly easy enough that anyone can do it. I'll give you an idea of what graphics programming looks like with the graphics gem, but that's only the tip of the iceberg. Let's look at more. Let's start with Jitter Divart. Here's criss-crossing lines with random Jitter added to their X, Ys. They're very, very lightly colored, translucent black, but there's 300 of them, so they quickly become solid dark black. This is the exact same code with slightly tweaked behavior. Instead of criss-crossing the lines, each segment is jittered away from the previous. This creates a fountain or jet effect. This is a complex wave pattern. What's cool is that this is not 3D math. It's actually really simple underneath. It's just emergent behavior looks 3D. This is contributed by Josh Cheek, who's been posting amazing videos using graphics, and he's got a lot more graphical skill than I do. Here's another piece by Josh Cheek. I just can't get over what he's done with the graphics gem. It kills me. These are set to music, mine or not. Sorry. This is an emulation of Piet Mondrian's artwork with animations added, which we'll see more of later. Finally, I went to an art exhibit a couple months back where several of the pieces actually showed their maths. I took a picture of this one and I translated it to Ruby. It's only 42 lines of code. God, I love that. Let's look at some more math-oriented examples. I hope this reads. Oh, it reads fine. That's great. This is a very simple proof of concept that you could implement the logo programming system in Ruby. This is taking a terminal input to a drawing turtle to draw a square. This is a standard fractal tree. This one was contributed by Justin Collins. This is a simple visualization of quadrant one maths with a polynomial. Often special, really simple. This also reads really well. This projector is fantastic. I want to buy one. This simulates bouncing balls expanding a polygon to calculate the value of pi. It's hard to see with the current estimate of pi is up in the Windows title, but it only gets to 3.13 by the time the video loops. It does hit 3.14 at some point. Now the part that fascinates me the most, simulations. Here's a more complete version of the bouncing balls example that I gave before. Two very different versions of Conway's Game of Life. Boids or a flocking simulation, which we'll see more of later. Two different versions of fluid dynamic simulations. These use the same maths entirely, but they use different visualizations to show the current density. A colored gamut is being used on the right, and then you can see little spheres as they hit the bottom and the density grows. Canvas drawing of the Vance or virtual ants algorithm, which we'll see more of later. This might be a little hard to see, but this is many, oh no, it looks great. I need to stop saying that. Many randomly moving bodies with collision detection handling, trails, so you can see their paths, and random goal changing behavior. The reason why I got into this in the first place, I wanted to write a zombie outbreak epidemic simulator. This shows one of the very few cases where the priests win and kill all the zombies. While it isn't an explicit goal of the gem, it is totally possible to implement games. This example shows the use of collision maps and sprites. This is a vector-based tank that is driven by the user poorly because I'm driving it. This is a bitmap version with slightly more sophisticated implementation. This is where I started to switch to the Model View Controller setup that I showed you before. It even has pew-pew noises when you shoot. This shows user interaction using the mouse, and finally, random maze generation using James Buck's Maze's for programmers book. He had a really good talk in Salt Lake a couple years back, and I was enthused by it, so I implemented it during his talk. Graphics. I'd like to take a moment to talk about how graphics feels different to me to emphasize my goals that I've been working on while I've been working on this. I want to optimize this to be approachable. I want to ideally target workshops for beginners, other non-computer science folk, and other non-game devs. I want this to be undemanding and easy to install. There are platform-specific external dependencies, and I'm sorry for that, but it's the only thing that makes this performant. I want the graphics gem to be a single gem install and work like any other gem. I've removed and normalized as much as I can to make it usable to anyone with any background. It is optimized to get the first pixels as quickly as possible, to see your code with as little boilerplate as possible. Any new developers, quick and easy feedback is very empowering. While there's no in-pros editing yet, single file is all it takes to experiment. It's simply a matter of edit and run. I want this to be any other script. I don't want it to be an app, all our Rails, or other large frameworks. Wherever possible, graphics uses real maths. Unless you need to use the computer's built-in trig functions, which are always specified in radians, everything in graphics uses degrees. It uses quadrant one maths, just like you used in grade school, unlike game math coordinates, which looks like quadrant four, but for some reason, why is still positive, even though it's going down? It uses the right-hand rule, where zero degrees is to the east, 90 to the north, et cetera, again, just like grade school. These allow simple things to work as expected. It means that all the stuff that you were taught in grade school still works as intended. It makes one less thing that you have to adjust in your brain when implementing your simulation. Graphics is opinionated. It wants to be pretty. There are over 20 graphic primitives available in graphics with more to come. Everything is anti-aliased by default. Alpha blending is automatic based on the color that you're drawing. There are plenty of helpers to make a code as clean as possible, probably more to come. Let's start to talk about how this works. First we'll talk about the update and draw loop. At the highest level possible, we have a model, which is the state of your simulation. Each turn that gets drawn or rendered into a canvas, and that in turn gets copied to the actual screen. As such, our loop is really just three calls. Update or change the model for this turn, draw, which renders the model onto a canvas, present, finally copy that canvas to the screen. As an aside, this is what I want things to feel like. On every turn, you have a new blank canvas. You draw some things, then you copy them to the screen. Under the covers, there might be two or more of those canvases, and they take turns updating and copying to the screen. These are details you don't really need to know most of the time, but they can trip you up on occasion. There's other stuff involved, but the basic run loop works just like this. Update end times, where N is usually one. Draw, then render to the screen. Update by default tells all registered bodies to update themselves. The default draw phase clears the canvas, then tells each registered body to render itself via that body's view class. It's a bit more than this, but this is the gist. And finally, the render phase calls down to the lower layers, the graphics engine, to do the stuff and make it actually display on the screen. There's a very basic class hierarchy. Abstract simulation has nearly all the meat in it. Anything, I should just rename it to simulation. Graphics simulation is the class you're going to use 80% of the time in the end. It has all the normal functionality I've described so far. If anything, you do changes over time, this is probably the class you want to start with. On each turn, it clears the window by default, and it uses very smooth drawing, and by that, I mean it uses that double buffering under the covers. The other 20% of the time, you might want to use graphics drawing. It does not clear the canvas on each turn, and it uses a single canvas that is persistent throughout all turns, and it uses single buffered drawing from that one canvas. It's static, so it's meant to draw things that are either entirely static at the start, they never change, or they're additive over time. This makes it good for plots and simulations or art where the canvas itself is part of the state. Okay, that's well and good, but what can you actually do? How do you draw things? You can pick at each one. For the sake of making these slides readable, I've defined these following values and methods to use in the examples. We have default margin of 50, random X and Y, a random color from all the registered colors that it defaults to have, a random bool which you won't really tell, but I've biased it towards false. For things that need multiple different values, I've got a R50 method that just gets a random 50. So, clear, not much to say here. First thing we almost always want to do on each turn is to clear the screen, but this happens by default when you use simulation. Point is the most basic building block, but not necessarily something you're going to directly use a lot. Here we're making a thousand random points on each turn with random colors, and it just basically becomes this noise. Line is pretty straightforward as well, with two pairs of X, Y values and a color. Here we're using the window height and the current turn number to animate. Angles like line, but you specify a single X, Y, and then an angle and a length instead of a second point. H line and V line take one coordinate and then span the whole window with your specified color. This is useful for plotting stuff. Erin, do not air drop me during the talk. I don't even have a mouse right now, jackass. Circle takes a middle point and a radius. This is the first primitive with an area, so it also takes a Boolean declaring whether or not it's built. Ellipse is just like circle, but it takes two radius values. Fast rect is a filled rect, always filled, taking an origin point on the bottom left, a width and a height and a color. It uses the lower level drawing function that should be faster, but these days it probably isn't anymore because everything is terribly fast. Too fast, in fact. I had to add v-sync to make things viewable. Rect is the same thing as filled rect, but it also takes a Boolean for whether or not it's filled. Polygon takes any number of points in a color. I probably need to extend this to allow for a filled polygon, but no one's asked for that yet. I always feel conflicted about this one. Bezier allows you to draw curves, but not in a terribly approachable way. You need to be more of a graphic step for this. I'm not going to explain this one too much. I want to add splines or something else that I find easier to use. This one's busy. It's one of the busiest slides I've got. Sprite, that call up top, takes dimensions and a block. During the block execution, drawing goes into a separate canvas and that image is returned. Blit takes that image or any image and draws it at a location with an angle and possibly optionally scale. Instead of using sprite to create a canvas, you can call image to load a bitmap from a file instead. Render text takes a string, a color, an optional font. There's always a default font. And returns an image. Put is just like Blit, except that it uses the bottom left corner instead of the center as its origin. Here you can see it rotating on that corner, unlike the tank and turret from before. Text renders, this has been opened, and draws a string at the specified point with a specified color. Okay. Finally, there's some helper functions that render to the screen. FPS prints the frames per second. It's a bad estimate, but it's close enough. Text returns the mouse date and I use that with the debug function that takes a format string and arguments to it and prints it. For debugging at the pixel level or just for fun, you can save the PNG of the current screen. And that's it. So let's build up some real examples, starting with generative art. I throw pottery. I enjoy it. It's fun to struggle with. I don't feel the same way about computer art. It's a struggle for me and I don't find that struggle fun. So I had an interview a few months back where they asked me to generate art in the style of Pitt Mondrian. I don't have time to go on about Mondrian. I could do that for a long time, but you should check out his work. It's really good. So normally I'd start with a hand-drawn sketch of what we're going for, but here we actually have a finished product to emulate. This is his actual work. So we start where we always start, the blank canvas. I've added a grid to help visualize where we want to go. There's stuff here I won't explain. It's mostly to make the slides more readable. Let's focus on the art method. So let's just think about where we want to go. We want thick lines that span the window. So we always have the width and height of the window, so let's just put two down. First vertically at 120 and next horizontally at 80. And W is the value 20, which is the width of the lines that we're going to make. Next let's use some helpers to generate a random number of grid aligned Xs and Ys, and then draw them using the previous code. It's really not much different except there's some loops. This looks good. Now we just need to fill in some regions. And as an aside, I just want to say, like, I love the fact that Ruby RAN could take number ranges. It makes code so much prettier than the way we used to do this shit and see offsets and adding and all that crap. So this is gross, I admit. I pulled it out and it's on method for this slide. Basically we randomly picked some regions based on the Xs and Ys that were previously generated. And then we find the area of those regions and we fill with a random primary color. What is going on next door? That's amazing. I hope this comes through on the audio for the recording. Honestly, that's not bad. And yet they still didn't offer me the job. Or did they offer me any feedback on how I could improve? So I wrote a blog post about it and published the interview question. NRI, you should treat your interviews better. Okay, slightly more relevant to your job. Let's work out some basic data visualization. Let's say you have a lot of data. You don't know what that data means yet. This is a bit of a bullshit example, but gathering data and plotting something like message size over time spent to process. As Rubyists, we're incredibly good at gathering data, whether that's parsing it out of text or slapping it from a database or whatever. Getting the data is never a real problem that we've ever had. So I'm not going to touch how that data is gathered. Just assume that I have it. And looking at a million data points, just under a million data points, and poking at it the way we would normally poke at it doesn't help. We need a way to make some sense of it. There might be a way to visualize it and see if there's a visible correlation. But doing something with that data, something visual, what do we as Rubyists do? Normally at this point, someone would probably suggest grabbing a whole new language like R. It's very good at importing and manipulating data. But it's also a pain in the ass language and overkill if you're not doing a lot of real stats. Or maybe someone else suggests using Excel, it has great plotting capabilities. But the data that you're dealing with is really, really big and getting it into a spreadsheet is a pain. Now you have two problems. We don't have many good options in Ruby for this. We grab for tools in other languages like R or D3 and JavaScript. And that keeps us from improving our own tooling. I think that's a problem. So let's try to address this a bit. I'm not solving this problem. I'm just addressing it. So let's try to do this ourselves playing to our strengths. We always start with a sketch of our system. That didn't come through very well, but whatever. Basically we're going to do a very basic plot, trying to show time over size to see where any problems might be. When I say sketch, I mean it. This is why this is a hand drawn. So as usual, we start with a very basic subclass of simulation to make a blank window. This time I've declared a grid to help. It's change initialized to store off the data being passed in. We scale it to the size of the window, which is max, which is set to 500. Then we use group i to gather and count the same values. And we apply math log to keep the values sane. Otherwise everything is just a big black screen. So now we can add the draw method. We simply walk over the data. We plot a gray point at x, y. I also plot a translucent red circle at that point with the log count as the radius. This will start to show where there's lots of spillover. If you zoom in, you can see that there's very light red is heavily spilling over in some areas. These are areas where you might want to look for trouble. There's huge amount of variance in those columns. Now we know where to look. Have fun with it. Add features. All those axes, put the values in there, make it easier to repeat later. This might not be the right data to visualize, though, or it might not be the right way to visualize it. You need to experiment in order to suss out information from raw data. Running simulations is why I wrote this library. Here are some simple examples on how to do that. Langston's ants, or vants, which is short for virtual ants. They live on a 2D plane and have a simple set of rules that they execute every turn. If an ant is on a white cell, make a black, turn left. If not, make it white, turn right. Move forward one and repeat for all ants. Turns out they're Turing complete. They're a Turing machine on a 2D surface where their state is their current position as well as direction. This is what it looks like for a single ant after about 11,000 iterations of this rule. As always, we start with a blank canvas. I'm leaving some of the mechanics out for brevity, but otherwise it looks the same. This reads pretty much like the rules. If it's white, set at black, turn left. If it's black, set at white, turn right. Finally, move. Here it is after only a few turns. This is it after about 6,000 iterations. Not bad ant farm. I really like it. Boidstance is an artificial life program which simulates the flocking behavior of birds, fish, or other systems. I love the fact that Wikipedia says this. Incidentally, Boidstance is also a New York metropolitan dialect pronunciation for bird. Boids use a combination of simple rules to determine flocking behavior. At a minimum, it has cohesion, separation, and alignment. More can be added, you can also have more added forces like wind or water for a more accurate simulation for whatever you're doing. We get off to the usual start. We declare a simulation with a white background and an FPS display because it was really slow to start. Then, register the boid body to be used. We'll have 25 boids and we'll initialize them with a random angle and speed. We'll declare that they get drawn as a simple black circle with a red line showing their vector. Boids try to fly towards the center, or rule number one, cohesion. Boids try to fly towards the center mass of nearby boids. That's pretty much exactly what this code says. Calculate the center mass, find the difference, and return 1% of that. Rule number two, separation. Boids try to keep a small distance away from others. This involves finding nearby boids that are too close and calculating a vector slightly away from them. Rule number three, alignment. Boids try to match the velocity of their nearby boids. This simply involves calculating the difference of the average velocity with their own. Now, each turn is just a matter of summing those vectors and adding them to the current velocity clamped to a limit. Finally, at the bottom, we move and wrap around the window. Here's the final result. Very quickly, the flocking behavior takes over and they converge while maintaining safe distance from each other. Some rules add up to very organic feeling behavior. Finally, games. Let's dissect that tank game from the gallery. I'm not going to implement the whole thing here for time, but I want to show how to do interactivity. Of course, we start with a sketch of what we want to do. Nothing fancy. Just a simple tank with an independent turret in an arena. So we sketch that up. We formulate the rules for how we want it to behave. For now, the tank is in the arena. Arrows let us move the tank. And S, turn the turret. Everything else, firing, et cetera, is an exercise for the viewer. As always, we start with a blank canvas. We initialize the tank to be in the middle. We have to add a turret for an angle independent of the body angle. Next, we draw our tank into a sprite. Pro tip. This is incredibly error-prone. So give each shape a different color until you figure all the coordinates out. Draw extra boundaries around the whole thing and draw diagonals to help you measure on stuff. Little things like that help a lot. Then we use blit to put the tank and the turret on the canvas. At this point, the game will start up and you can see the tank just sitting in the middle. That's boring because it can't do anything. So let's give the tank some behavior. The ability to turn the body and the turret and to move to accelerate or de-accelerate. Then once that's working correctly, we can go ahead and plug in those event handlers using add key handler. We declare the handlers and have them control the state of the tank. And here's our final result, for this talk at least. Obviously, this can be easily extended to fire bullets, have sound effects, multiple tanks, targets, scoring, et cetera. I just can't fit all of that into this talk. Sorry. So what are the patterns that we've seen so far? Start with a sketch. Figure out what you want to see. Figure out the behavior rules, if any. Start coding with a blank canvas and build up one small thing at a time. Don't try to do the whole thing at once. This is too much stuff. If you aren't sure, start explaining with drawing. Get your images, your sprites, your backgrounds, everything right. Move to simulation when you have more dynamic behavior. MVC is fantastic here. Model mostly body instances. Your view is all bodies have an external draw routine, stateless. And controller, simulation, running update and draw loop and maintaining the shared state. I think the declarative form works really well for graphical programming. So try to push towards doing less in your methods and using decorator modules instead. Just remember to always call super, otherwise they don't show up. Hopefully this talk has convinced you that you could benefit from added visualization tool in your toolbox and that graphics is a good fit for that. Thank you. The question is what lies underneath? And yes, it is a C library. It's called lib SDL. And the unreleased version of this code, which is eminent, has switched from SDL1 to SDL2, which is why it's so much more performant than it used to be. And is now pixel perfect colors, really good alpha blending. It's really improved. Is it possible to do this for M Ruby? I don't see why not. I don't know. But I'm going to say yes. This has a small C extension in it which provides the wrapper to SDL and so it should be just the same. What are the limitations that I currently see with it? I don't think there's enough decorators. I don't think there's enough drawing primitives. Josh uses this a lot and he's asked for things like thick lines instead of having to do it yourself, which is now possible with my switch to SDL2. The performance problems went away. I went from 30ish frames a second to 200 frames a second by switching to SDL2. And so I actually had to slow it down. Otherwise, the simulations are gone before you can see anything. And all the coloring stuff is better. Oh! Another thing that I don't like, and I don't have an example of this in this talk, I hate eyeballs and I hate brains. Color perception is really, really complex and difficult. It turns out magenta is not a real color and that crazy, wacky thing called brightness is really complex. So if you're wanting to do scientific visualizations where you're using, were you wanting to use the same level of intensity for your colors to differentiate, that's exceedingly hard. If you use the color system called HSV, hue saturation value, that supposedly lets you walk across the hues with the same saturation and value. But blue is darker than red and yellow and they all have the same saturation and value. And so you wind up with these extremely different versions of intensity. There is a color map, HCL, that tries to address color perception differences. It also means that as you're walking through the hue, some of those colors don't exist because they don't match the intensity. So you wind up with nil values and you wind up screwing up your thing. So I need to figure out how to address that. I don't know if I can. I hope I can, but I don't know. I'm still experimenting. I don't know if I understand your question. What do I suggest to tests in what? Okay. So what tests would I write while I'm doing one of these things? Probably test the crap out of your models. You don't have to test emergent behavior that will fall out on its own and I don't think that's something you need to write a test for. So I would test all of my models head to toe and then I would probably leave it at that. The way the controller works and the way the view code works, like I wouldn't write a test for the view method at all. If I did, it would probably be save off a bitmap and try to compare against a golden, but I hate golden tests. And then the controller, for the most part, is entirely declarative and I don't see the value in testing most declarative code. Can you do this headless? Totally. No, but I've been very, simulation is still available. I've been very seriously considering splitting the graphic stuff out to just graphics and having simulation being a separate gem so you can just do numeric stuff. Uh-oh, I'm at yellow. So there's a logging function in there where you can register a logger to run every 100 ticks or whatever and it would be really easy just to not create a window at all and have that go and have your routine do the update loop and log loop instead of an update and draw. I haven't broken that out yet, but it would be 10 lines of code to do it, change the initializer around to not do the window and have everything else be the same. And I can totally add that. Question is, is it possible to add new primitives? If you don't mind coding in C, it's totally possible. I'm using SDL GFX as an extender of the standard primitives available and I've wrapped all but 15 of them. I'm switching from SGE to GFX and I haven't wrapped everything yet so I'm going to get thick lines soon and I'm going to get some other stuff. The problem is that SGE also provided my collision detection bitmap comparator. Uh-oh. No, I got 30 more seconds to query on that. And I just haven't finished that. But if your primitives were done at the Ruby level, then yeah, it's just another method that you just throw into simulation. All the demos were just done in... So the question is, do I have other backgrounds? It defaults to black, the white decorator is there and I used it in all these slides because I was worried that I was going to have a bright room and a dim projector. But you can clear in any color you want. You can also blit a texture onto the background as well if you wanted to do a game. Yeah, yeah, totally. And I think that's all the time we have. So thank you very much.
 Hello!  Holy crap, it's packed.  I did not expect this many people for this type of talk.  Thank you so much for joining me today.  I'm Ryan Davis, and today I'm going to be talking about graphics and running simulations.  I should note that despite the title, there will be no Wizard of Oz references.  I'm very sorry about that, but that sets a level of expectation on my graphics prowess,  and I just don't have that.  So something I like to do in all my talks is to set expectations up front.  This is about half show and tell, and about half tutorial.  It's got a medium level of code compared to my normal talks, and it is suitable for all  levels of developer.  It's about 180 slides, which if I give some time for QN at the end, is about five slides  a minute.  But first, a little bit about me.  I've been coding professionally for a long time, 18 of those in Ruby.  I'm the founder of Seattle RAB, the first and oldest Ruby Brigade in the world.  I'm the author of many tests, FLOG, FLAY, Ruby parser, and a whole lot of other gems.  I figured out this year that I pushed over 1,000 gems into the Ruby ecosystem.  I'm a developer's developer.  I really like building tools that we use, and I run Seattle RAB Consulting at SeattleRAB.com.  I really hate these types of slides and these talks.  I'm really sorry about this, but I need new clients, and speaking is my primary lead generator.  So what is graphics, and why does it have such a boring name?  I was tired of nonsensical names like Miniskirt and Nokogiri, and both graphics and simulation  were available, and I literally flipped a coin and wound up with graphics.  Oh, hi, Aaron.  Quite simply, graphics is a gem that I wrote that you can install right now.  Unfortunately, it's still in beta, so you have to use dash dash pre.  I'm sorry about that.  I'm going to try to fix that today.  It tries to provide a simple framework to implement simulations, which I'll leave open  at this point.  Digital art, custom visualizations, and even games.  It's designed to follow grade school mathematical conventions, not game programming conventions,  and I'll get more into this later.  So I want to demystify graphical programming, generative art, simulation, scientific visualization,  simple games.  I want to show that these topics aren't actually hard.  Perhaps most importantly, though, I want to add a new tool to your toolbox, so you can  use it to add an extra dimension to your work or to understand a problem that you're having.  But first, a simple word of warning.  I am not good at art or graphical programming.  I'm aesthetically challenged, especially in electronic mediums.  I look at my website for proof.  I don't have the strongest math background, especially in linear algebra and other math  heavily used in graphics.  I'm not a scientist, and I'm not a game dev.  I will never write the next version of Doom.  This makes me a perfect candidate to try to write a graphics framework for the rest of  us, because I'm damn good at making tools to help me and other developers out.  So I've made this gem to be as useful and as powerful as possible, while at the same  time to be as simple as I can possibly get it.  But I prefer real examples over theory and marketing fluff, so let's quickly get concrete  with a real and tangible example.  Let's start with a blank window.  This is all it takes.  You require graphics.  You create a new class, and you subclass graphics, colon, colon, simulation, where G, colon,  colon, S is just slide shorthand, because nothing fit in this talk.  In the case of these talks, I'm going to be including a white background in all the classes,  because I wasn't quite sure how good the throw was going to be on the projector, and a thin  line on black might not come through.  And you instantiate it, and you tell it to run.  You wind up with a blank canvas that is defaulting to half the width and height of your actual  screen.  To actually draw something, we had a draw method.  We call it super, and then we declare a circle in the middle of the window with a radius  of 15.  We tell it to be blue and filled.  You see a circle, but there's no state or behavior with this setup.  For that, I want to make this a bit more complex.  I declare a body subclass called ball, and in that, I declare count to be one, and then  I create a class called view with a draw class method that takes the window and the ball  to draw.  The drawing code uses B.x and B.y, but as otherwise, very similar.  Then we get rid of the original draw method.  Then back in simulation, I add an initializer, and I declare that ball is going to be used  by calling register bodies with the result of populate ball.  It's not too much different.  Everything's just been displaced a little bit.  This allows for state and behavior, and this is when things start to get interesting.  We can add an initializer to ball, call super, and then set both the angle and magnitude  to random values.  The x and y of the body is already random via the super.  Now to add behavior, I want an update method to ball.  Is that running?  Interesting.  It is running, but it's not showing on my screen, so I need to stop looking at my screen.  For now, we're going to tell it to move and then wrap around as if necessary.  This is similar to how the classic game Asteroids works.  If we change from wrap to bounce zero, where zero means no friction, then we're going to  get behavior that's similar to the classic game POM.  If we added a small down vector to the ball's velocity and bounced with a default friction,  in other words, no zero, then we simulate how a real ball would bounce.  We've got three very different behaviors with three very small changes.  Finally, a simple one line change from count equals one to count equals 25 makes this much  more interesting dynamic example.  We've got a simulation class.  Declars a white background.  Declars that ball should be used.  That's all it does.  We've got a body named ball.  Populate should make 25 balls in the simulation.  We have a gravity constant, some random setup, and very basic behavior of balls bouncing  under constant gravity.  Finally, ball declares a view class that defines how to draw a ball in a stateless manner.  28 lines of code in the end.  It only looks complex because it's split across three slides to make this readable.  Here it is in its entirety, only barely formatted to fit on the slide, and it's not too bad,  in my opinion.  It's really, truly easy enough that anyone can do it.  I'll give you an idea of what graphics programming looks like with the graphics gem, but that's  only the tip of the iceberg.  Let's look at more.  Let's start with Jitter Divart.  Here's criss-crossing lines with random Jitter added to their X, Ys.  They're very, very lightly colored, translucent black, but there's 300 of them, so they quickly  become solid dark black.  This is the exact same code with slightly tweaked behavior.  Instead of criss-crossing the lines, each segment is jittered away from the previous.  This creates a fountain or jet effect.  This is a complex wave pattern.  What's cool is that this is not 3D math.  It's actually really simple underneath.  It's just emergent behavior looks 3D.  This is contributed by Josh Cheek, who's been posting amazing videos using graphics, and  he's got a lot more graphical skill than I do.  Here's another piece by Josh Cheek.  I just can't get over what he's done with the graphics gem.  It kills me.  These are set to music, mine or not.  Sorry.  This is an emulation of Piet Mondrian's artwork with animations added, which we'll see more  of later.  Finally, I went to an art exhibit a couple months back where several of the pieces actually  showed their maths.  I took a picture of this one and I translated it to Ruby.  It's only 42 lines of code.  God, I love that.  Let's look at some more math-oriented examples.  I hope this reads.  Oh, it reads fine.  That's great.  This is a very simple proof of concept that you could implement the logo programming system  in Ruby.  This is taking a terminal input to a drawing turtle to draw a square.  This is a standard fractal tree.  This one was contributed by Justin Collins.  This is a simple visualization of quadrant one maths with a polynomial.  Often special, really simple.  This also reads really well.  This projector is fantastic.  I want to buy one.  This simulates bouncing balls expanding a polygon to calculate the value of pi.  It's hard to see with the current estimate of pi is up in the Windows title, but it only  gets to 3.13 by the time the video loops.  It does hit 3.14 at some point.  Now the part that fascinates me the most, simulations.  Here's a more complete version of the bouncing balls example that I gave before.  Two very different versions of Conway's Game of Life.  Boids or a flocking simulation, which we'll see more of later.  Two different versions of fluid dynamic simulations.  These use the same maths entirely, but they use different visualizations to show the current  density.  A colored gamut is being used on the right, and then you can see little spheres as they  hit the bottom and the density grows.  Canvas drawing of the Vance or virtual ants algorithm, which we'll see more of later.  This might be a little hard to see, but this is many, oh no, it looks great.  I need to stop saying that.  Many randomly moving bodies with collision detection handling, trails, so you can see  their paths, and random goal changing behavior.  The reason why I got into this in the first place, I wanted to write a zombie outbreak  epidemic simulator.  This shows one of the very few cases where the priests win and kill all the zombies.  While it isn't an explicit goal of the gem, it is totally possible to implement games.  This example shows the use of collision maps and sprites.  This is a vector-based tank that is driven by the user poorly because I'm driving it.  This is a bitmap version with slightly more sophisticated implementation.  This is where I started to switch to the Model View Controller setup that I showed you before.  It even has pew-pew noises when you shoot.  This shows user interaction using the mouse, and finally, random maze generation using James  Buck's Maze's for programmers book.  He had a really good talk in Salt Lake a couple years back, and I was enthused by it, so I  implemented it during his talk.  Graphics.  I'd like to take a moment to talk about how graphics feels different to me to emphasize  my goals that I've been working on while I've been working on this.  I want to optimize this to be approachable.  I want to ideally target workshops for beginners, other non-computer science folk, and other  non-game devs.  I want this to be undemanding and easy to install.  There are platform-specific external dependencies, and I'm sorry for that, but it's the only  thing that makes this performant.  I want the graphics gem to be a single gem install and work like any other gem.  I've removed and normalized as much as I can to make it usable to anyone with any background.  It is optimized to get the first pixels as quickly as possible, to see your code with  as little boilerplate as possible.  Any new developers, quick and easy feedback is very empowering.  While there's no in-pros editing yet, single file is all it takes to experiment.  It's simply a matter of edit and run.  I want this to be any other script.  I don't want it to be an app, all our Rails, or other large frameworks.  Wherever possible, graphics uses real maths.  Unless you need to use the computer's built-in trig functions, which are always specified  in radians, everything in graphics uses degrees.  It uses quadrant one maths, just like you used in grade school, unlike game math coordinates,  which looks like quadrant four, but for some reason, why is still positive, even though  it's going down?  It uses the right-hand rule, where zero degrees is to the east, 90 to the north, et cetera,  again, just like grade school.  These allow simple things to work as expected.  It means that all the stuff that you were taught in grade school still works as intended.  It makes one less thing that you have to adjust in your brain when implementing your simulation.  Graphics is opinionated.  It wants to be pretty.  There are over 20 graphic primitives available in graphics with more to come.  Everything is anti-aliased by default.  Alpha blending is automatic based on the color that you're drawing.  There are plenty of helpers to make a code as clean as possible, probably more to come.  Let's start to talk about how this works.  First we'll talk about the update and draw loop.  At the highest level possible, we have a model, which is the state of your simulation.  Each turn that gets drawn or rendered into a canvas, and that in turn gets copied to  the actual screen.  As such, our loop is really just three calls.  Update or change the model for this turn, draw, which renders the model onto a canvas,  present, finally copy that canvas to the screen.  As an aside, this is what I want things to feel like.  On every turn, you have a new blank canvas.  You draw some things, then you copy them to the screen.  Under the covers, there might be two or more of those canvases, and they take turns updating  and copying to the screen.  These are details you don't really need to know most of the time, but they can trip  you up on occasion.  There's other stuff involved, but the basic run loop works just like this.  Update end times, where N is usually one.  Draw, then render to the screen.  Update by default tells all registered bodies to update themselves.  The default draw phase clears the canvas, then tells each registered body to render  itself via that body's view class.  It's a bit more than this, but this is the gist.  And finally, the render phase calls down to the lower layers, the graphics engine, to  do the stuff and make it actually display on the screen.  There's a very basic class hierarchy.  Abstract simulation has nearly all the meat in it.  Anything, I should just rename it to simulation.  Graphics simulation is the class you're going to use 80% of the time in the end.  It has all the normal functionality I've described so far.  If anything, you do changes over time, this is probably the class you want to start with.  On each turn, it clears the window by default, and it uses very smooth drawing, and by that,  I mean it uses that double buffering under the covers.  The other 20% of the time, you might want to use graphics drawing.  It does not clear the canvas on each turn, and it uses a single canvas that is persistent  throughout all turns, and it uses single buffered drawing from that one canvas.  It's static, so it's meant to draw things that are either entirely static at the start,  they never change, or they're additive over time.  This makes it good for plots and simulations or art where the canvas itself is part of the  state.  Okay, that's well and good, but what can you actually do?  How do you draw things?  You can pick at each one.  For the sake of making these slides readable, I've defined these following values and methods  to use in the examples.  We have default margin of 50, random X and Y, a random color from all the registered colors  that it defaults to have, a random bool which you won't really tell, but I've biased it  towards false.  For things that need multiple different values, I've got a R50 method that just gets a random  50.  So, clear, not much to say here.  First thing we almost always want to do on each turn is to clear the screen, but this  happens by default when you use simulation.  Point is the most basic building block, but not necessarily something you're going to  directly use a lot.  Here we're making a thousand random points on each turn with random colors, and it just  basically becomes this noise.  Line is pretty straightforward as well, with two pairs of X, Y values and a color.  Here we're using the window height and the current turn number to animate.  Angles like line, but you specify a single X, Y, and then an angle and a length instead  of a second point.  H line and V line take one coordinate and then span the whole window with your specified  color.  This is useful for plotting stuff.  Erin, do not air drop me during the talk.  I don't even have a mouse right now, jackass.  Circle takes a middle point and a radius.  This is the first primitive with an area, so it also takes a Boolean declaring whether  or not it's built.  Ellipse is just like circle, but it takes two radius values.  Fast rect is a filled rect, always filled, taking an origin point on the bottom left,  a width and a height and a color.  It uses the lower level drawing function that should be faster, but these days it probably  isn't anymore because everything is terribly fast.  Too fast, in fact.  I had to add v-sync to make things viewable.  Rect is the same thing as filled rect, but it also takes a Boolean for whether or not  it's filled.  Polygon takes any number of points in a color.  I probably need to extend this to allow for a filled polygon, but no one's asked for  that yet.  I always feel conflicted about this one.  Bezier allows you to draw curves, but not in a terribly approachable way.  You need to be more of a graphic step for this.  I'm not going to explain this one too much.  I want to add splines or something else that I find easier to use.  This one's busy.  It's one of the busiest slides I've got.  Sprite, that call up top, takes dimensions and a block.  During the block execution, drawing goes into a separate canvas and that image is returned.  Blit takes that image or any image and draws it at a location with an angle and possibly  optionally scale.  Instead of using sprite to create a canvas, you can call image to load a bitmap from a  file instead.  Render text takes a string, a color, an optional font.  There's always a default font.  And returns an image.  Put is just like Blit, except that it uses the bottom left corner instead of the center  as its origin.  Here you can see it rotating on that corner, unlike the tank and turret from before.  Text renders, this has been opened, and draws a string at the specified point with a specified  color.  Okay.  Finally, there's some helper functions that render to the screen.  FPS prints the frames per second.  It's a bad estimate, but it's close enough.  Text returns the mouse date and I use that with the debug function that takes a format  string and arguments to it and prints it.  For debugging at the pixel level or just for fun, you can save the PNG of the current screen.  And that's it.  So let's build up some real examples, starting with generative art.  I throw pottery.  I enjoy it.  It's fun to struggle with.  I don't feel the same way about computer art.  It's a struggle for me and I don't find that struggle fun.  So I had an interview a few months back where they asked me to generate art in the style  of Pitt Mondrian.  I don't have time to go on about Mondrian.  I could do that for a long time, but you should check out his work.  It's really good.  So normally I'd start with a hand-drawn sketch of what we're going for, but here we actually  have a finished product to emulate.  This is his actual work.  So we start where we always start, the blank canvas.  I've added a grid to help visualize where we want to go.  There's stuff here I won't explain.  It's mostly to make the slides more readable.  Let's focus on the art method.  So let's just think about where we want to go.  We want thick lines that span the window.  So we always have the width and height of the window, so let's just put two down.  First vertically at 120 and next horizontally at 80.  And W is the value 20, which is the width of the lines that we're going to make.  Next let's use some helpers to generate a random number of grid aligned Xs and Ys, and  then draw them using the previous code.  It's really not much different except there's some loops.  This looks good.  Now we just need to fill in some regions.  And as an aside, I just want to say, like, I love the fact that Ruby RAN could take  number ranges.  It makes code so much prettier than the way we used to do this shit and see offsets and  adding and all that crap.  So this is gross, I admit.  I pulled it out and it's on method for this slide.  Basically we randomly picked some regions based on the Xs and Ys that were previously  generated.  And then we find the area of those regions and we fill with a random primary color.  What is going on next door?  That's amazing.  I hope this comes through on the audio for the recording.  Honestly, that's not bad.  And yet they still didn't offer me the job.  Or did they offer me any feedback on how I could improve?  So I wrote a blog post about it and published the interview question.  NRI, you should treat your interviews better.  Okay, slightly more relevant to your job.  Let's work out some basic data visualization.  Let's say you have a lot of data.  You don't know what that data means yet.  This is a bit of a bullshit example, but gathering data and plotting something like message size  over time spent to process.  As Rubyists, we're incredibly good at gathering data, whether that's parsing it out of text  or slapping it from a database or whatever.  Getting the data is never a real problem that we've ever had.  So I'm not going to touch how that data is gathered.  Just assume that I have it.  And looking at a million data points, just under a million data points, and poking at  it the way we would normally poke at it doesn't help.  We need a way to make some sense of it.  There might be a way to visualize it and see if there's a visible correlation.  But doing something with that data, something visual, what do we as Rubyists do?  Normally at this point, someone would probably suggest grabbing a whole new language like  R.  It's very good at importing and manipulating data.  But it's also a pain in the ass language and overkill if you're not doing a lot of real  stats.  Or maybe someone else suggests using Excel, it has great plotting capabilities.  But the data that you're dealing with is really, really big and getting it into a spreadsheet  is a pain.  Now you have two problems.  We don't have many good options in Ruby for this.  We grab for tools in other languages like R or D3 and JavaScript.  And that keeps us from improving our own tooling.  I think that's a problem.  So let's try to address this a bit.  I'm not solving this problem.  I'm just addressing it.  So let's try to do this ourselves playing to our strengths.  We always start with a sketch of our system.  That didn't come through very well, but whatever.  Basically we're going to do a very basic plot, trying to show time over size to see where  any problems might be.  When I say sketch, I mean it.  This is why this is a hand drawn.  So as usual, we start with a very basic subclass of simulation to make a blank window.  This time I've declared a grid to help.  It's change initialized to store off the data being passed in.  We scale it to the size of the window, which is max, which is set to 500.  Then we use group i to gather and count the same values.  And we apply math log to keep the values sane.  Otherwise everything is just a big black screen.  So now we can add the draw method.  We simply walk over the data.  We plot a gray point at x, y.  I also plot a translucent red circle at that point with the log count as the radius.  This will start to show where there's lots of spillover.  If you zoom in, you can see that there's very light red is heavily spilling over in  some areas.  These are areas where you might want to look for trouble.  There's huge amount of variance in those columns.  Now we know where to look.  Have fun with it.  Add features.  All those axes, put the values in there, make it easier to repeat later.  This might not be the right data to visualize, though, or it might not be the right way to  visualize it.  You need to experiment in order to suss out information from raw data.  Running simulations is why I wrote this library.  Here are some simple examples on how to do that.  Langston's ants, or vants, which is short for virtual ants.  They live on a 2D plane and have a simple set of rules that they execute every turn.  If an ant is on a white cell, make a black, turn left.  If not, make it white, turn right.  Move forward one and repeat for all ants.  Turns out they're Turing complete.  They're a Turing machine on a 2D surface where their state is their current position as well  as direction.  This is what it looks like for a single ant after about 11,000 iterations of this rule.  As always, we start with a blank canvas.  I'm leaving some of the mechanics out for brevity, but otherwise it looks the same.  This reads pretty much like the rules.  If it's white, set at black, turn left.  If it's black, set at white, turn right.  Finally, move.  Here it is after only a few turns.  This is it after about 6,000 iterations.  Not bad ant farm.  I really like it.  Boidstance is an artificial life program which simulates the flocking behavior of birds,  fish, or other systems.  I love the fact that Wikipedia says this.  Incidentally, Boidstance is also a New York metropolitan dialect pronunciation for bird.  Boids use a combination of simple rules to determine flocking behavior.  At a minimum, it has cohesion, separation, and alignment.  More can be added, you can also have more added forces like wind or water for a more  accurate simulation for whatever you're doing.  We get off to the usual start.  We declare a simulation with a white background and an FPS display because it was really slow  to start.  Then, register the boid body to be used.  We'll have 25 boids and we'll initialize them with a random angle and speed.  We'll declare that they get drawn as a simple black circle with a red line showing their  vector.  Boids try to fly towards the center, or rule number one, cohesion.  Boids try to fly towards the center mass of nearby boids.  That's pretty much exactly what this code says.  Calculate the center mass, find the difference, and return 1% of that.  Rule number two, separation.  Boids try to keep a small distance away from others.  This involves finding nearby boids that are too close and calculating a vector slightly  away from them.  Rule number three, alignment.  Boids try to match the velocity of their nearby boids.  This simply involves calculating the difference of the average velocity with their own.  Now, each turn is just a matter of summing those vectors and adding them to the current  velocity clamped to a limit.  Finally, at the bottom, we move and wrap around the window.  Here's the final result.  Very quickly, the flocking behavior takes over and they converge while maintaining safe  distance from each other.  Some rules add up to very organic feeling behavior.  Finally, games.  Let's dissect that tank game from the gallery.  I'm not going to implement the whole thing here for time, but I want to show how to do  interactivity.  Of course, we start with a sketch of what we want to do.  Nothing fancy.  Just a simple tank with an independent turret in an arena.  So we sketch that up.  We formulate the rules for how we want it to behave.  For now, the tank is in the arena.  Arrows let us move the tank.  And S, turn the turret.  Everything else, firing, et cetera, is an exercise for the viewer.  As always, we start with a blank canvas.  We initialize the tank to be in the middle.  We have to add a turret for an angle independent of the body angle.  Next, we draw our tank into a sprite.  Pro tip.  This is incredibly error-prone.  So give each shape a different color until you figure all the coordinates out.  Draw extra boundaries around the whole thing and draw diagonals to help you measure on  stuff.  Little things like that help a lot.  Then we use blit to put the tank and the turret on the canvas.  At this point, the game will start up and you can see the tank just sitting in the middle.  That's boring because it can't do anything.  So let's give the tank some behavior.  The ability to turn the body and the turret and to move to accelerate or de-accelerate.  Then once that's working correctly, we can go ahead and plug in those event handlers  using add key handler.  We declare the handlers and have them control the state of the tank.  And here's our final result, for this talk at least.  Obviously, this can be easily extended to fire bullets, have sound effects, multiple tanks,  targets, scoring, et cetera.  I just can't fit all of that into this talk.  Sorry.  So what are the patterns that we've seen so far?  Start with a sketch.  Figure out what you want to see.  Figure out the behavior rules, if any.  Start coding with a blank canvas and build up one small thing at a time.  Don't try to do the whole thing at once.  This is too much stuff.  If you aren't sure, start explaining with drawing.  Get your images, your sprites, your backgrounds, everything right.  Move to simulation when you have more dynamic behavior.  MVC is fantastic here.  Model mostly body instances.  Your view is all bodies have an external draw routine, stateless.  And controller, simulation, running update and draw loop and maintaining the shared state.  I think the declarative form works really well for graphical programming.  So try to push towards doing less in your methods and using decorator modules instead.  Just remember to always call super, otherwise they don't show up.  Hopefully this talk has convinced you that you could benefit from added visualization  tool in your toolbox and that graphics is a good fit for that.  Thank you.  The question is what lies underneath?  And yes, it is a C library.  It's called lib SDL.  And the unreleased version of this code, which is eminent, has switched from SDL1 to SDL2,  which is why it's so much more performant than it used to be.  And is now pixel perfect colors, really good alpha blending.  It's really improved.  Is it possible to do this for M Ruby?  I don't see why not.  I don't know.  But I'm going to say yes.  This has a small C extension in it which provides the wrapper to SDL and so it should be just  the same.  What are the limitations that I currently see with it?  I don't think there's enough decorators.  I don't think there's enough drawing primitives.  Josh uses this a lot and he's asked for things like thick lines instead of having to do it  yourself, which is now possible with my switch to SDL2.  The performance problems went away.  I went from 30ish frames a second to 200 frames a second by switching to SDL2.  And so I actually had to slow it down.  Otherwise, the simulations are gone before you can see anything.  And all the coloring stuff is better.  Oh!  Another thing that I don't like, and I don't have an example of this in this talk, I hate  eyeballs and I hate brains.  Color perception is really, really complex and difficult.  It turns out magenta is not a real color and that crazy, wacky thing called brightness  is really complex.  So if you're wanting to do scientific visualizations where you're using, were you wanting to use  the same level of intensity for your colors to differentiate, that's exceedingly hard.  If you use the color system called HSV, hue saturation value, that supposedly lets you  walk across the hues with the same saturation and value.  But blue is darker than red and yellow and they all have the same saturation and value.  And so you wind up with these extremely different versions of intensity.  There is a color map, HCL, that tries to address color perception differences.  It also means that as you're walking through the hue, some of those colors don't exist  because they don't match the intensity.  So you wind up with nil values and you wind up screwing up your thing.  So I need to figure out how to address that.  I don't know if I can.  I hope I can, but I don't know.  I'm still experimenting.  I don't know if I understand your question.  What do I suggest to tests in what?  Okay.  So what tests would I write while I'm doing one of these things?  Probably test the crap out of your models.  You don't have to test emergent behavior that will fall out on its own and I don't think  that's something you need to write a test for.  So I would test all of my models head to toe and then I would probably leave it at that.  The way the controller works and the way the view code works, like I wouldn't write a test  for the view method at all.  If I did, it would probably be save off a bitmap and try to compare against a golden,  but I hate golden tests.  And then the controller, for the most part, is entirely declarative and I don't see the  value in testing most declarative code.  Can you do this headless?  Totally.  No, but I've been very, simulation is still available.  I've been very seriously considering splitting the graphic stuff out to just graphics and  having simulation being a separate gem so you can just do numeric stuff.  Uh-oh, I'm at yellow.  So there's a logging function in there where you can register a logger to run every 100  ticks or whatever and it would be really easy just to not create a window at all and have  that go and have your routine do the update loop and log loop instead of an update and  draw.  I haven't broken that out yet, but it would be 10 lines of code to do it, change the initializer  around to not do the window and have everything else be the same.  And I can totally add that.  Question is, is it possible to add new primitives?  If you don't mind coding in C, it's totally possible.  I'm using SDL GFX as an extender of the standard primitives available and I've wrapped all  but 15 of them.  I'm switching from SGE to GFX and I haven't wrapped everything yet so I'm going to get  thick lines soon and I'm going to get some other stuff.  The problem is that SGE also provided my collision detection bitmap comparator.  Uh-oh.  No, I got 30 more seconds to query on that.  And I just haven't finished that.  But if your primitives were done at the Ruby level, then yeah, it's just another method  that you just throw into simulation.  All the demos were just done in...  So the question is, do I have other backgrounds?  It defaults to black, the white decorator is there and I used it in all these slides  because I was worried that I was going to have a bright room and a dim projector.  But you can clear in any color you want.  You can also blit a texture onto the background as well if you wanted to do a game.  Yeah, yeah, totally.  And I think that's all the time we have.  So thank you very much.
RubyConf_2018_-_Graphics_and_Simulations__and_Game_5KVcsV_jseQ_0_59.png RubyConf_2018_-_Graphics_and_Simulations__and_Game_5KVcsV_jseQ_60_155.png RubyConf_2018_-_Graphics_and_Simulations__and_Game_5KVcsV_jseQ_156_971.png RubyConf_2018_-_Graphics_and_Simulations__and_Game_5KVcsV_jseQ_972_3311.png RubyConf_2018_-_Graphics_and_Simulations__and_Game_5KVcsV_jseQ_3312_3599.png RubyConf_2018_-_Graphics_and_Simulations__and_Game_5KVcsV_jseQ_3600_5027.png RubyConf_2018_-_Graphics_and_Simulations__and_Game_5KVcsV_jseQ_5028_10547.png RubyConf_2018_-_Graphics_and_Simulations__and_Game_5KVcsV_jseQ_10548_11207.png RubyConf_2018_-_Graphics_and_Simulations__and_Game_5KVcsV_jseQ_11208_12167.png RubyConf_2018_-_Graphics_and_Simulations__and_Game_5KVcsV_jseQ_12168_12251.png RubyConf_2018_-_Graphics_and_Simulations__and_Game_5KVcsV_jseQ_12252_15059.png RubyConf_2018_-_Graphics_and_Simulations__and_Game_5KVcsV_jseQ_15060_15431.png RubyConf_2018_-_Graphics_and_Simulations__and_Game_5KVcsV_jseQ_15432_15515.png RubyConf_2018_-_Graphics_and_Simulations__and_Game_5KVcsV_jseQ_15516_15659.png RubyConf_2018_-_Graphics_and_Simulations__and_Game_5KVcsV_jseQ_15660_18911.png RubyConf_2018_-_Graphics_and_Simulations__and_Game_5KVcsV_jseQ_18912_19715.png RubyConf_2018_-_Graphics_and_Simulations__and_Game_5KVcsV_jseQ_19716_21407.png RubyConf_2018_-_Graphics_and_Simulations__and_Game_5KVcsV_jseQ_21408_22679.png RubyConf_2018_-_Graphics_and_Simulations__and_Game_5KVcsV_jseQ_22680_22811.png RubyConf_2018_-_Graphics_and_Simulations__and_Game_5KVcsV_jseQ_22812_28811.png RubyConf_2018_-_Graphics_and_Simulations__and_Game_5KVcsV_jseQ_28812_32507.png RubyConf_2018_-_Graphics_and_Simulations__and_Game_5KVcsV_jseQ_32508_33539.png RubyConf_2018_-_Graphics_and_Simulations__and_Game_5KVcsV_jseQ_33540_34859.png RubyConf_2018_-_Graphics_and_Simulations__and_Game_5KVcsV_jseQ_34860_36779.png RubyConf_2018_-_Graphics_and_Simulations__and_Game_5KVcsV_jseQ_36780_37967.png RubyConf_2018_-_Graphics_and_Simulations__and_Game_5KVcsV_jseQ_37968_38123.png RubyConf_2018_-_Graphics_and_Simulations__and_Game_5KVcsV_jseQ_38124_38315.png RubyConf_2018_-_Graphics_and_Simulations__and_Game_5KVcsV_jseQ_38316_38591.png RubyConf_2018_-_Graphics_and_Simulations__and_Game_5KVcsV_jseQ_38592_39587.png RubyConf_2018_-_Graphics_and_Simulations__and_Game_5KVcsV_jseQ_39588_45911.png RubyConf_2018_-_Graphics_and_Simulations__and_Game_5KVcsV_jseQ_45912_47219.png RubyConf_2018_-_Graphics_and_Simulations__and_Game_5KVcsV_jseQ_47220_52667.png RubyConf_2018_-_Graphics_and_Simulations__and_Game_5KVcsV_jseQ_52668_56603.png RubyConf_2018_-_Graphics_and_Simulations__and_Game_5KVcsV_jseQ_56604_56747.png RubyConf_2018_-_Graphics_and_Simulations__and_Game_5KVcsV_jseQ_56748_56913.png
Hello! Holy crap, it's packed. I did not expect this many people for this type of talk. Thank you so much for joining me today. I'm Ryan Davis, and today I'm going to be talking about graphics and running simulations. I should note that despite the title, there will be no Wizard of Oz references. I'm very sorry about that, but that sets a level of expectation on my graphics prowess, and I just don't have that. So something I like to do in all my talks is to set expectations up front. This is about half show and tell, and about half tutorial. It's got a medium level of code compared to my normal talks, and it is suitable for all levels of developer. It's about 180 slides, which if I give some time for QN at the end, is about five slides a minute. But first, a little bit about me. I've been coding professionally for a long time, 18 of those in Ruby. I'm the founder of Seattle RAB, the first and oldest Ruby Brigade in the world. I'm the author of many tests, FLOG, FLAY, Ruby parser, and a whole lot of other gems. I figured out this year that I pushed over 1,000 gems into the Ruby ecosystem. I'm a developer's developer. I really like building tools that we use, and I run Seattle RAB Consulting at SeattleRAB.com. I really hate these types of slides and these talks. I'm really sorry about this, but I need new clients, and speaking is my primary lead generator. So what is graphics, and why does it have such a boring name? I was tired of nonsensical names like Miniskirt and Nokogiri, and both graphics and simulation were available, and I literally flipped a coin and wound up with graphics. Oh, hi, Aaron. Quite simply, graphics is a gem that I wrote that you can install right now. Unfortunately, it's still in beta, so you have to use dash dash pre. I'm sorry about that. I'm going to try to fix that today. It tries to provide a simple framework to implement simulations, which I'll leave open at this point. Digital art, custom visualizations, and even games. It's designed to follow grade school mathematical conventions, not game programming conventions, and I'll get more into this later. So I want to demystify graphical programming, generative art, simulation, scientific visualization, simple games. I want to show that these topics aren't actually hard. Perhaps most importantly, though, I want to add a new tool to your toolbox, so you can use it to add an extra dimension to your work or to understand a problem that you're having. But first, a simple word of warning. I am not good at art or graphical programming. I'm aesthetically challenged, especially in electronic mediums. I look at my website for proof. I don't have the strongest math background, especially in linear algebra and other math heavily used in graphics. I'm not a scientist, and I'm not a game dev. I will never write the next version of Doom. This makes me a perfect candidate to try to write a graphics framework for the rest of us, because I'm damn good at making tools to help me and other developers out. So I've made this gem to be as useful and as powerful as possible, while at the same time to be as simple as I can possibly get it. But I prefer real examples over theory and marketing fluff, so let's quickly get concrete with a real and tangible example. Let's start with a blank window. This is all it takes. You require graphics. You create a new class, and you subclass graphics, colon, colon, simulation, where G, colon, colon, S is just slide shorthand, because nothing fit in this talk. In the case of these talks, I'm going to be including a white background in all the classes, because I wasn't quite sure how good the throw was going to be on the projector, and a thin line on black might not come through. And you instantiate it, and you tell it to run. You wind up with a blank canvas that is defaulting to half the width and height of your actual screen. To actually draw something, we had a draw method. We call it super, and then we declare a circle in the middle of the window with a radius of 15. We tell it to be blue and filled. You see a circle, but there's no state or behavior with this setup. For that, I want to make this a bit more complex. I declare a body subclass called ball, and in that, I declare count to be one, and then I create a class called view with a draw class method that takes the window and the ball to draw. The drawing code uses B.x and B.y, but as otherwise, very similar. Then we get rid of the original draw method. Then back in simulation, I add an initializer, and I declare that ball is going to be used by calling register bodies with the result of populate ball. It's not too much different. Everything's just been displaced a little bit. This allows for state and behavior, and this is when things start to get interesting. We can add an initializer to ball, call super, and then set both the angle and magnitude to random values. The x and y of the body is already random via the super. Now to add behavior, I want an update method to ball. Is that running? Interesting. It is running, but it's not showing on my screen, so I need to stop looking at my screen. For now, we're going to tell it to move and then wrap around as if necessary. This is similar to how the classic game Asteroids works. If we change from wrap to bounce zero, where zero means no friction, then we're going to get behavior that's similar to the classic game POM. If we added a small down vector to the ball's velocity and bounced with a default friction, in other words, no zero, then we simulate how a real ball would bounce. We've got three very different behaviors with three very small changes. Finally, a simple one line change from count equals one to count equals 25 makes this much more interesting dynamic example. We've got a simulation class. Declars a white background. Declars that ball should be used. That's all it does. We've got a body named ball. Populate should make 25 balls in the simulation. We have a gravity constant, some random setup, and very basic behavior of balls bouncing under constant gravity. Finally, ball declares a view class that defines how to draw a ball in a stateless manner. 28 lines of code in the end. It only looks complex because it's split across three slides to make this readable. Here it is in its entirety, only barely formatted to fit on the slide, and it's not too bad, in my opinion. It's really, truly easy enough that anyone can do it. I'll give you an idea of what graphics programming looks like with the graphics gem, but that's only the tip of the iceberg. Let's look at more. Let's start with Jitter Divart. Here's criss-crossing lines with random Jitter added to their X, Ys. They're very, very lightly colored, translucent black, but there's 300 of them, so they quickly become solid dark black. This is the exact same code with slightly tweaked behavior. Instead of criss-crossing the lines, each segment is jittered away from the previous. This creates a fountain or jet effect. This is a complex wave pattern. What's cool is that this is not 3D math. It's actually really simple underneath. It's just emergent behavior looks 3D. This is contributed by Josh Cheek, who's been posting amazing videos using graphics, and he's got a lot more graphical skill than I do. Here's another piece by Josh Cheek. I just can't get over what he's done with the graphics gem. It kills me. These are set to music, mine or not. Sorry. This is an emulation of Piet Mondrian's artwork with animations added, which we'll see more of later. Finally, I went to an art exhibit a couple months back where several of the pieces actually showed their maths. I took a picture of this one and I translated it to Ruby. It's only 42 lines of code. God, I love that. Let's look at some more math-oriented examples. I hope this reads. Oh, it reads fine. That's great. This is a very simple proof of concept that you could implement the logo programming system in Ruby. This is taking a terminal input to a drawing turtle to draw a square. This is a standard fractal tree. This one was contributed by Justin Collins. This is a simple visualization of quadrant one maths with a polynomial. Often special, really simple. This also reads really well. This projector is fantastic. I want to buy one. This simulates bouncing balls expanding a polygon to calculate the value of pi. It's hard to see with the current estimate of pi is up in the Windows title, but it only gets to 3.13 by the time the video loops. It does hit 3.14 at some point. Now the part that fascinates me the most, simulations. Here's a more complete version of the bouncing balls example that I gave before. Two very different versions of Conway's Game of Life. Boids or a flocking simulation, which we'll see more of later. Two different versions of fluid dynamic simulations. These use the same maths entirely, but they use different visualizations to show the current density. A colored gamut is being used on the right, and then you can see little spheres as they hit the bottom and the density grows. Canvas drawing of the Vance or virtual ants algorithm, which we'll see more of later. This might be a little hard to see, but this is many, oh no, it looks great. I need to stop saying that. Many randomly moving bodies with collision detection handling, trails, so you can see their paths, and random goal changing behavior. The reason why I got into this in the first place, I wanted to write a zombie outbreak epidemic simulator. This shows one of the very few cases where the priests win and kill all the zombies. While it isn't an explicit goal of the gem, it is totally possible to implement games. This example shows the use of collision maps and sprites. This is a vector-based tank that is driven by the user poorly because I'm driving it. This is a bitmap version with slightly more sophisticated implementation. This is where I started to switch to the Model View Controller setup that I showed you before. It even has pew-pew noises when you shoot. This shows user interaction using the mouse, and finally, random maze generation using James Buck's Maze's for programmers book. He had a really good talk in Salt Lake a couple years back, and I was enthused by it, so I implemented it during his talk. Graphics. I'd like to take a moment to talk about how graphics feels different to me to emphasize my goals that I've been working on while I've been working on this. I want to optimize this to be approachable. I want to ideally target workshops for beginners, other non-computer science folk, and other non-game devs. I want this to be undemanding and easy to install. There are platform-specific external dependencies, and I'm sorry for that, but it's the only thing that makes this performant. I want the graphics gem to be a single gem install and work like any other gem. I've removed and normalized as much as I can to make it usable to anyone with any background. It is optimized to get the first pixels as quickly as possible, to see your code with as little boilerplate as possible. Any new developers, quick and easy feedback is very empowering. While there's no in-pros editing yet, single file is all it takes to experiment. It's simply a matter of edit and run. I want this to be any other script. I don't want it to be an app, all our Rails, or other large frameworks. Wherever possible, graphics uses real maths. Unless you need to use the computer's built-in trig functions, which are always specified in radians, everything in graphics uses degrees. It uses quadrant one maths, just like you used in grade school, unlike game math coordinates, which looks like quadrant four, but for some reason, why is still positive, even though it's going down? It uses the right-hand rule, where zero degrees is to the east, 90 to the north, et cetera, again, just like grade school. These allow simple things to work as expected. It means that all the stuff that you were taught in grade school still works as intended. It makes one less thing that you have to adjust in your brain when implementing your simulation. Graphics is opinionated. It wants to be pretty. There are over 20 graphic primitives available in graphics with more to come. Everything is anti-aliased by default. Alpha blending is automatic based on the color that you're drawing. There are plenty of helpers to make a code as clean as possible, probably more to come. Let's start to talk about how this works. First we'll talk about the update and draw loop. At the highest level possible, we have a model, which is the state of your simulation. Each turn that gets drawn or rendered into a canvas, and that in turn gets copied to the actual screen. As such, our loop is really just three calls. Update or change the model for this turn, draw, which renders the model onto a canvas, present, finally copy that canvas to the screen. As an aside, this is what I want things to feel like. On every turn, you have a new blank canvas. You draw some things, then you copy them to the screen. Under the covers, there might be two or more of those canvases, and they take turns updating and copying to the screen. These are details you don't really need to know most of the time, but they can trip you up on occasion. There's other stuff involved, but the basic run loop works just like this. Update end times, where N is usually one. Draw, then render to the screen. Update by default tells all registered bodies to update themselves. The default draw phase clears the canvas, then tells each registered body to render itself via that body's view class. It's a bit more than this, but this is the gist. And finally, the render phase calls down to the lower layers, the graphics engine, to do the stuff and make it actually display on the screen. There's a very basic class hierarchy. Abstract simulation has nearly all the meat in it. Anything, I should just rename it to simulation. Graphics simulation is the class you're going to use 80% of the time in the end. It has all the normal functionality I've described so far. If anything, you do changes over time, this is probably the class you want to start with. On each turn, it clears the window by default, and it uses very smooth drawing, and by that, I mean it uses that double buffering under the covers. The other 20% of the time, you might want to use graphics drawing. It does not clear the canvas on each turn, and it uses a single canvas that is persistent throughout all turns, and it uses single buffered drawing from that one canvas. It's static, so it's meant to draw things that are either entirely static at the start, they never change, or they're additive over time. This makes it good for plots and simulations or art where the canvas itself is part of the state. Okay, that's well and good, but what can you actually do? How do you draw things? You can pick at each one. For the sake of making these slides readable, I've defined these following values and methods to use in the examples. We have default margin of 50, random X and Y, a random color from all the registered colors that it defaults to have, a random bool which you won't really tell, but I've biased it towards false. For things that need multiple different values, I've got a R50 method that just gets a random 50. So, clear, not much to say here. First thing we almost always want to do on each turn is to clear the screen, but this happens by default when you use simulation. Point is the most basic building block, but not necessarily something you're going to directly use a lot. Here we're making a thousand random points on each turn with random colors, and it just basically becomes this noise. Line is pretty straightforward as well, with two pairs of X, Y values and a color. Here we're using the window height and the current turn number to animate. Angles like line, but you specify a single X, Y, and then an angle and a length instead of a second point. H line and V line take one coordinate and then span the whole window with your specified color. This is useful for plotting stuff. Erin, do not air drop me during the talk. I don't even have a mouse right now, jackass. Circle takes a middle point and a radius. This is the first primitive with an area, so it also takes a Boolean declaring whether or not it's built. Ellipse is just like circle, but it takes two radius values. Fast rect is a filled rect, always filled, taking an origin point on the bottom left, a width and a height and a color. It uses the lower level drawing function that should be faster, but these days it probably isn't anymore because everything is terribly fast. Too fast, in fact. I had to add v-sync to make things viewable. Rect is the same thing as filled rect, but it also takes a Boolean for whether or not it's filled. Polygon takes any number of points in a color. I probably need to extend this to allow for a filled polygon, but no one's asked for that yet. I always feel conflicted about this one. Bezier allows you to draw curves, but not in a terribly approachable way. You need to be more of a graphic step for this. I'm not going to explain this one too much. I want to add splines or something else that I find easier to use. This one's busy. It's one of the busiest slides I've got. Sprite, that call up top, takes dimensions and a block. During the block execution, drawing goes into a separate canvas and that image is returned. Blit takes that image or any image and draws it at a location with an angle and possibly optionally scale. Instead of using sprite to create a canvas, you can call image to load a bitmap from a file instead. Render text takes a string, a color, an optional font. There's always a default font. And returns an image. Put is just like Blit, except that it uses the bottom left corner instead of the center as its origin. Here you can see it rotating on that corner, unlike the tank and turret from before. Text renders, this has been opened, and draws a string at the specified point with a specified color. Okay. Finally, there's some helper functions that render to the screen. FPS prints the frames per second. It's a bad estimate, but it's close enough. Text returns the mouse date and I use that with the debug function that takes a format string and arguments to it and prints it. For debugging at the pixel level or just for fun, you can save the PNG of the current screen. And that's it. So let's build up some real examples, starting with generative art. I throw pottery. I enjoy it. It's fun to struggle with. I don't feel the same way about computer art. It's a struggle for me and I don't find that struggle fun. So I had an interview a few months back where they asked me to generate art in the style of Pitt Mondrian. I don't have time to go on about Mondrian. I could do that for a long time, but you should check out his work. It's really good. So normally I'd start with a hand-drawn sketch of what we're going for, but here we actually have a finished product to emulate. This is his actual work. So we start where we always start, the blank canvas. I've added a grid to help visualize where we want to go. There's stuff here I won't explain. It's mostly to make the slides more readable. Let's focus on the art method. So let's just think about where we want to go. We want thick lines that span the window. So we always have the width and height of the window, so let's just put two down. First vertically at 120 and next horizontally at 80. And W is the value 20, which is the width of the lines that we're going to make. Next let's use some helpers to generate a random number of grid aligned Xs and Ys, and then draw them using the previous code. It's really not much different except there's some loops. This looks good. Now we just need to fill in some regions. And as an aside, I just want to say, like, I love the fact that Ruby RAN could take number ranges. It makes code so much prettier than the way we used to do this shit and see offsets and adding and all that crap. So this is gross, I admit. I pulled it out and it's on method for this slide. Basically we randomly picked some regions based on the Xs and Ys that were previously generated. And then we find the area of those regions and we fill with a random primary color. What is going on next door? That's amazing. I hope this comes through on the audio for the recording. Honestly, that's not bad. And yet they still didn't offer me the job. Or did they offer me any feedback on how I could improve? So I wrote a blog post about it and published the interview question. NRI, you should treat your interviews better. Okay, slightly more relevant to your job. Let's work out some basic data visualization. Let's say you have a lot of data. You don't know what that data means yet. This is a bit of a bullshit example, but gathering data and plotting something like message size over time spent to process. As Rubyists, we're incredibly good at gathering data, whether that's parsing it out of text or slapping it from a database or whatever. Getting the data is never a real problem that we've ever had. So I'm not going to touch how that data is gathered. Just assume that I have it. And looking at a million data points, just under a million data points, and poking at it the way we would normally poke at it doesn't help. We need a way to make some sense of it. There might be a way to visualize it and see if there's a visible correlation. But doing something with that data, something visual, what do we as Rubyists do? Normally at this point, someone would probably suggest grabbing a whole new language like R. It's very good at importing and manipulating data. But it's also a pain in the ass language and overkill if you're not doing a lot of real stats. Or maybe someone else suggests using Excel, it has great plotting capabilities. But the data that you're dealing with is really, really big and getting it into a spreadsheet is a pain. Now you have two problems. We don't have many good options in Ruby for this. We grab for tools in other languages like R or D3 and JavaScript. And that keeps us from improving our own tooling. I think that's a problem. So let's try to address this a bit. I'm not solving this problem. I'm just addressing it. So let's try to do this ourselves playing to our strengths. We always start with a sketch of our system. That didn't come through very well, but whatever. Basically we're going to do a very basic plot, trying to show time over size to see where any problems might be. When I say sketch, I mean it. This is why this is a hand drawn. So as usual, we start with a very basic subclass of simulation to make a blank window. This time I've declared a grid to help. It's change initialized to store off the data being passed in. We scale it to the size of the window, which is max, which is set to 500. Then we use group i to gather and count the same values. And we apply math log to keep the values sane. Otherwise everything is just a big black screen. So now we can add the draw method. We simply walk over the data. We plot a gray point at x, y. I also plot a translucent red circle at that point with the log count as the radius. This will start to show where there's lots of spillover. If you zoom in, you can see that there's very light red is heavily spilling over in some areas. These are areas where you might want to look for trouble. There's huge amount of variance in those columns. Now we know where to look. Have fun with it. Add features. All those axes, put the values in there, make it easier to repeat later. This might not be the right data to visualize, though, or it might not be the right way to visualize it. You need to experiment in order to suss out information from raw data. Running simulations is why I wrote this library. Here are some simple examples on how to do that. Langston's ants, or vants, which is short for virtual ants. They live on a 2D plane and have a simple set of rules that they execute every turn. If an ant is on a white cell, make a black, turn left. If not, make it white, turn right. Move forward one and repeat for all ants. Turns out they're Turing complete. They're a Turing machine on a 2D surface where their state is their current position as well as direction. This is what it looks like for a single ant after about 11,000 iterations of this rule. As always, we start with a blank canvas. I'm leaving some of the mechanics out for brevity, but otherwise it looks the same. This reads pretty much like the rules. If it's white, set at black, turn left. If it's black, set at white, turn right. Finally, move. Here it is after only a few turns. This is it after about 6,000 iterations. Not bad ant farm. I really like it. Boidstance is an artificial life program which simulates the flocking behavior of birds, fish, or other systems. I love the fact that Wikipedia says this. Incidentally, Boidstance is also a New York metropolitan dialect pronunciation for bird. Boids use a combination of simple rules to determine flocking behavior. At a minimum, it has cohesion, separation, and alignment. More can be added, you can also have more added forces like wind or water for a more accurate simulation for whatever you're doing. We get off to the usual start. We declare a simulation with a white background and an FPS display because it was really slow to start. Then, register the boid body to be used. We'll have 25 boids and we'll initialize them with a random angle and speed. We'll declare that they get drawn as a simple black circle with a red line showing their vector. Boids try to fly towards the center, or rule number one, cohesion. Boids try to fly towards the center mass of nearby boids. That's pretty much exactly what this code says. Calculate the center mass, find the difference, and return 1% of that. Rule number two, separation. Boids try to keep a small distance away from others. This involves finding nearby boids that are too close and calculating a vector slightly away from them. Rule number three, alignment. Boids try to match the velocity of their nearby boids. This simply involves calculating the difference of the average velocity with their own. Now, each turn is just a matter of summing those vectors and adding them to the current velocity clamped to a limit. Finally, at the bottom, we move and wrap around the window. Here's the final result. Very quickly, the flocking behavior takes over and they converge while maintaining safe distance from each other. Some rules add up to very organic feeling behavior. Finally, games. Let's dissect that tank game from the gallery. I'm not going to implement the whole thing here for time, but I want to show how to do interactivity. Of course, we start with a sketch of what we want to do. Nothing fancy. Just a simple tank with an independent turret in an arena. So we sketch that up. We formulate the rules for how we want it to behave. For now, the tank is in the arena. Arrows let us move the tank. And S, turn the turret. Everything else, firing, et cetera, is an exercise for the viewer. As always, we start with a blank canvas. We initialize the tank to be in the middle. We have to add a turret for an angle independent of the body angle. Next, we draw our tank into a sprite. Pro tip. This is incredibly error-prone. So give each shape a different color until you figure all the coordinates out. Draw extra boundaries around the whole thing and draw diagonals to help you measure on stuff. Little things like that help a lot. Then we use blit to put the tank and the turret on the canvas. At this point, the game will start up and you can see the tank just sitting in the middle. That's boring because it can't do anything. So let's give the tank some behavior. The ability to turn the body and the turret and to move to accelerate or de-accelerate. Then once that's working correctly, we can go ahead and plug in those event handlers using add key handler. We declare the handlers and have them control the state of the tank. And here's our final result, for this talk at least. Obviously, this can be easily extended to fire bullets, have sound effects, multiple tanks, targets, scoring, et cetera. I just can't fit all of that into this talk. Sorry. So what are the patterns that we've seen so far? Start with a sketch. Figure out what you want to see. Figure out the behavior rules, if any. Start coding with a blank canvas and build up one small thing at a time. Don't try to do the whole thing at once. This is too much stuff. If you aren't sure, start explaining with drawing. Get your images, your sprites, your backgrounds, everything right. Move to simulation when you have more dynamic behavior. MVC is fantastic here. Model mostly body instances. Your view is all bodies have an external draw routine, stateless. And controller, simulation, running update and draw loop and maintaining the shared state. I think the declarative form works really well for graphical programming. So try to push towards doing less in your methods and using decorator modules instead. Just remember to always call super, otherwise they don't show up. Hopefully this talk has convinced you that you could benefit from added visualization tool in your toolbox and that graphics is a good fit for that. Thank you. The question is what lies underneath? And yes, it is a C library. It's called lib SDL. And the unreleased version of this code, which is eminent, has switched from SDL1 to SDL2, which is why it's so much more performant than it used to be. And is now pixel perfect colors, really good alpha blending. It's really improved. Is it possible to do this for M Ruby? I don't see why not. I don't know. But I'm going to say yes. This has a small C extension in it which provides the wrapper to SDL and so it should be just the same. What are the limitations that I currently see with it? I don't think there's enough decorators. I don't think there's enough drawing primitives. Josh uses this a lot and he's asked for things like thick lines instead of having to do it yourself, which is now possible with my switch to SDL2. The performance problems went away. I went from 30ish frames a second to 200 frames a second by switching to SDL2. And so I actually had to slow it down. Otherwise, the simulations are gone before you can see anything. And all the coloring stuff is better. Oh! Another thing that I don't like, and I don't have an example of this in this talk, I hate eyeballs and I hate brains. Color perception is really, really complex and difficult. It turns out magenta is not a real color and that crazy, wacky thing called brightness is really complex. So if you're wanting to do scientific visualizations where you're using, were you wanting to use the same level of intensity for your colors to differentiate, that's exceedingly hard. If you use the color system called HSV, hue saturation value, that supposedly lets you walk across the hues with the same saturation and value. But blue is darker than red and yellow and they all have the same saturation and value. And so you wind up with these extremely different versions of intensity. There is a color map, HCL, that tries to address color perception differences. It also means that as you're walking through the hue, some of those colors don't exist because they don't match the intensity. So you wind up with nil values and you wind up screwing up your thing. So I need to figure out how to address that. I don't know if I can. I hope I can, but I don't know. I'm still experimenting. I don't know if I understand your question. What do I suggest to tests in what? Okay. So what tests would I write while I'm doing one of these things? Probably test the crap out of your models. You don't have to test emergent behavior that will fall out on its own and I don't think that's something you need to write a test for. So I would test all of my models head to toe and then I would probably leave it at that. The way the controller works and the way the view code works, like I wouldn't write a test for the view method at all. If I did, it would probably be save off a bitmap and try to compare against a golden, but I hate golden tests. And then the controller, for the most part, is entirely declarative and I don't see the value in testing most declarative code. Can you do this headless? Totally. No, but I've been very, simulation is still available. I've been very seriously considering splitting the graphic stuff out to just graphics and having simulation being a separate gem so you can just do numeric stuff. Uh-oh, I'm at yellow. So there's a logging function in there where you can register a logger to run every 100 ticks or whatever and it would be really easy just to not create a window at all and have that go and have your routine do the update loop and log loop instead of an update and draw. I haven't broken that out yet, but it would be 10 lines of code to do it, change the initializer around to not do the window and have everything else be the same. And I can totally add that. Question is, is it possible to add new primitives? If you don't mind coding in C, it's totally possible. I'm using SDL GFX as an extender of the standard primitives available and I've wrapped all but 15 of them. I'm switching from SGE to GFX and I haven't wrapped everything yet so I'm going to get thick lines soon and I'm going to get some other stuff. The problem is that SGE also provided my collision detection bitmap comparator. Uh-oh. No, I got 30 more seconds to query on that. And I just haven't finished that. But if your primitives were done at the Ruby level, then yeah, it's just another method that you just throw into simulation. All the demos were just done in... So the question is, do I have other backgrounds? It defaults to black, the white decorator is there and I used it in all these slides because I was worried that I was going to have a bright room and a dim projector. But you can clear in any color you want. You can also blit a texture onto the background as well if you wanted to do a game. Yeah, yeah, totally. And I think that's all the time we have. So thank you very much.
and both graphics and simulation were available, mathematical conventions, not game programming music, mine or not. Sorry. This is an emulation see your code with as little boilerplate as possible. Any new developers, quick and easy time to go on about Mondrian. I could do that for something visual, what do we as Rubyists do? start with a very basic subclass of simulation to be the right way to visualize it. You need to extra boundaries around the whole thing and draw could benefit from added visualization tool in going to get some other stuff. The problem is was going to have a bright room and a dim