phpng - heap and stack allocation
Schlüter, Johannes
phpng - heap and stack allocation ······························ Speaker: Johannes Schlüter Event: FrOSCon 2014 by the Free and Open Source Software Conference (FrOSCon) e.V.
 to start in this B-Miles. Right. Thanks, Koehe. So the back story about this talk is I have a session, oh, maybe for an introduction, I'm Johannes Schlüter. Until last week, I was release manager for PHP 5.3, which now released its end of life, and it's dead, and it's gone, and I hope nobody of you is using PHP 5.3 or older anymore. And I hope you're all excited about PHP 5.6. Yeah, that's what I wanted to hear. And I do different talks. For some time, I did all the PHP 5.3 talks, and then the topic became boring, and new versions came out. And I had a talk about PHP under the hood, explaining some things about how PHP works, how reference counting works in PHP. And then there was some work done by a company called Zend to improve PHP's performance by massively changing the internals of PHP. So if I give this talk nowadays, it doesn't really apply for the future, and we know it won't apply what I'm going to talk about in that talk. Then for this year's Crosscon conference, they were looking for more technical talks, more experimental talks, not the default, oh, it's just web scripting talks, but in-depth talk. And Corin, I chatted shortly about it, whether something like that might work out, talking about PHP under the hood or PHP and G at this conference. But back in the times, I didn't really feel comfortable talking about PHP and G, since it was quite new, just announced by Zend that they are going to do it. It wasn't clear whether it will be merged into PHP and so on. And so that slowed down again. I kept it in the back of my head, but it didn't really prepare any talk about it yet. And then yesterday, I was noticed that the speaker was ill, and I said, well, yeah, let's try it. And therefore, I don't have any slides or anything. So in this PHP under the hood talk, what I'm talking about is what I'm trying to teach people is that references are bad. I assume everybody of you is knowing what PHP references are. So you have your variable and its $B. I hope you can roughly understand those dollar signs. My handwriting is bad and even worse on the whiteboard. And if you do this, this ampersand syntax, you get a full A here. So $A and $B directly refer to the same thing in memory. The thing in memory we are having is called a Zval, which is a send value derived from previous PHP three times. There it was called PHP value, but then proper or better engine sent. Engine came along, and the name was changed to Zval, to represent that. And a Zval is a quite complex structure. I won't write it in complete here. But it's a C structure. Let's take a shortcut. It's an angle bracket. And we have multiple fields in there. One is an integer about a ref count and an integer for is ref telling us how many of those variables are pointing to a specific variable. So after this line is executed, we have one of these data structures in memory. And we are having the ref count of two since $A and $B are showing on it. And as we have true alias, as we said, this is ref flag to the value of one, just tell us, OK, it's a value. We have more data in here. The important part is a union value. It should work on my handwriting next time, which has some different fields like integer for a long value, and another struct. Struct. I should go there. For string values, it's a char pointer and an int for the length, and some other things for other types around. It's not complete. It shouldn't be there. And so all different types we're having in PHP are represented in this data structure. And a union in C is a data type. We are basically the maximum value for the maximum type is allocated in memory. And depending on how you access it, it's interpreted in the according way. So the largest data type we're having in here is a char pointer and an integer for a string and a length. And so the size of the union will be the size of the char pointer, which is depending on your system, 32 bits, 64 bits, and the size of an integer. It's an integer. So yes, which might be 32 bits or something like that. And then we have these additional fields. And they are kept here in memory. And then we have an additional field called type. I'm not sure whether it's a correct name, but that's basically the set-world structure. So that we keep for every of our, you're taking a photo of my bad handwriting. You're all my fault. And there we keep all the data in memory. Now we're having an issue when doing it this way. That's the way how we reference to it. Since it's nice that we're having one container for those variables, but we need to point somewhere from these variables to this reference. And for that we have a data structure, which is a hash table. From memory perspective, we can simply follow our hash table and assume it's in C array. It's not exactly true, but it's the closest we can get to it. And if we look at a computer from Neumann architecture, what we are typically using these days, not exactly the form it is, or anybody here has seen a lecture about computer science 101 or Grundlagen informatik in German, a few studied people. Then that shouldn't be new to anybody. In a von Neumann architecture, you have your memory with slots of the same size. And they are all addressable with sequential numbers. So there's address 1, address 2, address 3, and so on. And we have somewhere on end with some other numbers. Yeah, it happens. Hems don't like me, and I don't like pens. So we have this memory layout where we can address the different areas. Now we're having an array for our values. So we place it in here, something like that. We have three pointers here. That's not exactly what it is, but we can assume it, pointing us to our structure, which are in some other memory address. So that's maybe here. And then we have a pointer pointing there. And we have a second pointer pointing there, too. So when accessing a variable, the PHP engine has to go first, or the system has to fetch that memory first. And then dereference the pointer and go here to fetch the actual data. So there's an in direction step. Now we have these things called CPUs. We're going deeper. If memory wasn't enough for you, we have our CPU, as you can clearly see. And the CPU, one key component is the Arithmetic Compulsion Unit, ACU, is that correct? ALU, Logic Unit. Or whatever. But that unit doing actually the math in the computer. Then you have your registers here, which are very small. Places of data basically can take the word size, so 32 bits or 64 bits or something like data, in one of those registers directly in the CPU. Then you have your L1 cache, your L2 cache, your L3 cache, the bit depending on your CPU. If you're looking at memory now, we know RAM sizes are in the gigabytes. I think I don't know what a typical server for you has, some quite few gigabytes. But if we're looking at here in the CPU level, we're talking about something like five megabytes or three megabytes or something like that, depending on your system's architecture, the specific CPU. And it becomes smaller and smaller and smaller. And here we are really at word size level. Now we have the second issue here, that it takes time to get from here to here, or data from there to there. Our limiting factor there is the absolute limiting factor. It's not the only one. It's called speed of light. Speed of light is, who knows it, how many kilometers per second? 300,000. 300,000, 299 or something like that. So then we have three gigahertz as a CPU frequency, which means in the time of one CPU cycle, the information can travel at most. Who can do the multiplication? It's 9 centimeters. Oh, depending on the 300,000 times the gigahertz, 8.9 is what I calculated yesterday. So the maximum distance our information can travel within one CPU cycle is, what did I say, 9 centimeters or something like that? And it has to go back and forth. And if you're looking at our main board, we see, we are a bit over that. So we are wasting CPU cycles while waiting for the memory. In real life numbers, just to give you an impression about what's happening here. Rough estimates, I found those numbers on the internet, so they must be true by definition. Accessing RAM, we can estimate about 100 nanoseconds. Accessing, I only have L2 cache, but it doesn't matter too much, it's 7 nanoseconds. And L1, 0.5 nanoseconds, and accessing registers more or less immediately. So by having to go to memory to access the data, we go from 7 nanoseconds. For data we have already cached here to 100 nanoseconds, which is a lot more. Of course, it's nothing if you look at it. But if you then think about, again, you have 3 gigahertz as frequency, and you're waiting for 100 nanoseconds for your data to process, it's quite some time you have to wait. And the CPU basically has nothing to do with it. So to improve this, the system, the CPU on one side and the compiler and everybody is trying to help by not reading the single data packet here, but trying to read more of it at once, and trying to put it in the caches. So we are not reading only this, but maybe if you're only accessing the LA, the CPU predicts that we're going to read this, and put at least an add of 3 cache, if there's enough space. That gives a huge improvement in the performance here. But we still, but the CPU can't predict that we're going to make this jump, and going to access that memory in most circumstances, depending on your CPU, depending on your compiler, that might look different. But in general, it's hard for the CPU to do. Now, we have to state a structure with this indirection here. And we want those references. And we have this issue. What can we do? We can change this behavior, and we can change these hash tables. When I thought about it, I thought about doing it next to each other, but I have to switch. So what the Sand guys came up with was redefining the hash table, inside the hash table. You're not actually storing the pointer, pointing there, but the actual value. So when we're drawing our memory again, we have to switch colors just to make that a bit more practical. We have our, that's now one entry for our hash table. So that's, all of that is $A. And inside there, directly, I don't have a value, but let's assume I have a value of 23 for that. We have some meta information, and we have the actual value of 23 right in that data structure. And if we are now having an array of data, the next element is directly next to it. Yeah, let's say $B, which already shows a small issue we're having. And that has some data. I'm not going into details right now. So what we're having here is, in the old version. Sorry, but there's two, it has some not really even now. Wait, wait. Coming exactly there. So in this old case here, we add inside this hash table with some meta data and so on, a reference to the actual data. And we could simply reuse this. In my old talk, I mentioned before, the main message was don't use references. Since doing this prevents an optimization inside the PHP engine, which tries to do copy and write. When we are not having a reference here but copies, and we don't change it, the picture here will be exactly the same. Only difference is that we are not marking it as references in the data structure here. But this stays the same. And if you have one reference and one copy, you can't reuse the container, but you need a second container to either hold the reference value or the copied value. And that's why references are bad, but that's another topic at talk. You can look at that with slides and everything on YouTube if you search for your hannish-luter PHP under the hood. There's a version from PHP UK conference about that. Now, as you directly saw, when we're needing a reference, and we are directly storing the value inside the actual data structure here, this doesn't work out that way. That's why there's more complication to it. What they already also did is I should have put it next to each other. I should have thought about it more. Changing the set world structure to a structure where you have type and you have a union and I think one more field, which I don't remember right now, but it doesn't really matter. And now the type here. Oh, let's take the other way around. The type flag in here directly represents the type you're seeing in PHP. If you're looking at the variable, it has a type of an integer or long, which is the same in PHP speak. It has the type of a string. It has the type of an array. It has the type of something like that. That's a numeric value. You have this is constant, I think, exported to PHP userland and the same ones are used here. In the new one, we have more types here. We have specific types saying it's long. We have specific types like here saying it's a string. And we have a specific type here saying it's a reference type. And if you say it's a reference type, then basically this happens again. But with more memory usage since the data structure is a bit, or that data structure is a bit bigger. And then we have this interaction again. But if we don't need an interaction for references, we can directly inline the data inside the hash table. Now, the next optimization this brings, or is that so for clear? The next optimization this brings by having those values next to each other in memory is, again, if you're looking at our different cache layers here, often operation we're doing is some form of iteration over in the data when searching for something, when looking in an array, when doing a for each on an array or something like that. By directly having the data next to each other, the CPU can take more of the memory at once closer to its core, to its calculation units. And without going back and forth all the time between different memory areas, then jumping somewhere else, and keeping in mind those are really small, it's quite nice to have this data really compact close to each other, everything. Now, there's another effect we're having here, which brings us back to our nice Mr. von Neumann architecture. Oh, just for more or less informational purpose. If you're reading data from an SSD drive, I just put some more numbers out just for giving a feeling what happens. We are at 150,000 nanoseconds, roughly for trusty SSD access. And then I have something about hard disk. So one megabyte reading from a spilling hard disk drive, one MB is 10 million nanoseconds. That's what I have brought here about numbers. So it's a huge gap if you're going to that kind of memory, just to give an impression of these numbers. So now, another thing is, if we're handling these variables in this old way, we have quite some common cases where we need temporal values. If you have in calculation, this is one working, 1 plus 2 plus 3, and equals $C. And as long as we don't optimize it, we need a temporal variable here. And then copy that, and then reference it again for that variable. And for all of that, we have to go to memory and find some space where we can put it. And basically, we have two choices when accessing memory from system perspective. One is putting it in heap memory, which basically means give me some memory area here, and reserve it for me so I can put data in and take data out. That takes quite some time on a system, speaking about in those many quads of time. The other approach is, again, looking at our von Neumann architecture. And our nice things we're having here is our stack. So we have one register, which is a stack pointer. And the stack, which is in Germany, nicely also called sometimes Kellerspeicher. So seller memory is at the bottom of our memory. So we have address one. And we have the maximum memory here. And when our program starts, the stack pointer points here. So this register has the address of that one. And each time, a function is called on a system level, so a C function from the PHP implementation. Somewhere annotated to that function is the information how much stack memory it's using. And it's simply changing this value accordingly, increasing by the needed space. And then the pointer shows here. And the function can use the space it needs. And that's a really fast way to access memory. Since all for allocation on a function call, you change that value. And when you leave the function, you change it at the other route. Unless you're using C++ with destructors or something like that, it's really, really a fast operation. I had some thought on my mind, but I forgot it. Anyways, I'm improvising, as you might have noticed. The data stored in here. And now, we don't have this overhead of putting it. My thought was, yeah, no typical issue about the PHP program, function foo calls function foo. And we all know what happens on PHP. We get a segmentation foo due to a stack overflow. What happens is we're calling function after function after function. So this one grows, grows, grows, moves up, moves up. And in some area, the operating system has a limit and says max stack space for this process is here, or for this thread, actually, not process. And if we reach that limit, the operating system, or actually the CPU, which knows this limit, will signal the operating system, which will signal the application. Sorry, no more stack space and goodbye process. And you get a segmentation fault. And detecting this in an incompatible way is something we don't do from PHP level. So you get a segmentation fault. So that's the reason for this one. Now, if we're having our temporary wayables and we are putting, taking this indirection out, we can directly put the temporary wayable here in our stack, put it there, keep it there. And if we need it, we copy it. If we need it somewhere else. So when the function exits, and we still need to value, we have to copy it out. Otherwise, we can forget about it. And it's gone from, well, it's still in memory. But in an area not accessible by anybody. And that's the second optimization we're having in PHP for the general purpose case. And these changes in data structures lead to numbers I don't know that anybody knows how much the factor is from you, well-informed people, something like one third of the performance improvement over PHP 5 for this PHP upcoming version, which nowadays has a version number 7. And mostly, you're really changing those data structures. While there's another change in the data structure, for instance, if you take Boolean values, currently in this implementation, we have a flag saying it's a Boolean variable, and then additional field for the value. They are now combined in one. So basically, true and the false are treated like they were types, not like values. So we need one lookup to access it's a true variable or it's a false variable, and not two lookups. And some more of those small improvements on there. Now, what can we ask you with this information? What's the time, by the way? So how much time? OK. Almost half time only. Oh, damn. It's the last talk, though. And party time is coming up. But what did I just say? All right. OK, so the initial issue which led to really changing these data structures was that Dimitri, who's the guy from Zendt, who's really in depth about doing all those optimization and thinking about those, wanted to do some just-in-time compilation of PHP code into native code to have even more performance impact. And when doing this, he figured out that this indirection kills every optimization since we can't really predict what will happen. With this model, there's a chance that we're not only seeing this improved data structures, which result in overall less memory usage and higher performance since access times become really slow. But that we can let the step towards just-in-time compilation, since with the values directly telling us the type, we can be sure about this and therefore give it to JIT engine to say, it's that type. Go with it. Optimize it. Make native code out of it. And? So almost work, I guess, like the optimization needs to be done, and then they have like experimental G to the engine of the app. Yeah. And then what do you do about it? Like, I think it will beat me, but the decision to simplify it's time. Well, the question is, this becomes way more complex if you look at it. If you really look at the code, it's more complex than what we had in the past. So it's not a simplification, but it makes it more complex. But for the user perspective, it's faster. It's more efficient. And you, if you're using PHP, you don't see anything of that. You see that. And but it's this first step towards JIT compilation. And there was a recent post by Dimitri starting a heated debate about one of everybody's favorite topic, static type hinting for scalar types. Since you really figured out, if we know in the declaration of our function we're having this type, we can do so much better JIT implementations. And after reading that statement, I thought, damn, he's quite close to having a JIT. I don't know any details about what he's really under. How close he is. But from that posting it sounded like there might be a chance that next week or something is coming up with a new idea for a JIT implementation, bringing again some huge performance improvements. The other thing in meantime Dimitri did, I never met him, but I really like what he does. Since he's really no big time about discussing, but delivering and improving things for quite some time in PHP. Even with that model, we have to allocate quite some memory on the heap or free memory or whatever you want to call that area, not the stack area, of the memory. And operating systems typically are optimized. But you start them at some time, allocate quite huge blocks of memory, keep them for, in computer speak, long time, and then free them again and do something else. What you're doing in the PHP script is you need these relatively small data blocks all the time. You need a new variable. You free it. You need a next variable. And new temporary table gives quite a lot of memory allocation operations. Memory allocating, finding good spot that you're not wasting too much memory and keeping it close to each other and keeping the total thing when multiple processes are running at the same time takes huge amount of time. And so we have an optimized memory manager already in PHP, which knows, OK, I'm having quite a lot of data with this size. And preallocates slots for that and reuses them and manages those. And what Dimitri, I haven't looked deeper into that implementation yet. And I think last week, or maybe the week before, it was a new memory manager. So he's not only working on changing the data structures, introducing a JIT engine, but also while writing a new memory manager. And again, improving performance of PHP quite a lot. So now there was quite some fuss in the PHP community related to this work. Now we're getting into politics. Or kindergarten. No, I won't read internal postings. So there's an operating system in a word which is called what's the name? Not doors, not gates, windows. That's it. And some people use that system. And there's a company in Redmond which really pushes that operating system and tries to sell it. I don't know why they are doing it, but they are doing it. Oh, this is recorded, isn't it? Damn. That's serious. I'm just. And what they are working on is they try to unify the experience, making it more portable that PHP scripts on different systems and different platforms behave the same way. And now we're having one thing in PHP, which actually is an error here. Now that I see it. This one is the long one. It's actually of type long. And the property of this type long is from the C standard, it's not clear what size long should be, which leads to different interpretations. In the Unix world, long is interpreted as if you're running on a 32-bit system, being of a size 32. If you're running on a 64-bit system, it's size 64-bit. On the Windows operating system, it's defined to be already 32-bit, just to make porting code easier on Windows from 32 to 64-bit. But I think that's also a change in the value when they would probably compile. I don't know about Windows, and I don't really. The point is there's this difference on the one side between Unix machines themselves, if you're running 32-bit and 64-bit, and with Windows machines, which then behave differently again. And what many people want from a scripting language is it behaves the same on all platforms. And those guys from Microsoft also figured out that we are using the int type here to represent the lengths of the string. If you look closer to C standard and recommend the programming guides, you see that int isn't the size one should take, but one should take the type called size t. And so they spent quite some time in an open process by early on telling everybody we're doing this and we want to do this to change those types. To make sure that on all platforms independently, whether it's Unix, 32-bit, and Unix, when saying Unix, I include Linux, which is a new Unix, other religious discussion, I'm working for a true Unix vendor, by the way. To always be 64-bit on all machines and all systems everywhere. And to have this type on a proper type, so size t. Long size t, S size t, just to be 64-bit all the time. So independently of your system, it behaves the same with high integers and so on. And long strings. I don't know if anybody needs strings longer than two gigabytes in memory, but if you do with PHP, it should behave the same on all platforms. That's the ideal. And it's a huge change in PHP to change that. And they announced that they were doing this. They were doing this publicly in PHP's Git repository in a specific additional branch, which was directly next to the main PHP branch. And suddenly, the send people came and said, well, we changed all the data structures. We worked hard on reducing memory usage. And we want to push that. And we didn't communicate with you beforehand, since it was an experiment, and we didn't know whether it would work out and or not. We don't want to send out all the ideas. We're having all the time, since that's too much, since they're really actually working continuously for years on different ideas, how to optimize PHP. And 90% of that are thrown away since it doesn't work out. That idea about changing the data type and so on, that seemed to have worked out. We don't have major bugs yet. Maybe you find them if you test what's now called PHP 7 from PHP repositories that has it with the applications. You might find bugs, but we don't know major bugs yet. Might want to push it. Now, this change here increases data usage. You need more memory if you're always having 64 bits here. You're using more memory if you're not using a small integer type here by a large integer type, which is kind of opposite. Now, you have the one company pushing it really hard to get their change in, since it's a huge improvement for PHP. And they are saying, that's what people are interested in. People want performance. And you have the other guy who's working for big Microsoft with other guys helping him, doing open source, open development from Microsoft, and doing this change. And they are more or less beaten down. I'm defending Pierre, by the way. New feeling, but heavens. And the same guy is saying, increasing memory usage and performance is the most important part. And they are saying, well, we need a consistent behavior, and we need better APIs internally, better types internally, and not ugly types, which that partly has internally. It's a more complex structure, needing more different operations and different types. And so there was a huge debate. In the end, they worked together silently and recently put this work on top of that work. And even in a way that in some cases, you even get more performance out of it. So out of the, oh, yes, so slow, they figured out if you're working, cooperating, we get something which is even better. We fulfill more or less both needs from both sides and putting stuff on top of it. And let me think, is there anything else on that level in the future of PHP? I don't think so. So if everything goes well, everything I talked about will be released as PHP 7. Now the debate is whether we will have a 5.7 release in between or not. From my personal opinion, I think we should focus on getting that one out. And if we figure out, oh, we can't do it. And we can do an immediate release. But if we say, oh, we can't do this. And we can't do it in a year, then we do some other stuff and implement other features which are piling up. People want to have big int support. So you can have integers higher than 64 bit, or that kind of stuff, and pushing for those changes. And then you get a 5.7. And then nobody has worked on that since the 5.7 got the priority. And then you figure out, oh, it will take so much more time until we have PHP 7 ready. So let's make a 5.8 and a 5.9 and a 5.10. And then we do the same what we did with PHP 6. And we kill PHP 7. And then we get to PHP 8. And so my opinion is we should try to push PHP 7. So we should demand, or you should demand, from the PHP developers, give us PHP 7 next year. And you should do your part in testing the Git repository snapshots of PHP 7. And try to bear breaks with your code base early. Since the earlier you report issues, you're seeing the earlier that can be fixed and the earlier it can be released. And that's mostly I can think of right now. And then the subject, if you have any questions related to that or any other questions, feel free to ask them. And maybe I'll answer them. And otherwise, if you don't have questions, there's beer and barbecue and stuff outside, I think. Or maybe inside. But sure, and not sure if it's from now. Oh, damn. So yeah, I need questions. So please ask questions. Yes? Does it break or extension? It's huge. I just had to make a joke first. So the question was about how much it affects extension developers, since there are people doing quite impressive work on top of the PHP API to make it usable with a proper programming language called C++, which really makes some things easier. And it's an issue since we have a migration guy telling you some of the things. But every place you're working was hash tables you have to change, basically. And some of the type usage you have to change. That's it for typical extension. You might, to some degree, automate it, basically when you have a set while star, star, take a star away. Keep the same macros. If you're using the isboolean macro, I think you have to change it. If you're using arrays, you have to slightly change those since there's a different type. Oh, and strings have been changed. They are now of type send string and not child pointer anymore. And then new string API. And then the string, and the string, and the string. But on the wiki PHP net, somewhere there's a migration guide. Just go to RFCs, then go to PHPNG. And on the bottom third of the RFC is linked to the migration guide for extension authors, which then also links by the way to the internals of PHPNG. So it's not on the RFC, but you have to go to the migration guide to get to the internals. And there you can find the information. But it's notable work to adopt to it. Other questions? Or if that's not enough, then? Well, the current set of extensions that I said, although there must be hundreds of extensions, they will all break. Yes. All extensions will break, and all extensions have to be adopted. And how much of the extensions are managed by PHP team? I don't know how much. So the question is how many extensions are managed by the PHP team, and as every extension basically has to be touched to be adopted to this new world. The number of extensions, I don't know, since there are many, many internal, only company specific extensions. Then there are tons of un-maintained and unused open-source extensions in PICL, in the PHP extension repository, PICL.PHP.NET. And then there's a number of used ones. And out of those used ones, there might be some which are worse maintained and some which are better maintained. So if you're looking at in common extension something like xDbug, which isn't part of the PHP distribution, but is very well maintained, that will be adopted quickly. If you're looking at some extension like the Memcache extension in the PICL repository of Memcache D, they are a bit less maintained. But I guess they are important enough that somebody will work on it. And if you're done going to even less used ones, it might take some time or your time. So the interesting point is if you have more or less simple PHP extensions, more or less wrapping only a C library, and not doing any object-oriented stuff, it's very little work to no work to get them from PHP 40 to PHP 5.6. The incompatibilities are quite small there. Less than an hour of work, pure work without all integration testing and what you could do. So there aren't many changes. And the changes there are kept in quite small areas. Now, this is a huge change. And this will really kill some really old extensions nobody should use. Anyway, since they are maintained, obviously, as they will hopefully be adopted. Since I think the performance improvements, the memory improvements, the springs are so much that the community will cry more than they will cry about some new syntactic sugar. I don't know what the latest syntactic sugar is, which is being introduced. I don't follow the syntactic sugar. I've come from an hosting company. We see sales very, very rarely. Like very exotic extensions. What we see is like between the community with the framework with the application, and they build all the certs. Mostly they are into core components. So some with that support SIP and HTTP and HTTP. Yeah, so HTTP extensions, a good example, that might probably become part of the core distribution. And the other extensions you mentioned are part of the core distribution. So either they are thrown out. I can imagine that some crappy stuff we're having in there, which nobody is using actually, might be thrown out in the process. I don't know. Some of the un-maintened PDO extensions. I won't comment on that since MySQL is the only database you should use anyways. I'm paid for saying that. So I won't comment on throwing out other databases. But yeah, in fact, that was an area I was thinking about. Some IBM DV2 PDO driver, which was broken anyways, and where nobody from the PHP developers can commit, since IBM lawyers don't like that. Such things might be thrown out. So most of that work is done by Zend, which is mostly dimetry, and they hired two guys more or less for that. I don't know if they hired them full time or just. No, not full time. Lawrence and PTC, are they working with that? Yeah, I don't know the contractual details. But so they sponsored at least work from two other guys. One is called himself LaRuins, which is a type of company from LaRuins, and he's a Chinese guy. He recently got into the PHP core community, and is also a great guy from technical perspective. He is a bit shy about communicating, since he feels his English isn't good enough. But that's the same for most of the contributors, which makes it a bit hard sometimes to discuss. But he's a great guy, and does lots of work. And Nikita is also quite into these things and send a sponsoring that. But I don't know how the relationship, who does what, and so on. And I don't care. I don't care who is paying whom for working on PHP for the most part of it, unless for stupid comments. So that was the question. I'm making sure that these two developers have a share of things together. So question is about our bus factor solution issue. So bus factor means having an important developer being run over by a bus. And what happens to the project? And well, you have Dimitri. I think he's still living somewhere in Russia. You have LaRuins in China, and Nikita somewhere in Europe. And I have a hard time imagining a situation where they travel from one location to the exact same location. They might probably go all due to maybe Send Consents, Send Past, and once their guy's there. And they might share a flight from Frankfurt or some other European hub. And I don't know if Send has any precautions of an accident happening there. I'm not so much into the disaster recovery stories from that. I know it from other companies, but barbecue has been opened. So if you're running out now, I get the impression that you don't like me. I prefer food and alcohol over me. Alternatively, I was so bad that you needed to become drunk. And anyway, thank you.
 to start in this B-Miles.  Right.  Thanks, Koehe.  So the back story about this talk is I have a session,  oh, maybe for an introduction, I'm Johannes Schlüter.  Until last week, I was release manager for PHP 5.3,  which now released its end of life, and it's dead,  and it's gone, and I hope nobody of you  is using PHP 5.3 or older anymore.  And I hope you're all excited about PHP 5.6.  Yeah, that's what I wanted to hear.  And I do different talks.  For some time, I did all the PHP 5.3 talks,  and then the topic became boring,  and new versions came out.  And I had a talk about PHP under the hood,  explaining some things about how PHP works,  how reference counting works in PHP.  And then there was some work done by a company called Zend  to improve PHP's performance by massively changing  the internals of PHP.  So if I give this talk nowadays, it  doesn't really apply for the future,  and we know it won't apply what I'm  going to talk about in that talk.  Then for this year's Crosscon conference,  they were looking for more technical talks,  more experimental talks, not the default, oh,  it's just web scripting talks, but in-depth talk.  And Corin, I chatted shortly about it,  whether something like that might work out,  talking about PHP under the hood or PHP  and G at this conference.  But back in the times, I didn't really  feel comfortable talking about PHP and G,  since it was quite new, just announced by Zend  that they are going to do it.  It wasn't clear whether it will be merged into PHP and so on.  And so that slowed down again.  I kept it in the back of my head, but it didn't really  prepare any talk about it yet.  And then yesterday, I was noticed that the speaker was ill,  and I said, well, yeah, let's try it.  And therefore, I don't have any slides or anything.  So in this PHP under the hood talk,  what I'm talking about is what I'm trying to teach people  is that references are bad.  I assume everybody of you is knowing what PHP references are.  So you have your variable and its $B.  I hope you can roughly understand those dollar signs.  My handwriting is bad and even worse on the whiteboard.  And if you do this, this ampersand syntax,  you get a full A here.  So $A and $B directly refer to the same thing in memory.  The thing in memory we are having is called a Zval, which  is a send value derived from previous PHP three times.  There it was called PHP value, but then proper or better  engine sent.  Engine came along, and the name was changed to Zval,  to represent that.  And a Zval is a quite complex structure.  I won't write it in complete here.  But it's a C structure.  Let's take a shortcut.  It's an angle bracket.  And we have multiple fields in there.  One is an integer about a ref count and an integer for is  ref telling us how many of those variables are pointing to  a specific variable.  So after this line is executed, we have one of these data  structures in memory.  And we are having the ref count of two since $A and $B  are showing on it.  And as we have true alias, as we said, this is ref flag to  the value of one, just tell us, OK, it's a value.  We have more data in here.  The important part is a union value.  It should work on my handwriting next time, which has  some different fields like integer for a long value,  and another struct.  Struct.  I should go there.  For string values, it's a char pointer and an int for the  length, and some other things for other types around.  It's not complete.  It shouldn't be there.  And so all different types we're having in PHP are  represented in this data structure.  And a union in C is a data type.  We are basically the maximum value for the maximum type  is allocated in memory.  And depending on how you access it, it's interpreted in  the according way.  So the largest data type we're having in here is a char  pointer and an integer for a string and a length.  And so the size of the union will be the size of the  char pointer, which is depending on your system,  32 bits, 64 bits, and the size of an integer.  It's an integer.  So yes, which might be 32 bits or something like that.  And then we have these additional fields.  And they are kept here in memory.  And then we have an additional field called type.  I'm not sure whether it's a correct name, but that's  basically the set-world structure.  So that we keep for every of our, you're taking a photo of  my bad handwriting.  You're all my fault.  And there we keep all the data in memory.  Now we're having an issue when doing it this way.  That's the way how we reference to it.  Since it's nice that we're having one container for  those variables, but we need to point somewhere from these  variables to this reference.  And for that we have a data structure, which is a hash  table.  From memory perspective, we can simply follow our hash  table and assume it's in C array.  It's not exactly true, but it's the closest we can get to it.  And if we look at a computer from Neumann architecture, what  we are typically using these days, not exactly the form  it is, or anybody here has seen a lecture about  computer science 101 or Grundlagen informatik in  German, a few studied people.  Then that shouldn't be new to anybody.  In a von Neumann architecture, you have your memory with  slots of the same size.  And they are all addressable with sequential numbers.  So there's address 1, address 2, address 3, and so on.  And we have somewhere on end with some other numbers.  Yeah, it happens.  Hems don't like me, and I don't like pens.  So we have this memory layout where we can address the  different areas.  Now we're having an array for our values.  So we place it in here, something like that.  We have three pointers here.  That's not exactly what it is, but we can assume it,  pointing us to our structure, which are in some other  memory address.  So that's maybe here.  And then we have a pointer pointing there.  And we have a second pointer pointing there, too.  So when accessing a variable, the PHP engine has to go  first, or the system has to fetch that memory first.  And then dereference the pointer and go here to fetch  the actual data.  So there's an in direction step.  Now we have these things called CPUs.  We're going deeper.  If memory wasn't enough for you, we have our CPU, as you  can clearly see.  And the CPU, one key component is the Arithmetic  Compulsion Unit, ACU, is that correct?  ALU, Logic Unit.  Or whatever.  But that unit doing actually the math in the computer.  Then you have your registers here, which are very small.  Places of data basically can take the word size, so 32  bits or 64 bits or something like data, in one of those  registers directly in the CPU.  Then you have your L1 cache, your L2 cache, your L3 cache,  the bit depending on your CPU.  If you're looking at memory now, we know RAM sizes are in  the gigabytes.  I think I don't know what a typical server for you has,  some quite few gigabytes.  But if we're looking at here in the CPU level, we're talking  about something like five megabytes or three megabytes  or something like that, depending on your system's  architecture, the specific CPU.  And it becomes smaller and smaller and smaller.  And here we are really at word size level.  Now we have the second issue here, that it takes time to  get from here to here, or data from there to there.  Our limiting factor there is the absolute limiting factor.  It's not the only one.  It's called speed of light.  Speed of light is, who knows it, how many kilometers per  second?  300,000.  300,000, 299 or something like that.  So then we have three gigahertz as a CPU frequency, which  means in the time of one CPU cycle, the information can  travel at most.  Who can do the multiplication?  It's 9 centimeters.  Oh, depending on the 300,000 times the gigahertz, 8.9 is  what I calculated yesterday.  So the maximum distance our information can travel within  one CPU cycle is, what did I say, 9 centimeters or  something like that?  And it has to go back and forth.  And if you're looking at our main board, we see, we are a  bit over that.  So we are wasting CPU cycles while waiting for the memory.  In real life numbers, just to give you an impression about  what's happening here.  Rough estimates, I found those numbers on the internet, so  they must be true by definition.  Accessing RAM, we can estimate about 100 nanoseconds.  Accessing, I only have L2 cache, but it doesn't matter  too much, it's 7 nanoseconds.  And L1, 0.5 nanoseconds, and accessing registers more or  less immediately.  So by having to go to memory to access the data, we go from  7 nanoseconds.  For data we have already cached here to 100 nanoseconds,  which is a lot more.  Of course, it's nothing if you look at it.  But if you then think about, again, you have 3 gigahertz as  frequency, and you're waiting for 100 nanoseconds for your  data to process, it's quite some time you have to wait.  And the CPU basically has nothing to do with it.  So to improve this, the system, the CPU on one side and the  compiler and everybody is trying to help by not reading the  single data packet here, but trying to read more of it at  once, and trying to put it in the caches.  So we are not reading only this, but maybe if you're only  accessing the LA, the CPU predicts that we're going to  read this, and put at least an add of 3 cache, if there's  enough space.  That gives a huge improvement in the performance here.  But we still, but the CPU can't predict that we're going to  make this jump, and going to access that memory in most  circumstances, depending on your CPU, depending on your  compiler, that might look different.  But in general, it's hard for the CPU to do.  Now, we have to state a structure with this  indirection here.  And we want those references.  And we have this issue.  What can we do?  We can change this behavior, and we can  change these hash tables.  When I thought about it, I thought about  doing it next to each other, but I have to switch.  So what the Sand guys came up with was redefining the hash  table, inside the hash table.  You're not actually storing the pointer, pointing there, but  the actual value.  So when we're drawing our memory again, we have to  switch colors just to make that a bit more practical.  We have our, that's now one entry for our hash table.  So that's, all of that is $A. And inside there, directly,  I don't have a value, but let's assume I have a value of  23 for that.  We have some meta information, and we have the actual  value of 23 right in that data structure.  And if we are now having an array of data, the next element  is directly next to it.  Yeah, let's say $B, which already shows a small issue  we're having.  And that has some data.  I'm not going into details right now.  So what we're having here is, in the old version.  Sorry, but there's two, it has some  not really even now.  Wait, wait.  Coming exactly there.  So in this old case here, we add inside this hash table  with some meta data and so on, a reference to the actual  data.  And we could simply reuse this.  In my old talk, I mentioned before, the main message was  don't use references.  Since doing this prevents an optimization inside the PHP  engine, which tries to do copy and write.  When we are not having a reference here but copies,  and we don't change it, the picture here will be exactly  the same.  Only difference is that we are not marking it as references  in the data structure here.  But this stays the same.  And if you have one reference and one copy, you can't  reuse the container, but you need a second container to  either hold the reference value or the copied value.  And that's why references are bad, but that's another topic  at talk.  You can look at that with slides and everything on  YouTube if you search for your hannish-luter PHP under the  hood.  There's a version from PHP UK conference about that.  Now, as you directly saw, when we're needing a reference,  and we are directly storing the value inside the actual  data structure here, this doesn't work out that way.  That's why there's more complication to it.  What they already also did is I should have put it next to  each other.  I should have thought about it more.  Changing the set world structure to a structure where  you have type and you have a union and I think one more  field, which I don't remember right now, but it  doesn't really matter.  And now the type here.  Oh, let's take the other way around.  The type flag in here directly represents the type you're  seeing in PHP.  If you're looking at the variable, it has a type of an  integer or long, which is the same in PHP speak.  It has the type of a string.  It has the type of an array.  It has the type of something like that.  That's a numeric value.  You have this is constant, I think, exported to PHP  userland and the same ones are used here.  In the new one, we have more types here.  We have specific types saying it's long.  We have specific types like here saying it's a string.  And we have a specific type here saying it's a reference type.  And if you say it's a reference type, then basically  this happens again.  But with more memory usage since the data structure is a  bit, or that data structure is a bit bigger.  And then we have this interaction again.  But if we don't need an interaction for references, we  can directly inline the data inside the hash table.  Now, the next optimization this brings, or is that so for  clear?  The next optimization this brings by having those values  next to each other in memory is, again, if you're looking at  our different cache layers here, often operation we're  doing is some form of iteration over in the data when  searching for something, when looking in an array, when  doing a for each on an array or something like that.  By directly having the data next to each other, the CPU  can take more of the memory at once closer to its core, to  its calculation units.  And without going back and forth all the time between  different memory areas, then jumping somewhere else, and  keeping in mind those are really small, it's quite nice to  have this data really compact close to each other,  everything.  Now, there's another effect we're having here, which  brings us back to our nice Mr. von Neumann architecture.  Oh, just for more or less informational purpose.  If you're reading data from an SSD drive, I just put some  more numbers out just for giving a feeling what happens.  We are at 150,000 nanoseconds, roughly for trusty SSD access.  And then I have something about hard disk.  So one megabyte reading from a spilling hard disk drive, one  MB is 10 million nanoseconds.  That's what I have brought here about numbers.  So it's a huge gap if you're going to that kind of memory,  just to give an impression of these numbers.  So now, another thing is, if we're handling these  variables in this old way, we have quite some common  cases where we need temporal values.  If you have in calculation, this is one working, 1 plus 2  plus 3, and equals $C. And as long as we don't optimize it,  we need a temporal variable here.  And then copy that, and then reference it again for that  variable.  And for all of that, we have to go to memory and find some  space where we can put it.  And basically, we have two choices when accessing memory  from system perspective.  One is putting it in heap memory, which basically means  give me some memory area here, and reserve it for me so I  can put data in and take data out.  That takes quite some time on a system, speaking about in  those many quads of time.  The other approach is, again, looking at our  von Neumann architecture.  And our nice things we're having here is our stack.  So we have one register, which is a stack pointer.  And the stack, which is in Germany, nicely also called  sometimes Kellerspeicher.  So seller memory is at the bottom of our memory.  So we have address one.  And we have the maximum memory here.  And when our program starts, the stack pointer points here.  So this register has the address of that one.  And each time, a function is called on a system level, so  a C function from the PHP implementation.  Somewhere annotated to that function is the information  how much stack memory it's using.  And it's simply changing this value accordingly, increasing  by the needed space.  And then the pointer shows here.  And the function can use the space it needs.  And that's a really fast way to access memory.  Since all for allocation on a function call, you change  that value.  And when you leave the function, you change it at the  other route.  Unless you're using C++ with destructors or something  like that, it's really, really a fast operation.  I had some thought on my mind, but I forgot it.  Anyways, I'm improvising, as you might have noticed.  The data stored in here.  And now, we don't have this overhead of putting it.  My thought was, yeah, no typical issue about the PHP  program, function foo calls function foo.  And we all know what happens on PHP.  We get a segmentation foo due to a stack overflow.  What happens is we're calling function after function after  function.  So this one grows, grows, grows, moves up, moves up.  And in some area, the operating system has a limit and  says max stack space for this process is here, or for this  thread, actually, not process.  And if we reach that limit, the operating system, or  actually the CPU, which knows this limit, will signal the  operating system, which will signal the application.  Sorry, no more stack space and goodbye process.  And you get a segmentation fault.  And detecting this in an incompatible way is something  we don't do from PHP level.  So you get a segmentation fault.  So that's the reason for this one.  Now, if we're having our temporary wayables and we are  putting, taking this indirection out, we can  directly put the temporary wayable here in our stack,  put it there, keep it there.  And if we need it, we copy it.  If we need it somewhere else.  So when the function exits, and we still need to value, we  have to copy it out.  Otherwise, we can forget about it.  And it's gone from, well, it's still in memory.  But in an area not accessible by anybody.  And that's the second optimization we're having in  PHP for the general purpose case.  And these changes in data structures lead to numbers I  don't know that anybody knows how much the factor is from  you, well-informed people, something like one third of  the performance improvement over PHP 5 for this PHP  upcoming version, which nowadays has a version number 7.  And mostly, you're really changing those data structures.  While there's another change in the data structure, for  instance, if you take Boolean values, currently in this  implementation, we have a flag saying it's a Boolean  variable, and then additional field for the value.  They are now combined in one.  So basically, true and the false are treated like they  were types, not like values.  So we need one lookup to access it's a true variable or  it's a false variable, and not two lookups.  And some more of those small improvements on there.  Now, what can we ask you with this information?  What's the time, by the way?  So how much time?  OK.  Almost half time only.  Oh, damn.  It's the last talk, though.  And party time is coming up.  But what did I just say?  All right.  OK, so the initial issue which led to really changing these  data structures was that Dimitri, who's the guy from  Zendt, who's really in depth about doing all those  optimization and thinking about those, wanted to do some  just-in-time compilation of PHP code into native code to  have even more performance impact.  And when doing this, he figured out that this  indirection kills every optimization since we can't  really predict what will happen.  With this model, there's a chance that we're not only  seeing this improved data structures, which result in  overall less memory usage and higher performance since  access times become really slow.  But that we can let the step towards just-in-time  compilation, since with the values directly telling us the  type, we can be sure about this and therefore give it to  JIT engine to say, it's that type.  Go with it.  Optimize it.  Make native code out of it.  And?  So almost work, I guess, like the optimization needs to be  done, and then they have like experimental G to the engine  of the app.  Yeah.  And then what do you do about it?  Like, I think it will beat me, but the decision to simplify  it's time.  Well, the question is, this becomes way more complex if  you look at it.  If you really look at the code, it's more complex than  what we had in the past.  So it's not a simplification, but it makes it more  complex.  But for the user perspective, it's faster.  It's more efficient.  And you, if you're using PHP, you don't see anything of that.  You see that.  And but it's this first step towards JIT compilation.  And there was a recent post by Dimitri starting a heated  debate about one of everybody's favorite topic, static type  hinting for scalar types.  Since you really figured out, if we know in the declaration  of our function we're having this type, we can do so much  better JIT implementations.  And after reading that statement, I thought, damn,  he's quite close to having a JIT.  I don't know any details about what he's really under.  How close he is.  But from that posting it sounded like there might be a  chance that next week or something is coming up with  a new idea for a JIT implementation, bringing  again some huge performance improvements.  The other thing in meantime Dimitri did, I never met him,  but I really like what he does.  Since he's really no big time about discussing, but  delivering and improving things for quite some time in PHP.  Even with that model, we have to allocate quite some memory  on the heap or free memory or whatever you want to call that  area, not the stack area, of the memory.  And operating systems typically are optimized.  But you start them at some time, allocate quite huge blocks  of memory, keep them for, in computer speak, long time, and  then free them again and do something else.  What you're doing in the PHP script is you need these  relatively small data blocks all the time.  You need a new variable.  You free it.  You need a next variable.  And new temporary table gives quite a lot of memory  allocation operations.  Memory allocating, finding good spot that you're not  wasting too much memory and keeping it close to each  other and keeping the total thing when multiple  processes are running at the same time takes huge amount of  time.  And so we have an optimized memory manager already in  PHP, which knows, OK, I'm having quite a lot of data  with this size.  And preallocates slots for that and reuses them and  manages those.  And what Dimitri, I haven't looked deeper into that  implementation yet.  And I think last week, or maybe the week before, it was a  new memory manager.  So he's not only working on changing the data structures,  introducing a JIT engine, but also while writing a new  memory manager.  And again, improving performance of PHP quite a lot.  So now there was quite some fuss in the PHP community  related to this work.  Now we're getting into politics.  Or kindergarten.  No, I won't read internal postings.  So there's an operating system in a word which is called  what's the name?  Not doors, not gates, windows.  That's it.  And some people use that system.  And there's a company in Redmond which really pushes that  operating system and tries to sell it.  I don't know why they are doing it, but they are doing it.  Oh, this is recorded, isn't it?  Damn.  That's serious.  I'm just.  And what they are working on is they try to unify the  experience, making it more portable that PHP scripts on  different systems and different platforms behave the same  way.  And now we're having one thing in PHP, which actually is an  error here.  Now that I see it.  This one is the long one.  It's actually of type long.  And the property of this type long is from the C standard,  it's not clear what size long should be, which leads to  different interpretations.  In the Unix world, long is interpreted as if you're  running on a 32-bit system, being of  a size 32.  If you're running on a 64-bit system, it's size 64-bit.  On the Windows operating system, it's defined to be  already 32-bit, just to make porting code easier on Windows  from 32 to 64-bit.  But I think that's also a change in the value when they  would probably compile.  I don't know about Windows, and I don't really.  The point is there's this difference on the one side  between Unix machines themselves, if you're running  32-bit and 64-bit, and with Windows machines, which then  behave differently again.  And what many people want from a scripting language is it  behaves the same on all platforms.  And those guys from Microsoft also figured out that we are  using the int type here to represent the  lengths of the string.  If you look closer to C standard and recommend the  programming guides, you see that int isn't the size one  should take, but one should take the type called size t.  And so they spent quite some time in an open process by  early on telling everybody we're doing this and we want to  do this to change those types.  To make sure that on all platforms independently,  whether it's Unix, 32-bit, and Unix, when saying Unix, I  include Linux, which is a new Unix, other religious  discussion, I'm working for a true Unix vendor, by the way.  To always be 64-bit on all machines and all systems  everywhere.  And to have this type on a proper type, so size t.  Long size t, S size t, just to be 64-bit all the time.  So independently of your system, it behaves the same  with high integers and so on.  And long strings.  I don't know if anybody needs strings longer than two  gigabytes in memory, but if you do with PHP, it should  behave the same on all platforms.  That's the ideal.  And it's a huge change in PHP to change that.  And they announced that they were doing this.  They were doing this publicly in PHP's Git repository in a  specific additional branch, which was directly next to the  main PHP branch.  And suddenly, the send people came and said, well, we  changed all the data structures.  We worked hard on reducing memory usage.  And we want to push that.  And we didn't communicate with you beforehand, since it was  an experiment, and we didn't know whether it would work out  and or not.  We don't want to send out all the ideas.  We're having all the time, since that's too much, since  they're really actually working continuously for years on  different ideas, how to optimize PHP.  And 90% of that are thrown away since it doesn't work out.  That idea about changing the data type and so on, that  seemed to have worked out.  We don't have major bugs yet.  Maybe you find them if you test what's now called PHP 7  from PHP repositories that has it with the applications.  You might find bugs, but we don't know major bugs yet.  Might want to push it.  Now, this change here increases data usage.  You need more memory if you're always having 64 bits here.  You're using more memory if you're not using a small  integer type here by a large integer type, which is kind  of opposite.  Now, you have the one company pushing it really hard to get  their change in, since it's a huge improvement for PHP.  And they are saying, that's what people are interested in.  People want performance.  And you have the other guy who's working for big Microsoft  with other guys helping him, doing open source, open  development from Microsoft, and doing this change.  And they are more or less beaten down.  I'm defending Pierre, by the way.  New feeling, but heavens.  And the same guy is saying, increasing memory usage and  performance is the most important part.  And they are saying, well, we need a consistent behavior,  and we need better APIs internally, better types  internally, and not ugly types, which that partly has  internally.  It's a more complex structure, needing more different  operations and different types.  And so there was a huge debate.  In the end, they worked together silently and recently  put this work on top of that work.  And even in a way that in some cases, you even get more  performance out of it.  So out of the, oh, yes, so slow, they figured out if you're  working, cooperating, we get something which is even better.  We fulfill more or less both needs from both sides and  putting stuff on top of it.  And let me think, is there anything else on that level  in the future of PHP?  I don't think so.  So if everything goes well, everything I talked about  will be released as PHP 7.  Now the debate is whether we will have a 5.7 release in  between or not.  From my personal opinion, I think we should focus on  getting that one out.  And if we figure out, oh, we can't do it.  And we can do an immediate release.  But if we say, oh, we can't do this.  And we can't do it in a year, then we do some other stuff  and implement other features which are piling up.  People want to have big int support.  So you can have integers higher than 64 bit, or that kind  of stuff, and pushing for those changes.  And then you get a 5.7.  And then nobody has worked on that since the 5.7 got the  priority.  And then you figure out, oh, it will take so much more time  until we have PHP 7 ready.  So let's make a 5.8 and a 5.9 and a 5.10.  And then we do the same what we did with PHP 6.  And we kill PHP 7.  And then we get to PHP 8.  And so my opinion is we should try to push PHP 7.  So we should demand, or you should demand, from the  PHP developers, give us PHP 7 next year.  And you should do your part in testing the Git repository  snapshots of PHP 7.  And try to bear breaks with your code base early.  Since the earlier you report issues, you're seeing the  earlier that can be fixed and the earlier it can be released.  And that's mostly I can think of right now.  And then the subject, if you have any questions related to  that or any other questions, feel free to ask them.  And maybe I'll answer them.  And otherwise, if you don't have questions, there's beer  and barbecue and stuff outside, I think.  Or maybe inside.  But sure, and not sure if it's from now.  Oh, damn.  So yeah, I need questions.  So please ask questions.  Yes?  Does it break or extension?  It's huge.  I just had to make a joke first.  So the question was about how much it affects extension  developers, since there are people doing quite impressive  work on top of the PHP API to make it usable with a proper  programming language called C++, which really makes some  things easier.  And it's an issue since we have a migration guy telling  you some of the things.  But every place you're working was hash tables you have to  change, basically.  And some of the type usage you have to change.  That's it for typical extension.  You might, to some degree, automate it, basically when  you have a set while star, star, take a star away.  Keep the same macros.  If you're using the isboolean macro, I think you have to  change it.  If you're using arrays, you have to slightly change those  since there's a different type.  Oh, and strings have been changed.  They are now of type send string and not child pointer  anymore.  And then new string API.  And then the string, and the string, and the string.  But on the wiki PHP net, somewhere there's a migration  guide.  Just go to RFCs, then go to PHPNG.  And on the bottom third of the RFC is linked to the  migration guide for extension authors, which then also links  by the way to the internals of PHPNG.  So it's not on the RFC, but you have to go to the migration  guide to get to the internals.  And there you can find the information.  But it's notable work to adopt to it.  Other questions?  Or if that's not enough, then?  Well, the current set of extensions that I said, although  there must be hundreds of extensions, they will all  break.  Yes.  All extensions will break, and all extensions have to be  adopted.  And how much of the extensions are managed by PHP team?  I don't know how much.  So the question is how many extensions are managed by the  PHP team, and as every extension basically has to be  touched to be adopted to this new world.  The number of extensions, I don't know, since there are  many, many internal, only company specific extensions.  Then there are tons of un-maintained and unused  open-source extensions in PICL, in the PHP extension  repository, PICL.PHP.NET.  And then there's a number of used ones.  And out of those used ones, there might be some which are  worse maintained and some which are better maintained.  So if you're looking at in common extension something  like xDbug, which isn't part of the PHP distribution, but  is very well maintained, that will be adopted quickly.  If you're looking at some extension like the Memcache  extension in the PICL repository of Memcache D, they  are a bit less maintained.  But I guess they are important enough that somebody  will work on it.  And if you're done going to even less used ones, it might  take some time or your time.  So the interesting point is if you have more or less simple  PHP extensions, more or less wrapping only a C library,  and not doing any object-oriented stuff, it's  very little work to no work to get them from PHP 40 to  PHP 5.6.  The incompatibilities are quite small there.  Less than an hour of work, pure work without all  integration testing and what you could do.  So there aren't many changes.  And the changes there are kept in quite small areas.  Now, this is a huge change.  And this will really kill some really old extensions  nobody should use.  Anyway, since they are maintained, obviously, as  they will hopefully be adopted.  Since I think the performance improvements, the memory  improvements, the springs are so much that the  community will cry more than they will cry about some  new syntactic sugar.  I don't know what the latest syntactic sugar is, which is  being introduced.  I don't follow the syntactic sugar.  I've come from an hosting company.  We see sales very, very rarely.  Like very exotic extensions.  What we see is like between the community with the  framework with the application, and they build  all the certs.  Mostly they are into core components.  So some with that support SIP and HTTP and HTTP.  Yeah, so HTTP extensions, a good example, that might  probably become part of the core distribution.  And the other extensions you mentioned are part of the core  distribution.  So either they are thrown out.  I can imagine that some crappy stuff we're having in  there, which nobody is using actually, might be thrown out  in the process.  I don't know.  Some of the un-maintened PDO extensions.  I won't comment on that since MySQL is the only database  you should use anyways.  I'm paid for saying that.  So I won't comment on throwing out other databases.  But yeah, in fact, that was an area I was thinking about.  Some IBM DV2 PDO driver, which was broken anyways, and  where nobody from the PHP developers can commit, since  IBM lawyers don't like that.  Such things might be thrown out.  So most of that work is done by Zend, which is mostly  dimetry, and they hired two guys more or less for that.  I don't know if they hired them full time or just.  No, not full time.  Lawrence and PTC, are they working with that?  Yeah, I don't know the contractual details.  But so they sponsored at least work from two other guys.  One is called himself LaRuins, which is a type of  company from LaRuins, and he's a Chinese guy.  He recently got into the PHP core community, and is also a  great guy from technical perspective.  He is a bit shy about communicating, since he feels  his English isn't good enough.  But that's the same for most of the contributors, which  makes it a bit hard sometimes to discuss.  But he's a great guy, and does lots of work.  And Nikita is also quite into these things and  send a sponsoring that.  But I don't know how the relationship, who does what,  and so on.  And I don't care.  I don't care who is paying whom for working on PHP for the  most part of it, unless for stupid comments.  So that was the question.  I'm making sure that these two developers have a share of  things together.  So question is about our bus factor solution issue.  So bus factor means having an important developer being  run over by a bus.  And what happens to the project?  And well, you have Dimitri.  I think he's still living somewhere in Russia.  You have LaRuins in China, and Nikita somewhere in Europe.  And I have a hard time imagining a situation where  they travel from one location to the exact same location.  They might probably go all due to maybe Send Consents, Send  Past, and once their guy's there.  And they might share a flight from Frankfurt or some other  European hub.  And I don't know if Send has any precautions of an  accident happening there.  I'm not so much into the disaster recovery stories  from that.  I know it from other companies, but barbecue has been  opened.  So if you're running out now, I get the impression that you  don't like me.  I prefer food and alcohol over me.  Alternatively, I was so bad that you  needed to become drunk.  And anyway, thank you.
phpng_-_heap_and_stack_allocation__FrOSCon14__0_755.png phpng_-_heap_and_stack_allocation__FrOSCon14__756_1127.png phpng_-_heap_and_stack_allocation__FrOSCon14__1128_1883.png phpng_-_heap_and_stack_allocation__FrOSCon14__1884_2255.png phpng_-_heap_and_stack_allocation__FrOSCon14__2256_2627.png phpng_-_heap_and_stack_allocation__FrOSCon14__2628_2999.png phpng_-_heap_and_stack_allocation__FrOSCon14__3000_3383.png phpng_-_heap_and_stack_allocation__FrOSCon14__3384_3755.png phpng_-_heap_and_stack_allocation__FrOSCon14__3756_4127.png phpng_-_heap_and_stack_allocation__FrOSCon14__4128_4499.png phpng_-_heap_and_stack_allocation__FrOSCon14__4500_4883.png phpng_-_heap_and_stack_allocation__FrOSCon14__4884_5255.png phpng_-_heap_and_stack_allocation__FrOSCon14__5256_5627.png phpng_-_heap_and_stack_allocation__FrOSCon14__5628_5999.png phpng_-_heap_and_stack_allocation__FrOSCon14__6000_6383.png phpng_-_heap_and_stack_allocation__FrOSCon14__6384_6755.png phpng_-_heap_and_stack_allocation__FrOSCon14__6756_7127.png phpng_-_heap_and_stack_allocation__FrOSCon14__7128_7499.png phpng_-_heap_and_stack_allocation__FrOSCon14__7500_7883.png phpng_-_heap_and_stack_allocation__FrOSCon14__7884_8255.png phpng_-_heap_and_stack_allocation__FrOSCon14__8256_8627.png phpng_-_heap_and_stack_allocation__FrOSCon14__8628_8999.png phpng_-_heap_and_stack_allocation__FrOSCon14__9000_9383.png phpng_-_heap_and_stack_allocation__FrOSCon14__9384_9755.png phpng_-_heap_and_stack_allocation__FrOSCon14__9756_10127.png phpng_-_heap_and_stack_allocation__FrOSCon14__10128_10499.png phpng_-_heap_and_stack_allocation__FrOSCon14__10500_10883.png phpng_-_heap_and_stack_allocation__FrOSCon14__10884_11255.png phpng_-_heap_and_stack_allocation__FrOSCon14__11256_11627.png phpng_-_heap_and_stack_allocation__FrOSCon14__11628_11999.png phpng_-_heap_and_stack_allocation__FrOSCon14__12000_12383.png phpng_-_heap_and_stack_allocation__FrOSCon14__12384_12755.png phpng_-_heap_and_stack_allocation__FrOSCon14__12756_13127.png phpng_-_heap_and_stack_allocation__FrOSCon14__13128_13499.png phpng_-_heap_and_stack_allocation__FrOSCon14__13500_13883.png phpng_-_heap_and_stack_allocation__FrOSCon14__13884_14255.png phpng_-_heap_and_stack_allocation__FrOSCon14__14256_14627.png phpng_-_heap_and_stack_allocation__FrOSCon14__14628_14999.png phpng_-_heap_and_stack_allocation__FrOSCon14__15000_15383.png phpng_-_heap_and_stack_allocation__FrOSCon14__15384_69791.png phpng_-_heap_and_stack_allocation__FrOSCon14__69792_73511.png phpng_-_heap_and_stack_allocation__FrOSCon14__73512_77915.png phpng_-_heap_and_stack_allocation__FrOSCon14__77916_79265.png
to start in this B-Miles. Right. Thanks, Koehe. So the back story about this talk is I have a session, oh, maybe for an introduction, I'm Johannes Schlüter. Until last week, I was release manager for PHP 5.3, which now released its end of life, and it's dead, and it's gone, and I hope nobody of you is using PHP 5.3 or older anymore. And I hope you're all excited about PHP 5.6. Yeah, that's what I wanted to hear. And I do different talks. For some time, I did all the PHP 5.3 talks, and then the topic became boring, and new versions came out. And I had a talk about PHP under the hood, explaining some things about how PHP works, how reference counting works in PHP. And then there was some work done by a company called Zend to improve PHP's performance by massively changing the internals of PHP. So if I give this talk nowadays, it doesn't really apply for the future, and we know it won't apply what I'm going to talk about in that talk. Then for this year's Crosscon conference, they were looking for more technical talks, more experimental talks, not the default, oh, it's just web scripting talks, but in-depth talk. And Corin, I chatted shortly about it, whether something like that might work out, talking about PHP under the hood or PHP and G at this conference. But back in the times, I didn't really feel comfortable talking about PHP and G, since it was quite new, just announced by Zend that they are going to do it. It wasn't clear whether it will be merged into PHP and so on. And so that slowed down again. I kept it in the back of my head, but it didn't really prepare any talk about it yet. And then yesterday, I was noticed that the speaker was ill, and I said, well, yeah, let's try it. And therefore, I don't have any slides or anything. So in this PHP under the hood talk, what I'm talking about is what I'm trying to teach people is that references are bad. I assume everybody of you is knowing what PHP references are. So you have your variable and its $B. I hope you can roughly understand those dollar signs. My handwriting is bad and even worse on the whiteboard. And if you do this, this ampersand syntax, you get a full A here. So $A and $B directly refer to the same thing in memory. The thing in memory we are having is called a Zval, which is a send value derived from previous PHP three times. There it was called PHP value, but then proper or better engine sent. Engine came along, and the name was changed to Zval, to represent that. And a Zval is a quite complex structure. I won't write it in complete here. But it's a C structure. Let's take a shortcut. It's an angle bracket. And we have multiple fields in there. One is an integer about a ref count and an integer for is ref telling us how many of those variables are pointing to a specific variable. So after this line is executed, we have one of these data structures in memory. And we are having the ref count of two since $A and $B are showing on it. And as we have true alias, as we said, this is ref flag to the value of one, just tell us, OK, it's a value. We have more data in here. The important part is a union value. It should work on my handwriting next time, which has some different fields like integer for a long value, and another struct. Struct. I should go there. For string values, it's a char pointer and an int for the length, and some other things for other types around. It's not complete. It shouldn't be there. And so all different types we're having in PHP are represented in this data structure. And a union in C is a data type. We are basically the maximum value for the maximum type is allocated in memory. And depending on how you access it, it's interpreted in the according way. So the largest data type we're having in here is a char pointer and an integer for a string and a length. And so the size of the union will be the size of the char pointer, which is depending on your system, 32 bits, 64 bits, and the size of an integer. It's an integer. So yes, which might be 32 bits or something like that. And then we have these additional fields. And they are kept here in memory. And then we have an additional field called type. I'm not sure whether it's a correct name, but that's basically the set-world structure. So that we keep for every of our, you're taking a photo of my bad handwriting. You're all my fault. And there we keep all the data in memory. Now we're having an issue when doing it this way. That's the way how we reference to it. Since it's nice that we're having one container for those variables, but we need to point somewhere from these variables to this reference. And for that we have a data structure, which is a hash table. From memory perspective, we can simply follow our hash table and assume it's in C array. It's not exactly true, but it's the closest we can get to it. And if we look at a computer from Neumann architecture, what we are typically using these days, not exactly the form it is, or anybody here has seen a lecture about computer science 101 or Grundlagen informatik in German, a few studied people. Then that shouldn't be new to anybody. In a von Neumann architecture, you have your memory with slots of the same size. And they are all addressable with sequential numbers. So there's address 1, address 2, address 3, and so on. And we have somewhere on end with some other numbers. Yeah, it happens. Hems don't like me, and I don't like pens. So we have this memory layout where we can address the different areas. Now we're having an array for our values. So we place it in here, something like that. We have three pointers here. That's not exactly what it is, but we can assume it, pointing us to our structure, which are in some other memory address. So that's maybe here. And then we have a pointer pointing there. And we have a second pointer pointing there, too. So when accessing a variable, the PHP engine has to go first, or the system has to fetch that memory first. And then dereference the pointer and go here to fetch the actual data. So there's an in direction step. Now we have these things called CPUs. We're going deeper. If memory wasn't enough for you, we have our CPU, as you can clearly see. And the CPU, one key component is the Arithmetic Compulsion Unit, ACU, is that correct? ALU, Logic Unit. Or whatever. But that unit doing actually the math in the computer. Then you have your registers here, which are very small. Places of data basically can take the word size, so 32 bits or 64 bits or something like data, in one of those registers directly in the CPU. Then you have your L1 cache, your L2 cache, your L3 cache, the bit depending on your CPU. If you're looking at memory now, we know RAM sizes are in the gigabytes. I think I don't know what a typical server for you has, some quite few gigabytes. But if we're looking at here in the CPU level, we're talking about something like five megabytes or three megabytes or something like that, depending on your system's architecture, the specific CPU. And it becomes smaller and smaller and smaller. And here we are really at word size level. Now we have the second issue here, that it takes time to get from here to here, or data from there to there. Our limiting factor there is the absolute limiting factor. It's not the only one. It's called speed of light. Speed of light is, who knows it, how many kilometers per second? 300,000. 300,000, 299 or something like that. So then we have three gigahertz as a CPU frequency, which means in the time of one CPU cycle, the information can travel at most. Who can do the multiplication? It's 9 centimeters. Oh, depending on the 300,000 times the gigahertz, 8.9 is what I calculated yesterday. So the maximum distance our information can travel within one CPU cycle is, what did I say, 9 centimeters or something like that? And it has to go back and forth. And if you're looking at our main board, we see, we are a bit over that. So we are wasting CPU cycles while waiting for the memory. In real life numbers, just to give you an impression about what's happening here. Rough estimates, I found those numbers on the internet, so they must be true by definition. Accessing RAM, we can estimate about 100 nanoseconds. Accessing, I only have L2 cache, but it doesn't matter too much, it's 7 nanoseconds. And L1, 0.5 nanoseconds, and accessing registers more or less immediately. So by having to go to memory to access the data, we go from 7 nanoseconds. For data we have already cached here to 100 nanoseconds, which is a lot more. Of course, it's nothing if you look at it. But if you then think about, again, you have 3 gigahertz as frequency, and you're waiting for 100 nanoseconds for your data to process, it's quite some time you have to wait. And the CPU basically has nothing to do with it. So to improve this, the system, the CPU on one side and the compiler and everybody is trying to help by not reading the single data packet here, but trying to read more of it at once, and trying to put it in the caches. So we are not reading only this, but maybe if you're only accessing the LA, the CPU predicts that we're going to read this, and put at least an add of 3 cache, if there's enough space. That gives a huge improvement in the performance here. But we still, but the CPU can't predict that we're going to make this jump, and going to access that memory in most circumstances, depending on your CPU, depending on your compiler, that might look different. But in general, it's hard for the CPU to do. Now, we have to state a structure with this indirection here. And we want those references. And we have this issue. What can we do? We can change this behavior, and we can change these hash tables. When I thought about it, I thought about doing it next to each other, but I have to switch. So what the Sand guys came up with was redefining the hash table, inside the hash table. You're not actually storing the pointer, pointing there, but the actual value. So when we're drawing our memory again, we have to switch colors just to make that a bit more practical. We have our, that's now one entry for our hash table. So that's, all of that is $A. And inside there, directly, I don't have a value, but let's assume I have a value of 23 for that. We have some meta information, and we have the actual value of 23 right in that data structure. And if we are now having an array of data, the next element is directly next to it. Yeah, let's say $B, which already shows a small issue we're having. And that has some data. I'm not going into details right now. So what we're having here is, in the old version. Sorry, but there's two, it has some not really even now. Wait, wait. Coming exactly there. So in this old case here, we add inside this hash table with some meta data and so on, a reference to the actual data. And we could simply reuse this. In my old talk, I mentioned before, the main message was don't use references. Since doing this prevents an optimization inside the PHP engine, which tries to do copy and write. When we are not having a reference here but copies, and we don't change it, the picture here will be exactly the same. Only difference is that we are not marking it as references in the data structure here. But this stays the same. And if you have one reference and one copy, you can't reuse the container, but you need a second container to either hold the reference value or the copied value. And that's why references are bad, but that's another topic at talk. You can look at that with slides and everything on YouTube if you search for your hannish-luter PHP under the hood. There's a version from PHP UK conference about that. Now, as you directly saw, when we're needing a reference, and we are directly storing the value inside the actual data structure here, this doesn't work out that way. That's why there's more complication to it. What they already also did is I should have put it next to each other. I should have thought about it more. Changing the set world structure to a structure where you have type and you have a union and I think one more field, which I don't remember right now, but it doesn't really matter. And now the type here. Oh, let's take the other way around. The type flag in here directly represents the type you're seeing in PHP. If you're looking at the variable, it has a type of an integer or long, which is the same in PHP speak. It has the type of a string. It has the type of an array. It has the type of something like that. That's a numeric value. You have this is constant, I think, exported to PHP userland and the same ones are used here. In the new one, we have more types here. We have specific types saying it's long. We have specific types like here saying it's a string. And we have a specific type here saying it's a reference type. And if you say it's a reference type, then basically this happens again. But with more memory usage since the data structure is a bit, or that data structure is a bit bigger. And then we have this interaction again. But if we don't need an interaction for references, we can directly inline the data inside the hash table. Now, the next optimization this brings, or is that so for clear? The next optimization this brings by having those values next to each other in memory is, again, if you're looking at our different cache layers here, often operation we're doing is some form of iteration over in the data when searching for something, when looking in an array, when doing a for each on an array or something like that. By directly having the data next to each other, the CPU can take more of the memory at once closer to its core, to its calculation units. And without going back and forth all the time between different memory areas, then jumping somewhere else, and keeping in mind those are really small, it's quite nice to have this data really compact close to each other, everything. Now, there's another effect we're having here, which brings us back to our nice Mr. von Neumann architecture. Oh, just for more or less informational purpose. If you're reading data from an SSD drive, I just put some more numbers out just for giving a feeling what happens. We are at 150,000 nanoseconds, roughly for trusty SSD access. And then I have something about hard disk. So one megabyte reading from a spilling hard disk drive, one MB is 10 million nanoseconds. That's what I have brought here about numbers. So it's a huge gap if you're going to that kind of memory, just to give an impression of these numbers. So now, another thing is, if we're handling these variables in this old way, we have quite some common cases where we need temporal values. If you have in calculation, this is one working, 1 plus 2 plus 3, and equals $C. And as long as we don't optimize it, we need a temporal variable here. And then copy that, and then reference it again for that variable. And for all of that, we have to go to memory and find some space where we can put it. And basically, we have two choices when accessing memory from system perspective. One is putting it in heap memory, which basically means give me some memory area here, and reserve it for me so I can put data in and take data out. That takes quite some time on a system, speaking about in those many quads of time. The other approach is, again, looking at our von Neumann architecture. And our nice things we're having here is our stack. So we have one register, which is a stack pointer. And the stack, which is in Germany, nicely also called sometimes Kellerspeicher. So seller memory is at the bottom of our memory. So we have address one. And we have the maximum memory here. And when our program starts, the stack pointer points here. So this register has the address of that one. And each time, a function is called on a system level, so a C function from the PHP implementation. Somewhere annotated to that function is the information how much stack memory it's using. And it's simply changing this value accordingly, increasing by the needed space. And then the pointer shows here. And the function can use the space it needs. And that's a really fast way to access memory. Since all for allocation on a function call, you change that value. And when you leave the function, you change it at the other route. Unless you're using C++ with destructors or something like that, it's really, really a fast operation. I had some thought on my mind, but I forgot it. Anyways, I'm improvising, as you might have noticed. The data stored in here. And now, we don't have this overhead of putting it. My thought was, yeah, no typical issue about the PHP program, function foo calls function foo. And we all know what happens on PHP. We get a segmentation foo due to a stack overflow. What happens is we're calling function after function after function. So this one grows, grows, grows, moves up, moves up. And in some area, the operating system has a limit and says max stack space for this process is here, or for this thread, actually, not process. And if we reach that limit, the operating system, or actually the CPU, which knows this limit, will signal the operating system, which will signal the application. Sorry, no more stack space and goodbye process. And you get a segmentation fault. And detecting this in an incompatible way is something we don't do from PHP level. So you get a segmentation fault. So that's the reason for this one. Now, if we're having our temporary wayables and we are putting, taking this indirection out, we can directly put the temporary wayable here in our stack, put it there, keep it there. And if we need it, we copy it. If we need it somewhere else. So when the function exits, and we still need to value, we have to copy it out. Otherwise, we can forget about it. And it's gone from, well, it's still in memory. But in an area not accessible by anybody. And that's the second optimization we're having in PHP for the general purpose case. And these changes in data structures lead to numbers I don't know that anybody knows how much the factor is from you, well-informed people, something like one third of the performance improvement over PHP 5 for this PHP upcoming version, which nowadays has a version number 7. And mostly, you're really changing those data structures. While there's another change in the data structure, for instance, if you take Boolean values, currently in this implementation, we have a flag saying it's a Boolean variable, and then additional field for the value. They are now combined in one. So basically, true and the false are treated like they were types, not like values. So we need one lookup to access it's a true variable or it's a false variable, and not two lookups. And some more of those small improvements on there. Now, what can we ask you with this information? What's the time, by the way? So how much time? OK. Almost half time only. Oh, damn. It's the last talk, though. And party time is coming up. But what did I just say? All right. OK, so the initial issue which led to really changing these data structures was that Dimitri, who's the guy from Zendt, who's really in depth about doing all those optimization and thinking about those, wanted to do some just-in-time compilation of PHP code into native code to have even more performance impact. And when doing this, he figured out that this indirection kills every optimization since we can't really predict what will happen. With this model, there's a chance that we're not only seeing this improved data structures, which result in overall less memory usage and higher performance since access times become really slow. But that we can let the step towards just-in-time compilation, since with the values directly telling us the type, we can be sure about this and therefore give it to JIT engine to say, it's that type. Go with it. Optimize it. Make native code out of it. And? So almost work, I guess, like the optimization needs to be done, and then they have like experimental G to the engine of the app. Yeah. And then what do you do about it? Like, I think it will beat me, but the decision to simplify it's time. Well, the question is, this becomes way more complex if you look at it. If you really look at the code, it's more complex than what we had in the past. So it's not a simplification, but it makes it more complex. But for the user perspective, it's faster. It's more efficient. And you, if you're using PHP, you don't see anything of that. You see that. And but it's this first step towards JIT compilation. And there was a recent post by Dimitri starting a heated debate about one of everybody's favorite topic, static type hinting for scalar types. Since you really figured out, if we know in the declaration of our function we're having this type, we can do so much better JIT implementations. And after reading that statement, I thought, damn, he's quite close to having a JIT. I don't know any details about what he's really under. How close he is. But from that posting it sounded like there might be a chance that next week or something is coming up with a new idea for a JIT implementation, bringing again some huge performance improvements. The other thing in meantime Dimitri did, I never met him, but I really like what he does. Since he's really no big time about discussing, but delivering and improving things for quite some time in PHP. Even with that model, we have to allocate quite some memory on the heap or free memory or whatever you want to call that area, not the stack area, of the memory. And operating systems typically are optimized. But you start them at some time, allocate quite huge blocks of memory, keep them for, in computer speak, long time, and then free them again and do something else. What you're doing in the PHP script is you need these relatively small data blocks all the time. You need a new variable. You free it. You need a next variable. And new temporary table gives quite a lot of memory allocation operations. Memory allocating, finding good spot that you're not wasting too much memory and keeping it close to each other and keeping the total thing when multiple processes are running at the same time takes huge amount of time. And so we have an optimized memory manager already in PHP, which knows, OK, I'm having quite a lot of data with this size. And preallocates slots for that and reuses them and manages those. And what Dimitri, I haven't looked deeper into that implementation yet. And I think last week, or maybe the week before, it was a new memory manager. So he's not only working on changing the data structures, introducing a JIT engine, but also while writing a new memory manager. And again, improving performance of PHP quite a lot. So now there was quite some fuss in the PHP community related to this work. Now we're getting into politics. Or kindergarten. No, I won't read internal postings. So there's an operating system in a word which is called what's the name? Not doors, not gates, windows. That's it. And some people use that system. And there's a company in Redmond which really pushes that operating system and tries to sell it. I don't know why they are doing it, but they are doing it. Oh, this is recorded, isn't it? Damn. That's serious. I'm just. And what they are working on is they try to unify the experience, making it more portable that PHP scripts on different systems and different platforms behave the same way. And now we're having one thing in PHP, which actually is an error here. Now that I see it. This one is the long one. It's actually of type long. And the property of this type long is from the C standard, it's not clear what size long should be, which leads to different interpretations. In the Unix world, long is interpreted as if you're running on a 32-bit system, being of a size 32. If you're running on a 64-bit system, it's size 64-bit. On the Windows operating system, it's defined to be already 32-bit, just to make porting code easier on Windows from 32 to 64-bit. But I think that's also a change in the value when they would probably compile. I don't know about Windows, and I don't really. The point is there's this difference on the one side between Unix machines themselves, if you're running 32-bit and 64-bit, and with Windows machines, which then behave differently again. And what many people want from a scripting language is it behaves the same on all platforms. And those guys from Microsoft also figured out that we are using the int type here to represent the lengths of the string. If you look closer to C standard and recommend the programming guides, you see that int isn't the size one should take, but one should take the type called size t. And so they spent quite some time in an open process by early on telling everybody we're doing this and we want to do this to change those types. To make sure that on all platforms independently, whether it's Unix, 32-bit, and Unix, when saying Unix, I include Linux, which is a new Unix, other religious discussion, I'm working for a true Unix vendor, by the way. To always be 64-bit on all machines and all systems everywhere. And to have this type on a proper type, so size t. Long size t, S size t, just to be 64-bit all the time. So independently of your system, it behaves the same with high integers and so on. And long strings. I don't know if anybody needs strings longer than two gigabytes in memory, but if you do with PHP, it should behave the same on all platforms. That's the ideal. And it's a huge change in PHP to change that. And they announced that they were doing this. They were doing this publicly in PHP's Git repository in a specific additional branch, which was directly next to the main PHP branch. And suddenly, the send people came and said, well, we changed all the data structures. We worked hard on reducing memory usage. And we want to push that. And we didn't communicate with you beforehand, since it was an experiment, and we didn't know whether it would work out and or not. We don't want to send out all the ideas. We're having all the time, since that's too much, since they're really actually working continuously for years on different ideas, how to optimize PHP. And 90% of that are thrown away since it doesn't work out. That idea about changing the data type and so on, that seemed to have worked out. We don't have major bugs yet. Maybe you find them if you test what's now called PHP 7 from PHP repositories that has it with the applications. You might find bugs, but we don't know major bugs yet. Might want to push it. Now, this change here increases data usage. You need more memory if you're always having 64 bits here. You're using more memory if you're not using a small integer type here by a large integer type, which is kind of opposite. Now, you have the one company pushing it really hard to get their change in, since it's a huge improvement for PHP. And they are saying, that's what people are interested in. People want performance. And you have the other guy who's working for big Microsoft with other guys helping him, doing open source, open development from Microsoft, and doing this change. And they are more or less beaten down. I'm defending Pierre, by the way. New feeling, but heavens. And the same guy is saying, increasing memory usage and performance is the most important part. And they are saying, well, we need a consistent behavior, and we need better APIs internally, better types internally, and not ugly types, which that partly has internally. It's a more complex structure, needing more different operations and different types. And so there was a huge debate. In the end, they worked together silently and recently put this work on top of that work. And even in a way that in some cases, you even get more performance out of it. So out of the, oh, yes, so slow, they figured out if you're working, cooperating, we get something which is even better. We fulfill more or less both needs from both sides and putting stuff on top of it. And let me think, is there anything else on that level in the future of PHP? I don't think so. So if everything goes well, everything I talked about will be released as PHP 7. Now the debate is whether we will have a 5.7 release in between or not. From my personal opinion, I think we should focus on getting that one out. And if we figure out, oh, we can't do it. And we can do an immediate release. But if we say, oh, we can't do this. And we can't do it in a year, then we do some other stuff and implement other features which are piling up. People want to have big int support. So you can have integers higher than 64 bit, or that kind of stuff, and pushing for those changes. And then you get a 5.7. And then nobody has worked on that since the 5.7 got the priority. And then you figure out, oh, it will take so much more time until we have PHP 7 ready. So let's make a 5.8 and a 5.9 and a 5.10. And then we do the same what we did with PHP 6. And we kill PHP 7. And then we get to PHP 8. And so my opinion is we should try to push PHP 7. So we should demand, or you should demand, from the PHP developers, give us PHP 7 next year. And you should do your part in testing the Git repository snapshots of PHP 7. And try to bear breaks with your code base early. Since the earlier you report issues, you're seeing the earlier that can be fixed and the earlier it can be released. And that's mostly I can think of right now. And then the subject, if you have any questions related to that or any other questions, feel free to ask them. And maybe I'll answer them. And otherwise, if you don't have questions, there's beer and barbecue and stuff outside, I think. Or maybe inside. But sure, and not sure if it's from now. Oh, damn. So yeah, I need questions. So please ask questions. Yes? Does it break or extension? It's huge. I just had to make a joke first. So the question was about how much it affects extension developers, since there are people doing quite impressive work on top of the PHP API to make it usable with a proper programming language called C++, which really makes some things easier. And it's an issue since we have a migration guy telling you some of the things. But every place you're working was hash tables you have to change, basically. And some of the type usage you have to change. That's it for typical extension. You might, to some degree, automate it, basically when you have a set while star, star, take a star away. Keep the same macros. If you're using the isboolean macro, I think you have to change it. If you're using arrays, you have to slightly change those since there's a different type. Oh, and strings have been changed. They are now of type send string and not child pointer anymore. And then new string API. And then the string, and the string, and the string. But on the wiki PHP net, somewhere there's a migration guide. Just go to RFCs, then go to PHPNG. And on the bottom third of the RFC is linked to the migration guide for extension authors, which then also links by the way to the internals of PHPNG. So it's not on the RFC, but you have to go to the migration guide to get to the internals. And there you can find the information. But it's notable work to adopt to it. Other questions? Or if that's not enough, then? Well, the current set of extensions that I said, although there must be hundreds of extensions, they will all break. Yes. All extensions will break, and all extensions have to be adopted. And how much of the extensions are managed by PHP team? I don't know how much. So the question is how many extensions are managed by the PHP team, and as every extension basically has to be touched to be adopted to this new world. The number of extensions, I don't know, since there are many, many internal, only company specific extensions. Then there are tons of un-maintained and unused open-source extensions in PICL, in the PHP extension repository, PICL.PHP.NET. And then there's a number of used ones. And out of those used ones, there might be some which are worse maintained and some which are better maintained. So if you're looking at in common extension something like xDbug, which isn't part of the PHP distribution, but is very well maintained, that will be adopted quickly. If you're looking at some extension like the Memcache extension in the PICL repository of Memcache D, they are a bit less maintained. But I guess they are important enough that somebody will work on it. And if you're done going to even less used ones, it might take some time or your time. So the interesting point is if you have more or less simple PHP extensions, more or less wrapping only a C library, and not doing any object-oriented stuff, it's very little work to no work to get them from PHP 40 to PHP 5.6. The incompatibilities are quite small there. Less than an hour of work, pure work without all integration testing and what you could do. So there aren't many changes. And the changes there are kept in quite small areas. Now, this is a huge change. And this will really kill some really old extensions nobody should use. Anyway, since they are maintained, obviously, as they will hopefully be adopted. Since I think the performance improvements, the memory improvements, the springs are so much that the community will cry more than they will cry about some new syntactic sugar. I don't know what the latest syntactic sugar is, which is being introduced. I don't follow the syntactic sugar. I've come from an hosting company. We see sales very, very rarely. Like very exotic extensions. What we see is like between the community with the framework with the application, and they build all the certs. Mostly they are into core components. So some with that support SIP and HTTP and HTTP. Yeah, so HTTP extensions, a good example, that might probably become part of the core distribution. And the other extensions you mentioned are part of the core distribution. So either they are thrown out. I can imagine that some crappy stuff we're having in there, which nobody is using actually, might be thrown out in the process. I don't know. Some of the un-maintened PDO extensions. I won't comment on that since MySQL is the only database you should use anyways. I'm paid for saying that. So I won't comment on throwing out other databases. But yeah, in fact, that was an area I was thinking about. Some IBM DV2 PDO driver, which was broken anyways, and where nobody from the PHP developers can commit, since IBM lawyers don't like that. Such things might be thrown out. So most of that work is done by Zend, which is mostly dimetry, and they hired two guys more or less for that. I don't know if they hired them full time or just. No, not full time. Lawrence and PTC, are they working with that? Yeah, I don't know the contractual details. But so they sponsored at least work from two other guys. One is called himself LaRuins, which is a type of company from LaRuins, and he's a Chinese guy. He recently got into the PHP core community, and is also a great guy from technical perspective. He is a bit shy about communicating, since he feels his English isn't good enough. But that's the same for most of the contributors, which makes it a bit hard sometimes to discuss. But he's a great guy, and does lots of work. And Nikita is also quite into these things and send a sponsoring that. But I don't know how the relationship, who does what, and so on. And I don't care. I don't care who is paying whom for working on PHP for the most part of it, unless for stupid comments. So that was the question. I'm making sure that these two developers have a share of things together. So question is about our bus factor solution issue. So bus factor means having an important developer being run over by a bus. And what happens to the project? And well, you have Dimitri. I think he's still living somewhere in Russia. You have LaRuins in China, and Nikita somewhere in Europe. And I have a hard time imagining a situation where they travel from one location to the exact same location. They might probably go all due to maybe Send Consents, Send Past, and once their guy's there. And they might share a flight from Frankfurt or some other European hub. And I don't know if Send has any precautions of an accident happening there. I'm not so much into the disaster recovery stories from that. I know it from other companies, but barbecue has been opened. So if you're running out now, I get the impression that you don't like me. I prefer food and alcohol over me. Alternatively, I was so bad that you needed to become drunk. And anyway, thank you.
Johannes Schlüter. Until last week, I was release accordingly, increasing by the needed space. And the application. Sorry, no more stack space and