The State of OpenJDK
Reinhold, Mark
A review of the past year in the life of the OpenJDK Community, and a look atwhat's ahead.
 Good morning and welcome to FOSDEM. My name is Mark Reinhold. I work on Java still. So I was putting these quick slides together for this presentation this morning, the State of OpenJDK, which I've been doing now for a while. And I ran across a logo that you've probably all seen. As you know, a couple of years ago, we celebrated Java being 20 years old. It's now old enough to drink in many jurisdictions. And it occurred to me, you know, this is 2017, and this is the 11th time I've been at FOSDEM, the 11th time many people from Sun and then Oracle have been at FOSDEM. So if I don't have a fence post error, this means that OpenJDK is 10 years old. And all the source code was released in May of 2007. So you know, just looking back a bit, you know, at least speaking personally, thank you everybody for your support. You know, I fondly remember the warm welcome that us folks from Sun had 11 years ago when we first came to FOSDEM, and it was us in the free Java crowd. And I think it's actually fairly amazing what we've all built together. So thank you very much. Okay, so as usual, we've had some growth. We have some improvements. I'd like to talk about a bit and then speak a little bit about the present and the future. In OpenJDK, we have a structure of groups and projects. There were no new groups this year or this past year. We have had a number of new projects. S390 export, those of you who really want to run Java on mainframes, you can do it. JDK 10, you know, 9 is starting to wind down. We need a place to start doing work for 10. So JDK 10 is spun up. We've got forests going and some of the initial sort of paperwork, busywork change sets to change version numbers and source and release levels and all that stuff is floating in there. And then there's a call for votes active right now on project amber, which I'll talk about a little bit more later on. Anyway, so we've got some new projects. We've got a bunch of new committers in this past year. The ones in orange do not work for Oracle. So that's 46 new committers, seven not from Oracle. The non-Oracle ones are range from Intel, Red Hat SAP, and one person from Lennaro. So that's good. We have some new reviewers. Reviewers have a little more stature than committers. They have the authority to review change sets going into the release projects. So these are people with a lot of experience who've earned a lot of trust. The ones in orange don't work for Oracle. The ones whose last names are in orange used to work for Oracle recently. Mr. Shepalev. So they're still a little tainted. Their first names are tainted and their last names are better. Yeah, Alexi, in a tragic career move, decided to leave the company and go to Red Hat. Michael Haupt is off doing other stuff. Anyway, so we have new reviewers. That's all great. Improvements. The vulnerability group. Lawyers. So, I spoke about this, was it three years ago now here, or was it only two? Two years ago, actually did a 30-minute presentation here with the basics of the proposal. The idea to have a group in Open JDK that, unlike any other group, has special dispensation from the governing board to talk about things in secret, because when you're talking about security issues, you need the ability to do that. And I think everybody understands that. To make all of that work, there needs to be a legal document that the lawyers of not just Oracle, spread the blame around, all the big companies, lawyers from all these companies need to read it and vet it and agree that it's reasonable. I'm told by George Saab that we have a near-final document that there's just one detail left to iron out that may or may not require additional changes to the document. With any luck, we'll be able to get this done. What, next few months? George says, thumbs up. Next few months. So definitely, definitely maybe. Oh, well. Okay. Room for improvement. So there have been a couple of fairly big sore points in terms of process and infrastructure. There are still things that were done inside Sun and then continue to be done inside Oracle. And this is, you know, it's not right for what is supposed to be an open development community, so we've been working to fix those. One of those issues is the mysterious CCC, this mysterious review body inside Oracle that sometimes gives people a hard time because they're making changes that are blatantly incompatible or have other issues. The other is the whole hotspot build and test problem where, you know, by now we have a number of experienced outside committers into the hotspot code base, but we still maintain the convention, the discipline which is valuable to keep high quality. Any change to hotspot code has to go through an Oracle internal build and test system. And well, obviously, this is not a good thing long term. So I'm happy to be able to report some progress on these. There's a proposal for a new group before the governing board right now for a new group called the compatibility and specification review group. So this is to replace the mysterious CCC. So a draft of its charter is here. You know, just to be clear, this is not a group of people that's expected to create things or innovate things. You can think of it much more as it's like the editorial board of, or the editorial staff of a newspaper. Their job is to look out for consistency, to look out for, make sure things are motivated to make sure that when an incompatible change is being made that it's justified and that it's risk is understood. And sometimes we do choose to make incompatible changes. So we need to make sure everybody knows about those. And finally, consistency, consistency with the interfaces. When an interface is being changed, it's important to be as consistent as reasonable with surrounding interfaces with the overall conventions of the platform. So the initial membership of the group is here. Doug Lee still doesn't work for Oracle. And he has been one of the principal people reminding us of the problem of the CCC. So as punishment for that, he gets to be on it. So we have prototype support for this process in the OpenJDK bug system. It's reasonably lightweight for a simple change. You create a CSR issue, you fill out some information. What's the problem you're solving? What's the rough shape of the solution? What are the details of the specification that you're changing? And we expect the process itself will be ironed out by the initial group members once the group is up and running, which should be soon. But I expect it will be a fairly lightweight, lazy consensus type of thing, where a proposal will go in, it'll be on a week timer, say, and if nobody objects within a week, then it's just approved unless somebody decides, oh, this is big and I need more time to think about it and they can put it in a pending state and go off and think about it and come back. But then, of course, they're obligated to let the submitter know what the story is. So that's the proposal there. I'm pretty hopeful that we can get this done in the next few months. All right. The second improvement to address the hotspot build and test problem is an experimental mechanism that we're calling Submission Forests. So I emphasize that this is experimental. At least two people in this room, Volker and Andrew, have tried this out, so they've seen the details. So this is a, well, it's kind of a cheap hack. And thinking about this problem more than a year ago, this whole problem, well, you've got an internal build and test system. And the whole thing is a huge effort that, well, yeah, eventually we want to do. But is there some cheaper thing we can do first with the infrastructure that we have without building any, you know, some new conduit for code to come in? Because if we have some new conduit, then, you know, Oracle, internals, network security people get all worried and you have to go through six months of approvals and just a nightmare. Well, we already have a conduit for getting code in, right? It's called Mercurial. We can make as many Mercurial Forests as we want. That's easy. That's an approved input channel. So this notion of submission forest is, it's a forest. On hg.openjdk.java.net, you can create a branch there, put your patch in, push that in, and the internal build system will pick up that patch, do a build, run all the tests, and let you know what the status is. And obviously, if it doesn't work, you've got more work to do. And if it involves some Oracle internal problem, then an Oracle person is, you know, Oracle is obligated to find a person to help you. But if it works, which is the common case, then it will automatically be merged into the hotspot master. So it's pretty simple. Here's an example of how it would work. You'd clone the submission forest, set the def path for it, make a new branch. In this case, we're just making some fake branch with a silly issue number. Here we're going to put the data on the readme. Wow. Commit that, go get your code review, and push that. You push that, you know, within some reasonable amount of time, it usually takes around an hour to do all the internal build and test stuff. You'll get an email from an internal bot saying, yes, it worked. And right after that, you'll get an email notifying you that the change set went in under your name. Or you'll get an email saying, no, it didn't work. Here are some people you could maybe contact to help you resolve the issue. So it's not an ideal solution, but it's some progress. There are more details on it at this Wiki page, on wiki.open.jdk, if you want to have a look at what's going on. So this is still kind of experimental. We're working out some kinks. Andrew and Volker have been very, very helpful in some initial trials. But we'll open this up more generally to people who commit regularly into hotspot fairly soon. Do we still need web reps, if we're going to do this? Do you still need web reps? Well, you still need code review. How you do it? It exists on the branch, on the source, and the server, right? I think the medium of your code review is between you and your reviewer. So if you just want to email a patch around and they're happy with that, then that's fine. Okay, so that's another improvement. So growth improvement present. So there's this big thing called nine. It's, we're trying to get it done. There's a fair amount of stuff in it. There's, believe it or not, a fair amount of stuff in it that's not jigsaw, despite what you may have heard. So here are all the projects of various sizes feeding into it. I'm not going to go through these in detail. There are a whole bunch of JEPs, 89 of them. I'm certainly not going to go through these in detail. But if you have any questions, if you can see them, yeah, they're all here. The JEP process has worked out reasonably well in terms of letting us track things that are finer granularity than project, but a bigger granularity than issues in the bug system. In terms of schedule, we are right about there. So we're in ramp down now. We're trying to get a handle on the overall bug count, trying to figure out, are there important bugs that, well, we really don't think we can fix in time? And this we as everybody, not just people work for Oracle. We need to be deferred to the next release. Are there low priority bugs that we can just shuffle off? All that sort of thing. When we get to ramp down phase two, the knobs will tighten a little more, and we'll be entering a mode where eventually only critical bugs can be fixed. So we're still aiming to have a final release candidate in early July and a GA release in late July. And to the best of my knowledge, those dates are still realistic. But it's software. Anything could happen. Beyond nine, number of active projects, well, or not so active projects that still might be nice to see in some release after nine, not saying any of these are bound for 10 or 11, 12, or anything in particular. Or is this new project that Brian gets just recently proposed? The goal of AMBER is to explore some smaller productivity-oriented Java language features. So you could think of it in complexity scope as somewhat more than coin was. Coin was really just kind of sanding off some small rough edges. Lambda was big. It was big stuff. Panama and Valhalla are also reasonably big. AMBER is somewhere between that. Some of the ideas that Brian wants to explore there include local variable type inference and enhanced enumerations and some of the leftovers from the Lambda design, some things that could be improved in Lambda expressions. Memory model update. This is a project Doug Lee and some others started. It would sure be nice to see some progress on this. I'm not sure. There is a progress. There are a lot of comments. There are, well, there are var handles. So var handles are part of that. But var handles are a feature in the JDK. The memory model is this complex document with lots of symbols in it and a lot of math. And we really need the, we need the people who love doing that stuff to update it. I enjoy reading it from time to time. I don't have the bandwidth to work on that kind of thing myself anymore, sadly. Anyway, memory model. Yeah, it'd be nice. Panama is about interconnecting Java and native code in a better way than JNI because we all know JNI kind of sucks. Intentionally so, by the way, for those of you who don't know. Shenandoah, new GC, making progress, getting there. Maybe that'll show up. And Valhalla is for value types and generic specialization, which is a huge change, not just to the language, but also some fairly deep changes to the Java virtual machine. Okay, I work for Oracle, which means you shouldn't believe the word I've said. And we have a few minutes left for questions before the timer goes off and the recording stops. Yeah. Mario, we have a mic for this purpose, right? We have a mic with a switch. Never mind. Oh, no, it works. Okay. Yes. So, how do you test from these for those statement forests? Are they derived from Oracle or just written by a community? And how frequent they are enhancing? So the tests that are run on the internal build-in test system, some of them are the tests that you find in open JDK, the usual regression in unit tests. Others are more akin to performance and robustness tests that are internal to Oracle in the long term. We'd like to publish them openly, but right now, we don't. Thank you. Oh. I assumed this role. I think it's your exercise that way. First of all, I have to congratulate you that, page with all the JEPs that go into nine is so cool. There's so much stuff in there. Congratulations for getting it all in. Congratulations to the people behind those JEPs. I merely collate them. But yeah, it's a lot of work. Nine is a pretty big release. My second thing is I have a lot of passion for languages on top of the JVM, and I'm wondering if you could say a word or two about the state of that. No. Languages on top of the JVM are a wonderful thing. You know, I'm an old Lisp guy. I love Clojure. I play with it. I write little things in it. It's very cool. If there's more stuff that we can do to make them run better, we should think about doing it. I don't really have much more to say. I don't track all those different language communities closely, but you know, JRuby is clearly really healthy. Charlie might be here somewhere. If not, he'll be here later. Charlie and Tom are here somewhere. Now they have their own dev room, I think. Oh, they have their own. Oh, well. I mean, Ruby dev room, I think, right? It could be. There's JRuby dev room. But yeah, I mean, there's JRuby, there's Scala, there's Clojure, you know, and a whole collection of somewhat smaller projects there. I mean, there's this guy doing small talk thing now. No, it's great. It's cool. It's not just about Java. This is the question. All right. So as you mentioned, that you plan to introduce some improvements on the build process of the OpenJDK. In particular, do you plan to make the build easier for Windows? Because it's quite difficult to build OpenJDK under Windows. It's a lot easier than it used to be. So I don't know if we're going to reply to that. Okay, I don't know of anybody who's planning to do more work on that if you wanted to show up and help do some. A proposal would be welcome. Obviously, Windows is kind of not really the primary development platform and is not the most pleasant platform to develop on anyway. But there it is. So no, I mean, five minutes left. Where's the board? No, no, because the recording won't get it. And if I repeat it, I will not get it right. A bit of a question about Jigsaw. And he's always GI community happy now with what we have in Jigsaw. That's off topic. That's the next talk. But if there are no other questions, I'll give you a quick answer, which is I don't know. I mean, there has always been a prominent OSGI person on the JSR376 expert group. For a long time it was Peter Green's. And now it's been Neil Bartlett. They appear content. I think at this point people understand that Jigsaw isn't OSGI. That we're not using OSGI to modularize the platform because we can't. There are things we need to be able to do that OSGI can't do. OSGI is still a useful technology if you need the dynamic life cycle stuff and all of that. Keep using it. It'll keep working. Mark, I have one question. We have reduced the footprint, for example, by using Jigsaw. The footprint of the GVM is smaller now. The question is, have anyone tracked what's the footprint of a fully working system running Java? Like if you take a kernel, use a space program, what's the minimal footprint required to run OpenUtk? That's a good question. I don't know the answer offhand, but we should measure it. Okay. And Sir Xease was asking a question that I don't know if you're going to address this in the next talk, but as a performance analyst, I'm wondering if you have any numbers or charts for us. No. Can you talk to the state of OpenJDK performance a little bit, 8 and 9? Are you still making pretty dashboards like I did for you before? Oh, yeah. There's still dashboards. Performance is hard. I mean, Shep 11 knows this, right? Performance is really hard. It's not worse, I would say that. It's not worse. We've had some entertaining adventures with Jigsaw because now at startup time, well, you've got this module system that's going and looking for modules and reading stuff in and doing resolution and building a graph and setting all the stuff up in the VM. Oh, my goodness, startup is going to really be horrible. For a while, it kind of was, but we pulled some tricks and invented some new techniques that got almost all of it back. Startup of the trivial hello world thing is pretty close to what it was in 8. In terms of mainline performance, when everything's warmed up and all that, we've continued to invest in C2 and other aspects of the runtime system. Since there are many JEPs, if you want to know about performance, it's probably prudent to just go to these JEPs and see their progress because they usually have the performance comparisons on their target workloads. Like compact strings, for instance, improve some very string-intensive workloads so you can see what's there. But really, the easiest way to do is just download the JDK9 Early Access build and try it yourself on your application because... Or build your own if you prefer. On Windows. On Windows, if you must. That's entirely up to you. All right, so time is just about up. Can I ask a quick one? You didn't mention perhaps one of the most dramatic things that's gone in really quite late in the process, which is AOT compilation, which is a pretty big deal, I think. AOT is a pretty big deal. It's still experimental, but when it works, it kicks ass. All right, thank you very much.
 Good morning and welcome to FOSDEM.  My name is Mark Reinhold. I work on Java still.  So I was putting these quick slides together for this presentation this morning, the State  of OpenJDK, which I've been doing now for a while.  And I ran across a logo that you've probably all seen.  As you know, a couple of years ago, we celebrated Java being 20 years old.  It's now old enough to drink in many jurisdictions.  And it occurred to me, you know, this is 2017, and this is the 11th time I've been at FOSDEM,  the 11th time many people from Sun and then Oracle have been at FOSDEM.  So if I don't have a fence post error, this means that OpenJDK is 10 years old.  And all the source code was released in May of 2007.  So you know, just looking back a bit, you know, at least speaking personally, thank  you everybody for your support.  You know, I fondly remember the warm welcome that us folks from Sun had 11 years ago when  we first came to FOSDEM, and it was us in the free Java crowd.  And I think it's actually fairly amazing what we've all built together.  So thank you very much.  Okay, so as usual, we've had some growth.  We have some improvements.  I'd like to talk about a bit and then speak a little bit about the present and the future.  In OpenJDK, we have a structure of groups and projects.  There were no new groups this year or this past year.  We have had a number of new projects.  S390 export, those of you who really want to run Java on mainframes, you can do it.  JDK 10, you know, 9 is starting to wind down.  We need a place to start doing work for 10.  So JDK 10 is spun up.  We've got forests going and some of the initial sort of paperwork, busywork change sets to  change version numbers and source and release levels and all that stuff is floating in there.  And then there's a call for votes active right now on project amber, which I'll talk about  a little bit more later on.  Anyway, so we've got some new projects.  We've got a bunch of new committers in this past year.  The ones in orange do not work for Oracle.  So that's 46 new committers, seven not from Oracle.  The non-Oracle ones are range from Intel, Red Hat SAP, and one person from Lennaro.  So that's good.  We have some new reviewers.  Reviewers have a little more stature than committers.  They have the authority to review change sets going into the release projects.  So these are people with a lot of experience who've earned a lot of trust.  The ones in orange don't work for Oracle.  The ones whose last names are in orange used to work for Oracle recently.  Mr. Shepalev.  So they're still a little tainted.  Their first names are tainted and their last names are better.  Yeah, Alexi, in a tragic career move, decided to leave the company and go to Red Hat.  Michael Haupt is off doing other stuff.  Anyway, so we have new reviewers.  That's all great.  Improvements.  The vulnerability group.  Lawyers.  So, I spoke about this, was it three years ago now here, or was it only two?  Two years ago, actually did a 30-minute presentation here with the basics of the proposal.  The idea to have a group in Open JDK that, unlike any other group, has special dispensation  from the governing board to talk about things in secret, because when you're talking about  security issues, you need the ability to do that.  And I think everybody understands that.  To make all of that work, there needs to be a legal document that the lawyers of not  just Oracle, spread the blame around, all the big companies, lawyers from all these  companies need to read it and vet it and agree that it's reasonable.  I'm told by George Saab that we have a near-final document that there's just one detail left  to iron out that may or may not require additional changes to the document.  With any luck, we'll be able to get this done.  What, next few months?  George says, thumbs up.  Next few months.  So definitely, definitely maybe.  Oh, well.  Okay.  Room for improvement.  So there have been a couple of fairly big sore points in terms of process and infrastructure.  There are still things that were done inside Sun and then continue to be done inside Oracle.  And this is, you know, it's not right for what is supposed to be an open development  community, so we've been working to fix those.  One of those issues is the mysterious CCC, this mysterious review body inside Oracle  that sometimes gives people a hard time because they're making changes that are blatantly  incompatible or have other issues.  The other is the whole hotspot build and test problem where, you know, by now we have a  number of experienced outside committers into the hotspot code base, but we still maintain  the convention, the discipline which is valuable to keep high quality.  Any change to hotspot code has to go through an Oracle internal build and test system.  And well, obviously, this is not a good thing long term.  So I'm happy to be able to report some progress on these.  There's a proposal for a new group before the governing board right now for a new group  called the compatibility and specification review group.  So this is to replace the mysterious CCC.  So a draft of its charter is here.  You know, just to be clear, this is not a group of people that's expected to create  things or innovate things.  You can think of it much more as it's like the editorial board of, or the editorial staff  of a newspaper.  Their job is to look out for consistency, to look out for, make sure things are motivated  to make sure that when an incompatible change is being made that it's justified and that  it's risk is understood.  And sometimes we do choose to make incompatible changes.  So we need to make sure everybody knows about those.  And finally, consistency, consistency with the interfaces.  When an interface is being changed, it's important to be as consistent as reasonable  with surrounding interfaces with the overall conventions of the platform.  So the initial membership of the group is here.  Doug Lee still doesn't work for Oracle.  And he has been one of the principal people reminding us of the problem of the CCC.  So as punishment for that, he gets to be on it.  So we have prototype support for this process in the OpenJDK bug system.  It's reasonably lightweight for a simple change.  You create a CSR issue, you fill out some information.  What's the problem you're solving?  What's the rough shape of the solution?  What are the details of the specification that you're changing?  And we expect the process itself will be ironed out by the initial group members once the  group is up and running, which should be soon.  But I expect it will be a fairly lightweight, lazy consensus type of thing, where a proposal  will go in, it'll be on a week timer, say, and if nobody objects within a week, then  it's just approved unless somebody decides, oh, this is big and I need more time to think  about it and they can put it in a pending state and go off and think about it and come  back.  But then, of course, they're obligated to let the submitter know what the story is.  So that's the proposal there.  I'm pretty hopeful that we can get this done in the next few months.  All right.  The second improvement to address the hotspot build and test problem is an experimental mechanism  that we're calling Submission Forests.  So I emphasize that this is experimental.  At least two people in this room, Volker and Andrew, have tried this out, so they've seen  the details.  So this is a, well, it's kind of a cheap hack.  And thinking about this problem more than a year ago, this whole problem, well, you've  got an internal build and test system.  And the whole thing is a huge effort that, well, yeah, eventually we want to do.  But is there some cheaper thing we can do first with the infrastructure that we have  without building any, you know, some new conduit for code to come in?  Because if we have some new conduit, then, you know, Oracle, internals, network security  people get all worried and you have to go through six months of approvals and just a  nightmare.  Well, we already have a conduit for getting code in, right?  It's called Mercurial.  We can make as many Mercurial Forests as we want.  That's easy.  That's an approved input channel.  So this notion of submission forest is, it's a forest.  On hg.openjdk.java.net, you can create a branch there, put your patch in, push that  in, and the internal build system will pick up that patch, do a build, run all the tests,  and let you know what the status is.  And obviously, if it doesn't work, you've got more work to do.  And if it involves some Oracle internal problem, then an Oracle person is, you know, Oracle  is obligated to find a person to help you.  But if it works, which is the common case, then it will automatically be merged into  the hotspot master.  So it's pretty simple.  Here's an example of how it would work.  You'd clone the submission forest, set the def path for it, make a new branch.  In this case, we're just making some fake branch with a silly issue number.  Here we're going to put the data on the readme.  Wow.  Commit that, go get your code review, and push that.  You push that, you know, within some reasonable amount of time, it usually takes around an  hour to do all the internal build and test stuff.  You'll get an email from an internal bot saying, yes, it worked.  And right after that, you'll get an email notifying you that the change set went in  under your name.  Or you'll get an email saying, no, it didn't work.  Here are some people you could maybe contact to help you resolve the issue.  So it's not an ideal solution, but it's some progress.  There are more details on it at this Wiki page, on wiki.open.jdk, if you want to have  a look at what's going on.  So this is still kind of experimental.  We're working out some kinks.  Andrew and Volker have been very, very helpful in some initial trials.  But we'll open this up more generally to people who commit regularly into hotspot fairly soon.  Do we still need web reps, if we're going to do this?  Do you still need web reps?  Well, you still need code review.  How you do it?  It exists on the branch, on the source, and the server, right?  I think the medium of your code review is between you and your reviewer.  So if you just want to email a patch around and they're happy with that, then that's  fine.  Okay, so that's another improvement.  So growth improvement present.  So there's this big thing called nine.  It's, we're trying to get it done.  There's a fair amount of stuff in it.  There's, believe it or not, a fair amount of stuff in it that's not jigsaw, despite what  you may have heard.  So here are all the projects of various sizes feeding into it.  I'm not going to go through these in detail.  There are a whole bunch of JEPs, 89 of them.  I'm certainly not going to go through these in detail.  But if you have any questions, if you can see them, yeah, they're all here.  The JEP process has worked out reasonably well in terms of letting us track things that  are finer granularity than project, but a bigger granularity than issues in the bug  system.  In terms of schedule, we are right about there.  So we're in ramp down now.  We're trying to get a handle on the overall bug count, trying to figure out, are there  important bugs that, well, we really don't think we can fix in time?  And this we as everybody, not just people work for Oracle.  We need to be deferred to the next release.  Are there low priority bugs that we can just shuffle off?  All that sort of thing.  When we get to ramp down phase two, the knobs will tighten a little more, and we'll be  entering a mode where eventually only critical bugs can be fixed.  So we're still aiming to have a final release candidate in early July and a GA release in  late July.  And to the best of my knowledge, those dates are still realistic.  But it's software.  Anything could happen.  Beyond nine, number of active projects, well, or not so active projects that still might  be nice to see in some release after nine, not saying any of these are bound for 10 or  11, 12, or anything in particular.  Or is this new project that Brian gets just recently proposed?  The goal of AMBER is to explore some smaller productivity-oriented Java language features.  So you could think of it in complexity scope as somewhat more than coin was.  Coin was really just kind of sanding off some small rough edges.  Lambda was big.  It was big stuff.  Panama and Valhalla are also reasonably big.  AMBER is somewhere between that.  Some of the ideas that Brian wants to explore there include local variable type inference  and enhanced enumerations and some of the leftovers from the Lambda design, some things  that could be improved in Lambda expressions.  Memory model update.  This is a project Doug Lee and some others started.  It would sure be nice to see some progress on this.  I'm not sure.  There is a progress.  There are a lot of comments.  There are, well, there are var handles.  So var handles are part of that.  But var handles are a feature in the JDK.  The memory model is this complex document with lots of symbols in it and a lot of math.  And we really need the, we need the people who love doing that stuff to update it.  I enjoy reading it from time to time.  I don't have the bandwidth to work on that kind of thing myself anymore, sadly.  Anyway, memory model.  Yeah, it'd be nice.  Panama is about interconnecting Java and native code in a better way than JNI because we all  know JNI kind of sucks.  Intentionally so, by the way, for those of you who don't know.  Shenandoah, new GC, making progress, getting there.  Maybe that'll show up.  And Valhalla is for value types and generic specialization, which is a huge change, not  just to the language, but also some fairly deep changes to the Java virtual machine.  Okay, I work for Oracle, which means you shouldn't believe the word I've said.  And we have a few minutes left for questions before the timer goes off and the recording  stops.  Yeah.  Mario, we have a mic for this purpose, right?  We have a mic with a switch.  Never mind.  Oh, no, it works.  Okay.  Yes.  So, how do you test from these for those statement forests?  Are they derived from Oracle or just written by a community?  And how frequent they are enhancing?  So the tests that are run on the internal build-in test system, some of them are the  tests that you find in open JDK, the usual regression in unit tests.  Others are more akin to performance and robustness tests that are internal to Oracle in the long  term.  We'd like to publish them openly, but right now, we don't.  Thank you.  Oh.  I assumed this role.  I think it's your exercise that way.  First of all, I have to congratulate you that, page with all the JEPs that go into nine is  so cool.  There's so much stuff in there.  Congratulations for getting it all in.  Congratulations to the people behind those JEPs.  I merely collate them.  But yeah, it's a lot of work.  Nine is a pretty big release.  My second thing is I have a lot of passion for languages on top of the JVM, and I'm wondering  if you could say a word or two about the state of that.  No.  Languages on top of the JVM are a wonderful thing.  You know, I'm an old Lisp guy.  I love Clojure.  I play with it.  I write little things in it.  It's very cool.  If there's more stuff that we can do to make them run better, we should think about doing  it.  I don't really have much more to say.  I don't track all those different language communities closely, but you know, JRuby is  clearly really healthy.  Charlie might be here somewhere.  If not, he'll be here later.  Charlie and Tom are here somewhere.  Now they have their own dev room, I think.  Oh, they have their own.  Oh, well.  I mean, Ruby dev room, I think, right?  It could be.  There's JRuby dev room.  But yeah, I mean, there's JRuby, there's Scala, there's Clojure, you know, and a whole collection  of somewhat smaller projects there.  I mean, there's this guy doing small talk thing now.  No, it's great.  It's cool.  It's not just about Java.  This is the question.  All right.  So as you mentioned, that you plan to introduce some improvements on the build process of  the OpenJDK.  In particular, do you plan to make the build easier for Windows?  Because it's quite difficult to build OpenJDK under Windows.  It's a lot easier than it used to be.  So I don't know if we're going to reply to that.  Okay, I don't know of anybody who's planning to do more work on that if you wanted to show  up and help do some.  A proposal would be welcome.  Obviously, Windows is kind of not really the primary development platform and is not the  most pleasant platform to develop on anyway.  But there it is.  So no, I mean, five minutes left.  Where's the board?  No, no, because the recording won't get it.  And if I repeat it, I will not get it right.  A bit of a question about Jigsaw.  And he's always GI community happy now with what we have in Jigsaw.  That's off topic.  That's the next talk.  But if there are no other questions, I'll give you a quick answer, which is I don't know.  I mean, there has always been a prominent OSGI person on the JSR376 expert group.  For a long time it was Peter Green's.  And now it's been Neil Bartlett.  They appear content.  I think at this point people understand that Jigsaw isn't OSGI.  That we're not using OSGI to modularize the platform because we can't.  There are things we need to be able to do that OSGI can't do.  OSGI is still a useful technology if you need the dynamic life cycle stuff and all of that.  Keep using it.  It'll keep working.  Mark, I have one question.  We have reduced the footprint, for example, by using Jigsaw.  The footprint of the GVM is smaller now.  The question is, have anyone tracked what's the footprint of a fully working system running  Java?  Like if you take a kernel, use a space program, what's the minimal footprint required to run  OpenUtk?  That's a good question.  I don't know the answer offhand, but we should measure it.  Okay.  And Sir Xease was asking a question that I don't know if you're going to address this  in the next talk, but as a performance analyst, I'm wondering if you have any numbers or charts  for us.  No.  Can you talk to the state of OpenJDK performance a little bit, 8 and 9?  Are you still making pretty dashboards like I did for you before?  Oh, yeah.  There's still dashboards.  Performance is hard.  I mean, Shep 11 knows this, right?  Performance is really hard.  It's not worse, I would say that.  It's not worse.  We've had some entertaining adventures with Jigsaw because now at startup time, well,  you've got this module system that's going and looking for modules and reading stuff  in and doing resolution and building a graph and setting all the stuff up in the VM.  Oh, my goodness, startup is going to really be horrible.  For a while, it kind of was, but we pulled some tricks and invented some new techniques  that got almost all of it back.  Startup of the trivial hello world thing is pretty close to what it was in 8.  In terms of mainline performance, when everything's warmed up and all that, we've continued to  invest in C2 and other aspects of the runtime system.  Since there are many JEPs, if you want to know about performance, it's probably prudent  to just go to these JEPs and see their progress because they usually have the performance  comparisons on their target workloads.  Like compact strings, for instance, improve some very string-intensive workloads so you  can see what's there.  But really, the easiest way to do is just download the JDK9 Early Access build and try  it yourself on your application because...  Or build your own if you prefer.  On Windows.  On Windows, if you must.  That's entirely up to you.  All right, so time is just about up.  Can I ask a quick one?  You didn't mention perhaps one of the most dramatic things that's gone in really quite  late in the process, which is AOT compilation, which is a pretty big deal, I think.  AOT is a pretty big deal.  It's still experimental, but when it works, it kicks ass.  All right, thank you very much.
The_State_of_OpenJDK_0_47.png The_State_of_OpenJDK_48_107.png The_State_of_OpenJDK_108_1079.png The_State_of_OpenJDK_1080_1775.png The_State_of_OpenJDK_1776_2867.png The_State_of_OpenJDK_2868_3227.png The_State_of_OpenJDK_3228_3563.png The_State_of_OpenJDK_3564_4475.png The_State_of_OpenJDK_4476_5087.png The_State_of_OpenJDK_5088_5495.png The_State_of_OpenJDK_5496_6035.png The_State_of_OpenJDK_6036_6395.png The_State_of_OpenJDK_6396_7475.png The_State_of_OpenJDK_7476_8555.png The_State_of_OpenJDK_8556_11699.png The_State_of_OpenJDK_11700_12155.png The_State_of_OpenJDK_12156_12227.png The_State_of_OpenJDK_12228_12275.png The_State_of_OpenJDK_12276_13487.png The_State_of_OpenJDK_13488_14243.png The_State_of_OpenJDK_14244_14603.png The_State_of_OpenJDK_14604_18275.png The_State_of_OpenJDK_18276_19535.png The_State_of_OpenJDK_19536_19931.png The_State_of_OpenJDK_19932_20267.png The_State_of_OpenJDK_20268_20423.png The_State_of_OpenJDK_20424_20627.png The_State_of_OpenJDK_20628_20675.png The_State_of_OpenJDK_20676_20843.png The_State_of_OpenJDK_20844_25487.png The_State_of_OpenJDK_25488_25607.png The_State_of_OpenJDK_25608_26075.png The_State_of_OpenJDK_26076_27911.png The_State_of_OpenJDK_27912_37487.png The_State_of_OpenJDK_37488_38447.png The_State_of_OpenJDK_38448_38495.png The_State_of_OpenJDK_38496_38543.png The_State_of_OpenJDK_38544_38568.png
Good morning and welcome to FOSDEM. My name is Mark Reinhold. I work on Java still. So I was putting these quick slides together for this presentation this morning, the State of OpenJDK, which I've been doing now for a while. And I ran across a logo that you've probably all seen. As you know, a couple of years ago, we celebrated Java being 20 years old. It's now old enough to drink in many jurisdictions. And it occurred to me, you know, this is 2017, and this is the 11th time I've been at FOSDEM, the 11th time many people from Sun and then Oracle have been at FOSDEM. So if I don't have a fence post error, this means that OpenJDK is 10 years old. And all the source code was released in May of 2007. So you know, just looking back a bit, you know, at least speaking personally, thank you everybody for your support. You know, I fondly remember the warm welcome that us folks from Sun had 11 years ago when we first came to FOSDEM, and it was us in the free Java crowd. And I think it's actually fairly amazing what we've all built together. So thank you very much. Okay, so as usual, we've had some growth. We have some improvements. I'd like to talk about a bit and then speak a little bit about the present and the future. In OpenJDK, we have a structure of groups and projects. There were no new groups this year or this past year. We have had a number of new projects. S390 export, those of you who really want to run Java on mainframes, you can do it. JDK 10, you know, 9 is starting to wind down. We need a place to start doing work for 10. So JDK 10 is spun up. We've got forests going and some of the initial sort of paperwork, busywork change sets to change version numbers and source and release levels and all that stuff is floating in there. And then there's a call for votes active right now on project amber, which I'll talk about a little bit more later on. Anyway, so we've got some new projects. We've got a bunch of new committers in this past year. The ones in orange do not work for Oracle. So that's 46 new committers, seven not from Oracle. The non-Oracle ones are range from Intel, Red Hat SAP, and one person from Lennaro. So that's good. We have some new reviewers. Reviewers have a little more stature than committers. They have the authority to review change sets going into the release projects. So these are people with a lot of experience who've earned a lot of trust. The ones in orange don't work for Oracle. The ones whose last names are in orange used to work for Oracle recently. Mr. Shepalev. So they're still a little tainted. Their first names are tainted and their last names are better. Yeah, Alexi, in a tragic career move, decided to leave the company and go to Red Hat. Michael Haupt is off doing other stuff. Anyway, so we have new reviewers. That's all great. Improvements. The vulnerability group. Lawyers. So, I spoke about this, was it three years ago now here, or was it only two? Two years ago, actually did a 30-minute presentation here with the basics of the proposal. The idea to have a group in Open JDK that, unlike any other group, has special dispensation from the governing board to talk about things in secret, because when you're talking about security issues, you need the ability to do that. And I think everybody understands that. To make all of that work, there needs to be a legal document that the lawyers of not just Oracle, spread the blame around, all the big companies, lawyers from all these companies need to read it and vet it and agree that it's reasonable. I'm told by George Saab that we have a near-final document that there's just one detail left to iron out that may or may not require additional changes to the document. With any luck, we'll be able to get this done. What, next few months? George says, thumbs up. Next few months. So definitely, definitely maybe. Oh, well. Okay. Room for improvement. So there have been a couple of fairly big sore points in terms of process and infrastructure. There are still things that were done inside Sun and then continue to be done inside Oracle. And this is, you know, it's not right for what is supposed to be an open development community, so we've been working to fix those. One of those issues is the mysterious CCC, this mysterious review body inside Oracle that sometimes gives people a hard time because they're making changes that are blatantly incompatible or have other issues. The other is the whole hotspot build and test problem where, you know, by now we have a number of experienced outside committers into the hotspot code base, but we still maintain the convention, the discipline which is valuable to keep high quality. Any change to hotspot code has to go through an Oracle internal build and test system. And well, obviously, this is not a good thing long term. So I'm happy to be able to report some progress on these. There's a proposal for a new group before the governing board right now for a new group called the compatibility and specification review group. So this is to replace the mysterious CCC. So a draft of its charter is here. You know, just to be clear, this is not a group of people that's expected to create things or innovate things. You can think of it much more as it's like the editorial board of, or the editorial staff of a newspaper. Their job is to look out for consistency, to look out for, make sure things are motivated to make sure that when an incompatible change is being made that it's justified and that it's risk is understood. And sometimes we do choose to make incompatible changes. So we need to make sure everybody knows about those. And finally, consistency, consistency with the interfaces. When an interface is being changed, it's important to be as consistent as reasonable with surrounding interfaces with the overall conventions of the platform. So the initial membership of the group is here. Doug Lee still doesn't work for Oracle. And he has been one of the principal people reminding us of the problem of the CCC. So as punishment for that, he gets to be on it. So we have prototype support for this process in the OpenJDK bug system. It's reasonably lightweight for a simple change. You create a CSR issue, you fill out some information. What's the problem you're solving? What's the rough shape of the solution? What are the details of the specification that you're changing? And we expect the process itself will be ironed out by the initial group members once the group is up and running, which should be soon. But I expect it will be a fairly lightweight, lazy consensus type of thing, where a proposal will go in, it'll be on a week timer, say, and if nobody objects within a week, then it's just approved unless somebody decides, oh, this is big and I need more time to think about it and they can put it in a pending state and go off and think about it and come back. But then, of course, they're obligated to let the submitter know what the story is. So that's the proposal there. I'm pretty hopeful that we can get this done in the next few months. All right. The second improvement to address the hotspot build and test problem is an experimental mechanism that we're calling Submission Forests. So I emphasize that this is experimental. At least two people in this room, Volker and Andrew, have tried this out, so they've seen the details. So this is a, well, it's kind of a cheap hack. And thinking about this problem more than a year ago, this whole problem, well, you've got an internal build and test system. And the whole thing is a huge effort that, well, yeah, eventually we want to do. But is there some cheaper thing we can do first with the infrastructure that we have without building any, you know, some new conduit for code to come in? Because if we have some new conduit, then, you know, Oracle, internals, network security people get all worried and you have to go through six months of approvals and just a nightmare. Well, we already have a conduit for getting code in, right? It's called Mercurial. We can make as many Mercurial Forests as we want. That's easy. That's an approved input channel. So this notion of submission forest is, it's a forest. On hg.openjdk.java.net, you can create a branch there, put your patch in, push that in, and the internal build system will pick up that patch, do a build, run all the tests, and let you know what the status is. And obviously, if it doesn't work, you've got more work to do. And if it involves some Oracle internal problem, then an Oracle person is, you know, Oracle is obligated to find a person to help you. But if it works, which is the common case, then it will automatically be merged into the hotspot master. So it's pretty simple. Here's an example of how it would work. You'd clone the submission forest, set the def path for it, make a new branch. In this case, we're just making some fake branch with a silly issue number. Here we're going to put the data on the readme. Wow. Commit that, go get your code review, and push that. You push that, you know, within some reasonable amount of time, it usually takes around an hour to do all the internal build and test stuff. You'll get an email from an internal bot saying, yes, it worked. And right after that, you'll get an email notifying you that the change set went in under your name. Or you'll get an email saying, no, it didn't work. Here are some people you could maybe contact to help you resolve the issue. So it's not an ideal solution, but it's some progress. There are more details on it at this Wiki page, on wiki.open.jdk, if you want to have a look at what's going on. So this is still kind of experimental. We're working out some kinks. Andrew and Volker have been very, very helpful in some initial trials. But we'll open this up more generally to people who commit regularly into hotspot fairly soon. Do we still need web reps, if we're going to do this? Do you still need web reps? Well, you still need code review. How you do it? It exists on the branch, on the source, and the server, right? I think the medium of your code review is between you and your reviewer. So if you just want to email a patch around and they're happy with that, then that's fine. Okay, so that's another improvement. So growth improvement present. So there's this big thing called nine. It's, we're trying to get it done. There's a fair amount of stuff in it. There's, believe it or not, a fair amount of stuff in it that's not jigsaw, despite what you may have heard. So here are all the projects of various sizes feeding into it. I'm not going to go through these in detail. There are a whole bunch of JEPs, 89 of them. I'm certainly not going to go through these in detail. But if you have any questions, if you can see them, yeah, they're all here. The JEP process has worked out reasonably well in terms of letting us track things that are finer granularity than project, but a bigger granularity than issues in the bug system. In terms of schedule, we are right about there. So we're in ramp down now. We're trying to get a handle on the overall bug count, trying to figure out, are there important bugs that, well, we really don't think we can fix in time? And this we as everybody, not just people work for Oracle. We need to be deferred to the next release. Are there low priority bugs that we can just shuffle off? All that sort of thing. When we get to ramp down phase two, the knobs will tighten a little more, and we'll be entering a mode where eventually only critical bugs can be fixed. So we're still aiming to have a final release candidate in early July and a GA release in late July. And to the best of my knowledge, those dates are still realistic. But it's software. Anything could happen. Beyond nine, number of active projects, well, or not so active projects that still might be nice to see in some release after nine, not saying any of these are bound for 10 or 11, 12, or anything in particular. Or is this new project that Brian gets just recently proposed? The goal of AMBER is to explore some smaller productivity-oriented Java language features. So you could think of it in complexity scope as somewhat more than coin was. Coin was really just kind of sanding off some small rough edges. Lambda was big. It was big stuff. Panama and Valhalla are also reasonably big. AMBER is somewhere between that. Some of the ideas that Brian wants to explore there include local variable type inference and enhanced enumerations and some of the leftovers from the Lambda design, some things that could be improved in Lambda expressions. Memory model update. This is a project Doug Lee and some others started. It would sure be nice to see some progress on this. I'm not sure. There is a progress. There are a lot of comments. There are, well, there are var handles. So var handles are part of that. But var handles are a feature in the JDK. The memory model is this complex document with lots of symbols in it and a lot of math. And we really need the, we need the people who love doing that stuff to update it. I enjoy reading it from time to time. I don't have the bandwidth to work on that kind of thing myself anymore, sadly. Anyway, memory model. Yeah, it'd be nice. Panama is about interconnecting Java and native code in a better way than JNI because we all know JNI kind of sucks. Intentionally so, by the way, for those of you who don't know. Shenandoah, new GC, making progress, getting there. Maybe that'll show up. And Valhalla is for value types and generic specialization, which is a huge change, not just to the language, but also some fairly deep changes to the Java virtual machine. Okay, I work for Oracle, which means you shouldn't believe the word I've said. And we have a few minutes left for questions before the timer goes off and the recording stops. Yeah. Mario, we have a mic for this purpose, right? We have a mic with a switch. Never mind. Oh, no, it works. Okay. Yes. So, how do you test from these for those statement forests? Are they derived from Oracle or just written by a community? And how frequent they are enhancing? So the tests that are run on the internal build-in test system, some of them are the tests that you find in open JDK, the usual regression in unit tests. Others are more akin to performance and robustness tests that are internal to Oracle in the long term. We'd like to publish them openly, but right now, we don't. Thank you. Oh. I assumed this role. I think it's your exercise that way. First of all, I have to congratulate you that, page with all the JEPs that go into nine is so cool. There's so much stuff in there. Congratulations for getting it all in. Congratulations to the people behind those JEPs. I merely collate them. But yeah, it's a lot of work. Nine is a pretty big release. My second thing is I have a lot of passion for languages on top of the JVM, and I'm wondering if you could say a word or two about the state of that. No. Languages on top of the JVM are a wonderful thing. You know, I'm an old Lisp guy. I love Clojure. I play with it. I write little things in it. It's very cool. If there's more stuff that we can do to make them run better, we should think about doing it. I don't really have much more to say. I don't track all those different language communities closely, but you know, JRuby is clearly really healthy. Charlie might be here somewhere. If not, he'll be here later. Charlie and Tom are here somewhere. Now they have their own dev room, I think. Oh, they have their own. Oh, well. I mean, Ruby dev room, I think, right? It could be. There's JRuby dev room. But yeah, I mean, there's JRuby, there's Scala, there's Clojure, you know, and a whole collection of somewhat smaller projects there. I mean, there's this guy doing small talk thing now. No, it's great. It's cool. It's not just about Java. This is the question. All right. So as you mentioned, that you plan to introduce some improvements on the build process of the OpenJDK. In particular, do you plan to make the build easier for Windows? Because it's quite difficult to build OpenJDK under Windows. It's a lot easier than it used to be. So I don't know if we're going to reply to that. Okay, I don't know of anybody who's planning to do more work on that if you wanted to show up and help do some. A proposal would be welcome. Obviously, Windows is kind of not really the primary development platform and is not the most pleasant platform to develop on anyway. But there it is. So no, I mean, five minutes left. Where's the board? No, no, because the recording won't get it. And if I repeat it, I will not get it right. A bit of a question about Jigsaw. And he's always GI community happy now with what we have in Jigsaw. That's off topic. That's the next talk. But if there are no other questions, I'll give you a quick answer, which is I don't know. I mean, there has always been a prominent OSGI person on the JSR376 expert group. For a long time it was Peter Green's. And now it's been Neil Bartlett. They appear content. I think at this point people understand that Jigsaw isn't OSGI. That we're not using OSGI to modularize the platform because we can't. There are things we need to be able to do that OSGI can't do. OSGI is still a useful technology if you need the dynamic life cycle stuff and all of that. Keep using it. It'll keep working. Mark, I have one question. We have reduced the footprint, for example, by using Jigsaw. The footprint of the GVM is smaller now. The question is, have anyone tracked what's the footprint of a fully working system running Java? Like if you take a kernel, use a space program, what's the minimal footprint required to run OpenUtk? That's a good question. I don't know the answer offhand, but we should measure it. Okay. And Sir Xease was asking a question that I don't know if you're going to address this in the next talk, but as a performance analyst, I'm wondering if you have any numbers or charts for us. No. Can you talk to the state of OpenJDK performance a little bit, 8 and 9? Are you still making pretty dashboards like I did for you before? Oh, yeah. There's still dashboards. Performance is hard. I mean, Shep 11 knows this, right? Performance is really hard. It's not worse, I would say that. It's not worse. We've had some entertaining adventures with Jigsaw because now at startup time, well, you've got this module system that's going and looking for modules and reading stuff in and doing resolution and building a graph and setting all the stuff up in the VM. Oh, my goodness, startup is going to really be horrible. For a while, it kind of was, but we pulled some tricks and invented some new techniques that got almost all of it back. Startup of the trivial hello world thing is pretty close to what it was in 8. In terms of mainline performance, when everything's warmed up and all that, we've continued to invest in C2 and other aspects of the runtime system. Since there are many JEPs, if you want to know about performance, it's probably prudent to just go to these JEPs and see their progress because they usually have the performance comparisons on their target workloads. Like compact strings, for instance, improve some very string-intensive workloads so you can see what's there. But really, the easiest way to do is just download the JDK9 Early Access build and try it yourself on your application because... Or build your own if you prefer. On Windows. On Windows, if you must. That's entirely up to you. All right, so time is just about up. Can I ask a quick one? You didn't mention perhaps one of the most dramatic things that's gone in really quite late in the process, which is AOT compilation, which is a pretty big deal, I think. AOT is a pretty big deal. It's still experimental, but when it works, it kicks ass. All right, thank you very much.
candidate in early July and a GA release in late improvements on the build process of the OpenJDK.