Introducing FreeBSD in new environment: the good, the bad, the ugly
Daroussin, Baptiste
Over the last 15 years, the author have introduced successfully FreeBSD in various professional environments. This talk will focus on the reactions discovering FreeBSD, it will cover the good surprises from end users, the bad ones and the really ugly ones. During the last 15 years, the author has introduced FreeBSD in various companies, for various usage, from remote desktops to database servers, from routers to storage servers. The talk will cover the myth about FreeBSD people often have and how there usually surprised in a positive way by FreeBSD when actually really using it. What are the main blockers? How do people react when having to change some of there linux centric habits? In some cases FreeBSD has really simplified the workload and maintenance, in other cases it has been a real pain to do things that were actually really simple on linux.
 Hello, everyone. Thank you for attending my talk. So as you may expect, it's not really a technical talk. It's a talk I wanted to give for a while based on my experience and because I wanted to share how through this, the other in perceiving the multiple companies I've been working in and where previously I ended up being introduced and used because I keep hearing from time to time that BSD is not gaining any place in new companies or I keep hearing things from developers which are not really reflecting what I've lived during the last 15 years. So I thought it would be a good idea to share my views. So first, it is highly subjective. It's my point of view. I've been working in various companies and my net reflects your experience, my net reflects what you think. It's also what I believe has a student in France, in Europe, so it might be different here in North America. So first, who am I? I'm a free BSD user since 1998 basically. I won't be a free BSD developer right now and I won't be using free BSD if free BSD was not running on my laptop back in the time and if I wasn't able to use it on my laptop. So I think it's very important that we keep running properly on desktop and laptops just because of that as a student even more than after because I mean you need to use it to work on it to be able to do it. So using free BSD as a student, I became a free BSD committer in 2010. I think my first PR was in 1999, something because I wanted to watch TV on my laptop and I got a BKTR driver which was not working properly which I closed in 2012 supersede by Evans and never been committed. So I know what it is waiting behind the PR for a while. Becoming a committer, I got quite quickly involved into the governance of the project. First being I've been introduced as a support manager in 2011 and since 2014 I'm in the core team. I'm not running again for this time, so I will be free for that hopefully in a couple of weeks. Another important thing is most of my contribution, I've been doing that all on my free time and nothing being paid for. Unfortunately, I'm no more a student since 2003, so I had to work, find a work. I've been working mostly in Europe, in France and also quite a bit in Africa, in French Africa where I also have put some free BSD in production and that's also part of the experience. I've been doing quite different jobs and only one ended up being free BSD related for real which was the previous job, not the job I'm working on right now. So who are the targets, meaning the people who are the kind of usage of free BSD I've met during those 15 years? So we have, I've been working with old Cousins and Min, meaning that they set up free BSD by just installing it on the server manually and then running it daily manually. I've been working on helping creating some appliances based on free BSD. I've been also working in with more modern usage in the cloud, let's say, where you deploy everything with a lot of the mission and you don't want to end up manually connecting to the machine each day to see what happened. And last is my last experience with being able to deploy some free BSD, totally diskless, where you still have a lot of automation to specialize it but you have one single image in the beginning. So I think I pretty much cover a large scope of usage people can do with free BSD. The audience I have is mostly CIS admins and for the CIS admins it was, I would say there was two era. The first era was from the vendor time. Unix people were mostly, at least in France, in the company I was working for, people that were either from IBM, HP, so they were, when you come with the Unix, is it IX because your vendor is ABM or, but they didn't know much about other available operating system and didn't trust that a community-driven operating system could be worse using in large scale. And right now I'm more facing the youngsters coming all from the Linux world and what is free BSD, what kind of Linux destroy that and think that, well, in both case free BSD was pretty much unknown from the people I have in the companies besides the name. And the other people I met were developers, so two kind of developers, I would say the regular developers they just have an application, they don't care what it runs on and then it has some, you have to teach them that free BSD runs like this or like that if they want that to be performance or people that are, that will be hacking on the base system to improve it or to modify things. And last is PCI DSS author. I don't know if you have tried to auditors. I don't know if you have, everyone knows what PCI DSS is. Probably not. That's compliance to do if you want to do some bank, to play with the banks. And so you need to comply to a bunch of rules that were designed by Visa mostly. The thing is usually PCI DSS auditors are applying, I would say, some rules that they write on a book. And so it's quite fits if you have a red hat or if you have even an Ubuntu it could work. If you have free BSD they are totally lost and they don't know where to look at and that's quite complicated to explain them how to deal with that. The other thing is when people got introduced to free BSD they got a bunch of myths if they knew it. So the first one is you come and people say, hey, BSD is for hardcore people who are just for random regular C-set means so I'm not sure we want to use that. It will be complicated for us. Leading to free BSD, well, BSD in general is hard to use. BSD is only a source operating system and I'm running in production. I need to run binary packages because I need to deploy quite quickly and quite safely. And last they don't understand that there is not one BSD and BSD means nothing. There are multiple different operating systems and they mix up what they write on open BSD, on net BSD and free BSD and they end up with something which is not working on any of the three. So last thing is when they are, that goes for every single open source project. They think because the BSD have an old, it's quite old project with people hacking on it for a while, if something is not available as a feature or as a tool it probably means there is a history behind that and it's not a good idea. While the real reason each time is no one did it and that's all. And so go ahead and do it. So that's a bunch of myths I have to face. My methodology to each time I introduce free BSD was pretty clear. Never impose free BSD, never come and say this is free BSD and you need to install it. Do not try to convert people. If they are good with Linux, keep them with Linux because if you try to convert them you will get all the blame each time it breaks and they won't really use it. Be accessible. So first is it's not you who are driving them to go to free BSD. They use and they choose free BSD because they figure out that it might be the right tool for the right solution, for the right problem. But if you leave them alone they mostly have problems so you might be, you might be still accessible so that they come to you asking questions. I never wanted to introduce free BSD in those companies. They always ended up using it by themselves. I talk to them but I don't want to be the free BSD guy having everything on the shoulder if something is broken. And be reactive is when they are having a problem if you let them in the problem for a couple of weeks or months then they will probably switch back to Linux quite quickly or to whatever operating system they will use usually. They were, besides the fact that they didn't necessarily knew about free BSD by itself. They were already facing free BSD at least. In France you can always find free BSD somewhere in a company in different forms. So through NetAskQ, I don't know if you heard of it but NetAskQ is a firewall company, a French one in the beginning. It's now named StormShield. It's quite big. It's belong to Airbus and it's deployed everywhere in Europe. So you always find some free BSD in big and small companies in Europe. PFCense is, we can find it quite a lot in a lot of companies. So that's also a good start to find some free BSD so you can tell them if they don't trust it. You are using PFCense. You're very happy with it, meaning that your operating system is reliable for you. Maybe you should give it a try on other areas. Frenaz obviously. So this one, I found it in a couple of telecom company, Nixon. I think we have some committers working over there. It was quite interesting when I figured out it was a free BSD. And obviously, Juniper and Isilon, you can find them quite a lot. So the thing is, okay, they are now trying free BSD, trying to use it, but they want to feel at home. So the first thing they do is usually they mingle the system. So the first thing, if you just let them alone installing as a system in a free BSD and try to run in, you will quickly figure out that they have installed a log rotate. They have installed a RCSlog or whatever is the name of the application they like on Linux without even thinking that maybe we have our own thing. So you need to talk to them before they totally mingle the system and you have to insert from scratch. So usually you teach them and then they, oh, new C-Slog, what that thing? And oh, it's quite nice. Or they discover that the C-Slog we have is able to do quite a bunch of things where you don't necessarily need RCSlog. I mean, if there is features in RCSlog you need them. Of course, you switch, but most of the time you don't need to. So going back to, going to the ugly, the good, the bad and the ugly. So in 15 years, the package management has changed quite a lot on free BSD. So at the old time, the only good thing I heard, I needed to try to make a good thing about it. The only good thing I heard about it is, oh, that's pretty cool because it's highly customizable. So because of the Pulse 3, mostly more than the tools. So if I have my own patches, if I want to customize the way something is installed, it's very easy. But it's almost impossible to reproduce a setup. So you have a box which is running properly and then you want to take all the binary packages and reinstall that somewhere else. And then now imagine deploying 200 different boxes with exactly the same setup. It's almost impossible. The other solution is to, was to rebuild the ports on every single of those machines, which is not very convenient when you have a lot of boxes. And it's so very hard also to maintain a repo. Basically, to have your own repo, it was build everything on your system, package create dash A, and then try to put that somewhere else. It was very not convenient. It was close to impossible to automate. I don't know if some have tried back in the time to play with, well, Popet was not there, but CFNG was there, all the tools there. I should look at the recipes for those tools back in the time. Not very nice. Most of them were backed around port upgrade or port master, meaning that when you have a run, it takes hours if one of those things to build is very large. Not very convenient. Binary maintenance is close to impossible. It was impossible to upgrade, I don't know, Aperture has a vulnerability, PHP as well, and I want to just upgrade those two packages on my running servers to get the fixes. There was no mechanism for that. So nowadays, we have a new package manager. It's not perfect, but at least it allows you to easily create and maintain repositories. From what I hear from the people using it, it's very usable. Of course, it has some bugs, but the common line interface is quite easy when they come from the Linux world, in particular when they come from Debian. They find it quite easy because you don't have three different commands to do the same thing. APT catch for search, APT get for install and whatever. So now they merge that into one single APT, so they're pretty much in shape. If you come from Red Hat, the YUM UI was already pretty close to what Pekage is doing, but they always, you don't have to teach them. It's very, it's defined by themselves, the right commands easily, so the UI is okay. And the other thing is we have up-to-date packages available all the time. That's something people like. So before that, the binary repositories were updated whenever the cluster was not breaking, and so we got a lot of port manager just looking at logs, preying so that the whole build goes and we could deploy it. Now a days, everything is automated. We have updates almost every day for the quarterly branches, every two or three days for the latest branch. We have a quarterly branch, which we didn't have back in the time. The bad thing is they still want because they have views to, they are used to the Linux world. They want to have some kind of long-term support repository where you just have a long-term being more than a year where you have the same version of the packages on the security fixes, small bug fixes so that they can use all the automation tools that, where they decide to do some pinning on some version because whatever. And multi-repository support is still very crappy in package. It kind of works, but still a crap. So nowadays, they have upspeople the law of automation, and so they don't want to manually edit one configuration file. They want to have a tool which will, if you run it twice, will do twice the same thing on the machine and make sure that the machine is in a given state with the given configuration wherever you want. When I've been working on a puppet, there was a talk just before about puppets, or when I've been working with Ansible and other things like that, people were very happy with what usually FreeBSD provides. The RC.com.d is very useful for being able to deploy a configuration just by adding a new file somewhere and you don't have to edit automatically. So basically in automation, one thing you hate is having a giant configuration file where you have to edit the line which is between this one and this one and make sure that you haven't broken anything because try to do that programmatically, it's kind of painful. While if you have a directory where you have a file which just have the lines you need, the lines you want, it's very easy. So you don't want to edit RC.com in that case, you really want to use RC.com.d and that works pretty well. Basically the whole base system is now quite automation-friendly with.d files everywhere. There was two exception which I fixed recently based on this experiment. It was CRUNTAPS and C-SLOG. So at Gandhi, we were working at the time, they were using a lot, a lot of CRUNTAPS and those CRUNTAPS were deployed through Puppet and you don't want to edit the CRUNTAPS files. You don't want to do things like that. But what you want is being able to say, for this service, if you have a CRUNTAP, I would have files slash.etc slash CRUNTAPD slash the name of my service and if I want 10 CRUNTAPS, I will just add 10 files. So I modified the CRUNTAPS to that, CRUNTAPS to that, it's able to read that and since CRUNTAPS is now friendly to automation. C-SLOG was also painful. You really don't want to, same way, C-SLOG.conf and append things all the time and then there is an update and you need to merge things and you might have problems. So we modified C-SLOG to learn the C-SLOG.conf format so that there is a new keyword which is include here or include, I don't remember. And by default we include user local, C-SLOG.d and C-SLOG.d so that if you're, if a package want to install a specific log configuration, you just put the file there. If C-SLOG wants to do that, he modify the patch, check whatever tool they like or are unable to put something in that place. The batting are, so gel.conf is, basically people like the jail when they used it, like gel.conf because it's easy, there's a lot of right. But the problem with gel.conf is it's one single static file so I can have, I can't have one, one jail per, one configuration file per jail. Well, actually I can but I need to specify when I start the jail through the dash C command. And if you do that when you do jail dash R, it will forget where the file was and then it will mess up the shutdown of the jail. So that's a bad one that needs to be fixed. At least adding some includes and better also keeping, keeping memory, keep in mind the configuration it used when it was started. The second one is INEDD. So a lot of people, in particular in the new generation find me crazy because I keep using it and it did quite a lot in companies. But I find it very useful and rather than each time coding something that will listen on the network, INEDD is doing that pretty well by itself. The problem is I need to append a new line to INEDD and so I need to do that with POPET or with Ansible and so it's a mess because it's an append or a modification. It's not adding in a given directory and then I also need to modify ATC services which has, which suffers the same problem. So we have a new one. It's actually a very cool tool called BlacklistD. It allows to, when connected to SSHG for example, to Blacklist some IPs when there are problems. The thing is the configuration file is done the old way so it's the same kind of syntax as INEDD which is okay except that there is absolutely no way to add new configuration file but this one. Basically everything which does not allow you to install a simple file is not automation friendly and but in previous the most of the base system is cool in that area. Yeah? What services on you go behind INEDD successfully? Behind? With INEDD? INEDD? Quite a lot of things. ZFS send receive for example was my last example but it's usually homemade things. You know you have all made tooling on main. I wrote some, we will say nowadays microservices behind INEDD. I mean the two old time tools that I use a lot are INEDD, ATD and CRON tabs. I don't know why I would each time reinvent my own mechanism for that. They are there, they work. Right, so about the tooling of the base system. So they come, keep in mind that they come either from AI, old Unix or about Linux. Usually utilities you can find on those, at least old Unix are quite rough. On previous, usually they have very good surprises. We come with a lot of utilities out of box in the base system which are not used to. So I won't list all of them but all the control, all the utility, I mean I have in a utility per LSI cards is something which is very, very convenient. I really hate using the official tools for them. They also, for statistics, we get very good utility. So if you wanted to use just what CSTAT provides or what Tops provide on Linux for example, you need to install a bunch of external utilities. So for example, top-mio, you would need to install IOTOP or you have the choice between two or three different utilities that just present the same thing. CSTAT-SAT is something that they usually surprise to find. So we have arch statistics in CSTAT in the top way. That's very useful people like that. The man pages. Yes, we have man pages for almost all our utilities. It's might sound crazy that we put that as a good thing but actually on the Linux tool you don't always have them or most of the time when you have them, they are very, let's say, small, not much information and you have to switch to the info pages if they have some or to dash, dash help. So having man pages for almost everything including drivers is very useful and they usually very have good surprises just to figure out that. The G-part common, I'm surprised but it's each time something that people comes to me since we have it saying, okay, so we can, you can do partitioning on the desk in a simple manner. You don't need to have a very complicated tool you don't understand and it's also a scriptable manner because you can script very easily what all G-part is doing and you don't need to read something like 10 pages of manual to be able to understand what G-part is doing. So that's usually a good surprise. The bad things, three firewalls, it's with another L is better. So it might sound not bad surprises but it is one because when they set up something they end up, what should they use? And so usually it's actually two firewalls because one of the three no one ever heard about it. So at least for the new people. So IP filter can be removed in my opinion but that's just me. But yeah, that's a bad thing. Soundmail, I don't say, I'm not saying it's ugly, I say it's a bad thing. Usually people don't even know that there is a soundmail and they don't care about it. It's just when they figure out that it is a full soundmail that they freak out. But the way it is set up is just delivering mail locally. It's not that big deal. VI, I put that in the bad things because they are all nowadays at least used to VIM. And so in most Linux distribution there is a minimal version of VIM which they call VI. And so they try to do a lot of fun to sing inside of VI and they end up yelling a lot in front of the desk. And the old, so obviously we have G-pop now but I had four wild problems just to tell the people that no FD is not the same as FD from Linux. And yes, we need BSD labels and yes, you need to read the documentation because you probably don't know what you're doing otherwise. And that was very complicated. Something else that is bad, so when I say bad it's not bad necessarily for me. It's bad from the point of view of the people I had in front of me. It's the common strictness. I actually like this one but on Linux they have the habit to put the argument at the end after some, well the option at the end after some arguments and this versa. On BSD usually you just put the options first and then the arguments. And the last one is it's kind of funny when you don't say the people that they are logging to a previous deboxes and then they try for every single command dash dash help all the time because that's the way they expect documentation to come out, not demand pages. I like that one and I don't want us to add more dash help because it's funny. The ugly, it's kind of interesting. The number one complaint from far I got about previously in production from far CSH. So I like to know who likes CSH in this room. Okay, so yeah, sorry, TCSH. So what I figured out is most of the people including the one coming from, well the one coming from AIX doesn't know how to use, how to use KSH already correctly so they don't have any quadratic shell so they are lost nowhere. But basically people are lost in front of CSH just because it doesn't work like every other born compatible shell, interactive shell. It's crazy to see how much anger it gives to people. My opinion is we should keep CSH because obviously some people like it here and all the time but we should probably import an interactive shell which is more let's say born compatible. There are two options in my opinion, MKSH and OKSH which could be imported. There is no need to import a big bash or import a big ZSH. So that's probably something we should do and please if we do that switch routes to default on it so that that pleases my users. And so as I was saying we have no good shell, interactive shell as should have said in the base system. Our business edge is not an interactive shell. What it is but. Deploying free BSD. So one of the good thing people discover on free BSD is free BSD updates. They are usually very happy about it because it allows you to upgrade things without. I don't remember last time I got a real break edge in production based on that and I've been using that since almost free BSD 18 production and people really likes it simple fast well fast for patches not fast for upgrade between releases but actually upgrade between releases works which is something they don't have the habit of when they come from Linux. The batting is nowadays we a lot more in cloudish environments and if you take free BSD out of box it doesn't know how to play correctly in the cloud. So for example we don't have cloud in it's equivalency so cloud in it is a bunch of crap you won't ever say to me it's good but we need something which is able to at least provide easily images in various cloud providers. And we always forgot in the free BSD project that a lot of deployment might be done in machines which doesn't have access to internet. It's not that it's quite frequent actually that there are machines which are completely disconnected from the internet and those machines need still to have access to packages to binary updates and things like that. And I don't know if you try to mirror a free BSD update we have no mechanism provided by the project. So what I end up doing is one of the mirrors has index on so you can LFTP mirror and then put that somewhere else that's not really convenient would be nice if we provide our sync mechanism or maybe we have but I'm not aware of it. The ugly that's the very most awful part of the free BSD project when you're trying to deal with modern deployment. Pixi. Pixi on free BSD so I picked in particular to do seamless installation to boot some run disks diskless machine. I've been giving talk about how I did that at Gandhi and how I modified the loader to do that and I'm not speaking I don't want any NFS route. I mean no not in production. So have a look at my talk I gave two years ago to see how I work without that but it's really painful. It prevents adding free BSD in part of an already existing IPX installation in a clean way. We didn't need to fix that. Having a local free BSD update server is also a very huge pain. I don't know if you try to do that but if I'm making my own modified version of free BSD for various reasons in production I want to be able to upgrade it to add some patches and so I want to have a free BSD update server because I built my own I can't use the free BSD update one and it's painful. So we're speaking of the cloud. The good thing, 10.0 support it's quite stable it does the job. I got some feedbacks about users that use it. It's pretty good. The bad thing, the behind front end, the command line usually people run away just when they figure out that the loader is there and then they will come and there is something in the middle and where is my VM they don't know. I mean it makes sense when you understand when you talk to the people who made a VIVE when you see that and VM run they say she's not a solution for that. So if we could end up with front end, a UI which is better it would be nice. So lack of some features, there is two features that VIVE likes a lot and which is preventing the usage in production. So if you're running a cloud provider you basically don't want at any time to reboot the client machine so you need live migration because there would be security issues that would be update on the hypervisor and the hypervisor you want to make and you don't want to reboot the client space and that. So without live migration it's almost impossible to imagine having so I don't know joined how they are handling that case but the other one is okay having VIVE is pretty cool on my laptop when I use FreeBSD because I can do a lot of things. Usually what I want to do is be able to use things that are not running on FreeBSD properly. Let's say some proprietary program like Cisco, WebEx teams for example, things like that and for that what I want is to have some pass through because it wants access to my webcam for example so USB pass through and things like that would be useful. Like of Dynamics CPU is badly written it's I've actually talked to three smallish cloud providers which adds so Gandhi was one of them, the two others have not been working with them but have been in contact with them who has a feature where they dynamically add CPU to the guest VMs and the complainant FreeBSD was not supported that. Well actually I don't think that's a big problem because nowadays people reboot when they add CPUs but. The use case is dynamic scaling so my application is taking all my CPU I had a new one. Don't reboot. Storage, I won't spend much time on the storage area because I think everyone knows the good thing and the bad things there but basically people likes that FS the very good surprise I got myself when working at Gandhi was CTL. It works very well it's compared to the operating system we're using before for that we were very happy to switch to it and all the utility we have in the base system I was already saying all the LSI cards have a utility. I wrote myself the new Sessute so that we got the generic thing to blink the disk in a J-Buds. The GM tools that you have being able to have things like a GNOP to do a lot of testing is wonderful people so we're quite surprised. The ugly thing is so at first it's a good thing and when you start using it in production for real with a lot of people it becomes an ugly thing. It's NFS performances so at Gandhi we use quite a lot NFS. We were very happy when we switched from an Accenture to FreeBSD we got good performances so we added a lot of clients on it and then boom. So it was very, it's still very painful in terms of performance, very bad performances, very bad default tweaks, tunings and NFSv4 is even, I mean have you tried really in production NFSv4 against NFSv4 clients on Linux? Try to mess both of them speak the same language and with a lot of IUs you will end up with lots of problems. So it's mostly under heavy load so you need to have a heavy load usage to figure that but this was very ugly for us. So hacking, I don't know why, I think I lost the habit of the Linux machine in the time but people were very surprised to be able to have three, four different tool chains and being on the same system and being able to build with everything. Formado is okay, no reason to not have those tool chains so that's a good point for FreeBSD. Dtrace is a good one. PMCstat when people discuss the tool is a good one, they were happy with it. The bad thing is the PMCstat UI. I mean you use perfectly valid options, you flip them in the command line and you keep them in a valid order but you flip them and then it does things completely differently. It's really an unfriendly UI for that kind of tool. The lack of sanitizers in the LLVM land or GCC land, most of the sanitizers that has been written recently are not working. I think only Libasan is working correctly. But so the force, so that most people don't see it, they don't care about it but as soon as you need to do some modification in the loader, I had to do some in Gandhi and I don't want to spend two, three, four weeks or having one of my engineers spending two, three, four weeks learning force just to do some small modifications. It's a pain. And I'm really speaking of small modification, add a new entry in the menu, add something like that. So yes, I'm expecting a lot from the LLVM loader. And GDB66, that, I mean the people that do not understand why we have such old GDB in the base system, of course I tell them you can use GDB from the ports but then they have to call it GDB8, blah, blah, blah, something and not just GDB something. And so it's mostly, it's not bad not in a really because it's mostly a surprise and they always, they switch quite quickly to the recent GDB or they try to learn LLDB but GDB66 is something, I mean, it's archeology. Community feedbacks. The good thing is when I try to push people to the community, usually I try not to go myself instead of them. I try to tell them I'll go there, speak to this one and this one so that they learn. The good thing I have is it's not worse than anywhere else. So sometimes there is, it's a black hole, sometimes you have friendly people, sometimes you have yelling people, it depends but basically people are just, it's okay. They're happy to find quite a good documentation with multiple handbooks. The bad thing is if you try to read the communities through PR, through reviews, through whatever mechanism we have, if you want to catch attention of a developer you need to mail him directly or to go through mailing list. I'm not sure it's a bad thing but it's a bad thing the fact that it's not documented as such and so when people are looking for a contribution guide they should know that yeah, mailing list is the main way to reach people. I don't know what a word contribution, I think I forgot some words. Anyway, localized documentation, please can we rip out every single localized documentation which is not up to date because I'm in France, I was also working in French Africa, so they speak French in all case, so they read the French documentation and they learned about FreeBSD4 nowadays. So that should be removed and also if we could get some kind of modern encoding rather than this thing where people are just, oh, FreeBSD is not reporting in Unicode, yes that's the question I have just because of the link on the URL. The really ugly part but that's the same for every project from what I heard is bug reporting if you don't know someone. Even if it's someone who knows someone, who knows someone, who knows someone, most of the time your bug will stay there for a while and you need to go to IRC being someone, I'm always doing the proxy so I've been doing the proxy recently on ZFS, I've been doing that for a bunch of company just to, hey, I know you know that, can you please talk to those and try to help them fixing that or they have a patch, can you review? That's pretty bad but from what I hear it's almost the same on every single open source project. So, oh, I remember contribution. So contribution is the way we document, no, that's the next slide, sorry. Contributing. So the good thing about contributing to the FreeBSD project, as I said earlier, when you discuss with the community, you're basically not better but not worse than the other project so it's fine. The bad thing is subversion. Now there is subversion is a bad thing. Not that they don't want to use it but the surprise is basically the same that when we switch to subversion and before that we said we were on CVS and people were on CVS, that's exactly the same thing today. I mean it's not a bad tool, it does the job but people have the habit of both Git and Mercury also. The first thing is, whoa, what is this outdated project? Then you choose them, you tell them and it's fine enough and they can use it. Lack of documentation. We don't have a good documentation on how to contribute. Where should I go? Who should I go to? What is the right mechanism? Should I send my patch to Buxila? Should I send my patch to Fabricator? Should I open a pull request on the GitHub we have? That's the problem. Oh, I should have said in bad GitHub. The GitHub mirror, yes? So, I have a course on the question on the lack of documentation. Is it bad that we have the version number of what is or is it bad that we don't document what they are and document what the expectations for each one should be? Both and it comes just after. So, I forgot to mention in the bad GitHub. We have a GitHub mirror but there is nothing on it saying that it's a read only mirror. So, people are trying to use it or don't know it. It has more confusion on how to contribute. I know that people are pushing saying that GitHub is the way to go. All the people I talked to that were contributing through GitHub, through whatever, they don't care. As long as they have a process, I mean they like Fabricator as much as GitHub. GitHub is easy because it could mirror but as long as they have a patch that they could send through a simple command, that's all they care about most 99% of the time. But we don't have documentation of the processing. We have one single way to contribute. Right now, some people, some developer are refusing to review outside of mailing list. Some wants to have the patch in the Xela. Some wants to have the patch in Fabricator. Not having one way to contribute is mostly what is driving people crazy. If you look at other operating systems that are having contribution, they all have one way. Have you tried to contribute to Linux, for example? You will never go through GitHub. You go through a process when you send to mailing list. And then usually the first reply you get from the mailing list is something quite, what the fuck have you done here? Is a link to a documentation which explains you how to contribute and who you should send things to and what kind of quality reports you should do. And that works. I mean, I don't think we can say that the Linux community is a dying community. And they don't use GitHub. They just have one properly documented way to contribute. It does. It's the same for most of the other communities. I mean, OpenBSD, I think there are some OpenBSD folks in the room. They, this is the same, basically. You send a mail to a mailing list. There is one single way. They don't have bugs anymore because they don't have bug tracker anymore so that you have only one way to send things. Desktop lactaves. So surprisingly, even in companies, most of the people that are daily facing freebies, the in-production, they start willing to have it on their laptop, their desktop. I don't think it's a surprise because it's easier to test locally before doing something. So the thing is, it kind of works beside what's claimed. It kind of works most of the time. It's usable. And the very good surprise they have is the sound while they're coming from far. I mean, have you ever played with Alza or even worse? So the sound is a very good surprise most of the time. The bad thing is where is drivers? We are often behind, but kind of works anyway. GPU drivers, it's way better recently, but you have to be on stable or on current. Users are using releases. So I know a lot of people are pushing to have stability to have everyone moving on stable or current. Users are using releases. That's what we do, releases. So we need to have things that works on releases. Out-summarized by, usually freebies is a surprise and a good surprise for the people when they use it. I've never faced places where they were, what is this thing? Usually they say, oh, I can maybe use FreebiesD instead of that operating system here because it seems to be better and they try to extend it. A good example was at Gandhi, I tried not to put any FreebiesD anywhere and they were adding more and more. That thing, if you don't know someone, it's very hard to get in. There is a company who shadow who is doing some, let's say, gamer boxes, cloudish gamer boxes. They were using some, they're trying to use some FreebiesD in the know one. It was very hard. I had to play the proxy for them. It was very hard for them to report and get people on their bugs and they got a lot of, a lot of the FSS issues. So we fixed most of them since I made the proxy. But the bad thing is regression unstable. So the same company, they were using, so they tried to use Curent because I told them to use head and Curent like on FreebiesD then because all the FSS patches they needed were unstable, they switched to stable because it was easier to track if you don't have a FreebiesD developer. Even at Gandhi where I was, we switched back to stable because it was easier to track in the end. And they now, I know at least three companies that decided to move out of FreebiesD because of some regression in particular, I don't, I haven't put it there. In particular, the IF Lib in stable 11, it broke aggregation, it broke performance. I don't remember the details. There was a bunch of issues. It was open for a while. On Curent, I couldn't understand that. It takes some time to fix and it's progress. I mean, I think we need it. But having that on a stable branch which becomes release was very painful. And so at least three companies moved out of FreebiesD because of that. So if we could be very careful about when we merge, what we merge, what we do there. Lack of Erata, as I was saying, most people are using releases if you have a bug in a program in a release, an important bug or an important regression in a given release, we need an Erata. I know why we don't have Erata. We need to improve in that place. But there is a lot of, a lot of things that should, at least after a release, deserve an Erata. It's very hard to say to the users, okay, you use a release, you have, you're facing this bug. It will be fixed on the next release. You probably should use a stable. In particular, two or three weeks after the release. Another bad thing is we are not cool anymore. So that relates to when I was a student, the random reason why I needed to use FreebiesD was we were cool. PSD was in every single university. There was wonderful swags. So in IT and general school, having a PSD T-shirt, I mean, open PSD folks are pretty good at doing cool T-shirts was, you know, you were the cool guy in that case. And nowadays, if you go to those universities, they don't have any PSD anymore, at least in Europe, as the main desktop. The students are not attracted at all by any way. Our swags are all very, I would say, company compliant. And that's, it plays a lot in people, in young people, at least in Europe, not playing as much with the PSD as we used to do when we were kids. And in the project, you can have a look at the middle edge. For Europe, at least, they are all about, most of the contributors are about my age, more plus or minus five years. That's used to not be the case. I'm sure that something is related also to that. I'm sorry, man. And that's all. So, don't know if you have any questions? No? Thank you.
 Hello, everyone. Thank you for attending my talk. So as you may expect, it's not really  a technical talk. It's a talk I wanted to give for a while based on my experience and  because I wanted to share how through this, the other in perceiving the multiple companies  I've been working in and where previously I ended up being introduced and used because  I keep hearing from time to time that BSD is not gaining any place in new companies  or I keep hearing things from developers which are not really reflecting what I've  lived during the last 15 years. So I thought it would be a good idea to share my views.  So first, it is highly subjective. It's my point of view. I've been working in various  companies and my net reflects your experience, my net reflects what you think. It's also  what I believe has a student in France, in Europe, so it might be different here in North  America. So first, who am I? I'm a free BSD user since 1998 basically. I won't be a free  BSD developer right now and I won't be using free BSD if free BSD was not running on my  laptop back in the time and if I wasn't able to use it on my laptop. So I think it's very  important that we keep running properly on desktop and laptops just because of that as  a student even more than after because I mean you need to use it to work on it to be able  to do it. So using free BSD as a student, I became a free BSD committer in 2010. I think  my first PR was in 1999, something because I wanted to watch TV on my laptop and I got  a BKTR driver which was not working properly which I closed in 2012 supersede by Evans  and never been committed. So I know what it is waiting behind the PR for a while. Becoming  a committer, I got quite quickly involved into the governance of the project. First  being I've been introduced as a support manager in 2011 and since 2014 I'm in the core team.  I'm not running again for this time, so I will be free for that hopefully in a couple  of weeks. Another important thing is most of my contribution, I've been doing that all  on my free time and nothing being paid for. Unfortunately, I'm no more a student since  2003, so I had to work, find a work. I've been working mostly in Europe, in France and  also quite a bit in Africa, in French Africa where I also have put some free BSD in production  and that's also part of the experience. I've been doing quite different jobs and only one  ended up being free BSD related for real which was the previous job, not the job I'm working  on right now. So who are the targets, meaning the people who are the kind of usage of free  BSD I've met during those 15 years? So we have, I've been working with old Cousins and  Min, meaning that they set up free BSD by just installing it on the server manually and  then running it daily manually. I've been working on helping creating some appliances  based on free BSD. I've been also working in with more modern usage in the cloud, let's  say, where you deploy everything with a lot of the mission and you don't want to end up  manually connecting to the machine each day to see what happened. And last is my last  experience with being able to deploy some free BSD, totally diskless, where you still  have a lot of automation to specialize it but you have one single image in the beginning.  So I think I pretty much cover a large scope of usage people can do with free BSD. The audience  I have is mostly CIS admins and for the CIS admins it was, I would say there was two era.  The first era was from the vendor time. Unix people were mostly, at least in France, in  the company I was working for, people that were either from IBM, HP, so they were, when  you come with the Unix, is it IX because your vendor is ABM or, but they didn't know much  about other available operating system and didn't trust that a community-driven operating  system could be worse using in large scale. And right now I'm more facing the youngsters  coming all from the Linux world and what is free BSD, what kind of Linux destroy that  and think that, well, in both case free BSD was pretty much unknown from the people I  have in the companies besides the name. And the other people I met were developers, so  two kind of developers, I would say the regular developers they just have an application, they  don't care what it runs on and then it has some, you have to teach them that free BSD  runs like this or like that if they want that to be performance or people that are, that  will be hacking on the base system to improve it or to modify things.  And last is PCI DSS author. I don't know if you have tried to auditors. I don't know  if you have, everyone knows what PCI DSS is. Probably not. That's compliance to do if you  want to do some bank, to play with the banks. And so you need to comply to a bunch of rules  that were designed by Visa mostly. The thing is usually PCI DSS auditors are applying,  I would say, some rules that they write on a book. And so it's quite fits if you have  a red hat or if you have even an Ubuntu it could work. If you have free BSD they are  totally lost and they don't know where to look at and that's quite complicated to explain  them how to deal with that. The other thing is when people got introduced  to free BSD they got a bunch of myths if they knew it. So the first one is you come and people  say, hey, BSD is for hardcore people who are just for random regular C-set means so I'm  not sure we want to use that. It will be complicated for us.  Leading to free BSD, well, BSD in general is hard to use. BSD is only a source operating  system and I'm running in production. I need to run binary packages because I need to deploy  quite quickly and quite safely. And last they don't understand that there is not one BSD  and BSD means nothing. There are multiple different operating systems and they mix up  what they write on open BSD, on net BSD and free BSD and they end up with something which  is not working on any of the three. So last thing is when they are, that goes for every  single open source project. They think because the BSD have an old, it's quite old project  with people hacking on it for a while, if something is not available as a feature or  as a tool it probably means there is a history behind that and it's not a good idea. While  the real reason each time is no one did it and that's all. And so go ahead and do it.  So that's a bunch of myths I have to face. My methodology to each time I introduce free  BSD was pretty clear. Never impose free BSD, never come and say this is free BSD and you  need to install it. Do not try to convert people. If they are good with Linux, keep them with  Linux because if you try to convert them you will get all the blame each time it breaks  and they won't really use it. Be accessible. So first is it's not you who are driving them  to go to free BSD. They use and they choose free BSD because they figure out that it might  be the right tool for the right solution, for the right problem. But if you leave them  alone they mostly have problems so you might be, you might be still accessible so that  they come to you asking questions. I never wanted to introduce free BSD in those companies.  They always ended up using it by themselves. I talk to them but I don't want to be the  free BSD guy having everything on the shoulder if something is broken. And be reactive is  when they are having a problem if you let them in the problem for a couple of weeks or  months then they will probably switch back to Linux quite quickly or to whatever operating  system they will use usually. They were, besides the fact that they didn't necessarily  knew about free BSD by itself. They were already facing free BSD at least. In France you can  always find free BSD somewhere in a company in different forms. So through NetAskQ, I  don't know if you heard of it but NetAskQ is a firewall company, a French one in the  beginning. It's now named StormShield. It's quite big. It's belong to Airbus and it's  deployed everywhere in Europe. So you always find some free BSD in big and small companies  in Europe. PFCense is, we can find it quite a lot in a lot of companies. So that's also  a good start to find some free BSD so you can tell them if they don't trust it. You  are using PFCense. You're very happy with it, meaning that your operating system is reliable  for you. Maybe you should give it a try on other areas. Frenaz obviously. So this one,  I found it in a couple of telecom company, Nixon. I think we have some committers working  over there. It was quite interesting when I figured out it was a free BSD. And obviously,  Juniper and Isilon, you can find them quite a lot. So the thing is, okay, they are now  trying free BSD, trying to use it, but they want to feel at home. So the first thing they  do is usually they mingle the system. So the first thing, if you just let them alone installing  as a system in a free BSD and try to run in, you will quickly figure out that they have  installed a log rotate. They have installed a RCSlog or whatever is the name of the application  they like on Linux without even thinking that maybe we have our own thing. So you need to  talk to them before they totally mingle the system and you have to insert from scratch.  So usually you teach them and then they, oh, new C-Slog, what that thing? And oh, it's  quite nice. Or they discover that the C-Slog we have is able to do quite a bunch of things  where you don't necessarily need RCSlog. I mean, if there is features in RCSlog you  need them. Of course, you switch, but most of the time you don't need to. So going back  to, going to the ugly, the good, the bad and the ugly. So in 15 years, the package management  has changed quite a lot on free BSD. So at the old time, the only good thing I heard,  I needed to try to make a good thing about it. The only good thing I heard about it is,  oh, that's pretty cool because it's highly customizable. So because of the Pulse 3, mostly  more than the tools. So if I have my own patches, if I want to customize the way something is  installed, it's very easy. But it's almost impossible to reproduce a setup. So you have  a box which is running properly and then you want to take all the binary packages and reinstall  that somewhere else. And then now imagine deploying 200 different boxes with exactly  the same setup. It's almost impossible. The other solution is to, was to rebuild the  ports on every single of those machines, which is not very convenient when you have a lot  of boxes. And it's so very hard also to maintain a repo. Basically, to have your own repo,  it was build everything on your system, package create dash A, and then try to put that somewhere  else. It was very not convenient. It was close to impossible to automate. I don't know if  some have tried back in the time to play with, well, Popet was not there, but CFNG was there,  all the tools there. I should look at the recipes for those tools back in the time. Not  very nice. Most of them were backed around port upgrade or port master, meaning that  when you have a run, it takes hours if one of those things to build is very large. Not  very convenient. Binary maintenance is close to impossible. It was impossible to upgrade,  I don't know, Aperture has a vulnerability, PHP as well, and I want to just upgrade those  two packages on my running servers to get the fixes. There was no mechanism for that.  So nowadays, we have a new package manager. It's not perfect, but at least it allows you  to easily create and maintain repositories. From what I hear from the people using it,  it's very usable. Of course, it has some bugs, but the common line interface is quite easy  when they come from the Linux world, in particular when they come from Debian. They find it  quite easy because you don't have three different commands to do the same thing. APT catch for  search, APT get for install and whatever. So now they merge that into one single APT,  so they're pretty much in shape. If you come from Red Hat, the YUM UI was already pretty  close to what Pekage is doing, but they always, you don't have to teach them. It's very, it's  defined by themselves, the right commands easily, so the UI is okay. And the other thing  is we have up-to-date packages available all the time. That's something people like. So  before that, the binary repositories were updated whenever the cluster was not breaking, and  so we got a lot of port manager just looking at logs, preying so that the whole build goes  and we could deploy it. Now a days, everything is automated. We have updates almost every  day for the quarterly branches, every two or three days for the latest branch. We have  a quarterly branch, which we didn't have back in the time. The bad thing is they still want  because they have views to, they are used to the Linux world. They want to have some  kind of long-term support repository where you just have a long-term being more than a  year where you have the same version of the packages on the security fixes, small bug  fixes so that they can use all the automation tools that, where they decide to do some pinning  on some version because whatever. And multi-repository support is still very crappy in package. It  kind of works, but still a crap. So nowadays, they have upspeople the law of automation,  and so they don't want to manually edit one configuration file. They want to have a tool  which will, if you run it twice, will do twice the same thing on the machine and make sure  that the machine is in a given state with the given configuration wherever you want.  When I've been working on a puppet, there was a talk just before about puppets, or when  I've been working with Ansible and other things like that, people were very happy with what  usually FreeBSD provides. The RC.com.d is very useful for being able to deploy a configuration  just by adding a new file somewhere and you don't have to edit automatically. So basically  in automation, one thing you hate is having a giant configuration file where you have  to edit the line which is between this one and this one and make sure that you haven't  broken anything because try to do that programmatically, it's kind of painful. While if you have a  directory where you have a file which just have the lines you need, the lines you want,  it's very easy. So you don't want to edit RC.com in that case, you really want to use RC.com.d  and that works pretty well. Basically the whole base system is now quite automation-friendly  with.d files everywhere. There was two exception which I fixed recently based on this experiment.  It was CRUNTAPS and C-SLOG. So at Gandhi, we were working at the time, they were using  a lot, a lot of CRUNTAPS and those CRUNTAPS were deployed through Puppet and you don't  want to edit the CRUNTAPS files. You don't want to do things like that. But what you  want is being able to say, for this service, if you have a CRUNTAP, I would have files slash .etc slash CRUNTAPD slash the name of my service and if I want 10 CRUNTAPS, I will just add  10 files. So I modified the CRUNTAPS to that, CRUNTAPS to that, it's able to read that and  since CRUNTAPS is now friendly to automation. C-SLOG was also painful. You really don't  want to, same way, C-SLOG.conf and append things all the time and then there is an update and  you need to merge things and you might have problems. So we modified C-SLOG to learn the  C-SLOG.conf format so that there is a new keyword which is include here or include, I  don't remember. And by default we include user local, C-SLOG.d and C-SLOG.d so that  if you're, if a package want to install a specific log configuration, you just put the  file there. If C-SLOG wants to do that, he modify the patch, check whatever tool they  like or are unable to put something in that place. The batting are, so gel.conf is, basically  people like the jail when they used it, like gel.conf because it's easy, there's a lot  of right. But the problem with gel.conf is it's one single static file so I can have,  I can't have one, one jail per, one configuration file per jail. Well, actually I can but I  need to specify when I start the jail through the dash C command. And if you do that when  you do jail dash R, it will forget where the file was and then it will mess up the shutdown  of the jail. So that's a bad one that needs to be fixed. At least adding some includes  and better also keeping, keeping memory, keep in mind the configuration it used when it  was started. The second one is INEDD. So a lot of people, in particular in the new generation  find me crazy because I keep using it and it did quite a lot in companies. But I find  it very useful and rather than each time coding something that will listen on the network,  INEDD is doing that pretty well by itself. The problem is I need to append a new line  to INEDD and so I need to do that with POPET or with Ansible and so it's a mess because  it's an append or a modification. It's not adding in a given directory and then I also  need to modify ATC services which has, which suffers the same problem. So we have a new  one. It's actually a very cool tool called BlacklistD. It allows to, when connected to  SSHG for example, to Blacklist some IPs when there are problems. The thing is the configuration  file is done the old way so it's the same kind of syntax as INEDD which is okay except  that there is absolutely no way to add new configuration file but this one. Basically  everything which does not allow you to install a simple file is not automation friendly and  but in previous the most of the base system is cool in that area. Yeah?  What services on you go behind INEDD successfully?  Behind?  With INEDD?  INEDD?  Quite a lot of things. ZFS send receive for example was my last example but it's usually  homemade things. You know you have all made tooling on main. I wrote some, we will say  nowadays microservices behind INEDD. I mean the two old time tools that I use a lot are  INEDD, ATD and CRON tabs. I don't know why I would each time reinvent my own mechanism  for that. They are there, they work. Right, so about the tooling of the base system.  So they come, keep in mind that they come either from AI, old Unix or about Linux. Usually  utilities you can find on those, at least old Unix are quite rough. On previous, usually  they have very good surprises. We come with a lot of utilities out of box in the base  system which are not used to. So I won't list all of them but all the control, all the  utility, I mean I have in a utility per LSI cards is something which is very, very convenient.  I really hate using the official tools for them. They also, for statistics, we get very  good utility. So if you wanted to use just what CSTAT provides or what Tops provide on  Linux for example, you need to install a bunch of external utilities. So for example, top-mio,  you would need to install IOTOP or you have the choice between two or three different  utilities that just present the same thing. CSTAT-SAT is something that they usually surprise  to find. So we have arch statistics in CSTAT in the top way. That's very useful people  like that. The man pages. Yes, we have man pages for almost all our utilities. It's might  sound crazy that we put that as a good thing but actually on the Linux tool you don't always  have them or most of the time when you have them, they are very, let's say, small, not  much information and you have to switch to the info pages if they have some or to dash,  dash help. So having man pages for almost everything including drivers is very useful  and they usually very have good surprises just to figure out that. The G-part common,  I'm surprised but it's each time something that people comes to me since we have it saying,  okay, so we can, you can do partitioning on the desk in a simple manner. You don't need  to have a very complicated tool you don't understand and it's also a scriptable manner  because you can script very easily what all G-part is doing and you don't need to read  something like 10 pages of manual to be able to understand what G-part is doing. So that's  usually a good surprise. The bad things, three firewalls, it's with another L is better. So  it might sound not bad surprises but it is one because when they set up something they  end up, what should they use? And so usually it's actually two firewalls because one of  the three no one ever heard about it. So at least for the new people. So IP filter can  be removed in my opinion but that's just me. But yeah, that's a bad thing. Soundmail, I  don't say, I'm not saying it's ugly, I say it's a bad thing. Usually people don't even  know that there is a soundmail and they don't care about it. It's just when they figure  out that it is a full soundmail that they freak out. But the way it is set up is just  delivering mail locally. It's not that big deal. VI, I put that in the bad things because  they are all nowadays at least used to VIM. And so in most Linux distribution there is  a minimal version of VIM which they call VI. And so they try to do a lot of fun to sing  inside of VI and they end up yelling a lot in front of the desk. And the old, so obviously  we have G-pop now but I had four wild problems just to tell the people that no FD is not  the same as FD from Linux. And yes, we need BSD labels and yes, you need to read the documentation  because you probably don't know what you're doing otherwise. And that was very complicated.  Something else that is bad, so when I say bad it's not bad necessarily for me. It's  bad from the point of view of the people I had in front of me. It's the common strictness.  I actually like this one but on Linux they have the habit to put the argument at the  end after some, well the option at the end after some arguments and this versa. On BSD  usually you just put the options first and then the arguments. And the last one is it's  kind of funny when you don't say the people that they are logging to a previous deboxes  and then they try for every single command dash dash help all the time because that's  the way they expect documentation to come out, not demand pages. I like that one and  I don't want us to add more dash help because it's funny. The ugly, it's kind of interesting.  The number one complaint from far I got about previously in production from far CSH. So  I like to know who likes CSH in this room. Okay, so yeah, sorry, TCSH. So what I figured  out is most of the people including the one coming from, well the one coming from AIX  doesn't know how to use, how to use KSH already correctly so they don't have any quadratic  shell so they are lost nowhere. But basically people are lost in front of CSH just because  it doesn't work like every other born compatible shell, interactive shell. It's crazy to see  how much anger it gives to people. My opinion is we should keep CSH because obviously some  people like it here and all the time but we should probably import an interactive shell  which is more let's say born compatible. There are two options in my opinion, MKSH and OKSH  which could be imported. There is no need to import a big bash or import a big ZSH.  So that's probably something we should do and please if we do that switch routes to  default on it so that that pleases my users. And so as I was saying we have no good shell,  interactive shell as should have said in the base system. Our business edge is not an interactive  shell. What it is but. Deploying free BSD. So one of the good thing people discover on  free BSD is free BSD updates. They are usually very happy about it because it allows you  to upgrade things without. I don't remember last time I got a real break edge in production  based on that and I've been using that since almost free BSD 18 production and people really  likes it simple fast well fast for patches not fast for upgrade between releases but actually  upgrade between releases works which is something they don't have the habit of when they come  from Linux. The batting is nowadays we a lot more in  cloudish environments and if you take free BSD out of box it doesn't know how to play  correctly in the cloud. So for example we don't have cloud in it's equivalency so cloud  in it is a bunch of crap you won't ever say to me it's good but we need something which  is able to at least provide easily images in various cloud providers. And we always forgot  in the free BSD project that a lot of deployment might be done in machines which doesn't have  access to internet. It's not that it's quite frequent actually that there are machines  which are completely disconnected from the internet and those machines need still to  have access to packages to binary updates and things like that. And I don't know if you  try to mirror a free BSD update we have no mechanism provided by the project. So what  I end up doing is one of the mirrors has index on so you can LFTP mirror and then put that  somewhere else that's not really convenient would be nice if we provide our sync mechanism  or maybe we have but I'm not aware of it. The ugly that's the very most awful part of  the free BSD project when you're trying to deal with modern deployment. Pixi. Pixi on  free BSD so I picked in particular to do seamless installation to boot some run disks diskless  machine. I've been giving talk about how I did that at Gandhi and how I modified the  loader to do that and I'm not speaking I don't want any NFS route. I mean no not in production.  So have a look at my talk I gave two years ago to see how I work without that but it's  really painful. It prevents adding free BSD in part of an already existing IPX installation  in a clean way. We didn't need to fix that. Having a local free BSD update server is also  a very huge pain. I don't know if you try to do that but if I'm making my own modified  version of free BSD for various reasons in production I want to be able to upgrade it  to add some patches and so I want to have a free BSD update server because I built my  own I can't use the free BSD update one and it's painful. So we're speaking of the cloud.  The good thing, 10.0 support it's quite stable it does the job. I got some feedbacks about  users that use it. It's pretty good. The bad thing, the behind front end, the command line  usually people run away just when they figure out that the loader is there and then they  will come and there is something in the middle and where is my VM they don't know. I mean  it makes sense when you understand when you talk to the people who made a VIVE when you  see that and VM run they say she's not a solution for that. So if we could end up with front  end, a UI which is better it would be nice. So lack of some features, there is two features  that VIVE likes a lot and which is preventing the usage in production. So if you're running  a cloud provider you basically don't want at any time to reboot the client machine so  you need live migration because there would be security issues that would be update on  the hypervisor and the hypervisor you want to make and you don't want to reboot the client  space and that. So without live migration it's almost impossible to imagine having so I don't  know joined how they are handling that case but the other one is okay having VIVE is pretty  cool on my laptop when I use FreeBSD because I can do a lot of things. Usually what I want  to do is be able to use things that are not running on FreeBSD properly. Let's say some  proprietary program like Cisco, WebEx teams for example, things like that and for that  what I want is to have some pass through because it wants access to my webcam for example so  USB pass through and things like that would be useful. Like of Dynamics CPU is badly  written it's I've actually talked to three smallish cloud providers which adds so Gandhi  was one of them, the two others have not been working with them but have been in contact  with them who has a feature where they dynamically add CPU to the guest VMs and the complainant  FreeBSD was not supported that. Well actually I don't think that's a big problem because  nowadays people reboot when they add CPUs but. The use case is dynamic scaling so my application  is taking all my CPU I had a new one. Don't reboot. Storage, I won't spend much time on  the storage area because I think everyone knows the good thing and the bad things there but  basically people likes that FS the very good surprise I got myself when working at Gandhi  was CTL. It works very well it's compared to the operating system we're using before  for that we were very happy to switch to it and all the utility we have in the base system  I was already saying all the LSI cards have a utility. I wrote myself the new Sessute  so that we got the generic thing to blink the disk in a J-Buds. The GM tools that you  have being able to have things like a GNOP to do a lot of testing is wonderful people  so we're quite surprised. The ugly thing is so at first it's a good thing and when you  start using it in production for real with a lot of people it becomes an ugly thing.  It's NFS performances so at Gandhi we use quite a lot NFS. We were very happy when we switched  from an Accenture to FreeBSD we got good performances so we added a lot of clients on it and then  boom. So it was very, it's still very painful in terms of performance, very bad performances,  very bad default tweaks, tunings and NFSv4 is even, I mean have you tried really in production  NFSv4 against NFSv4 clients on Linux? Try to mess both of them speak the same language  and with a lot of IUs you will end up with lots of problems. So it's mostly under heavy  load so you need to have a heavy load usage to figure that but this was very ugly for  us. So hacking, I don't know why, I think I lost the habit of the Linux machine in the  time but people were very surprised to be able to have three, four different tool chains  and being on the same system and being able to build with everything. Formado is okay,  no reason to not have those tool chains so that's a good point for FreeBSD.  Dtrace is a good one. PMCstat when people discuss the tool is a good one, they were  happy with it. The bad thing is the PMCstat UI. I mean you use perfectly valid options,  you flip them in the command line and you keep them in a valid order but you flip them  and then it does things completely differently. It's really an unfriendly UI for that kind  of tool. The lack of sanitizers in the LLVM land or GCC land, most of the sanitizers that  has been written recently are not working. I think only Libasan is working correctly.  But so the force, so that most people don't see it, they don't care about it but as soon  as you need to do some modification in the loader, I had to do some in Gandhi and I don't  want to spend two, three, four weeks or having one of my engineers spending two, three, four  weeks learning force just to do some small modifications. It's a pain. And I'm really  speaking of small modification, add a new entry in the menu, add something like that.  So yes, I'm expecting a lot from the LLVM loader. And GDB66, that, I mean the people  that do not understand why we have such old GDB in the base system, of course I tell them  you can use GDB from the ports but then they have to call it GDB8, blah, blah, blah, something  and not just GDB something. And so it's mostly, it's not bad not in a really because it's  mostly a surprise and they always, they switch quite quickly to the recent GDB or they try  to learn LLDB but GDB66 is something, I mean, it's archeology. Community feedbacks. The  good thing is when I try to push people to the community, usually I try not to go myself  instead of them. I try to tell them I'll go there, speak to this one and this one so that  they learn. The good thing I have is it's not worse than anywhere else. So sometimes  there is, it's a black hole, sometimes you have friendly people, sometimes you have yelling  people, it depends but basically people are just, it's okay. They're happy to find quite  a good documentation with multiple handbooks. The bad thing is if you try to read the communities  through PR, through reviews, through whatever mechanism we have, if you want to catch attention  of a developer you need to mail him directly or to go through mailing list. I'm not sure  it's a bad thing but it's a bad thing the fact that it's not documented as such and  so when people are looking for a contribution guide they should know that yeah, mailing  list is the main way to reach people. I don't know what a word contribution, I think I forgot  some words. Anyway, localized documentation, please can we rip out every single localized  documentation which is not up to date because I'm in France, I was also working in French  Africa, so they speak French in all case, so they read the French documentation and they  learned about FreeBSD4 nowadays. So that should be removed and also if we could get some kind  of modern encoding rather than this thing where people are just, oh, FreeBSD is not  reporting in Unicode, yes that's the question I have just because of the link on the URL.  The really ugly part but that's the same for every project from what I heard is bug reporting  if you don't know someone. Even if it's someone who knows someone, who knows someone, who  knows someone, most of the time your bug will stay there for a while and you need to go  to IRC being someone, I'm always doing the proxy so I've been doing the proxy recently  on ZFS, I've been doing that for a bunch of company just to, hey, I know you know that,  can you please talk to those and try to help them fixing that or they have a patch, can  you review? That's pretty bad but from what I hear it's almost the same on every single  open source project. So, oh, I remember contribution. So contribution is the way we document, no,  that's the next slide, sorry. Contributing. So the good thing about contributing to the  FreeBSD project, as I said earlier, when you discuss with the community, you're basically  not better but not worse than the other project so it's fine. The bad thing is subversion.  Now there is subversion is a bad thing. Not that they don't want to use it but the surprise  is basically the same that when we switch to subversion and before that we said we were  on CVS and people were on CVS, that's exactly the same thing today. I mean it's not a bad  tool, it does the job but people have the habit of both Git and Mercury also. The first  thing is, whoa, what is this outdated project? Then you choose them, you tell them and it's  fine enough and they can use it. Lack of documentation. We don't have a good documentation on how  to contribute. Where should I go? Who should I go to? What is the right mechanism? Should  I send my patch to Buxila? Should I send my patch to Fabricator? Should I open a pull  request on the GitHub we have? That's the problem. Oh, I should have said in bad GitHub.  The GitHub mirror, yes?  So, I have a course on the question on the lack of documentation. Is it bad that we have  the version number of what is or is it bad that we don't document what they are and  document what the expectations for each one should be?  Both and it comes just after. So, I forgot to mention in the bad GitHub. We have a GitHub  mirror but there is nothing on it saying that it's a read only mirror. So, people are trying  to use it or don't know it. It has more confusion on how to contribute. I know that people are  pushing saying that GitHub is the way to go. All the people I talked to that were contributing  through GitHub, through whatever, they don't care. As long as they have a process, I mean  they like Fabricator as much as GitHub. GitHub is easy because it could mirror but as long  as they have a patch that they could send through a simple command, that's all they care  about most 99% of the time. But we don't have documentation of the processing. We have one  single way to contribute. Right now, some people, some developer are refusing to review  outside of mailing list. Some wants to have the patch in the Xela. Some wants to have the  patch in Fabricator. Not having one way to contribute is mostly what is driving people  crazy. If you look at other operating systems that are having contribution, they all have  one way. Have you tried to contribute to Linux, for example? You will never go through GitHub.  You go through a process when you send to mailing list. And then usually the first reply you  get from the mailing list is something quite, what the fuck have you done here? Is a link  to a documentation which explains you how to contribute and who you should send things  to and what kind of quality reports you should do. And that works. I mean, I don't think we  can say that the Linux community is a dying community. And they don't use GitHub. They  just have one properly documented way to contribute. It does. It's the same for most of the other  communities. I mean, OpenBSD, I think there are some OpenBSD folks in the room. They,  this is the same, basically. You send a mail to a mailing list. There is one single way.  They don't have bugs anymore because they don't have bug tracker anymore so that you  have only one way to send things. Desktop lactaves. So surprisingly, even in companies,  most of the people that are daily facing freebies, the in-production, they start willing to have  it on their laptop, their desktop. I don't think it's a surprise because it's easier  to test locally before doing something. So the thing is, it kind of works beside what's  claimed. It kind of works most of the time. It's usable. And the very good surprise they  have is the sound while they're coming from far. I mean, have you ever played with Alza  or even worse? So the sound is a very good surprise most of the time. The bad thing is  where is drivers? We are often behind, but kind of works anyway. GPU drivers, it's way  better recently, but you have to be on stable or on current. Users are using releases. So  I know a lot of people are pushing to have stability to have everyone moving on stable  or current. Users are using releases. That's what we do, releases. So we need to have things  that works on releases. Out-summarized by, usually freebies is a surprise and a good  surprise for the people when they use it. I've never faced places where they were, what  is this thing? Usually they say, oh, I can maybe use FreebiesD instead of that operating  system here because it seems to be better and they try to extend it. A good example was  at Gandhi, I tried not to put any FreebiesD anywhere and they were adding more and more.  That thing, if you don't know someone, it's very hard to get in. There is a company who  shadow who is doing some, let's say, gamer boxes, cloudish gamer boxes. They were using  some, they're trying to use some FreebiesD in the know one. It was very hard. I had to  play the proxy for them. It was very hard for them to report and get people on their  bugs and they got a lot of, a lot of the FSS issues. So we fixed most of them since I  made the proxy. But the bad thing is regression unstable. So the same company, they were using,  so they tried to use Curent because I told them to use head and Curent like on FreebiesD  then because all the FSS patches they needed were unstable, they switched to stable because  it was easier to track if you don't have a FreebiesD developer. Even at Gandhi where  I was, we switched back to stable because it was easier to track in the end. And they  now, I know at least three companies that decided to move out of FreebiesD because of  some regression in particular, I don't, I haven't put it there. In particular, the IF  Lib in stable 11, it broke aggregation, it broke performance. I don't remember the details.  There was a bunch of issues. It was open for a while. On Curent, I couldn't understand  that. It takes some time to fix and it's progress. I mean, I think we need it. But having that  on a stable branch which becomes release was very painful. And so at least three companies  moved out of FreebiesD because of that. So if we could be very careful about when we merge,  what we merge, what we do there. Lack of Erata, as I was saying, most people are using releases  if you have a bug in a program in a release, an important bug or an important regression  in a given release, we need an Erata. I know why we don't have Erata. We need to improve  in that place. But there is a lot of, a lot of things that should, at least after a release,  deserve an Erata. It's very hard to say to the users, okay, you use a release, you have,  you're facing this bug. It will be fixed on the next release. You probably should use  a stable. In particular, two or three weeks after the release. Another bad thing is we  are not cool anymore. So that relates to when I was a student, the random reason why I needed  to use FreebiesD was we were cool. PSD was in every single university. There was wonderful  swags. So in IT and general school, having a PSD T-shirt, I mean, open PSD folks are  pretty good at doing cool T-shirts was, you know, you were the cool guy in that case.  And nowadays, if you go to those universities, they don't have any PSD anymore, at least  in Europe, as the main desktop. The students are not attracted at all by any way. Our swags  are all very, I would say, company compliant. And that's, it plays a lot in people, in young  people, at least in Europe, not playing as much with the PSD as we used to do when we  were kids. And in the project, you can have a look at the middle edge. For Europe, at  least, they are all about, most of the contributors are about my age, more plus or minus five  years. That's used to not be the case. I'm sure that something is related also to that.  I'm sorry, man. And that's all. So, don't know if you have any questions? No? Thank  you.
Daroussin__Introducing_FreeBSD_in_new_environment_the_good_the_bad_0_107.png Daroussin__Introducing_FreeBSD_in_new_environment_the_good_the_bad_108_251.png Daroussin__Introducing_FreeBSD_in_new_environment_the_good_the_bad_252_1415.png Daroussin__Introducing_FreeBSD_in_new_environment_the_good_the_bad_1416_1967.png Daroussin__Introducing_FreeBSD_in_new_environment_the_good_the_bad_1968_4319.png Daroussin__Introducing_FreeBSD_in_new_environment_the_good_the_bad_4320_5099.png Daroussin__Introducing_FreeBSD_in_new_environment_the_good_the_bad_5100_7367.png Daroussin__Introducing_FreeBSD_in_new_environment_the_good_the_bad_7368_7415.png Daroussin__Introducing_FreeBSD_in_new_environment_the_good_the_bad_7416_10751.png Daroussin__Introducing_FreeBSD_in_new_environment_the_good_the_bad_10752_14039.png Daroussin__Introducing_FreeBSD_in_new_environment_the_good_the_bad_14040_17243.png Daroussin__Introducing_FreeBSD_in_new_environment_the_good_the_bad_17244_18011.png Daroussin__Introducing_FreeBSD_in_new_environment_the_good_the_bad_18012_19655.png Daroussin__Introducing_FreeBSD_in_new_environment_the_good_the_bad_19656_20783.png Daroussin__Introducing_FreeBSD_in_new_environment_the_good_the_bad_20784_23267.png Daroussin__Introducing_FreeBSD_in_new_environment_the_good_the_bad_23268_24335.png Daroussin__Introducing_FreeBSD_in_new_environment_the_good_the_bad_24336_29051.png Daroussin__Introducing_FreeBSD_in_new_environment_the_good_the_bad_29052_29231.png Daroussin__Introducing_FreeBSD_in_new_environment_the_good_the_bad_29232_31499.png Daroussin__Introducing_FreeBSD_in_new_environment_the_good_the_bad_31500_32375.png Daroussin__Introducing_FreeBSD_in_new_environment_the_good_the_bad_32376_33275.png Daroussin__Introducing_FreeBSD_in_new_environment_the_good_the_bad_33276_33767.png Daroussin__Introducing_FreeBSD_in_new_environment_the_good_the_bad_33768_37595.png Daroussin__Introducing_FreeBSD_in_new_environment_the_good_the_bad_37596_41327.png Daroussin__Introducing_FreeBSD_in_new_environment_the_good_the_bad_41328_43883.png Daroussin__Introducing_FreeBSD_in_new_environment_the_good_the_bad_43884_46763.png Daroussin__Introducing_FreeBSD_in_new_environment_the_good_the_bad_46764_48731.png Daroussin__Introducing_FreeBSD_in_new_environment_the_good_the_bad_48732_55799.png Daroussin__Introducing_FreeBSD_in_new_environment_the_good_the_bad_55800_59291.png Daroussin__Introducing_FreeBSD_in_new_environment_the_good_the_bad_59292_62027.png Daroussin__Introducing_FreeBSD_in_new_environment_the_good_the_bad_62028_63431.png Daroussin__Introducing_FreeBSD_in_new_environment_the_good_the_bad_63432_78125.png
Hello, everyone. Thank you for attending my talk. So as you may expect, it's not really a technical talk. It's a talk I wanted to give for a while based on my experience and because I wanted to share how through this, the other in perceiving the multiple companies I've been working in and where previously I ended up being introduced and used because I keep hearing from time to time that BSD is not gaining any place in new companies or I keep hearing things from developers which are not really reflecting what I've lived during the last 15 years. So I thought it would be a good idea to share my views. So first, it is highly subjective. It's my point of view. I've been working in various companies and my net reflects your experience, my net reflects what you think. It's also what I believe has a student in France, in Europe, so it might be different here in North America. So first, who am I? I'm a free BSD user since 1998 basically. I won't be a free BSD developer right now and I won't be using free BSD if free BSD was not running on my laptop back in the time and if I wasn't able to use it on my laptop. So I think it's very important that we keep running properly on desktop and laptops just because of that as a student even more than after because I mean you need to use it to work on it to be able to do it. So using free BSD as a student, I became a free BSD committer in 2010. I think my first PR was in 1999, something because I wanted to watch TV on my laptop and I got a BKTR driver which was not working properly which I closed in 2012 supersede by Evans and never been committed. So I know what it is waiting behind the PR for a while. Becoming a committer, I got quite quickly involved into the governance of the project. First being I've been introduced as a support manager in 2011 and since 2014 I'm in the core team. I'm not running again for this time, so I will be free for that hopefully in a couple of weeks. Another important thing is most of my contribution, I've been doing that all on my free time and nothing being paid for. Unfortunately, I'm no more a student since 2003, so I had to work, find a work. I've been working mostly in Europe, in France and also quite a bit in Africa, in French Africa where I also have put some free BSD in production and that's also part of the experience. I've been doing quite different jobs and only one ended up being free BSD related for real which was the previous job, not the job I'm working on right now. So who are the targets, meaning the people who are the kind of usage of free BSD I've met during those 15 years? So we have, I've been working with old Cousins and Min, meaning that they set up free BSD by just installing it on the server manually and then running it daily manually. I've been working on helping creating some appliances based on free BSD. I've been also working in with more modern usage in the cloud, let's say, where you deploy everything with a lot of the mission and you don't want to end up manually connecting to the machine each day to see what happened. And last is my last experience with being able to deploy some free BSD, totally diskless, where you still have a lot of automation to specialize it but you have one single image in the beginning. So I think I pretty much cover a large scope of usage people can do with free BSD. The audience I have is mostly CIS admins and for the CIS admins it was, I would say there was two era. The first era was from the vendor time. Unix people were mostly, at least in France, in the company I was working for, people that were either from IBM, HP, so they were, when you come with the Unix, is it IX because your vendor is ABM or, but they didn't know much about other available operating system and didn't trust that a community-driven operating system could be worse using in large scale. And right now I'm more facing the youngsters coming all from the Linux world and what is free BSD, what kind of Linux destroy that and think that, well, in both case free BSD was pretty much unknown from the people I have in the companies besides the name. And the other people I met were developers, so two kind of developers, I would say the regular developers they just have an application, they don't care what it runs on and then it has some, you have to teach them that free BSD runs like this or like that if they want that to be performance or people that are, that will be hacking on the base system to improve it or to modify things. And last is PCI DSS author. I don't know if you have tried to auditors. I don't know if you have, everyone knows what PCI DSS is. Probably not. That's compliance to do if you want to do some bank, to play with the banks. And so you need to comply to a bunch of rules that were designed by Visa mostly. The thing is usually PCI DSS auditors are applying, I would say, some rules that they write on a book. And so it's quite fits if you have a red hat or if you have even an Ubuntu it could work. If you have free BSD they are totally lost and they don't know where to look at and that's quite complicated to explain them how to deal with that. The other thing is when people got introduced to free BSD they got a bunch of myths if they knew it. So the first one is you come and people say, hey, BSD is for hardcore people who are just for random regular C-set means so I'm not sure we want to use that. It will be complicated for us. Leading to free BSD, well, BSD in general is hard to use. BSD is only a source operating system and I'm running in production. I need to run binary packages because I need to deploy quite quickly and quite safely. And last they don't understand that there is not one BSD and BSD means nothing. There are multiple different operating systems and they mix up what they write on open BSD, on net BSD and free BSD and they end up with something which is not working on any of the three. So last thing is when they are, that goes for every single open source project. They think because the BSD have an old, it's quite old project with people hacking on it for a while, if something is not available as a feature or as a tool it probably means there is a history behind that and it's not a good idea. While the real reason each time is no one did it and that's all. And so go ahead and do it. So that's a bunch of myths I have to face. My methodology to each time I introduce free BSD was pretty clear. Never impose free BSD, never come and say this is free BSD and you need to install it. Do not try to convert people. If they are good with Linux, keep them with Linux because if you try to convert them you will get all the blame each time it breaks and they won't really use it. Be accessible. So first is it's not you who are driving them to go to free BSD. They use and they choose free BSD because they figure out that it might be the right tool for the right solution, for the right problem. But if you leave them alone they mostly have problems so you might be, you might be still accessible so that they come to you asking questions. I never wanted to introduce free BSD in those companies. They always ended up using it by themselves. I talk to them but I don't want to be the free BSD guy having everything on the shoulder if something is broken. And be reactive is when they are having a problem if you let them in the problem for a couple of weeks or months then they will probably switch back to Linux quite quickly or to whatever operating system they will use usually. They were, besides the fact that they didn't necessarily knew about free BSD by itself. They were already facing free BSD at least. In France you can always find free BSD somewhere in a company in different forms. So through NetAskQ, I don't know if you heard of it but NetAskQ is a firewall company, a French one in the beginning. It's now named StormShield. It's quite big. It's belong to Airbus and it's deployed everywhere in Europe. So you always find some free BSD in big and small companies in Europe. PFCense is, we can find it quite a lot in a lot of companies. So that's also a good start to find some free BSD so you can tell them if they don't trust it. You are using PFCense. You're very happy with it, meaning that your operating system is reliable for you. Maybe you should give it a try on other areas. Frenaz obviously. So this one, I found it in a couple of telecom company, Nixon. I think we have some committers working over there. It was quite interesting when I figured out it was a free BSD. And obviously, Juniper and Isilon, you can find them quite a lot. So the thing is, okay, they are now trying free BSD, trying to use it, but they want to feel at home. So the first thing they do is usually they mingle the system. So the first thing, if you just let them alone installing as a system in a free BSD and try to run in, you will quickly figure out that they have installed a log rotate. They have installed a RCSlog or whatever is the name of the application they like on Linux without even thinking that maybe we have our own thing. So you need to talk to them before they totally mingle the system and you have to insert from scratch. So usually you teach them and then they, oh, new C-Slog, what that thing? And oh, it's quite nice. Or they discover that the C-Slog we have is able to do quite a bunch of things where you don't necessarily need RCSlog. I mean, if there is features in RCSlog you need them. Of course, you switch, but most of the time you don't need to. So going back to, going to the ugly, the good, the bad and the ugly. So in 15 years, the package management has changed quite a lot on free BSD. So at the old time, the only good thing I heard, I needed to try to make a good thing about it. The only good thing I heard about it is, oh, that's pretty cool because it's highly customizable. So because of the Pulse 3, mostly more than the tools. So if I have my own patches, if I want to customize the way something is installed, it's very easy. But it's almost impossible to reproduce a setup. So you have a box which is running properly and then you want to take all the binary packages and reinstall that somewhere else. And then now imagine deploying 200 different boxes with exactly the same setup. It's almost impossible. The other solution is to, was to rebuild the ports on every single of those machines, which is not very convenient when you have a lot of boxes. And it's so very hard also to maintain a repo. Basically, to have your own repo, it was build everything on your system, package create dash A, and then try to put that somewhere else. It was very not convenient. It was close to impossible to automate. I don't know if some have tried back in the time to play with, well, Popet was not there, but CFNG was there, all the tools there. I should look at the recipes for those tools back in the time. Not very nice. Most of them were backed around port upgrade or port master, meaning that when you have a run, it takes hours if one of those things to build is very large. Not very convenient. Binary maintenance is close to impossible. It was impossible to upgrade, I don't know, Aperture has a vulnerability, PHP as well, and I want to just upgrade those two packages on my running servers to get the fixes. There was no mechanism for that. So nowadays, we have a new package manager. It's not perfect, but at least it allows you to easily create and maintain repositories. From what I hear from the people using it, it's very usable. Of course, it has some bugs, but the common line interface is quite easy when they come from the Linux world, in particular when they come from Debian. They find it quite easy because you don't have three different commands to do the same thing. APT catch for search, APT get for install and whatever. So now they merge that into one single APT, so they're pretty much in shape. If you come from Red Hat, the YUM UI was already pretty close to what Pekage is doing, but they always, you don't have to teach them. It's very, it's defined by themselves, the right commands easily, so the UI is okay. And the other thing is we have up-to-date packages available all the time. That's something people like. So before that, the binary repositories were updated whenever the cluster was not breaking, and so we got a lot of port manager just looking at logs, preying so that the whole build goes and we could deploy it. Now a days, everything is automated. We have updates almost every day for the quarterly branches, every two or three days for the latest branch. We have a quarterly branch, which we didn't have back in the time. The bad thing is they still want because they have views to, they are used to the Linux world. They want to have some kind of long-term support repository where you just have a long-term being more than a year where you have the same version of the packages on the security fixes, small bug fixes so that they can use all the automation tools that, where they decide to do some pinning on some version because whatever. And multi-repository support is still very crappy in package. It kind of works, but still a crap. So nowadays, they have upspeople the law of automation, and so they don't want to manually edit one configuration file. They want to have a tool which will, if you run it twice, will do twice the same thing on the machine and make sure that the machine is in a given state with the given configuration wherever you want. When I've been working on a puppet, there was a talk just before about puppets, or when I've been working with Ansible and other things like that, people were very happy with what usually FreeBSD provides. The RC.com.d is very useful for being able to deploy a configuration just by adding a new file somewhere and you don't have to edit automatically. So basically in automation, one thing you hate is having a giant configuration file where you have to edit the line which is between this one and this one and make sure that you haven't broken anything because try to do that programmatically, it's kind of painful. While if you have a directory where you have a file which just have the lines you need, the lines you want, it's very easy. So you don't want to edit RC.com in that case, you really want to use RC.com.d and that works pretty well. Basically the whole base system is now quite automation-friendly with.d files everywhere. There was two exception which I fixed recently based on this experiment. It was CRUNTAPS and C-SLOG. So at Gandhi, we were working at the time, they were using a lot, a lot of CRUNTAPS and those CRUNTAPS were deployed through Puppet and you don't want to edit the CRUNTAPS files. You don't want to do things like that. But what you want is being able to say, for this service, if you have a CRUNTAP, I would have files slash.etc slash CRUNTAPD slash the name of my service and if I want 10 CRUNTAPS, I will just add 10 files. So I modified the CRUNTAPS to that, CRUNTAPS to that, it's able to read that and since CRUNTAPS is now friendly to automation. C-SLOG was also painful. You really don't want to, same way, C-SLOG.conf and append things all the time and then there is an update and you need to merge things and you might have problems. So we modified C-SLOG to learn the C-SLOG.conf format so that there is a new keyword which is include here or include, I don't remember. And by default we include user local, C-SLOG.d and C-SLOG.d so that if you're, if a package want to install a specific log configuration, you just put the file there. If C-SLOG wants to do that, he modify the patch, check whatever tool they like or are unable to put something in that place. The batting are, so gel.conf is, basically people like the jail when they used it, like gel.conf because it's easy, there's a lot of right. But the problem with gel.conf is it's one single static file so I can have, I can't have one, one jail per, one configuration file per jail. Well, actually I can but I need to specify when I start the jail through the dash C command. And if you do that when you do jail dash R, it will forget where the file was and then it will mess up the shutdown of the jail. So that's a bad one that needs to be fixed. At least adding some includes and better also keeping, keeping memory, keep in mind the configuration it used when it was started. The second one is INEDD. So a lot of people, in particular in the new generation find me crazy because I keep using it and it did quite a lot in companies. But I find it very useful and rather than each time coding something that will listen on the network, INEDD is doing that pretty well by itself. The problem is I need to append a new line to INEDD and so I need to do that with POPET or with Ansible and so it's a mess because it's an append or a modification. It's not adding in a given directory and then I also need to modify ATC services which has, which suffers the same problem. So we have a new one. It's actually a very cool tool called BlacklistD. It allows to, when connected to SSHG for example, to Blacklist some IPs when there are problems. The thing is the configuration file is done the old way so it's the same kind of syntax as INEDD which is okay except that there is absolutely no way to add new configuration file but this one. Basically everything which does not allow you to install a simple file is not automation friendly and but in previous the most of the base system is cool in that area. Yeah? What services on you go behind INEDD successfully? Behind? With INEDD? INEDD? Quite a lot of things. ZFS send receive for example was my last example but it's usually homemade things. You know you have all made tooling on main. I wrote some, we will say nowadays microservices behind INEDD. I mean the two old time tools that I use a lot are INEDD, ATD and CRON tabs. I don't know why I would each time reinvent my own mechanism for that. They are there, they work. Right, so about the tooling of the base system. So they come, keep in mind that they come either from AI, old Unix or about Linux. Usually utilities you can find on those, at least old Unix are quite rough. On previous, usually they have very good surprises. We come with a lot of utilities out of box in the base system which are not used to. So I won't list all of them but all the control, all the utility, I mean I have in a utility per LSI cards is something which is very, very convenient. I really hate using the official tools for them. They also, for statistics, we get very good utility. So if you wanted to use just what CSTAT provides or what Tops provide on Linux for example, you need to install a bunch of external utilities. So for example, top-mio, you would need to install IOTOP or you have the choice between two or three different utilities that just present the same thing. CSTAT-SAT is something that they usually surprise to find. So we have arch statistics in CSTAT in the top way. That's very useful people like that. The man pages. Yes, we have man pages for almost all our utilities. It's might sound crazy that we put that as a good thing but actually on the Linux tool you don't always have them or most of the time when you have them, they are very, let's say, small, not much information and you have to switch to the info pages if they have some or to dash, dash help. So having man pages for almost everything including drivers is very useful and they usually very have good surprises just to figure out that. The G-part common, I'm surprised but it's each time something that people comes to me since we have it saying, okay, so we can, you can do partitioning on the desk in a simple manner. You don't need to have a very complicated tool you don't understand and it's also a scriptable manner because you can script very easily what all G-part is doing and you don't need to read something like 10 pages of manual to be able to understand what G-part is doing. So that's usually a good surprise. The bad things, three firewalls, it's with another L is better. So it might sound not bad surprises but it is one because when they set up something they end up, what should they use? And so usually it's actually two firewalls because one of the three no one ever heard about it. So at least for the new people. So IP filter can be removed in my opinion but that's just me. But yeah, that's a bad thing. Soundmail, I don't say, I'm not saying it's ugly, I say it's a bad thing. Usually people don't even know that there is a soundmail and they don't care about it. It's just when they figure out that it is a full soundmail that they freak out. But the way it is set up is just delivering mail locally. It's not that big deal. VI, I put that in the bad things because they are all nowadays at least used to VIM. And so in most Linux distribution there is a minimal version of VIM which they call VI. And so they try to do a lot of fun to sing inside of VI and they end up yelling a lot in front of the desk. And the old, so obviously we have G-pop now but I had four wild problems just to tell the people that no FD is not the same as FD from Linux. And yes, we need BSD labels and yes, you need to read the documentation because you probably don't know what you're doing otherwise. And that was very complicated. Something else that is bad, so when I say bad it's not bad necessarily for me. It's bad from the point of view of the people I had in front of me. It's the common strictness. I actually like this one but on Linux they have the habit to put the argument at the end after some, well the option at the end after some arguments and this versa. On BSD usually you just put the options first and then the arguments. And the last one is it's kind of funny when you don't say the people that they are logging to a previous deboxes and then they try for every single command dash dash help all the time because that's the way they expect documentation to come out, not demand pages. I like that one and I don't want us to add more dash help because it's funny. The ugly, it's kind of interesting. The number one complaint from far I got about previously in production from far CSH. So I like to know who likes CSH in this room. Okay, so yeah, sorry, TCSH. So what I figured out is most of the people including the one coming from, well the one coming from AIX doesn't know how to use, how to use KSH already correctly so they don't have any quadratic shell so they are lost nowhere. But basically people are lost in front of CSH just because it doesn't work like every other born compatible shell, interactive shell. It's crazy to see how much anger it gives to people. My opinion is we should keep CSH because obviously some people like it here and all the time but we should probably import an interactive shell which is more let's say born compatible. There are two options in my opinion, MKSH and OKSH which could be imported. There is no need to import a big bash or import a big ZSH. So that's probably something we should do and please if we do that switch routes to default on it so that that pleases my users. And so as I was saying we have no good shell, interactive shell as should have said in the base system. Our business edge is not an interactive shell. What it is but. Deploying free BSD. So one of the good thing people discover on free BSD is free BSD updates. They are usually very happy about it because it allows you to upgrade things without. I don't remember last time I got a real break edge in production based on that and I've been using that since almost free BSD 18 production and people really likes it simple fast well fast for patches not fast for upgrade between releases but actually upgrade between releases works which is something they don't have the habit of when they come from Linux. The batting is nowadays we a lot more in cloudish environments and if you take free BSD out of box it doesn't know how to play correctly in the cloud. So for example we don't have cloud in it's equivalency so cloud in it is a bunch of crap you won't ever say to me it's good but we need something which is able to at least provide easily images in various cloud providers. And we always forgot in the free BSD project that a lot of deployment might be done in machines which doesn't have access to internet. It's not that it's quite frequent actually that there are machines which are completely disconnected from the internet and those machines need still to have access to packages to binary updates and things like that. And I don't know if you try to mirror a free BSD update we have no mechanism provided by the project. So what I end up doing is one of the mirrors has index on so you can LFTP mirror and then put that somewhere else that's not really convenient would be nice if we provide our sync mechanism or maybe we have but I'm not aware of it. The ugly that's the very most awful part of the free BSD project when you're trying to deal with modern deployment. Pixi. Pixi on free BSD so I picked in particular to do seamless installation to boot some run disks diskless machine. I've been giving talk about how I did that at Gandhi and how I modified the loader to do that and I'm not speaking I don't want any NFS route. I mean no not in production. So have a look at my talk I gave two years ago to see how I work without that but it's really painful. It prevents adding free BSD in part of an already existing IPX installation in a clean way. We didn't need to fix that. Having a local free BSD update server is also a very huge pain. I don't know if you try to do that but if I'm making my own modified version of free BSD for various reasons in production I want to be able to upgrade it to add some patches and so I want to have a free BSD update server because I built my own I can't use the free BSD update one and it's painful. So we're speaking of the cloud. The good thing, 10.0 support it's quite stable it does the job. I got some feedbacks about users that use it. It's pretty good. The bad thing, the behind front end, the command line usually people run away just when they figure out that the loader is there and then they will come and there is something in the middle and where is my VM they don't know. I mean it makes sense when you understand when you talk to the people who made a VIVE when you see that and VM run they say she's not a solution for that. So if we could end up with front end, a UI which is better it would be nice. So lack of some features, there is two features that VIVE likes a lot and which is preventing the usage in production. So if you're running a cloud provider you basically don't want at any time to reboot the client machine so you need live migration because there would be security issues that would be update on the hypervisor and the hypervisor you want to make and you don't want to reboot the client space and that. So without live migration it's almost impossible to imagine having so I don't know joined how they are handling that case but the other one is okay having VIVE is pretty cool on my laptop when I use FreeBSD because I can do a lot of things. Usually what I want to do is be able to use things that are not running on FreeBSD properly. Let's say some proprietary program like Cisco, WebEx teams for example, things like that and for that what I want is to have some pass through because it wants access to my webcam for example so USB pass through and things like that would be useful. Like of Dynamics CPU is badly written it's I've actually talked to three smallish cloud providers which adds so Gandhi was one of them, the two others have not been working with them but have been in contact with them who has a feature where they dynamically add CPU to the guest VMs and the complainant FreeBSD was not supported that. Well actually I don't think that's a big problem because nowadays people reboot when they add CPUs but. The use case is dynamic scaling so my application is taking all my CPU I had a new one. Don't reboot. Storage, I won't spend much time on the storage area because I think everyone knows the good thing and the bad things there but basically people likes that FS the very good surprise I got myself when working at Gandhi was CTL. It works very well it's compared to the operating system we're using before for that we were very happy to switch to it and all the utility we have in the base system I was already saying all the LSI cards have a utility. I wrote myself the new Sessute so that we got the generic thing to blink the disk in a J-Buds. The GM tools that you have being able to have things like a GNOP to do a lot of testing is wonderful people so we're quite surprised. The ugly thing is so at first it's a good thing and when you start using it in production for real with a lot of people it becomes an ugly thing. It's NFS performances so at Gandhi we use quite a lot NFS. We were very happy when we switched from an Accenture to FreeBSD we got good performances so we added a lot of clients on it and then boom. So it was very, it's still very painful in terms of performance, very bad performances, very bad default tweaks, tunings and NFSv4 is even, I mean have you tried really in production NFSv4 against NFSv4 clients on Linux? Try to mess both of them speak the same language and with a lot of IUs you will end up with lots of problems. So it's mostly under heavy load so you need to have a heavy load usage to figure that but this was very ugly for us. So hacking, I don't know why, I think I lost the habit of the Linux machine in the time but people were very surprised to be able to have three, four different tool chains and being on the same system and being able to build with everything. Formado is okay, no reason to not have those tool chains so that's a good point for FreeBSD. Dtrace is a good one. PMCstat when people discuss the tool is a good one, they were happy with it. The bad thing is the PMCstat UI. I mean you use perfectly valid options, you flip them in the command line and you keep them in a valid order but you flip them and then it does things completely differently. It's really an unfriendly UI for that kind of tool. The lack of sanitizers in the LLVM land or GCC land, most of the sanitizers that has been written recently are not working. I think only Libasan is working correctly. But so the force, so that most people don't see it, they don't care about it but as soon as you need to do some modification in the loader, I had to do some in Gandhi and I don't want to spend two, three, four weeks or having one of my engineers spending two, three, four weeks learning force just to do some small modifications. It's a pain. And I'm really speaking of small modification, add a new entry in the menu, add something like that. So yes, I'm expecting a lot from the LLVM loader. And GDB66, that, I mean the people that do not understand why we have such old GDB in the base system, of course I tell them you can use GDB from the ports but then they have to call it GDB8, blah, blah, blah, something and not just GDB something. And so it's mostly, it's not bad not in a really because it's mostly a surprise and they always, they switch quite quickly to the recent GDB or they try to learn LLDB but GDB66 is something, I mean, it's archeology. Community feedbacks. The good thing is when I try to push people to the community, usually I try not to go myself instead of them. I try to tell them I'll go there, speak to this one and this one so that they learn. The good thing I have is it's not worse than anywhere else. So sometimes there is, it's a black hole, sometimes you have friendly people, sometimes you have yelling people, it depends but basically people are just, it's okay. They're happy to find quite a good documentation with multiple handbooks. The bad thing is if you try to read the communities through PR, through reviews, through whatever mechanism we have, if you want to catch attention of a developer you need to mail him directly or to go through mailing list. I'm not sure it's a bad thing but it's a bad thing the fact that it's not documented as such and so when people are looking for a contribution guide they should know that yeah, mailing list is the main way to reach people. I don't know what a word contribution, I think I forgot some words. Anyway, localized documentation, please can we rip out every single localized documentation which is not up to date because I'm in France, I was also working in French Africa, so they speak French in all case, so they read the French documentation and they learned about FreeBSD4 nowadays. So that should be removed and also if we could get some kind of modern encoding rather than this thing where people are just, oh, FreeBSD is not reporting in Unicode, yes that's the question I have just because of the link on the URL. The really ugly part but that's the same for every project from what I heard is bug reporting if you don't know someone. Even if it's someone who knows someone, who knows someone, who knows someone, most of the time your bug will stay there for a while and you need to go to IRC being someone, I'm always doing the proxy so I've been doing the proxy recently on ZFS, I've been doing that for a bunch of company just to, hey, I know you know that, can you please talk to those and try to help them fixing that or they have a patch, can you review? That's pretty bad but from what I hear it's almost the same on every single open source project. So, oh, I remember contribution. So contribution is the way we document, no, that's the next slide, sorry. Contributing. So the good thing about contributing to the FreeBSD project, as I said earlier, when you discuss with the community, you're basically not better but not worse than the other project so it's fine. The bad thing is subversion. Now there is subversion is a bad thing. Not that they don't want to use it but the surprise is basically the same that when we switch to subversion and before that we said we were on CVS and people were on CVS, that's exactly the same thing today. I mean it's not a bad tool, it does the job but people have the habit of both Git and Mercury also. The first thing is, whoa, what is this outdated project? Then you choose them, you tell them and it's fine enough and they can use it. Lack of documentation. We don't have a good documentation on how to contribute. Where should I go? Who should I go to? What is the right mechanism? Should I send my patch to Buxila? Should I send my patch to Fabricator? Should I open a pull request on the GitHub we have? That's the problem. Oh, I should have said in bad GitHub. The GitHub mirror, yes? So, I have a course on the question on the lack of documentation. Is it bad that we have the version number of what is or is it bad that we don't document what they are and document what the expectations for each one should be? Both and it comes just after. So, I forgot to mention in the bad GitHub. We have a GitHub mirror but there is nothing on it saying that it's a read only mirror. So, people are trying to use it or don't know it. It has more confusion on how to contribute. I know that people are pushing saying that GitHub is the way to go. All the people I talked to that were contributing through GitHub, through whatever, they don't care. As long as they have a process, I mean they like Fabricator as much as GitHub. GitHub is easy because it could mirror but as long as they have a patch that they could send through a simple command, that's all they care about most 99% of the time. But we don't have documentation of the processing. We have one single way to contribute. Right now, some people, some developer are refusing to review outside of mailing list. Some wants to have the patch in the Xela. Some wants to have the patch in Fabricator. Not having one way to contribute is mostly what is driving people crazy. If you look at other operating systems that are having contribution, they all have one way. Have you tried to contribute to Linux, for example? You will never go through GitHub. You go through a process when you send to mailing list. And then usually the first reply you get from the mailing list is something quite, what the fuck have you done here? Is a link to a documentation which explains you how to contribute and who you should send things to and what kind of quality reports you should do. And that works. I mean, I don't think we can say that the Linux community is a dying community. And they don't use GitHub. They just have one properly documented way to contribute. It does. It's the same for most of the other communities. I mean, OpenBSD, I think there are some OpenBSD folks in the room. They, this is the same, basically. You send a mail to a mailing list. There is one single way. They don't have bugs anymore because they don't have bug tracker anymore so that you have only one way to send things. Desktop lactaves. So surprisingly, even in companies, most of the people that are daily facing freebies, the in-production, they start willing to have it on their laptop, their desktop. I don't think it's a surprise because it's easier to test locally before doing something. So the thing is, it kind of works beside what's claimed. It kind of works most of the time. It's usable. And the very good surprise they have is the sound while they're coming from far. I mean, have you ever played with Alza or even worse? So the sound is a very good surprise most of the time. The bad thing is where is drivers? We are often behind, but kind of works anyway. GPU drivers, it's way better recently, but you have to be on stable or on current. Users are using releases. So I know a lot of people are pushing to have stability to have everyone moving on stable or current. Users are using releases. That's what we do, releases. So we need to have things that works on releases. Out-summarized by, usually freebies is a surprise and a good surprise for the people when they use it. I've never faced places where they were, what is this thing? Usually they say, oh, I can maybe use FreebiesD instead of that operating system here because it seems to be better and they try to extend it. A good example was at Gandhi, I tried not to put any FreebiesD anywhere and they were adding more and more. That thing, if you don't know someone, it's very hard to get in. There is a company who shadow who is doing some, let's say, gamer boxes, cloudish gamer boxes. They were using some, they're trying to use some FreebiesD in the know one. It was very hard. I had to play the proxy for them. It was very hard for them to report and get people on their bugs and they got a lot of, a lot of the FSS issues. So we fixed most of them since I made the proxy. But the bad thing is regression unstable. So the same company, they were using, so they tried to use Curent because I told them to use head and Curent like on FreebiesD then because all the FSS patches they needed were unstable, they switched to stable because it was easier to track if you don't have a FreebiesD developer. Even at Gandhi where I was, we switched back to stable because it was easier to track in the end. And they now, I know at least three companies that decided to move out of FreebiesD because of some regression in particular, I don't, I haven't put it there. In particular, the IF Lib in stable 11, it broke aggregation, it broke performance. I don't remember the details. There was a bunch of issues. It was open for a while. On Curent, I couldn't understand that. It takes some time to fix and it's progress. I mean, I think we need it. But having that on a stable branch which becomes release was very painful. And so at least three companies moved out of FreebiesD because of that. So if we could be very careful about when we merge, what we merge, what we do there. Lack of Erata, as I was saying, most people are using releases if you have a bug in a program in a release, an important bug or an important regression in a given release, we need an Erata. I know why we don't have Erata. We need to improve in that place. But there is a lot of, a lot of things that should, at least after a release, deserve an Erata. It's very hard to say to the users, okay, you use a release, you have, you're facing this bug. It will be fixed on the next release. You probably should use a stable. In particular, two or three weeks after the release. Another bad thing is we are not cool anymore. So that relates to when I was a student, the random reason why I needed to use FreebiesD was we were cool. PSD was in every single university. There was wonderful swags. So in IT and general school, having a PSD T-shirt, I mean, open PSD folks are pretty good at doing cool T-shirts was, you know, you were the cool guy in that case. And nowadays, if you go to those universities, they don't have any PSD anymore, at least in Europe, as the main desktop. The students are not attracted at all by any way. Our swags are all very, I would say, company compliant. And that's, it plays a lot in people, in young people, at least in Europe, not playing as much with the PSD as we used to do when we were kids. And in the project, you can have a look at the middle edge. For Europe, at least, they are all about, most of the contributors are about my age, more plus or minus five years. That's used to not be the case. I'm sure that something is related also to that. I'm sorry, man. And that's all. So, don't know if you have any questions? No? Thank you.
I've lived during the last 15 years. So I thought view. I've been working in various companies and pretty much cover a large scope of usage people for, people that were either from IBM, HP, so that. The other thing is when people got always ended up using it by themselves. I talk to they usually very have good surprises just to can do partitioning on the desk in a simple remember last time I got a real break edge in about how I did that at Gandhi and how I modified pain. I don't know if you try to do that but if does the job. I got some feedbacks about users the good thing and the bad things there but so we're quite surprised. The ugly thing is so at the sanitizers that has been written recently are the main way to reach people. I don't know what a the URL. The really ugly part but that's the same