Can JS also build the metaverse?
Benetou, Fabien (Utopiah)
We keep on hearing about the metaverse but what is it and more importantly, can JS be used to build it? We'll briefly clarify what the metaverse is and give practical examples today with federated virtual reality servers managed by different persons. Behind the buzzword from Facebook/Meta there is a truly interesting concept : connecting virtual worlds! To do so there is no need to be one of the largest advertising company. In fact there are several solutions working today allowing to navigate from 3D or VR web pages and even keep information across, like a profile. During this presentation we will explore the WebXR specifications, in particular links between pages, and test an implementation running today named ImmersSpace based on Mozilla Hubs and ActivityPub.
 you Hello everyone, my name is Fabien and we're going to discuss about the metaverse today. I'm going to start by introducing myself, define what the metaverse is, because otherwise if we start to build or see how it can be built properly without defining what it is, then probably we're not going to talk about the same thing. So it's going to be a definition, obviously their canonical definition. And then I'm going to show you how to build one version of the metaverse, thanks to existing tools. Why do I start, let's say with a non-technical part is because I think honestly starting with why it matters, why it's interesting is more important, otherwise we build really cool stuff, but there is no point. So even worse is dangerous. So that's the little visual that you can see there on the screen. It's a bit of a mess, so I'm going to decompose it. And I'm going to start by introducing myself. So my name is Fabien Benetou. I work at the European Parliament. Here you can see me letting the former president of the European Parliament try some AR glasses and using augmented reality do some facial recognition. So a lot of proof of concepts like this. Testing is the same hardware, but how it could be used and a lot more different prototypes, events, creating things, focusing on virtual and augmented reality on the web. What motivates me to do this, besides the art school new tech, is because I have a bunch of notes and I want to organize them. So you can see some example of documents there and being in a virtual world. Not even in video actually, because you can see the video and someone else being there. So it's social. And then I have my notes on it and a bunch of different examples like contrary to my live bulbs, organizing the documents again, managing your camera anyway. A bunch of different prototypes and proof of concepts. I mentioned the proof of concepts part because that also means I'm not working on production ready software. So my perspective is going to be a lot more about discovery, thinkability, but not necessarily like long term maintenance. I think we're at that stage now anyway for the topic, both of VR or the metaverse that it's still early stage. So that's also why the architecture in my opinion is so important to get right, not just making stuff. And yeah, you might have heard the metaverse term recently, but the first time I not just used it, but participate in a professional meeting was then and that is in 2016 in San Jose, the Samsung HQ. The goal was to define actually one of the core components, which is link traversal. So the WPC for those who don't I imagine most people into JavaScript know what it is, but it's basically defining standards for the browser. So how the browser are going to behave the same way when we ask them to do the same thing. That's very important, especially at least to know what it is and how it works, especially when we have so few browsers in general and especially in the other. So very few browsers that are WebEx are compatible. So knowing who is doing what and also being aware of their motivation, I think it's pretty crucial. But anyway, the point is, I'm not deciding to make a metaverse presentation for first them out of the blue. I've been working on this for more than five years now. So here's my VR. First of all, you don't need VR to be in the metaverse. You can really well without any problem go around in a 3D world connected to another and see people work on solving a problem without one of those fancy headsets. First of all, they are fancy, but they're getting cheaper and cheaper. You have either standalone ones or ones like this that connect to a desktop, a desktop that can run Linux. It's working really well. I'm not saying it's going to be trivial plug and play five minutes setup, but it's really working properly. But again, why in the first place? Okay, you could use a VR headset to join the metaverse. So why? Because you're going to feel there. I know it sounds silly. And if you've never tried it, that's going to be actually my first recommendation is do try it. And you can remain critical about it. Because this is far from perfect. But it works. So that's the amazing part is like you put the headset on and then you feel or you know you're somewhere else. You're physically present here, let's say, working from home, little office. And yet you see a different place and you feel you there. So this is amazing. This is like an amazing opportunity. It's not a requirement once again. But I do recommend you give it a go at least once again, even if you say, well, this is a gadget that is stupid, I'm never going to need it. Perfectly fine. And my perspective on this is also that you don't need every tool all the time. You might need a pen and paper to sketch through. You might need a computer to develop. You might need your headset to visit an architectural model, but you're not going to visit an architectural model through text. You can that's going to be just too challenging and you're going to have different interpretation. The same way you're not going to read a 300 pages in a view headset, you can do that, but I don't recommend it. So then it begs the question like what is your for when is it actually interesting. So I left a couple of examples here. So this is for the toys. It's a game if you want. And you're exploring what it means to manipulate not just one D2D3D but also for the objects in a mathematical space. So I think for this kind of usages, that's really, really interesting. That's really fun because it's I don't know how to do this otherwise. And in that's the kind of, let's say use case where VR really shines is you just you just don't know how to make that without a headset without some manipulation and being in space. The other part is places where you shouldn't be because it's too dangerous or it's too far away. Let's say we're not going to go to Mars, not you, not me, but we can be there through VR and games like this. I'm going to the whole video play, but it's kind of crazy. It is. So you have some pretty scary creatures and places and then you have a mission. You have some game dynamics. So that's another really cool usage is that you're telling a story shaping a moment in time for someone. And again, it works. So the other part is so I said I wouldn't read a 300 pages document, but my own motivation for VR is here I have a small office and I want to have my posters, my notes, because I take a bunch of notes on any kind of support. And I want to be able to organize. So I don't want to read my notes in VR because it's just like the pixel density is too low. And it's just not as pleasant. And maybe I'm so not used to it. Maybe I like to be on my couch and just scribble on my notes. So this I don't want to give up on. But one part that I really like and I don't know if some of you have done it is like toss all the papers I need to read on the floor and then organize it or stick it on a whiteboard or pin them on the wall to get some structure out of it. I think this is really cool. So I've done a lot of prototypes, different ways to do it, a kind of notes city where each of those are actually pages of my wiki. So this is basically impossible to do otherwise, especially if you keep it in sync. So in my opinion, that's number one, you don't need a VR headset to explore the metaverse or build on the metaverse. It is amazing if you do because then you're going to feel there and with a shared space with colleagues or friends or strangers, but you don't need it. And if you do need it, if you do rather decide that you're going to use a VR headset, make it worth your well, make it worth enough time, excitement, enjoyment or find a radically new way to do things that would otherwise be impossible. VR is not necessary. It's just another tool in your toolbox. It's nothing more than that, nothing less than that. The same way you would use a pen or a desktop with a keyboard or a tablet, whatever device or screwdriver, whatever. It is what you make use out of it or so. One yet another tool in your toolbox. It makes the metaverse a lot more there once you're in it. So I'm going to give a practical example of what are the building blocks, let's say, of the metaverse. I'm going to use Mozilla Hubs because it is open source and it's web-based. So that's the JavaScript part. It's using React for the 2D UI and a frame for the 3D and VR elements or components and itself a frame is based on 3DS. So I'm going to start by creating a room, which is going to be a 3D world and as I said before, VR is optional so I meant it. You don't need a VR headset. If you have one, try the ones. It is amazing. If you don't, then you can use a desktop. You can even use your phone or a tablet. So here I'm just navigating around, looking at the 3D world and you can modify that of course because it is on GitHub. The server side is relatively complex but then the client, which allows you to see all this and connect with other people, is much easier to tinker with. And I have a bunch of different prototypes so feel free to ask me what's possible or not happy to clarify. So what of course I can do is connect with another machine to make as if we were there in the same space together because that is one of the points. Entering the room and I'll go navigate where my other little robot is. I'm going in front of it so that we can see the mute quickly. And voilà. We can see each other let's say. So that's a good step. But then okay, just like one 3D world, one on the web across platform. That's cool but that's not the metaverse, is it? Sure. But then how do I make my metaverse? So I think starting with hubs is good because it shows the potential of a full blown experience. Again, across different solutions, phone, headsets, desktops that I'm going to show. But I think it's quite heavy because there are a lot of different moving pieces. So an easier example is this. It's called network A-frame. It's still the same basic component, meaning A-frame with 3GS to do 3D on the web. Here I see two different Avertar but very basic. What's a bit different, let's say, from a traditional immerse or rather network A-frame experience is that I can log in here in the top right corner. So what I can do is I can log in through this. So I'm going to go through this server, immerse.ovh, because I logged in as utopia.at immerse.ovh. Accept. And then it's going to load my avatar. It's a cube so it's not super exciting. But yeah, you can see it working there. And then what I can do here on the other side is, logging as not utopia and the interesting part is it's going to use another server, immerse-first-dem.ovh. So I'm going to log in there and allow it. And you can see my nickname changed. I don't think I had an avatar on this so it kept the very cute, let's say, round person. So that's it. That's the metaverse in my opinion. We can stop here. I'm going to explain or at least decompose a little bit what happened in that session and then how to do this with hubs at the end. So what happened is that I can click on my profile as not utopia among immerse.first-dem.ovh. Whereas if I go on the other profile, I'm on immerse.ovh. No first-dem in there. So I made those two servers a couple of days ago and it took me a couple of hours for each and it's really on affordable servers. I'll put links on how to set those up. But yeah, that's the important part is that I have two different persons or profiles, but they want two different servers and yet they can talk to each other or they could before I moved there. So how does that work? Well, maybe you're familiar with Mastodont or PeerTube. So I have my own PeerTube server and I put a bunch of video including this one, how to set up hubs. And the beauty of it, besides being PeerTube and working, is this aspect of the federation, meaning that I'm on my server with a couple of users, but also I have 35 followers but they are not personal followers. They are instead entire instances, which also pose their own video, with their dedicated topic, dedicated culture, etc. And I don't have a son what they're doing on their server. They don't have a son what I'm doing on my server and that's again the beauty of federation. We have something and we're connected through a protocol, specifically here, ActivityPub. And that's the same thing on eMirrorz, being for network day frame, or for Mozilla hubs, or maybe if you make your own implementation, there are eMirrorz clients that are also available. And then through ActivityPub, you can say I'm doing this, I'm doing that, I'm on that server, here is my friend list, here is my profile list. But that's the beauty of it, is that through this federation, I'm part of a bigger network, but I still define my own rules on my own server. So I have those two different eMirrorz space and then I can see also the, only I can see the log of my different activities, some chat where I had this, who I met, etc. So now the interesting thing is there is this one page here on how to set it up and that it works with Mozilla hubs. So what I can do is, for example, go on my hubs experiment server. I'm already logged in, so it's not going to prompt me just on the previous example for network day frame. Yeah, it's going to check if I want to log in with this, accept and then it already finds my nickname and my avatar. And then what I can do is connect with a private browser window where I connected with another profile. It connects me as, let's say, an eMirrorz or randomly generated nickname, but if I use my eMirrorz login, it attempts to log in and I can see who I am. So I think I had a not utopia and then to prove, let's say, to show the demonstrate the federated login, I'll use it on this other server. And that's also, that's the key part that when you're new to federation might be a bit strange. It's like, oh, you have, I don't know, your Twitter login or you have your Gmail address. You need here to put the domain because maybe you're not using the most famous server. And again, that sounds a bit maybe strange in the first place when you do that. But the most popular is not actually the most interesting. And again, it depends on your own values. So I'm going to try to log in, hopefully, so it redirects to another server. Hopefully it has my yes, because I, it's generated. So I don't remember what it is for these examples. And then I can decide what I want to share, what I don't want to share, giving everything or giving nothing or giving everything. I suggest this one, I'll allow this. And now we change my nickname to not utopia. And then I can join, put an avatar. I don't care for the sound. And voila, we can see each other. I'll try to be here. So you can see on both windows. And here, voila, I have my two avatars seeing each other in this virtual space. So again, it's not convenient to show, but otherwise that works with your headset. Still optional. But in my opinion, that's the most interesting kind of experience you can get. And yes, we can use the chat to say hi. And I have it here and I'll have it also in my e-merc history. So I can close that and I left a mold on my own now to clarify a bit more what happened there. So what happens is, what did a hubs brings this 3d interface e-mers is going to add on top of it some customization. And that customization is going to be this login button is going to be the friend list are going to be whatever new behavior you want to set up. And thanks to activity pub is going to keep the updates and login. So you can do this with network day frame. Another way to build your 3d world with hubs or if you have another one, so frame VR is an alternative. You can also use that. So you already have a network day frame, modular hubs, frame VR, by Vibella, to at least have three different implementation of VR 3d that are compatible with e-mers. So to me, that's the exciting part. e-mers per se give a set of functionalities, hubs is nice, frame VR is nice for other things. It's closed source, but hubs is open source. But the point is they can still communicate to each other and I can communicate with somebody with another profile from another platform. So that's the core aspect to me. That's the core aspect that e-mers is bringing to 3d or VR and that's the core component of what I believe is a proper metaverse, meaning something federated, something you keep on having control on, something that if you want to join servers together, network together, you can. But according to rules and forms of behavior that you believe are just or fair or things like this. A quick word that this also means you need money or you should have a way to pay for your hard work or your services. And that's why also it's interesting that it's not required also, but e-mers propose directly web monetization. So for some of you who are not aware of it, web monetization is, for example, using Coil or other platforms going to give a little bit of money every second you're on the server. And then you can have access to a new feature, let's say other scenes, premium scenes or other avatars. And then you're going to find a way why it's interesting. It's because then you're not going to have to rely on advertising. Advertising per se is not necessarily bad, but if it destroys privacy, the consequences for our society, for the political system is pretty terrible. It's not just about yet another type of soda. It goes much deeper than this is much more riskier than this, I would say. So yeah, that's a set of components that I think make a lot of sense together. WebXR, how you're going to bring 3D content to a headset with a six-defeat freedom moving in space and with your controllers. GLTF for the 3D assets, how you can have an avatar or a backpack, whatever you want to put and then show. And then Immerse relying on Activity Pub in order to have a federated server. So I'm going to summarize this in a minute or so. So what we saw was you can have 3D or 2D or even text as ways to converse on the metaverse. The number of dimensions or if it's using Vio or not, of course, makes a different experience, but it's not the most important. We can have one example of an implementation like a Hubs room. And you can have two rooms on the same server, let's say the Modzilla.com server. But what that means is that we have one server and we have the same person owning it, the same person paying for it. And you can even have two servers. You can have one Modzilla Hubs.modzilla.com or whatever Modzilla.com that are two physical servers. It's still paid by the same person. And when what people usually consider is that, okay, it's technical. But what I'm trying to argue here is that it's different. It's about power so that if you have, you like Modzilla or maybe you don't. And maybe you like some of the rules they set in place, maybe you don't. So what's interesting is if you have, for example, two different Hubs server, one owned by Modzilla, another owned by you or me. And we don't have exactly the same rules. And yet we find a way to connect. We find a way that I can link to your server or you can link to mine. It means going to have different owners and different rules. What's also interesting is you can do that, but with different Hubs or services, you can have Hubs with, for example, Network A-frame as I showed, also frame VR, which is not open source, hence the little lock there. But what's interesting again is when we can do that, go from a server to another with an owner that is different with even an implementation that is different. But instead of having just, let's say, going there, we have some form of data portability. We have some form of bringing or relying on the same, for example, profile. Because the profile can have a list of friends, can have a 3D avatar, can have whatever you can come up with. But basically, it means being able to go from a server to another with different owners, with different rules, and not starting from scratch or being, let's say, someone else. So that's what federation is all about. That federation works in 3D or in VR is actually crucial or I would say more interesting, not, let's say, for the technical aspect, even though it is interesting, but for what some would say, to quote McLuhan, the medium is the message, what does it mean when we have a different medium? Here we're going to, let's say, walk around in VR, but then it's basically a metaphor for social networks, all those different 3D worlds together. So in social network, the message is society. What kind of society do we want to live in, work in, etc. And the structure, the technical structure underlying the metaverse is going to change our society. So that's why I believe using open source, using a federation type of network is so important. It's not just because it's a cool tech, it's because it's going to impact back our society the way we want to live together. So that's why I encourage you to check Activity Pub and the eMERS implementation. That's why I encourage you to try hubs and how you can have different federated server and maybe set up your own server. Maybe try your own implementation of Activity Hubs with another server, but as long as we can connect back together in a way where we don't even need or want to use a central server, then I think that's going to shape back a society that is much more resilient, much more interesting, and for us as individuals, much better than a central actor in the middle that is driven by advertising. So thanks a lot for your time and just let me know if you want to meet on the metaverse, if you want to build yours or if you have any questions. Happy to try to help if I can. Take care.
 you  Hello everyone, my name is Fabien and we're going to discuss about the metaverse today. I'm going to start by introducing myself, define what the metaverse is, because otherwise if we start to build or see how it can be built properly without defining what it is, then probably we're not going to talk about the same thing.  So it's going to be a definition, obviously their canonical definition. And then I'm going to show you how to build one version of the metaverse, thanks to existing tools.  Why do I start, let's say with a non-technical part is because I think honestly starting with why it matters, why it's interesting is more important, otherwise we build really cool stuff, but there is no point.  So even worse is dangerous. So that's the little visual that you can see there on the screen. It's a bit of a mess, so I'm going to decompose it.  And I'm going to start by introducing myself. So my name is Fabien Benetou. I work at the European Parliament. Here you can see me letting the former president of the European Parliament try some AR glasses and using augmented reality do some facial recognition.  So a lot of proof of concepts like this. Testing is the same hardware, but how it could be used and a lot more different prototypes, events, creating things, focusing on virtual and augmented reality on the web.  What motivates me to do this, besides the art school new tech, is because I have a bunch of notes and I want to organize them. So you can see some example of documents there and being in a virtual world.  Not even in video actually, because you can see the video and someone else being there. So it's social. And then I have my notes on it and a bunch of different examples like contrary to my live bulbs, organizing the documents again, managing your camera anyway.  A bunch of different prototypes and proof of concepts. I mentioned the proof of concepts part because that also means I'm not working on production ready software.  So my perspective is going to be a lot more about discovery, thinkability, but not necessarily like long term maintenance.  I think we're at that stage now anyway for the topic, both of VR or the metaverse that it's still early stage. So that's also why the architecture in my opinion is so important to get right, not just making stuff.  And yeah, you might have heard the metaverse term recently, but the first time I not just used it, but participate in a professional meeting was then and that is in 2016 in San Jose, the Samsung HQ.  The goal was to define actually one of the core components, which is link traversal. So the WPC for those who don't I imagine most people into JavaScript know what it is, but it's basically defining standards for the browser.  So how the browser are going to behave the same way when we ask them to do the same thing. That's very important, especially at least to know what it is and how it works, especially when we have so few browsers in general and especially in the other.  So very few browsers that are WebEx are compatible. So knowing who is doing what and also being aware of their motivation, I think it's pretty crucial. But anyway, the point is, I'm not deciding to make a metaverse presentation for first them out of the blue.  I've been working on this for more than five years now.  So here's my VR. First of all, you don't need VR to be in the metaverse. You can really well without any problem go around in a 3D world connected to another and see people work on solving a problem without one of those fancy headsets.  First of all, they are fancy, but they're getting cheaper and cheaper. You have either standalone ones or ones like this that connect to a desktop, a desktop that can run Linux.  It's working really well. I'm not saying it's going to be trivial plug and play five minutes setup, but it's really working properly. But again, why in the first place?  Okay, you could use a VR headset to join the metaverse. So why? Because you're going to feel there. I know it sounds silly. And if you've never tried it, that's going to be actually my first recommendation is do try it.  And you can remain critical about it. Because this is far from perfect.  But it works. So that's the amazing part is like you put the headset on and then you feel or you know you're somewhere else. You're physically present here, let's say, working from home, little office.  And yet you see a different place and you feel you there. So this is amazing. This is like an amazing opportunity. It's not a requirement once again.  But I do recommend you give it a go at least once again, even if you say, well, this is a gadget that is stupid, I'm never going to need it. Perfectly fine.  And my perspective on this is also that you don't need every tool all the time. You might need a pen and paper to sketch through. You might need a computer to develop.  You might need your headset to visit an architectural model, but you're not going to visit an architectural model through text. You can that's going to be just too challenging and you're going to have different interpretation.  The same way you're not going to read a 300 pages in a view headset, you can do that, but I don't recommend it.  So then it begs the question like what is your for when is it actually interesting.  So I left a couple of examples here. So this is for the toys. It's a game if you want. And you're exploring what it means to manipulate not just one D2D3D but also for the objects in a mathematical space.  So I think for this kind of usages, that's really, really interesting. That's really fun because it's I don't know how to do this otherwise.  And in that's the kind of, let's say use case where VR really shines is you just you just don't know how to make that without a headset without some manipulation and being in space.  The other part is places where you shouldn't be because it's too dangerous or it's too far away. Let's say we're not going to go to Mars, not you, not me, but we can be there through VR and games like this.  I'm going to the whole video play, but it's kind of crazy. It is. So you have some pretty scary creatures and places and then you have a mission. You have some game dynamics.  So that's another really cool usage is that you're telling a story shaping a moment in time for someone. And again, it works.  So the other part is so I said I wouldn't read a 300 pages document, but my own motivation for VR is here I have a small office and I want to have my posters, my notes, because I take a bunch of notes on any kind of support.  And I want to be able to organize. So I don't want to read my notes in VR because it's just like the pixel density is too low. And it's just not as pleasant. And maybe I'm so not used to it.  Maybe I like to be on my couch and just scribble on my notes. So this I don't want to give up on.  But one part that I really like and I don't know if some of you have done it is like toss all the papers I need to read on the floor and then organize it or stick it on a whiteboard or pin them on the wall to get some structure out of it.  I think this is really cool. So I've done a lot of prototypes, different ways to do it, a kind of notes city where each of those are actually pages of my wiki.  So this is basically impossible to do otherwise, especially if you keep it in sync.  So in my opinion, that's number one, you don't need a VR headset to explore the metaverse or build on the metaverse.  It is amazing if you do because then you're going to feel there and with a shared space with colleagues or friends or strangers, but you don't need it.  And if you do need it, if you do rather decide that you're going to use a VR headset, make it worth your well, make it worth enough time, excitement, enjoyment or find a radically new way to do things that would otherwise be impossible.  VR is not necessary. It's just another tool in your toolbox. It's nothing more than that, nothing less than that. The same way you would use a pen or a desktop with a keyboard or a tablet, whatever device or screwdriver, whatever.  It is what you make use out of it or so. One yet another tool in your toolbox.  It makes the metaverse a lot more there once you're in it.  So I'm going to give a practical example of what are the building blocks, let's say, of the metaverse. I'm going to use Mozilla Hubs because it is open source and it's web-based.  So that's the JavaScript part. It's using React for the 2D UI and a frame for the 3D and VR elements or components and itself a frame is based on 3DS.  So I'm going to start by creating a room, which is going to be a 3D world and as I said before, VR is optional so I meant it. You don't need a VR headset.  If you have one, try the ones. It is amazing. If you don't, then you can use a desktop. You can even use your phone or a tablet.  So here I'm just navigating around, looking at the 3D world and you can modify that of course because it is on GitHub.  The server side is relatively complex but then the client, which allows you to see all this and connect with other people, is much easier to tinker with.  And I have a bunch of different prototypes so feel free to ask me what's possible or not happy to clarify.  So what of course I can do is connect with another machine to make as if we were there in the same space together because that is one of the points.  Entering the room and I'll go navigate where my other little robot is.  I'm going in front of it so that we can see the mute quickly.  And voilà. We can see each other let's say. So that's a good step.  But then okay, just like one 3D world, one on the web across platform. That's cool but that's not the metaverse, is it?  Sure. But then how do I make my metaverse?  So I think starting with hubs is good because it shows the potential of a full blown experience.  Again, across different solutions, phone, headsets, desktops that I'm going to show.  But I think it's quite heavy because there are a lot of different moving pieces.  So an easier example is this. It's called network A-frame. It's still the same basic component, meaning A-frame with 3GS to do 3D on the web.  Here I see two different Avertar but very basic.  What's a bit different, let's say, from a traditional immerse or rather network A-frame experience is that I can log in here in the top right corner.  So what I can do is I can log in through this. So I'm going to go through this server, immerse.ovh, because I logged in as utopia.at immerse.ovh.  Accept. And then it's going to load my avatar. It's a cube so it's not super exciting.  But yeah, you can see it working there. And then what I can do here on the other side is,  logging as not utopia and the interesting part is it's going to use another server, immerse-first-dem.ovh.  So I'm going to log in there and allow it.  And you can see my nickname changed. I don't think I had an avatar on this so it kept the very cute, let's say, round person.  So that's it. That's the metaverse in my opinion. We can stop here.  I'm going to explain or at least decompose a little bit what happened in that session and then how to do this with hubs at the end.  So what happened is that I can click on my profile as not utopia among immerse.first-dem.ovh.  Whereas if I go on the other profile, I'm on immerse.ovh. No first-dem in there. So I made those two servers a couple of days ago  and it took me a couple of hours for each and it's really on affordable servers.  I'll put links on how to set those up. But yeah, that's the important part is that I have two different persons or profiles,  but they want two different servers and yet they can talk to each other or they could before I moved there.  So how does that work? Well, maybe you're familiar with Mastodont or PeerTube.  So I have my own PeerTube server and I put a bunch of video including this one, how to set up hubs.  And the beauty of it, besides being PeerTube and working, is this aspect of the federation, meaning that I'm on my server with a couple of users,  but also I have 35 followers but they are not personal followers. They are instead entire instances, which also pose their own video,  with their dedicated topic, dedicated culture, etc. And I don't have a son what they're doing on their server.  They don't have a son what I'm doing on my server and that's again the beauty of federation.  We have something and we're connected through a protocol, specifically here, ActivityPub.  And that's the same thing on eMirrorz, being for network day frame, or for Mozilla hubs,  or maybe if you make your own implementation, there are eMirrorz clients that are also available.  And then through ActivityPub, you can say I'm doing this, I'm doing that, I'm on that server, here is my friend list, here is my profile list.  But that's the beauty of it, is that through this federation, I'm part of a bigger network, but I still define my own rules on my own server.  So I have those two different eMirrorz space and then I can see also the, only I can see the log of my different activities,  some chat where I had this, who I met, etc.  So now the interesting thing is there is this one page here on how to set it up and that it works with Mozilla hubs.  So what I can do is, for example, go on my hubs experiment server.  I'm already logged in, so it's not going to prompt me just on the previous example for network day frame.  Yeah, it's going to check if I want to log in with this, accept and then it already finds my nickname and my avatar.  And then what I can do is connect with a private browser window where I connected with another profile.  It connects me as, let's say, an eMirrorz or randomly generated nickname, but if I use my eMirrorz login, it attempts to log in and I can see who I am.  So I think I had a not utopia and then to prove, let's say, to show the demonstrate the federated login, I'll use it on this other server.  And that's also, that's the key part that when you're new to federation might be a bit strange.  It's like, oh, you have, I don't know, your Twitter login or you have your Gmail address.  You need here to put the domain because maybe you're not using the most famous server.  And again, that sounds a bit maybe strange in the first place when you do that.  But the most popular is not actually the most interesting.  And again, it depends on your own values.  So I'm going to try to log in, hopefully, so it redirects to another server.  Hopefully it has my yes, because I, it's generated.  So I don't remember what it is for these examples.  And then I can decide what I want to share, what I don't want to share, giving everything or giving nothing or giving everything.  I suggest this one, I'll allow this.  And now we change my nickname to not utopia.  And then I can join, put an avatar. I don't care for the sound.  And voila, we can see each other.  I'll try to be here.  So you can see on both windows.  And here, voila, I have my two avatars seeing each other in this virtual space.  So again, it's not convenient to show, but otherwise that works with your headset.  Still optional.  But in my opinion, that's the most interesting kind of experience you can get.  And yes, we can use the chat to say hi.  And I have it here and I'll have it also in my e-merc history.  So I can close that and I left a mold on my own now to clarify a bit more what happened there.  So what happens is, what did a hubs brings this 3d interface e-mers is going to add on top of it some customization.  And that customization is going to be this login button is going to be the friend list are going to be whatever new behavior you want to set up.  And thanks to activity pub is going to keep the updates and login.  So you can do this with network day frame.  Another way to build your 3d world with hubs or if you have another one, so frame VR is an alternative.  You can also use that.  So you already have a network day frame, modular hubs, frame VR, by Vibella, to at least have three different implementation of VR 3d that are compatible with e-mers.  So to me, that's the exciting part.  e-mers per se give a set of functionalities, hubs is nice, frame VR is nice for other things.  It's closed source, but hubs is open source.  But the point is they can still communicate to each other and I can communicate with somebody with another profile from another platform.  So that's the core aspect to me.  That's the core aspect that e-mers is bringing to 3d or VR and that's the core component of what I believe is a proper metaverse, meaning something federated, something you keep on having control on, something that if you want to join servers together, network together, you can.  But according to rules and forms of behavior that you believe are just or fair or things like this.  A quick word that this also means you need money or you should have a way to pay for your hard work or your services.  And that's why also it's interesting that it's not required also, but e-mers propose directly web monetization.  So for some of you who are not aware of it, web monetization is, for example, using Coil or other platforms going to give a little bit of money every second you're on the server.  And then you can have access to a new feature, let's say other scenes, premium scenes or other avatars.  And then you're going to find a way why it's interesting.  It's because then you're not going to have to rely on advertising.  Advertising per se is not necessarily bad, but if it destroys privacy, the consequences for our society, for the political system is pretty terrible.  It's not just about yet another type of soda. It goes much deeper than this is much more riskier than this, I would say.  So yeah, that's a set of components that I think make a lot of sense together.  WebXR, how you're going to bring 3D content to a headset with a six-defeat freedom moving in space and with your controllers.  GLTF for the 3D assets, how you can have an avatar or a backpack, whatever you want to put and then show.  And then Immerse relying on Activity Pub in order to have a federated server.  So I'm going to summarize this in a minute or so.  So what we saw was you can have 3D or 2D or even text as ways to converse on the metaverse.  The number of dimensions or if it's using Vio or not, of course, makes a different experience, but it's not the most important.  We can have one example of an implementation like a Hubs room.  And you can have two rooms on the same server, let's say the Modzilla.com server.  But what that means is that we have one server and we have the same person owning it, the same person paying for it.  And you can even have two servers.  You can have one Modzilla Hubs.modzilla.com or whatever Modzilla.com that are two physical servers.  It's still paid by the same person.  And when what people usually consider is that, okay, it's technical.  But what I'm trying to argue here is that it's different.  It's about power so that if you have, you like Modzilla or maybe you don't.  And maybe you like some of the rules they set in place, maybe you don't.  So what's interesting is if you have, for example, two different Hubs server, one owned by Modzilla, another owned by you or me.  And we don't have exactly the same rules.  And yet we find a way to connect.  We find a way that I can link to your server or you can link to mine.  It means going to have different owners and different rules.  What's also interesting is you can do that, but with different Hubs or services, you can have Hubs with, for example,  Network A-frame as I showed, also frame VR, which is not open source, hence the little lock there.  But what's interesting again is when we can do that, go from a server to another with an owner that is different with even an implementation that is different.  But instead of having just, let's say, going there, we have some form of data portability.  We have some form of bringing or relying on the same, for example, profile.  Because the profile can have a list of friends, can have a 3D avatar, can have whatever you can come up with.  But basically, it means being able to go from a server to another with different owners, with different rules,  and not starting from scratch or being, let's say, someone else.  So that's what federation is all about.  That federation works in 3D or in VR is actually crucial or I would say more interesting,  not, let's say, for the technical aspect, even though it is interesting, but for what some would say, to quote McLuhan,  the medium is the message, what does it mean when we have a different medium?  Here we're going to, let's say, walk around in VR, but then it's basically a metaphor for social networks, all those different 3D worlds together.  So in social network, the message is society.  What kind of society do we want to live in, work in, etc.  And the structure, the technical structure underlying the metaverse is going to change our society.  So that's why I believe using open source, using a federation type of network is so important.  It's not just because it's a cool tech, it's because it's going to impact back our society the way we want to live together.  So that's why I encourage you to check Activity Pub and the eMERS implementation.  That's why I encourage you to try hubs and how you can have different federated server and maybe set up your own server.  Maybe try your own implementation of Activity Hubs with another server,  but as long as we can connect back together in a way where we don't even need or want to use a central server,  then I think that's going to shape back a society that is much more resilient, much more interesting,  and for us as individuals, much better than a central actor in the middle that is driven by advertising.  So thanks a lot for your time and just let me know if you want to meet on the metaverse,  if you want to build yours or if you have any questions.  Happy to try to help if I can. Take care.
metaverse_0_167.png metaverse_168_2543.png metaverse_2544_3215.png metaverse_3216_3395.png metaverse_3396_3467.png metaverse_3468_4115.png metaverse_4116_4247.png metaverse_4248_4355.png metaverse_4356_4499.png metaverse_4500_5243.png metaverse_5244_5459.png metaverse_5460_10703.png metaverse_10704_10991.png metaverse_10992_12659.png metaverse_12660_14255.png metaverse_14256_16043.png metaverse_16044_17003.png metaverse_17004_17159.png metaverse_17160_17291.png metaverse_17292_17555.png metaverse_17556_18695.png metaverse_18696_19007.png metaverse_19008_19223.png metaverse_19224_20003.png metaverse_20004_21419.png metaverse_21420_21503.png metaverse_21504_22271.png metaverse_22272_26723.png metaverse_26724_26987.png metaverse_26988_31031.png metaverse_31032_31091.png metaverse_31092_34643.png metaverse_34644_34727.png metaverse_34728_34847.png metaverse_34848_36851.png metaverse_36852_37715.png metaverse_37716_43630.png
you Hello everyone, my name is Fabien and we're going to discuss about the metaverse today. I'm going to start by introducing myself, define what the metaverse is, because otherwise if we start to build or see how it can be built properly without defining what it is, then probably we're not going to talk about the same thing. So it's going to be a definition, obviously their canonical definition. And then I'm going to show you how to build one version of the metaverse, thanks to existing tools. Why do I start, let's say with a non-technical part is because I think honestly starting with why it matters, why it's interesting is more important, otherwise we build really cool stuff, but there is no point. So even worse is dangerous. So that's the little visual that you can see there on the screen. It's a bit of a mess, so I'm going to decompose it. And I'm going to start by introducing myself. So my name is Fabien Benetou. I work at the European Parliament. Here you can see me letting the former president of the European Parliament try some AR glasses and using augmented reality do some facial recognition. So a lot of proof of concepts like this. Testing is the same hardware, but how it could be used and a lot more different prototypes, events, creating things, focusing on virtual and augmented reality on the web. What motivates me to do this, besides the art school new tech, is because I have a bunch of notes and I want to organize them. So you can see some example of documents there and being in a virtual world. Not even in video actually, because you can see the video and someone else being there. So it's social. And then I have my notes on it and a bunch of different examples like contrary to my live bulbs, organizing the documents again, managing your camera anyway. A bunch of different prototypes and proof of concepts. I mentioned the proof of concepts part because that also means I'm not working on production ready software. So my perspective is going to be a lot more about discovery, thinkability, but not necessarily like long term maintenance. I think we're at that stage now anyway for the topic, both of VR or the metaverse that it's still early stage. So that's also why the architecture in my opinion is so important to get right, not just making stuff. And yeah, you might have heard the metaverse term recently, but the first time I not just used it, but participate in a professional meeting was then and that is in 2016 in San Jose, the Samsung HQ. The goal was to define actually one of the core components, which is link traversal. So the WPC for those who don't I imagine most people into JavaScript know what it is, but it's basically defining standards for the browser. So how the browser are going to behave the same way when we ask them to do the same thing. That's very important, especially at least to know what it is and how it works, especially when we have so few browsers in general and especially in the other. So very few browsers that are WebEx are compatible. So knowing who is doing what and also being aware of their motivation, I think it's pretty crucial. But anyway, the point is, I'm not deciding to make a metaverse presentation for first them out of the blue. I've been working on this for more than five years now. So here's my VR. First of all, you don't need VR to be in the metaverse. You can really well without any problem go around in a 3D world connected to another and see people work on solving a problem without one of those fancy headsets. First of all, they are fancy, but they're getting cheaper and cheaper. You have either standalone ones or ones like this that connect to a desktop, a desktop that can run Linux. It's working really well. I'm not saying it's going to be trivial plug and play five minutes setup, but it's really working properly. But again, why in the first place? Okay, you could use a VR headset to join the metaverse. So why? Because you're going to feel there. I know it sounds silly. And if you've never tried it, that's going to be actually my first recommendation is do try it. And you can remain critical about it. Because this is far from perfect. But it works. So that's the amazing part is like you put the headset on and then you feel or you know you're somewhere else. You're physically present here, let's say, working from home, little office. And yet you see a different place and you feel you there. So this is amazing. This is like an amazing opportunity. It's not a requirement once again. But I do recommend you give it a go at least once again, even if you say, well, this is a gadget that is stupid, I'm never going to need it. Perfectly fine. And my perspective on this is also that you don't need every tool all the time. You might need a pen and paper to sketch through. You might need a computer to develop. You might need your headset to visit an architectural model, but you're not going to visit an architectural model through text. You can that's going to be just too challenging and you're going to have different interpretation. The same way you're not going to read a 300 pages in a view headset, you can do that, but I don't recommend it. So then it begs the question like what is your for when is it actually interesting. So I left a couple of examples here. So this is for the toys. It's a game if you want. And you're exploring what it means to manipulate not just one D2D3D but also for the objects in a mathematical space. So I think for this kind of usages, that's really, really interesting. That's really fun because it's I don't know how to do this otherwise. And in that's the kind of, let's say use case where VR really shines is you just you just don't know how to make that without a headset without some manipulation and being in space. The other part is places where you shouldn't be because it's too dangerous or it's too far away. Let's say we're not going to go to Mars, not you, not me, but we can be there through VR and games like this. I'm going to the whole video play, but it's kind of crazy. It is. So you have some pretty scary creatures and places and then you have a mission. You have some game dynamics. So that's another really cool usage is that you're telling a story shaping a moment in time for someone. And again, it works. So the other part is so I said I wouldn't read a 300 pages document, but my own motivation for VR is here I have a small office and I want to have my posters, my notes, because I take a bunch of notes on any kind of support. And I want to be able to organize. So I don't want to read my notes in VR because it's just like the pixel density is too low. And it's just not as pleasant. And maybe I'm so not used to it. Maybe I like to be on my couch and just scribble on my notes. So this I don't want to give up on. But one part that I really like and I don't know if some of you have done it is like toss all the papers I need to read on the floor and then organize it or stick it on a whiteboard or pin them on the wall to get some structure out of it. I think this is really cool. So I've done a lot of prototypes, different ways to do it, a kind of notes city where each of those are actually pages of my wiki. So this is basically impossible to do otherwise, especially if you keep it in sync. So in my opinion, that's number one, you don't need a VR headset to explore the metaverse or build on the metaverse. It is amazing if you do because then you're going to feel there and with a shared space with colleagues or friends or strangers, but you don't need it. And if you do need it, if you do rather decide that you're going to use a VR headset, make it worth your well, make it worth enough time, excitement, enjoyment or find a radically new way to do things that would otherwise be impossible. VR is not necessary. It's just another tool in your toolbox. It's nothing more than that, nothing less than that. The same way you would use a pen or a desktop with a keyboard or a tablet, whatever device or screwdriver, whatever. It is what you make use out of it or so. One yet another tool in your toolbox. It makes the metaverse a lot more there once you're in it. So I'm going to give a practical example of what are the building blocks, let's say, of the metaverse. I'm going to use Mozilla Hubs because it is open source and it's web-based. So that's the JavaScript part. It's using React for the 2D UI and a frame for the 3D and VR elements or components and itself a frame is based on 3DS. So I'm going to start by creating a room, which is going to be a 3D world and as I said before, VR is optional so I meant it. You don't need a VR headset. If you have one, try the ones. It is amazing. If you don't, then you can use a desktop. You can even use your phone or a tablet. So here I'm just navigating around, looking at the 3D world and you can modify that of course because it is on GitHub. The server side is relatively complex but then the client, which allows you to see all this and connect with other people, is much easier to tinker with. And I have a bunch of different prototypes so feel free to ask me what's possible or not happy to clarify. So what of course I can do is connect with another machine to make as if we were there in the same space together because that is one of the points. Entering the room and I'll go navigate where my other little robot is. I'm going in front of it so that we can see the mute quickly. And voilà. We can see each other let's say. So that's a good step. But then okay, just like one 3D world, one on the web across platform. That's cool but that's not the metaverse, is it? Sure. But then how do I make my metaverse? So I think starting with hubs is good because it shows the potential of a full blown experience. Again, across different solutions, phone, headsets, desktops that I'm going to show. But I think it's quite heavy because there are a lot of different moving pieces. So an easier example is this. It's called network A-frame. It's still the same basic component, meaning A-frame with 3GS to do 3D on the web. Here I see two different Avertar but very basic. What's a bit different, let's say, from a traditional immerse or rather network A-frame experience is that I can log in here in the top right corner. So what I can do is I can log in through this. So I'm going to go through this server, immerse.ovh, because I logged in as utopia.at immerse.ovh. Accept. And then it's going to load my avatar. It's a cube so it's not super exciting. But yeah, you can see it working there. And then what I can do here on the other side is, logging as not utopia and the interesting part is it's going to use another server, immerse-first-dem.ovh. So I'm going to log in there and allow it. And you can see my nickname changed. I don't think I had an avatar on this so it kept the very cute, let's say, round person. So that's it. That's the metaverse in my opinion. We can stop here. I'm going to explain or at least decompose a little bit what happened in that session and then how to do this with hubs at the end. So what happened is that I can click on my profile as not utopia among immerse.first-dem.ovh. Whereas if I go on the other profile, I'm on immerse.ovh. No first-dem in there. So I made those two servers a couple of days ago and it took me a couple of hours for each and it's really on affordable servers. I'll put links on how to set those up. But yeah, that's the important part is that I have two different persons or profiles, but they want two different servers and yet they can talk to each other or they could before I moved there. So how does that work? Well, maybe you're familiar with Mastodont or PeerTube. So I have my own PeerTube server and I put a bunch of video including this one, how to set up hubs. And the beauty of it, besides being PeerTube and working, is this aspect of the federation, meaning that I'm on my server with a couple of users, but also I have 35 followers but they are not personal followers. They are instead entire instances, which also pose their own video, with their dedicated topic, dedicated culture, etc. And I don't have a son what they're doing on their server. They don't have a son what I'm doing on my server and that's again the beauty of federation. We have something and we're connected through a protocol, specifically here, ActivityPub. And that's the same thing on eMirrorz, being for network day frame, or for Mozilla hubs, or maybe if you make your own implementation, there are eMirrorz clients that are also available. And then through ActivityPub, you can say I'm doing this, I'm doing that, I'm on that server, here is my friend list, here is my profile list. But that's the beauty of it, is that through this federation, I'm part of a bigger network, but I still define my own rules on my own server. So I have those two different eMirrorz space and then I can see also the, only I can see the log of my different activities, some chat where I had this, who I met, etc. So now the interesting thing is there is this one page here on how to set it up and that it works with Mozilla hubs. So what I can do is, for example, go on my hubs experiment server. I'm already logged in, so it's not going to prompt me just on the previous example for network day frame. Yeah, it's going to check if I want to log in with this, accept and then it already finds my nickname and my avatar. And then what I can do is connect with a private browser window where I connected with another profile. It connects me as, let's say, an eMirrorz or randomly generated nickname, but if I use my eMirrorz login, it attempts to log in and I can see who I am. So I think I had a not utopia and then to prove, let's say, to show the demonstrate the federated login, I'll use it on this other server. And that's also, that's the key part that when you're new to federation might be a bit strange. It's like, oh, you have, I don't know, your Twitter login or you have your Gmail address. You need here to put the domain because maybe you're not using the most famous server. And again, that sounds a bit maybe strange in the first place when you do that. But the most popular is not actually the most interesting. And again, it depends on your own values. So I'm going to try to log in, hopefully, so it redirects to another server. Hopefully it has my yes, because I, it's generated. So I don't remember what it is for these examples. And then I can decide what I want to share, what I don't want to share, giving everything or giving nothing or giving everything. I suggest this one, I'll allow this. And now we change my nickname to not utopia. And then I can join, put an avatar. I don't care for the sound. And voila, we can see each other. I'll try to be here. So you can see on both windows. And here, voila, I have my two avatars seeing each other in this virtual space. So again, it's not convenient to show, but otherwise that works with your headset. Still optional. But in my opinion, that's the most interesting kind of experience you can get. And yes, we can use the chat to say hi. And I have it here and I'll have it also in my e-merc history. So I can close that and I left a mold on my own now to clarify a bit more what happened there. So what happens is, what did a hubs brings this 3d interface e-mers is going to add on top of it some customization. And that customization is going to be this login button is going to be the friend list are going to be whatever new behavior you want to set up. And thanks to activity pub is going to keep the updates and login. So you can do this with network day frame. Another way to build your 3d world with hubs or if you have another one, so frame VR is an alternative. You can also use that. So you already have a network day frame, modular hubs, frame VR, by Vibella, to at least have three different implementation of VR 3d that are compatible with e-mers. So to me, that's the exciting part. e-mers per se give a set of functionalities, hubs is nice, frame VR is nice for other things. It's closed source, but hubs is open source. But the point is they can still communicate to each other and I can communicate with somebody with another profile from another platform. So that's the core aspect to me. That's the core aspect that e-mers is bringing to 3d or VR and that's the core component of what I believe is a proper metaverse, meaning something federated, something you keep on having control on, something that if you want to join servers together, network together, you can. But according to rules and forms of behavior that you believe are just or fair or things like this. A quick word that this also means you need money or you should have a way to pay for your hard work or your services. And that's why also it's interesting that it's not required also, but e-mers propose directly web monetization. So for some of you who are not aware of it, web monetization is, for example, using Coil or other platforms going to give a little bit of money every second you're on the server. And then you can have access to a new feature, let's say other scenes, premium scenes or other avatars. And then you're going to find a way why it's interesting. It's because then you're not going to have to rely on advertising. Advertising per se is not necessarily bad, but if it destroys privacy, the consequences for our society, for the political system is pretty terrible. It's not just about yet another type of soda. It goes much deeper than this is much more riskier than this, I would say. So yeah, that's a set of components that I think make a lot of sense together. WebXR, how you're going to bring 3D content to a headset with a six-defeat freedom moving in space and with your controllers. GLTF for the 3D assets, how you can have an avatar or a backpack, whatever you want to put and then show. And then Immerse relying on Activity Pub in order to have a federated server. So I'm going to summarize this in a minute or so. So what we saw was you can have 3D or 2D or even text as ways to converse on the metaverse. The number of dimensions or if it's using Vio or not, of course, makes a different experience, but it's not the most important. We can have one example of an implementation like a Hubs room. And you can have two rooms on the same server, let's say the Modzilla.com server. But what that means is that we have one server and we have the same person owning it, the same person paying for it. And you can even have two servers. You can have one Modzilla Hubs.modzilla.com or whatever Modzilla.com that are two physical servers. It's still paid by the same person. And when what people usually consider is that, okay, it's technical. But what I'm trying to argue here is that it's different. It's about power so that if you have, you like Modzilla or maybe you don't. And maybe you like some of the rules they set in place, maybe you don't. So what's interesting is if you have, for example, two different Hubs server, one owned by Modzilla, another owned by you or me. And we don't have exactly the same rules. And yet we find a way to connect. We find a way that I can link to your server or you can link to mine. It means going to have different owners and different rules. What's also interesting is you can do that, but with different Hubs or services, you can have Hubs with, for example, Network A-frame as I showed, also frame VR, which is not open source, hence the little lock there. But what's interesting again is when we can do that, go from a server to another with an owner that is different with even an implementation that is different. But instead of having just, let's say, going there, we have some form of data portability. We have some form of bringing or relying on the same, for example, profile. Because the profile can have a list of friends, can have a 3D avatar, can have whatever you can come up with. But basically, it means being able to go from a server to another with different owners, with different rules, and not starting from scratch or being, let's say, someone else. So that's what federation is all about. That federation works in 3D or in VR is actually crucial or I would say more interesting, not, let's say, for the technical aspect, even though it is interesting, but for what some would say, to quote McLuhan, the medium is the message, what does it mean when we have a different medium? Here we're going to, let's say, walk around in VR, but then it's basically a metaphor for social networks, all those different 3D worlds together. So in social network, the message is society. What kind of society do we want to live in, work in, etc. And the structure, the technical structure underlying the metaverse is going to change our society. So that's why I believe using open source, using a federation type of network is so important. It's not just because it's a cool tech, it's because it's going to impact back our society the way we want to live together. So that's why I encourage you to check Activity Pub and the eMERS implementation. That's why I encourage you to try hubs and how you can have different federated server and maybe set up your own server. Maybe try your own implementation of Activity Hubs with another server, but as long as we can connect back together in a way where we don't even need or want to use a central server, then I think that's going to shape back a society that is much more resilient, much more interesting, and for us as individuals, much better than a central actor in the middle that is driven by advertising. So thanks a lot for your time and just let me know if you want to meet on the metaverse, if you want to build yours or if you have any questions. Happy to try to help if I can. Take care.
going to discuss about the metaverse today. I'm you how to build one version of the metaverse, really cool stuff, but there is no point. So even but how it could be used and a lot more different virtual and augmented reality on the web. What VR. First of all, you don't need VR to be in the a VR headset to explore the metaverse or build on frame is based on 3DS. So I'm going to start by experience. Again, across different solutions, have two different persons or profiles, but they implementation, there are eMirrorz clients that metaverse, meaning something federated, something you keep on having control on, something that if summarize this in a minute or so. So what we saw of an implementation like a Hubs room. And you interesting is you can do that, but with about. That federation works in 3D or in VR is