PASTE: Fast End System Networking with netmap
Honda, Michio
Lettieri, Guiseppe
Eggert, Lars
Santry, Douglas
We introduce PASTE, an extension to the netmap framework for end systems to exploit high speed NICs, kernel TCP/IP stack and emerging persistent memory. We describe design and implementation of PASTE, as well as how to write applications on top of it. New hardware, such as high-speed NICs and persistent memory introduces significant challenges in system software design and implementation. Today's socket APIs are already unable to serve on 10 Gbps under message-oriented workloads. Suppose a server that runs a poll/kqueue event loop to handle concurrent TCP connections. Arriving messages create a long queue of ready sockets where each must be processed by a pair of read/write system calls; If the application is notified of 100 messages on parallel TCP connections, it requires 200 system calls to process this single event, significantly reducing throughput and increasing end-to-end latency. Persistent memory eliminates a bottleneck of disks/SSDs because of 2-3 orders of magnitude lower access latency, which makes the network stack be a bottleneck even when the application persists receiving data. In particular, data copy from socket buffer to persistent memory comes at significant cost. To solve these problems, we introduce PASTE, an extension to the netmap framework. Unlike the original netmap, it supports TCP/IP in the kernel, yet offering the netmap API to applications for zero copy and batching of system calls and packet I/O, even across multiple TCP connections. PASTE further supports emerging persistent memory, allowing applications to persist data without copying data since NIC's DMA. Applications can form persistent data structures using netmap's packet buffers backed by a file. PASTE was initially implemented in Linux for research purpose and demonstrated notable performance improvements. We are now porting it to FreeBSD. We will report software architecture and performance of our FreeBSD prototype. We then would like to discuss necessary extensions beyond the netmap framework, which include socket structures and NVDIMM supports. With PASTE, we believe the netmap API to be an even more generic, promising networking API of FreeBSD beyond packet I/O framework, and to enable building efficient end systems by exploiting emerging hardware.
 All right. So maybe can I start? All right. So nice to meet you. My name is Michio Honda and today I'm going to present a paste which is a high performance and the system networking framework using Netmap API. And so this is the joint work with my awesome colleagues, Giuseppe Eleti-Airi, Lars Egert and Doug Santhry. If you have any questions or something later on, so please just drop me a message on Twitter or email, whatever. And the call using today is here, but I think it will disappear at some point after merging it to original Netmap. So first of all, so what is this talk about? So this talk is about what are the problems with current network stack and socket API today and how do we solve that problem. And this talk is not about current network stack is slow. Let's bypass everything. I have awesome user space network stack. This talk is not about that. So if you expect something like this, I recommend you to go some other excellent sessions. All right. Never this. The problem is kind of similar. So the current socket API is not that fast. So to get some intuition, so let's see some benchmark with some simple request response workload. So in this experiment, we send some, the 1400 bytes HTTP post and replying back of HTTP OK, which is some 64 byte message. This is experimented down with Linux, but we can expect quite similar with free BST. And this uses just HTTP and TCP. So the typically the server calls doing this kind of processing looks like this. You fetch the ready file descriptors. So new messages are arriving. And you iterate over ready file descriptor one by one by issuing pair of read and write system call. So the shared result, so the left is throughput and the right is latency for the different number of concurrent TCP connections. So you see the throughput. It is only this user single core, but still it can achieve only 2.8 gigabits. This is not that great because today's modern network speed is 10 gigabit or 40 or even 100 gigabit per second. So latency as well. For the single TCP connection, it's actually pretty good. So we can see 23 microseconds for the HTTP run type of data. But when we have 100 concurrent TCP connections, the latency goes up to 400 microseconds. And you see, no, that's great. So this result shows that for modern network speed, socket API is not that good. So I think you got some intuition now. So the second problem is current network stack cannot support emerging non-volatile memory very well. That's the second motivation in the problem. So I think some of you may never heard about non-volatile memory. So let's quickly review it. So non-volatile memory is a literary memory whose contents survive reboot. As a result, non-volatile memory offers very fast, much faster than disk SSDs. And byte addressable not based on block, access to persistent data. So the visual looks like this. We have CPU cache at the top and underneath. You have main memory. And access there is very fast. But the contents go away when you are machine reboot. And to store data persistently, you have disk SSD at the bottom. And it has much higher capacity. And the contents survive over reboot. But access there is very slow. And you have to issue the system calls, like read, write, or some xing. But having non-volatile memory here, now you can access persistent data just using load of store and privilege instruction and access there is almost as fast as DRAM. Maybe slightly slower, but still two, three orders of magnitude faster than the hard drives and SSDs. So once NVMN put into the system, then the entire architecture looks like this. So you can create a file system on non-volatile memory, something like you create file system on RAM disk. But once the file there is N mapped into user address space, you can directly load and store the data there without using any buffer cache. That is the main point of non-volatile memory. So let's go back to the problem definition and let's see how the end-to-end system is affected by end-to-end transaction speed is affected by non-volatile memory. So just using previous code, but we have now some of the processing that store data from the socket buffer to non-volatile memory. So we have memory copied to NVMN and also flashing data from CPU cache to the actual physical media. So here is the result. So this compares the case which doesn't store data on NVMN at all and the case which store data there. So it's surprising that throughput gets almost half and latency increase almost doubles. So yes. Oh yes, that's a good point. Actually we have the same experiment but actually it related to even worse performance. So because memory copy is implemented using optimized instructions like SSC, but we don't like to move data using the normal instruction. So I can tell you the more result by showing the graph later. It's a good point. But it's the same. So the point here is, okay, so even for non-volatile memory, don't copy data if you really want high performance. So the summary of the previous two problems is let's avoid per socket system calls and IO requests. Okay, so that's the first summary or first requirement. And the second requirement is let's avoid data copy at all even for non-volatile memory. This decreases throughput and increases latency. So before the going to solution, so let's get access to write. And the question is how do we address these problems while preserving benefits offered by current network stack and socket API today? So that's the main question. So here's the page what we propose. And the page is framework that achieves scalable, flexible end system networking. And page offers true zero copy since the NICDMA even for non-volatile memory. So that's the first thing. The second point is you can batch system call and IO operations across multiple sockets. And the third point is you can use existing excellent kernel TCP, IP, CTP, UDP, IPv6 stack which has a lot of extensions not just doesn't implement the original RFC but implement almost all the RFCs. Also, but API itself on the other hand is pretty protocol independent. So it's using a net map API so it's not bound to specific transport protocols. So we can for both blocking and busy polling depending on what you want. For example, if you want some low CPU utilization or some efficiency, you could use the blocking. If you want ultimate low latency even at idle time period, then you can use the busy polling. You can also benefit from protection so your application can crash but the rest of the system is not affected. Actually the last four features are exactly what we benefit from socket API today. So going to a little bit more details, so the page consists of the two net map extensions. So net map is a packet Iof framework to efficiently move data between the kernel and the user space and that is already in previous D. The first extension is stack port. So net maps depending on type of the interface, net maps provide multiple port types where application access. So the stack port is some of the net map API but which the data goes to kernel TCP, IP, stack. So the same level of abstraction with the pipe and the valid port if you're familiar with net map already. So the other extension is Xmem subsystem. So by default, net map creates shared memory region in the kernel by using standard MROC. It is actually contigual MROC but it doesn't matter. So the memory region is kind of anonymous. But using Xmem subsystem, now the user can give a bit more memory region on its virtual address space to the kernel to create net map shared buffers. So by creating file system on nonvolta memory and the M mapping file there and giving it to the net map. So you can create NVMM backed net map memory allocator or shared memory region. So these two net map extensions are part of the paste. So okay, so let's take a look at how the paste works end to end. So here we have user in the kernel space and in the kernel. You have TCPIP stack and NICs. And you also have shared memory region used by net map which contains several types of net map objects like packet buffers which are fixed size and statically allocated. You also have net map link. This is a set of pointers to buffers. So the important point is ring buffer doesn't contain buffer itself. It just contains buffer indexes. So here the ring is initialized with the point, initialized by pointers to packet buffer 20 to 27. Okay? So and user can access this shared memory region via this ring. Okay? So without copying any data. Okay. So imagine that so how it works. So application first issue system call, like pull system call. Okay? So the pull system call internally triggers NIC packet IO. And in this case suppose that application is trying to receive some packets. Okay? So the pull initiate NIC packet IO and have the TCPIP stack process receiving packets. Okay? So imagine that so we are now getting seven packets and imagine that TCPIP stack identifies these other, these are in order TCP segments. So in the end so the kernel or net map sets these seven in order TCP segments to application visible net map link. So it places the packet buffer 0 to 7, sorry 0 to 6 to the application visible net map link. So the ring now contains packet buffer 0 to 6 and 27 is not used yet. And to indicate the application, the data is ready from the car to tail. So kernel also advances tail pointer. Okay? So at this point the application recognized, so now the application returns from the pull system call and recognizes that okay from car to tail. So the packet buffer 0 to 6 are ready to process. Okay? So the application process, so these data may belong to different file descriptors. Okay? So the application processes data by traversing the ring from car to tail and after that application returns these buffers pointed by link slots to the kernel on the next pull system call. Okay? So this is how it works end to end. So by the way, so the application may want, may not want to return packet buffer to the kernel because imagine that you are building some key value store. Okay? So you want to keep data without returning buffer to the kernel. Okay? And also application don't want to copy data to store data. Okay? So in that case, before returning buffer to the kernel, so application can swap out packet buffer 1 to 6 and, sorry, 1 to 6 and instead you can fill some free packet buffer in this case 8, 9, 10 to the same slot. If application returns the slot with these buffer indexes, then the kernel recycles packet buffer 0, 8, 9, 3, 4, 5, 10. Okay? So packet buffer 1 to 3 were already swapped out. So the strong orange parts, so they are not recycled. So application can maintain and organize these packet buffers using some convenient various data structures. In this picture, application organizes these packet buffers using B plus 3. So you see that packet buffer 1 to 6. For example, for packet buffer 1, it's from packet buffer 1 offset 96 bytes, skipping some unnecessary TCP IP header and also for the length of 120 bytes. So application can describe the extent of the particular packet buffer by using, also using that any data structures. So now, if your application wants to persist data, so suppose that application wants to keep the data across reboot, in that case, using X-memex extension, so you can create file system on NVMM and create packet buffer on, create packet buffers or shared memory region on NVMM and we can do the same. Then if we organize all the data on NVMM and also storing B plus 3 on somewhere persistent or on NVMM or on disks, whatever, then the application creates persistent data structure which can be accessed without any data copy since NIC's DMA. In this way, because in this way, so NIC's DMA packets directly onto NVMM. So now, I think I hope you understood that it seems to be useful but now going to the probably most important part of this talk. So the hard application code looks like. So let's take some time to understand this. So as usual, we first open Netmap port by giving a name. So since we use Stackport, a kind of Netmap port, and we create Netmap ports, we open Netmap port with NM Open using this stack name, Stackport 0. So this indicates Stackport. So we get some descriptor for Netmap. Second, we also register some NIC to that Stackport. So in other words, it associates a NIC to that Stackport using existing IO control for Netmap. After that, after setting up some data path from the Netmap port and the NIC, we open the socket and bind the listen as usual. So let's imagine we got the socket S. So now we create the array of two file descriptors. One is for Netmap file descriptor and the other is for listening socket descriptor. And now we're going to infinite loop and the port system calls the monitor to file descriptor. So why is the Netmap one, the other is listening socket one. So let's check that listening socket first. If there is some new, if the port returns and if new request or new TCP connection is arriving, then the file descriptor is one. It should be port in. So let's just accept this file descriptor and register, associate this file descriptor to Netmap file descriptor using this IO control. So if the Netmap file descriptor is available, then the retrovastor link, retrovastor link slots by manipulating its data. So you can know to which file descriptor this data belongs to by taking file descriptor field of Netmap slot. And of course, you can get some actual battery address pointer to the data using normal Netmap, the translation macro. Also we introduce some offset field in the slot. But we're not extending Netmap slot at all. We are using some unnecessary fields. So I think that it looks simple enough and hope everybody understood. So what's going on internally? This is a little bit complex and may need some knowledge about the internal kernels. So the number describes some order of execution. So after calling the power system call, the kernel itchates over the nick link after making packets available on the main memory. So the kernel pick packet buffer or packet descriptor and wrap the data using nbuff and point this Netmap buffer from the nbuff and put the nbuff into IF input. Then the data goes up to the TCP IP implementation. And if it is in order TCP segment, you know that some of the SO, the app call is invoked. So actually in the accepting IO control, it's internally the set SO app call. So something is set to the socket, accepting socket. So the IO that is in the segment is marked in the step number three, it says our own stuff. And in this way, so step number two, it's a record, it creates a list of ready packets, list of packets which is ready to be consumed by application. So in the end, after processing all the packets against TCP IP stack, the kernel creates, itchates over that ready packet to be consumed by application and sets the Netmap buffer to application visible link. So in this way, we can batch system call, we can batch IO and we can avoid per socket system call as well. So here is the performance. So this is experiment with single CPU core and the left side is the case which we don't store data, so we adjust to the consuming data. So compared to the normal Linux, we can get say 50% performance improvement, also latency also gets reduced. So in case of storing data to NVMM, if we store data to NVMM using just memory copy, we get say about 30% improvement, but if we directly DM a data into NVMM, we can get 50% or even more improvement, which is orange bar over Linux. So I think this is pretty good. So here is the multiple CPU core scalability. So the axis is number of CPU cores and y axis is transaction per second, million transactions per second. So what we compared against is no-passed is just a case which doesn't store data at all, so just consuming data. So copy is storing data using memory copy and your copy is directly DM a into NVMM. So the point where the throughput becomes flat is reaching to 10 gigabit line rate. So in all the cases, it's not perfect, but it scales pretty well, right, so at least until 10 CPU cores. So I would say this multi-core scalability here is pretty good. So we also ported the paste to the Redis, Redis is a popular persistent key value store, and in this case, we directly DM a packets onto NVMM. And so we tried different right sides, which is 64 bytes and 1 kilobyte. We also changed the ratio of right request, also changed the distribution of data, which is distribution of keys, which is uniform and different 0.99. So you may familiar with the key value store, so many people use YCSP benchmark, which contains set of workloads, like read mostly or update heavy. So these two circles correspond to YCSP workloads. So the comparison to Linux and pasted head to head comparison, but we also measured the case which runs Redis on normal Linux, but configured not to pass this data at all. But even for that case, we can get much higher performance. So the paste supports Linux and FreeBSD, and actually, I initially implemented for Linux, but now porting to FreeBSD. So Linux, actually, we don't have to modify Linux core at all. We can do everything just within Netmap as a Netmap component. But for FreeBSD, unfortunately, we need a little bit of change in the current network stack. So this is a patch what I want to submit. So unless you strongly disagree right now, I'm going to submit it. And the problem is Linux has the paste that manipulates sockets within the kernel, so it associates the sockets into the Netmap object in the kernel. So when application calls close on the socket, the kernel side of the Netmap wants to know that. So in Linux, there's such a callback, but in FreeBSD, there is not. So I want to add this. So I know some relevant people here, so let's talk later. And here is the summary. So the using paste, you can integrate kernel TCP IP stack with Netmap API. Also supports emerging non-volatile main memory. And the status is now in progress with upstreaming to the Netmap with Giuseppe. And there is also some academic papers in case of you are interested in. And it's not really exciting, but all the experiments or some numbers shown in this talk were obtained using Linux, but it's not good. So I want to show it's also running with FreeBSD. So what is this? Okay. Can you see text? I think. I think you can see text. Wait a minute. Okay. So it's a FreeBSD. It's not Linux. So let's run some normal HTTP server. So this HTTP server is a pretty normal one. It's just the KQ-based program, which I showed in the beginning of the presentation. Let's run some normal server first. So this is the normal HTTP benchmark tool called WRK. So in this command line, we are running experiments for three seconds and 20 using 20 TCP connections and threads. It is continually creating HTTP posts with 1200 bytes. So this is VM. So it is just to show something running. You see that throughput is about 96, so nine kilo of the request per second is pretty slow. But okay, this is normal FreeBSD. So this is very slow because this is Linux. Sorry, this is a virtual machine. Okay, now let's use the paste. Let's see what happens. I hope it's done in the crash. Okay, it's ready. You see that it's quite fast, right? Given that the VM, so even the VM, we can see some about 40% performance improvement with much lower latency actually. So latency of 50% time was 1.74 milliseconds. Now, but it's a lower latency with higher throughput. That's pretty good. So that's it. I'm very happy because of the demonstration.
 All right.  So maybe can I start?  All right.  So nice to meet you.  My name is Michio Honda and today I'm going to present a paste which is a high performance  and the system networking framework using Netmap API.  And so this is the joint work with my awesome colleagues, Giuseppe Eleti-Airi, Lars Egert  and Doug Santhry.  If you have any questions or something later on, so please just drop me a message on Twitter  or email, whatever.  And the call using today is here, but I think it will disappear at some point after merging  it to original Netmap.  So first of all, so what is this talk about?  So this talk is about what are the problems with current network stack and socket API today  and how do we solve that problem.  And this talk is not about current network stack is slow.  Let's bypass everything.  I have awesome user space network stack.  This talk is not about that.  So if you expect something like this, I recommend you to go some other excellent sessions.  All right.  Never this.  The problem is kind of similar.  So the current socket API is not that fast.  So to get some intuition, so let's see some benchmark with some simple request response  workload.  So in this experiment, we send some, the 1400 bytes HTTP post and replying back of HTTP  OK, which is some 64 byte message.  This is experimented down with Linux, but we can expect quite similar with free BST.  And this uses just HTTP and TCP.  So the typically the server calls doing this kind of processing looks like this.  You fetch the ready file descriptors.  So new messages are arriving.  And you iterate over ready file descriptor one by one by issuing pair of read and write  system call.  So the shared result, so the left is throughput and the right is latency for the different  number of concurrent TCP connections.  So you see the throughput.  It is only this user single core, but still it can achieve only 2.8 gigabits.  This is not that great because today's modern network speed is 10 gigabit or 40 or even 100  gigabit per second.  So latency as well.  For the single TCP connection, it's actually pretty good.  So we can see 23 microseconds for the HTTP run type of data.  But when we have 100 concurrent TCP connections, the latency goes up to 400 microseconds.  And you see, no, that's great.  So this result shows that for modern network speed, socket API is not that good.  So I think you got some intuition now.  So the second problem is current network stack cannot support emerging non-volatile memory  very well.  That's the second motivation in the problem.  So I think some of you may never heard about non-volatile memory.  So let's quickly review it.  So non-volatile memory is a literary memory whose contents survive reboot.  As a result, non-volatile memory offers very fast, much faster than disk SSDs.  And byte addressable not based on block, access to persistent data.  So the visual looks like this.  We have CPU cache at the top and underneath.  You have main memory.  And access there is very fast.  But the contents go away when you are machine reboot.  And to store data persistently, you have disk SSD at the bottom.  And it has much higher capacity.  And the contents survive over reboot.  But access there is very slow.  And you have to issue the system calls, like read, write, or some xing.  But having non-volatile memory here, now you can access persistent data just using load  of store and privilege instruction and access there is almost as fast as DRAM.  Maybe slightly slower, but still two, three orders of magnitude faster than the hard drives  and SSDs.  So once NVMN put into the system, then the entire architecture looks like this.  So you can create a file system on non-volatile memory, something like you create file system  on RAM disk.  But once the file there is N mapped into user address space, you can directly load and store  the data there without using any buffer cache.  That is the main point of non-volatile memory.  So let's go back to the problem definition and let's see how the end-to-end system is  affected by end-to-end transaction speed is affected by non-volatile memory.  So just using previous code, but we have now some of the processing that store data from  the socket buffer to non-volatile memory.  So we have memory copied to NVMN and also flashing data from CPU cache to the actual  physical media.  So here is the result.  So this compares the case which doesn't store data on NVMN at all and the case which store  data there.  So it's surprising that throughput gets almost half and latency increase almost doubles.  So yes.  Oh yes, that's a good point.  Actually we have the same experiment but actually it related to even worse performance.  So because memory copy is implemented using optimized instructions like SSC, but we don't  like to move data using the normal instruction.  So I can tell you the more result by showing the graph later.  It's a good point.  But it's the same.  So the point here is, okay, so even for non-volatile memory, don't copy data if you really want  high performance.  So the summary of the previous two problems is let's avoid per socket system calls and  IO requests.  Okay, so that's the first summary or first requirement.  And the second requirement is let's avoid data copy at all even for non-volatile memory.  This decreases throughput and increases latency.  So before the going to solution, so let's get access to write.  And the question is how do we address these problems while preserving benefits offered  by current network stack and socket API today?  So that's the main question.  So here's the page what we propose.  And the page is framework that achieves scalable, flexible end system networking.  And page offers true zero copy since the NICDMA even for non-volatile memory.  So that's the first thing.  The second point is you can batch system call and IO operations across multiple sockets.  And the third point is you can use existing excellent kernel TCP, IP, CTP, UDP, IPv6 stack  which has a lot of extensions not just doesn't implement the original RFC but implement almost  all the RFCs.  Also, but API itself on the other hand is pretty protocol independent.  So it's using a net map API so it's not bound to specific transport protocols.  So we can for both blocking and busy polling depending on what you want.  For example, if you want some low CPU utilization or some efficiency, you could use the blocking.  If you want ultimate low latency even at idle time period, then you can use the busy polling.  You can also benefit from protection so your application can crash but the rest of the  system is not affected.  Actually the last four features are exactly what we benefit from socket API today.  So going to a little bit more details, so the page consists of the two net map extensions.  So net map is a packet Iof framework to efficiently move data between the kernel and the user  space and that is already in previous D.  The first extension is stack port.  So net maps depending on type of the interface, net maps provide multiple port types where  application access.  So the stack port is some of the net map API but which the data goes to kernel TCP, IP,  stack.  So the same level of abstraction with the pipe and the valid port if you're familiar  with net map already.  So the other extension is Xmem subsystem.  So by default, net map creates shared memory region in the kernel by using standard MROC.  It is actually contigual MROC but it doesn't matter.  So the memory region is kind of anonymous.  But using Xmem subsystem, now the user can give a bit more memory region on its virtual  address space to the kernel to create net map shared buffers.  So by creating file system on nonvolta memory and the M mapping file there and giving it  to the net map.  So you can create NVMM backed net map memory allocator or shared memory region.  So these two net map extensions are part of the paste.  So okay, so let's take a look at how the paste works end to end.  So here we have user in the kernel space and in the kernel.  You have TCPIP stack and NICs.  And you also have shared memory region used by net map which contains several types of  net map objects like packet buffers which are fixed size and statically allocated.  You also have net map link.  This is a set of pointers to buffers.  So the important point is ring buffer doesn't contain buffer itself.  It just contains buffer indexes.  So here the ring is initialized with the point, initialized by pointers to packet buffer 20  to 27.  Okay?  So and user can access this shared memory region via this ring.  Okay?  So without copying any data.  Okay.  So imagine that so how it works.  So application first issue system call, like pull system call.  Okay?  So the pull system call internally triggers NIC packet IO.  And in this case suppose that application is trying to receive some packets.  Okay?  So the pull initiate NIC packet IO and have the TCPIP stack process receiving packets.  Okay?  So imagine that so we are now getting seven packets and imagine that TCPIP stack identifies  these other, these are in order TCP segments.  So in the end so the kernel or net map sets these seven in order TCP segments to application  visible net map link.  So it places the packet buffer 0 to 7, sorry 0 to 6 to the application visible net map link.  So the ring now contains packet buffer 0 to 6 and 27 is not used yet.  And to indicate the application, the data is ready from the car to tail.  So kernel also advances tail pointer.  Okay?  So at this point the application recognized, so now the application returns from the pull  system call and recognizes that okay from car to tail.  So the packet buffer 0 to 6 are ready to process.  Okay?  So the application process, so these data may belong to different file descriptors.  Okay?  So the application processes data by traversing the ring from car to tail and after that application  returns these buffers pointed by link slots to the kernel on the next pull system call.  Okay?  So this is how it works end to end.  So by the way, so the application may want, may not want to return packet buffer to the  kernel because imagine that you are building some key value store.  Okay?  So you want to keep data without returning buffer to the kernel.  Okay?  And also application don't want to copy data to store data.  Okay?  So in that case, before returning buffer to the kernel, so application can swap out packet  buffer 1 to 6 and, sorry, 1 to 6 and instead you can fill some free packet buffer in this  case 8, 9, 10 to the same slot.  If application returns the slot with these buffer indexes, then the kernel recycles  packet buffer 0, 8, 9, 3, 4, 5, 10.  Okay?  So packet buffer 1 to 3 were already swapped out.  So the strong orange parts, so they are not recycled.  So application can maintain and organize these packet buffers using some convenient various  data structures.  In this picture, application organizes these packet buffers using B plus 3.  So you see that packet buffer 1 to 6.  For example, for packet buffer 1, it's from packet buffer 1 offset 96 bytes, skipping  some unnecessary TCP IP header and also for the length of 120 bytes.  So application can describe the extent of the particular packet buffer by using, also  using that any data structures.  So now, if your application wants to persist data, so suppose that application wants to  keep the data across reboot, in that case, using X-memex extension, so you can create  file system on NVMM and create packet buffer on, create packet buffers or shared memory  region on NVMM and we can do the same.  Then if we organize all the data on NVMM and also storing B plus 3 on somewhere persistent  or on NVMM or on disks, whatever, then the application creates persistent data structure  which can be accessed without any data copy since NIC's DMA.  In this way, because in this way, so NIC's DMA packets directly onto NVMM.  So now, I think I hope you understood that it seems to be useful but now going to the  probably most important part of this talk.  So the hard application code looks like.  So let's take some time to understand this.  So as usual, we first open Netmap port by giving a name.  So since we use Stackport, a kind of Netmap port, and we create Netmap ports, we open  Netmap port with NM Open using this stack name, Stackport 0.  So this indicates Stackport.  So we get some descriptor for Netmap.  Second, we also register some NIC to that Stackport.  So in other words, it associates a NIC to that Stackport using existing IO control for Netmap.  After that, after setting up some data path from the Netmap port and the NIC, we open  the socket and bind the listen as usual.  So let's imagine we got the socket S. So now we create the array of two file descriptors.  One is for Netmap file descriptor and the other is for listening socket descriptor.  And now we're going to infinite loop and the port system calls the monitor to file descriptor.  So why is the Netmap one, the other is listening socket one.  So let's check that listening socket first.  If there is some new, if the port returns and if new request or new TCP connection is  arriving, then the file descriptor is one.  It should be port in.  So let's just accept this file descriptor and register, associate this file descriptor  to Netmap file descriptor using this IO control.  So if the Netmap file descriptor is available, then the retrovastor link, retrovastor link  slots by manipulating its data.  So you can know to which file descriptor this data belongs to by taking file descriptor  field of Netmap slot.  And of course, you can get some actual battery address pointer to the data using normal Netmap,  the translation macro.  Also we introduce some offset field in the slot.  But we're not extending Netmap slot at all.  We are using some unnecessary fields.  So I think that it looks simple enough and hope everybody understood.  So what's going on internally?  This is a little bit complex and may need some knowledge about the internal kernels.  So the number describes some order of execution.  So after calling the power system call, the kernel itchates over the nick link after making  packets available on the main memory.  So the kernel pick packet buffer or packet descriptor and wrap the data using nbuff and  point this Netmap buffer from the nbuff and put the nbuff into IF input.  Then the data goes up to the TCP IP implementation.  And if it is in order TCP segment, you know that some of the SO, the app call is invoked.  So actually in the accepting IO control, it's internally the set SO app call.  So something is set to the socket, accepting socket.  So the IO that is in the segment is marked in the step number three, it says our own  stuff.  And in this way, so step number two, it's a record, it creates a list of ready packets,  list of packets which is ready to be consumed by application.  So in the end, after processing all the packets against TCP IP stack, the kernel creates,  itchates over that ready packet to be consumed by application and sets the Netmap buffer  to application visible link.  So in this way, we can batch system call, we can batch IO and we can avoid per socket  system call as well.  So here is the performance.  So this is experiment with single CPU core and the left side is the case which we don't  store data, so we adjust to the consuming data.  So compared to the normal Linux, we can get say 50% performance improvement, also latency  also gets reduced.  So in case of storing data to NVMM, if we store data to NVMM using just memory copy,  we get say about 30% improvement, but if we directly DM a data into NVMM, we can get  50% or even more improvement, which is orange bar over Linux.  So I think this is pretty good.  So here is the multiple CPU core scalability.  So the axis is number of CPU cores and y axis is transaction per second, million transactions  per second.  So what we compared against is no-passed is just a case which doesn't store data at all,  so just consuming data.  So copy is storing data using memory copy and your copy is directly DM a into NVMM.  So the point where the throughput becomes flat is reaching to 10 gigabit line rate.  So in all the cases, it's not perfect, but it scales pretty well, right, so at least  until 10 CPU cores.  So I would say this multi-core scalability here is pretty good.  So we also ported the paste to the Redis, Redis is a popular persistent key value store,  and in this case, we directly DM a packets onto NVMM.  And so we tried different right sides, which is 64 bytes and 1 kilobyte.  We also changed the ratio of right request, also changed the distribution of data, which  is distribution of keys, which is uniform and different 0.99.  So you may familiar with the key value store, so many people use YCSP benchmark, which contains  set of workloads, like read mostly or update heavy.  So these two circles correspond to YCSP workloads.  So the comparison to Linux and pasted head to head comparison, but we also measured the  case which runs Redis on normal Linux, but configured not to pass this data at all.  But even for that case, we can get much higher performance.  So the paste supports Linux and FreeBSD, and actually, I initially implemented for Linux,  but now porting to FreeBSD.  So Linux, actually, we don't have to modify Linux core at all.  We can do everything just within Netmap as a Netmap component.  But for FreeBSD, unfortunately, we need a little bit of change in the current network  stack.  So this is a patch what I want to submit.  So unless you strongly disagree right now, I'm going to submit it.  And the problem is Linux has the paste that manipulates sockets within the kernel, so  it associates the sockets into the Netmap object in the kernel.  So when application calls close on the socket, the kernel side of the Netmap wants to know  that.  So in Linux, there's such a callback, but in FreeBSD, there is not.  So I want to add this.  So I know some relevant people here, so let's talk later.  And here is the summary.  So the using paste, you can integrate kernel TCP IP stack with Netmap API.  Also supports emerging non-volatile main memory.  And the status is now in progress with upstreaming to the Netmap with Giuseppe.  And there is also some academic papers in case of you are interested in.  And it's not really exciting, but all the experiments or some numbers shown in this  talk were obtained using Linux, but it's not good.  So I want to show it's also running with FreeBSD.  So what is this?  Okay.  Can you see text?  I think.  I think you can see text.  Wait a minute.  Okay.  So it's a FreeBSD.  It's not Linux.  So let's run some normal HTTP server.  So this HTTP server is a pretty normal one.  It's just the KQ-based program, which I showed in the beginning of the presentation.  Let's run some normal server first.  So this is the normal HTTP benchmark tool called WRK.  So in this command line, we are running experiments for three seconds and 20 using 20 TCP connections  and threads.  It is continually creating HTTP posts with 1200 bytes.  So this is VM.  So it is just to show something running.  You see that throughput is about 96, so nine kilo of the request per second is pretty slow.  But okay, this is normal FreeBSD.  So this is very slow because this is Linux.  Sorry, this is a virtual machine.  Okay, now let's use the paste.  Let's see what happens.  I hope it's done in the crash.  Okay, it's ready.  You see that it's quite fast, right?  Given that the VM, so even the VM, we can see some about 40% performance improvement  with much lower latency actually.  So latency of 50% time was 1.74 milliseconds.  Now, but it's a lower latency with higher throughput.  That's pretty good.  So that's it.  I'm very happy because of the demonstration.
Michio_Honda__PASTE__Fast_End_System_Networking_with_netmap_-_BSDCan_2018_y_F-8HijfOA_0_119.png Michio_Honda__PASTE__Fast_End_System_Networking_with_netmap_-_BSDCan_2018_y_F-8HijfOA_120_7487.png Michio_Honda__PASTE__Fast_End_System_Networking_with_netmap_-_BSDCan_2018_y_F-8HijfOA_7488_8255.png Michio_Honda__PASTE__Fast_End_System_Networking_with_netmap_-_BSDCan_2018_y_F-8HijfOA_8256_10523.png Michio_Honda__PASTE__Fast_End_System_Networking_with_netmap_-_BSDCan_2018_y_F-8HijfOA_10524_11543.png Michio_Honda__PASTE__Fast_End_System_Networking_with_netmap_-_BSDCan_2018_y_F-8HijfOA_11544_11615.png Michio_Honda__PASTE__Fast_End_System_Networking_with_netmap_-_BSDCan_2018_y_F-8HijfOA_11616_13907.png Michio_Honda__PASTE__Fast_End_System_Networking_with_netmap_-_BSDCan_2018_y_F-8HijfOA_13908_20735.png Michio_Honda__PASTE__Fast_End_System_Networking_with_netmap_-_BSDCan_2018_y_F-8HijfOA_20736_35099.png Michio_Honda__PASTE__Fast_End_System_Networking_with_netmap_-_BSDCan_2018_y_F-8HijfOA_35100_38567.png Michio_Honda__PASTE__Fast_End_System_Networking_with_netmap_-_BSDCan_2018_y_F-8HijfOA_38568_39491.png Michio_Honda__PASTE__Fast_End_System_Networking_with_netmap_-_BSDCan_2018_y_F-8HijfOA_39492_39647.png Michio_Honda__PASTE__Fast_End_System_Networking_with_netmap_-_BSDCan_2018_y_F-8HijfOA_39648_39719.png Michio_Honda__PASTE__Fast_End_System_Networking_with_netmap_-_BSDCan_2018_y_F-8HijfOA_39720_39767.png Michio_Honda__PASTE__Fast_End_System_Networking_with_netmap_-_BSDCan_2018_y_F-8HijfOA_39768_39839.png Michio_Honda__PASTE__Fast_End_System_Networking_with_netmap_-_BSDCan_2018_y_F-8HijfOA_39840_40463.png Michio_Honda__PASTE__Fast_End_System_Networking_with_netmap_-_BSDCan_2018_y_F-8HijfOA_40464_40535.png Michio_Honda__PASTE__Fast_End_System_Networking_with_netmap_-_BSDCan_2018_y_F-8HijfOA_40536_40631.png Michio_Honda__PASTE__Fast_End_System_Networking_with_netmap_-_BSDCan_2018_y_F-8HijfOA_40632_40727.png Michio_Honda__PASTE__Fast_End_System_Networking_with_netmap_-_BSDCan_2018_y_F-8HijfOA_40728_40847.png Michio_Honda__PASTE__Fast_End_System_Networking_with_netmap_-_BSDCan_2018_y_F-8HijfOA_40848_41111.png Michio_Honda__PASTE__Fast_End_System_Networking_with_netmap_-_BSDCan_2018_y_F-8HijfOA_41112_41339.png Michio_Honda__PASTE__Fast_End_System_Networking_with_netmap_-_BSDCan_2018_y_F-8HijfOA_41340_41399.png Michio_Honda__PASTE__Fast_End_System_Networking_with_netmap_-_BSDCan_2018_y_F-8HijfOA_41400_41651.png Michio_Honda__PASTE__Fast_End_System_Networking_with_netmap_-_BSDCan_2018_y_F-8HijfOA_41652_42191.png Michio_Honda__PASTE__Fast_End_System_Networking_with_netmap_-_BSDCan_2018_y_F-8HijfOA_42192_42323.png Michio_Honda__PASTE__Fast_End_System_Networking_with_netmap_-_BSDCan_2018_y_F-8HijfOA_42324_42431.png Michio_Honda__PASTE__Fast_End_System_Networking_with_netmap_-_BSDCan_2018_y_F-8HijfOA_42432_42875.png Michio_Honda__PASTE__Fast_End_System_Networking_with_netmap_-_BSDCan_2018_y_F-8HijfOA_42876_43043.png Michio_Honda__PASTE__Fast_End_System_Networking_with_netmap_-_BSDCan_2018_y_F-8HijfOA_43044_43139.png Michio_Honda__PASTE__Fast_End_System_Networking_with_netmap_-_BSDCan_2018_y_F-8HijfOA_43140_43259.png Michio_Honda__PASTE__Fast_End_System_Networking_with_netmap_-_BSDCan_2018_y_F-8HijfOA_43260_43326.png
All right. So maybe can I start? All right. So nice to meet you. My name is Michio Honda and today I'm going to present a paste which is a high performance and the system networking framework using Netmap API. And so this is the joint work with my awesome colleagues, Giuseppe Eleti-Airi, Lars Egert and Doug Santhry. If you have any questions or something later on, so please just drop me a message on Twitter or email, whatever. And the call using today is here, but I think it will disappear at some point after merging it to original Netmap. So first of all, so what is this talk about? So this talk is about what are the problems with current network stack and socket API today and how do we solve that problem. And this talk is not about current network stack is slow. Let's bypass everything. I have awesome user space network stack. This talk is not about that. So if you expect something like this, I recommend you to go some other excellent sessions. All right. Never this. The problem is kind of similar. So the current socket API is not that fast. So to get some intuition, so let's see some benchmark with some simple request response workload. So in this experiment, we send some, the 1400 bytes HTTP post and replying back of HTTP OK, which is some 64 byte message. This is experimented down with Linux, but we can expect quite similar with free BST. And this uses just HTTP and TCP. So the typically the server calls doing this kind of processing looks like this. You fetch the ready file descriptors. So new messages are arriving. And you iterate over ready file descriptor one by one by issuing pair of read and write system call. So the shared result, so the left is throughput and the right is latency for the different number of concurrent TCP connections. So you see the throughput. It is only this user single core, but still it can achieve only 2.8 gigabits. This is not that great because today's modern network speed is 10 gigabit or 40 or even 100 gigabit per second. So latency as well. For the single TCP connection, it's actually pretty good. So we can see 23 microseconds for the HTTP run type of data. But when we have 100 concurrent TCP connections, the latency goes up to 400 microseconds. And you see, no, that's great. So this result shows that for modern network speed, socket API is not that good. So I think you got some intuition now. So the second problem is current network stack cannot support emerging non-volatile memory very well. That's the second motivation in the problem. So I think some of you may never heard about non-volatile memory. So let's quickly review it. So non-volatile memory is a literary memory whose contents survive reboot. As a result, non-volatile memory offers very fast, much faster than disk SSDs. And byte addressable not based on block, access to persistent data. So the visual looks like this. We have CPU cache at the top and underneath. You have main memory. And access there is very fast. But the contents go away when you are machine reboot. And to store data persistently, you have disk SSD at the bottom. And it has much higher capacity. And the contents survive over reboot. But access there is very slow. And you have to issue the system calls, like read, write, or some xing. But having non-volatile memory here, now you can access persistent data just using load of store and privilege instruction and access there is almost as fast as DRAM. Maybe slightly slower, but still two, three orders of magnitude faster than the hard drives and SSDs. So once NVMN put into the system, then the entire architecture looks like this. So you can create a file system on non-volatile memory, something like you create file system on RAM disk. But once the file there is N mapped into user address space, you can directly load and store the data there without using any buffer cache. That is the main point of non-volatile memory. So let's go back to the problem definition and let's see how the end-to-end system is affected by end-to-end transaction speed is affected by non-volatile memory. So just using previous code, but we have now some of the processing that store data from the socket buffer to non-volatile memory. So we have memory copied to NVMN and also flashing data from CPU cache to the actual physical media. So here is the result. So this compares the case which doesn't store data on NVMN at all and the case which store data there. So it's surprising that throughput gets almost half and latency increase almost doubles. So yes. Oh yes, that's a good point. Actually we have the same experiment but actually it related to even worse performance. So because memory copy is implemented using optimized instructions like SSC, but we don't like to move data using the normal instruction. So I can tell you the more result by showing the graph later. It's a good point. But it's the same. So the point here is, okay, so even for non-volatile memory, don't copy data if you really want high performance. So the summary of the previous two problems is let's avoid per socket system calls and IO requests. Okay, so that's the first summary or first requirement. And the second requirement is let's avoid data copy at all even for non-volatile memory. This decreases throughput and increases latency. So before the going to solution, so let's get access to write. And the question is how do we address these problems while preserving benefits offered by current network stack and socket API today? So that's the main question. So here's the page what we propose. And the page is framework that achieves scalable, flexible end system networking. And page offers true zero copy since the NICDMA even for non-volatile memory. So that's the first thing. The second point is you can batch system call and IO operations across multiple sockets. And the third point is you can use existing excellent kernel TCP, IP, CTP, UDP, IPv6 stack which has a lot of extensions not just doesn't implement the original RFC but implement almost all the RFCs. Also, but API itself on the other hand is pretty protocol independent. So it's using a net map API so it's not bound to specific transport protocols. So we can for both blocking and busy polling depending on what you want. For example, if you want some low CPU utilization or some efficiency, you could use the blocking. If you want ultimate low latency even at idle time period, then you can use the busy polling. You can also benefit from protection so your application can crash but the rest of the system is not affected. Actually the last four features are exactly what we benefit from socket API today. So going to a little bit more details, so the page consists of the two net map extensions. So net map is a packet Iof framework to efficiently move data between the kernel and the user space and that is already in previous D. The first extension is stack port. So net maps depending on type of the interface, net maps provide multiple port types where application access. So the stack port is some of the net map API but which the data goes to kernel TCP, IP, stack. So the same level of abstraction with the pipe and the valid port if you're familiar with net map already. So the other extension is Xmem subsystem. So by default, net map creates shared memory region in the kernel by using standard MROC. It is actually contigual MROC but it doesn't matter. So the memory region is kind of anonymous. But using Xmem subsystem, now the user can give a bit more memory region on its virtual address space to the kernel to create net map shared buffers. So by creating file system on nonvolta memory and the M mapping file there and giving it to the net map. So you can create NVMM backed net map memory allocator or shared memory region. So these two net map extensions are part of the paste. So okay, so let's take a look at how the paste works end to end. So here we have user in the kernel space and in the kernel. You have TCPIP stack and NICs. And you also have shared memory region used by net map which contains several types of net map objects like packet buffers which are fixed size and statically allocated. You also have net map link. This is a set of pointers to buffers. So the important point is ring buffer doesn't contain buffer itself. It just contains buffer indexes. So here the ring is initialized with the point, initialized by pointers to packet buffer 20 to 27. Okay? So and user can access this shared memory region via this ring. Okay? So without copying any data. Okay. So imagine that so how it works. So application first issue system call, like pull system call. Okay? So the pull system call internally triggers NIC packet IO. And in this case suppose that application is trying to receive some packets. Okay? So the pull initiate NIC packet IO and have the TCPIP stack process receiving packets. Okay? So imagine that so we are now getting seven packets and imagine that TCPIP stack identifies these other, these are in order TCP segments. So in the end so the kernel or net map sets these seven in order TCP segments to application visible net map link. So it places the packet buffer 0 to 7, sorry 0 to 6 to the application visible net map link. So the ring now contains packet buffer 0 to 6 and 27 is not used yet. And to indicate the application, the data is ready from the car to tail. So kernel also advances tail pointer. Okay? So at this point the application recognized, so now the application returns from the pull system call and recognizes that okay from car to tail. So the packet buffer 0 to 6 are ready to process. Okay? So the application process, so these data may belong to different file descriptors. Okay? So the application processes data by traversing the ring from car to tail and after that application returns these buffers pointed by link slots to the kernel on the next pull system call. Okay? So this is how it works end to end. So by the way, so the application may want, may not want to return packet buffer to the kernel because imagine that you are building some key value store. Okay? So you want to keep data without returning buffer to the kernel. Okay? And also application don't want to copy data to store data. Okay? So in that case, before returning buffer to the kernel, so application can swap out packet buffer 1 to 6 and, sorry, 1 to 6 and instead you can fill some free packet buffer in this case 8, 9, 10 to the same slot. If application returns the slot with these buffer indexes, then the kernel recycles packet buffer 0, 8, 9, 3, 4, 5, 10. Okay? So packet buffer 1 to 3 were already swapped out. So the strong orange parts, so they are not recycled. So application can maintain and organize these packet buffers using some convenient various data structures. In this picture, application organizes these packet buffers using B plus 3. So you see that packet buffer 1 to 6. For example, for packet buffer 1, it's from packet buffer 1 offset 96 bytes, skipping some unnecessary TCP IP header and also for the length of 120 bytes. So application can describe the extent of the particular packet buffer by using, also using that any data structures. So now, if your application wants to persist data, so suppose that application wants to keep the data across reboot, in that case, using X-memex extension, so you can create file system on NVMM and create packet buffer on, create packet buffers or shared memory region on NVMM and we can do the same. Then if we organize all the data on NVMM and also storing B plus 3 on somewhere persistent or on NVMM or on disks, whatever, then the application creates persistent data structure which can be accessed without any data copy since NIC's DMA. In this way, because in this way, so NIC's DMA packets directly onto NVMM. So now, I think I hope you understood that it seems to be useful but now going to the probably most important part of this talk. So the hard application code looks like. So let's take some time to understand this. So as usual, we first open Netmap port by giving a name. So since we use Stackport, a kind of Netmap port, and we create Netmap ports, we open Netmap port with NM Open using this stack name, Stackport 0. So this indicates Stackport. So we get some descriptor for Netmap. Second, we also register some NIC to that Stackport. So in other words, it associates a NIC to that Stackport using existing IO control for Netmap. After that, after setting up some data path from the Netmap port and the NIC, we open the socket and bind the listen as usual. So let's imagine we got the socket S. So now we create the array of two file descriptors. One is for Netmap file descriptor and the other is for listening socket descriptor. And now we're going to infinite loop and the port system calls the monitor to file descriptor. So why is the Netmap one, the other is listening socket one. So let's check that listening socket first. If there is some new, if the port returns and if new request or new TCP connection is arriving, then the file descriptor is one. It should be port in. So let's just accept this file descriptor and register, associate this file descriptor to Netmap file descriptor using this IO control. So if the Netmap file descriptor is available, then the retrovastor link, retrovastor link slots by manipulating its data. So you can know to which file descriptor this data belongs to by taking file descriptor field of Netmap slot. And of course, you can get some actual battery address pointer to the data using normal Netmap, the translation macro. Also we introduce some offset field in the slot. But we're not extending Netmap slot at all. We are using some unnecessary fields. So I think that it looks simple enough and hope everybody understood. So what's going on internally? This is a little bit complex and may need some knowledge about the internal kernels. So the number describes some order of execution. So after calling the power system call, the kernel itchates over the nick link after making packets available on the main memory. So the kernel pick packet buffer or packet descriptor and wrap the data using nbuff and point this Netmap buffer from the nbuff and put the nbuff into IF input. Then the data goes up to the TCP IP implementation. And if it is in order TCP segment, you know that some of the SO, the app call is invoked. So actually in the accepting IO control, it's internally the set SO app call. So something is set to the socket, accepting socket. So the IO that is in the segment is marked in the step number three, it says our own stuff. And in this way, so step number two, it's a record, it creates a list of ready packets, list of packets which is ready to be consumed by application. So in the end, after processing all the packets against TCP IP stack, the kernel creates, itchates over that ready packet to be consumed by application and sets the Netmap buffer to application visible link. So in this way, we can batch system call, we can batch IO and we can avoid per socket system call as well. So here is the performance. So this is experiment with single CPU core and the left side is the case which we don't store data, so we adjust to the consuming data. So compared to the normal Linux, we can get say 50% performance improvement, also latency also gets reduced. So in case of storing data to NVMM, if we store data to NVMM using just memory copy, we get say about 30% improvement, but if we directly DM a data into NVMM, we can get 50% or even more improvement, which is orange bar over Linux. So I think this is pretty good. So here is the multiple CPU core scalability. So the axis is number of CPU cores and y axis is transaction per second, million transactions per second. So what we compared against is no-passed is just a case which doesn't store data at all, so just consuming data. So copy is storing data using memory copy and your copy is directly DM a into NVMM. So the point where the throughput becomes flat is reaching to 10 gigabit line rate. So in all the cases, it's not perfect, but it scales pretty well, right, so at least until 10 CPU cores. So I would say this multi-core scalability here is pretty good. So we also ported the paste to the Redis, Redis is a popular persistent key value store, and in this case, we directly DM a packets onto NVMM. And so we tried different right sides, which is 64 bytes and 1 kilobyte. We also changed the ratio of right request, also changed the distribution of data, which is distribution of keys, which is uniform and different 0.99. So you may familiar with the key value store, so many people use YCSP benchmark, which contains set of workloads, like read mostly or update heavy. So these two circles correspond to YCSP workloads. So the comparison to Linux and pasted head to head comparison, but we also measured the case which runs Redis on normal Linux, but configured not to pass this data at all. But even for that case, we can get much higher performance. So the paste supports Linux and FreeBSD, and actually, I initially implemented for Linux, but now porting to FreeBSD. So Linux, actually, we don't have to modify Linux core at all. We can do everything just within Netmap as a Netmap component. But for FreeBSD, unfortunately, we need a little bit of change in the current network stack. So this is a patch what I want to submit. So unless you strongly disagree right now, I'm going to submit it. And the problem is Linux has the paste that manipulates sockets within the kernel, so it associates the sockets into the Netmap object in the kernel. So when application calls close on the socket, the kernel side of the Netmap wants to know that. So in Linux, there's such a callback, but in FreeBSD, there is not. So I want to add this. So I know some relevant people here, so let's talk later. And here is the summary. So the using paste, you can integrate kernel TCP IP stack with Netmap API. Also supports emerging non-volatile main memory. And the status is now in progress with upstreaming to the Netmap with Giuseppe. And there is also some academic papers in case of you are interested in. And it's not really exciting, but all the experiments or some numbers shown in this talk were obtained using Linux, but it's not good. So I want to show it's also running with FreeBSD. So what is this? Okay. Can you see text? I think. I think you can see text. Wait a minute. Okay. So it's a FreeBSD. It's not Linux. So let's run some normal HTTP server. So this HTTP server is a pretty normal one. It's just the KQ-based program, which I showed in the beginning of the presentation. Let's run some normal server first. So this is the normal HTTP benchmark tool called WRK. So in this command line, we are running experiments for three seconds and 20 using 20 TCP connections and threads. It is continually creating HTTP posts with 1200 bytes. So this is VM. So it is just to show something running. You see that throughput is about 96, so nine kilo of the request per second is pretty slow. But okay, this is normal FreeBSD. So this is very slow because this is Linux. Sorry, this is a virtual machine. Okay, now let's use the paste. Let's see what happens. I hope it's done in the crash. Okay, it's ready. You see that it's quite fast, right? Given that the VM, so even the VM, we can see some about 40% performance improvement with much lower latency actually. So latency of 50% time was 1.74 milliseconds. Now, but it's a lower latency with higher throughput. That's pretty good. So that's it. I'm very happy because of the demonstration.
high performance and the system networking stack and socket API today and how do we solve by issuing pair of read and write system call. So concurrent TCP connections. So you see the speed is 10 gigabit or 40 or even 100 gigabit per second. So latency as well. For the single TCP you can access persistent data just using load of but still two, three orders of magnitude faster into the system, then the entire architecture looks like this. So you can create a file system end-to-end system is affected by end-to-end now some of the processing that store data from the socket buffer to non-volatile memory. So we how do we address these problems while preserving system networking. And page offers true zero copy multiple sockets. And the third point is you can doesn't implement the original RFC but implement extensions. So net map is a packet Iof framework net map already. So the other extension is Xmem giving it to the net map. So you can create NVMM NIC packet IO and have the TCPIP stack process yet. And to indicate the application, the data is this is how it works end to end. So by the way, store. Okay? So you want to keep data without can maintain and organize these packet buffers using some convenient various data structures. In describe the extent of the particular packet persist data, so suppose that application wants data copy since NIC's DMA. In this way, because to infinite loop and the port system calls the translation macro. Also we introduce some offset slot at all. We are using some unnecessary which is ready to be consumed by application. So get 50% or even more improvement, which is orange into NVMM. So the point where the throughput Linux and FreeBSD, and actually, I initially implemented for Linux, but now porting to change in the current network stack. So this is a the kernel. So when application calls close on to know that. So in Linux, there's such a stack with Netmap API. Also supports emerging performance improvement with much lower latency