From Silicon to Compiler
Zonenberg, Andrew
Programmable logic devices have historically been locked up behind proprietary vendor toolchains and undocumented firmware formats, preventing the creation of a third-party compiler or decompiler. While the vendor typically prohibits reverse engineering of their software in the license agreement, no such ban applies to the silicon. Given the choice between REing gigabytes of spaghetti code and looking at clean, regular die layout, the choice is clear. This talk describes my reverse engineering of the Xilinx XC2C32A, a 180nm 32-macrocell CPLD, at the silicon level and my progress toward a fully open-source toolchain (compiler, decompiler, and floorplanner) for the device. A live demonstration of firmware generated by my tools running on actual hardware is included.
 You guys hear me now? Okay, sounds good. So my talk is called from Silicon to compiler and it's pretty much that. So we're going to start off with what we're doing, why we're doing this, a little bit of architectural background on programmable logic for those of you who have not done work on programmable logic before. Then jump to a block diagram of the device. We'll start with the high level overview of the silicon and then we'll drill down into more interesting stuff down to transistor level and gate level circuit analysis. At the end I do have a live demo of firmware produced by my tools running on live silicon. Sadly there are no cute pictures in this presentation, sorry. So a little background about me, I pretty much like to build and break everything. I will do web stuff if they pay me enough. I mostly like to live down in low level, a bit of ring zero, firmware, low level board design, RTL design and now getting down into transistor level. I just finished up my PhD a few weeks ago. During that time I designed and created what I believe is the first ever college class on semiconductor reverse engineering. I tried quite hard to find another one that I could borrow notes and slides from as far as I can tell none existed. And I'm also obviously a significant contributor to Silicon Prone. So if you see all the guys walking around with decap chips in their shirts, say hi. I've only been with IO active since January. A lot of this work was actually done before I joined the company but they've supported me continuing the work. So I acknowledge them for that obviously. So as far as what we're actually trying to do and why, program logic is really everywhere these days. You may not know it but a lot of especially high end networking AV, all sorts of stuff has program logic in it because A6 these days the tape out costs are so high especially for leading edge process nodes it usually is not actually cost effective to make custom silicon unless you're making a huge number of them like for a smart phone or something. For anything else you're probably better off running on an FPGA. And the problem is they're full of black boxes. Nobody really knows what happens when you compile your bit stream onto a device. You know okay well we've got these lookup tables, we've got these RAM blocks but what is actually going on underneath? How do I know that the RTLI gave the compiler is actually equivalent to what the actual device's behavior is? We don't know. Or how do I do development on a platform that is not x86 or x64 windows or one of the very short list of Linux distros they support? Sorry, out of luck. And let's say we think we found a compiler bug. Oh we just pop the code and obj dump and look at it. Nope there is no decompiler for the bit streams so if you think it's generating bad code, sorry you're screwed. And of course reverse engineering this is recon. The vendors want people to think bit stream reversing is hard. They advertise their closed source proprietary format as being impossible to reverse engineer. It's not. So as far as the methodology here, I just decided earlier today to take a look at the size of the directory I had installed the Xilinx tools in. 18 gigabytes. I don't know about you but I don't have time to look through 18 gigabytes of stuff in IDA. There are much better things I could spend my time on. And plus the license agreement says I'm not supposed to be a reverse engineering software. I never agreed not to look at the silicon. So open source spending means necessary. So this is our target. It is the Xilinx XC2C32A. I chose it for a bunch of reasons. One is that it is very cheap. They're about $1.15 each on Digi-key in single units as of a couple of weeks ago. I think the price just went up to $1.20 something. But they're still cheap enough you can afford to kill large numbers of them testing. You have your choice of a nice big QFP package that's easy to hand solder, has plenty of plastic around the package so it's easy to decap while keeping bond wires intact. Or you can do chip scale BGA or QFN if you just want to dunk the thing in acid and get rid of all the plastic and take a look at the silicon. It's a nice friendly 1.8 nanometer process for metal UMC. So you can actually read out most of the upper metal layers optically. You don't have to bother going into the electron microscope except for the lowest layers. So that helps. It's a lot easier to use an optical microscope in a SAM. It's a lot quicker to shoot a lot of images quickly. The bitstream is also not that big. It's around 12K. So there's not really that much better reverse. And it's also a fairly simple architecture. You're not going to find block ram and embedded arm cores and all kinds of stuff like that. It's just pure programmable logic. And the two vendor tools are free which always helps. Hackers are cheap. So the bitstream format for this thing at a high level, the output of the compiler is a JDEK programming file. It's basically the equivalent of IHACS but instead of being HACS lines, it's binary lines, ASCII 1s and 0s. So this is the exact data that gets written to the chip. Not in the actual order. I'll get to that later. But it is, there is a one-to-one correspondence between these 1s and 0s and configuration bits on the device. And the nice thing is that, as you can see on the right side here, the bitstream generated by this I-Link's tools, if you're so inclined, does have comments. Turns out they're not very useful. They just say, okay, block zero ZIA. Well, what is a ZIA? Look in the datasheet. There is not one reference to ZIA. Okay. It doesn't really help me much. So let's get a little bit into the architecture of what a CPLD actually is. So any digital equations can be expressed using a form known as sum of products. It's a canonical representation of digital equations in which you take a series of terms, you bitwise and them together, either of the input or its complement. You can just treat the complement as coming into your device. So you've got A and B and C or C and D and E, et cetera. So you can express any arbitrary digital equation this way. So let's look at how we'll actually make a chip to do this. So if we have a large, say, 32 input AND gate and we have a MUX at the input to the gate, that lets me select between a constant one or the input to my circuit, I can effectively MUX any subset of these inputs and just leave the others as one, which is the identity for the bitwise AND operation. So I can AND together any subset of the possible inputs to this AND gate. The same thing can be done for OR. Obviously, you have to use zero as the identity element for bitwise OR. But what this means is you can create a gate that has a huge number of inputs and I can pick any subset from those at runtime with MUX settings. So this leads to a natural structure for a programmable logic device. You make a grid of gates with a 2-1 MUX at each input. You've got inputs coming in, outputs going down, and then you just have a series of gates in sequence and stuff together. So this is actually a render for my tool. You can see we've got the input coming in. We select either X or X prime and that with the data coming in from overhead. Take the output of that. Now here we skip the AND gate. This is more of a logical view, so I'm not actually showing the actual MUXes in the chip. This is more of a schematic view of what the circuit logically does. I'm also rendering it as a cascaded sequence of gates end to end. In practice, the actual implementation on silicon is usually a tree because nobody wants order AND propagation delay when you could have order log AND instead. So if we take this and we want to build a full programmable logic device, what we do is we take a bunch of signals coming from our registers that store state. We take a bunch of signals from our input pins. Then we feed these into an AND array. We invert them so we have two times however many inputs. We take M product terms out of that. The product terms then go into a programmable array. The output of that gives us, we'll just call it our outputs. Then we feed those either into an output pin or into a flip-flop of one of the state machines or something like that. So this is really all it takes to build a simple programmable logic device or SPLD. The problem is SPLDs scale poorly. You end up having quadratic scaling with the number of inputs. Nobody wants to die whose size scales quadratically with the amount of work it can do. That's terrible. So how can we improve this? Well, it turns out we can make a grid of small SPLDs. So it doesn't show up as well as I had hoped on the projector, but we've got one SPLD block here. We've got one here and there's a crossbar switch in the middle. So now we can create a bunch of outputs from one SPLD, a bunch of outputs from the other, and then just feed them all into one big routing fabric, pick out whichever signals I'm interested in processing on this half of the chip or that half of the chip, and then feed them into the SPLD. So this is a SPLD. So now let's look at the specific SPLD that we've been targeting today. So there are 32 GPIO pins which are full input output plus one input only pin. I'm pretty sure this is because they intended to package the die in a 44 pin QFP package. All of the other stuff for JTAG power and the 32 GPIOs use 43 pins. They didn't want to have one NC pin. So they just threw one extra signal into the global routing that you can't drive and it just serves as an extra input. And then the remainder of the device is two function blocks which are basically SPLDs. Each one has 16 GPIO pins, 16 flip flops. Of the 65 signals in the global routing, so 32 IOs, 32 flip flops plus one input only 65. We pick 40 of those, feed those into an 80 by 56 and array, then a 56 by 16 array. This is all documented in the data sheet. What's not documented is, okay, we can make a technology math net list. We know, okay, I have these inputs added together, these outputs or together. Now, how do I actually make the chip do what I want? So time to put in our lab codes. I am not going to talk about decapping and imaging. This has been beaten to death in other talks like recon last year and I think the year before that and probably the year before that. So we're not going to cover that. There's a lot of good stuff up on SiliconPron. There's, if you read the lecture notes from the class I taught at RPI last year, there's a lot of good material there. But this talk is about reverse engineering, not on sample preparation. We're not going to be teaching you how IDA works. We're teaching you how to actually reverse engineer this specific device. So here is the metal 4 overview of the device. We can see that there is a roughly left-right symmetry. It's not exact. You can see here there's something that's not mirrored here, but up here it looks like the device is pretty symmetrical. So first impression, we've probably got the two function blocks left-right symmetric on the device. Let's go down a little bit. After we've etched off all of the metal and poly layers, we're now looking at the implant layer. This is after a process known as dash-etch, which stains P-type doping either brown or it shows up raised under the electron microscope. Mainly, though, it's useful because it provides contrast in areas of gates. So you can see we've got pair-wise symmetry here and here. Those are probably the function blocks. There's something down the middle. That's probably the global routing. Then there's a large memory right here. Pretty obviously the EEPROM where the bitstream is stored. Then if we actually follow the bond wires from the die out the pins in the package, you can see these are the JTAG pins, CDI, TMS, TCKI, I believe top to bottom. The TDI pin is right there. So we can conclude that the JTAG shift register probably runs left to right across this configuration area and somehow allows you to write this EEPROM here. There's also a few small pairs. There's one up there. There's one up there. I think there's one on the upper left. There's a total of six. I have not yet figured out what these do. It wasn't necessary to reverse the portions of the device I needed. So that remains future work. So let's take a closer look at the function block. So we know there are 16 macros cells. Each macros cell contains one flip-flop and stores the output of one term of the ORA and has a little bit more glue logic I'm going to talk about later on. So there is 16 wide symmetry here. 16 identical copies of this. Just by looking at this, we can be pretty sure we're looking at the macros cells. Then we see some structures up here and here. This looks symmetric. This looks symmetric. This looks symmetric but different. So as it turns out, the AND array is not actually one solid block. The AND array is split in half. They've got 20 signals here, 20 signals here. Collectively that forms 40. Then they AND the outputs of those two individual blocks together. That gives you our product terms. And then that goes into the ORA here and then out the macros cells. So now let's take a quick look at the configuration bit structure before we dive into detailed die analysis. So the programming documentation does talk a little bit about how the device is put together. It turns out that even though the JED format is supposed to be something you can actually just feed to an EPROM program and load the device, that's not the case here. The bit ordering in the JED file is actually virtual addressing in which they abstract away all of the quarks of the silicon. So for example, all of the AND array bits and all of the ORA array bits are in the same order in the JED file for each one. Turns out half the actual AND and ORA array blocks on the device are mirrored left right. So you actually have to do address translation before you can take a bit stream generated by their tools and flash to the chip. Luckily they do sort of document this. There's a big Excel file they publish to people who are making program error adapters that is just a big grid of output and input and just has a intrate each cell. It says, okay, which config bit from the JED file goes this physical address? So it doesn't really tell you much about how the chip is put together. The actual structure is 48 rows by 260 columns. There's one extra row full of configuration metadata. I'll get to that later. But it stores seven lock bits, which some of them are always one. I'm not entirely sure what purpose they serve. The remainder are one for an unlocked device and zero for a locked device. Then there's two done bits, which indicate, okay, this bit stream is valid. We have a legal firmware and the chip has been fully flashed. And it turns out also only 258 of the 260 columns are usable. The remainder are what we call transfer bits, which as far as I can tell, you just put them as a constant zero. It just indicates, hey, this row of EPROM has actually been programmed. We didn't abort halfway through programming. There is some documentation of this in a patent from Xilinx, but it didn't really give enough detail for me to figure out, okay, why are they doing this? And the other intuition we can get from this is that since it is EPROM based, then FF is going to be the state of the memory when the chip is blank. So therefore, we expect most of the memory on the device is going to be active low. So most things should be turned off when the bit is high and on when the bit is low. We just don't know exactly what that bit does yet. So now let's take a look at the actual die structure. So you can see that the configuration memory is not actually one block. We've got an array here, one here, one here, one here, and one hiding all the way over here. And as it turns out, the size of these are symmetric about the center, and they go directly up to the corresponding logic. So it's pretty obvious we found the memory that configures that part of the chip. The data just flows straight up during the boot process. So this configures the end of the array, this configures the global routing, this configures these macro cells, this configures these macro cells. We can confirm this if we look up at the metal two. You can actually see the lines coming off the send sample fires for the EPROM going up and vertically into the array. And if you trace it out all the way, you can actually see it writing to the individual configuration SRAM cells in the die. There are a few bit lines here and here, for example, that are connected and I believe metal three instead of metal two. They couldn't fan it all out in one layer. So now let's take a look at the main logic array. If we actually look at where the SRAM cells are and count how many rows high each individual block are, we see that the end array is 20 rows high, the or arrays eight rows high, other half is 20 rows high, and each macro cell is three rows high. So that gives us a pretty good idea. We know left, right, which bits in the bit stream configure which logic just by what's physically proximate to it. And we can make a pretty good guess that since we have a 2DS for MRA here in the configuration area, we have a 2D flash array, we probably have either top to bottom or bottom up addressing. There's actually some gray code going on. It's a little more complex than that, but logically we should have the addressing going across vertically. So if we take a closer look at the end array, we know that there are 56 product terms. We know that there are supposed to be 40 rows and 40 complements of rows for a total of 80. If we actually count how many SRAM cells are, we'll see there's one 12 bits wide, just 56 times two. There's two blocks of 20 rows each. So the conclusion is that since we know there's 40 inputs coming in from the crossbar, we've got two blocks that are each 20 rows high, each row probably corresponds to one row of input. And then we know that the width of the array is double 56. So it's probably two bits per product term. One selects X, one selects non-X, or maybe some sort of other code. As it turns out, it is one-hot coded. I just figured this out by experiment. It was pretty easy. Just try one, see if it works. If not, try something else. There's only four possibilities. So the or array turns out to be 56 and terms, 16 outputs. If we count the config bits, it's still one 12 bits wide. It's the same, coming up off the same EEPROM, but it's only eight rows high. And there's only 56 inputs. Since the or array does not have an X and X prime input, my conclusion is that we still have one hot select thing for one particular input, but we do have two actual arrays of the or array interleaved in one configuration row. I got a little tinkering. It was pretty simple to figure out the actual bit ordering. So now let's take a look at the macro cells. There are 27 configuration bits per macro cell. If we look, you can see there's one 60S RAM cell, one 60S RAM cell, and so on. There's a 9 by 3 grid. It turns out this 9 by 3 grid does not actually control one macro cell. The bottom two rows of this RAM control the bottom one. This goes to the upper one. This threw me off a little bit when I started looking at the circuit, but it turns out that yes it is 27 cells, yes it is 9 by 3. No, the 9 by 3 structures in the die do not actually correspond to this. So this makes intuitive sense. The EEPROM at the bottom of the die is, well, okay, it's 10 bits wide, but we know one of them is a transfer bit. So the other nine bits go directly up to here. I've figured out a significant fraction of the functionality, not quite all. There's still some clocking stuff I'm a little unsure on right now. These also configure IO buffering and stuff like that. So there are lines coming out from these to the side of the device and controlling the IO buffers. Now I will, as a quick aside, jump into the security bits. We know there are nine done bits and lock bits somewhere on the device. We know that the physical address of these is in the right hand macro cell memory. It's in the top row. There's nine 60S RAM cells right here that don't appear to be hooked up to any of the actual logic array. I have not actually tried fibbing these. I don't know for certain that these are the lock bits, but it is pretty obvious. Especially when we know of those nine bits, four of them have to be held low to lock the device. There's a four input NOR gate right here and an inverter right there. Hmm, I wonder what happens when I cut that line. I don't know. So now we'll get to the global routing. So we know between the left and the right halves of the end array, there is something. We know that it is 20 configuration bits high for each half. We know it's 16 bits wide. We don't know anything more than that. The data sheet has about two sentences that talk about the global routing. So there is, we have no idea whatsoever how it works. What we do know is that of those 65 signals coming in, we know that 20 of them go to the left function block, 20 go to the right. Those subsets may or may not have any relationship. And we know since it's 16 bits wide, we probably have eight bits selecting what goes left and eight bits selecting what goes right. So now here's the question. How do you make a 65 to one MUX with eight bits? If you just had zero to 65, you'd only need seven bits. If it was one hot, you'd need 65 bits. So what sort of strange, perverse code are they using here? The data sheet is of exactly no help. Well, time to get dirty. So if we jump into the electron microscope and take a look at the implant layer after dash-ups, so you can see these areas here show up raised. Those are P-channel. These are N-channel. You can see that it's slightly lower. The raised areas are the individual channels for the FETs. This is zoomed out. The original image is a lot higher resolution, but I can't really fit it in the projector. So this is the implant layer. What we can see immediately is that there's some sort of symmetry going on, both horizontally and vertically. If we jump up to metal four, we see that there are six small buses of 11 signals each. The rightmost is 10. All right? Five times 11 plus 10. What's that, anybody? 65. Hmm. There are 65 signals going into global routing. I think we found them. So I spent quite a while in Inkscape vectorizing this. My automated tools are not quite as well-developed as I had hoped. So it did take me a while, but I do now have a full vectorization of the entire global routing matrix from implant layer all the way up to metal four. The little gray boxes are not actually layout. There are standard cell outlines. So we've got a inverter here, another inverter here, another inverter here, and so on. So we can see that there are six identical blocks going left to right here. There's two weird blocks over there that are identical to each other at first glance, but not identical to the rest. Okay. This is starting to get interesting. So those big drivers on either side are pretty clear of the buffers driving the PLI. We've got a fan out of 112. We need a fairly big buffer. There are, I believe, 20-some fingers on this last inverter. So that looks like a high fan out driver. It's actually a three-stage inverter because the low-level stuff in the programmable logic doesn't actually have the drive current to drive that much gate capacitance. So they actually have a three-stage driver inverting it, inverting it again and increasing the current once again to actually drive the output. And of those eight blocks, each one contains two S-rem cells. So it seems to make sense that we've got eight blocks. We know that we have eight bits controlling the left output. We have eight bits controlling the right output. So we probably have one bit per block controlling what goes left and one bit controlling what goes right. Then there's six identical blocks and there's six groups of wiring up in Metal 4. So there's probably some sort of correspondence going on here. Again, we don't know what it is, but we know there's some relationship. So now let's take a look up at Metal 3. So we can see there's power and ground routing here. We'll ignore all of that for now. But there are sets of one, two, three, four, five, six vias per row. What's more interesting is there are always six vias per row, but they're not all in the same place. So there is, if we actually trace it out for all of the rows, each one of those vias is under each of the Metal 4 groups. There is exactly one via in each row column intersection, but not at the same place. So now I finally realized what is going on. The routing matrix is not actually a full crossbar. It is a sparse crossbar in which of the 40 rows, each row can only pick six of the 65 inputs. But there's a different subset for each one. And the end result is that using all of these subsets, you can select any unique subset of 40 of the 65. But you don't actually need a 40 by 65 crossbar. So now let's take a look at how the actual implementation works. We have a pretty good hunch as to how this is structured at a high level. We don't actually know what the implementation is. So I do apologize for some of the misalignment. This was a quick tracing. And so not all of the vias and metal layers line up exactly. This is not meant to be something you can clone the chip from. It's meant to be something I can figure out how it works from. So you can see there's a big pass transistor here. We've got the signal coming in from the upper layer here and here. And then we've got an output of the mux here. There's an SREM cell that goes through a two input NOR gate and then drives that. So if we put this all together, what we see is that each row is indeed an 8 to 1 tri-state box mux. So there's one pass transistor that selects each of the six possible outputs for a row. Then there's a single discrete NMOS and a single discrete PMOS that lets me drive constant 1 or constant 0 as well. It turns out that the driver for constant 1 is active high. All the rest are active low. So this means that a blank bit stream of FF won't cause bus fights. It'll put all the outputs in a well-defined state. So this makes sense. And all the other signals are active low. There's also one additional signal I've called O-Gate. I have no idea exactly where it comes from, but I'm pretty sure that it's used during the boot process to basically pause the device, don't drive any of the outputs. It'll consume less power when the chip is idle. And it also prevents bus fights between drivers that aren't fully configured. If we've got half the device flashed and half not flashed during the boot process, we don't want to be driving signals onto nets that some other thing might still be driving with the old firmware. So as long as the signal is high, it gates all the outputs. And the rows are, again, not identical. But it turns out we can actually do the routing using MaxFlow. We just create a source node for each of the nets we want to route. We create a directed graph with paths for each of the legal connections to each row. Then we create a sync node with however many nets that we want to draw out. And we can just use MaxFlow to route everything. So as it turns out, the structure is pretty much that. I do have a full schematic. This is not it. I've kind of truncated the bottom to make it fit on the slides. But we've got a single configuration bit. We've got the NOR, which I've actually drawn as a negative logic AND because that's really how it's functioning. Output of that goes to a single PMOS pulling high. We've got an NMOS pulling low. Then we've got MUX in top A0, MUX in top 90, and so on. Those are the actual hex codes in the bit stream for selecting that particular input. So we make that row be set to A0 if we wanted to write to this element. And I do also have a table in the source for my tools that includes all the MUX settings for this device. So I now can fully control the global routing matrix. But there's one last bit. We know the ordering of the inputs. We know how to select one of the inputs and drive something. We don't know which input is actually hooked up to which signal on this bus. So time to get dirty again. There were a couple of options I considered, but it turned out that one of the simplest was to make a few educated guesses about how things were structured. So for example, all of the macrocells in function block one, they are flip flops, are probably in a contiguous order on the bus. We don't know where on the bus they are, but they're probably contiguous. All of the input pins for function block two are probably contiguous as well. So let's tinker around a little bit. Okay. The inputs to function block two are probably contiguous. So there's not really that many orderings for the global input pin and these four sets of inputs. Let's just try tinkering and see which one works. So I went up to campus, hopped on the focused ion beam, drilled a few holes in the insulator over some wires, and laid down some nice gigantic 20 micron square probe pads. And then I just started driving signals onto each net. Here's the code I'm using. It's just a counter going from a 2 megahertz input dividing down to, I believe, a 2 hertz LED. And then I have one signal before that. So 4 hertz is on a specific flip flop that I've constrained the two. I was like, okay, here, function block two, macrocell five. So let's see. If we probe function block two, macrocell five, or if we put a signal on function block two, macrocell five, it's supposed to be 4 hertz. And we've got a probe pad on this signal. Is it a 4 hertz square wave? Oh yeah, it is. All right. I guess it's right. So it didn't take too long to figure out the actual bit ordering. So here's the basic summary of the actual layout. We've got gates on metal one and poly. Metal two is vertical routing, SRM bit lines, O-gate. Then horizontal routing on M3. Then M4 is the input bus. It turns out the actual ordering is the GPIOs for function block one, the global input, GPIOs for function block two, then the flip flops left to right. So now we know pretty much enough to configure the PLA. Unfortunately, there's more to it. Because it turns out all of the product terms, or nearly all of them, are dual purpose. They can be used as general purpose logic. You can feed them into the OR array. But each of the 16 macrocells also has three product terms that have separate dedicated connections to it. These are used for set, reset, clock, enable, and so on. I'll cover some of this in the next slide. Four more have special connections to the entire function block. And the last four, as far as they can tell from reading the datasheet, they have no special purpose. The datasheet does tell us these things exist. They tell us roughly what they do. They don't tell us which of these 56 product terms does what. So I will just briefly cover what these terms are. So we've got per function block, we have a local clock that we can use. So we don't want to waste global clock resources for the whole chip. But we also don't want to have a per flip flop clock because that will both increase skew and use a lot of product terms that we really don't need. Then we have dedicated set and reset and dedicated output enables. Then per macrocell, we've got product terms A, B, and C, very inventive naming. So product term A is one of several legal sources for set, reset. There's also a, the control term set and reset can be used instead if you want. There's product term B, which is one of several possible sources for the IO buffer output enable. And then product term C, which can be used for a couple of things. If you've got a clock enable, it's used as a clock enable. If you want a per macrocell clock, for just one flip flop that you're clocking off one weird clock that nothing else is clocked by, you can use that. And it also drives an XOR gate. So it turns out that this isn't quite a conventional CPLD structure. The output of the PLA is XOR with product term C or its complement or a constant one or constant zero. This is a Xilinx proprietary optimization. That is, again, documented in the data sheet, but they don't tell you how to configure in the bit stream. The intention here is to allow you to do efficient adders. You've got XOR. You don't have to do not X or Y, not X and Y or not Y and X. So you can just use the XOR gate. So the question is, okay, we've got 56 product terms. Which one is PTC? No idea. Well, it turns out we can configure the PLA as much as you want. We already know how to configure that. We've reversed all the bets for that. We can configure the global routing as much as you want. And it's fairly easy to generate a bit stream that is known to use product term C from the tools. All we have to do is either synthesize something with an XOR in it or it turns out there's actually an optimization of the tools. If you are not using the OR array, just say we just have Y equals A and B. It turns out that if we set the OR array to output a constant zero and then we XOR that with product term C, we no longer have the OR array in our critical path. So you can sheave about 500 picoseconds off the propagation delay by doing this. The compiler trying to optimize speed does this by default if you're not using the OR array. So therefore, just by creating an equation like Y equals X in your source code, you can trivially produce a bit stream that uses product term C. So all we have to do is say, okay, let's make something that uses product term C. We'll spam these black box macro cell bits into all of the outputs and just start feeding inputs into various product terms until we find one that hits. Well, it turns out it is zero based term 3N plus 10. Unfortunately, I have things called customers that kept me from working on this project as much as I would like. So I never actually figured out what product terms A and B are. I'm pretty sure that they're either just above or just below product term C. I don't know which. The control terms, if you remember, product term C is 3N plus 10. So below number 10, none of these terms are actually being used for anything in PTABC. So the control terms are probably among the first couple of terms. But again, I have not actually had time to figure out which is which. One more wrinkle is that there are some global configuration bits. So if you remember, we've got the structure of the die on the left side and the right side. I've got the function blocks and the top. I've got global routing and the bottom. I've got global routing. So that pretty much makes a big donut shape. In the middle of that, there's wasted space. But no, it's not actually wasted. There's 22 single configuration bits, the middle of the die. We know three of them control the VT for the IO banks. The device has two IO banks, but the original XC2C32 only had one IO bank. So they did a clever hack. They are now bitstream compatible with the 2C32. They added four more configuration bits. So the way they did this is they have one global bit. If this bit is set, then it will use the old school XC2C32 configuration setting for this. The newer ones are two additional per bank bits. So it actually just does a bitwise add between the global bit and the per bank bit. If you erase the device and you don't program these last couple of fuses, then you will get the default XC2C32 behavior in which you've got one IO standard for everything. If you're compiling for the 32A, now you set that other bit as a constant one. Leave the other one set the way you want. And now you get the ability to set VT for each bank independently. So it's actually pretty clever. Then there are other miscellaneous things that are there. The comments in the bitstream were a little bit helpful for this. So I know there's three bits that configure global clocking, two bits that configure global set reset, eight that configure global output enable. I do not actually know what the coding for these bits are at this point. So there's still some work in progress going on. Now I will get to the actual tool chain that I'm producing. I've called it libcrowbar after the flying crowbar project on SiliconPron. The goal is to, in general, produce open source tools for programmable logic. It does need a lot of refactoring and tweaking. It does not do nearly everything that I want. It only supports the 2C32A right now. We can scale to other devices. The 64 cell device is pretty much just a scale up of the 32. I've decapped it. I've looked at the top layer. It's just a 2x2 grid of function blocks instead of 1x2. The ROM for the routing fabric will have to be decoded separately, but that shouldn't take too long. The ordering of the bits will have to be decoded, but again, that shouldn't take too long. I can probably just brute force it. And the macro cell configuration and everything else is pretty much exactly the same. The larger device is the 128, 256, 384, and 512. It adds some additional features to macro cell. So in addition to figuring out the actual routing stuff, I will actually have to do some additional reversing of the macro cell logic. So that remains a work in progress, but that is a goal. The library is BSD licensed. You can grab the code from there. I do not recommend attempting to use it in its current state. It's more of something to look at and understand how the chip works. And maybe in a couple of months, it will have something that's actually stable enough you can actually use. But I do have a quick usage example here. So we have a bunch of IO pins. We select P8, P6, P38, P37, whatever. Then we get the IOBs for each of these. We can now select the IO standards, some output enables. We're not going to use termination. We're not going to use Schmidt trigger. These are all bits in the macro cell that I was able to figure out. There are still quite a few related to the clocking that remain unknown at this point. And then there's quite a bit more going on below this. I can pull out full source if anybody is interested in looking at it. But the end result is that I can go from an in-memory technology map net list created like this as C++ objects. I can then place and route this, produce a valid bit stream, flash the device and have it work. I can also go the other way around. I can go from a program device. I can read out the bit stream. I can turn it into an ASCII art schematic. And I can turn it into synthesizable Veralog as long as you're not using the features I haven't yet reversed the functionality of. The remaining thing in the forward tool chain is I would like to integrate with YOSIS for synthesis, take the output of YOSIS, do technology mapping on that and then feed it into libcrowbar. At that point I will have a full Veralog to bit stream tool chain. I'm not quite there yet, but it's coming close. The other tool I've made, if any of you have used plan ahead from Xilinx, I called mine very inventively FC plan for the flying crowbar floor planner. So it is a floor planner and physical layout viewer. I currently only have rendering for the and array and the global routing. The or array, I do have the functionality reversed. I have not written the code to actually draw it, but I can actually open up a bit stream and look at the individual settings in the and array. The macro cells also, I've reversed some of the bits, not all of them, but I have not yet written code to actually render the appearance of them as check boxes for, okay, instrument trigger is enabled, the terminator is enabled, the slew is fast or slow and so on. So those all remain in the wish list at this point. The configuration bits are known. You can access them from libcrowbar. You cannot do so in the GUI. So here's one more view of FC plan. This is showing as you zoom out, the product terms shrink down to smaller columns. You can see the full global routing matrix here. This is the actual via pattern. It doesn't show up as well in the projector again as I would have liked, but you can see the actual mux bits in there. When you zoom in in the tool, you can actually see the signals coming in from the global routing going out through these vias and into the and an arrays. Before we get the demo, I do want to thank John McMaster from SiliconPron. Hi, John. He did most of the large scale optical imagery for this. He also did the dash etching, which was very helpful. Then Ray Dove and Brian Colwell at RPI run the material science lab in the clean room respectively. They were quite helpful when it came time to getting access to the electron microscope in the clean room and so on. As far as I know, I am the only computer science student there, whoever ran the electron microscopes. But I did actually get access to the clean room, the SEM, the FIB. I was trained on all of them and they were quite helpful. Then the SiliconPron team in general was quite handy when it came to just getting feedback and sanity checking and does this look okay to you? I can't quite make out what this connects to. Do you have any ideas? Or just sharing process suggestions, okay? What are some ways I can get this edge to be a little more even? Something like that. Now, I'm to attempt the demo gods. I have my JTAG daemonic connected already. This is my dev board. It has a small FTI 232 on there that I'm using for JTAG. What I'm going to do is I'm going to create a bit stream using lib crowbar that will act as a not gate and a buffer. Then I'm going to bit bang inputs going into there with the FTI chip. We should see one LED and then the other alternative lighting up. I don't know if you guys can see. Right up here, these LEDs are blinking back and forth. If we scroll through the output here of the tool, you can see we start out initializing. We connect to the JTAG chain. We see okay, there is a device. One odd note, the cool runners are the only devices I've ever actually looked at in which the JTAG device ID actually includes the package as well as the device. What this means is that they've actually got a fuse somewhere in the die that says, oh, this is a TQFP. This is a BGA. I've never seen that on any other device. Any other FPGA or microcontroller, anything I've used, the device ID was the same between different packages. Anyway, so we confirm, okay, yes, the STDI chip has GPIOs on it. We generate our net list. We figure out the actual function block and macrocell locations of the IO buffers we're using. Then it runs the fitter through IO banks, macrocells, function blocks, does global routing, repeats for the other function block, does global routing for that, confirms okay, fitting is complete, took 620 microseconds. If any of you have used the Xilinx Compilers, they're not that fast. Then we finish generating the bit stream. That's kind of slow. Right now, it took a couple of milliseconds. I'm pretty sure I can optimize this. Monk, this is a debug build of profiling turned on. So it'll be a lot faster if I actually compile the optimizations. Then we actually configure the device. We verify, okay, yes, we are talking to the right device. We've got the right number of fuses. We erase it. We make sure it's blank. Then we just bit bang the out as 0 and 1. It confirms that the loop back going back to the FDI chip does have a value that we expect from one of MsX and one of MsX prime. That looks good. Now, let's see what happens if we actually want to reverse this. Let me see if I can up the font size. Can people read this? I'm guessing no. Is that better? All right. So this is what happens when we actually try dumping the bit stream. This is not the same bit stream. This is a different one. I just picked that random I had sitting around. So we have a nice ask your schematic showing all the inputs coming in from the PLA or coming in from the global routing. Each of these represents one connection. That's one configuration, but it's actually connected to something. We have the X and the X bar outputs. Nice big grid. Then 56 product terms to the or array. We've got outputs going to the agris array. This table looks a lot nicer at 1080p when your font isn't quite so big, but I do actually have details on, okay, this output is floating. This one is configured as an input, this is an input, this is an input. There's an output in here somewhere, probably on the other one. Whatever, I must have scrolled past it. So now on the output, we have the IO standard. It turns out even though there's about a dozen IO standards you can pick from its emphasis time, in the bit stream there's only two. There is the high voltage and the low voltage. So there's 1.8 and 1.5 and there's high voltage. So my guess is what this does internally. I have not fully reverse engineered all of the IO driver's arbitrary, but what I'm pretty sure it does is it selects a single threshold or another and then thicker a thinner gate oxide in the output. There's about four or five different sets of drivers in the output for fast and slow, different voltages and so on. I haven't actually done TEM cross sections and measured VTVs transistors or anything like that. It wasn't necessary for this. And then we've got a bunch of stuff here for the global clock muxes, global reset muxes and everything. And all this remains unknown. I know roughly, I know which bits these are. I don't know what the functionality of those bits is. And then, let's get the fun part. Let me actually switch to the adder test.jed file. This one has a simpler, this is a 4-bit adder, which will make a lot more sense than the 32-bit adder I was showing the other one. So here's our RTL. It is pretty much the assembly language of AeroLog. This is a direct analog of the actual PLI structure. I have not attempted to figure out any higher level structure. This is actually equivalent. I have not tested synthesizing it. I actually just found a bug last night here on lock equals. That should be quoted. So right now the output as formatted won't synthesize, but it would be fairly trivial to fix that and make it actually synthesize and work. And then I've got function block 2. There's not really anything in there. So now before we get to the questions, I'm just going to jump up to FC plan and show you guys what the actual structure looks like in the floor planner. I guess I found a bug. The scroll on this thing is really sensitive. All right. Can I go just one click in? All right. That's probably good enough. So you can see what I've got. Initially we're bypassing all the inputs. So we're not using the first couple of rows coming off the global routing. Then down here I've got one signal coming in. It's hooked up over here to X. It's hooked over here to X crime. Goes into the AND gate. Then we go down there and so on. So we do actually have a full physical layout viewer at this point. So now I'm just going to jump back to the slides for just one or two more before we get back to... So what remains as far as future work? We still have to figure out the last of the special product terms. There still are about six or eight bits in each macrosail that we have not yet figured out the functionality of. There's still some more in the global devices. The global bits. As I did mention, there is a little bit more I need to do to support larger devices. As far as the tool chain, I want to do more work in the decompiler. My long-term goal with this work is to integrate with some other bitstream reversing projects and so on and create the IDA for hardware. I would like to be able to go from bitstreams for CoolRunner, bitstreams for Part 6, bitstreams for Stratix 4 and everything, and go from that up to a device-dependent technology map net list. Abstract that up to a device-independent net list. Something more along the lines of LLVM IR for hardware and then do higher-level analysis on that to figure out, okay, this combination of XOR gates and MUX is looks like a 32-bit adder. This combination of stuff looks like a 10-to-1 MUX. This combination of stuff looks like a Cortex M3. I would like to eventually be able to have something along the lines of flirt here. I do know that subgraph isomorphism is NP-complete. I don't know if I can make a randomized or approximated algorithm that's fast enough that remains a topic for future work, but it is on the wish list whether it is physically possible or not. So, at this point I'll take questions. Seeing none, I guess we're done. Thank you. Thank you.
 You guys hear me now? Okay, sounds good. So my talk is called from Silicon to compiler  and it's pretty much that. So we're going to start off with what we're doing, why we're  doing this, a little bit of architectural background on programmable logic for those  of you who have not done work on programmable logic before. Then jump to a block diagram  of the device. We'll start with the high level overview of the silicon and then we'll drill  down into more interesting stuff down to transistor level and gate level circuit analysis. At  the end I do have a live demo of firmware produced by my tools running on live silicon.  Sadly there are no cute pictures in this presentation, sorry. So a little background about me, I pretty  much like to build and break everything. I will do web stuff if they pay me enough. I  mostly like to live down in low level, a bit of ring zero, firmware, low level board design,  RTL design and now getting down into transistor level. I just finished up my PhD a few weeks  ago. During that time I designed and created what I believe is the first ever college class  on semiconductor reverse engineering. I tried quite hard to find another one that I could  borrow notes and slides from as far as I can tell none existed. And I'm also obviously  a significant contributor to Silicon Prone. So if you see all the guys walking around  with decap chips in their shirts, say hi. I've only been with IO active since January.  A lot of this work was actually done before I joined the company but they've supported  me continuing the work. So I acknowledge them for that obviously. So as far as what we're  actually trying to do and why, program logic is really everywhere these days. You may not  know it but a lot of especially high end networking AV, all sorts of stuff has program logic in  it because A6 these days the tape out costs are so high especially for leading edge process  nodes it usually is not actually cost effective to make custom silicon unless you're making  a huge number of them like for a smart phone or something. For anything else you're probably  better off running on an FPGA. And the problem is they're full of black boxes. Nobody really  knows what happens when you compile your bit stream onto a device. You know okay well we've  got these lookup tables, we've got these RAM blocks but what is actually going on underneath?  How do I know that the RTLI gave the compiler is actually equivalent to what the actual  device's behavior is? We don't know. Or how do I do development on a platform that is  not x86 or x64 windows or one of the very short list of Linux distros they support?  Sorry, out of luck. And let's say we think we found a compiler bug. Oh we just pop the  code and obj dump and look at it. Nope there is no decompiler for the bit streams so if  you think it's generating bad code, sorry you're screwed. And of course reverse engineering  this is recon. The vendors want people to think bit stream reversing is hard. They advertise  their closed source proprietary format as being impossible to reverse engineer. It's  not. So as far as the methodology here, I just decided earlier today to take a look at the  size of the directory I had installed the Xilinx tools in. 18 gigabytes. I don't know about  you but I don't have time to look through 18 gigabytes of stuff in IDA. There are much  better things I could spend my time on. And plus the license agreement says I'm not supposed  to be a reverse engineering software. I never agreed not to look at the silicon. So open  source spending means necessary. So this is our target. It is the Xilinx XC2C32A. I chose  it for a bunch of reasons. One is that it is very cheap. They're about $1.15 each on  Digi-key in single units as of a couple of weeks ago. I think the price just went up  to $1.20 something. But they're still cheap enough you can afford to kill large numbers  of them testing. You have your choice of a nice big QFP package that's easy to hand solder,  has plenty of plastic around the package so it's easy to decap while keeping bond wires  intact. Or you can do chip scale BGA or QFN if you just want to dunk the thing in acid  and get rid of all the plastic and take a look at the silicon. It's a nice friendly  1.8 nanometer process for metal UMC. So you can actually read out most of the upper metal  layers optically. You don't have to bother going into the electron microscope except  for the lowest layers. So that helps. It's a lot easier to use an optical microscope  in a SAM. It's a lot quicker to shoot a lot of images quickly. The bitstream is also not  that big. It's around 12K. So there's not really that much better reverse. And it's  also a fairly simple architecture. You're not going to find block ram and embedded  arm cores and all kinds of stuff like that. It's just pure programmable logic. And the  two vendor tools are free which always helps. Hackers are cheap. So the bitstream format  for this thing at a high level, the output of the compiler is a JDEK programming file.  It's basically the equivalent of IHACS but instead of being HACS lines, it's binary lines,  ASCII 1s and 0s. So this is the exact data that gets written to the chip. Not in the  actual order. I'll get to that later. But it is, there is a one-to-one correspondence  between these 1s and 0s and configuration bits on the device. And the nice thing is  that, as you can see on the right side here, the bitstream generated by this I-Link's tools,  if you're so inclined, does have comments. Turns out they're not very useful. They just  say, okay, block zero ZIA. Well, what is a ZIA? Look in the datasheet. There is not one  reference to ZIA. Okay. It doesn't really help me much.  So let's get a little bit into the architecture of what a CPLD actually is. So any digital  equations can be expressed using a form known as sum of products. It's a canonical representation  of digital equations in which you take a series of terms, you bitwise and them together, either  of the input or its complement. You can just treat the complement as coming into your device.  So you've got A and B and C or C and D and E, et cetera. So you can express any arbitrary  digital equation this way. So let's look at how we'll actually make a chip to do this.  So if we have a large, say, 32 input AND gate and we have a MUX at the input to the gate,  that lets me select between a constant one or the input to my circuit, I can effectively  MUX any subset of these inputs and just leave the others as one, which is the identity for  the bitwise AND operation. So I can AND together any subset of the possible inputs to this AND  gate. The same thing can be done for OR. Obviously, you have to use zero as the identity element  for bitwise OR. But what this means is you can create a gate that has a huge number of  inputs and I can pick any subset from those at runtime with MUX settings. So this leads  to a natural structure for a programmable logic device. You make a grid of gates with  a 2-1 MUX at each input. You've got inputs coming in, outputs going down, and then you  just have a series of gates in sequence and stuff together. So this is actually a render  for my tool. You can see we've got the input coming in. We select either X or X prime and  that with the data coming in from overhead. Take the output of that. Now here we skip  the AND gate. This is more of a logical view, so I'm not actually showing the actual MUXes  in the chip. This is more of a schematic view of what the circuit logically does. I'm also  rendering it as a cascaded sequence of gates end to end. In practice, the actual implementation  on silicon is usually a tree because nobody wants order AND propagation delay when you  could have order log AND instead. So if we take this and we want to build a full programmable  logic device, what we do is we take a bunch of signals coming from our registers that  store state. We take a bunch of signals from our input pins. Then we feed these into an  AND array. We invert them so we have two times however many inputs. We take M product terms  out of that. The product terms then go into a programmable array. The output of that gives  us, we'll just call it our outputs. Then we feed those either into an output pin or into  a flip-flop of one of the state machines or something like that. So this is really all  it takes to build a simple programmable logic device or SPLD. The problem is SPLDs scale  poorly. You end up having quadratic scaling with the number of inputs. Nobody wants to  die whose size scales quadratically with the amount of work it can do. That's terrible.  So how can we improve this? Well, it turns out we can make a grid of small SPLDs. So  it doesn't show up as well as I had hoped on the projector, but we've got one SPLD block  here. We've got one here and there's a crossbar switch in the middle. So now we can create  a bunch of outputs from one SPLD, a bunch of outputs from the other, and then just feed  them all into one big routing fabric, pick out whichever signals I'm interested in processing  on this half of the chip or that half of the chip, and then feed them into the SPLD. So  this is a SPLD. So now let's look at the specific SPLD that we've been targeting today. So there  are 32 GPIO pins which are full input output plus one input only pin. I'm pretty sure this  is because they intended to package the die in a 44 pin QFP package. All of the other  stuff for JTAG power and the 32 GPIOs use 43 pins. They didn't want to have one NC pin.  So they just threw one extra signal into the global routing that you can't drive and it  just serves as an extra input. And then the remainder of the device is two function blocks  which are basically SPLDs. Each one has 16 GPIO pins, 16 flip flops. Of the 65 signals  in the global routing, so 32 IOs, 32 flip flops plus one input only 65. We pick 40 of  those, feed those into an 80 by 56 and array, then a 56 by 16 array. This is all documented  in the data sheet. What's not documented is, okay, we can make a technology math net list.  We know, okay, I have these inputs added together, these outputs or together. Now, how do I actually  make the chip do what I want? So time to put in our lab codes. I am not going to talk about  decapping and imaging. This has been beaten to death in other talks like recon last year  and I think the year before that and probably the year before that. So we're not going to  cover that. There's a lot of good stuff up on SiliconPron. There's, if you read the lecture  notes from the class I taught at RPI last year, there's a lot of good material there.  But this talk is about reverse engineering, not on sample preparation. We're not going  to be teaching you how IDA works. We're teaching you how to actually reverse engineer this  specific device. So here is the metal 4 overview of the device. We can see that there is a  roughly left-right symmetry. It's not exact. You can see here there's something that's  not mirrored here, but up here it looks like the device is pretty symmetrical. So first  impression, we've probably got the two function blocks left-right symmetric on the device.  Let's go down a little bit. After we've etched off all of the metal and poly layers, we're  now looking at the implant layer. This is after a process known as dash-etch, which stains  P-type doping either brown or it shows up raised under the electron microscope. Mainly, though,  it's useful because it provides contrast in areas of gates. So you can see we've got pair-wise  symmetry here and here. Those are probably the function blocks. There's something down  the middle. That's probably the global routing. Then there's a large memory right here. Pretty  obviously the EEPROM where the bitstream is stored. Then if we actually follow the bond  wires from the die out the pins in the package, you can see these are the JTAG pins, CDI, TMS,  TCKI, I believe top to bottom. The TDI pin is right there. So we can conclude that the  JTAG shift register probably runs left to right across this configuration area and somehow  allows you to write this EEPROM here. There's also a few small pairs. There's one up there.  There's one up there. I think there's one on the upper left. There's a total of six.  I have not yet figured out what these do. It wasn't necessary to reverse the portions  of the device I needed. So that remains future work. So let's take a closer look at the function  block. So we know there are 16 macros cells. Each macros cell contains one flip-flop and  stores the output of one term of the ORA and has a little bit more glue logic I'm going  to talk about later on. So there is 16 wide symmetry here. 16 identical copies of this.  Just by looking at this, we can be pretty sure we're looking at the macros cells. Then we  see some structures up here and here. This looks symmetric. This looks symmetric. This  looks symmetric but different. So as it turns out, the AND array is not actually one solid  block. The AND array is split in half. They've got 20 signals here, 20 signals here. Collectively  that forms 40. Then they AND the outputs of those two individual blocks together. That  gives you our product terms. And then that goes into the ORA here and then out the macros  cells. So now let's take a quick look at the configuration bit structure before we dive  into detailed die analysis. So the programming documentation does talk a little bit about  how the device is put together. It turns out that even though the JED format is supposed  to be something you can actually just feed to an EPROM program and load the device, that's  not the case here. The bit ordering in the JED file is actually virtual addressing in  which they abstract away all of the quarks of the silicon. So for example, all of the  AND array bits and all of the ORA array bits are in the same order in the JED file for each  one. Turns out half the actual AND and ORA array blocks on the device are mirrored left  right. So you actually have to do address translation before you can take a bit stream  generated by their tools and flash to the chip. Luckily they do sort of document this.  There's a big Excel file they publish to people who are making program error adapters that  is just a big grid of output and input and just has a intrate each cell. It says, okay,  which config bit from the JED file goes this physical address? So it doesn't really tell  you much about how the chip is put together. The actual structure is 48 rows by 260 columns.  There's one extra row full of configuration metadata. I'll get to that later. But it stores  seven lock bits, which some of them are always one. I'm not entirely sure what purpose they  serve. The remainder are one for an unlocked device and zero for a locked device. Then  there's two done bits, which indicate, okay, this bit stream is valid. We have a legal firmware  and the chip has been fully flashed. And it turns out also only 258 of the 260 columns  are usable. The remainder are what we call transfer bits, which as far as I can tell,  you just put them as a constant zero. It just indicates, hey, this row of EPROM has actually  been programmed. We didn't abort halfway through programming. There is some documentation  of this in a patent from Xilinx, but it didn't really give enough detail for me to figure  out, okay, why are they doing this? And the other intuition we can get from this is that  since it is EPROM based, then FF is going to be the state of the memory when the chip  is blank. So therefore, we expect most of the memory on the device is going to be active  low. So most things should be turned off when the bit is high and on when the bit is low.  We just don't know exactly what that bit does yet.  So now let's take a look at the actual die structure. So you can see that the configuration  memory is not actually one block. We've got an array here, one here, one here, one here,  and one hiding all the way over here. And as it turns out, the size of these are symmetric  about the center, and they go directly up to the corresponding logic. So it's pretty  obvious we found the memory that configures that part of the chip. The data just flows  straight up during the boot process. So this configures the end of the array, this configures  the global routing, this configures these macro cells, this configures these macro cells.  We can confirm this if we look up at the metal two. You can actually see the lines coming  off the send sample fires for the EPROM going up and vertically into the array. And if you  trace it out all the way, you can actually see it writing to the individual configuration  SRAM cells in the die. There are a few bit lines here and here, for example, that are  connected and I believe metal three instead of metal two. They couldn't fan it all out  in one layer.  So now let's take a look at the main logic array. If we actually look at where the SRAM  cells are and count how many rows high each individual block are, we see that the end  array is 20 rows high, the or arrays eight rows high, other half is 20 rows high, and  each macro cell is three rows high. So that gives us a pretty good idea. We know left,  right, which bits in the bit stream configure which logic just by what's physically proximate  to it. And we can make a pretty good guess that since we have a 2DS for MRA here in the  configuration area, we have a 2D flash array, we probably have either top to bottom or bottom  up addressing. There's actually some gray code going on. It's a little more complex than  that, but logically we should have the addressing going across vertically.  So if we take a closer look at the end array, we know that there are 56 product terms. We  know that there are supposed to be 40 rows and 40 complements of rows for a total of  80. If we actually count how many SRAM cells are, we'll see there's one 12 bits wide, just  56 times two. There's two blocks of 20 rows each. So the conclusion is that since we know  there's 40 inputs coming in from the crossbar, we've got two blocks that are each 20 rows  high, each row probably corresponds to one row of input. And then we know that the width  of the array is double 56. So it's probably two bits per product term. One selects X,  one selects non-X, or maybe some sort of other code. As it turns out, it is one-hot coded.  I just figured this out by experiment. It was pretty easy. Just try one, see if it works.  If not, try something else. There's only four possibilities.  So the or array turns out to be 56 and terms, 16 outputs. If we count the config bits, it's  still one 12 bits wide. It's the same, coming up off the same EEPROM, but it's only eight  rows high. And there's only 56 inputs. Since the or array does not have an X and X prime  input, my conclusion is that we still have one hot select thing for one particular input,  but we do have two actual arrays of the or array interleaved in one configuration row.  I got a little tinkering. It was pretty simple to figure out the actual bit ordering.  So now let's take a look at the macro cells. There are 27 configuration bits per macro  cell. If we look, you can see there's one 60S RAM cell, one 60S RAM cell, and so on.  There's a 9 by 3 grid. It turns out this 9 by 3 grid does not actually control one macro  cell. The bottom two rows of this RAM control the bottom one. This goes to the upper one.  This threw me off a little bit when I started looking at the circuit, but it turns out that  yes it is 27 cells, yes it is 9 by 3. No, the 9 by 3 structures in the die do not actually  correspond to this. So this makes intuitive sense. The EEPROM at the bottom of the die  is, well, okay, it's 10 bits wide, but we know one of them is a transfer bit. So the  other nine bits go directly up to here. I've figured out a significant fraction of the functionality,  not quite all. There's still some clocking stuff I'm a little unsure on right now. These  also configure IO buffering and stuff like that. So there are lines coming out from these  to the side of the device and controlling the IO buffers.  Now I will, as a quick aside, jump into the security bits. We know there are nine done  bits and lock bits somewhere on the device. We know that the physical address of these  is in the right hand macro cell memory. It's in the top row. There's nine 60S RAM cells  right here that don't appear to be hooked up to any of the actual logic array. I have  not actually tried fibbing these. I don't know for certain that these are the lock bits,  but it is pretty obvious. Especially when we know of those nine bits, four of them have  to be held low to lock the device. There's a four input NOR gate right here and an inverter  right there. Hmm, I wonder what happens when I cut that line. I don't know.  So now we'll get to the global routing. So we know between the left and the right halves  of the end array, there is something. We know that it is 20 configuration bits high for  each half. We know it's 16 bits wide. We don't know anything more than that. The data sheet  has about two sentences that talk about the global routing. So there is, we have no idea  whatsoever how it works. What we do know is that of those 65 signals coming in, we know  that 20 of them go to the left function block, 20 go to the right. Those subsets may or may  not have any relationship. And we know since it's 16 bits wide, we probably have eight  bits selecting what goes left and eight bits selecting what goes right. So now here's the  question. How do you make a 65 to one MUX with eight bits? If you just had zero to 65,  you'd only need seven bits. If it was one hot, you'd need 65 bits. So what sort of strange,  perverse code are they using here? The data sheet is of exactly no help. Well, time to  get dirty. So if we jump into the electron microscope and take a look at the implant layer  after dash-ups, so you can see these areas here show up raised. Those are P-channel. These  are N-channel. You can see that it's slightly lower. The raised areas are the individual  channels for the FETs. This is zoomed out. The original image is a lot higher resolution,  but I can't really fit it in the projector. So this is the implant layer. What we can  see immediately is that there's some sort of symmetry going on, both horizontally and  vertically. If we jump up to metal four, we see that there are six small buses of 11  signals each. The rightmost is 10. All right? Five times 11 plus 10. What's that, anybody?  65. Hmm. There are 65 signals going into global routing. I think we found them. So I spent  quite a while in Inkscape vectorizing this. My automated tools are not quite as well-developed  as I had hoped. So it did take me a while, but I do now have a full vectorization of  the entire global routing matrix from implant layer all the way up to metal four. The little  gray boxes are not actually layout. There are standard cell outlines. So we've got a inverter  here, another inverter here, another inverter here, and so on. So we can see that there  are six identical blocks going left to right here. There's two weird blocks over there that  are identical to each other at first glance, but not identical to the rest. Okay. This  is starting to get interesting. So those big drivers on either side are pretty clear  of the buffers driving the PLI. We've got a fan out of 112. We need a fairly big buffer.  There are, I believe, 20-some fingers on this last inverter. So that looks like a high fan  out driver. It's actually a three-stage inverter because the low-level stuff in the programmable  logic doesn't actually have the drive current to drive that much gate capacitance. So they  actually have a three-stage driver inverting it, inverting it again and increasing the  current once again to actually drive the output. And of those eight blocks, each one contains  two S-rem cells. So it seems to make sense that we've got eight blocks. We know that  we have eight bits controlling the left output. We have eight bits controlling the right output.  So we probably have one bit per block controlling what goes left and one bit controlling what  goes right. Then there's six identical blocks and there's six groups of wiring up in Metal  4. So there's probably some sort of correspondence going on here. Again, we don't know what it  is, but we know there's some relationship. So now let's take a look up at Metal 3. So  we can see there's power and ground routing here. We'll ignore all of that for now. But  there are sets of one, two, three, four, five, six vias per row. What's more interesting  is there are always six vias per row, but they're not all in the same place. So there  is, if we actually trace it out for all of the rows, each one of those vias is under  each of the Metal 4 groups. There is exactly one via in each row column intersection, but  not at the same place. So now I finally realized what is going on. The routing matrix is not  actually a full crossbar. It is a sparse crossbar in which of the 40 rows, each row can only  pick six of the 65 inputs. But there's a different subset for each one. And the end result is  that using all of these subsets, you can select any unique subset of 40 of the 65. But you  don't actually need a 40 by 65 crossbar. So now let's take a look at how the actual  implementation works. We have a pretty good hunch as to how this is structured at a high  level. We don't actually know what the implementation is. So I do apologize for some of the misalignment.  This was a quick tracing. And so not all of the vias and metal layers line up exactly.  This is not meant to be something you can clone the chip from. It's meant to be something  I can figure out how it works from. So you can see there's a big pass transistor here.  We've got the signal coming in from the upper layer here and here. And then we've got an  output of the mux here. There's an SREM cell that goes through a two input NOR gate and  then drives that. So if we put this all together, what we see is that each row is indeed an  8 to 1 tri-state box mux. So there's one pass transistor that selects each of the six possible  outputs for a row. Then there's a single discrete NMOS and a single discrete PMOS that lets  me drive constant 1 or constant 0 as well. It turns out that the driver for constant  1 is active high. All the rest are active low. So this means that a blank bit stream  of FF won't cause bus fights. It'll put all the outputs in a well-defined state. So this  makes sense. And all the other signals are active low. There's also one additional signal  I've called O-Gate. I have no idea exactly where it comes from, but I'm pretty sure that  it's used during the boot process to basically pause the device, don't drive any of the outputs.  It'll consume less power when the chip is idle. And it also prevents bus fights between drivers  that aren't fully configured. If we've got half the device flashed and half not flashed  during the boot process, we don't want to be driving signals onto nets that some other  thing might still be driving with the old firmware. So as long as the signal is high,  it gates all the outputs. And the rows are, again, not identical. But it turns out we  can actually do the routing using MaxFlow. We just create a source node for each of the  nets we want to route. We create a directed graph with paths for each of the legal connections  to each row. Then we create a sync node with however many nets that we want to draw out.  And we can just use MaxFlow to route everything. So as it turns out, the structure is pretty  much that. I do have a full schematic. This is not it. I've kind of truncated the bottom  to make it fit on the slides. But we've got a single configuration bit. We've got the  NOR, which I've actually drawn as a negative logic AND because that's really how it's  functioning. Output of that goes to a single PMOS pulling high. We've got an NMOS pulling  low. Then we've got MUX in top A0, MUX in top 90, and so on. Those are the actual hex  codes in the bit stream for selecting that particular input. So we make that row be  set to A0 if we wanted to write to this element. And I do also have a table in the source for  my tools that includes all the MUX settings for this device. So I now can fully control  the global routing matrix. But there's one last bit. We know the ordering of the inputs.  We know how to select one of the inputs and drive something. We don't know which input  is actually hooked up to which signal on this bus. So time to get dirty again. There were  a couple of options I considered, but it turned out that one of the simplest was to make a  few educated guesses about how things were structured. So for example, all of the macrocells  in function block one, they are flip flops, are probably in a contiguous order on the  bus. We don't know where on the bus they are, but they're probably contiguous. All of the  input pins for function block two are probably contiguous as well. So let's tinker around  a little bit. Okay. The inputs to function block two are probably contiguous. So there's  not really that many orderings for the global input pin and these four sets of inputs. Let's  just try tinkering and see which one works. So I went up to campus, hopped on the focused  ion beam, drilled a few holes in the insulator over some wires, and laid down some nice gigantic  20 micron square probe pads. And then I just started driving signals onto each net. Here's  the code I'm using. It's just a counter going from a 2 megahertz input dividing down to,  I believe, a 2 hertz LED. And then I have one signal before that. So 4 hertz is on a  specific flip flop that I've constrained the two. I was like, okay, here, function block  two, macrocell five. So let's see. If we probe function block two, macrocell five, or if  we put a signal on function block two, macrocell five, it's supposed to be 4 hertz. And we've  got a probe pad on this signal. Is it a 4 hertz square wave? Oh yeah, it is. All right.  I guess it's right. So it didn't take too long to figure out the actual bit ordering.  So here's the basic summary of the actual layout. We've got gates on metal one and poly. Metal  two is vertical routing, SRM bit lines, O-gate. Then horizontal routing on M3. Then M4 is  the input bus. It turns out the actual ordering is the GPIOs for function block one, the global  input, GPIOs for function block two, then the flip flops left to right. So now we know  pretty much enough to configure the PLA. Unfortunately, there's more to it. Because it turns out all  of the product terms, or nearly all of them, are dual purpose. They can be used as general  purpose logic. You can feed them into the OR array. But each of the 16 macrocells also  has three product terms that have separate dedicated connections to it. These are used  for set, reset, clock, enable, and so on. I'll cover some of this in the next slide. Four  more have special connections to the entire function block. And the last four, as far as  they can tell from reading the datasheet, they have no special purpose. The datasheet does  tell us these things exist. They tell us roughly what they do. They don't tell us which of  these 56 product terms does what. So I will just briefly cover what these terms are. So  we've got per function block, we have a local clock that we can use. So we don't want to  waste global clock resources for the whole chip. But we also don't want to have a per  flip flop clock because that will both increase skew and use a lot of product terms that we  really don't need. Then we have dedicated set and reset and dedicated output enables.  Then per macrocell, we've got product terms A, B, and C, very inventive naming. So product  term A is one of several legal sources for set, reset. There's also a, the control term  set and reset can be used instead if you want. There's product term B, which is one of several  possible sources for the IO buffer output enable. And then product term C, which can  be used for a couple of things. If you've got a clock enable, it's used as a clock  enable. If you want a per macrocell clock, for just one flip flop that you're clocking  off one weird clock that nothing else is clocked by, you can use that. And it also drives  an XOR gate. So it turns out that this isn't quite a conventional CPLD structure. The output  of the PLA is XOR with product term C or its complement or a constant one or constant zero.  This is a Xilinx proprietary optimization. That is, again, documented in the data sheet,  but they don't tell you how to configure in the bit stream. The intention here is to allow  you to do efficient adders. You've got XOR. You don't have to do not X or Y, not X and  Y or not Y and X. So you can just use the XOR gate. So the question is, okay, we've got  56 product terms. Which one is PTC? No idea. Well, it turns out we can configure the PLA  as much as you want. We already know how to configure that. We've reversed all the bets  for that. We can configure the global routing as much as you want. And it's fairly easy  to generate a bit stream that is known to use product term C from the tools. All we have  to do is either synthesize something with an XOR in it or it turns out there's actually  an optimization of the tools. If you are not using the OR array, just say we just have  Y equals A and B. It turns out that if we set the OR array to output a constant zero  and then we XOR that with product term C, we no longer have the OR array in our critical  path. So you can sheave about 500 picoseconds off the propagation delay by doing this. The  compiler trying to optimize speed does this by default if you're not using the OR array.  So therefore, just by creating an equation like Y equals X in your source code, you can  trivially produce a bit stream that uses product term C. So all we have to do is say, okay,  let's make something that uses product term C. We'll spam these black box macro cell bits  into all of the outputs and just start feeding inputs into various product terms until we  find one that hits. Well, it turns out it is zero based term 3N plus 10. Unfortunately,  I have things called customers that kept me from working on this project as much as I  would like. So I never actually figured out what product terms A and B are. I'm pretty  sure that they're either just above or just below product term C. I don't know which.  The control terms, if you remember, product term C is 3N plus 10. So below number 10,  none of these terms are actually being used for anything in PTABC. So the control terms  are probably among the first couple of terms. But again, I have not actually had time to  figure out which is which. One more wrinkle is that there are some global configuration  bits. So if you remember, we've got the structure of the die on the left side and the right  side. I've got the function blocks and the top. I've got global routing and the bottom.  I've got global routing. So that pretty much makes a big donut shape. In the middle of  that, there's wasted space. But no, it's not actually wasted. There's 22 single configuration  bits, the middle of the die. We know three of them control the VT for the IO banks. The  device has two IO banks, but the original XC2C32 only had one IO bank. So they did a  clever hack. They are now bitstream compatible with the 2C32. They added four more configuration  bits. So the way they did this is they have one global bit. If this bit is set, then it  will use the old school XC2C32 configuration setting for this. The newer ones are two additional  per bank bits. So it actually just does a bitwise add between the global bit and the  per bank bit. If you erase the device and you don't program these last couple of fuses,  then you will get the default XC2C32 behavior in which you've got one IO standard for everything.  If you're compiling for the 32A, now you set that other bit as a constant one. Leave the  other one set the way you want. And now you get the ability to set VT for each bank independently.  So it's actually pretty clever. Then there are other miscellaneous things that are there.  The comments in the bitstream were a little bit helpful for this. So I know there's three  bits that configure global clocking, two bits that configure global set reset, eight  that configure global output enable. I do not actually know what the coding for these bits  are at this point. So there's still some work in progress going on.  Now I will get to the actual tool chain that I'm producing. I've called it libcrowbar after  the flying crowbar project on SiliconPron. The goal is to, in general, produce open source  tools for programmable logic. It does need a lot of refactoring and tweaking. It does  not do nearly everything that I want. It only supports the 2C32A right now. We can scale  to other devices. The 64 cell device is pretty much just a scale up of the 32. I've decapped  it. I've looked at the top layer. It's just a 2x2 grid of function blocks instead of 1x2.  The ROM for the routing fabric will have to be decoded separately, but that shouldn't  take too long. The ordering of the bits will have to be decoded, but again, that shouldn't  take too long. I can probably just brute force it.  And the macro cell configuration and everything else is pretty much exactly the same. The  larger device is the 128, 256, 384, and 512. It adds some additional features to macro  cell. So in addition to figuring out the actual routing stuff, I will actually have to do  some additional reversing of the macro cell logic. So that remains a work in progress,  but that is a goal. The library is BSD licensed. You can grab the code from there. I do not  recommend attempting to use it in its current state. It's more of something to look at and  understand how the chip works. And maybe in a couple of months, it will have something  that's actually stable enough you can actually use. But I do have a quick usage example here.  So we have a bunch of IO pins. We select P8, P6, P38, P37, whatever. Then we get the  IOBs for each of these. We can now select the IO standards, some output enables. We're  not going to use termination. We're not going to use Schmidt trigger. These are all bits  in the macro cell that I was able to figure out. There are still quite a few related to  the clocking that remain unknown at this point. And then there's quite a bit more going on  below this. I can pull out full source if anybody is interested in looking at it. But  the end result is that I can go from an in-memory technology map net list created like this  as C++ objects. I can then place and route this, produce a valid bit stream, flash the  device and have it work. I can also go the other way around. I can go from a program  device. I can read out the bit stream. I can turn it into an ASCII art schematic. And  I can turn it into synthesizable Veralog as long as you're not using the features I haven't  yet reversed the functionality of. The remaining thing in the forward tool chain is I would  like to integrate with YOSIS for synthesis, take the output of YOSIS, do technology mapping  on that and then feed it into libcrowbar. At that point I will have a full Veralog to  bit stream tool chain. I'm not quite there yet, but it's coming close. The other tool  I've made, if any of you have used plan ahead from Xilinx, I called mine very inventively  FC plan for the flying crowbar floor planner. So it is a floor planner and physical layout  viewer. I currently only have rendering for the and array and the global routing. The  or array, I do have the functionality reversed. I have not written the code to actually draw  it, but I can actually open up a bit stream and look at the individual settings in the  and array. The macro cells also, I've reversed some of the bits, not all of them, but I have  not yet written code to actually render the appearance of them as check boxes for, okay,  instrument trigger is enabled, the terminator is enabled, the slew is fast or slow and so  on. So those all remain in the wish list at this point. The configuration bits are known.  You can access them from libcrowbar. You cannot do so in the GUI. So here's one more view  of FC plan. This is showing as you zoom out, the product terms shrink down to smaller columns.  You can see the full global routing matrix here. This is the actual via pattern. It doesn't  show up as well in the projector again as I would have liked, but you can see the actual  mux bits in there. When you zoom in in the tool, you can actually see the signals coming  in from the global routing going out through these vias and into the and an arrays. Before  we get the demo, I do want to thank John McMaster from SiliconPron. Hi, John. He did most of  the large scale optical imagery for this. He also did the dash etching, which was very  helpful. Then Ray Dove and Brian Colwell at RPI run the material science lab in the clean  room respectively. They were quite helpful when it came time to getting access to the  electron microscope in the clean room and so on. As far as I know, I am the only computer  science student there, whoever ran the electron microscopes. But I did actually get access  to the clean room, the SEM, the FIB. I was trained on all of them and they were quite  helpful. Then the SiliconPron team in general was quite handy when it came to just getting  feedback and sanity checking and does this look okay to you? I can't quite make out what  this connects to. Do you have any ideas? Or just sharing process suggestions, okay? What  are some ways I can get this edge to be a little more even? Something like that. Now,  I'm to attempt the demo gods. I have my JTAG daemonic connected already. This is my dev  board. It has a small FTI 232 on there that I'm using for JTAG. What I'm going to do  is I'm going to create a bit stream using lib crowbar that will act as a not gate and  a buffer. Then I'm going to bit bang inputs going into there with the FTI chip. We should  see one LED and then the other alternative lighting up. I don't know if you guys can  see. Right up here, these LEDs are blinking back and forth. If we scroll through the output  here of the tool, you can see we start out initializing. We connect to the JTAG chain.  We see okay, there is a device. One odd note, the cool runners are the only devices I've  ever actually looked at in which the JTAG device ID actually includes the package as  well as the device. What this means is that they've actually got a fuse somewhere in the  die that says, oh, this is a TQFP. This is a BGA. I've never seen that on any other device.  Any other FPGA or microcontroller, anything I've used, the device ID was the same between  different packages. Anyway, so we confirm, okay, yes, the STDI chip has GPIOs on it. We  generate our net list. We figure out the actual function block and macrocell locations of  the IO buffers we're using. Then it runs the fitter through IO banks, macrocells, function  blocks, does global routing, repeats for the other function block, does global routing  for that, confirms okay, fitting is complete, took 620 microseconds. If any of you have  used the Xilinx Compilers, they're not that fast. Then we finish generating the bit stream.  That's kind of slow. Right now, it took a couple of milliseconds. I'm pretty sure I  can optimize this. Monk, this is a debug build of profiling turned on. So it'll be a lot  faster if I actually compile the optimizations. Then we actually configure the device. We  verify, okay, yes, we are talking to the right device. We've got the right number of fuses.  We erase it. We make sure it's blank. Then we just bit bang the out as 0 and 1. It confirms  that the loop back going back to the FDI chip does have a value that we expect from one  of MsX and one of MsX prime. That looks good. Now, let's see what happens if we actually  want to reverse this. Let me see if I can up the font size. Can people read this? I'm  guessing no. Is that better? All right. So this is what happens when we actually try  dumping the bit stream. This is not the same bit stream. This is a different one. I just  picked that random I had sitting around. So we have a nice ask your schematic showing  all the inputs coming in from the PLA or coming in from the global routing. Each of these  represents one connection. That's one configuration, but it's actually connected to something. We  have the X and the X bar outputs. Nice big grid. Then 56 product terms to the or array.  We've got outputs going to the agris array. This table looks a lot nicer at 1080p when  your font isn't quite so big, but I do actually have details on, okay, this output is floating.  This one is configured as an input, this is an input, this is an input. There's an output  in here somewhere, probably on the other one. Whatever, I must have scrolled past it.  So now on the output, we have the IO standard. It turns out even though there's about a dozen  IO standards you can pick from its emphasis time, in the bit stream there's only two. There  is the high voltage and the low voltage. So there's 1.8 and 1.5 and there's high voltage.  So my guess is what this does internally. I have not fully reverse engineered all of  the IO driver's arbitrary, but what I'm pretty sure it does is it selects a single threshold  or another and then thicker a thinner gate oxide in the output. There's about four or  five different sets of drivers in the output for fast and slow, different voltages and  so on. I haven't actually done TEM cross sections and measured VTVs transistors or anything like  that. It wasn't necessary for this. And then we've got a bunch of stuff here for the global  clock muxes, global reset muxes and everything. And all this remains unknown. I know roughly,  I know which bits these are. I don't know what the functionality of those bits is.  And then, let's get the fun part. Let me actually switch to the adder test.jed file. This one  has a simpler, this is a 4-bit adder, which will make a lot more sense than the 32-bit  adder I was showing the other one. So here's our RTL. It is pretty much the assembly language  of AeroLog. This is a direct analog of the actual PLI structure. I have not attempted  to figure out any higher level structure. This is actually equivalent. I have not tested  synthesizing it. I actually just found a bug last night here on lock equals. That should  be quoted. So right now the output as formatted won't synthesize, but it would be fairly trivial  to fix that and make it actually synthesize and work. And then I've got function block  2. There's not really anything in there. So now before we get to the questions, I'm just  going to jump up to FC plan and show you guys what the actual structure looks like in the  floor planner. I guess I found a bug. The scroll on this thing is really sensitive. All right.  Can I go just one click in? All right. That's probably good enough. So you can see what  I've got. Initially we're bypassing all the inputs. So we're not using the first couple  of rows coming off the global routing. Then down here I've got one signal coming in. It's  hooked up over here to X. It's hooked over here to X crime. Goes into the AND gate. Then  we go down there and so on. So we do actually have a full physical layout viewer at this  point. So now I'm just going to jump back to the slides for just one or two more before  we get back to... So what remains as far as future work? We still have to figure out the  last of the special product terms. There still are about six or eight bits in each macrosail  that we have not yet figured out the functionality of. There's still some more in the global devices.  The global bits. As I did mention, there is a little bit more I need to do to support  larger devices. As far as the tool chain, I want to do more work in the decompiler.  My long-term goal with this work is to integrate with some other bitstream reversing projects  and so on and create the IDA for hardware. I would like to be able to go from bitstreams  for CoolRunner, bitstreams for Part 6, bitstreams for Stratix 4 and everything, and go from  that up to a device-dependent technology map net list. Abstract that up to a device-independent  net list. Something more along the lines of LLVM IR for hardware and then do higher-level  analysis on that to figure out, okay, this combination of XOR gates and MUX is looks  like a 32-bit adder. This combination of stuff looks like a 10-to-1 MUX. This combination  of stuff looks like a Cortex M3. I would like to eventually be able to have something along  the lines of flirt here. I do know that subgraph isomorphism is NP-complete. I don't know if  I can make a randomized or approximated algorithm that's fast enough that remains a topic for  future work, but it is on the wish list whether it is physically possible or not. So, at this  point I'll take questions. Seeing none, I guess we're done. Thank you.  Thank you.
recon2015-18-andrew-zonenberg-From-Silicon-to-Compiler_0_5639.png recon2015-18-andrew-zonenberg-From-Silicon-to-Compiler_5640_7415.png recon2015-18-andrew-zonenberg-From-Silicon-to-Compiler_7416_8495.png recon2015-18-andrew-zonenberg-From-Silicon-to-Compiler_8496_11399.png recon2015-18-andrew-zonenberg-From-Silicon-to-Compiler_11400_14963.png recon2015-18-andrew-zonenberg-From-Silicon-to-Compiler_14964_15719.png recon2015-18-andrew-zonenberg-From-Silicon-to-Compiler_15720_16223.png recon2015-18-andrew-zonenberg-From-Silicon-to-Compiler_16224_22895.png recon2015-18-andrew-zonenberg-From-Silicon-to-Compiler_22896_26291.png recon2015-18-andrew-zonenberg-From-Silicon-to-Compiler_26292_29927.png recon2015-18-andrew-zonenberg-From-Silicon-to-Compiler_29928_30611.png recon2015-18-andrew-zonenberg-From-Silicon-to-Compiler_30612_34835.png recon2015-18-andrew-zonenberg-From-Silicon-to-Compiler_34836_37679.png recon2015-18-andrew-zonenberg-From-Silicon-to-Compiler_37680_41339.png recon2015-18-andrew-zonenberg-From-Silicon-to-Compiler_41340_46727.png recon2015-18-andrew-zonenberg-From-Silicon-to-Compiler_46728_53615.png recon2015-18-andrew-zonenberg-From-Silicon-to-Compiler_53616_55439.png recon2015-18-andrew-zonenberg-From-Silicon-to-Compiler_55440_56903.png recon2015-18-andrew-zonenberg-From-Silicon-to-Compiler_56904_57023.png recon2015-18-andrew-zonenberg-From-Silicon-to-Compiler_57024_57119.png recon2015-18-andrew-zonenberg-From-Silicon-to-Compiler_57120_58235.png recon2015-18-andrew-zonenberg-From-Silicon-to-Compiler_58236_58379.png recon2015-18-andrew-zonenberg-From-Silicon-to-Compiler_58380_59531.png recon2015-18-andrew-zonenberg-From-Silicon-to-Compiler_59532_59627.png recon2015-18-andrew-zonenberg-From-Silicon-to-Compiler_59628_60371.png recon2015-18-andrew-zonenberg-From-Silicon-to-Compiler_60372_60431.png recon2015-18-andrew-zonenberg-From-Silicon-to-Compiler_60432_60731.png recon2015-18-andrew-zonenberg-From-Silicon-to-Compiler_60732_61583.png recon2015-18-andrew-zonenberg-From-Silicon-to-Compiler_61584_61955.png recon2015-18-andrew-zonenberg-From-Silicon-to-Compiler_61956_62003.png recon2015-18-andrew-zonenberg-From-Silicon-to-Compiler_62004_62051.png recon2015-18-andrew-zonenberg-From-Silicon-to-Compiler_62052_62219.png recon2015-18-andrew-zonenberg-From-Silicon-to-Compiler_62220_62267.png recon2015-18-andrew-zonenberg-From-Silicon-to-Compiler_62268_62315.png recon2015-18-andrew-zonenberg-From-Silicon-to-Compiler_62316_62375.png recon2015-18-andrew-zonenberg-From-Silicon-to-Compiler_62376_62423.png recon2015-18-andrew-zonenberg-From-Silicon-to-Compiler_62424_62471.png recon2015-18-andrew-zonenberg-From-Silicon-to-Compiler_62472_62579.png recon2015-18-andrew-zonenberg-From-Silicon-to-Compiler_62580_62639.png recon2015-18-andrew-zonenberg-From-Silicon-to-Compiler_62640_62687.png recon2015-18-andrew-zonenberg-From-Silicon-to-Compiler_62688_62771.png recon2015-18-andrew-zonenberg-From-Silicon-to-Compiler_62772_64295.png recon2015-18-andrew-zonenberg-From-Silicon-to-Compiler_64296_64343.png recon2015-18-andrew-zonenberg-From-Silicon-to-Compiler_64344_64391.png recon2015-18-andrew-zonenberg-From-Silicon-to-Compiler_64392_64439.png recon2015-18-andrew-zonenberg-From-Silicon-to-Compiler_64440_64535.png recon2015-18-andrew-zonenberg-From-Silicon-to-Compiler_64536_64595.png recon2015-18-andrew-zonenberg-From-Silicon-to-Compiler_64596_65495.png recon2015-18-andrew-zonenberg-From-Silicon-to-Compiler_65496_65555.png recon2015-18-andrew-zonenberg-From-Silicon-to-Compiler_65556_66551.png recon2015-18-andrew-zonenberg-From-Silicon-to-Compiler_66552_67055.png recon2015-18-andrew-zonenberg-From-Silicon-to-Compiler_67056_67499.png recon2015-18-andrew-zonenberg-From-Silicon-to-Compiler_67500_68951.png recon2015-18-andrew-zonenberg-From-Silicon-to-Compiler_68952_69359.png recon2015-18-andrew-zonenberg-From-Silicon-to-Compiler_69360_69707.png recon2015-18-andrew-zonenberg-From-Silicon-to-Compiler_69708_69776.png
You guys hear me now? Okay, sounds good. So my talk is called from Silicon to compiler and it's pretty much that. So we're going to start off with what we're doing, why we're doing this, a little bit of architectural background on programmable logic for those of you who have not done work on programmable logic before. Then jump to a block diagram of the device. We'll start with the high level overview of the silicon and then we'll drill down into more interesting stuff down to transistor level and gate level circuit analysis. At the end I do have a live demo of firmware produced by my tools running on live silicon. Sadly there are no cute pictures in this presentation, sorry. So a little background about me, I pretty much like to build and break everything. I will do web stuff if they pay me enough. I mostly like to live down in low level, a bit of ring zero, firmware, low level board design, RTL design and now getting down into transistor level. I just finished up my PhD a few weeks ago. During that time I designed and created what I believe is the first ever college class on semiconductor reverse engineering. I tried quite hard to find another one that I could borrow notes and slides from as far as I can tell none existed. And I'm also obviously a significant contributor to Silicon Prone. So if you see all the guys walking around with decap chips in their shirts, say hi. I've only been with IO active since January. A lot of this work was actually done before I joined the company but they've supported me continuing the work. So I acknowledge them for that obviously. So as far as what we're actually trying to do and why, program logic is really everywhere these days. You may not know it but a lot of especially high end networking AV, all sorts of stuff has program logic in it because A6 these days the tape out costs are so high especially for leading edge process nodes it usually is not actually cost effective to make custom silicon unless you're making a huge number of them like for a smart phone or something. For anything else you're probably better off running on an FPGA. And the problem is they're full of black boxes. Nobody really knows what happens when you compile your bit stream onto a device. You know okay well we've got these lookup tables, we've got these RAM blocks but what is actually going on underneath? How do I know that the RTLI gave the compiler is actually equivalent to what the actual device's behavior is? We don't know. Or how do I do development on a platform that is not x86 or x64 windows or one of the very short list of Linux distros they support? Sorry, out of luck. And let's say we think we found a compiler bug. Oh we just pop the code and obj dump and look at it. Nope there is no decompiler for the bit streams so if you think it's generating bad code, sorry you're screwed. And of course reverse engineering this is recon. The vendors want people to think bit stream reversing is hard. They advertise their closed source proprietary format as being impossible to reverse engineer. It's not. So as far as the methodology here, I just decided earlier today to take a look at the size of the directory I had installed the Xilinx tools in. 18 gigabytes. I don't know about you but I don't have time to look through 18 gigabytes of stuff in IDA. There are much better things I could spend my time on. And plus the license agreement says I'm not supposed to be a reverse engineering software. I never agreed not to look at the silicon. So open source spending means necessary. So this is our target. It is the Xilinx XC2C32A. I chose it for a bunch of reasons. One is that it is very cheap. They're about $1.15 each on Digi-key in single units as of a couple of weeks ago. I think the price just went up to $1.20 something. But they're still cheap enough you can afford to kill large numbers of them testing. You have your choice of a nice big QFP package that's easy to hand solder, has plenty of plastic around the package so it's easy to decap while keeping bond wires intact. Or you can do chip scale BGA or QFN if you just want to dunk the thing in acid and get rid of all the plastic and take a look at the silicon. It's a nice friendly 1.8 nanometer process for metal UMC. So you can actually read out most of the upper metal layers optically. You don't have to bother going into the electron microscope except for the lowest layers. So that helps. It's a lot easier to use an optical microscope in a SAM. It's a lot quicker to shoot a lot of images quickly. The bitstream is also not that big. It's around 12K. So there's not really that much better reverse. And it's also a fairly simple architecture. You're not going to find block ram and embedded arm cores and all kinds of stuff like that. It's just pure programmable logic. And the two vendor tools are free which always helps. Hackers are cheap. So the bitstream format for this thing at a high level, the output of the compiler is a JDEK programming file. It's basically the equivalent of IHACS but instead of being HACS lines, it's binary lines, ASCII 1s and 0s. So this is the exact data that gets written to the chip. Not in the actual order. I'll get to that later. But it is, there is a one-to-one correspondence between these 1s and 0s and configuration bits on the device. And the nice thing is that, as you can see on the right side here, the bitstream generated by this I-Link's tools, if you're so inclined, does have comments. Turns out they're not very useful. They just say, okay, block zero ZIA. Well, what is a ZIA? Look in the datasheet. There is not one reference to ZIA. Okay. It doesn't really help me much. So let's get a little bit into the architecture of what a CPLD actually is. So any digital equations can be expressed using a form known as sum of products. It's a canonical representation of digital equations in which you take a series of terms, you bitwise and them together, either of the input or its complement. You can just treat the complement as coming into your device. So you've got A and B and C or C and D and E, et cetera. So you can express any arbitrary digital equation this way. So let's look at how we'll actually make a chip to do this. So if we have a large, say, 32 input AND gate and we have a MUX at the input to the gate, that lets me select between a constant one or the input to my circuit, I can effectively MUX any subset of these inputs and just leave the others as one, which is the identity for the bitwise AND operation. So I can AND together any subset of the possible inputs to this AND gate. The same thing can be done for OR. Obviously, you have to use zero as the identity element for bitwise OR. But what this means is you can create a gate that has a huge number of inputs and I can pick any subset from those at runtime with MUX settings. So this leads to a natural structure for a programmable logic device. You make a grid of gates with a 2-1 MUX at each input. You've got inputs coming in, outputs going down, and then you just have a series of gates in sequence and stuff together. So this is actually a render for my tool. You can see we've got the input coming in. We select either X or X prime and that with the data coming in from overhead. Take the output of that. Now here we skip the AND gate. This is more of a logical view, so I'm not actually showing the actual MUXes in the chip. This is more of a schematic view of what the circuit logically does. I'm also rendering it as a cascaded sequence of gates end to end. In practice, the actual implementation on silicon is usually a tree because nobody wants order AND propagation delay when you could have order log AND instead. So if we take this and we want to build a full programmable logic device, what we do is we take a bunch of signals coming from our registers that store state. We take a bunch of signals from our input pins. Then we feed these into an AND array. We invert them so we have two times however many inputs. We take M product terms out of that. The product terms then go into a programmable array. The output of that gives us, we'll just call it our outputs. Then we feed those either into an output pin or into a flip-flop of one of the state machines or something like that. So this is really all it takes to build a simple programmable logic device or SPLD. The problem is SPLDs scale poorly. You end up having quadratic scaling with the number of inputs. Nobody wants to die whose size scales quadratically with the amount of work it can do. That's terrible. So how can we improve this? Well, it turns out we can make a grid of small SPLDs. So it doesn't show up as well as I had hoped on the projector, but we've got one SPLD block here. We've got one here and there's a crossbar switch in the middle. So now we can create a bunch of outputs from one SPLD, a bunch of outputs from the other, and then just feed them all into one big routing fabric, pick out whichever signals I'm interested in processing on this half of the chip or that half of the chip, and then feed them into the SPLD. So this is a SPLD. So now let's look at the specific SPLD that we've been targeting today. So there are 32 GPIO pins which are full input output plus one input only pin. I'm pretty sure this is because they intended to package the die in a 44 pin QFP package. All of the other stuff for JTAG power and the 32 GPIOs use 43 pins. They didn't want to have one NC pin. So they just threw one extra signal into the global routing that you can't drive and it just serves as an extra input. And then the remainder of the device is two function blocks which are basically SPLDs. Each one has 16 GPIO pins, 16 flip flops. Of the 65 signals in the global routing, so 32 IOs, 32 flip flops plus one input only 65. We pick 40 of those, feed those into an 80 by 56 and array, then a 56 by 16 array. This is all documented in the data sheet. What's not documented is, okay, we can make a technology math net list. We know, okay, I have these inputs added together, these outputs or together. Now, how do I actually make the chip do what I want? So time to put in our lab codes. I am not going to talk about decapping and imaging. This has been beaten to death in other talks like recon last year and I think the year before that and probably the year before that. So we're not going to cover that. There's a lot of good stuff up on SiliconPron. There's, if you read the lecture notes from the class I taught at RPI last year, there's a lot of good material there. But this talk is about reverse engineering, not on sample preparation. We're not going to be teaching you how IDA works. We're teaching you how to actually reverse engineer this specific device. So here is the metal 4 overview of the device. We can see that there is a roughly left-right symmetry. It's not exact. You can see here there's something that's not mirrored here, but up here it looks like the device is pretty symmetrical. So first impression, we've probably got the two function blocks left-right symmetric on the device. Let's go down a little bit. After we've etched off all of the metal and poly layers, we're now looking at the implant layer. This is after a process known as dash-etch, which stains P-type doping either brown or it shows up raised under the electron microscope. Mainly, though, it's useful because it provides contrast in areas of gates. So you can see we've got pair-wise symmetry here and here. Those are probably the function blocks. There's something down the middle. That's probably the global routing. Then there's a large memory right here. Pretty obviously the EEPROM where the bitstream is stored. Then if we actually follow the bond wires from the die out the pins in the package, you can see these are the JTAG pins, CDI, TMS, TCKI, I believe top to bottom. The TDI pin is right there. So we can conclude that the JTAG shift register probably runs left to right across this configuration area and somehow allows you to write this EEPROM here. There's also a few small pairs. There's one up there. There's one up there. I think there's one on the upper left. There's a total of six. I have not yet figured out what these do. It wasn't necessary to reverse the portions of the device I needed. So that remains future work. So let's take a closer look at the function block. So we know there are 16 macros cells. Each macros cell contains one flip-flop and stores the output of one term of the ORA and has a little bit more glue logic I'm going to talk about later on. So there is 16 wide symmetry here. 16 identical copies of this. Just by looking at this, we can be pretty sure we're looking at the macros cells. Then we see some structures up here and here. This looks symmetric. This looks symmetric. This looks symmetric but different. So as it turns out, the AND array is not actually one solid block. The AND array is split in half. They've got 20 signals here, 20 signals here. Collectively that forms 40. Then they AND the outputs of those two individual blocks together. That gives you our product terms. And then that goes into the ORA here and then out the macros cells. So now let's take a quick look at the configuration bit structure before we dive into detailed die analysis. So the programming documentation does talk a little bit about how the device is put together. It turns out that even though the JED format is supposed to be something you can actually just feed to an EPROM program and load the device, that's not the case here. The bit ordering in the JED file is actually virtual addressing in which they abstract away all of the quarks of the silicon. So for example, all of the AND array bits and all of the ORA array bits are in the same order in the JED file for each one. Turns out half the actual AND and ORA array blocks on the device are mirrored left right. So you actually have to do address translation before you can take a bit stream generated by their tools and flash to the chip. Luckily they do sort of document this. There's a big Excel file they publish to people who are making program error adapters that is just a big grid of output and input and just has a intrate each cell. It says, okay, which config bit from the JED file goes this physical address? So it doesn't really tell you much about how the chip is put together. The actual structure is 48 rows by 260 columns. There's one extra row full of configuration metadata. I'll get to that later. But it stores seven lock bits, which some of them are always one. I'm not entirely sure what purpose they serve. The remainder are one for an unlocked device and zero for a locked device. Then there's two done bits, which indicate, okay, this bit stream is valid. We have a legal firmware and the chip has been fully flashed. And it turns out also only 258 of the 260 columns are usable. The remainder are what we call transfer bits, which as far as I can tell, you just put them as a constant zero. It just indicates, hey, this row of EPROM has actually been programmed. We didn't abort halfway through programming. There is some documentation of this in a patent from Xilinx, but it didn't really give enough detail for me to figure out, okay, why are they doing this? And the other intuition we can get from this is that since it is EPROM based, then FF is going to be the state of the memory when the chip is blank. So therefore, we expect most of the memory on the device is going to be active low. So most things should be turned off when the bit is high and on when the bit is low. We just don't know exactly what that bit does yet. So now let's take a look at the actual die structure. So you can see that the configuration memory is not actually one block. We've got an array here, one here, one here, one here, and one hiding all the way over here. And as it turns out, the size of these are symmetric about the center, and they go directly up to the corresponding logic. So it's pretty obvious we found the memory that configures that part of the chip. The data just flows straight up during the boot process. So this configures the end of the array, this configures the global routing, this configures these macro cells, this configures these macro cells. We can confirm this if we look up at the metal two. You can actually see the lines coming off the send sample fires for the EPROM going up and vertically into the array. And if you trace it out all the way, you can actually see it writing to the individual configuration SRAM cells in the die. There are a few bit lines here and here, for example, that are connected and I believe metal three instead of metal two. They couldn't fan it all out in one layer. So now let's take a look at the main logic array. If we actually look at where the SRAM cells are and count how many rows high each individual block are, we see that the end array is 20 rows high, the or arrays eight rows high, other half is 20 rows high, and each macro cell is three rows high. So that gives us a pretty good idea. We know left, right, which bits in the bit stream configure which logic just by what's physically proximate to it. And we can make a pretty good guess that since we have a 2DS for MRA here in the configuration area, we have a 2D flash array, we probably have either top to bottom or bottom up addressing. There's actually some gray code going on. It's a little more complex than that, but logically we should have the addressing going across vertically. So if we take a closer look at the end array, we know that there are 56 product terms. We know that there are supposed to be 40 rows and 40 complements of rows for a total of 80. If we actually count how many SRAM cells are, we'll see there's one 12 bits wide, just 56 times two. There's two blocks of 20 rows each. So the conclusion is that since we know there's 40 inputs coming in from the crossbar, we've got two blocks that are each 20 rows high, each row probably corresponds to one row of input. And then we know that the width of the array is double 56. So it's probably two bits per product term. One selects X, one selects non-X, or maybe some sort of other code. As it turns out, it is one-hot coded. I just figured this out by experiment. It was pretty easy. Just try one, see if it works. If not, try something else. There's only four possibilities. So the or array turns out to be 56 and terms, 16 outputs. If we count the config bits, it's still one 12 bits wide. It's the same, coming up off the same EEPROM, but it's only eight rows high. And there's only 56 inputs. Since the or array does not have an X and X prime input, my conclusion is that we still have one hot select thing for one particular input, but we do have two actual arrays of the or array interleaved in one configuration row. I got a little tinkering. It was pretty simple to figure out the actual bit ordering. So now let's take a look at the macro cells. There are 27 configuration bits per macro cell. If we look, you can see there's one 60S RAM cell, one 60S RAM cell, and so on. There's a 9 by 3 grid. It turns out this 9 by 3 grid does not actually control one macro cell. The bottom two rows of this RAM control the bottom one. This goes to the upper one. This threw me off a little bit when I started looking at the circuit, but it turns out that yes it is 27 cells, yes it is 9 by 3. No, the 9 by 3 structures in the die do not actually correspond to this. So this makes intuitive sense. The EEPROM at the bottom of the die is, well, okay, it's 10 bits wide, but we know one of them is a transfer bit. So the other nine bits go directly up to here. I've figured out a significant fraction of the functionality, not quite all. There's still some clocking stuff I'm a little unsure on right now. These also configure IO buffering and stuff like that. So there are lines coming out from these to the side of the device and controlling the IO buffers. Now I will, as a quick aside, jump into the security bits. We know there are nine done bits and lock bits somewhere on the device. We know that the physical address of these is in the right hand macro cell memory. It's in the top row. There's nine 60S RAM cells right here that don't appear to be hooked up to any of the actual logic array. I have not actually tried fibbing these. I don't know for certain that these are the lock bits, but it is pretty obvious. Especially when we know of those nine bits, four of them have to be held low to lock the device. There's a four input NOR gate right here and an inverter right there. Hmm, I wonder what happens when I cut that line. I don't know. So now we'll get to the global routing. So we know between the left and the right halves of the end array, there is something. We know that it is 20 configuration bits high for each half. We know it's 16 bits wide. We don't know anything more than that. The data sheet has about two sentences that talk about the global routing. So there is, we have no idea whatsoever how it works. What we do know is that of those 65 signals coming in, we know that 20 of them go to the left function block, 20 go to the right. Those subsets may or may not have any relationship. And we know since it's 16 bits wide, we probably have eight bits selecting what goes left and eight bits selecting what goes right. So now here's the question. How do you make a 65 to one MUX with eight bits? If you just had zero to 65, you'd only need seven bits. If it was one hot, you'd need 65 bits. So what sort of strange, perverse code are they using here? The data sheet is of exactly no help. Well, time to get dirty. So if we jump into the electron microscope and take a look at the implant layer after dash-ups, so you can see these areas here show up raised. Those are P-channel. These are N-channel. You can see that it's slightly lower. The raised areas are the individual channels for the FETs. This is zoomed out. The original image is a lot higher resolution, but I can't really fit it in the projector. So this is the implant layer. What we can see immediately is that there's some sort of symmetry going on, both horizontally and vertically. If we jump up to metal four, we see that there are six small buses of 11 signals each. The rightmost is 10. All right? Five times 11 plus 10. What's that, anybody? 65. Hmm. There are 65 signals going into global routing. I think we found them. So I spent quite a while in Inkscape vectorizing this. My automated tools are not quite as well-developed as I had hoped. So it did take me a while, but I do now have a full vectorization of the entire global routing matrix from implant layer all the way up to metal four. The little gray boxes are not actually layout. There are standard cell outlines. So we've got a inverter here, another inverter here, another inverter here, and so on. So we can see that there are six identical blocks going left to right here. There's two weird blocks over there that are identical to each other at first glance, but not identical to the rest. Okay. This is starting to get interesting. So those big drivers on either side are pretty clear of the buffers driving the PLI. We've got a fan out of 112. We need a fairly big buffer. There are, I believe, 20-some fingers on this last inverter. So that looks like a high fan out driver. It's actually a three-stage inverter because the low-level stuff in the programmable logic doesn't actually have the drive current to drive that much gate capacitance. So they actually have a three-stage driver inverting it, inverting it again and increasing the current once again to actually drive the output. And of those eight blocks, each one contains two S-rem cells. So it seems to make sense that we've got eight blocks. We know that we have eight bits controlling the left output. We have eight bits controlling the right output. So we probably have one bit per block controlling what goes left and one bit controlling what goes right. Then there's six identical blocks and there's six groups of wiring up in Metal 4. So there's probably some sort of correspondence going on here. Again, we don't know what it is, but we know there's some relationship. So now let's take a look up at Metal 3. So we can see there's power and ground routing here. We'll ignore all of that for now. But there are sets of one, two, three, four, five, six vias per row. What's more interesting is there are always six vias per row, but they're not all in the same place. So there is, if we actually trace it out for all of the rows, each one of those vias is under each of the Metal 4 groups. There is exactly one via in each row column intersection, but not at the same place. So now I finally realized what is going on. The routing matrix is not actually a full crossbar. It is a sparse crossbar in which of the 40 rows, each row can only pick six of the 65 inputs. But there's a different subset for each one. And the end result is that using all of these subsets, you can select any unique subset of 40 of the 65. But you don't actually need a 40 by 65 crossbar. So now let's take a look at how the actual implementation works. We have a pretty good hunch as to how this is structured at a high level. We don't actually know what the implementation is. So I do apologize for some of the misalignment. This was a quick tracing. And so not all of the vias and metal layers line up exactly. This is not meant to be something you can clone the chip from. It's meant to be something I can figure out how it works from. So you can see there's a big pass transistor here. We've got the signal coming in from the upper layer here and here. And then we've got an output of the mux here. There's an SREM cell that goes through a two input NOR gate and then drives that. So if we put this all together, what we see is that each row is indeed an 8 to 1 tri-state box mux. So there's one pass transistor that selects each of the six possible outputs for a row. Then there's a single discrete NMOS and a single discrete PMOS that lets me drive constant 1 or constant 0 as well. It turns out that the driver for constant 1 is active high. All the rest are active low. So this means that a blank bit stream of FF won't cause bus fights. It'll put all the outputs in a well-defined state. So this makes sense. And all the other signals are active low. There's also one additional signal I've called O-Gate. I have no idea exactly where it comes from, but I'm pretty sure that it's used during the boot process to basically pause the device, don't drive any of the outputs. It'll consume less power when the chip is idle. And it also prevents bus fights between drivers that aren't fully configured. If we've got half the device flashed and half not flashed during the boot process, we don't want to be driving signals onto nets that some other thing might still be driving with the old firmware. So as long as the signal is high, it gates all the outputs. And the rows are, again, not identical. But it turns out we can actually do the routing using MaxFlow. We just create a source node for each of the nets we want to route. We create a directed graph with paths for each of the legal connections to each row. Then we create a sync node with however many nets that we want to draw out. And we can just use MaxFlow to route everything. So as it turns out, the structure is pretty much that. I do have a full schematic. This is not it. I've kind of truncated the bottom to make it fit on the slides. But we've got a single configuration bit. We've got the NOR, which I've actually drawn as a negative logic AND because that's really how it's functioning. Output of that goes to a single PMOS pulling high. We've got an NMOS pulling low. Then we've got MUX in top A0, MUX in top 90, and so on. Those are the actual hex codes in the bit stream for selecting that particular input. So we make that row be set to A0 if we wanted to write to this element. And I do also have a table in the source for my tools that includes all the MUX settings for this device. So I now can fully control the global routing matrix. But there's one last bit. We know the ordering of the inputs. We know how to select one of the inputs and drive something. We don't know which input is actually hooked up to which signal on this bus. So time to get dirty again. There were a couple of options I considered, but it turned out that one of the simplest was to make a few educated guesses about how things were structured. So for example, all of the macrocells in function block one, they are flip flops, are probably in a contiguous order on the bus. We don't know where on the bus they are, but they're probably contiguous. All of the input pins for function block two are probably contiguous as well. So let's tinker around a little bit. Okay. The inputs to function block two are probably contiguous. So there's not really that many orderings for the global input pin and these four sets of inputs. Let's just try tinkering and see which one works. So I went up to campus, hopped on the focused ion beam, drilled a few holes in the insulator over some wires, and laid down some nice gigantic 20 micron square probe pads. And then I just started driving signals onto each net. Here's the code I'm using. It's just a counter going from a 2 megahertz input dividing down to, I believe, a 2 hertz LED. And then I have one signal before that. So 4 hertz is on a specific flip flop that I've constrained the two. I was like, okay, here, function block two, macrocell five. So let's see. If we probe function block two, macrocell five, or if we put a signal on function block two, macrocell five, it's supposed to be 4 hertz. And we've got a probe pad on this signal. Is it a 4 hertz square wave? Oh yeah, it is. All right. I guess it's right. So it didn't take too long to figure out the actual bit ordering. So here's the basic summary of the actual layout. We've got gates on metal one and poly. Metal two is vertical routing, SRM bit lines, O-gate. Then horizontal routing on M3. Then M4 is the input bus. It turns out the actual ordering is the GPIOs for function block one, the global input, GPIOs for function block two, then the flip flops left to right. So now we know pretty much enough to configure the PLA. Unfortunately, there's more to it. Because it turns out all of the product terms, or nearly all of them, are dual purpose. They can be used as general purpose logic. You can feed them into the OR array. But each of the 16 macrocells also has three product terms that have separate dedicated connections to it. These are used for set, reset, clock, enable, and so on. I'll cover some of this in the next slide. Four more have special connections to the entire function block. And the last four, as far as they can tell from reading the datasheet, they have no special purpose. The datasheet does tell us these things exist. They tell us roughly what they do. They don't tell us which of these 56 product terms does what. So I will just briefly cover what these terms are. So we've got per function block, we have a local clock that we can use. So we don't want to waste global clock resources for the whole chip. But we also don't want to have a per flip flop clock because that will both increase skew and use a lot of product terms that we really don't need. Then we have dedicated set and reset and dedicated output enables. Then per macrocell, we've got product terms A, B, and C, very inventive naming. So product term A is one of several legal sources for set, reset. There's also a, the control term set and reset can be used instead if you want. There's product term B, which is one of several possible sources for the IO buffer output enable. And then product term C, which can be used for a couple of things. If you've got a clock enable, it's used as a clock enable. If you want a per macrocell clock, for just one flip flop that you're clocking off one weird clock that nothing else is clocked by, you can use that. And it also drives an XOR gate. So it turns out that this isn't quite a conventional CPLD structure. The output of the PLA is XOR with product term C or its complement or a constant one or constant zero. This is a Xilinx proprietary optimization. That is, again, documented in the data sheet, but they don't tell you how to configure in the bit stream. The intention here is to allow you to do efficient adders. You've got XOR. You don't have to do not X or Y, not X and Y or not Y and X. So you can just use the XOR gate. So the question is, okay, we've got 56 product terms. Which one is PTC? No idea. Well, it turns out we can configure the PLA as much as you want. We already know how to configure that. We've reversed all the bets for that. We can configure the global routing as much as you want. And it's fairly easy to generate a bit stream that is known to use product term C from the tools. All we have to do is either synthesize something with an XOR in it or it turns out there's actually an optimization of the tools. If you are not using the OR array, just say we just have Y equals A and B. It turns out that if we set the OR array to output a constant zero and then we XOR that with product term C, we no longer have the OR array in our critical path. So you can sheave about 500 picoseconds off the propagation delay by doing this. The compiler trying to optimize speed does this by default if you're not using the OR array. So therefore, just by creating an equation like Y equals X in your source code, you can trivially produce a bit stream that uses product term C. So all we have to do is say, okay, let's make something that uses product term C. We'll spam these black box macro cell bits into all of the outputs and just start feeding inputs into various product terms until we find one that hits. Well, it turns out it is zero based term 3N plus 10. Unfortunately, I have things called customers that kept me from working on this project as much as I would like. So I never actually figured out what product terms A and B are. I'm pretty sure that they're either just above or just below product term C. I don't know which. The control terms, if you remember, product term C is 3N plus 10. So below number 10, none of these terms are actually being used for anything in PTABC. So the control terms are probably among the first couple of terms. But again, I have not actually had time to figure out which is which. One more wrinkle is that there are some global configuration bits. So if you remember, we've got the structure of the die on the left side and the right side. I've got the function blocks and the top. I've got global routing and the bottom. I've got global routing. So that pretty much makes a big donut shape. In the middle of that, there's wasted space. But no, it's not actually wasted. There's 22 single configuration bits, the middle of the die. We know three of them control the VT for the IO banks. The device has two IO banks, but the original XC2C32 only had one IO bank. So they did a clever hack. They are now bitstream compatible with the 2C32. They added four more configuration bits. So the way they did this is they have one global bit. If this bit is set, then it will use the old school XC2C32 configuration setting for this. The newer ones are two additional per bank bits. So it actually just does a bitwise add between the global bit and the per bank bit. If you erase the device and you don't program these last couple of fuses, then you will get the default XC2C32 behavior in which you've got one IO standard for everything. If you're compiling for the 32A, now you set that other bit as a constant one. Leave the other one set the way you want. And now you get the ability to set VT for each bank independently. So it's actually pretty clever. Then there are other miscellaneous things that are there. The comments in the bitstream were a little bit helpful for this. So I know there's three bits that configure global clocking, two bits that configure global set reset, eight that configure global output enable. I do not actually know what the coding for these bits are at this point. So there's still some work in progress going on. Now I will get to the actual tool chain that I'm producing. I've called it libcrowbar after the flying crowbar project on SiliconPron. The goal is to, in general, produce open source tools for programmable logic. It does need a lot of refactoring and tweaking. It does not do nearly everything that I want. It only supports the 2C32A right now. We can scale to other devices. The 64 cell device is pretty much just a scale up of the 32. I've decapped it. I've looked at the top layer. It's just a 2x2 grid of function blocks instead of 1x2. The ROM for the routing fabric will have to be decoded separately, but that shouldn't take too long. The ordering of the bits will have to be decoded, but again, that shouldn't take too long. I can probably just brute force it. And the macro cell configuration and everything else is pretty much exactly the same. The larger device is the 128, 256, 384, and 512. It adds some additional features to macro cell. So in addition to figuring out the actual routing stuff, I will actually have to do some additional reversing of the macro cell logic. So that remains a work in progress, but that is a goal. The library is BSD licensed. You can grab the code from there. I do not recommend attempting to use it in its current state. It's more of something to look at and understand how the chip works. And maybe in a couple of months, it will have something that's actually stable enough you can actually use. But I do have a quick usage example here. So we have a bunch of IO pins. We select P8, P6, P38, P37, whatever. Then we get the IOBs for each of these. We can now select the IO standards, some output enables. We're not going to use termination. We're not going to use Schmidt trigger. These are all bits in the macro cell that I was able to figure out. There are still quite a few related to the clocking that remain unknown at this point. And then there's quite a bit more going on below this. I can pull out full source if anybody is interested in looking at it. But the end result is that I can go from an in-memory technology map net list created like this as C++ objects. I can then place and route this, produce a valid bit stream, flash the device and have it work. I can also go the other way around. I can go from a program device. I can read out the bit stream. I can turn it into an ASCII art schematic. And I can turn it into synthesizable Veralog as long as you're not using the features I haven't yet reversed the functionality of. The remaining thing in the forward tool chain is I would like to integrate with YOSIS for synthesis, take the output of YOSIS, do technology mapping on that and then feed it into libcrowbar. At that point I will have a full Veralog to bit stream tool chain. I'm not quite there yet, but it's coming close. The other tool I've made, if any of you have used plan ahead from Xilinx, I called mine very inventively FC plan for the flying crowbar floor planner. So it is a floor planner and physical layout viewer. I currently only have rendering for the and array and the global routing. The or array, I do have the functionality reversed. I have not written the code to actually draw it, but I can actually open up a bit stream and look at the individual settings in the and array. The macro cells also, I've reversed some of the bits, not all of them, but I have not yet written code to actually render the appearance of them as check boxes for, okay, instrument trigger is enabled, the terminator is enabled, the slew is fast or slow and so on. So those all remain in the wish list at this point. The configuration bits are known. You can access them from libcrowbar. You cannot do so in the GUI. So here's one more view of FC plan. This is showing as you zoom out, the product terms shrink down to smaller columns. You can see the full global routing matrix here. This is the actual via pattern. It doesn't show up as well in the projector again as I would have liked, but you can see the actual mux bits in there. When you zoom in in the tool, you can actually see the signals coming in from the global routing going out through these vias and into the and an arrays. Before we get the demo, I do want to thank John McMaster from SiliconPron. Hi, John. He did most of the large scale optical imagery for this. He also did the dash etching, which was very helpful. Then Ray Dove and Brian Colwell at RPI run the material science lab in the clean room respectively. They were quite helpful when it came time to getting access to the electron microscope in the clean room and so on. As far as I know, I am the only computer science student there, whoever ran the electron microscopes. But I did actually get access to the clean room, the SEM, the FIB. I was trained on all of them and they were quite helpful. Then the SiliconPron team in general was quite handy when it came to just getting feedback and sanity checking and does this look okay to you? I can't quite make out what this connects to. Do you have any ideas? Or just sharing process suggestions, okay? What are some ways I can get this edge to be a little more even? Something like that. Now, I'm to attempt the demo gods. I have my JTAG daemonic connected already. This is my dev board. It has a small FTI 232 on there that I'm using for JTAG. What I'm going to do is I'm going to create a bit stream using lib crowbar that will act as a not gate and a buffer. Then I'm going to bit bang inputs going into there with the FTI chip. We should see one LED and then the other alternative lighting up. I don't know if you guys can see. Right up here, these LEDs are blinking back and forth. If we scroll through the output here of the tool, you can see we start out initializing. We connect to the JTAG chain. We see okay, there is a device. One odd note, the cool runners are the only devices I've ever actually looked at in which the JTAG device ID actually includes the package as well as the device. What this means is that they've actually got a fuse somewhere in the die that says, oh, this is a TQFP. This is a BGA. I've never seen that on any other device. Any other FPGA or microcontroller, anything I've used, the device ID was the same between different packages. Anyway, so we confirm, okay, yes, the STDI chip has GPIOs on it. We generate our net list. We figure out the actual function block and macrocell locations of the IO buffers we're using. Then it runs the fitter through IO banks, macrocells, function blocks, does global routing, repeats for the other function block, does global routing for that, confirms okay, fitting is complete, took 620 microseconds. If any of you have used the Xilinx Compilers, they're not that fast. Then we finish generating the bit stream. That's kind of slow. Right now, it took a couple of milliseconds. I'm pretty sure I can optimize this. Monk, this is a debug build of profiling turned on. So it'll be a lot faster if I actually compile the optimizations. Then we actually configure the device. We verify, okay, yes, we are talking to the right device. We've got the right number of fuses. We erase it. We make sure it's blank. Then we just bit bang the out as 0 and 1. It confirms that the loop back going back to the FDI chip does have a value that we expect from one of MsX and one of MsX prime. That looks good. Now, let's see what happens if we actually want to reverse this. Let me see if I can up the font size. Can people read this? I'm guessing no. Is that better? All right. So this is what happens when we actually try dumping the bit stream. This is not the same bit stream. This is a different one. I just picked that random I had sitting around. So we have a nice ask your schematic showing all the inputs coming in from the PLA or coming in from the global routing. Each of these represents one connection. That's one configuration, but it's actually connected to something. We have the X and the X bar outputs. Nice big grid. Then 56 product terms to the or array. We've got outputs going to the agris array. This table looks a lot nicer at 1080p when your font isn't quite so big, but I do actually have details on, okay, this output is floating. This one is configured as an input, this is an input, this is an input. There's an output in here somewhere, probably on the other one. Whatever, I must have scrolled past it. So now on the output, we have the IO standard. It turns out even though there's about a dozen IO standards you can pick from its emphasis time, in the bit stream there's only two. There is the high voltage and the low voltage. So there's 1.8 and 1.5 and there's high voltage. So my guess is what this does internally. I have not fully reverse engineered all of the IO driver's arbitrary, but what I'm pretty sure it does is it selects a single threshold or another and then thicker a thinner gate oxide in the output. There's about four or five different sets of drivers in the output for fast and slow, different voltages and so on. I haven't actually done TEM cross sections and measured VTVs transistors or anything like that. It wasn't necessary for this. And then we've got a bunch of stuff here for the global clock muxes, global reset muxes and everything. And all this remains unknown. I know roughly, I know which bits these are. I don't know what the functionality of those bits is. And then, let's get the fun part. Let me actually switch to the adder test.jed file. This one has a simpler, this is a 4-bit adder, which will make a lot more sense than the 32-bit adder I was showing the other one. So here's our RTL. It is pretty much the assembly language of AeroLog. This is a direct analog of the actual PLI structure. I have not attempted to figure out any higher level structure. This is actually equivalent. I have not tested synthesizing it. I actually just found a bug last night here on lock equals. That should be quoted. So right now the output as formatted won't synthesize, but it would be fairly trivial to fix that and make it actually synthesize and work. And then I've got function block 2. There's not really anything in there. So now before we get to the questions, I'm just going to jump up to FC plan and show you guys what the actual structure looks like in the floor planner. I guess I found a bug. The scroll on this thing is really sensitive. All right. Can I go just one click in? All right. That's probably good enough. So you can see what I've got. Initially we're bypassing all the inputs. So we're not using the first couple of rows coming off the global routing. Then down here I've got one signal coming in. It's hooked up over here to X. It's hooked over here to X crime. Goes into the AND gate. Then we go down there and so on. So we do actually have a full physical layout viewer at this point. So now I'm just going to jump back to the slides for just one or two more before we get back to... So what remains as far as future work? We still have to figure out the last of the special product terms. There still are about six or eight bits in each macrosail that we have not yet figured out the functionality of. There's still some more in the global devices. The global bits. As I did mention, there is a little bit more I need to do to support larger devices. As far as the tool chain, I want to do more work in the decompiler. My long-term goal with this work is to integrate with some other bitstream reversing projects and so on and create the IDA for hardware. I would like to be able to go from bitstreams for CoolRunner, bitstreams for Part 6, bitstreams for Stratix 4 and everything, and go from that up to a device-dependent technology map net list. Abstract that up to a device-independent net list. Something more along the lines of LLVM IR for hardware and then do higher-level analysis on that to figure out, okay, this combination of XOR gates and MUX is looks like a 32-bit adder. This combination of stuff looks like a 10-to-1 MUX. This combination of stuff looks like a Cortex M3. I would like to eventually be able to have something along the lines of flirt here. I do know that subgraph isomorphism is NP-complete. I don't know if I can make a randomized or approximated algorithm that's fast enough that remains a topic for future work, but it is on the wish list whether it is physically possible or not. So, at this point I'll take questions. Seeing none, I guess we're done. Thank you. Thank you.
talk is called from Silicon to compiler and it's with the high level overview of the silicon and firmware produced by my tools running on live look at it. Nope there is no decompiler for the They advertise their closed source proprietary license agreement says I'm not supposed to be a reverse engineering software. I never agreed not Xilinx XC2C32A. I chose it for a bunch of of them testing. You have your choice of a nice here, the bitstream generated by this I-Link's between a constant one or the input to my showing the actual MUXes in the chip. This is layers, we're now looking at the implant layer. unlocked device and zero for a locked device. firmware and the chip has been fully flashed. And directly up to the corresponding logic. So it's