Emulating the full NTSC stack
Harte, Thomas
Many emulators offer a CRT filter, an artist's rendition of classic video. This presentation describes emulation of the an entire NTSC or PAL video device, to produce an engineer's rendition — starting from sync discrimination and separation, through PLLs into scan placement and via QAM to extracting colour. In the implementation discussed work is split between CPU and GPU and a range of emulated source machines are demonstrated, including in-phase machines such as the Atari 2600, machines that routinely generate sync errors such as the ZX80, machines that generate a colour signal in software such as the Apple II, and interlaced machines such as the Acorn Electron.
 Hello and welcome to emulating full NTSC stack or creating objective video artifacts. My name is Thomas Hart. So, starting from the basics. A classic video display device, out of the purpose of this talk I'm going to treat cathode ray tubes as exemplar, outputs content by scanning from top left to bottom right. To achieve this, it maintains two current deflection positions, one horizontal, one vertical, which presides as approximate sawtooth. They're completely independent of one another, so the flying spot is always moving horizontally and it is always moving vertically. Therefore, the output is a series of shallow diagonal scans, as there's no such thing as a horizontal scan in traditional analog video output. That's the output, so what's the input? For the purposes of this presentation, the input is just the stream of intensities to paint separated by sync pulses. Approximately speaking, sync pulses are either long or short, indicating either a vertical or horizontal retrace, so you're going to need to measure the length of syncs and classify them the one way or the other. In real life, there are also things called the front and back porch, which help to ensure that no content is painted during retrace, and establish the amplitude range of what follows, and the sync pulses should carry things called equalization pulses, but I'm going to ignore all of that for the purposes of it today. Since the CRT's output spot is always moving, the sync inputs are advisory only. A CRT will try to pull its deflection generator into phase with them via a couple of phase locked loops, but their best effort only. To put it another way, a CRT is not going to pause and wait for a sync input, it's just going to keep on moving. A common kind of phase locked loop deployed in classic analog electronics is a simple flywheel sync. Each time the CRT discerns, say, a horizontal sync, it will measure how much error there is between the timing of its internal generator and the timing of the input. If its generator would have fired a little too early, then it slows it down a little. If it would have fired a little too late, it speeds it up. So let's look at what all that implies for potential implementations up front. We're going to have a bunch of pixels coming in, which we need to end up on the screen, so there's going to be some streaming to the GPU there. Output is going to be a series of one-dimensional scans, but since the scans actually have heights, we're going to generate them as quads, and those also are going to need to be streamed to the GPU. That suggests, in general, a pipeline where we're dividing data and generating scans on the CPU, posting them off to the GPU means we're going to have to have ideally some sort of shared memory space for passing them off, and if your graphics API allows it, seriously consider whether you want to disable textures to whistling. Normally, what a GPU does with a texture without trying to get too far into it is it tries to ensure good cache locality for pixels that are nearby in 2D terms, but in this case, you're always going to be referencing content from the left to right, top to bottom. Ideally you would keep a classic raster structure for caching purposes. So supposing you've implemented all of that, let's see where we are. Monochrome video works! We're also accurately modelling the effect of sync errors. I'm treating a little here by showing you a British computer with 50Hz power style timings, but the point carries across. This computer, the ZX80, is unable to generate video at the same time as it performs processing, and doesn't know what should be in the frame when it resumes video, so typing anything needs to a sync discontinuity. As you can see, our model display is initially out of phase each time I press a button, but it quickly locks back in. As the flywheel sync makes an acceleration decision, proportional to position, you end up with a dampened simple harmonic motion, which is a fancy way of saying that the error bounces like a sine wave as it resolves. And if that mention of mathematics is upset you, you probably want to skip the rest of this presentation. This is going to be a slog, specifically because I want to get into how colour was added to the existing monochrome television broadcasts while maintaining backwards compatibility. So the decision was made to divide up the existing luminous signal into a low frequency part, which would continue to describe luminance, and a high frequency part for chrominance, so in this context high frequency means around 3.58 MHz, or 227.5 cycles per line. So then the question arises, how can you ensure that chrominance information stays at the high frequency end of the spectrum? Now if it were one dimensional, then prima facie you could just amplitude modulate it, that is multiply the input level by a carrier wave, here a simple sine wave, but chrominance is actually two dimensional. So if you're going to follow that line of argument, then you need to find a two dimensional version of amplitude modulation, and luckily that exists. It's called quadrature amplitude modulation, in which the incoming two chrominance signals, here U and V, are multiplied by two different sine waves, which are exactly 90 degrees out of phase, which is called being in quadrature. So then allowing for luminance, the formula for the instantaneous output of an NTSC signal is as shown on this slide, it is Y for luminance, plus U times sine theta, plus V times cos theta for chrominance signals U and V. In real life, the U and V signals are scaled down a bit, so they have a lesser total amplitude than does the lumens, but it doesn't make a substantial difference to the mathematics, so I've omitted that constant here. I'm also glossing over a bunch of other smart moves that the designer has made, with regards to backwards compatibility. Rest assured that if you were to view a colour signal on a purely monochrome set that predated the introduction of the colour standards, then you would get what looked like a slightly noisy signal, but there wouldn't be anything too obviously wrong with it. That's partly because the colour subcarrier is not an integral multiple of the line rate, so the exact noise that it contributes varies from line to line, in this case it's exactly the opposite on one line to the next, and it's also because the total length of a field means that the noise on one specific line will change from field to field, so it's not consistent over time and it is not consistent over space. So once again, let's look at where we are. Here is a screenshot from the Atari 2600 of the video game Pitfall, now with composite colour information added. A few points worth making. The Atari 2600 actually generates off-spec video, specifically it makes each horizontal line slightly longer than it should be in order to squeeze in 228 colour cycles. As a result the colour subcarrier here shows as consistent vertical stripes. It's actually a fairly common thing for NTSC machines to do, the Apple II does the same thing, the TMS9918 machines and those that descend from it, such as the master system do the same thing, and the reason is that it simplifies the electronics, I mean you're outputting the same colour burst every single line and all the other timings are exactly in phase, it just simplifies the electronics. Okay so we've managed to add colour to the video signal, but how can we extract it again? Well let's assume that a thing called a low pass filter exists, which would be smart because it does, and what such a thing does is that it is a black box and you feed it an input signal and it gives you an output signal which is the same as the input except that all the high frequency parts have been taken out. Well if we had such a thing then we could get luminance back and leave chromatite zone and luckily we have many options for implementing such a thing. The most generic one is a finite impulse response filter which in implementation terms just amounts to a weighted average around an output point, you might have heard of them as processing kernels especially in the image processing world and they're naturally parallel stuff, ideal for GPUs. They're certainly not the only option, especially in a case like this there are some specific possibilities like comb filters and box filters and I will seek to return to those later. Okay so you've got your low pass filter, you've extracted your luminance, of course you can then subtract the luminance from the original signal to give you just the chromatite part so you're sitting on top of u times sine theta plus v times cos theta. How are you going to get back to UNV? Well we've multiplied by some things so maybe we could divide by some things but you can't really hear because both sine and cos sine go through zero so you would produce an asymptotic result. Okay well take a leap with me on this one, let's instead multiply. I'm going to focus for a bit on extracting just the u component and I'm going to assume that theta is known. Of course once you can extract u you can also extract v because there are just two things around a phase at 90 to theta so let's go with the multiplying. Okay, welcome to the most fun slide of the presentation, we're going to have a lot of fun here, let's go. Here in this box some product identities, we're going to take these as given, you can look them up if you like but all you need to know are and again trust me it's true. It's that sine theta sine phi equals cos theta minus phi minus cos theta plus phi over 2 and that cos theta sine phi equals sine theta plus phi minus sine theta minus phi all over 2. Those are just facts, roll with it. So starting up here, here's the u times sine theta plus v times cos theta we started with and as I told you we're going to multiply by sine theta and see what happens. Well that's not hard to write out more neatly, it's u times sine theta sine theta plus v times cos theta sine theta. Great, where did that get us? It got us to the point where we can use our product identities. So sine theta sine theta, we're going to treat that as sine theta sine phi and therefore substitute it in cos theta minus phi, well okay but in this case it's 2theta so theta equals phi so that's cos 0 and then subtract from that cos theta plus phi, well again they're both theta so that's cos 2theta and divide by 2 and then rolling on to the next side we've got v times cos theta sine theta, well that's this one then isn't it? Cos theta sine phi is sine theta plus phi so for us it's sine 2theta minus sine theta minus phi which is sine of 0 and divide that by 2, well okay but cos theta sorry cos 0 and sine 0 are clearly constants specifically cos 0 is 1 and sine 0 is 0 so we can simplify that a bit more it's now u times 1 minus cos 2theta over 2 plus v times sine 2theta over 2 which if we pull this apart a bit more is basically u times 1 over 2 or u over 2 plus a whole bunch of things that's a trigonometric functions of 2theta but if you'll recall on the last slide we assume to the thing called a low pass filter exists so we can just use another one of those we're going to get rid of everything that hangs around the 2theta sort of range and just keep the low part and magically we have extracted you, wow. Okay so that was thankfully the peak of the mathematics that I need to discuss today let's have another look back at the concrete implementation so this is how I implemented it, on the CPU I am still dividing the input into scans and producing quads and throwing them all off to the GPU and letting it deal with it, on the GPU I have two passes, in pass 1 apply a low pass filter to separate luminance from chrominance and at that stage multiply chrominance by a sine and cosine so the input is single channel the output is three channels in actual fact it's four for reasons to do with retaining information from the colour burst but again I'm kind of glossing over that for now. For GPU pass 2 then the input to that is the luminance signal plus the two multiplied chrominance signals so it's going to low pass the two chrominance signals in order to get their true values get rid of the 2theta part that we just discussed and I also chucked in an additional high pass filter on luminance so I could have done more of a windowing filter in pass 1 but then I would have had low information from luminance leaking into chrominance so I just decided to do it in two steps I'm sure that smarter people might do something smarter. At this point it's immediately before output so this is where I apply gamma correction if relevant. Most computers today have a hypothetical at least gamma of 2.2 which is exactly the same as NCSC so there's nothing to do there but PAL has a nominal gamma of 2.8 so if you were processing a PAL signal you'd want to do some further gamma correction. And that's it success well done you've implemented the whole thing here's pitfall in colour here's printer Persia in colour life is good. So let's return to the topic of low pass filtering because I kind of glossed over it a moment ago so as I said there's a very generic solution which is called a fur filter and it ends up being just a weighted average of input samples around a desired output sample so a question you might have in your mind is what coefficients what are you talking about well personally I use Kaiser Bezel to generate the coefficients that's pretty mathematically complicated I'm not sure I could adequately explain it but trust me it exists and it makes sense you'll also see windowed sync documented a lot so if you wanted in your code to have a generic I want to filter low pass cutoff frequency as this then I would recommend looking into one of those but there are other options in both coding terms and in real life a comb filter was eventually popular for filtering the luminance part out of an NTSC signal it's kind of a bit more expensive to implement on the analog side so we're talking later televisions here but regardless it makes the observation that if you were to sum y plus sin theta and y plus sin theta 180 degrees later you'd just get to y none of the sin stuff the sin stuff would drop out because sin theta plus 180 equals minus sin theta so if you took two samples that were 227.5 color cycles apart conformant NTSC terms that's one directly vertically above or below the other and you average them then supposing they had originally had exactly the same lumines and exactly the same chromance you would get exactly the correct output luminance later TVs actually did more of a I guess three or four point averaging not just across lines but also across fields and that's that's commonly referred to as a 3d comb I mean higher dimensionalities too but 3d comb will do is the marketing term that's really really late stage television stuff will probably in the late 90s now you've definitely got some digital electronics in there another easy solution for implementing on a GPU or a CPU is certainly easy to comprehend is just a box filter if you averaged every sample across the entire cycle of a sine wave then you should end up with a sum of zero so you can average across a single wave of the color subcarrier and hope to extract luminance that way that's obviously a strictly horizontal operation and is another one as I use sometimes it tends to produce a sharper output than a blind fire impulse response filter at least for the same number of input samples first are really great but ideally you would turn up the the number of input samples very high and that gets prohibitive in strictly real life implementation terms any case the point I'm making here I guess is play around with so a lot of this presentation has been specifically about cathode ray tubes so let's talk about for a moment some specifics of them and the question is what about phosphor decay so it's not really it's not really NTSC related but on a CRT many people perceive that frames one after another blend into one another that's not really true so when an actual CRT is painting a field actually only you know a fifth to a quarter of the display is listed any given time because phosphor decay really really quickly but psychology plus persistence of vision makes up for the rest you see a solid image because your brain thinks that's a more likely outcome than what it's actually seeing so I decided to implement phosphor decay and this was my first implementation buffer scans so the quads I produce just keep them around for longer keep them for several frames and every time I need to produce a frame of output I redraw every single one is still buffered with appropriate logarithmic decay or brightness which you know it's it's it's not too dissimilar from how a real display would be perceived other than the you know the fiction of talking about discrete lines and times of in terms of decay but it produces rolling brightness errors similar to those that you might get if you pointed a classic analog camcorder at a classic CRT so it didn't really like that implementation well I mean I liked the implementation I didn't like the result if it's even worth distinguishing those things so instead at the minute I use this kind of hand waving solution I came at it from this angle the objective is that every pixel on the screen should be consistent brightness so I can achieve that must easily by just keeping a persistent frame buffer which is sort of an accumulation buffer but not in the classic GP API sense but it is one accumulating things so go with it and I'm doing standard alpha blending of skins that is to say that new scants will be drawn with a certain opacity whatever let's say it's 0.7 and they will definitely be drawn by doing you know input times 0.7 plus whatever is already there times 0.3 which ends up being an oiler-esque discrete approximation of logarithmic decay for anything that is currently on screen since it will be progressively multiplied by 0.3 and then again by 0.3 so it's 0.3 to the to the number of times it's been overdrawn there's only one big drawback to that implementation and that is that you may not touch every pixel on display every field because during moments of sync error that are going to be parts of display that are not painted at all so to get around that I decided to use a stencil buffer so that means dividing scans into the frames in which they sit which is not something that I was otherwise doing and join each frame using the stencil to mark as painted anywhere that a pixel has been updated and then at the end of the frame of course I can inspect the stencil and apply a simple you know multiplied by 0.3 I'm going to keep sticking to those example constants for any pixel that was not overpainted and that gives you know let's roll with it kind of approximation of quote unquote phosphor decay and again I really want to emphasize this this is this is not a lot of science on this this slide this is a perception thing if you wanted nice solid vibrant colors you would definitely never ever do any blending and indeed ideally you'll do some sort of black frame insertion which is the exact opposite of what I'm doing here but I think that perceptually it's more accurate but I don't know attack me as you will so as a quick bonus let's discuss pal pal is like ntsc version 2 it was designed a few years after ntsc had gone into broadcast and was a response to some of the real life practical problems that ntsc creates specifically if you've ever seen an old-fashioned American television it has a tint control and that is to correct for phase errors which result from the analog electronics involved in the fact that in you know real analog terms things don't charge and discharge instantly and signals don't propagate instantly and the implementation doesn't quite match the mathematics. Pal attempts to resolve such phase errors by alternating phase each line indeed that's what the name comes from so on one line you will have phase going positively upwards it will start on the left at for argument's sake well it's just called n and it will end at the right on n plus something but in pal well on the next line the phase will go negatively so it will start at n and end at n minus something that's you might observe that cos of theta is equal to cos of minus theta and sin of theta is equal to minus sin of minus theta and therefore another way to describe the changes made in pal is that it flips the sign of one of the color subcomponent as amplitude modulated per line mathematically I do have these descriptions of the fixed pal makes is correct so because of that change a fully featured pal decoder can do something clever it can and I'm going to hand wave this one also it can kind of average the chromance line after line and thereby eliminate phase errors at the cost of some resolution our chromance resolution only lumens remains high resolution but if you if you don't want to you know implement a delay line another costly electronics it's costly in 1950s terms anyway 1960s then you can just implement a pal s decoder which acts almost exactly like ntsc every line is decoded separately and the colors come out as the colors come out and because of the specific exigencies of my implementation it's not particularly easy to correlate data from one line to the next so I've only implemented pal s also back in the day you might have seen pal s because of a desire to avoid some telefunken patents and you know boo patents so so let's see how my implementation does with pal s here we go so on the left is an undecoded version of the composite signal on the right there it is decoded and in the box out there is a zoom in of a different part of the same software title showing the the artifact that results in this case it's jumping up and down vertically a little that's because the particular computer this signal is a simulation of which is the acorn electron outputs an interlaced signal and in this case by gift capture it's kind of making it jump a bit more than it does when you're actually looking at it with your eyes. Trying to quick corollary of the video stack discussed so far today that your output is no longer frame centric the multipaths stop in my implementation breaks down only as far as individual lines but yours may be entirely continuous and the benefit of that is freedom from a classic emulation loop in which the emulated calculates and posts a frame blocks on bsync then calculates the next etc etc etc and the issue with that loop is that latency gets worse as computers get faster e.g. on a host computer that is infinitely fast calculating the next frame will be instant so with a loop as just discussed i.e. you know post and block then your emulator will wake up on vsync calculate a frame instantly push into the pipeline then block for the entirety of the output frame before waking again so you've at least one output frame of latency before any of what you just drew becomes visible and a full two frames of output latency before the final part of what you just drew hits the display so that's a one point five frame average but if you've modeled video output is continuous as per the presentation today then you can capture the current state of the CRT at any point by pull rather than by push assuming you can also observe vsync without blocking on it or make a reasonable projection of when it will occur which is what i do under cute just an average and a couple of standard deviations i probably need to test that more anyway that means the moment the host retrace begins you can quickly capture the current state of the emulated CRT and post that before the host display starts its next frame so the host reaches the bottom of its frame you then update the frame buffer while the raster is retracing the first bit of content that you've just posted will of course appear essentially immediately so latency of zero but the final bit will not appear until the end of the next frame which is therefore one frame but that gives you an average of zero point five frames latency rather than one point five so you've you've cut off two thirds of your average output latency and furthermore those frames that you're posting or the latency rather that i'm measuring that in that zero point five quote unquote frames their host frames not emulated machine frames so if your emulator is running on a machine with a high refresh rate monitor such as 120 hertz 144 hertz etc then that user will see a corresponding decrease in latency they you're actually giving them some value for the specialized equipment that they have invested their money in so they will probably come to your house and give you gifts of course this is quite different from the way things work in a classic pusher frame and weight emulator in which case such an emulator is going to output 60 frames a second end of story and those are going to appear on the 144 hertz monitor however they appear as a bit of an aside this is the other reason that I currently use an accumulation frame buffer above and beyond sort of attempting to appear crts with regards to phosphor decay in persistence it substantially softens the appearance of tearing in cases where the host and the emulated frequencies don't match up that well it's not only true of high frequency rate monitors it's also true of displaying 50 hertz signals at 60 hertz which speaking as a British person who now lives with the modern suite of available computers is a very good thing basically the choice there is between motion aliasing and softness so I picked softness my implementation at least on the Mac seeks further to minimize tearing by adding a phase locked loop to pull the host and the emulated vertical retraces into sync or into phase rather would be a more accurate way of saying that provided that they are sufficiently quote unquote compatible which in practice means that the host display has to be very close to being a divider of the emulated display in that case you shouldn't see any tearing at all provided that the mechanism is working and now something I personally haven't implemented yet but which is obviously another potential big win is full rate full racing of the raster so no on windows you can currently know you can directly pull for current raster position and on other platforms you can make a reasonable guess if you've observed that your vertical retrace interrupts soon to be coming in 60 times a second and you know it's been one one twentieth of a second since the last one you can reduce it you're probably about halfway down display although you can actually do any better job of that by making reasonable assumptions about retrace periods and so on regardless the point is you can have a pretty good idea of where the beam is just based on observing suitably high precision clock and using that information on any platform that allows tearing you can of course fully race the raster you can run your emulator completely in lockstep with the actual on the wire video output thereby almost completely eliminating output latency. So a quick grab bag of other items I should mention before concluding this presentation number one the colour burst I've kind of overlooked much mention of that at all so far the colour burst is a signal that occurs shortly before the visible part of the line which demonstrates the current phase and amplitude of the colour subcarrier. If there's no colour burst then a good display should decline to decode colour at all treating the input signal as purely monochrome so it's probably smart to include a pathway for that in any code you're writing in particular some old machines are monochrome such as the ZX80 that we already saw and some such as the Apple II can output a colour burst or no colour burst depending on the graphics mode. Number two PCM sampling rate let's suppose that you want to come up with a PCM sampling of NTSU data either because you're actually sampling an existing signal or because you're emulating it in that case make sure you do so at at least four times the colour clock rate I can't really go thoroughly into the mathematics behind that because I'm not especially qualified but here's a version I hope will persuade you if you had a single amplitude modulated signal then it would have an inherent frequency of whatever the frequency of the amplitude modulation is and therefore per Nyquist you would want to PCM sample that at twice the rate of the subcarrier but in this case we're talking quadrature amplitude modulation so we've kind of got the two amplitude modulated signals 90 degrees out of phase so you know you're kind of need to do like double Nyquist which turns into four times the colour clock I don't know if anybody is rolling their eyes I certainly hope I don't have any real mathematicians in the audience but that's the that's that's the rule of thumb four times the colour clock in my particular implementation I use whatever is the least integer multiple of the input pixel clock there's also at least four times the colour clock often leads to lines that are more like you know 1500 samples long than 1000 samples long but it's it's no big deal I should also say that in my particular implementation there is an additional GPU step additional pass that I kind of didn't mention because it's not relevant to a generic implementation that does input data conversion to raw PCM and clock rate conversion these are both kind of basic pixel manipulation things so they're more GPU stuff the objective there was that these CPU codes should have to do as little as possible to convert between the native pixel format of the machine being emulated and something it can post to the GPU there's still going to be some conversion because I have now nailed myself down to a fixed I know six or seven or eight pixel formats that I support but it's hopefully limited in particular I support some direct PCM format there's a couple of straightforward ones there's you know like one bit luminance data there's eight bit luminance data there's one slightly less straightforward one where you supply four luminance samples and the GPU picks whichever of those is appropriate given the current phase it helps with avoiding unrealistic numbers for certain machines it's a pragmatic pick I've got a couple that are luminance plus phase related that's a special case of color encoding that is used on at least a couple of notable machines from early in the home computer market specifically I'm thinking of well the Atari 2600 which isn't a home computer it's a home console and the various Vic machines from Commodore so the Vic 20 and the Commodore 64 they both internally represent color as as a luminance plus a phase offset so they have a single sine wave and they just push it back and forth depending on the selected color so I have a couple of a couple of pixel formats that directly represent that which is actually really neat so my Atari 2600 there is no RGB lookup table anywhere as directly just lumens plus phase which is calculated algorithmically based on the palette index and then the the entity decoding takes over and produces hopefully the correct colors and you saw the screenshots make your own mind up I also have various positions of RGB that was another slightly pragmatic choice there are some machines where RGB is you know two bits per channel for example well the master system is two bits per channel the abstract CPC is actually slightly less than that because it is three potential values per channel and in that case I didn't want to be streaming more data to the GPU than I really needed to so that that is an 8-bit RGB format for that a reasonable number of machines have you know 4096 color palette the Atari ST actually that's that's the ST I'm probably talking about there the Amiga the Apple 2GS so there's a there's a you know two byte format and then there's a full 4 byte for anything else it doesn't fit and also in my emulator it's not just necessarily decoding Palo NTSC sometimes it is encoding it because some machines are natively internally RGB and they just have black box Palo NTSC encoders about which often very little is known so for those I use generic encoders and just hope that the output looks reasonably close I mean it usually does so I don't know I can't claim 100% accuracy on those machines but that's the best information I have notably absent from that list are the Apple 2 and the Apple 2 a programmer actually directly generates the luminance output signal it's a one bit thing and they're directly poking bits high or low go straight onto the wire there is no color encoder the other one is the Oric the Oric is a British home computer and it has a color ROM so it has a off the top of my head I want to say 128 byte ROM probably have to look that up but inputs are phase as a two input actually it's in quadrature so two bits for that plus four bits for the currently selected color and the output is just the correct digital level to put directly on the wire which goes to a basic resistor ladder digital to analog converter so that ROM has been dumped and the emulator can use it directly and then of course there's a lot of the machines I mentioned there are actually luminance plus phase the ones I actually support these are 2600 and a bit 20 and in both of those cases you know it's not a generic power NTSC encoder it's a it's a generic luminance plus phase offset encoder but that's still queuing relatively close to the original hardware so that's everything I had to say here are some contact details I look forward to talking to you in a moment. Seems like we're live? Okay. I hope. Okay I'm going to take this last question and start. Okay it seems like we're live so hi Tom thanks for your really interesting talk I watched yesterday actually but I raised a lot of questions for me because that's it's way out of my league but whatever we're here to learn right so thanks again for being here didn't see that many questions yet though they're starting to come in so I'll start with the first one I saw is that is this implementation portable across multiple hardware implementations NTSC is NTSC but to certain computers or comes also weird stuff with a signal that has to be accounted for? So it certainly tries to be portable against all potential video generators and some of the machines I've added to the emulator have been specifically to test that so the emulator is actually like my specific implementation of this is coming on I don't know five years old maybe maybe six I'm not really I should pay more attention to these things but initially it was just the Atari 2600 which does generate video with frequency kind of baked in I'm losing track of my thoughts here. I've added a bunch of machines since but so a recent one actually the Apple 2 I only added a couple of years ago and that's because the Apple 2 the program is responsible for generating a large part of the color signals that was supposed to be the first time but I hadn't written like both sides of the equation in case I had a fundamental misunderstanding and I wasn't revealing it because I wasn't testing it but incredibly that did just work so yeah in principle if you've if you've allowed for the timings properly like different computers will have different relative phases line online although in NTSC world it's basically just either doing it correctly or remaining in phases is the main two as long as you're keeping that stuff keeping track of that and then in principle the same back end should work for any machine. Okay, I've actually got a couple of them by the way there's an open GL implementation and there's a metal implementation because Apple says that I must they're slightly different in the details but hopefully equally compatible. Yeah, thanks, I saw another question about how about I don't know how to pronounce it in English but the SAKEM I think. Oh yeah, so that's not actually using as I understand it not actually using quadrature amplitude modulation it's just like alternating which color signal it amplitude modulates per line so the idea there was that a phase doesn't matter then you can't have phase errors and so you would decode that in a in a different way. Okay, thanks. Somebody else asks where can I buy the hardware to record PCM at I don't know Samosper or whatever I don't know what the meaning of MSPS per second. I cannot offer buying recommendations. Okay, somebody answered an oscilloscope apparently. Yeah, yeah, that's obviously for sampling a real video signal from a real device. Okay, don't see another question I had a question myself was how would you go about verifying that your results are as correct as can be is it possible to like quantify it in certain ways especially with the for example though the say like physical effects of phosphor gradient decay or whatever else. Yeah, I wish I've been more rigorous about that stuff so let's see so back when I was starting to try to do this was actually more than six years ago so about a decade ago I wrote a basic ZX81 emulator which is a machine in which the programmer is wholly responsible for producing sync and that made me curious about how this all works and that's when I started vaguely trying to collect documentation. I had a few I managed at the time to find some lovely captured samples of real PAL video back then. So at least I was able to start with a comparison between how I was decoding that and the sort of the recommended the provided sample decoding but that was only I was only like you know frame and a half it was a very small amount of data so on the other things it has been somewhat subjective and empirical. I spent a long time I lived in San Francisco for a bit there's a museum there I have temporarily I'm not going to risk a pronunciation it's a name in French it's got a lot of old video game machines I arcade machines I definitely trolled around there taking phone shots close up of the CRT for a while to try to get a sense of that. So it's a bit of a bit in the eye of the holder right? Yeah especially for stuff like the phosphor decay which is a bit subjective and not necessarily completely mathematical. Okay cool thanks let's see I see question apologies if it has already been covered but I only is it approach amenable to artifacts that happen in the CRT? Things like dot crawl certainly so you know dot crawl is just improper separation of luma and chroma and the point is you know you can't really perfectly separate them because of the way they're multiplexed and therefore actually emulating the signal by combining them and then attempting to separate them rather than a more traditional like NTSC filter type approach i.e. emulators that start with a perfect signal and then you know mess it up a bit. Part of the point of actually generating the signal and then attempting to decode the signal is that the real issues that come when you try to decode such a signal do in fact manifest. Okay thanks another new question how does one reverse engineer CRT? I'm unclear if the scope of that question is pretty broad. Perhaps someone can narrow it down a little bit I'll take an aspect question for now. Have you considered having a warping effect to simulate the convex screens of this place? I haven't actually looked into it at all so I'm kind of on the opposite side of the curve compared to a lot of emulators also in this given that I'm generating the actual composite signal I actually try constantly to do a better job of separating it and decoding it rather than you know a worse job I don't want to call warping a worse job warping is just a physical fact of curved screens and yeah I should look into things like that. Okay thank you let's see do you intend to go further in the fast-force simulation in the future for instance CRT's pixels are not perfect dots and can blend with adjacent ones depending on things such as the intensity of the beam. I mean I intend to do a lot of things in the future but yeah that's another factor of real CRT's that I am not currently simulating so it's definitely something that I should try to tackle. Okay don't see one yet somebody typing so perhaps we get a question in a second I have one more question but it's also perhaps a bit a little bit broad but I don't know anything about these things like NTC and how it works and it's inner working so a lot of things in your presentation I didn't understand had to look up I wonder if you have like especially with no electronics background would we be a good resource to start learning about like this. Oh yeah no I had no electronics background I had a half a mathematics undergraduate from which I apparently learnt some trig identities I think I started with I think it was the Noon's video guidebook so like some of the old repair manuals for analogue engineers and most of it went straight over my head because I don't know a transistor from a resistor that's a good they have a different number of legs that's the clue but no so yeah that's why I came in it mostly from the mathematical side I think because my physical skills as an engineer are very slight. Okay thanks I know somebody was talking but apparently not a question. Do I know of anyone else who has a question or is there a problem? Let's see. Somebody says there are questions without votes so I'm going to take a quick look. So I guess I'll elaborate more on that last answer since I didn't necessarily answer the question so well. Oh yeah so I started off I managed to find some actual captures of real analogue frames that was a good place to start because I could plot those and get a good conceptual sense of what the signal looks like you know lots of wavy lines and dips. I have no idea what signal I've not just described. Then just trying to break that up and actually getting the code started like you know a lot of emulators authors there's a desire to get your hands dirty and just get something running that constantly fails and then look at where it fails and figure out what's failing and move on from there so you know discerning the syncs first then categorizing the syncs then you've got something of a black and white signal on screen and then you like well how do I turn this into colour and you can make some broad guesses of things like colour phase before you figure out how to lock to the colour burst similarly hard codes and relative intensities and keep moving. Okay thanks I've got another question now. Triniton displays have a different shape of the colour mask in other TVs can this be simulated? Would it happen after the final pass or somewhere between the two passes? Oh definitely actually I had some simulation of the different colour masks very early on and then I think I got rid of it just because of the programming reason but yeah I mean so you know the absolute simplest level actually one of the PNGs might still be in the repository doesn't matter yeah the basic level you get to the point where you're outputting the RGB you can just sample an additional source texture which is the mask map and multiply on through so like one extra sample one extra multiply per output pixel. Okay thanks we'll take a look again if I see some more questions which we're not sure what did they think we got from. I think especially as monitors go you know much higher pixel density there's a lot of new options for things like you know convincing colour mask simulation. Yes, I'm waiting on somebody else to start thinking again. Not that good in smart talk. Probably should have made the presentation longer. On our next steps you're going to take with this project in the near future. So I mean the actual substance of the project ends up being another yet another multi-machine emulator so the specific thing I'm working on right now is another additional machine it's the Apple 2GS right now which I don't know whether that was a wise choice because there's really only one set of documentation which is the official documentation and like most official documentation it's frequently ambiguous or incomplete and sadly the machine wasn't popular enough to get a lot of third party documentation and I don't have access to a real one so yeah this is like how we used to have to write emulators in the 1990s but hopefully after that's done I don't know I've been kind of working my way up so it was only like a 6502 emulator back in 2015 when I started. I gained the Z80, I gained the 68000 and there's all been attempts to do actual without trying to foreshadow the other presentation I'm giving today cycle perfect emulations of those processors but I've been looking more recently at stuff like you know platforms that are defined in terms of an instruction set rather than an actual concrete processor in which case the bus timings are just not an aspect of the way the platform is defined so I have another emulator which does CPM on, it's still called CPM for OS10 because it's pretty old now, I don't know CPM for Mac OS, doesn't even sound good but yeah that's another one of those platforms of course where you know there is a processor in there but the specifics of it are beside the point it's just supposed to be an instruction set in a binary interface. I don't know some of those which are completely disjoint of course from NTSC. Now they still sound very interesting and perhaps that's a nice like a hot to the question like which is that somebody finds this a really interesting obsession I don't know if we've got an obsession but what's the top of it? Oh I don't know, like I was writing so let's think about this it was for an airplane ride inevitably as these things often are sitting in the airport, SFO, trying to wait to, let's think about it. Oh so at the time I was a mobile developer doing iOS and this was around the time of probably iOS 4, Apple had just finally imported lambdas slash closures into their language which the rest of you will be laughing about and I was like oh I better learn some new syntax I better fire up something and do something so I started writing a half cycle position Z80 emulator based on you know in queuing future bus steps to take as lambdas slash closures slash whatever they're actually called in Objective C and once you've got that you're asking yourself well what can I run against it and that's how I got to the ZX80 slash 81 the first time around and yeah those machines are really interesting they kind of use the processor to generate the video so there will be a display buffer in memory and the processor will jump to it and then the hardware underneath will trick the processor by stealing whatever it fetches and forcing a knob to the actual processor so the processor will think it's just running lots of knobs but actually it's doing all of the video fetching just lovely so all of the graphics data ends up on the bus and there's a bunch of ways the other signals are used to figure out how long the line should be and where the sync should be and stuff like that but so you end up with a fully programmatic generation of video so you know there's the built-in ROM which does a pretty basic character based display and then there's clever people who figured out how to get a full pixel addressable display out of the exact same hardware even though it wasn't even an advertised feature and so you've got coming out of that you know a video signal it's just one dimensional syncs and pixels and the question is how do you decode that accurately especially because people who actually use those machines way back when and I'm not even old enough myself although clearly I'm close how do you decode that and make it look accurate and that's how I started getting onto the PLL slash sync separation side of things it's only a monochrome machine so from there it was a short leap to well how do people add color to this? Super, I saw like in the corner of my questions coming in so just continue I'm lucky enough to have a couple of CRT displays is there anything special I can use these for any experiments maybe that would help and emulate your development? Let's think about this I mean so what has been really useful to me most recently has been like high resolution high frame rate captures of a CRT and I'd better try to ask this in the next minute so yeah you kind of need more specialized equipment for capturing what it looks like because at this point I mean it depends on the age of CRTs how they want to be doing the color decoding.
 Hello and welcome to emulating full NTSC stack or creating objective video artifacts.  My name is Thomas Hart.  So, starting from the basics.  A classic video display device, out of the purpose of this talk I'm going to treat cathode  ray tubes as exemplar, outputs content by scanning from top left to bottom right.  To achieve this, it maintains two current deflection positions, one horizontal, one vertical,  which presides as approximate sawtooth.  They're completely independent of one another, so the flying spot is always moving horizontally  and it is always moving vertically.  Therefore, the output is a series of shallow diagonal scans, as there's no such thing  as a horizontal scan in traditional analog video output.  That's the output, so what's the input?  For the purposes of this presentation, the input is just the stream of intensities to  paint separated by sync pulses.  Approximately speaking, sync pulses are either long or short, indicating either a vertical  or horizontal retrace, so you're going to need to measure the length of syncs and classify  them the one way or the other.  In real life, there are also things called the front and back porch, which help to ensure  that no content is painted during retrace, and establish the amplitude range of what  follows, and the sync pulses should carry things called equalization pulses, but I'm  going to ignore all of that for the purposes of it today.  Since the CRT's output spot is always moving, the sync inputs are advisory only.  A CRT will try to pull its deflection generator into phase with them via a couple of phase  locked loops, but their best effort only.  To put it another way, a CRT is not going to pause and wait for a sync input, it's just  going to keep on moving.  A common kind of phase locked loop deployed in classic analog electronics is a simple  flywheel sync.  Each time the CRT discerns, say, a horizontal sync, it will measure how much error there  is between the timing of its internal generator and the timing of the input.  If its generator would have fired a little too early, then it slows it down a little.  If it would have fired a little too late, it speeds it up.  So let's look at what all that implies for potential implementations up front.  We're going to have a bunch of pixels coming in, which we need to end up on the screen,  so there's going to be some streaming to the GPU there.  Output is going to be a series of one-dimensional scans, but since the scans actually have  heights, we're going to generate them as quads, and those also are going to need to be streamed  to the GPU.  That suggests, in general, a pipeline where we're dividing data and generating scans on  the CPU, posting them off to the GPU means we're going to have to have ideally some sort  of shared memory space for passing them off, and if your graphics API allows it, seriously  consider whether you want to disable textures to whistling.  Normally, what a GPU does with a texture without trying to get too far into it is it tries to  ensure good cache locality for pixels that are nearby in 2D terms, but in this case,  you're always going to be referencing content from the left to right, top to bottom.  Ideally you would keep a classic raster structure for caching purposes.  So supposing you've implemented all of that, let's see where we are.  Monochrome video works!  We're also accurately modelling the effect of sync errors.  I'm treating a little here by showing you a British computer with 50Hz power style timings,  but the point carries across.  This computer, the ZX80, is unable to generate video at the same time as it performs processing,  and doesn't know what should be in the frame when it resumes video, so typing anything  needs to a sync discontinuity.  As you can see, our model display is initially out of phase each time I press a button, but  it quickly locks back in.  As the flywheel sync makes an acceleration decision, proportional to position, you end  up with a dampened simple harmonic motion, which is a fancy way of saying that the error  bounces like a sine wave as it resolves.  And if that mention of mathematics is upset you, you probably want to skip the rest of  this presentation.  This is going to be a slog, specifically because I want to get into how colour was added to  the existing monochrome television broadcasts while maintaining backwards compatibility.  So the decision was made to divide up the existing luminous signal into a low frequency  part, which would continue to describe luminance, and a high frequency part for chrominance,  so in this context high frequency means around 3.58 MHz, or 227.5 cycles per line.  So then the question arises, how can you ensure that chrominance information stays at the  high frequency end of the spectrum?  Now if it were one dimensional, then prima facie you could just amplitude modulate it,  that is multiply the input level by a carrier wave, here a simple sine wave, but chrominance  is actually two dimensional.  So if you're going to follow that line of argument, then you need to find a two dimensional  version of amplitude modulation, and luckily that exists.  It's called quadrature amplitude modulation, in which the incoming two chrominance signals,  here U and V, are multiplied by two different sine waves, which are exactly 90 degrees out  of phase, which is called being in quadrature.  So then allowing for luminance, the formula for the instantaneous output of an NTSC signal  is as shown on this slide, it is Y for luminance, plus U times sine theta, plus V times cos  theta for chrominance signals U and V.  In real life, the U and V signals are scaled down a bit, so they have a lesser total amplitude  than does the lumens, but it doesn't make a substantial difference to the mathematics,  so I've omitted that constant here.  I'm also glossing over a bunch of other smart moves that the designer has made, with regards  to backwards compatibility.  Rest assured that if you were to view a colour signal on a purely monochrome set that predated  the introduction of the colour standards, then you would get what looked like a slightly  noisy signal, but there wouldn't be anything too obviously wrong with it.  That's partly because the colour subcarrier is not an integral multiple of the line rate,  so the exact noise that it contributes varies from line to line, in this case it's exactly  the opposite on one line to the next, and it's also because the total length of a field  means that the noise on one specific line will change from field to field, so it's not  consistent over time and it is not consistent over space.  So once again, let's look at where we are.  Here is a screenshot from the Atari 2600 of the video game Pitfall, now with composite  colour information added.  A few points worth making.  The Atari 2600 actually generates off-spec video, specifically it makes each horizontal  line slightly longer than it should be in order to squeeze in 228 colour cycles.  As a result the colour subcarrier here shows as consistent vertical stripes.  It's actually a fairly common thing for NTSC machines to do, the Apple II does the same  thing, the TMS9918 machines and those that descend from it, such as the master system  do the same thing, and the reason is that it simplifies the electronics, I mean you're  outputting the same colour burst every single line and all the other timings are exactly  in phase, it just simplifies the electronics.  Okay so we've managed to add colour to the video signal, but how can we extract it again?  Well let's assume that a thing called a low pass filter exists, which would be smart because  it does, and what such a thing does is that it is a black box and you feed it an input  signal and it gives you an output signal which is the same as the input except that all the  high frequency parts have been taken out.  Well if we had such a thing then we could get luminance back and leave chromatite zone  and luckily we have many options for implementing such a thing.  The most generic one is a finite impulse response filter which in implementation terms just  amounts to a weighted average around an output point, you might have heard of them as processing  kernels especially in the image processing world and they're naturally parallel stuff,  ideal for GPUs.  They're certainly not the only option, especially in a case like this there are some specific  possibilities like comb filters and box filters and I will seek to return to those later.  Okay so you've got your low pass filter, you've extracted your luminance, of course you can  then subtract the luminance from the original signal to give you just the chromatite part  so you're sitting on top of u times sine theta plus v times cos theta.  How are you going to get back to UNV?  Well we've multiplied by some things so maybe we could divide by some things but you can't  really hear because both sine and cos sine go through zero so you would produce an asymptotic  result.  Okay well take a leap with me on this one, let's instead multiply.  I'm going to focus for a bit on extracting just the u component and I'm going to assume  that theta is known.  Of course once you can extract u you can also extract v because there are just two things  around a phase at 90 to theta so let's go with the multiplying.  Okay, welcome to the most fun slide of the presentation, we're going to have a lot of  fun here, let's go.  Here in this box some product identities, we're going to take these as given, you can  look them up if you like but all you need to know are and again trust me it's true.  It's that sine theta sine phi equals cos theta minus phi minus cos theta plus phi over 2 and  that cos theta sine phi equals sine theta plus phi minus sine theta minus phi all over  2.  Those are just facts, roll with it.  So starting up here, here's the u times sine theta plus v times cos theta we started with  and as I told you we're going to multiply by sine theta and see what happens.  Well that's not hard to write out more neatly, it's u times sine theta sine theta plus v  times cos theta sine theta.  Great, where did that get us?  It got us to the point where we can use our product identities.  So sine theta sine theta, we're going to treat that as sine theta sine phi and therefore  substitute it in cos theta minus phi, well okay but in this case it's 2theta so theta  equals phi so that's cos 0 and then subtract from that cos theta plus phi, well again they're  both theta so that's cos 2theta and divide by 2 and then rolling on to the next side  we've got v times cos theta sine theta, well that's this one then isn't it?  Cos theta sine phi is sine theta plus phi so for us it's sine 2theta minus sine theta  minus phi which is sine of 0 and divide that by 2, well okay but cos theta sorry cos 0 and  sine 0 are clearly constants specifically cos 0 is 1 and sine 0 is 0 so we can simplify  that a bit more it's now u times 1 minus cos 2theta over 2 plus v times sine 2theta over  2 which if we pull this apart a bit more is basically u times 1 over 2 or u over 2 plus  a whole bunch of things that's a trigonometric functions of 2theta but if you'll recall on  the last slide we assume to the thing called a low pass filter exists so we can just use  another one of those we're going to get rid of everything that hangs around the 2theta  sort of range and just keep the low part and magically we have extracted you, wow.  Okay so that was thankfully the peak of the mathematics that I need to discuss today let's  have another look back at the concrete implementation so this is how I implemented it, on the CPU  I am still dividing the input into scans and producing quads and throwing them all off  to the GPU and letting it deal with it, on the GPU I have two passes, in pass 1 apply  a low pass filter to separate luminance from chrominance and at that stage multiply chrominance  by a sine and cosine so the input is single channel the output is three channels in actual  fact it's four for reasons to do with retaining information from the colour burst but again  I'm kind of glossing over that for now. For GPU pass 2 then the input to that is the  luminance signal plus the two multiplied chrominance signals so it's going to low pass the two  chrominance signals in order to get their true values get rid of the 2theta part that  we just discussed and I also chucked in an additional high pass filter on luminance so  I could have done more of a windowing filter in pass 1 but then I would have had low information  from luminance leaking into chrominance so I just decided to do it in two steps I'm sure  that smarter people might do something smarter. At this point it's immediately before output  so this is where I apply gamma correction if relevant. Most computers today have a hypothetical  at least gamma of 2.2 which is exactly the same as NCSC so there's nothing to do there  but PAL has a nominal gamma of 2.8 so if you were processing a PAL signal you'd want to  do some further gamma correction. And that's it success well done you've implemented the  whole thing here's pitfall in colour here's printer Persia in colour life is good.  So let's return to the topic of low pass filtering because I kind of glossed over it a moment  ago so as I said there's a very generic solution which is called a fur filter and it ends up  being just a weighted average of input samples around a desired output sample so a question  you might have in your mind is what coefficients what are you talking about well personally  I use Kaiser Bezel to generate the coefficients that's pretty mathematically complicated I'm  not sure I could adequately explain it but trust me it exists and it makes sense you'll  also see windowed sync documented a lot so if you wanted in your code to have a generic  I want to filter low pass cutoff frequency as this then I would recommend looking into  one of those but there are other options in both coding terms and in real life a comb  filter was eventually popular for filtering the luminance part out of an NTSC signal it's  kind of a bit more expensive to implement on the analog side so we're talking later televisions  here but regardless it makes the observation that if you were to sum y plus sin theta and  y plus sin theta 180 degrees later you'd just get to y none of the sin stuff the sin stuff  would drop out because sin theta plus 180 equals minus sin theta so if you took two  samples that were 227.5 color cycles apart conformant NTSC terms that's one directly  vertically above or below the other and you average them then supposing they had originally  had exactly the same lumines and exactly the same chromance you would get exactly the  correct output luminance later TVs actually did more of a I guess three or four point  averaging not just across lines but also across fields and that's that's commonly referred  to as a 3d comb I mean higher dimensionalities too but 3d comb will do is the marketing term  that's really really late stage television stuff will probably in the late 90s now you've  definitely got some digital electronics in there another easy solution for implementing  on a GPU or a CPU is certainly easy to comprehend is just a box filter if you averaged every  sample across the entire cycle of a sine wave then you should end up with a sum of zero  so you can average across a single wave of the color subcarrier and hope to extract luminance  that way that's obviously a strictly horizontal operation and is another one as I use sometimes  it tends to produce a sharper output than a blind fire impulse response filter at least  for the same number of input samples first are really great but ideally you would turn  up the the number of input samples very high and that gets prohibitive in strictly real  life implementation terms any case the point I'm making here I guess is play around with  so a lot of this presentation has been specifically about cathode ray tubes so let's talk about  for a moment some specifics of them and the question is what about phosphor decay so it's  not really it's not really NTSC related but on a CRT many people perceive that frames  one after another blend into one another that's not really true so when an actual CRT is painting  a field actually only you know a fifth to a quarter of the display is listed any given  time because phosphor decay really really quickly but psychology plus persistence of  vision makes up for the rest you see a solid image because your brain thinks that's a more  likely outcome than what it's actually seeing so I decided to implement phosphor decay and  this was my first implementation buffer scans so the quads I produce just keep them around  for longer keep them for several frames and every time I need to produce a frame of output  I redraw every single one is still buffered with appropriate logarithmic decay or brightness  which you know it's it's it's not too dissimilar from how a real display would be perceived  other than the you know the fiction of talking about discrete lines and times of in terms  of decay but it produces rolling brightness errors similar to those that you might get  if you pointed a classic analog camcorder at a classic CRT so it didn't really like that  implementation well I mean I liked the implementation I didn't like the result if it's even worth  distinguishing those things so instead at the minute I use this kind of hand waving  solution I came at it from this angle the objective is that every pixel on the screen  should be consistent brightness so I can achieve that must easily by just keeping a persistent  frame buffer which is sort of an accumulation buffer but not in the classic GP API sense  but it is one accumulating things so go with it and I'm doing standard alpha blending of  skins that is to say that new scants will be drawn with a certain opacity whatever let's  say it's 0.7 and they will definitely be drawn by doing you know input times 0.7 plus whatever  is already there times 0.3 which ends up being an oiler-esque discrete approximation of logarithmic  decay for anything that is currently on screen since it will be progressively multiplied  by 0.3 and then again by 0.3 so it's 0.3 to the to the number of times it's been overdrawn  there's only one big drawback to that implementation and that is that you may not touch every pixel  on display every field because during moments of sync error that are going to be parts of  display that are not painted at all so to get around that I decided to use a stencil buffer  so that means dividing scans into the frames in which they sit which is not something that  I was otherwise doing and join each frame using the stencil to mark as painted anywhere  that a pixel has been updated and then at the end of the frame of course I can inspect  the stencil and apply a simple you know multiplied by 0.3 I'm going to keep sticking to those  example constants for any pixel that was not overpainted and that gives you know let's  roll with it kind of approximation of quote unquote phosphor decay and again I really  want to emphasize this this is this is not a lot of science on this this slide this is  a perception thing if you wanted nice solid vibrant colors you would definitely never  ever do any blending and indeed ideally you'll do some sort of black frame insertion which  is the exact opposite of what I'm doing here but I think that perceptually it's more accurate  but I don't know attack me as you will so as a quick bonus let's discuss pal pal is  like ntsc version 2 it was designed a few years after ntsc had gone into broadcast and  was a response to some of the real life practical problems that ntsc creates specifically if  you've ever seen an old-fashioned American television it has a tint control and that  is to correct for phase errors which result from the analog electronics involved in the  fact that in you know real analog terms things don't charge and discharge instantly and signals  don't propagate instantly and the implementation doesn't quite match the mathematics.  Pal attempts to resolve such phase errors by alternating phase each line indeed that's  what the name comes from so on one line you will have phase going positively upwards it  will start on the left at for argument's sake well it's just called n and it will end  at the right on n plus something but in pal well on the next line the phase will go negatively  so it will start at n and end at n minus something that's you might observe that cos of theta  is equal to cos of minus theta and sin of theta is equal to minus sin of minus theta and therefore  another way to describe the changes made in pal is that it flips the sign of one of the  color subcomponent as amplitude modulated per line mathematically I do have these descriptions  of the fixed pal makes is correct so because of that change a fully featured pal decoder  can do something clever it can and I'm going to hand wave this one also it can kind of  average the chromance line after line and thereby eliminate phase errors at the cost  of some resolution our chromance resolution only lumens remains high resolution but if  you if you don't want to you know implement a delay line another costly electronics it's  costly in 1950s terms anyway 1960s then you can just implement a pal s decoder which acts  almost exactly like ntsc every line is decoded separately and the colors come out as the  colors come out and because of the specific exigencies of my implementation it's not  particularly easy to correlate data from one line to the next so I've only implemented  pal s also back in the day you might have seen pal s because of a desire to avoid some  telefunken patents and you know boo patents so so let's see how my implementation does  with pal s here we go so on the left is an undecoded version of the composite signal on  the right there it is decoded and in the box out there is a zoom in of a different part  of the same software title showing the the artifact that results in this case it's jumping  up and down vertically a little that's because the particular computer this signal is a simulation  of which is the acorn electron outputs an interlaced signal and in this case by gift capture  it's kind of making it jump a bit more than it does when you're actually looking at it  with your eyes.  Trying to quick corollary of the video stack discussed so far today that your output is  no longer frame centric the multipaths stop in my implementation breaks down only as far  as individual lines but yours may be entirely continuous and the benefit of that is freedom  from a classic emulation loop in which the emulated calculates and posts a frame blocks  on bsync then calculates the next etc etc etc and the issue with that loop is that latency  gets worse as computers get faster e.g. on a host computer that is infinitely fast calculating  the next frame will be instant so with a loop as just discussed i.e. you know post and block  then your emulator will wake up on vsync calculate a frame instantly push into the pipeline then  block for the entirety of the output frame before waking again so you've at least one  output frame of latency before any of what you just drew becomes visible and a full two  frames of output latency before the final part of what you just drew hits the display  so that's a one point five frame average but if you've modeled video output is continuous  as per the presentation today then you can capture the current state of the CRT at any  point by pull rather than by push assuming you can also observe vsync without blocking  on it or make a reasonable projection of when it will occur which is what i do under cute  just an average and a couple of standard deviations i probably need to test that more anyway that  means the moment the host retrace begins you can quickly capture the current state of the  emulated CRT and post that before the host display starts its next frame so the host  reaches the bottom of its frame you then update the frame buffer while the raster is retracing  the first bit of content that you've just posted will of course appear essentially immediately  so latency of zero but the final bit will not appear until the end of the next frame  which is therefore one frame but that gives you an average of zero point five frames latency  rather than one point five so you've you've cut off two thirds of your average output  latency and furthermore those frames that you're posting or the latency rather that  i'm measuring that in that zero point five quote unquote frames their host frames not  emulated machine frames so if your emulator is running on a machine with a high refresh  rate monitor such as 120 hertz 144 hertz etc then that user will see a corresponding decrease  in latency they you're actually giving them some value for the specialized equipment that  they have invested their money in so they will probably come to your house and give you gifts  of course this is quite different from the way things work in a classic pusher frame  and weight emulator in which case such an emulator is going to output 60 frames a second  end of story and those are going to appear on the 144 hertz monitor however they appear  as a bit of an aside this is the other reason that I currently use an accumulation frame  buffer above and beyond sort of attempting to appear crts with regards to phosphor decay  in persistence it substantially softens the appearance of tearing in cases where the host  and the emulated frequencies don't match up that well it's not only true of high frequency  rate monitors it's also true of displaying 50 hertz signals at 60 hertz which speaking  as a British person who now lives with the modern suite of available computers is a very  good thing basically the choice there is between motion aliasing and softness so I picked softness  my implementation at least on the Mac seeks further to minimize tearing by adding a phase  locked loop to pull the host and the emulated vertical retraces into sync or into phase  rather would be a more accurate way of saying that provided that they are sufficiently quote  unquote compatible which in practice means that the host display has to be very close  to being a divider of the emulated display in that case you shouldn't see any tearing  at all provided that the mechanism is working and now something I personally haven't implemented  yet but which is obviously another potential big win is full rate full racing of the raster  so no on windows you can currently know you can directly pull for current raster position  and on other platforms you can make a reasonable guess if you've observed that your vertical  retrace interrupts soon to be coming in 60 times a second and you know it's been one  one twentieth of a second since the last one you can reduce it you're probably about halfway  down display although you can actually do any better job of that by making reasonable assumptions  about retrace periods and so on regardless the point is you can have a pretty good idea  of where the beam is just based on observing suitably high precision clock and using that  information on any platform that allows tearing you can of course fully race the raster you  can run your emulator completely in lockstep with the actual on the wire video output thereby  almost completely eliminating output latency.  So a quick grab bag of other items I should mention before concluding this presentation  number one the colour burst I've kind of overlooked much mention of that at all so far the colour  burst is a signal that occurs shortly before the visible part of the line which demonstrates  the current phase and amplitude of the colour subcarrier.  If there's no colour burst then a good display should decline to decode colour at all treating  the input signal as purely monochrome so it's probably smart to include a pathway for that  in any code you're writing in particular some old machines are monochrome such as the ZX80  that we already saw and some such as the Apple II can output a colour burst or no colour  burst depending on the graphics mode.  Number two PCM sampling rate let's suppose that you want to come up with a PCM sampling  of NTSU data either because you're actually sampling an existing signal or because you're  emulating it in that case make sure you do so at at least four times the colour clock  rate I can't really go thoroughly into the mathematics behind that because I'm not especially  qualified but here's a version I hope will persuade you if you had a single amplitude  modulated signal then it would have an inherent frequency of whatever the frequency of the  amplitude modulation is and therefore per Nyquist you would want to PCM sample that at twice  the rate of the subcarrier but in this case we're talking quadrature amplitude modulation  so we've kind of got the two amplitude modulated signals 90 degrees out of phase so you know  you're kind of need to do like double Nyquist which turns into four times the colour clock  I don't know if anybody is rolling their eyes I certainly hope I don't have any real mathematicians  in the audience but that's the that's that's the rule of thumb four times the colour clock  in my particular implementation I use whatever is the least integer multiple of the input  pixel clock there's also at least four times the colour clock often leads to lines that  are more like you know 1500 samples long than 1000 samples long but it's it's no big deal  I should also say that in my particular implementation there is an additional GPU step additional  pass that I kind of didn't mention because it's not relevant to a generic implementation  that does input data conversion to raw PCM and clock rate conversion these are both kind  of basic pixel manipulation things so they're more GPU stuff the objective there was that  these CPU codes should have to do as little as possible to convert between the native  pixel format of the machine being emulated and something it can post to the GPU there's  still going to be some conversion because I have now nailed myself down to a fixed I  know six or seven or eight pixel formats that I support but it's hopefully limited in particular  I support some direct PCM format there's a couple of straightforward ones there's you  know like one bit luminance data there's eight bit luminance data there's one slightly less  straightforward one where you supply four luminance samples and the GPU picks whichever  of those is appropriate given the current phase it helps with avoiding unrealistic numbers  for certain machines it's a pragmatic pick I've got a couple that are luminance plus  phase related that's a special case of color encoding that is used on at least a couple  of notable machines from early in the home computer market specifically I'm thinking  of well the Atari 2600 which isn't a home computer it's a home console and the various  Vic machines from Commodore so the Vic 20 and the Commodore 64 they both internally represent  color as as a luminance plus a phase offset so they have a single sine wave and they just  push it back and forth depending on the selected color so I have a couple of a couple of pixel  formats that directly represent that which is actually really neat so my Atari 2600 there  is no RGB lookup table anywhere as directly just lumens plus phase which is calculated  algorithmically based on the palette index and then the the entity decoding takes over  and produces hopefully the correct colors and you saw the screenshots make your own mind  up I also have various positions of RGB that was another slightly pragmatic choice there  are some machines where RGB is you know two bits per channel for example well the master  system is two bits per channel the abstract CPC is actually slightly less than that because  it is three potential values per channel and in that case I didn't want to be streaming  more data to the GPU than I really needed to so that that is an 8-bit RGB format for  that a reasonable number of machines have you know 4096 color palette the Atari ST actually  that's that's the ST I'm probably talking about there the Amiga the Apple 2GS so there's  a there's a you know two byte format and then there's a full 4 byte for anything else it  doesn't fit and also in my emulator it's not just necessarily decoding Palo NTSC sometimes  it is encoding it because some machines are natively internally RGB and they just have  black box Palo NTSC encoders about which often very little is known so for those I use generic  encoders and just hope that the output looks reasonably close I mean it usually does so  I don't know I can't claim 100% accuracy on those machines but that's the best information  I have notably absent from that list are the Apple 2 and the Apple 2 a programmer actually  directly generates the luminance output signal it's a one bit thing and they're directly  poking bits high or low go straight onto the wire there is no color encoder the other one  is the Oric the Oric is a British home computer and it has a color ROM so it has a off the  top of my head I want to say 128 byte ROM probably have to look that up but inputs are phase  as a two input actually it's in quadrature so two bits for that plus four bits for the  currently selected color and the output is just the correct digital level to put directly  on the wire which goes to a basic resistor ladder digital to analog converter so that  ROM has been dumped and the emulator can use it directly and then of course there's  a lot of the machines I mentioned there are actually luminance plus phase the ones I actually  support these are 2600 and a bit 20 and in both of those cases you know it's not a generic  power NTSC encoder it's a it's a generic luminance plus phase offset encoder but that's still  queuing relatively close to the original hardware so that's everything I had to say here are  some contact details I look forward to talking to you in a moment.  Seems like we're live?  Okay.  I hope.  Okay I'm going to take this last question and start.  Okay it seems like we're live so hi Tom thanks for your really interesting talk I watched  yesterday actually but I raised a lot of questions for me because that's it's way out of my league  but whatever we're here to learn right so thanks again for being here didn't see that  many questions yet though they're starting to come in so I'll start with the first one  I saw is that is this implementation portable across multiple hardware implementations NTSC  is NTSC but to certain computers or comes also weird stuff with a signal that has to  be accounted for?  So it certainly tries to be portable against all potential video generators and some of  the machines I've added to the emulator have been specifically to test that so the emulator  is actually like my specific implementation of this is coming on I don't know five years  old maybe maybe six I'm not really I should pay more attention to these things but initially  it was just the Atari 2600 which does generate video with frequency kind of baked in I'm  losing track of my thoughts here.  I've added a bunch of machines since but so a recent one actually the Apple 2 I only  added a couple of years ago and that's because the Apple 2 the program is responsible for  generating a large part of the color signals that was supposed to be the first time but  I hadn't written like both sides of the equation in case I had a fundamental misunderstanding  and I wasn't revealing it because I wasn't testing it but incredibly that did just work  so yeah in principle if you've if you've allowed for the timings properly like different computers  will have different relative phases line online although in NTSC world it's basically just  either doing it correctly or remaining in phases is the main two as long as you're keeping  that stuff keeping track of that and then in principle the same back end should work  for any machine.  Okay, I've actually got a couple of them by the way there's an open GL implementation  and there's a metal implementation because Apple says that I must they're slightly different  in the details but hopefully equally compatible.  Yeah, thanks, I saw another question about how about I don't know how to pronounce it  in English but the SAKEM I think.  Oh yeah, so that's not actually using as I understand it not actually using quadrature  amplitude modulation it's just like alternating which color signal it amplitude modulates  per line so the idea there was that a phase doesn't matter then you can't have phase  errors and so you would decode that in a in a different way.  Okay, thanks.  Somebody else asks where can I buy the hardware to record PCM at I don't know Samosper or  whatever I don't know what the meaning of MSPS per second.  I cannot offer buying recommendations.  Okay, somebody answered an oscilloscope apparently.  Yeah, yeah, that's obviously for sampling a real video signal from a real device.  Okay, don't see another question I had a question myself was how would you go about verifying  that your results are as correct as can be is it possible to like quantify it in certain  ways especially with the for example though the say like physical effects of phosphor  gradient decay or whatever else.  Yeah, I wish I've been more rigorous about that stuff so let's see so back when I was  starting to try to do this was actually more than six years ago so about a decade ago I  wrote a basic ZX81 emulator which is a machine in which the programmer is wholly responsible  for producing sync and that made me curious about how this all works and that's when I  started vaguely trying to collect documentation.  I had a few I managed at the time to find some lovely captured samples of real PAL video  back then.  So at least I was able to start with a comparison between how I was decoding that and the sort  of the recommended the provided sample decoding but that was only I was only like you know  frame and a half it was a very small amount of data so on the other things it has been  somewhat subjective and empirical.  I spent a long time I lived in San Francisco for a bit there's a museum there I have temporarily  I'm not going to risk a pronunciation it's a name in French it's got a lot of old video  game machines I arcade machines I definitely trolled around there taking phone shots close  up of the CRT for a while to try to get a sense of that.  So it's a bit of a bit in the eye of the holder right?  Yeah especially for stuff like the phosphor decay which is a bit subjective and not necessarily  completely mathematical.  Okay cool thanks let's see I see question apologies if it has already been covered  but I only is it approach amenable to artifacts that happen in the CRT?  Things like dot crawl certainly so you know dot crawl is just improper separation of  luma and chroma and the point is you know you can't really perfectly separate them  because of the way they're multiplexed and therefore actually emulating the signal by  combining them and then attempting to separate them rather than a more traditional like NTSC  filter type approach i.e. emulators that start with a perfect signal and then you know mess  it up a bit.  Part of the point of actually generating the signal and then attempting to decode the signal  is that the real issues that come when you try to decode such a signal do in fact manifest.  Okay thanks another new question how does one reverse engineer CRT?  I'm unclear if the scope of that question is pretty broad.  Perhaps someone can narrow it down a little bit I'll take an aspect question for now.  Have you considered having a warping effect to simulate the convex screens of this place?  I haven't actually looked into it at all so I'm kind of on the opposite side of the curve  compared to a lot of emulators also in this given that I'm generating the actual composite  signal I actually try constantly to do a better job of separating it and decoding it rather  than you know a worse job I don't want to call warping a worse job warping is just a  physical fact of curved screens and yeah I should look into things like that.  Okay thank you let's see do you intend to go further in the fast-force simulation in  the future for instance CRT's pixels are not perfect dots and can blend with adjacent  ones depending on things such as the intensity of the beam.  I mean I intend to do a lot of things in the future but yeah that's another factor of real  CRT's that I am not currently simulating so it's definitely something that I should try  to tackle.  Okay don't see one yet somebody typing so perhaps we get a question in a second I have  one more question but it's also perhaps a bit a little bit broad but I don't know anything  about these things like NTC and how it works and it's inner working so a lot of things  in your presentation I didn't understand had to look up I wonder if you have like especially  with no electronics background would we be a good resource to start learning about like  this.  Oh yeah no I had no electronics background I had a half a mathematics undergraduate from  which I apparently learnt some trig identities I think I started with I think it was the  Noon's video guidebook so like some of the old repair manuals for analogue engineers  and most of it went straight over my head because I don't know a transistor from a resistor  that's a good they have a different number of legs that's the clue but no so yeah that's  why I came in it mostly from the mathematical side I think because my physical skills as  an engineer are very slight.  Okay thanks I know somebody was talking but apparently not a question.  Do I know of anyone else who has a question or is there a problem?  Let's see.  Somebody says there are questions without votes so I'm going to take a quick look.  So I guess I'll elaborate more on that last answer since I didn't necessarily answer the  question so well.  Oh yeah so I started off I managed to find some actual captures of real analogue frames  that was a good place to start because I could plot those and get a good conceptual sense  of what the signal looks like you know lots of wavy lines and dips.  I have no idea what signal I've not just described.  Then just trying to break that up and actually getting the code started like you know a lot  of emulators authors there's a desire to get your hands dirty and just get something running  that constantly fails and then look at where it fails and figure out what's failing and  move on from there so you know discerning the syncs first then categorizing the syncs  then you've got something of a black and white signal on screen and then you like well how  do I turn this into colour and you can make some broad guesses of things like colour phase  before you figure out how to lock to the colour burst similarly hard codes and relative intensities  and keep moving.  Okay thanks I've got another question now.  Triniton displays have a different shape of the colour mask in other TVs can this be simulated?  Would it happen after the final pass or somewhere between the two passes?  Oh definitely actually I had some simulation of the different colour masks very early on  and then I think I got rid of it just because of the programming reason but yeah I mean  so you know the absolute simplest level actually one of the PNGs might still be in the repository  doesn't matter yeah the basic level you get to the point where you're outputting the RGB  you can just sample an additional source texture which is the mask map and multiply on through  so like one extra sample one extra multiply per output pixel.  Okay thanks we'll take a look again if I see some more questions which we're not sure  what did they think we got from. I think especially as monitors go you know much higher pixel  density there's a lot of new options for things like you know convincing colour mask simulation.  Yes, I'm waiting on somebody else to start thinking again. Not that good in smart talk.  Probably should have made the presentation longer. On our next steps you're going to  take with this project in the near future. So I mean the actual substance of the project  ends up being another yet another multi-machine emulator so the specific thing I'm working  on right now is another additional machine it's the Apple 2GS right now which I don't  know whether that was a wise choice because there's really only one set of documentation  which is the official documentation and like most official documentation it's frequently  ambiguous or incomplete and sadly the machine wasn't popular enough to get a lot of third  party documentation and I don't have access to a real one so yeah this is like how we  used to have to write emulators in the 1990s but hopefully after that's done I don't know  I've been kind of working my way up so it was only like a 6502 emulator back in 2015  when I started. I gained the Z80, I gained the 68000 and there's all been attempts to  do actual without trying to foreshadow the other presentation I'm giving today cycle  perfect emulations of those processors but I've been looking more recently at stuff like  you know platforms that are defined in terms of an instruction set rather than an actual  concrete processor in which case the bus timings are just not an aspect of the way the platform  is defined so I have another emulator which does CPM on, it's still called CPM for OS10  because it's pretty old now, I don't know CPM for Mac OS, doesn't even sound good but  yeah that's another one of those platforms of course where you know there is a processor  in there but the specifics of it are beside the point it's just supposed to be an instruction  set in a binary interface. I don't know some of those which are completely disjoint of  course from NTSC.  Now they still sound very interesting and perhaps that's a nice like a hot to the question  like which is that somebody finds this a really interesting obsession I don't know if we've  got an obsession but what's the top of it?  Oh I don't know, like I was writing so let's think about this it was for an airplane ride  inevitably as these things often are sitting in the airport, SFO, trying to wait to, let's  think about it. Oh so at the time I was a mobile developer doing iOS and this was around  the time of probably iOS 4, Apple had just finally imported lambdas slash closures into  their language which the rest of you will be laughing about and I was like oh I better  learn some new syntax I better fire up something and do something so I started writing a half  cycle position Z80 emulator based on you know in queuing future bus steps to take as lambdas  slash closures slash whatever they're actually called in Objective C and once you've got  that you're asking yourself well what can I run against it and that's how I got to the  ZX80 slash 81 the first time around and yeah those machines are really interesting they  kind of use the processor to generate the video so there will be a display buffer in  memory and the processor will jump to it and then the hardware underneath will trick the  processor by stealing whatever it fetches and forcing a knob to the actual processor  so the processor will think it's just running lots of knobs but actually it's doing all  of the video fetching just lovely so all of the graphics data ends up on the bus and there's  a bunch of ways the other signals are used to figure out how long the line should be  and where the sync should be and stuff like that but so you end up with a fully programmatic  generation of video so you know there's the built-in ROM which does a pretty basic character  based display and then there's clever people who figured out how to get a full pixel addressable  display out of the exact same hardware even though it wasn't even an advertised feature  and so you've got coming out of that you know a video signal it's just one dimensional  syncs and pixels and the question is how do you decode that accurately especially because  people who actually use those machines way back when and I'm not even old enough myself  although clearly I'm close how do you decode that and make it look accurate and that's  how I started getting onto the PLL slash sync separation side of things it's only a monochrome  machine so from there it was a short leap to well how do people add color to this?  Super, I saw like in the corner of my questions coming in so just continue I'm lucky enough  to have a couple of CRT displays is there anything special I can use these for any experiments  maybe that would help and emulate your development?  Let's think about this I mean so what has been really useful to me most recently has  been like high resolution high frame rate captures of a CRT and I'd better try to ask  this in the next minute so yeah you kind of need more specialized equipment for capturing  what it looks like because at this point I mean it depends on the age of CRTs how they  want to be doing the color decoding.
ntsc_0_179.png ntsc_180_431.png ntsc_432_1415.png ntsc_1416_5507.png ntsc_5508_5711.png ntsc_5712_5807.png ntsc_5808_5855.png ntsc_5856_5903.png ntsc_5904_5963.png ntsc_5964_6035.png ntsc_6036_6095.png ntsc_6096_6143.png ntsc_6144_6263.png ntsc_6264_6383.png ntsc_6384_6443.png ntsc_6444_6635.png ntsc_6636_6695.png ntsc_6696_6779.png ntsc_6780_6827.png ntsc_6828_12911.png ntsc_12912_16151.png ntsc_16152_20303.png ntsc_20304_23651.png ntsc_23652_28763.png ntsc_28764_35519.png ntsc_35520_39995.png ntsc_39996_41027.png ntsc_41028_61319.png ntsc_61320_62111.png ntsc_62112_62759.png ntsc_62760_62831.png ntsc_62832_62999.png ntsc_63000_63311.png ntsc_63312_63371.png ntsc_63372_63815.png ntsc_63816_67763.png ntsc_67764_68483.png ntsc_68484_70787.png ntsc_70788_72119.png ntsc_72120_72503.png ntsc_72504_74243.png ntsc_74244_74615.png ntsc_74616_74711.png ntsc_74712_75779.png ntsc_75780_75923.png ntsc_75924_77867.png ntsc_77868_83291.png ntsc_83292_85451.png ntsc_85452_87035.png ntsc_87036_87539.png ntsc_87540_87611.png ntsc_87612_88367.png ntsc_88368_88483.png
Hello and welcome to emulating full NTSC stack or creating objective video artifacts. My name is Thomas Hart. So, starting from the basics. A classic video display device, out of the purpose of this talk I'm going to treat cathode ray tubes as exemplar, outputs content by scanning from top left to bottom right. To achieve this, it maintains two current deflection positions, one horizontal, one vertical, which presides as approximate sawtooth. They're completely independent of one another, so the flying spot is always moving horizontally and it is always moving vertically. Therefore, the output is a series of shallow diagonal scans, as there's no such thing as a horizontal scan in traditional analog video output. That's the output, so what's the input? For the purposes of this presentation, the input is just the stream of intensities to paint separated by sync pulses. Approximately speaking, sync pulses are either long or short, indicating either a vertical or horizontal retrace, so you're going to need to measure the length of syncs and classify them the one way or the other. In real life, there are also things called the front and back porch, which help to ensure that no content is painted during retrace, and establish the amplitude range of what follows, and the sync pulses should carry things called equalization pulses, but I'm going to ignore all of that for the purposes of it today. Since the CRT's output spot is always moving, the sync inputs are advisory only. A CRT will try to pull its deflection generator into phase with them via a couple of phase locked loops, but their best effort only. To put it another way, a CRT is not going to pause and wait for a sync input, it's just going to keep on moving. A common kind of phase locked loop deployed in classic analog electronics is a simple flywheel sync. Each time the CRT discerns, say, a horizontal sync, it will measure how much error there is between the timing of its internal generator and the timing of the input. If its generator would have fired a little too early, then it slows it down a little. If it would have fired a little too late, it speeds it up. So let's look at what all that implies for potential implementations up front. We're going to have a bunch of pixels coming in, which we need to end up on the screen, so there's going to be some streaming to the GPU there. Output is going to be a series of one-dimensional scans, but since the scans actually have heights, we're going to generate them as quads, and those also are going to need to be streamed to the GPU. That suggests, in general, a pipeline where we're dividing data and generating scans on the CPU, posting them off to the GPU means we're going to have to have ideally some sort of shared memory space for passing them off, and if your graphics API allows it, seriously consider whether you want to disable textures to whistling. Normally, what a GPU does with a texture without trying to get too far into it is it tries to ensure good cache locality for pixels that are nearby in 2D terms, but in this case, you're always going to be referencing content from the left to right, top to bottom. Ideally you would keep a classic raster structure for caching purposes. So supposing you've implemented all of that, let's see where we are. Monochrome video works! We're also accurately modelling the effect of sync errors. I'm treating a little here by showing you a British computer with 50Hz power style timings, but the point carries across. This computer, the ZX80, is unable to generate video at the same time as it performs processing, and doesn't know what should be in the frame when it resumes video, so typing anything needs to a sync discontinuity. As you can see, our model display is initially out of phase each time I press a button, but it quickly locks back in. As the flywheel sync makes an acceleration decision, proportional to position, you end up with a dampened simple harmonic motion, which is a fancy way of saying that the error bounces like a sine wave as it resolves. And if that mention of mathematics is upset you, you probably want to skip the rest of this presentation. This is going to be a slog, specifically because I want to get into how colour was added to the existing monochrome television broadcasts while maintaining backwards compatibility. So the decision was made to divide up the existing luminous signal into a low frequency part, which would continue to describe luminance, and a high frequency part for chrominance, so in this context high frequency means around 3.58 MHz, or 227.5 cycles per line. So then the question arises, how can you ensure that chrominance information stays at the high frequency end of the spectrum? Now if it were one dimensional, then prima facie you could just amplitude modulate it, that is multiply the input level by a carrier wave, here a simple sine wave, but chrominance is actually two dimensional. So if you're going to follow that line of argument, then you need to find a two dimensional version of amplitude modulation, and luckily that exists. It's called quadrature amplitude modulation, in which the incoming two chrominance signals, here U and V, are multiplied by two different sine waves, which are exactly 90 degrees out of phase, which is called being in quadrature. So then allowing for luminance, the formula for the instantaneous output of an NTSC signal is as shown on this slide, it is Y for luminance, plus U times sine theta, plus V times cos theta for chrominance signals U and V. In real life, the U and V signals are scaled down a bit, so they have a lesser total amplitude than does the lumens, but it doesn't make a substantial difference to the mathematics, so I've omitted that constant here. I'm also glossing over a bunch of other smart moves that the designer has made, with regards to backwards compatibility. Rest assured that if you were to view a colour signal on a purely monochrome set that predated the introduction of the colour standards, then you would get what looked like a slightly noisy signal, but there wouldn't be anything too obviously wrong with it. That's partly because the colour subcarrier is not an integral multiple of the line rate, so the exact noise that it contributes varies from line to line, in this case it's exactly the opposite on one line to the next, and it's also because the total length of a field means that the noise on one specific line will change from field to field, so it's not consistent over time and it is not consistent over space. So once again, let's look at where we are. Here is a screenshot from the Atari 2600 of the video game Pitfall, now with composite colour information added. A few points worth making. The Atari 2600 actually generates off-spec video, specifically it makes each horizontal line slightly longer than it should be in order to squeeze in 228 colour cycles. As a result the colour subcarrier here shows as consistent vertical stripes. It's actually a fairly common thing for NTSC machines to do, the Apple II does the same thing, the TMS9918 machines and those that descend from it, such as the master system do the same thing, and the reason is that it simplifies the electronics, I mean you're outputting the same colour burst every single line and all the other timings are exactly in phase, it just simplifies the electronics. Okay so we've managed to add colour to the video signal, but how can we extract it again? Well let's assume that a thing called a low pass filter exists, which would be smart because it does, and what such a thing does is that it is a black box and you feed it an input signal and it gives you an output signal which is the same as the input except that all the high frequency parts have been taken out. Well if we had such a thing then we could get luminance back and leave chromatite zone and luckily we have many options for implementing such a thing. The most generic one is a finite impulse response filter which in implementation terms just amounts to a weighted average around an output point, you might have heard of them as processing kernels especially in the image processing world and they're naturally parallel stuff, ideal for GPUs. They're certainly not the only option, especially in a case like this there are some specific possibilities like comb filters and box filters and I will seek to return to those later. Okay so you've got your low pass filter, you've extracted your luminance, of course you can then subtract the luminance from the original signal to give you just the chromatite part so you're sitting on top of u times sine theta plus v times cos theta. How are you going to get back to UNV? Well we've multiplied by some things so maybe we could divide by some things but you can't really hear because both sine and cos sine go through zero so you would produce an asymptotic result. Okay well take a leap with me on this one, let's instead multiply. I'm going to focus for a bit on extracting just the u component and I'm going to assume that theta is known. Of course once you can extract u you can also extract v because there are just two things around a phase at 90 to theta so let's go with the multiplying. Okay, welcome to the most fun slide of the presentation, we're going to have a lot of fun here, let's go. Here in this box some product identities, we're going to take these as given, you can look them up if you like but all you need to know are and again trust me it's true. It's that sine theta sine phi equals cos theta minus phi minus cos theta plus phi over 2 and that cos theta sine phi equals sine theta plus phi minus sine theta minus phi all over 2. Those are just facts, roll with it. So starting up here, here's the u times sine theta plus v times cos theta we started with and as I told you we're going to multiply by sine theta and see what happens. Well that's not hard to write out more neatly, it's u times sine theta sine theta plus v times cos theta sine theta. Great, where did that get us? It got us to the point where we can use our product identities. So sine theta sine theta, we're going to treat that as sine theta sine phi and therefore substitute it in cos theta minus phi, well okay but in this case it's 2theta so theta equals phi so that's cos 0 and then subtract from that cos theta plus phi, well again they're both theta so that's cos 2theta and divide by 2 and then rolling on to the next side we've got v times cos theta sine theta, well that's this one then isn't it? Cos theta sine phi is sine theta plus phi so for us it's sine 2theta minus sine theta minus phi which is sine of 0 and divide that by 2, well okay but cos theta sorry cos 0 and sine 0 are clearly constants specifically cos 0 is 1 and sine 0 is 0 so we can simplify that a bit more it's now u times 1 minus cos 2theta over 2 plus v times sine 2theta over 2 which if we pull this apart a bit more is basically u times 1 over 2 or u over 2 plus a whole bunch of things that's a trigonometric functions of 2theta but if you'll recall on the last slide we assume to the thing called a low pass filter exists so we can just use another one of those we're going to get rid of everything that hangs around the 2theta sort of range and just keep the low part and magically we have extracted you, wow. Okay so that was thankfully the peak of the mathematics that I need to discuss today let's have another look back at the concrete implementation so this is how I implemented it, on the CPU I am still dividing the input into scans and producing quads and throwing them all off to the GPU and letting it deal with it, on the GPU I have two passes, in pass 1 apply a low pass filter to separate luminance from chrominance and at that stage multiply chrominance by a sine and cosine so the input is single channel the output is three channels in actual fact it's four for reasons to do with retaining information from the colour burst but again I'm kind of glossing over that for now. For GPU pass 2 then the input to that is the luminance signal plus the two multiplied chrominance signals so it's going to low pass the two chrominance signals in order to get their true values get rid of the 2theta part that we just discussed and I also chucked in an additional high pass filter on luminance so I could have done more of a windowing filter in pass 1 but then I would have had low information from luminance leaking into chrominance so I just decided to do it in two steps I'm sure that smarter people might do something smarter. At this point it's immediately before output so this is where I apply gamma correction if relevant. Most computers today have a hypothetical at least gamma of 2.2 which is exactly the same as NCSC so there's nothing to do there but PAL has a nominal gamma of 2.8 so if you were processing a PAL signal you'd want to do some further gamma correction. And that's it success well done you've implemented the whole thing here's pitfall in colour here's printer Persia in colour life is good. So let's return to the topic of low pass filtering because I kind of glossed over it a moment ago so as I said there's a very generic solution which is called a fur filter and it ends up being just a weighted average of input samples around a desired output sample so a question you might have in your mind is what coefficients what are you talking about well personally I use Kaiser Bezel to generate the coefficients that's pretty mathematically complicated I'm not sure I could adequately explain it but trust me it exists and it makes sense you'll also see windowed sync documented a lot so if you wanted in your code to have a generic I want to filter low pass cutoff frequency as this then I would recommend looking into one of those but there are other options in both coding terms and in real life a comb filter was eventually popular for filtering the luminance part out of an NTSC signal it's kind of a bit more expensive to implement on the analog side so we're talking later televisions here but regardless it makes the observation that if you were to sum y plus sin theta and y plus sin theta 180 degrees later you'd just get to y none of the sin stuff the sin stuff would drop out because sin theta plus 180 equals minus sin theta so if you took two samples that were 227.5 color cycles apart conformant NTSC terms that's one directly vertically above or below the other and you average them then supposing they had originally had exactly the same lumines and exactly the same chromance you would get exactly the correct output luminance later TVs actually did more of a I guess three or four point averaging not just across lines but also across fields and that's that's commonly referred to as a 3d comb I mean higher dimensionalities too but 3d comb will do is the marketing term that's really really late stage television stuff will probably in the late 90s now you've definitely got some digital electronics in there another easy solution for implementing on a GPU or a CPU is certainly easy to comprehend is just a box filter if you averaged every sample across the entire cycle of a sine wave then you should end up with a sum of zero so you can average across a single wave of the color subcarrier and hope to extract luminance that way that's obviously a strictly horizontal operation and is another one as I use sometimes it tends to produce a sharper output than a blind fire impulse response filter at least for the same number of input samples first are really great but ideally you would turn up the the number of input samples very high and that gets prohibitive in strictly real life implementation terms any case the point I'm making here I guess is play around with so a lot of this presentation has been specifically about cathode ray tubes so let's talk about for a moment some specifics of them and the question is what about phosphor decay so it's not really it's not really NTSC related but on a CRT many people perceive that frames one after another blend into one another that's not really true so when an actual CRT is painting a field actually only you know a fifth to a quarter of the display is listed any given time because phosphor decay really really quickly but psychology plus persistence of vision makes up for the rest you see a solid image because your brain thinks that's a more likely outcome than what it's actually seeing so I decided to implement phosphor decay and this was my first implementation buffer scans so the quads I produce just keep them around for longer keep them for several frames and every time I need to produce a frame of output I redraw every single one is still buffered with appropriate logarithmic decay or brightness which you know it's it's it's not too dissimilar from how a real display would be perceived other than the you know the fiction of talking about discrete lines and times of in terms of decay but it produces rolling brightness errors similar to those that you might get if you pointed a classic analog camcorder at a classic CRT so it didn't really like that implementation well I mean I liked the implementation I didn't like the result if it's even worth distinguishing those things so instead at the minute I use this kind of hand waving solution I came at it from this angle the objective is that every pixel on the screen should be consistent brightness so I can achieve that must easily by just keeping a persistent frame buffer which is sort of an accumulation buffer but not in the classic GP API sense but it is one accumulating things so go with it and I'm doing standard alpha blending of skins that is to say that new scants will be drawn with a certain opacity whatever let's say it's 0.7 and they will definitely be drawn by doing you know input times 0.7 plus whatever is already there times 0.3 which ends up being an oiler-esque discrete approximation of logarithmic decay for anything that is currently on screen since it will be progressively multiplied by 0.3 and then again by 0.3 so it's 0.3 to the to the number of times it's been overdrawn there's only one big drawback to that implementation and that is that you may not touch every pixel on display every field because during moments of sync error that are going to be parts of display that are not painted at all so to get around that I decided to use a stencil buffer so that means dividing scans into the frames in which they sit which is not something that I was otherwise doing and join each frame using the stencil to mark as painted anywhere that a pixel has been updated and then at the end of the frame of course I can inspect the stencil and apply a simple you know multiplied by 0.3 I'm going to keep sticking to those example constants for any pixel that was not overpainted and that gives you know let's roll with it kind of approximation of quote unquote phosphor decay and again I really want to emphasize this this is this is not a lot of science on this this slide this is a perception thing if you wanted nice solid vibrant colors you would definitely never ever do any blending and indeed ideally you'll do some sort of black frame insertion which is the exact opposite of what I'm doing here but I think that perceptually it's more accurate but I don't know attack me as you will so as a quick bonus let's discuss pal pal is like ntsc version 2 it was designed a few years after ntsc had gone into broadcast and was a response to some of the real life practical problems that ntsc creates specifically if you've ever seen an old-fashioned American television it has a tint control and that is to correct for phase errors which result from the analog electronics involved in the fact that in you know real analog terms things don't charge and discharge instantly and signals don't propagate instantly and the implementation doesn't quite match the mathematics. Pal attempts to resolve such phase errors by alternating phase each line indeed that's what the name comes from so on one line you will have phase going positively upwards it will start on the left at for argument's sake well it's just called n and it will end at the right on n plus something but in pal well on the next line the phase will go negatively so it will start at n and end at n minus something that's you might observe that cos of theta is equal to cos of minus theta and sin of theta is equal to minus sin of minus theta and therefore another way to describe the changes made in pal is that it flips the sign of one of the color subcomponent as amplitude modulated per line mathematically I do have these descriptions of the fixed pal makes is correct so because of that change a fully featured pal decoder can do something clever it can and I'm going to hand wave this one also it can kind of average the chromance line after line and thereby eliminate phase errors at the cost of some resolution our chromance resolution only lumens remains high resolution but if you if you don't want to you know implement a delay line another costly electronics it's costly in 1950s terms anyway 1960s then you can just implement a pal s decoder which acts almost exactly like ntsc every line is decoded separately and the colors come out as the colors come out and because of the specific exigencies of my implementation it's not particularly easy to correlate data from one line to the next so I've only implemented pal s also back in the day you might have seen pal s because of a desire to avoid some telefunken patents and you know boo patents so so let's see how my implementation does with pal s here we go so on the left is an undecoded version of the composite signal on the right there it is decoded and in the box out there is a zoom in of a different part of the same software title showing the the artifact that results in this case it's jumping up and down vertically a little that's because the particular computer this signal is a simulation of which is the acorn electron outputs an interlaced signal and in this case by gift capture it's kind of making it jump a bit more than it does when you're actually looking at it with your eyes. Trying to quick corollary of the video stack discussed so far today that your output is no longer frame centric the multipaths stop in my implementation breaks down only as far as individual lines but yours may be entirely continuous and the benefit of that is freedom from a classic emulation loop in which the emulated calculates and posts a frame blocks on bsync then calculates the next etc etc etc and the issue with that loop is that latency gets worse as computers get faster e.g. on a host computer that is infinitely fast calculating the next frame will be instant so with a loop as just discussed i.e. you know post and block then your emulator will wake up on vsync calculate a frame instantly push into the pipeline then block for the entirety of the output frame before waking again so you've at least one output frame of latency before any of what you just drew becomes visible and a full two frames of output latency before the final part of what you just drew hits the display so that's a one point five frame average but if you've modeled video output is continuous as per the presentation today then you can capture the current state of the CRT at any point by pull rather than by push assuming you can also observe vsync without blocking on it or make a reasonable projection of when it will occur which is what i do under cute just an average and a couple of standard deviations i probably need to test that more anyway that means the moment the host retrace begins you can quickly capture the current state of the emulated CRT and post that before the host display starts its next frame so the host reaches the bottom of its frame you then update the frame buffer while the raster is retracing the first bit of content that you've just posted will of course appear essentially immediately so latency of zero but the final bit will not appear until the end of the next frame which is therefore one frame but that gives you an average of zero point five frames latency rather than one point five so you've you've cut off two thirds of your average output latency and furthermore those frames that you're posting or the latency rather that i'm measuring that in that zero point five quote unquote frames their host frames not emulated machine frames so if your emulator is running on a machine with a high refresh rate monitor such as 120 hertz 144 hertz etc then that user will see a corresponding decrease in latency they you're actually giving them some value for the specialized equipment that they have invested their money in so they will probably come to your house and give you gifts of course this is quite different from the way things work in a classic pusher frame and weight emulator in which case such an emulator is going to output 60 frames a second end of story and those are going to appear on the 144 hertz monitor however they appear as a bit of an aside this is the other reason that I currently use an accumulation frame buffer above and beyond sort of attempting to appear crts with regards to phosphor decay in persistence it substantially softens the appearance of tearing in cases where the host and the emulated frequencies don't match up that well it's not only true of high frequency rate monitors it's also true of displaying 50 hertz signals at 60 hertz which speaking as a British person who now lives with the modern suite of available computers is a very good thing basically the choice there is between motion aliasing and softness so I picked softness my implementation at least on the Mac seeks further to minimize tearing by adding a phase locked loop to pull the host and the emulated vertical retraces into sync or into phase rather would be a more accurate way of saying that provided that they are sufficiently quote unquote compatible which in practice means that the host display has to be very close to being a divider of the emulated display in that case you shouldn't see any tearing at all provided that the mechanism is working and now something I personally haven't implemented yet but which is obviously another potential big win is full rate full racing of the raster so no on windows you can currently know you can directly pull for current raster position and on other platforms you can make a reasonable guess if you've observed that your vertical retrace interrupts soon to be coming in 60 times a second and you know it's been one one twentieth of a second since the last one you can reduce it you're probably about halfway down display although you can actually do any better job of that by making reasonable assumptions about retrace periods and so on regardless the point is you can have a pretty good idea of where the beam is just based on observing suitably high precision clock and using that information on any platform that allows tearing you can of course fully race the raster you can run your emulator completely in lockstep with the actual on the wire video output thereby almost completely eliminating output latency. So a quick grab bag of other items I should mention before concluding this presentation number one the colour burst I've kind of overlooked much mention of that at all so far the colour burst is a signal that occurs shortly before the visible part of the line which demonstrates the current phase and amplitude of the colour subcarrier. If there's no colour burst then a good display should decline to decode colour at all treating the input signal as purely monochrome so it's probably smart to include a pathway for that in any code you're writing in particular some old machines are monochrome such as the ZX80 that we already saw and some such as the Apple II can output a colour burst or no colour burst depending on the graphics mode. Number two PCM sampling rate let's suppose that you want to come up with a PCM sampling of NTSU data either because you're actually sampling an existing signal or because you're emulating it in that case make sure you do so at at least four times the colour clock rate I can't really go thoroughly into the mathematics behind that because I'm not especially qualified but here's a version I hope will persuade you if you had a single amplitude modulated signal then it would have an inherent frequency of whatever the frequency of the amplitude modulation is and therefore per Nyquist you would want to PCM sample that at twice the rate of the subcarrier but in this case we're talking quadrature amplitude modulation so we've kind of got the two amplitude modulated signals 90 degrees out of phase so you know you're kind of need to do like double Nyquist which turns into four times the colour clock I don't know if anybody is rolling their eyes I certainly hope I don't have any real mathematicians in the audience but that's the that's that's the rule of thumb four times the colour clock in my particular implementation I use whatever is the least integer multiple of the input pixel clock there's also at least four times the colour clock often leads to lines that are more like you know 1500 samples long than 1000 samples long but it's it's no big deal I should also say that in my particular implementation there is an additional GPU step additional pass that I kind of didn't mention because it's not relevant to a generic implementation that does input data conversion to raw PCM and clock rate conversion these are both kind of basic pixel manipulation things so they're more GPU stuff the objective there was that these CPU codes should have to do as little as possible to convert between the native pixel format of the machine being emulated and something it can post to the GPU there's still going to be some conversion because I have now nailed myself down to a fixed I know six or seven or eight pixel formats that I support but it's hopefully limited in particular I support some direct PCM format there's a couple of straightforward ones there's you know like one bit luminance data there's eight bit luminance data there's one slightly less straightforward one where you supply four luminance samples and the GPU picks whichever of those is appropriate given the current phase it helps with avoiding unrealistic numbers for certain machines it's a pragmatic pick I've got a couple that are luminance plus phase related that's a special case of color encoding that is used on at least a couple of notable machines from early in the home computer market specifically I'm thinking of well the Atari 2600 which isn't a home computer it's a home console and the various Vic machines from Commodore so the Vic 20 and the Commodore 64 they both internally represent color as as a luminance plus a phase offset so they have a single sine wave and they just push it back and forth depending on the selected color so I have a couple of a couple of pixel formats that directly represent that which is actually really neat so my Atari 2600 there is no RGB lookup table anywhere as directly just lumens plus phase which is calculated algorithmically based on the palette index and then the the entity decoding takes over and produces hopefully the correct colors and you saw the screenshots make your own mind up I also have various positions of RGB that was another slightly pragmatic choice there are some machines where RGB is you know two bits per channel for example well the master system is two bits per channel the abstract CPC is actually slightly less than that because it is three potential values per channel and in that case I didn't want to be streaming more data to the GPU than I really needed to so that that is an 8-bit RGB format for that a reasonable number of machines have you know 4096 color palette the Atari ST actually that's that's the ST I'm probably talking about there the Amiga the Apple 2GS so there's a there's a you know two byte format and then there's a full 4 byte for anything else it doesn't fit and also in my emulator it's not just necessarily decoding Palo NTSC sometimes it is encoding it because some machines are natively internally RGB and they just have black box Palo NTSC encoders about which often very little is known so for those I use generic encoders and just hope that the output looks reasonably close I mean it usually does so I don't know I can't claim 100% accuracy on those machines but that's the best information I have notably absent from that list are the Apple 2 and the Apple 2 a programmer actually directly generates the luminance output signal it's a one bit thing and they're directly poking bits high or low go straight onto the wire there is no color encoder the other one is the Oric the Oric is a British home computer and it has a color ROM so it has a off the top of my head I want to say 128 byte ROM probably have to look that up but inputs are phase as a two input actually it's in quadrature so two bits for that plus four bits for the currently selected color and the output is just the correct digital level to put directly on the wire which goes to a basic resistor ladder digital to analog converter so that ROM has been dumped and the emulator can use it directly and then of course there's a lot of the machines I mentioned there are actually luminance plus phase the ones I actually support these are 2600 and a bit 20 and in both of those cases you know it's not a generic power NTSC encoder it's a it's a generic luminance plus phase offset encoder but that's still queuing relatively close to the original hardware so that's everything I had to say here are some contact details I look forward to talking to you in a moment. Seems like we're live? Okay. I hope. Okay I'm going to take this last question and start. Okay it seems like we're live so hi Tom thanks for your really interesting talk I watched yesterday actually but I raised a lot of questions for me because that's it's way out of my league but whatever we're here to learn right so thanks again for being here didn't see that many questions yet though they're starting to come in so I'll start with the first one I saw is that is this implementation portable across multiple hardware implementations NTSC is NTSC but to certain computers or comes also weird stuff with a signal that has to be accounted for? So it certainly tries to be portable against all potential video generators and some of the machines I've added to the emulator have been specifically to test that so the emulator is actually like my specific implementation of this is coming on I don't know five years old maybe maybe six I'm not really I should pay more attention to these things but initially it was just the Atari 2600 which does generate video with frequency kind of baked in I'm losing track of my thoughts here. I've added a bunch of machines since but so a recent one actually the Apple 2 I only added a couple of years ago and that's because the Apple 2 the program is responsible for generating a large part of the color signals that was supposed to be the first time but I hadn't written like both sides of the equation in case I had a fundamental misunderstanding and I wasn't revealing it because I wasn't testing it but incredibly that did just work so yeah in principle if you've if you've allowed for the timings properly like different computers will have different relative phases line online although in NTSC world it's basically just either doing it correctly or remaining in phases is the main two as long as you're keeping that stuff keeping track of that and then in principle the same back end should work for any machine. Okay, I've actually got a couple of them by the way there's an open GL implementation and there's a metal implementation because Apple says that I must they're slightly different in the details but hopefully equally compatible. Yeah, thanks, I saw another question about how about I don't know how to pronounce it in English but the SAKEM I think. Oh yeah, so that's not actually using as I understand it not actually using quadrature amplitude modulation it's just like alternating which color signal it amplitude modulates per line so the idea there was that a phase doesn't matter then you can't have phase errors and so you would decode that in a in a different way. Okay, thanks. Somebody else asks where can I buy the hardware to record PCM at I don't know Samosper or whatever I don't know what the meaning of MSPS per second. I cannot offer buying recommendations. Okay, somebody answered an oscilloscope apparently. Yeah, yeah, that's obviously for sampling a real video signal from a real device. Okay, don't see another question I had a question myself was how would you go about verifying that your results are as correct as can be is it possible to like quantify it in certain ways especially with the for example though the say like physical effects of phosphor gradient decay or whatever else. Yeah, I wish I've been more rigorous about that stuff so let's see so back when I was starting to try to do this was actually more than six years ago so about a decade ago I wrote a basic ZX81 emulator which is a machine in which the programmer is wholly responsible for producing sync and that made me curious about how this all works and that's when I started vaguely trying to collect documentation. I had a few I managed at the time to find some lovely captured samples of real PAL video back then. So at least I was able to start with a comparison between how I was decoding that and the sort of the recommended the provided sample decoding but that was only I was only like you know frame and a half it was a very small amount of data so on the other things it has been somewhat subjective and empirical. I spent a long time I lived in San Francisco for a bit there's a museum there I have temporarily I'm not going to risk a pronunciation it's a name in French it's got a lot of old video game machines I arcade machines I definitely trolled around there taking phone shots close up of the CRT for a while to try to get a sense of that. So it's a bit of a bit in the eye of the holder right? Yeah especially for stuff like the phosphor decay which is a bit subjective and not necessarily completely mathematical. Okay cool thanks let's see I see question apologies if it has already been covered but I only is it approach amenable to artifacts that happen in the CRT? Things like dot crawl certainly so you know dot crawl is just improper separation of luma and chroma and the point is you know you can't really perfectly separate them because of the way they're multiplexed and therefore actually emulating the signal by combining them and then attempting to separate them rather than a more traditional like NTSC filter type approach i.e. emulators that start with a perfect signal and then you know mess it up a bit. Part of the point of actually generating the signal and then attempting to decode the signal is that the real issues that come when you try to decode such a signal do in fact manifest. Okay thanks another new question how does one reverse engineer CRT? I'm unclear if the scope of that question is pretty broad. Perhaps someone can narrow it down a little bit I'll take an aspect question for now. Have you considered having a warping effect to simulate the convex screens of this place? I haven't actually looked into it at all so I'm kind of on the opposite side of the curve compared to a lot of emulators also in this given that I'm generating the actual composite signal I actually try constantly to do a better job of separating it and decoding it rather than you know a worse job I don't want to call warping a worse job warping is just a physical fact of curved screens and yeah I should look into things like that. Okay thank you let's see do you intend to go further in the fast-force simulation in the future for instance CRT's pixels are not perfect dots and can blend with adjacent ones depending on things such as the intensity of the beam. I mean I intend to do a lot of things in the future but yeah that's another factor of real CRT's that I am not currently simulating so it's definitely something that I should try to tackle. Okay don't see one yet somebody typing so perhaps we get a question in a second I have one more question but it's also perhaps a bit a little bit broad but I don't know anything about these things like NTC and how it works and it's inner working so a lot of things in your presentation I didn't understand had to look up I wonder if you have like especially with no electronics background would we be a good resource to start learning about like this. Oh yeah no I had no electronics background I had a half a mathematics undergraduate from which I apparently learnt some trig identities I think I started with I think it was the Noon's video guidebook so like some of the old repair manuals for analogue engineers and most of it went straight over my head because I don't know a transistor from a resistor that's a good they have a different number of legs that's the clue but no so yeah that's why I came in it mostly from the mathematical side I think because my physical skills as an engineer are very slight. Okay thanks I know somebody was talking but apparently not a question. Do I know of anyone else who has a question or is there a problem? Let's see. Somebody says there are questions without votes so I'm going to take a quick look. So I guess I'll elaborate more on that last answer since I didn't necessarily answer the question so well. Oh yeah so I started off I managed to find some actual captures of real analogue frames that was a good place to start because I could plot those and get a good conceptual sense of what the signal looks like you know lots of wavy lines and dips. I have no idea what signal I've not just described. Then just trying to break that up and actually getting the code started like you know a lot of emulators authors there's a desire to get your hands dirty and just get something running that constantly fails and then look at where it fails and figure out what's failing and move on from there so you know discerning the syncs first then categorizing the syncs then you've got something of a black and white signal on screen and then you like well how do I turn this into colour and you can make some broad guesses of things like colour phase before you figure out how to lock to the colour burst similarly hard codes and relative intensities and keep moving. Okay thanks I've got another question now. Triniton displays have a different shape of the colour mask in other TVs can this be simulated? Would it happen after the final pass or somewhere between the two passes? Oh definitely actually I had some simulation of the different colour masks very early on and then I think I got rid of it just because of the programming reason but yeah I mean so you know the absolute simplest level actually one of the PNGs might still be in the repository doesn't matter yeah the basic level you get to the point where you're outputting the RGB you can just sample an additional source texture which is the mask map and multiply on through so like one extra sample one extra multiply per output pixel. Okay thanks we'll take a look again if I see some more questions which we're not sure what did they think we got from. I think especially as monitors go you know much higher pixel density there's a lot of new options for things like you know convincing colour mask simulation. Yes, I'm waiting on somebody else to start thinking again. Not that good in smart talk. Probably should have made the presentation longer. On our next steps you're going to take with this project in the near future. So I mean the actual substance of the project ends up being another yet another multi-machine emulator so the specific thing I'm working on right now is another additional machine it's the Apple 2GS right now which I don't know whether that was a wise choice because there's really only one set of documentation which is the official documentation and like most official documentation it's frequently ambiguous or incomplete and sadly the machine wasn't popular enough to get a lot of third party documentation and I don't have access to a real one so yeah this is like how we used to have to write emulators in the 1990s but hopefully after that's done I don't know I've been kind of working my way up so it was only like a 6502 emulator back in 2015 when I started. I gained the Z80, I gained the 68000 and there's all been attempts to do actual without trying to foreshadow the other presentation I'm giving today cycle perfect emulations of those processors but I've been looking more recently at stuff like you know platforms that are defined in terms of an instruction set rather than an actual concrete processor in which case the bus timings are just not an aspect of the way the platform is defined so I have another emulator which does CPM on, it's still called CPM for OS10 because it's pretty old now, I don't know CPM for Mac OS, doesn't even sound good but yeah that's another one of those platforms of course where you know there is a processor in there but the specifics of it are beside the point it's just supposed to be an instruction set in a binary interface. I don't know some of those which are completely disjoint of course from NTSC. Now they still sound very interesting and perhaps that's a nice like a hot to the question like which is that somebody finds this a really interesting obsession I don't know if we've got an obsession but what's the top of it? Oh I don't know, like I was writing so let's think about this it was for an airplane ride inevitably as these things often are sitting in the airport, SFO, trying to wait to, let's think about it. Oh so at the time I was a mobile developer doing iOS and this was around the time of probably iOS 4, Apple had just finally imported lambdas slash closures into their language which the rest of you will be laughing about and I was like oh I better learn some new syntax I better fire up something and do something so I started writing a half cycle position Z80 emulator based on you know in queuing future bus steps to take as lambdas slash closures slash whatever they're actually called in Objective C and once you've got that you're asking yourself well what can I run against it and that's how I got to the ZX80 slash 81 the first time around and yeah those machines are really interesting they kind of use the processor to generate the video so there will be a display buffer in memory and the processor will jump to it and then the hardware underneath will trick the processor by stealing whatever it fetches and forcing a knob to the actual processor so the processor will think it's just running lots of knobs but actually it's doing all of the video fetching just lovely so all of the graphics data ends up on the bus and there's a bunch of ways the other signals are used to figure out how long the line should be and where the sync should be and stuff like that but so you end up with a fully programmatic generation of video so you know there's the built-in ROM which does a pretty basic character based display and then there's clever people who figured out how to get a full pixel addressable display out of the exact same hardware even though it wasn't even an advertised feature and so you've got coming out of that you know a video signal it's just one dimensional syncs and pixels and the question is how do you decode that accurately especially because people who actually use those machines way back when and I'm not even old enough myself although clearly I'm close how do you decode that and make it look accurate and that's how I started getting onto the PLL slash sync separation side of things it's only a monochrome machine so from there it was a short leap to well how do people add color to this? Super, I saw like in the corner of my questions coming in so just continue I'm lucky enough to have a couple of CRT displays is there anything special I can use these for any experiments maybe that would help and emulate your development? Let's think about this I mean so what has been really useful to me most recently has been like high resolution high frame rate captures of a CRT and I'd better try to ask this in the next minute so yeah you kind of need more specialized equipment for capturing what it looks like because at this point I mean it depends on the age of CRTs how they want to be doing the color decoding.
classic video display device, out of the purpose and establish the amplitude range of what ZX80, is unable to generate video at the same skip the rest of this presentation. This is going are. Here is a screenshot from the Atari 2600 of implementing on a GPU or a CPU is certainly easy sometimes it tends to produce a sharper output instantly and the implementation doesn't quite an interlaced signal and in this case by gift host and the emulated vertical retraces into sync writing in particular some old machines are and some such as the Apple II can output a colour NTSC filter type approach i.e. emulators that