You had mentioned I think when I was
visiting you last, that the Connection
Machine you saw as sort of a dead end.
Were you talking about hardware? Because
what you have just told me is that all
the concepts that - I mean I know that you
were, you wanted ... Well I don't think it was a dead end,
I think it was ahead of its time, and the
technology wasn't quite there to do what
we wanted. But it definitely showed that
something like that was possible, and it
got people thinking in a way that
as the technology came along, that you
could build, you know, you could do that
economically on a large enough scale that
people sort of knew how to program it, knew
how to use it, and so on. I remember when I
wrote the first Scientific
American article on the Connection
Machine, I had an hypothesis, I said look,
once we had this, there's no need to have
a computer ... the power of the computer
in your home so much. The real
power will be out, you know, in some
centralized location. I didn't call it
the Cloud but that's where you need computations, because
computations want to be near each other.
They don't need to be near you once you
have the bandwidth for that. And I said so you
can imagine your have big computers that you
use something across the nation, and
then the Scientific American
people made me - they were like, "that's just too
implausible. We'll let you say, like,
'in the same city.'" So I was like, "Okay, well fine."
So you know, one city, someplace in
the center of the city, but they actually -
that was the negotiation of them pushing
it back because it just sounded too crazy.
Wow. Well can we say
that the CM-1 as a SIMD machine, that
the hardware experience there flowed -
hardware and software experience there -
flowed into the GPU processors. And
the CM-5 ...
Yeah. A GPU chip
or even a microprocessor
that has a GPU unit or a vector unit
on it, you know, they have SIMD
instructions on them, so you know, those
are  fundamentally little Connection
Machines and I think if you read ...
its clear ... I think probably people like
Nvidia even acknowledge that, you know,
there're little Connection Machines on the
chip, and they can put several - they actually
don't have all the things the Connection
Machine had, because they don't have the ...
they don't have the connection part of
it so much, but they have the map part
more than they have the reduce part.
Or they have the connection part, the
communicate part. They're able to do  reduction.
... So that's one sense
in which it exists, and then, you know,
the Cloud is much more like the CM-5
it's, you know, racks and racks of
micro processors that are connected by a
fast network. And so, you know,
that was very much the architecture
that we evolved toward, and they're programmed very much
like the CM-5 was programmed.
Although they actually don't have special
hardware for reduce network anymore
they just do that with the general network.
And they don't have special hardware
for the map, like
the graphics processors do. So it turns
out for the neural networks actually the
more efficient way to do it is on a
bunch of the graphics processors because
they do have that special, they do have
that special hardware.
So both things definitely take
elements from the way that we did things
on the Connection Machine. It's
interesting though, that in many ways the software
is more primitive than ... we really had it so
that you can take your Fortran program
and compile it to run on 10,000 processors and
that pretty much doesn't exist now. Why not???
The technology kind of lost in the whole
meltdown of the super computer industry and
... you know, I think it's getting ...
there are DARPA programs to deal with it ... again.
But it's funny seeing a technology
get lost in your lifetime.
Yeah, yeah. It will get
rebuilt eventually. But people were able
to do enough with the simpler
programming paradigms ... you know,
the simple kind of things
like MapReduce, which was definitely one
of the things we used but wasn't the only
thing we used. But that's turned out to be
powerful enough to do an awful lot. So it sounds
like even though Thinking
Machines went belly up after, you
know, little over 10 years that ...
in some sense it was worthwhile, both the
hardware and the software and the ideas
have gone on and ... Well also the
people have gone on, I mean it's amazing
the people that ... some of them have gone on to win Nobel Prizes,
or start giant research institutes, or
found companies or ... So what's interesting is
if you would - actually what would have
been the best investment portfolio would've
just been to invest  in everybody in
that building - whatever they did!
And so, you know, fantastic people
if you remember, like, you know, Eric Lander
was just starting to play with
biology using the ... and is now the head of
one of the biggest biology Institutes.
Sydney Brenner you know was ...
... only insiders knew who Sydney Brenner was,
he certainly hadn't won the Nobel Prize at
that point, you know. So it was
things like Brewster's search engine
and his archiving the Internet just seemed like
crazy ideas, nobody understood what they were.
So I think a lot of ... I think, you know,
people went on to do kind of amazing
things and ... so maybe that's how it has it's biggest
impact was a set of people got - it's like
the Manhattan Project in that sense,
a set of people got together,
and sort of inspired each other to go
off and do great things. And yeah we
did do a bunch of stuff and we did, I
mean, the project that Dick Feynman was
working, on which was quantum computing, and
it was so absurd that only Dick Feynman
was working on it, you know, that's become
a whole field! And I'm quite sure
that Thinking Machines was the first company
that ever worked on that! Just because they
had this nutty guy called Richard Feynman
who had this nutty idea that he was
working on in his spare time. Yeah, yeah, yeah. So
well I think it was ... I think we were
very lucky to have been at that
time in that place,  and it was certainly an
extraordinary thing. If I had known even a
quarter as much about this business and
how the world works as I do now,
it would have been able to save it for
long enough to, you know, for the web
come along and parallel computing to come
along, but you know, I was a bad
business person, made a lot of stupid
mistakes and you know, I can now go back
and see all kinds of things that I did
wrong that I would never do today, but
it wasn't - it wasn't what I knew about, it
wasn't what I was paying attention to then.
Yeah and it did go belly up right
when the web started out didn't it, that was
that was a bit of an ... And remember, we were working
on a web server and nobody - we couldn't
raise any money on it because nobody knew
what the web was, right? Right. Sigh. But in fact,
you know, a lot of those people went
to Sun, and turned it - helped turn it
from a workstation company into a web
company. So ....
