MeadInterv i ew 1 Fi le : 1 Carver.doe

Transcription

1 MeadInterv i ew 1 Fi le : 1 Carver.doe 10/00 CARVER MEAD INTERVIEW By : Gene Youngb l ood (Document 1 of 4 of Mead Interv i ew) The progress i on toward a speed/power i nvers i on has been happen i ng cont i nuous l y s i nce 1959 when the i ntegra ted c i rcu i t was i nvented and ye t today we st i l l can ' t bu i l d a supercomputer tha t doesn ' t requ i re acres of f reon tubes for coo l i ng. CARVER : Oh yes you can. I t ' s j ust they ' re not do i ng i t. You see the prob l em has been twofo l d. One, as the techno l ogy deve l oped the b i g computer peop l e - - the Amdhah l s and the Crays - - stayed w i th the o l der b i po l ar ga te ar ray techno l ogy. They haven ' t par t i c i pa ted i n the revo l ut i on. Now the semi conduc tor guys don ' t know much about computers so they ' ve cop i ed a bunch of anc i ent arch i tec tures. So wha t we have i n the persona l computers i s creaky o l d arch i tec tures w i th th i s wonder fu l techno l ogy. So the peop l e dr i v i ng the techno l ogy don ' t know anyth i ng about systems and the peop l e who have bu i l t t rad i t i ona l l arge computer systems haven ' t par t i c i pa ted i n the techno l ogy revo l ut i on. Supercomputers are an ext reme l y i ne f f i c i ent use of power and space. They j ust brute- forced i t. They sa i d we ' re not go i ng to use any c l everness a t a l l, we ' re j ust go i ng to pour the coa l to i t and see how fast we can make i t run. So they j ust turn up the knob and go for i t. And you end up w i th these g i ant steam eng i nes tha t b l ow of f a l l th i s hea t. I t doesn ' t make any sense a t a l l f rom any po i nt of v i ew. But, you see, the s i tua t i on i s ac tua l l y much be t ter than tha t. And tha t ' s the par t nobody counts. I mean i f you take today ' s techno- l ogy and use i t to do a rea l l y nove l arch i tec ture you can ge t a fac tor of 10,000 r i ght now. Today. But you don ' t do i t by runn i ng the c l ock tha t fast. You do i t through para l l e l i sm. There ' s a l ot more to be ga i ned through arch i tec ture than there i s i n c l ock speed. You can ge t 10 GHz out of a para l l e l ar ray of 10 MHz c l ocks. But aren ' t there ser i ous prob l ems w i th sof tware for para l l e l arch i tec tures? There seems to be cont roversy whe ther schemes l i ke da taf l ow and func t i ona l programmi ng can ac tua l l y overcome the commun i ca t i on prob l ems. CARVER : As l ong as you ' re ta l k i ng sof tware you ' re mi ss i ng the po i nt. Because you ' re th i nk i ng of some th i ng programmab l e. And those schemes w i l l never work out i n terms of enormous para l l e l i sm. Tha t ' s why I th i nk th i s super -computer th i ng w i l l turn out to be a ne t l oss to our count ry. Because they ' re st i l l c l i ng i ng to the be l i e f tha t we ' re go i ng to make programmab l e h i gh l y para l l e l mach i nes. I worked on tha t for a l ong t i me and f i na l l y came to the conc l us i on tha t i t j ust wasn ' t go i ng to make i t. You were go i ng to ge t fac tors of ten or a hundred and tha t was the end of i t. We l l tha t ' s not enough. We need e i ght or n i ne orders of magn i tude to do the k i nds of th i ngs we want to do w i th computers today. There ' s fac tors of a mi l l i on there i f you do i t r i ght. But for tha t you can ' t separa te the arch i tec ture f rom the a l gor i thm. You have to bu i l d tha t a l gor i thm i n s i l i con, not program i t somehow. I th i nk they 'd be t ter be fac i ng st ra i ght i nto the fac t tha t there are ded i ca ted arch i tec tures for these enormous tasks. Wha t do you ca l l the k i nd of arch i tec ture you ' re ta l k i ng about? CARVER : I ca l l them s i l i con a l gor i thms. You j ust map the app l i ca t i on r i ght i nto s i l i con. And the techno l ogy for do i ng tha t we understand pre t ty we l l. I don ' t mean we ' ve work- ed i t a l l out. But we ' ve deve l oped the techno l ogy for tak i ng h i gh- l eve l descr i pt i ons and put t i ng them on s i l i con. I t ' s ca l l ed s i l i con comp i l a t i on, wh i ch i s the obv i ous next step. You need to comp i l e th i ngs r i ght down i nto the ch i p i nstead of comp i l i ng them i nto some code tha t you feed to th i s g i ant mass of ch i ps tha t gr i nds away on i t t ry i ng to i nterpre t i t. So the s i l i con comp i l er so l ves not on l y the comp l ex i ty prob l em but a l so the prob l em of mass i ve l y para l l e l arch i tec tures by a l l ow i ng us to exper i ment w i th the ded i ca ted hardware tha t a l one can rea l i ze the fu l l potent i a l of tha t arch i tec ture. CARVER : You be t. Otherw i se there ' s no hope. Because then you ' re stuck. I f i t takes f i ve years to des i gn a ch i p nobody ' s go i ng to exper i ment w i th a l gor i thms i n s i l i con, r i ght? Because i t takes years to ge t anyp l ace. And tha t ' s i n ser i es w i th your l earn i ng curve. So you have to have a techno l ogy for exper i ment i ng w i th des i gns i n s i l i con more rap i d l y than tha t.

2 MeadInterv i ew 2 F i l e : 1 Carver.doc Then mi croe l ec t ron i cs and computer sc i ence have become synonymous. CARVER : They have to. You see, the th i ng tha t a l l owed peop l e to separa te the mach i nery of comput i ng f rom the prac t i ce of comput i ng - - hardware f rom sof tware, i f you l i ke - - was the i dea tha t wha t computers d i d was a sequent i a l process. The o l d Tur i ng or Von Neumann i dea. Once you star t to do th i ngs i n a h i gh l y concur rent way you can ' t separa te the a l gor i thm f rom the arch i tec ture any more. You ' re conf ronted w i th the necess i ty of mapp i ng c l asses of app l i ca t i ons d i rec t l y i nto s i l i con. There ' s no such th i ng any more as the genera l purpose programmab l e mach i ne when i t comes to rea l l y h i gh bandw i dth comput i ng - - wh i ch i s the on l y reason you need concur rent process i ng i n the f i rst p l ace. So tha t doesn ' t mean Von Neumann mach i nes w i l l go away ; they ' l l be used as genera l purpose cont ro l l ers for these spec i a l purpose i nst ruments. The user has to ta l k to some th i ng tha t ' s coherent, tha t has a l anguage and an opera t i ng system and so for th. But i nstead of tha t poor l i t t l e th i ng t ry i ng to execute a l l those i nst ruc t i ons, you ' re go i ng to have spec i a l -purpose eng i nes i t ca l l s upon to do these enormous tasks. So you can th i nk of the ded i ca ted ch i ps as ext reme l y capab l e i nst ruc t i ons tha t you ca l l f rom your persona l computer. Wou l d a persona l computer based on one of the new 32-b i t mi croprocessors be power fu l enough to cont ro l a such a h i gh speed eng i ne? CARVER : Oh sure. The very f i rst one of those on the marke t i s the S i l i con Graph i cs I r i s mach i ne w i th the Geome t ry Eng i ne i n i t tha t J i m Cl ark des i gned. I t ' s got a tha t cont ro l s an enormous graph i cs p i pe l i ne w i th a t l east 100 t i mes more computat i on capab i l i ty, wh i ch never the l ess i s a s l ave to the mi croprocessor. The reason, of course, i s tha t one sma l l i nst ruc t i on can cause an enormous amount of work to ge t done. So the mi croprocessor i s used to upda te the p i c ture. Even w i th ded i ca ted hardware there ' s a commun i ca t i on prob l em i n any mass i ve l y para l l e l arch i tec ture. So peop l e star t ta l k i ng about wa fer -sca l e i ntegra t i on. Is wa fer sca l e techno l ogy necessary for h i gh l y concur rent mach i nes? CARVER : I t ' s not essent i a l but i t wou l d he l p a l ot. There are many d i mens i ons to the prob l em. But essent i a l l y you ' re r i ght tha t every t i me you come of f the ch i p and have to dr i ve a b i g w i re i nstead of the k i nd of w i res you have on the ch i p, you pay for tha t i n speed. So to the extent tha t you can put the w i r i ng on a wa fer i nstead of on a c i rcu i t board, the th i ng w i l l run tha t much faster and be tha t much more cost e f fec t i ve. The prob l em i s nobody has ye t come up w i th a v i ab l e scheme for to l era t i ng the fau l ts tha t a l ways occur. None of the redundancy schemes are v i ab l e ye t i n terms of e f f i c i ent use of the techno l ogy. I t has not a t t rac ted the magn i tude of research e f for t tha t i t ' s wor th. I th i nk i t ' s a very exc i t i ng area. But as an a l terna t i ve to wa fer -sca l e i ntegra t i on you can ar range the ch i ps as i s l ands of computat i on where the l i nks be tween them aren ' t too huge. I t depends ent i re l y on the na ture of the a l gor i thm. How t i ght l y connec ted are the c l umps of computat i on? You cou l d, for examp l e, ar range the i s l ands to be synchronous and where they ' re not connec ted too t i ght l y use an asynchronous protoco l so you don ' t have to l ock a l l the c l ocks toge ther. You 'd st i l l have a fa i r l y sma l l box. So even i f you have to go of f ch i p you ar range i t so everyth i ng tha t ' s on the ch i p i s t i ght l y connec ted. Tha t ' s the who l e game of mak i ng concur rent arch i tec tures whe ther you ' re on a wa fer or not. I t has to be based on l oca l i ty. Otherw i se the w i r i ng mess j ust ge ts comp l e te l y out of hand. Inc i denta l l y, the bra i n has the same prob l em. Your cor tex i s bas i ca l l y two-d i mens i ona l. I t ' s on l y a mi l l i me ter th i ck. Tha t ' s a l ot th i cker than a ch i p but i t ' s by no means a fu l l y connec ted three-d i mens i ona l vo l ume. Peop l e th i nk tha t because the bra i n i s encased i n th i s l i t t l e round th i ng i t ' s three-d i mens i ona l. I t ' s not. I t ' s two-d i mens i ona l, and i t ' s w i red i n a twod i mens i ona l way. In fac t there are two l ayers. There ' s the gray ma t ter on top of the cor tex wh i ch i s where the process i ng happens, and there ' s the wh i te ma t ter on the bot tom ha l f of the cor tex wh i ch i s where the w i res are - - a so l i d ma t of w i res about ha l f or one- th i rd as th i ck as the cor tex. Tha t ' s exac t l y wha t we have i n s i l i con except we don ' t have qu i te as many l ayers of th i ckness. But i t doesn ' t qua l i tat i ve l y change the na ture of the prob l em. And we ' re ge t t i ng more l ayers faster than the bra i n i s. So the who l e i dea of put t i ng pr i or i ty on l oca l i ty i s as t rue i n the bra i n as i n a ch i p. Are s i l i con comp i l ers based on ar t i f i c i a l i nte l l i gence? Are they exper t systems? CARVER : You cou l d th i nk of them as exper t systems, i f you l i ke, i n the sense tha t they capture the exper t i se of the des i gners, but they don ' t do i t by be i ng ar t i f i c i a l i nte l l i gens i a ; they do i t because some rea l l y smar t peop l e worked the prob l em. S i l i con comp i l ers are based on ord i nary, o l d- fash i oned, good systems des i gn. Tha t ' s a who l e d i f ferent

3 MeadInterv i ew 3 Fi le : 1 Carver.doe game. In my op i n i on, ar t i f i c i a l i nte l l i gence has done abso l ute l y noth i ng tha t ever he l ped anyone to do a ch i p des i gn. They may he l p i n the future but so far they haven ' t. W i l l the s i l i con comp i l er become the un i versa l me thod of ch i p des i gn? CARVER : Yes, for h i gh l y concur rent arch i tec tures. Regard l ess of the comp l ex i ty of the ch i p? CARVER : We l l you s i mp l y can ' t i mp l ement most of these a l gor i thms w i thout the comp l ex i ty. There has to be a cer ta i n sca l e of i ntegra t i on be fore i t makes sense. But yes, you ' re go i ng to need a s i l i con comp i l er to des i gn even a Von Neumann-sty l e ch i p a t the comp l ex i ty l eve l of VLSI But once you have VLSI wha t ' s to stop you f rom do i ng some th i ng a who l e l ot more i nterest i ng than a Von Neumann computer? Tha t ' s rea l l y the po i nt. Peop l e ta l k about the comp l ex i ty as though i t were a prob l em. I t ' s not ; i t ' s an oppor tun i ty. We have to th i nk about i t as an oppor tun i ty for new i deas, not as a prob l em tha t has to be overcome so we can make one more Von Neumann computer. Tha t ' s a ter r i b l e waste of a very beaut i fu l techno l ogy. And tha t ' s wha t we ' re see i ng r i ght now. The s i tua t i on i s ana l ogous to wha t happened i n n i ne teenth century Eng l and a f ter Far raday demonst ra ted tha t you cou l d make e l ec t r i c genera tors and motors. The fac tor i es of tha t day were l ong sheds w i th huge steam eng i nes tha t drove a rotat i ng sha f t tha t ran a l ong under the r i dge po l e of the roof w i th b i g pu l l eys on i t and be l ts down to a l l the mach i nes. I f you wanted to stop a l a the you s l i d the be l t f rom the dr i ve pu l l ey onto an i d l er. We l l, when they bu i l t the f i rst e l ec t r i c motors they bu i l t enormous motors and used them i n p l ace of the steam eng i ne. You st i l l had the sha f t and pu l l eys and be l ts. They cou l d not conceptua l i ze tha t they rea l l y ought to have f rac t i ona l horsepower motors d i st r i buted around - - wha t we now ca l l the para l l e l process i ng approach. But the other th i ng was tha t of course they cou l dn ' t a f ford to change everyth i ng. We l ook a t tha t today and say we l l tha t was rea l l y dumb. But i f you were l i v i ng back then you 'd probab l y have a thousand arguments for why i t was the on l y sens i b l e th i ng to do. Tha t ' s where we are today i n computer sc i ence. We have th i s wonder fu l techno l ogy and we ' re bu i l d i ng one mono l i th i c computer. And i f tha t turns out to be i ne f f i c i ent we ' l l make a b i g one on a ch i p. So they ta l k about mi croma i nf rames. I t remi nds me of a car toon of the board of d i rec tors i n De t ro i t and one of the d i rec tors i s say i ng "We l l, i f we have to make a compac t, we ' l l make the b i ggest compac t i n the i ndust ry! " CARVER : The reason I got i nto the who l e sord i d a f fa i r f i f teen years ago i s tha t we as a cu l ture were sta l l ed. The rea l l y smar t i nnova tors, who are a l ways the l i t t l e guys, were unab l e to ge t a t the techno l ogy. And tha t ' s cr i mi na l. And of course the b i g compan i es l i ked i t tha t way. Because i t ' s fat l azy comfor tab l e, r i ght? Wh i ch i s why they don ' t l i ke me. I 'm rock i ng the boa t. But the i mpor tant th i ng tha t has happened i s tha t the l i t t l e guy l i ke J i m Cl ark, one guy, can ge t a t i t and turn the who l e marke t ups i de down. Tha t Geome t ry Eng i ne tha t J i m Cl ark bu i l t i s a d i f ferent arch i tec ture but the same i dea tha t I proposed to Dave Evans [of Evans and Suther l and] i n 1975 as prec i se l y the r i ght th i ng to do w i th the techno l ogy, name l y to do the t ransforma t i on and c l i pp i ng stuf f. Here was the company tha t was supposed to be l ead i ng the way i n graph i cs unw i l l i ng to do anyth i ng about i t. They cou l d se l l these megabuck boxes, why d i d they need a cheap one tha t d i d the same th i ng? F i na l l y J i m Cl ark d i d i t, s i x years l a ter. As ear l y as 1972 you were say i ng we 'd soon reach the l i mi ts of Von Neumann arch i tec ture. And ye t even today severa l more orders of magn i tude i n per formance can st i l l be got ten out of tha t arch i tec ture through submi cron sca l i ng and i ncreased compac t i on. CARVER : But you see the Von Neumann mach i ne doesn ' t take very good advantage of tha t. For one th i ng i t t rea ts a l l memory as i f i t were equ i d i stant. I t makes everyth i ng as hard to ge t to as everyth i ng e l se. They t ry to compensa te w i th th i ngs l i ke caches, where they put some memory c l oser to the processor, but tha t ' s a f ter the fac t. I t ' s not bu i l t i nto the arch i tec ture. So as a resu l t Von Neumann arch i tec ture ac tua l l y works aga i nst i ncreased compac t i on a t the ch i p l eve l. I t doesn ' t sca l e we l l a t a l l. I t ' s not a way you 'd ever des i gn a computer i f you had VLSI techno l ogy i n mi nd. I f you want to ge t the fu l l bene f i t of th i s marve l ous new techno l ogy - - th i s new med i um, i f you w i l l - - you need a who l e new way of th i nk i ng about computer arch i tec ture. Everybody v i ewed th i s techno l ogy as a cost reduc t i on mechan i sm for t rad i t i ona l des i gns i nstead of as a new med i um w i th wh i ch to rea l i ze new c l asses of arch i tec ture. I ' ve spent the l ast f i f teen years of my l i fe t ry i ng to ge t peop l e to th i nk i n a f resh new way. And they haven ' t been do i ng tha t. The so l i d fundamenta l work tha t ' s necessary even to understand wha t ' s sens i b l e to do i sn ' t be i ng pursued i n very many p l aces. I mean the fundamenta l l i mi tat i ons work f rom an arch i tec tura l and a l gor i thmi c standpo i nt, not f rom a techno l ogy po i nt of v i ew. We ' re i n pre t ty good shape techno l og i ca l l y. There ' s a l ot of depth

4 MeadInterv i ew 4 F i l e : 1 Carver.doc there across the board. But i t ' s not para l l e l ed by s i mi l ar depth i n the l ead i ng-edge systems area. We don ' t have good peop l e i n every ma j or depar tment a t every ma j or un i vers i ty do i ng exce l l ent work on the cut t i ng edge of arch i tec ture as we do i n dev i ce phys i cs and ma ter i a l s research. Bas i ca l l y tha t ' s j ust star t i ng to happen now. There have been some s i gns of l i fe recent l y, but tha t ' s a f ter f i f teen years of bea t i ng on i t. You sa i d i n 1977 tha t there was an e i ght -order -of -magn i tude poss i b i l i ty i n compac t i on of i ntegra ted c i rcu i ts - - potent i a l l y 100 mi l l i on t rans i stors on a ch i p. How do those ear l y pro j ec t i ons l ook to you today? CARVER : The or i g i na l ana l ys i s we d i d i n 1972 of how sma l l you cou l d make dev i ces - - one quar ter of a mi cron - - i s ho l d i ng up remarkab l y we l l. The ch i p s i zes have a l so been go i ng up, a l though s l ow l y. In tha t 1977 repor t we were th i nk i ng of a ch i p about a square cent i me ter, approx i ma te l y wha t they are today. So i f you sca l e the dev i ces to a quar ter -mi cron and en l arge the ch i p area a l i t t l e more you can ge t a hundred-mi l l i on t rans i stors on a ch i p. Tha t ' s w i th the techno l ogy tha t ' s known today. There ' s not a s i ng l e step i n there tha t hasn ' t been proven feas i b l e. You 'd use i on-beam dry e tch i ng and X- ray l i thography. Tha t w i l l be done i n the next decade. There shou l d be peop l e do i ng i t now. How much more comput i ng power i s ava i l ab l e beyond today ' s techno l ogy j ust by sca l i ng everyth i ng to the l i mi ts? CARVER : I t depends on wha t you count. You can count computat i on per un i t power or un i t area. I th i nk the fa i rest th i ng i s computat i on per do l l ar. Wha t are you go i ng to ge t for your money i n terms of cyc l es per second of rea l computat i on? From now on tha t number i s go i ng to go rough l y l i ke the square of the sca l i ng. Idea l l y, when you sca l e everyth i ng the speed/power produc t goes l i ke the cube : sw i tch i ng speed i ncreases l i near l y as you sca l e down and dens i ty goes up as the square. I f i t ' s two to one i n d i mens i on i t ' s four to one i n dens i ty - - i f you reduce a t rans i stor by a fac tor of two on each s i de, four l i t t l e t rans i stors now f i t where the b i g one used to be. But i n add i t i on to tha t they ' re runn i ng tw i ce as fast. So every t i me you go down a fac tor of two i n s i ze you ge t a fac tor of e i ght - - a fac tor of four i n the number of ga tes and a fac tor of two i n speed. Tha t says ga te cyc l es per un i t area goes l i ke the cube a l so. But as you approach phys i ca l l i mi ts you don ' t qu i te ge t tha t fu l l cube l aw anymore, wh i ch i s why I say tha t f rom now on i t ' s go i ng to go l i ke the square. So i f we sca l e f rom, say, 1.25 down to.25 mi cron, tha t ' s f i ve to one, wh i ch i s 125 t i mes i n ga te cyc l es per un i t area and a l so 125 t i mes i n how much computat i on you ge t per un i t power ; i t ' s probab l y not qu i te tha t good due to approach i ng phys i ca l l i mi ts, but we ' l l cer ta i n l y ge t a fac tor of 100. And remember we ' re not rea l l y a t 1.25 mi cron ye t. I t ' s rea l l y a 1.5 process today, and tha t ' s on l y the l ead i ng edge ; three mi crons i s the standard produc t i on process i n today ' s commerc i a l wor l d. So tha t means we ' re ta l k i ng about ch i ps a thousand t i mes faster than the ones i n today ' s persona l computer once we reach a l l those l i mi ts. And tha t ' s not count i ng concur rent opera t i on. A computer w i th a thousand of those ch i ps runn i ng concur rent l y wou l d be a mi l l i on t i mes faster than today ' s mach i ne. Wha t about three-d i mens i ona l ch i ps? CARVER : These th i ngs are bu i l t up i n l ayers and i n pr i nc i p l e you cou l d keep go i ng. Everybody knows how to do tha t. For examp l e, SOI - - s i l i con on i nsu l a tor - - i s probab l y the most promi s i ng way to star t stack i ng th i ngs. But there are other ways. The prob l em i s tha t the y i e l d - - the overa l l probab i l i ty tha t the th i ng works - - goes down exponent i a l l y w i th the number of l ayers i n the process, even i f i t ' s j ust an i nsu l a t i on l ayer or a l ayer of w i r i ng. And exponent i a l s are dead l y th i ngs. Af ter a wh i l e they rea l l y ge t you. And i t ' s not j ust the y i e l d prob l em. As you make th i ngs sma l l er they a l so become much more suscept i b l e to th i ngs l i ke cosmi c rays and a l l k i nds of other mechan i sms of fa i l ure. Compound ma ter i a l s l i ke ga l l i um arsen i de are supposed to be l ess vu l nerab l e to sof t er rors. CARVER : Every techno l ogy has i ts own hor r i b l e prob l ems and tha t i nc l udes ga l l i um arsen i de. There ' s no mag i c. The number of e l ec t rons crea ted by i on i z i ng par t i c l es i sn ' t t remendous l y d i f ferent be tween ma ter i a l s, not by huge orders of magn i tude. I t ' s rea l l y a s i ze quest i on, a st ra i ght - forward sca l i ng i ssue - - how many e l ec t rons represent your s i gna l versus how many e l ec t rons are crea ted when you put an i on i z i ng par t i c l e through the ma ter i a l? As you sca l e down there are fewer e l ec t rons per dev i ce, so i t becomes eas i er for a random par t i c l e to sw i tch the t rans i stor for the same reason i t ' s eas i er for you to sw i tch i t.

5 MeadInterv i ew 5 Fi le : 1 Carver.doe CARVER : I be l i eve tha t the v i s i on of the or i g i na l founders of AI - - the M i nskys and McCar thys - - was ext reme l y cor rec t. But when they got i nto i t they d i scovered tha t wha t they were rea l l y headed for took e i ght or n i ne orders of magn i tude more computat i on than you cou l d ge t out of a regu l ar computer. And so there came to be two groups : one went l ook i ng for tha t e i ght or n i ne orders of magn i tude ; the other j ust punted and faked i t - - and tha t ' s the vast ma j or i ty of the AI commun i ty today. I 'm one of the peop l e who ' s of f t ry i ng to f i nd tha t e i ght or n i ne orders of magn i tude. I ' ve heard ce l l u l ar automa ta used as a genera l term for mass i ve l y para l l e l arch i tec tures. CARVER : Ce l l u l ar automa ta are an i nvent i on of Von Neumann, st range l y enough. He descr i bed them back i n the for t i es. I t was the f i rst u l t ra-concur rent arch i tec ture, a rea l l y good f i rst step i n wha t ' s now become a l arge c l ass of th i ngs. For examp l e, you can v i ew Kung ' s systo l i c a l gor i thms as an extens i on of ce l l u l ar automa ta. And i f you l ook a t i t tha t way, i t ' s a very na tura l evo l ut i on. But i n fac t a ce l l u l ar automa ton i s an ext reme l y prec i se th i ng - - an i nterconnec ted se t of f i n i te automa ta. Trad i t i ona l ce l l u l ar automa ta have on l y been connec ted to ne i ghbors, and as such they represent on l y very l oca l computat i ons. Any such l oca l computat i on can be expressed as a ce l l u l ar automa ton but i t ' s of ten not re l evant to do so. Very few a l gor i thms map we l l i nto tha t doma i n. So they ' ve k i nd of boxed themse l ves out of a l ot of the more i nterest i ng v i ews of computat i on. Wha t about the demarca t i ons be tween l eve l s of i ntegra t i on, l i ke SSI, MSI, and so for th? CARVER : To me the d i f ferences aren ' t so much i n t rans i stor count as i n the l eve l of func t i ona l i ty, tha t i s, how you have to th i nk about i t i n terms of wha t i t ' s do i ng. The f i rst i ntegra ted c i rcu i t i n 1959 had one ga te or one f l i p- f l op, so you cou l d th i nk of them as e l ementary l og i c e l ements. Med i um-sca l e ch i ps had th i ngs l i ke counters and reg i sters. Now peop l e cou l d star t th i nk i ng a t a h i gher l eve l than j ust ga tes. Tha t was i n the mi dd l e s i xt i es. The next rea l step happened when memor i es and mi croprocessors were i ntegra ted on the ch i p around Tha t ' s when LSI beg i ns for me, not because of the number of t rans i stors but because now i t was a comp l e te system- l eve l func t i on. Those ear l y mi cros on l y had about three thousand t rans i stors but they were a processor never the l ess and they changed the way peop l e thought. Now ac tua l l y i f you l ook a t the Inte l 286 today, i t i sn ' t rea l l y more capab l e - - or not very much - - than those ear l y ch i ps. I t ' s got a l ot more t rans i stors but the arch i tec ture ' s pre t ty much the same ; the i nst ruc t i on se t ' s pre t ty much the same. So I don ' t put i t i n a d i f ferent c l ass f rom the Al so, accord i ng to th i s v i ew the ga te-ar ray bus i ness represents a throwback : ga te ar rays are the SSI l eve l and standard ce l l s are the MSI l eve l. beg i n? I f 32-b i t mi croprocessors aren ' t qua l i tat i ve l y d i f ferent f rom 8-b i t vers i ons, then when does VLSI CARVER : I 'm l ook i ng forward to the mach i ne tha t ' l l do ray- t rac i ng i n rea l t i me. I t ' s not beyond our capab i l i ty r i ght now. Prov i ded you comp i l ed the a l gor i thms i nto s i l i con i n a mass i ve ar ray. I don ' t be l i eve tha t po l ygona l representat i on for shapes i s the r i ght one. I th i nk stuf f l i ke superquadr i cs i s a who l e l ot more sens i b l e - - maybe not exac t l y tha t representat i on, but some th i ng tha t has some n i ce ma thema t i ca l proper t i es when i t comes to ray- t rac i ng. So i f you ' re go i ng to use.cp10 a ray- t rac i ng a l gor i thm you 'd be t ter use some th i ng other than po l ygons. In fac t, you have to i nvent the representat i on and the a l gor i thm toge ther. Tha t ' s why nobody ' s go i ng to do i t un l ess they ge t the i r head around the who l e prob l em. CARVER : My own work i s concerned w i th mus i c. As you know, i f you t ry to s i mu l a te even one mus i ca l i nst rument you ' re i n for a t l east 10 MIPs wor th of computat i on for a s i ng l e vo i ce - - one st r i ng on a gu i tar or wha tever. In some cases you ge t a who l e i nst rument l i ke a f l ute, but i f you t ry to do a p i ano you ' ve got to have a huge number of processors. So you ge t i nto the g i gaops very qu i ck l y to s i mu l a te any reasonab l e ensemb l e a t a l l. A who l e orchest ra i s a l ot of g i gaops. And i t turns out you can do tha t. We ' re r i ght now l ook i ng a t about 10 MIPS per ch i p for do i ng mus i c. A spec i f i c arch i tec ture for do i ng mus i c, not good for anyth i ng e l se, and you can make some of the most beaut i fu l vo i ces you ' ve ever heard i n your l i fe. We ' l l be ab l e to s i mu l a te a very conv i nc i ng orchest ra. Now I 'm not say i ng we ' re ever go i ng to rep l ace a rea l orchest ra, but there ' s a se t of th i ngs you can do i f you have the ab i l i ty to crea te your own orchest ra. Th i s i s wha t the synthes i zer guys ought to be do i ng and can ' t ; and the reason they can ' t i s tha t you need four or f i ve more orders of computat i on than i s ava i l ab l e today. Tha t ' s wha t prevents them f rom t rue orchest ra l s i mu l a t i on. A standard synthes i zer sounds l i ke he l l and i t doesn ' t have to, but you ' ve got to f i nd tha t four or four or f i ve orders of magn i tude - - i n th i s case i t i sn ' t e i ght or n i ne, on l y four or f i ve, but i t ' s st i l l a l ot. And we can put i t i n a persona l workstat i on for use by composers. I be l i eve tha t ' s the way compos i t i on w i l l be done. Composers

6 MeadInterv i ew 6 F i l e : 1 Carver.doc s i mp l y cannot a f ford to exper i ment. You can ' t a f ford to pay a hundred un i on mus i c i ans to foo l around. So composers are rea l l y stuck, j ust be- cause they need tha t four or f i ve orders of magn i tude. We ' re st i l l i n the very ear l y exper i menta l stages but we can a l ready mi x amaz i ng vo i ces. Most peop l e say i t sounds l i ke a rea l i nst rument, not l i ke a computer. Except you can t ransform i t and move i t i nto d i f ferent spaces. We can s i mu l a te a mar i mba bar tha t wou l d have to be 27 fee t l ong. End of document I

7 MeadInterv i ew2 1 F i l e : 2 Carver.doc 10/00 CARVER MEAD INTERVIEW PART TWO Document 2 of 4 of Mead Interv i ew By : Gene Youngb l ood CARVER : Fu l l custom handcra f ted ch i ps typ i ca l l y take three years. Wha t tha t means i s tha t peop l e hand des i gn cer ta i n p i eces and then they use some k i nd of computer a i d to he l p them put the stuf f toge ther. Nobody s i ts down and draws the who l e ch i p on one g i ant shee t of my l ar any more l i ke they used to. Tha t ' s i mposs i b l e. At l east ten years ago peop l e began us i ng s i mp l e CAD systems to he l p p l ace the p i eces they 'd dra f ted by hand. Tha t ' s j ust p l a i n common sense. Even the o l d k i nd of des i gn a i ds he l ped. And there ' s been th i s cont i nua l evo l ut i on of th i ngs tha t made tha t eas i er to do. So any ch i p done i n the l ast ten years, nobody sa t down and drew the who l e th i ng on one p i ece of paper. They drew p i eces of i t and then they p l ot ted i t and then they drew some more, and so on. You can ge t there tha t way but i t ' s an enormous amount of e f for t. I f you have a good me thodo l ogy and good peop l e tha t understand the des i gn you can do some pre t ty amb i t i ous ch i ps us i ng not very advanced too l s, i f you ' ve thought the th i ng through. By wha t fac tor does a s i l i con comp i l er reduce des i gn t i me? CARVER : Maybe 75% or an order of magn i tude. From severa l weeks to severa l months. I t depends on wha t you count ; a l ot of des i gn goes i nto j ust conceptua l i z i ng wha t you need. One th i ng a s i l i con comp i l er a l l ows you to do wh i ch we never cou l d do be fore i s exp l ora tory arch i tec ture, where you can ac tua l l y t ry a des i gn and see how i t comes out ; i f i t doesn ' t work you t ry some th i ng e l se. We cou l d never tha t be fore because the amount of energy to i mp l ement a des i gn i s so h i gh tha t once you ge t on an approach you force i t to f i n i sh, even i f i t ' s a hor r i b l e ch i p when you ' re done. And there ' s a l ot of examp l es on the marke t. They got f i n i shed because economi ca l l y i t ' s not feas i b l e to scrap them and star t over. W i l l des i gn t i me ever be i ndependent of ch i p comp l ex i ty? CARVER : I t depends on wha t you mean by comp l ex i ty. I t ' s cer ta i n l y not a d i rec t func t i on of the number of t rans i stors ; i t ' s a func t i on of how much the des i gner has to th i nk about the des i gn. Tha t ' s qu i te a d i f ferent ma t ter. Tha t has not been a d i mens i on a l ong wh i ch peop l e have measured th i ngs very much. So des i gn t i me w i l l i ndeed become i ndependent of raw t rans i stor count, but peop l e w i l l f i gure out conceptua l l y more comp l ex th i ngs and then i t ' l l take them l onger to ge t those f i gured out. Wha t a s i l i con comp i l er does i s make the i mp l ementat i on t r i v i a l. I t doesn ' t mean ge t t i ng the i dea i s any eas i er. W i l l some th i ng equ i va l ent to supercomputer power be necessary for s i l i con comp i l a t i on as we approach the mu l t i mi l l i on- t rans i stor l eve l of comp l ex i ty? Even now Cray i s a l ready promot i ng i ts mach i nes for VLSI des i gn. CARVER : Wha t peop l e th i nk of when they say supercomputer i s go i ng to change. I a l ready showed you a ch i p tha t ' l l do about 600 VAXes wor th of computat i on. The ac tua l numbers of adds and mu l t i p l i es are on l y maybe f i ve or ten mi l l i on per second ; tha t sounds l i ke maybe ten VAXes. But by the t i me you ge t done shuf f l i ng a l l the da ta around and ge t t i ng everyth i ng i n the r i ght p l ace and a l l tha t, i t ends up be i ng a few hundreds t i mes rea l t i me i nstead of a few tens t i mes rea l t i me. most of wha t goes on i n a standard computer i s j ust da ta shuf f l i ng. And tha t doesn ' t change w i th a supercomputer. I t ' s the same prob l em. So peop l e are go i ng to reconceptua l i ze wha t they th i nk about supercomputers. I t ' s go i ng to be para l l e l arch i tec tures and a l ot of spec i a l -purpose arch i tec ture. Sure, there ' l l be some standard "genera l -purpose " arch i tec tures but the b i g break throughs w i l l be i n spec i a l purpose arch i tec tures. The who l e not i on of supercomput i ng i s go i ng to change. The peop l e i n tha t bus i ness don ' t want i t to change, because they ' re i n tha t

8 MeadInterv i ew2 2 F i l e : 2 Carver.doc bus i ness. We l l, l e t us say then, w i l l power i n the supercomputer range, no ma t ter how i t ' s ach i eved, be necessary for VLSI des i gn? CARVER : Sure. But wha t ' s rea l l y go i ng to happen i s tha t i nstead of do i ng a supercomputer peop l e w i l l bu i l d spec i a l l i t t l e acce l era tors to do p i eces of the prob l em. There ' l l be a l i t t l e s i mu l a t i on eng i ne tha t runs s i mu l a t i ons l i ke crazy. These var i ous th i ngs tha t do the par t i cu l ar computat i ons, and you ' l l p l ug those boards i nto your workstat i on. Why i s the s i l i con comp i l er d i e s i ze l arger than hand-cra f ted ch i ps? CARVER : There are a l ot of approaches to s i l i con comp i l a t i on today. The one I worked on, the ch i p s i ze for a g i ven func t i on i s much sma l l er than ga te ar rays or standard ce l l s but st i l l l arger than hand des i gn of course, because tha t ' s wha t hand des i gn does - - i f you see some space you pack some stuf f i nto i t. In a very funny way i t does ge t to the po i nt where tha t ' s compensa ted. Wha t rea l l y happens i s peop l e redo the arch i tec ture to make the ch i p more e f f i c i ent. So ac tua l l y for a g i ven func t i on they can ge t th i ngs tha t are ac tua l l y sma l l er than hand des i gns by exp l or i ng the arch i tec tura l approaches. But i f you took any one of those once you ' re f i n i shed and redes i gned i t by hand i t wou l d be sma l l er. But you can ' t a f ford to do tha t i n VLSI. So the who l e th i ng i s rea l l y decept i ve because wha t a s i l i con comp i l er a l l ows you to do i s exper i ment w i th the arch i tec ture unt i l you f i nd the most e f f i c i ent arch i tec ture, and you can never do tha t i n hand des i gn. Peop l e w i l l l ook a t a g i ven ch i p and say " Oh, I cou l d make tha t sma l l er by hand, " but they never wou l d have got ten there by hand. So you ' re work i ng a d i f ferent end of the prob l em ; you ' re l e t t i ng peop l e exper i ment on the arch i tec tura l end. Then of course i f they wanted to take tha t par t i cu l ar arch i tec ture and repack i t by hand you can a l ways ge t i t phys i ca l l y sma l l er. I t ' s a l ways t rue tha t star t i ng w i th some th i ng tha t ex i sts you can a l ways make i t be t ter by work i ng on i t. Wha t the s i l i con comp i l er does i s l e t you star t f rom scra tch and ge t to some th i ng tha t ex i sts tha t ' s rea l l y qu i te e f f i c i ent. Is " fu l l custom" synonymous w i th s i l i con a l gor i thms? CARVER : I t means hand cra f ted. I t ' s not synonymous w i th a ded i ca ted arch i tec ture. H i stor i ca l l y, " fu l l custom" has meant tha t you g i ve the th i ng to somebody i n a l i t t l e des i gn shop and they do a hand-cra f ted des i gn for tha t purpose. In the l i tera ture they on l y say tha t s i l i con comp i l ers are for fu l l custom ch i ps ; they don ' t spec i f i ca l l y ta l k about ded i ca ted arch i tec tures. Tha t ' s i ncred i b l e, g i ven the fac t tha t everyone wou l d acknow l edge tha t ded i ca ted arch i tec tures are i nherent l y faster than genera l ones, and the on l y h i stor i ca l bar r i er to do i ng tha t has been the des i gn t i me, and now you have a techno l ogy tha t can do i t and they don ' t even remark tha t ach i evement. CARVER : I t ' s a l ways t rue when you have a new techno l ogy tha t the on l y terms i n wh i ch peop l e can d i scuss i t are the o l d terms. So i t ' s rea l l y d i f f i cu l t to exp l a i n to peop l e how th i ngs have to work when you have a new techno l ogy. The on l y th i ng peop l e can do i s compare w i th wha t they know. The on l y th i ng they know i s tha t h i stor i ca l l y peop l e have hand-cra f ted some des i gns. There ' s the i ssue of who spec i f i es the arch i tec ture, the des i gners or the comp i l er. Today we ' re not ye t a t the po i nt where the computer s i mp l y doesn ' t have the i nte l l i gence to do tha t. So the des i gner spec i f i es the arch i tec ture and then exper i ments w i th i t through the s i l i con comp i l er. But apparent l y there are peop l e pursu i ng the AI approach or exper t systems approach where the comp i l er wou l d have suf f i c i ent i nte l l i gence to ac tua l l y make arch i tec tura l dec i s i ons. CARVER : We l l the AI peop l e haven ' t done anyth i ng ye t so i t ' s hard to make comments about i t. Tha t doesn ' t mean they won ' t. There ' s cer ta i n l y room for heur i st i cs i n he l p i ng w i th the process, to

9 MeadInterv i ew2 3 F i l e : 2 Carver.doc the extent tha t wha t we mean by AI i s a heur i st i c approach to opt i mi za t i on - - of course there ' s room for tha t and everybody ' s go i ng to be do i ng tha t. But there ' s another th i ng go i ng on tha t you need to know about. Wha t a l ot of peop l e mean by automa t i c arch i tec ture i s f i xed arch i tec ture. They have p i cked the arch i tec ture ahead of t i me and they comp i l e a ch i p i n tha t c l ass. And a l l the ear l y s i l i con comp i l ers were l i ke tha t. I wrote one myse l f i n 1971 tha t was l i ke tha t. i t on l y d i d one l i t t l e arch i tec ture. And i t d i d a l l the s i l i con comp i l i ng th i ngs - - i t took i n the th i ng and genera ted the s i mu l a t i on f rom the same source tha t genera ted the ar twork, but i t on l y d i d th i s one l i t t l e t i ny arch i tec ture. And then when Dave Johannsen d i d h i s th i ng he d i d a w i der c l ass of arch i tec ture. And a l ot of peop l e now are do i ng an even w i der c l ass of arch i tec tures, but they ' re st i l l w i th i n l i mi ts. There ' s noth i ng wrong w i th tha t i f tha t arch i tec ture f i ts wha t you want to do, but i t doesn ' t a l l ow you to do arch i tec tura l exp l ora t i on, wh i ch i s one of the d i mens i ons of s i l i con comp i l a t i on tha t ' s rea l l y h i gh l y i mpor tant. The approach tha t the S i l i con Comp i l er Inc. peop l e too l - - wh i ch i s Dave Johannsen and h i s peop l e - - i s to bu i l d a too l for peop l e to do genera l arch i tec ture work. So they bu i l t a too l for arch i tec ts ; i t does i mp l ementat i ons of systems for peop l e who want to do arch i tec ture - - to bu i l d a l gor i thms i n s i l i con, i f you l i ke. tha t i sn ' t the on l y th i ng tha t needs to be done. A coup l e of compan i es back east have bu i l t s i l i con comp i l ers for bu i l d i ng mi crocode eng i nes. Tha t ' s go i ng to have an i mpac t on peop l e who want to bu i l d mi crocode eng i nes - - tha t ' s a da ta pa th w i th a se t of f i n i te-state mach i nes tha t sequences i t. Our own ear l y s i l i con comp i l ers were bas i ca l l y a i med a t tha t th i ng too. s i nce then they ' ve become genera l i zed. And of course once you have a genera l too l you can a l ways put a l ayer on top of i t tha t has a h i gher l eve l representat i on and a l l ows you then to genera te the i nput for the genera l ch i p comp i l er. So the way I l ook a t the ac tua l evo l ut i on of the i ndust ry i s there w i l l be a few rea l l y genera l ch i p comp i l ers, a bunch of f ront -ends for those tha t a l l ow peop l e to take a spec i a l th i nk i ng process or heur i st i c programs or wha tever and genera te i nput for tha t, then there w i l l be some honest - to-god spec i a l - purpose s i l i con comp i l ers. Can we ach i eve VLSI dens i ty w i thout sacr i f i c i ng c l ock speed? For examp l e, b i po l ar and CMOS are converg i ng today and peop l e say CMOS w i l l outper form b i po l ar. So w i l l be we ab l e to car ry a 100 MHz c l ock i nto VLSI? CARVER : Oh sure. The prob l em w i th tha t i s not tha t you can ' t make dense th i ngs tha t run a t h i gh c l ock ra tes, i t ' s tha t you ' re not go i ng to have the ent i re ch i p comp l e te l y l ockstep a t those c l ock ra tes. Because j ust d i st r i but i ng a c l ock a t those f requenc i es means tha t you ' re go i ng to l ose the synchrony over the ch i p. You shou l d ask Chuck Se i t z about the fac t tha t d i f ferent p i eces of the ch i p are go i ng to have to be ab l e to opera te autonomous l y. And you coup l e them i n such a way tha t the i r c l ocks don ' t have to be abso l ute l y i n phase. You run p i eces of the ch i p on very fast c l ocks and you commun i ca te be tween them w i th some other f requency. I f you th i nk of any mach i ne as be i ng a c l ock, i n the norma l human-sca l ed wor l d there seems to be a constant ru l e wh i ch says the fastest c l ock has to use the most energy, and the one uses the most energy usua l l y has some s i ze const ra i nts on i t i n order to d i ss i pa te tha t hea t. There fore, i n the norma l phys i ca l human wor l d the fastest c l ock tha t wou l d a l so use the l east amount of energy i s an i mposs i b l e c l ock. But when you ge t down to the submi cron doma i n you have an i mposs i b l e c l ock, because i n order to ach i eve both h i gh speed and h i gh dens i ty compac t i on, i t has to use the l east amount of energy compara t i ve l y speak i ng. CARVER : We l l, remember tha t wha t you sa i d i n the beg i nn i ng i s rea l l y t rue : i f you t ry to take a who l e VLSI ch i p and c l ock i t a t 100 MHz i t w i l l i ndeed d i ss i pa te an i mposs i b l e amount of energy. But you don ' t have to run the who l e ch i p a t the same c l ock ra te and you don ' t have to c l ock every e l ement every t i me - - there are a l ot of ways you can ge t the e f fec t of tha t speed w i thout hav i ng th i s g l oba l th i ng tha t ' s j ust pump i ng a l l tha t charge a l l the t i me. Tha t ' s j ust phys i cs. I f you have every e l ement i n there sw i tch i ng every t i me, then there ' s j ust the amount of stored energy t i mes the f requency, and tha t doesn ' t change depend i ng upon how many e l ements. You make the e l ements sma l l er, there are more of them, so i n tha t sense you can run more th i ngs faster w i th the same power ; but the bas i c phys i cs doesn ' t change. Pump i ng th i ngs up and down a t a fast ra te takes a huge amount of energy and the faster the ra te and the more th i ngs, the more energy. Tha t par t

10 MeadInterv i ew2 4 F i l e : 2 Carver.doc doesn ' t change. I t ' s rea l l y the area : how much area do you pump? Th i nk of the ch i p as a b i g capac i tor and you ' re j ust pump i ng charge i n and out of i t, and every t i me you pump i t i t ' s one-ha l f CV squared tha t you l ose i n energy ; so the power i s one-ha l f CV squared t i mes the f requency. Per i od. Capac i tance t i mes V-squared hasn ' t been chang i ng a l ot w i th ch i p evo l ut i on ; capac i tance has been ge t t i ng a l i t t l e b i gger, the vo l tage a l i t t l e l ower ; i t ' s ge t t i ng a l i t t l e be t ter but bas i ca l l y tha t ' s one of those th i ngs tha t doesn ' t change much. Wha t ' s rea l l y go i ng on i s tha t we can make i nd i v i dua l p i eces of the c i rcu i t ext reme l y fast and then you do some th i ng to not have to run the who l e c i rcu i t tha t fast. For examp l e, commun i ca t i ons cont ro l l ers have to decode stuf f comi ng i n of f of, say, a l oca l -area ne twork a t 10 MHz ; tha t means you ' ve got to have some mu l t i p l e of tha t i n your reso l ut i on of th i ngs, so tha t ' s got to be fast on the f ront -end and for er ror -cor rec t i ng, but then you ge t i t i n and you go i nto some para l l e l th i ng tha t can run a l ot s l ower, so you do most of your process i ng a t a l ot s l ower ra te, but i n para l l e l. So your h i gh c l ock ra tes are pr i mar i l y for i nter fac i ng the ch i p w i th the outs i de wor l d. I f you ' re i nter fac i ng w i th an opt i ca l f i ber you ' ve got to be runn i ng i n the g i gaher t z range. But somehow there ' s a way of not hav i ng to sw i tch every e l ement every t i me. (Chuck can te l l you some n i ce th i ngs about the who l e se l f - t i mi ng th i ng where you on l y do sw i tch i ng i f you need i t. Th i ngs don ' t sw i tch un l ess they change). Geof f rey Fox c l a i ms the speedup through concur rency i s a l i near func t i on of the number of PE' s, per i od. CARVER : I have gone on record as say i ng tha t the spec i a l -purpose arch i tec tures are go i ng to be the way to approach h i gh l y concur rent arch i tec tures. The reason for tha t i sn ' t tha t i sn ' t tha t - - yeah, for Geof f rey ' s par t i cu l ar prob l em, he can ar range th i ngs i n such a way tha t you on l y have nearest ne i ghbor commun i ca t i ons and then you can ge t a l i near i ncrease i n computat i on w i th the number of e l ements. And tha t ' l l work unt i l he wants to do some th i ng a l i t t l e more soph i st i ca ted. The computat i ons they ' re do i ng i sn ' t very d i f ferent f rom the one you have to do i n mus i c, and they ' re ge t t i ng about one VAX per c i rcu i t board wor th of computat i on and we ' re ge t t i ng about 600 VAXes per ch i p. Tha t shou l d ca l i bra te you a l i t t l e b i t. Wha t ' s the prob l em w i th do i ng f l oa t i ng-po i nt i n s i l i con? CARVER : We chose to do 64-b i t f i xed po i nt i nstead of 16-b i t f l oa t i ng po i nt i n our mus i c ch i ps because i t ' s much c l eaner and for a l ot of app l i ca t i ons i f you go to a rea l l y l ong word you can use a f i xed po i nt number every b i t as e f fec t i ve l y as you can use a shor ter f l oa t i ng po i nt number, and the computat i on does ge t enormous l y s i mp l i f i ed. I mean, do i ng f l oa t i ng po i nt i s a b i tch. There ' s no quest i on about tha t. Because you have a l l the i nterac t i on be tween the exponent and the mant i ssa ' s. In par t i cu l ar, adds are hor r i b l e. There ' s a l ot of f l oa t i ng po i nt i n graph i cs. CARVER : We l l nobody ' s ever thought through i f you rea l l y need to do tha t or not. I t ' s j ust tha t, they l ook a t the dynami c range they have, wh i ch i s huge, and they say gee we ' ve got to do f l oa t i ng po i nt. On the other hand they ' ve on l y got a 1000 by 1000 ma t r i x, so i t ' s cer ta i n l y a f i n i te reso l ut i on of th i ngs. So I be l i eve nobody ' s rea l l y thought through whe ther you can do tha t w i th a l ong-word f i xed po i nt ar i thme t i c. Nobody has a l ong-word, f i xed po i nt eng i ne, i t ' s a l l 16-b i t or 32-b i t. Al gor i thms i n s i l i con wou l d fac i l i tate tha t. But I 'm rea l l y not sure whe ther i t wou l d be more e f fec t i ve to buy a bunch of f l oa t i ng po i nt eng i nes to do i t or whe ther i t wou l d be more e f fec t i ve to j ust have some l ong-word f i xed-po i nt eng i nes - - of wh i ch you cou l d have a l ot more on the same s i l i con. And then you ask yourse l f wha t the t radeof f i s. And i t ' s j ust an eng i neer i ng t radeof f, i t ' s not a re l i g i ous i ssue. Geof f rey Fox sa i d tha t for h i s par t i cu l ar sc i ent i f i c prob l ems, the l arger the prob l em the l ess v i ab l e i t i s to bu i l d i t i nto hardware. The more par ts you have w i th grea ter degrees of f reedom, the more genera l the prob l em and you need a genera l purpose computer. CARVER : Tha t ' s the o l d l ore of the programmab l e mach i ne peop l e. They know how to program ;

11 MeadInterv i ew2 5 F i l e : 2 Carver.doc they don ' t know how to des i gn ch i ps. So they assume i ts eas i er to wr i te a program. They can ' t conce i ve tha t w i th a ch i p comp i l er you cou l d comp i l e a spec i a l -purpose arch i tec ture for a prob l em eas i er than you can f i gure out how to use one of those stup i d programmab l e h i gh l y concur rent mach i nes. R i ght now i f you l ook a t the amount of programmi ng t i me i t takes use an ar ray processor, you cou l d eas i l y have a spec i a l -purpose arch i tec ture up and runn i ng be fore you cou l d have the same a l gor i thms runn i ng on an ar ray mach i ne. I 'm not say i ng tha t ' s a l ways t rue ; there i s a l ot of mer i t to programmab l e th i ngs you can change i n some way ; but i t ' s a b i g mi stake to th i nk tha t sequent i a l programmi ng l anguages are go i ng to work on concur rent mach i nes for anyth i ng except the most s i mp l e prob l ems. I t ' s rea l easy when everyth i ng ' s do i ng the same th i ng, to wr i te down wha t i t does and you have a bunch of Von Neumann mach i nes work i ng on i t and i ts st ra i ght forward. But when i t ' s more comp l i ca ted than tha t i t ' s not so t r i v i a l any more. I ' l l once aga i n re fer to the mus i c prob l em, where the i nst ruments are separab l e i n the sense tha t a vo i ce i s separa te f rom other vo i ces, but then they have to ge t choreograph- ed toge ther w i th some k i nd of conduc tor k i nd of th i ng, and each one ' s a l i t t l e d i f ferent, and now i t i sn ' t qu i te so t r i v i a l anymore. I t ' s probab l y the same th i ng i n graph i cs i f you want to do photorea l s i mu l a t i ons of na tura l phenomena. Tha t ' s very i nterest i ng. The phys i c i sts w i l l say they ' re do i ng the rea l ser i ous work, mode l i ng the un i verse, and i f you want to p l ay w i th the b i g boys and mode l the un i verse, then you need a genera l purpose supercomputer. But i t seems to me tha t mode l i ng na tura l phenomena for v i sua l s i mu l a t i on i s a t l east as comp l ex i f not more so i n terms of the dynami c comp l ex i ty of the prob l em. CARVER : I t has one other advantage : you can te l l i f you ' ve done i t. A who l e l ot of phys i cs these days has the prob l em of how many ange l s are danc i ng on the head of a p i n. So cou l d i t be tha t mode l i ng the un i verse i s a s i mp l er prob l em than mode l i ng an orchest ra or a sma l l ensemb l e? CARVER : Yes, i t ' s more rea l. Phys i cs has got ten i tse l f of f i n l e f t f i e l d. I don ' t want to ge t my phys i cs f r i ends mad a t me. I do a l ot of phys i cs myse l f. But the phys i cs commun i ty i s i n a b i t of t roub l e nowadays and the reason i s i t has l ost a l ot of contac t w i th rea l i ty. I mean the k i nds of exper i ments where they l ook for these par t i c l es a t some enormous energ i es and they have some symme t ry groups wh i ch they th i nk exp l a i n the par t i c l es, wh i ch are rea l l y j ust a way of ca ta l og i ng. A symme t ry group doesn ' t exp l a i n anyth i ng. I t ' s l i ke Mende l ev be fore we understood wha t made the per i od i c tab l e ; you cou l d see there was per i od i c i ty i n i t and you cou l d make some pred i c t i ons f rom tha t. We l l tha t ' s n i ce but i t doesn ' t say why i t ' s there. My percept i on i s tha t they ' ve got a very pecu l i ar v i ew go i ng i n phys i cs and they cover tha t up w i th a l ot of ego. I t ' s very, very far f rom anyth i ng d i rec t and exper i encab l e by rea l human be i ngs or even measurab l e by rea l human be i ngs. I f you l ook a t the exper i ments tha t are done by casts of thousands on b i l l i on-do l l ar fac i l i t i es, the who l e th i ng has taken on an a i r of unrea l i ty tha t ' s j ust monumenta l. I f the 286 i sn ' t s i gn i f i cant l y d i f ferent f rom the 8008 i n the sense tha t i t ' s a mi croprocessor, then a t wha t l eve l of th i nk i ng does VLSI beg i n for you? CARVER : I t ' s cer ta i n l y t rue tha t the mi croprocessor star ted an era where computat i on and s i l i con weren ' t separa te any more. In other words the rea l s i gn i f i cance of the mi cro- processor was tha t peop l e cou l d no l onger th i nk of computat i on as go i ng separa te f rom i ts techno l ogy base. In fac t i t never had been separa te. I t ' s j ust tha t peop l e thought of i t tha t way. There was the computer i ndust ry and there was some other p l ace where they bought the i r par ts, and those were j ust par ts and they d i dn ' t rea l l y ma t ter. We l l the comp l ex i ty of th i ngs cont i nued to grow because the bas i c fab process a l l owed you to put more and more t rans i stors down ; but bu i l d i ng a b i g memory i s no d i f ferent than bu i l d i ng a l i t t l e memory. So i n terms of the way the fab guys have to th i nk, i t ' s VLSI a l l r i ght because they ' ve got to make the th i ng y i e l d. So f rom the i r po i nt of v i ew i t ' s VLSI the moment i t ge ts to 100,000 t rans i stors. But f rom the po i nt of v i ew of the des i gner those memor i es are no d i f ferent than they ever were. They ' re j ust l i t t l e w i zened c i rcu i ts. And the memory des i gner ' s j ob i s no harder, i nvo l ves no more i nte l l ec tua l content, than i t d i d be fore VLSI. Have

12 MeadInterv i ew2 6 F i l e : 2 Carver.doc you heard the term " a l gor i thmi c comp l ex i ty?" The i dea i s, how wou l d you charac ter i ze the comp l ex i ty of a mach i ne? We l l, i f you l ook a t a crysta l, for examp l e, i t can have ten to the twentyfour th a toms i n i t. But you can spec i fy i t by the shape of a un i t ce l l and the way the un i t ce l l s are stacked toge ther. So there ' s rea l l y on l y two p i eces of i nforma t i on. So i n a few tens of b i ts you can spec i fy a crysta l. So i t ' s not anywhere the ten to the twenty- four th k i nd of comp l ex i ty because of the very regu l ar na ture of i t. Programs are the same way. You wr i te a l oop and the l oop can go and crea te a god-awfu l amount of stuf f, but i f i t ' s one s i mp l e l i t t l e th i ng, no ma t ter how much i t ' s repea ted i t ' s not comp l ex somehow. So i f you l ook a t i t f rom tha t po i nt of v i ew and say we l l how can we ta l k about the comp l ex i ty of s i l i con a l gor i thms, i t ' s rea l l y the sor t of un i que func t i on and the amount of un i que i nterconnec t i on tha t have to be spec i f i ed, tha t aren ' t j ust repea ted regu l ar l y, tha t g i ves you the na ture of the comp l ex i ty of the th i ng. So l e t ' s l ook a t Geof f rey Fox ' s mach i ne. I t ' s a crysta l. He ' s a l ready to l d you tha t. No ma t ter how b i g i t i s i t ' s a crysta l. I t has one k i nd of un i t ce l l tha t ' s hooked up i n some way to i ts ne i ghbors. so you have to spec i fy exac t l y wha t you have to spec i fy i n a crysta l : i t ' s no more comp l ex than the descr i pt i on of i ts un i t ce l l and of i ts connec t i ons. And the fac t tha t i t mi ght have a l onger -word ALU doesn ' t make i t more comp l i ca ted. But i f i t has a more comp l i ca ted ALU - - i f i t does f l oa t i ng-po i nt, tha t ' s a l ot more comp l i ca ted than a f i xed-po i nt ALU. But i f you go f rom a 16-b i t ALU to a 64-b i t ALU tha t doesn ' t make i t more comp l i ca ted, a l gor i thmi ca l l y. I t ' s harder for the fab guy to ge t y i e l d out of i t, so f rom h i s po i nt of v i ew i t may be qua l i tat i ve l y d i f ferent. So the way I l ook a t i t i s tha t the fab guys have crea ted a techno l ogy tha t ' s i ndependent of wha t peop l e use i t for : a systems des i gner can sue i t for e i ther a soph i st i ca ted purpose or a dead s i mp l e purpose. I f peop l e dec i de to j ust make crysta l s out of i t - - make a memory - - wh i ch doesn ' t have any more conceptua l content than memor i es ever d i d, then i t ' s j ust a cost - reduc t i on mechan i sm. Tha t ' s a l l i t i s. I t hasn ' t added any rea l va l ue except tha t you ge t more memory i n a box of the same s i ze for the same pr i ce. And tha t ' s n i ce. But the rea l va l ue of i t added by wha t you put i nto the memory. And tha t ' s why memor i es are cheap : there ' s no conceptua l content i n them. But now you take some th i ng l i ke a 32- b i t mi croprocessor - - and I was a l i t t l e face t i ous : the 286 i s rea l l y qu i te a b i t more soph i st i ca ted than the so i n fac t they ' ve added more va l ue there than you wou l d i f i t had been a memory, because i t does a l ot of th i ngs be t ter ; on the other hand, i f you take tha t another l eve l and ask i f there ' s any new conceptua l content i n i t, we l l no, i t ' s do i ng wha t mi n i computers a l ways d i d. So i t ' s new to s i l i con i n some way maybe but i t adds noth i ng new to the theory of comput i ng. Inc i denta l l y, th i s i s why there ' s a l ways wha t peop l e ca l l the hardware / sof tware t radeof f. I t means where i s the va l ue? Is the va l ue i n the pa t terns on the s i l i con or i s the va l ue i n the l i t t l e b i t -pa t terns i n the memory a f ter you ' ve made the s i l i con? And you can a l ways t rade tha t of f. The i nforma t i on ' s a l ways go i ng to be on the s i l i con, but i t may be i n l i t t l e e l ec t r i ca l charges on the memory l oca t i ons ; i n wh i ch case i t was the va l ue added by the guy tha t wrote the sof tware. Or i t may be tha t someone ac tua l l y crea ted a st ruc ture tha t embod i ed tha t a l gor i thm, i n wh i ch case he put i t d i rec t l y on the s i l i con - - i n the w i r i ng, i n the content of those p i eces of s i l i con. So for me, VLSI beg i ns when you can star t th i nk i ng about a l gor i thmi c comp l ex i ty. Tha t ' s when i t ge ts i nterest i ng. And I ' ve a l ways thought of i t tha t way, ever s i nce i t was j ust LSI. Because the potent i a l was there to bu i l d rea l l y comp l ex th i ngs. The s i l i con a l gor i thm i s a photograph tha t does wha t i t represents. CARVER : Fur thermore they ' re beaut i fu l. As forms, they ' re beaut i fu l. I f you ' ve l i ved w i th them for awh i l e you can apprec i a te them as an ar t form. You can te l l a beaut i fu l des i gn f rom an ug l y des i gn tha t does the same th i ng very eas i l y. CARVER : There are two t rans i stors per memory l oca t i on i n a DRAM and s i x i n a SRAM. And three or four t rans i stors per ga te, i f you ' re bu i l d i ng ga tes. But the par t they don ' t te l l you - - these peop l e tha t l i ke to count ga tes - - i s tha t i f you take a good we l l -des i gned s i l i con ch i p tha t ' s done by exper ts and you make a ga te d i agram for wha t i t does - - see, the th i ng i tse l f can ' t be represented by ga tes, because they don ' t represent a l ot of the func t i ona l i ty you can ge t w i th t rans i stors. Bi t i f you make a ga te d i agram of the th i ng i t turns out tha t we ge t about one equ i va l ent ga te for every one and a ha l f t rans i stors. Because you use t rans i stors not for mak i ng ga tes but for mak i ng th i ngs much more c l ever than ga tes. L i ke you use a s i ng l e-pass t rans i stor as a memory l oca t i on. I f you

13 MeadInterv i ew2 7 F i l e : 2 Carver.doc had to bu i l d tha t w i th ga tes you 'd have to make two cross- coup l ed AND ga tes and some other stuf f and i t 'd take a t l east four ga tes to make wha t one s i ng l e-pass t rans i stor does. So ga tes aren ' t an appropr i a te descr i pt i on for wha t happens i n VLSI. and there ' s th i s enormous argument about we l l, tha t ' s 30,000 t rans i stors tha t ' s on l y 8000 ga tes. No 30,000 t rans i stors i s probab l y l i ke 20,000 ga tes - - i f you d i d i t w i th ga tes, l i ke i n a ga te-ar ray. Tha t par t doesn ' t ge t to l d. But i t means you ' re us i ng the s i l i con i n a much more e f fec t i ve way than mak i ng ga tes out of i t. So to summar i ze, i f i t ' s done we l l the number of t rans i stors per ga te-equ i va l ent i s be tween one and two. Tha t ' s because you don ' t use them to make ga tes ; i f you used them to make ga tes i t 'd be three or four. So you ' re a l ready us i ng them tw i ce as e f fec t i ve l y as you wou l d i f you made ga tes, j ust by be i ng the s l i ghtest b i t i nte l l i gent. Wha t i s meant by "dev i ces per ch i p" and how much of the ch i p area i s devoted to "ga tes" and how much to w i res? CARVER : Dev i ces usua l l y re fers to t rans i stors, and tha t ' s mi s l ead i ng because w i res take up about 95 percent of the ch i p area. Le t me say i t another way : i f you put the w i res down tha t have to be there you can usua l l y s l i p the t rans i stors undernea th and not not i ce. I t ' s down on the bot tom l eve l s and the w i res go over the top, and usua l l y the w i res you can work i n w i th p i eces of the t rans i stor and i t turns out tha t i f they were shrunk to zero area for the ac tua l t rans i stors themse l ves you wou l dn ' t save more than f i ve or ten percent of ch i p area. So when you say 100 mi l l i on t rans i stors per ch i p you ' re on l y ta l k i ng about ten percent of the ch i p area? CARVER : Yeah, but you see, i f i t were j ust the t rans i stors there they wou l dn ' t do anyth i ng. Wha t they do i s de termi ned by how they ' re i nterconnec ted. So ac tua l l y i t ' s not a bad measure, because when peop l e have a work i ng ch i p they te l l you how many t rans i stors they were ab l e to i nterconnec t const ruc t i ve l y. And tha t ' s an i mpor tant number. Tha t number i s one d i mens i on of th i s comp l ex i ty i ssue : i t doesn ' t te l l you i f they ' re ext reme l y regu l ar or i f they have more i nforma t i on than tha t i n them. And j ust because a th i ng appears regu l ar doesn ' t mean there i sn ' t i nforma t i on i n i t tha t makes th i ngs not equ i va l ent to other th i ngs. L i ke a ROM l ooks ext reme l y regu l ar but i t may have a l ot of i nforma t i on i n i t because of the pa t terns i n ROMs. L i ke Geof f rey Fox ' s computer : i f you were to put a d i f ferent program i n every mach i ne, i t wou l d be a much more comp l ex th i ng than i f you have one program i n a l l the mach i nes. Tha t ' s wha t I meant by a l gor i thmi c comp l ex i ty. Of course, f i gur i ng out how to use i t wou l d a l so be tha t much more comp l ex. Is there a ru l e of thumb by wh i ch you can re l a te the mi n i mum fea ture s i ze or des i gn ru l e to the tota l area of a t rans i stor i n terms of square mi crons? CARVER : A typ i ca l t rans i stor i n today ' s wor l d i s of the order of one des i gn ru l e by one des i gn ru l e. Three by three i s the most common t rans i stor. I read some th i ng tha t sa i d 480 square mi crons. CARVER : Tha t ' s the amount of ch i p per dev i ce. I f you take the tota l number of dev i ces and d i v i de by ch i p area tha t ' s wha t you ge t. So then i mmed i a te l y you can take those two numbers and f i gure out wha t the f rac t i on of ut i l i za t i on i s. Wha t i s the computer revo l ut i on? CARVER : There are a number of l eve l s. The most obv i ous l eve l i s tha t s i nce anybody can own an ord i nary Von Neumann-sty l e computer, peop l e are now d i scover i ng tha t they can do a l ot of th i ngs, so everybody ' s got a PC a t home. They wr i te programs to do a l l k i nds of th i ngs. Tha t ' s become a th i ng you can ta l k about wh i ch you cou l d never ta l k about be fore. I t ' s l i ke the car. Enough peop l e own cars tha t you can ta l k about them as a me taphor for exp l a i n i ng other th i ngs. To me tha t ' s been the va l ue of the mi crocomputer phenomenon : not so much wha t they do but the

14 MeadInterv i ew2 8 F i l e : 2 Carver.doc fac t tha t they ' re obv i ous enough and use fu l enough and ub i qu i tous enough tha t everybody knows a t l east some th i ng about them. They ' re not grea t b i g th i ngs tha t gobb l e you up anymore. So tha t ' s been a b i g he l p. But much more i mpor tant than tha t i n terms of mi crocomputers are the mi crocomputers tha t ge t bu i l t i nto th i ngs - - te l ephones and typewr i ters and VCRs and a l l the ord i nary everyday l i fe app l i ances and too l s. E l ec t r i c dr i l l s. A l ot of peop l e don ' t understand tha t the reason e l ec t r i c dr i l l s don ' t ge t stuck as much as they used to i s tha t they have a mi croprocessor i n them tha t l ooks a t how fast the chuck ' s go i ng around and where your f i nger i s on the t r i gger and f i gures out wha t they ought to do. Those are rea l l y much more power fu l i n the revo l ut i on they crea te because they so l ve rea l prob l ems i nstead of be i ng some th i ng tha t you have to program to so l ve a rea l prob l em. They ac tua l l y j ust do i t. Somebody worked the who l e th i ng out i nstead of j ust say i ng we l l here, now i t ' s your turn. Al l of th i s i s i n the context of wha t peop l e th i nk of as computers. The fac t tha t there has occur red a v i ab l e th i rd-par ty sof tware i ndust ry where i t ' s now cons i dered OK to wr i te sof tware, you can ac tua l l y make money do i ng tha t. Tha t ' s not some th i ng tha t was perce i ved a few years ago. Tha t ' s where most of the ac tua l va l ue i s added i n terms of a l gor i thmi c comp l ex i ty. The va l ue i s added i n the programs, not the hardware i tse l f. The hardware i s t r i v i a l. Most computers don ' t have much i n them. They ' re rea l l y j ust a receptac l e for sof tware. So of course the rea l l y i mpor tant th i ng tha t ' s comi ng up - - and we haven ' t rea l l y begun to see i t - - i s the revo l ut i on of a l gor i thms i n s i l i con, where a l l the th i ngs you cou l d never do w i th computers - - wh i ch were the th i ngs you rea l l y wanted to do - - l i ke make p i c tures and mus i c and a l l of the th i ngs tha t re l a te to peop l e and to wha t peop l e rea l l y want to do - - ord i nary computers can ' t touch tha t. They ' re of f by a t l east f i ve orders of magn i tude i n computat i ona l power. Tha t to me i s the rea l l y exc i t i ng th i ng and i t ' s the th i ng tha t hasn ' t been touched ye t by wha t peop l e are ta l k i ng about as the computer revo l ut i on. But I th i nk i f you d i g undernea th a l l tha t, peop l e ' s not i on of va l ue i s chang i ng. I t used to be tha t peop l e thought of ma ter i a l as va l ue, so they 'd th i nk about stee l or a l umi num or go l d or d i amonds. I t was th i ngs tha t were va l uab l e. Af ter a wh i l e i t became c l ear tha t i t was a l so energy tha t was va l uab l e : energy a l l owed you to t ransform th i ngs i nto other th i ngs. Up unt i l very recent l y tha t ' s been peop l e ' s i mage of va l ue : th i ngs he l d va l ue. And the fac t tha t i t ' s ac tua l l y i nforma t i on tha t ' s the th i ng tha t ' s va l uab l e ra ther than the substance, i s some th i ng tha t ' s a new i dea. Tha t ' s why nobody wou l d g i ve you a d i me for sof tware. Because wha t the he l l i t ' s j ust a tape you can copy so why i s there any va l ue i n i t. You cou l d have sa i d the same for mot i on p i c tures except tha t was i n a d i f ferent doma i n so i t had d i f ferent l aws and i t grew up understand i ng tha t the va l ue wasn ' t i n the ce l l u l o i d. But everyp l ace e l se, espec i a l l y i n techno l ogy, peop l e had th i s very we i rd v i ew of where the va l ue rea l l y was. To me the most fundamenta l revo l ut i on tha t ' s go i ng on i s peop l e are now beg i nn i ng to perce i ve tha t not j ust i n enter ta i nment i s the i nforma t i on the i mpor tant th i ng. And tha t ' s a l ways been t rue i n enter ta i nment, i n mus i c and mot i on p i c tures and pa i nt i ngs. I t wasn ' t those l i t t l e tubes of pa i nt and a p i ece of canvas, i t was the i nforma t i on tha t was there. So i t ' s a l ways been t rue i n the ar ts but i t has taken a l ong t i me to dawn i n other areas. And now i t ' s happen i ng i n mach i nes and bus i nesses. Peop l e are beg i nn i ng to see tha t the i nforma t i on i s the va l uab l e th i ng. I t br i ngs these c l oser to the f i ne ar ts and the human pursu i ts, not far ther away. Tha t ' s not genera l l y perce i ved but i t ' s very t rue. Is i t fa i r to say tha t h i gh l y concur rent s i l i con a l gor i thms put many orders of magn i tude more comput i ng power i n the hands of average peop l e? Is tha t par t of the revo l ut i on? CARVER : Oh yeah. Th i ngs tha t the ord i nary computer cou l d never do. Vi s i on. Speech understand i ng. Mus i c. Vi sua l s i mu l a t i on. End of Carver

15 Carver 3 1 Fi le : 3 Carver.doe 10/00 Carver Mead Interv i ew Par t I I I (W i th D i ck Lyon, Sch l umberger AI researcher ) By : Gene Youngb l ood Document 3 of 4 of Mead Interv i ew CARVER : When I say tha t one of our mus i c ch i ps i s equa l to 600 VAXes, we l l, " equa l " i s a funny word. Wha t I mean i s tha t i f you do the same ca l cu l a t i ons on a VAX i t takes about ten mi nutes to do one second of sound. So one of those ch i ps does as much computat i on of tha t par t i cu l ar sor t as 600 VAXes cou l d do. Now VAXes are very poor l y se t up for do i ng computat i ons of tha t sor t, but so i s every other genera l purpose computer. Now there are spec i a l -purpose eng i nes tha t can do comparab l e th i ngs ; so mi ne i sn ' t the on l y way to do i t. Wha t d i d you mean by "by the t i me you ge t done shuf f l i ng a l l the da ta around, you w i nd up w i th some th i ng tha t ' s a few hundred t i mes rea l t i me i nstead of a few tens t i mes rea l t i me?" Wha t do you mean by rea l t i me? CARVER : You ought to be ab l e to genera te the sound as fast as you need the samp l es. To compute the samp l es as fast as they ' re needed for you to hear the sound. You cant do tha t on a regu l ar computer. I f you l ook a t how many mu l t i p l i es and adds i t takes per second to do these k i nds of computat i ons, i t ' s on l y f i ve or ten mi l l i on per second. Tha t sounds l i ke f i ve or ten t i mes as many as a VAX can do. But i t turns out tha t ' s not a l l the VAX i s do i ng. I t ends up shuf f l i ng a l l the da ta around to be i n the r i ght p l ace - - you fetch i t and you ge t i t i n the mach i ne and you mu l t i p l y i ng i t and then you ge t i t back out aga i n. And a l l those how many mu l t i p l i es you can do per second don ' t count i f you ' re on l y ge t t i ng the da ta to mu l t i p l y, you see. We were very surpr i sed. Peop l e had to l d me th i s for years, and I a l ways thought we l l yeah, i t ' s t rue for genera l programs, but I never thought tha t for a program l i ke th i s i t wou l d be t rue tha t da ta shuf f l i ng took most of the t i me. But i t turns out tha t there ' s enough p i eces of da ta tha t you have to gr i nd on and put i nto other p l aces tha t tha t ends up tak i ng more t i me than a l l the mu l t i p l i es. Every t i me you want a p i ece of da ta you ' ve got to compute i t ' s address, then do a par t i a l ca l cu l a t i on, then f i gure out where to put the answer, and then put i t back, and so on. Al l those th i ngs ea t you a l i ve. So on l y about one- tenth of the i nst ruc t i ons ac tua l l y end up mu l t i p l y i ng anyth i ng. So you cou l d do the ac tua l mu l t i p l i es i n zero t i me and i t doesn ' t speed th i ngs up very much. Whereas w i th spec i a l -purpose arch i tec ture you ar range i t so the da ta automa t i ca l l y f l ows to the r i ght p l ace a t the r i ght t i me. In fac t i n our ch i p and others tha t was the key to the who l e des i gn - - to make sure tha t tha t ' s wha t happened. Is there a ru l e of thumb for the amount of speedup you ge t f rom a ded i ca ted arch i tec ture. Al l e l se be i ng equa l, i f you ' re go i ng to put the same a l gor i thm i n s i l i con as opposed to runn i ng i t on a VAX. Wha t k i nd of speedup i n pr i nc i p l e, j ust by do i ng tha t? CARVER : For a l ot of the app l i ca t i ons we ' ve l ooked a t i t ' s more than two orders of magn i tude. One way to l ook a t i t i s tha t you ge t one fac tor of ten f rom wha t was j ust be i ng d i scussed - - f rom the fac t tha t i t ' s spec i a l i zed - - and then as many more fac tors of ten as you l i ke f rom the fac t tha t you ' re put t i ng more of the spec i a l i zed hard- ware i n tha t - - you ge t a fac tor of ten or 100 because you used 10 or 100 mu l t i p l i ers and another fac tor of ten because i t ' s not genera l purpose. So tha t ' s a hundred or a thousand r i ght there. So tha t ' s why we ' re a t a fac tor of the order of because you ' ve got th i ngs work i ng i n para l l e l and then you ' re not do i ng any da ta-shuf f l i ng. So tha t ' s another factor of ten. DICK: My speech recogn i t i on mach i ne ' s ge t t i ng the same fac tor w i th respec t to our genera l -purpose computer ; I ge t somewhere be tween a fac tor of 1000 and 5000 speedup. I 'm do i ng speech recogn i t i on by i mp l ement i ng a mode l of how the ear processes sounds. Sounds of a l l sor ts i nc l ud i ng mus i c. CARVER : So my mach i ne does th i s par t i cu l ar c l ass of computat i ons about 600 t i mes faster than a VAX. But i t won ' t run a PASCAL program and i t doesn ' t run UNIX and i t doesn ' t ma i nta i n a f i l e system. The way I th i nk of genera l purpose computers i s they ' re the l i t t l e env i ronment tha t does a l l those stup i d th i ngs - - hooks up to Etherne ts, hooks up to te l ephone l i nes, hooks up to keyboards, hooks up to humans, comp i l es l anguages, accesses f i l e systems - - does a l l those th i ngs AND i ssues commands to the spec i a l -purpose eng i nes tha t do the rea l l y hard

16 Carver 3 2 F i l e : 3 Carver.doc grunt -work a thousand or more t i mes faster than i t cou l d. CARVER : S i x hundred t i mes a VAX wou l d be about ten t i mes as fast as a Cray. I haven ' t programmed th i s on a Cray so I wou l dn ' t say tha t ; i n fac t a Cray i s se t up to do th i s k i nd of ca l cu l a t i on reasonab l y we l l a l so, so tha t for th i s par t i cu l ar task the Cray mi ght be more than 60 t i mes a VAX. I want to be very care fu l not to mi s l ead peop l e about th i s. One way of th i nk i ng about i t i s tha t you ' re not j ust tak i ng the VAX and mak i ng a spec i a l purpose th i ng ; you ' re tak i ng the who l e program tha t does a spec i a l th i ng on the VAX and you ' re say i ng tha t I 'm go i ng to take tha t very spec i a l app l i ca t i on and i nstead of a genera l purpose programmi ng l anguage on a genera l purpose computer, I 'm go i ng to put the who l e th i ng - - app l i ca t i on and a l l - - r i ght down i nto s i l i con. Who i nvented the s i l i con comp i l er? CARVER : I t evo l ved gradua l l y over a per i od of about e i ght years. But the f i rst one tha t was recogn i zab l e as a comp i l er i nstead of some th i ng l ess genera l was done by Dave Johannsen i n I d i d a very spec i a l purpose one i n 1971 for f i n i te-state mach i nes, but i t wasn ' t a s i l i con comp i l er i n the sense tha t we ta l k about them today. The appearance of s i l i con comp i l ers i s a s i gn i f i cant h i stor i ca l event ; does wha t makes them poss i b l e come out of your st ruc tured approach to VLSI des i gn? CARVER : I t had been poss i b l e for a l ong t i me. I t i sn ' t a ma t ter of mak i ng i t poss i b l e ; i t ' s tha t peop l e had been th i nk i ng of hand-cra f t i ng everyth i ng because i n the ear l y days you never had enough s i l i con to waste any a t a l l. And any st ruc tured approach to des i gn i s i n some sense s l i ght l y i ne f f i c i ent a t the bot tom l eve l. You can a l ways f i nd l i t t l e p l aces tha t aren ' t a l l f i l l ed up w i th t rans i stors. Exac t l y the same h i story happened i n code. Back when mach i nes on l y had a 4K memory, peop l e wrote every mach i ne i nst ruc t i on because you cou l dn ' t a f ford any waste a t a l l. But when you have a megabyte of memory you can l e t a comp i l er a l l oca te var i ab l es because i f you waste ten percent i t doesn ' t ma t ter any more. The same th i ng i s t rue of s i l i con comp i l a t i on : peop l e d i d not foresee the fac t tha t we wou l d have so much rea l estate ava i l ab l e ; tha t the rea l prob l em was the des i gn cost, not the cost of the rea l estate. When I t r i ed to te l l peop l e tha t i n the ear l y days they l aughed a t me. Tha t was w i th i n f i ve years of the t i me they were stand i ng up and say i ng i t ' s the most i mpor tant prob l em for the semi conduc tor i ndust ry. They were l augh i ng when you wou l d say tha t the des i gn cost was go i ng to be l arger than the produc t i on cost for comp l i ca ted par ts. Now tha t i t ' s t rue i t ' s them tha t i nvented i t. But a t the t i me I remember t ry i ng to ge t the Inte l peop l e i nterested - - th i s was i n and Gordon Moore sa i d i t was an academi c prob l em, and by the l a te 1970s he was pub l i sh i ng h i s curve of the des i gn costs go i ng up to the moon. The comp i l er v i ewpo i nt comes f rom say i ng we ' re go i ng to have enough rea l estate and we ' re not go i ng to have enough eng i neer i ng man-hours, so wha t you ought to be work i ng on i s the des i gn prob l em, not how dense i s the s i l i con. DICK: But i t ' s st i l l fa i r to say tha t th i s approach of do i ng th i ngs i n a more st ruc tured way i s wha t made s i l i con comp i l ers poss i b l e, by se t t i ng up tha t way of th i nk i ng. To go f rom hand-cra f t i ng to st ruc tura l bu i l d i ng b l ocks se ts the stage for the next th i ng, wh i ch i s comp i l a t i on. So i t ' s fa i r to say tha t the Mead-Conway approach made s i l i con comp i l ers poss i b l e. CARVER : And Dave Johannsen was the one who f i rst rea l l y saw how to pu l l the who l e th i ng toge ther i n a much more genera l way than peop l e had done be fore. So i t ' s a k i nd of synergy be tween VLSI, wh i ch makes poss i b l e a more s i mp l i f i ed st ruc tured approach, p l us tha t mak i ng poss i b l e the s i l i con comp i l er - - a l l of tha t toge ther resu l ts i n the ab i l i ty to put a l gor i thms i n s i l i con w i th a fast turnaround t i me. You can ' t separa te the comp i l er f rom the st ruc tured approach and you can ' t separa te the st ruc tured approach f rom VLSI. CARVER : There are a l ot of th i ngs tha t can ' t be separa ted. Hav i ng peop l e i n bus i ness who are w i l l i ng to fab ch i ps tha t you des i gn. In fac t we had to star t a who l e new se t of compan i es to do tha t because the t rad i t i ona l semi conduc tor peop l e d i dn ' t want to do i t. They st i l l don ' t want to do i t. They ' l l do i t for the i r l arge customers who want to make a mi l l i on par ts. So th i s th i ng cou l dn ' t happen unt i l we had access to fabr i ca t i on. And there was a ma j or amount of energy went i nto ge t t i ng peop l e to understand tha t there was a good bus i ness i n tha t ; and r i ght now tha t ' s the on l y par t of the semi conduc tor bus i ness tha t ' s mak i ng money. I t ' s happen i ng i n a b i g way now. DARPA prov i des fab serv i ces a t a very l ow cost to research pro j ec ts i n a few hundred un i vers i t i es. But there are

17 Carver 3 3 Fi le : 3 Carver.doe a l so a number of commerc i a l p l aces tha t w i l l do tha t k i nd of th i ng now for l ow-vo l ume eng i neer i ng prototypes. How do you ta l k about wha t i s ac tua l l y on a ded i ca ted ch i p? Wha t makes the adders and mu l t i p l i ers i n your ch i p ded i ca ted? CARVER : We l l, on one of our ch i ps we ' re apt to have f i f ty mu l t i p l i ers a l l work i ng a t the same t i me ; on a mi croprocessor you have to have one - - or zero, more l i ke l y, because they use adders i nstead and make up the mu l t i p l y out of add i t i ons. Al most a l l mi croprocessors i nc l ud i ng the MC68000 have zero mu l t i p l i ers. Both i n D i ck ' s speech ch i p and the ch i p we use for mus i c, the key i s there ' s a way to se t i t up so tha t where the da ta comes f rom and where i t goes to i s some th i ng tha t can be se t up and j ust run and not take a bunch of i nst ruc t i ons to f i gure out everyth i ng. Because the i nst ruc t i ons are the ch i p? CARVER : The ch i p has a spec i a l i zed way of se t t i ng up where the da ta shou l d come f rom and where i t shou l d go, so tha t the fac i l i t i es for mov i ng da ta e f f i c i ent l y f rom every p l ace i t mi ght want to come f rom to every p l ace i t mi ght want to go to are a b i g par t of the ch i p. And i t ' s not hardw i red to a par t i cu l ar a l gor i thm ; i t ' s got enough f l ex i b i l i ty to be ab l e to run a reasonab l e number of a l gor i thms w i th i n a c l ass. L i ke d i f ferent mus i c i nst rument mode l s can be put on our ch i p. D i f ferent k i nds of d i g i ta l f i l ter i ng st ruc tures can be put on D i ck ' s speech ch i p. So i t ' s not accura te to say tha t th i s ch i p i s a ce l l o? CARVER : I t can be conf i gured to be any k i nd of i nst rument, but i t ' s ded i ca ted i n the sense tha t i t per forms tha t l i mi ted c l ass of ca l cu l a t i ons wh i ch are re l evant to those phys i cs formu l as. The b i g cha l l enge i n th i s spec i a l i zed hardware game i s to make a p i ece of hardware tha t ' s not so spec i a l i zed tha t i t ' s on l y good for one th i ng. You want to l eave a cer ta i n amount of programmab i l i ty of some sor t. Th i s resu l ts i n a good chunk of your s i l i con area - - maybe as much as ha l f - - be i ng ded i ca ted to the j ob of ge t t i ng da ta f rom where i t ' s comi ng f rom to where i t ' s go i ng. In a genera l purpose computer tha t takes n i ne ty percent of your resources ; i n our ch i ps i t ' s probab l y l ess than 20 percent, i n D i ck ' s speech ch i p i t ' s f i f ty percent. You ge t d i f ferent amounts of genera l i ty for the d i f ferent amounts of overhead. How many t rans i stors i s on your ch i p? CARVER : I t has 64 mu l t i p l i ers, each g i v i ng a 32 by 32 mu l t i p l y w i th a 64-b i t produc t. There are 32 stages of the mu l t i p l y. I t has about 200,000 t rans i stors. Tha t ' s a l ot l ess than the Hew l e t t -Packard ch i p tha t has 450,000. But for th i s par t i cu l ar app l i ca t i on our ch i p i s about 1000 t i mes faster. The t i me-honored way of ge t t i ng a h i gh t rans i stor count i s to put a l ot of memory on the ch i p. Our ch i p has a l ot of memory wh i ch i s mi xed i n w i th the processors, and the very th i ng tha t makes i t e f f i c i ent i s tha t you mi x the process i ng i n w i th the memory so you don ' t have to move th i ngs f rom the memory to a process i ng e l ement and back. You mi x them toge ther so process i ng i s go i ng on a l l the t i me. Le t me say wh i l e we ' re a t i t tha t the 600 VAXes est i ma te i s ext reme l y conserva t i ve because the c l ock for our ch i p i s s l owed down so tha t i t ma tches the samp l e ra te wh i ch i s the standard tha t a l l D to A conver ters use. There 'd be noth i ng to prevent us f rom runn i ng the ch i p f i ve t i mes faster i f you wanted to do graph i cs i nstead of sound. But i n a sense tha t ' s not fa i r because we ' re comput i ng these samp l es ; and the fac t tha t you cou l d compute them faster than rea l t i me doesn ' t he l p. On the other hand, because a VAX computes them s l ower than rea l t i me we ought to be ab l e to a t l east count tha t somehow. But essent i a l l y once you ' re a t rea l t i me a t a g i ven samp l i ng reso l ut i on you don ' t need to go any faster. Your ear cou l dn ' t hear i t anyway. Now i f we were work i ng on ba ts i nstead of humans, we l l ba ts can hear a t 200 KHz i nstead of they ' re 10 t i mes faster than peop l e - - so then we 'd have to run the c l ock 10 t i mes as fast. We cou l d do tha t, too, i n wh i ch case i t 'd be the equ i va l ent of 6000 VAXes. Now the other th i ng i s tha t you may have the need to genera te 10 mus i ca l i nst ruments a t once, and w i th our ch i p i f you run i t 10 t i mes faster i t st i l l cant qu i te do tha t - - i t can genera te one very fast i nst rument but not ten a t once. D i ck ' s speech ch i p has an ext ra degree of f l ex i b i l i ty tha t l e ts you be ten th i ngs a t once i nstead of one th i ng very fast. In fac t h i s ch i p i s not very good a t do i ng one th i ng very fast, so he cou l dn ' t make very good mus i c for ba ts but he cou l d maybe ana l yze sounds f rom severa l mi crophones a t once. Tha t costs h i m a l ot ; he has ha l f h i s ch i p ded i ca ted to ge t t i ng da ta f rom where i t ' s comi ng f rom to where i t ' s go i ng and by the way stor i ng i t temporar i l y i n be tween. Tha t ' s the ext ra cost of tha t ext ra genera l i ty. And tha t ' s the t radeof f you make depend i ng on wha t you want to do. The d i scuss i on among peop l e who do these spec i a l i zed arch i tec tures i s another l eve l up f rom a l l th i s,

18 Carver 3 4 F i l e : 3 Carver.doc name l y, now tha t you ' ve got th i ngs tha t run 1000 t i mes as fast as a genera l purpose computer wou l d on the same prob l em, can you make i t 10,000 t i mes or 100,000 t i mes faster - - can you ge t an even be t ter ma tch to the prob l em wh i ch uses the s i l i con i n even a be t ter way? I t ' s j ust amaz i ng when peop l e star t p l ay i ng th i s game. I t ' s not j ust more concur rency, i t ' s a l gor i thms tha t are more thought - through, t i ghter. In fac t both D i ck and I have t raded of f for genera l i ty the u l t i ma te i n per formance. There wou l d have been ways for each of us i f we 'd wanted to be rea l l y spec i a l i zed to make the th i ngs another order of magn i tude faster. We chose to have some genera l i ty i nstead. For examp l e we both use a l ot more b i ts of accuracy than are needed for most of the prob l ems we ' re a t tack i ng, because occas i ona l l y you sudden l y need more b i ts, so you spend the b i ts a l ways and pay a b i g cost, but th i s means you don ' t of ten run i nto a prob l em where you don ' t have enough b i ts. Tha t ' s one of the d i mens i ons of var i ab i l i ty, how many b i ts you choose to put i n. We use 64-b i t accumu l a tors. Most peop l e who do mus i c synthes i s wou l d say tha t ' s out rageous, we ' re throw i ng away resources. Today to ge t an upda ted vers i on of a sof tware package I buy a f l oppy d i sk. How wou l d I upda te my computer i f everyth i ng ' s i n hardware? Buy a new board? CARVER : There ' s a l i t t l e company i n the bay area ca l l ed S i l i con So l ut i ons, Inc. You shou l d ta l k to them because they make an a l gor i thm i n s i l i con. They founded th i s company on the premi se tha t a l gor i thms i n s i l i con are go i ng to be the future. The i r f i rst produc t checks des i gn ru l es on i ntegra ted c i rcu i ts. So i t ' s an i ntegra ted c i rcu i t tha t checks for er rors i n the next i ntegra ted c i rcu i t. The next th i ng they ' re bu i l d i ng i s a ch i p s i mu l a tor. Wha t they d i d i s make an add-on for a standard workstat i on. So you can p l ug the card i nto your workstat i on ' s mu l t i bus s l ot. And there ' s a l i t t l e sof tware package tha t goes w i th your standard UNIX opera t i ng system - - workstat i ons are a l l UNIX based - - and now you can be up and runn i ng w i th the i r des i gn- ru l e-checker i n your workstat i on. So i nstead of a d i sk tha t you p l ug i nto your d i sk s l ot there ' s a board you p l ug i nto the board s l ot. There ' s a ch i p tha t i mp l emented the Etherne t protoco l. I t got bu i l t on a l i t t l e board tha t p l ugs i nto the PC. So tha t ' s an examp l e of an a l ready ex i st i ng approach. When you buy sof tware on a d i sk you pay f i ve do l l ars for the f l oppy and three-hundred do l l ars for the i deas i n i t. A board costs about the same, maybe ten or twenty do l l ars for a cheap board i nstead of f i ve do l l ars for a f l oppy d i sk. Or even i f i t was $100 for the board you ' re ge t t i ng i nstead of some th i ng new but j ust as s l ow as ever, i t does some th i ng new but 1000 t i mes faster. So tha t fac tor of 1000 costs you a hundred bucks over wha t i t wou l d be i f i t was a sof tware package. And you ge t a fac tor of Tha t ' s a pre t ty good pr i ce. Tha t ' s a fac tor of ten per do l l ar. End of Carver 3

19 Carver 3 5 Fi le : 3 Carver.doc

20 Intervi ew : Mead 4 1 Fi le : 4 Carver.doe 10/00 Carver Mead Interv i ew Par t IV (W i th D i ck Lyon, Sch l umberger AI researcher ) By : Gene Youngb l ood (Document 4 of 4 of Mead Interv i ew) DICK: You asked about wha t i t ' s l i ke to des i gn a l gor i thms when i t ' s not j ust Pasca l programs or some th i ng. I t ' s a d i f ferent game, but peop l e who des i gn a l gor i thms a l ways do i t w i th some targe t comput i ng techno l ogy i n mi nd, whe ther i t ' s a genera l purpose computer or fu l l custom s i l i con or maybe you ' re des i gn i ng the i nterconnec t pa t tern for Carver ' s ch i p to make a new i nst rument mode l. Wha tever your targe t techno l ogy i s, tha t de f i nes the game you ' re p l ay i ng when you des i gn a l gor i thms. The nea t th i ng about custom s i l i con i s tha t you ' ve got a l ot of new k i nds of games other than j ust t ry i ng new programs. So des i gn i ng mus i ca l i nst ruments i s a new k i nd of game. You come up w i th a l gor i thms tha t you cou l d have wr i t ten i n a genera l purpose programmi ng l anguage but you probab l y wou l dn ' t have, because i f you had a genera l purpose computer to p l ay w i th, you wou l d have done some th i ng d i f ferent - - more comp l i ca ted, probab l y ; more e f f i c i ent maybe. I f you ' re des i gn i ng f rom bare s i l i con and you have to make a new a l gor i thm, the game i s w i de open. Al l you know i s you ' re mak i ng i t out of submi croscop i c t rans i stors. I f you ' re des i gn i ng w i th i n the context of somebody ' s s i l i con comp i l er, you ' re a l i t t l e more rest r i c ted than tha t - - you know you can make i t out of prede f i ned modu l e types and reg i ster types and bus st ruc tures and so on. so tha t de f i nes the ru l es of the game. A programmi ng l anguage puts a l ot of preconcept i ons i n your head about wha t you shou l d do w i th a computer. CARVER : In fac t you do th i ngs very d i f ferent l y i n d i f ferent programmi ng l anguages. Not because you cou l dn ' t do the same th i ng i n two d i f ferent l anguages but because the way you th i nk i n one l anguage l eads to a way of do i ng i t wh i ch i s na tura l i n tha t l anguage. Even though you cou l d have thought i n another way i t ' s not as na tura l. So the l anguage of s i l i con has cer ta i n th i ngs tha t are ext reme l y na tura l to i t. The i dea of regu l ar i ty ma tches very we l l the i dea tha t commun i ca t i on i s very expens i ve. I f you th i nk of th i ngs l oca l l y and have them commun i ca te not too g l oba l l y w i th too many other th i ngs, a) : i t he l ps you to th i nk about i t and make regu l ar st ruc tures, and b) : i t a l so makes th i ngs tha t are very h i gh per formance. Tha t was the par t tha t the o l d s i l i con hackers mi ssed. They d i dn ' t understand tha t a regu l ar des i gn can produce be t ter per formance than some i r regu l ar way. Because you force your thought processes to encompass the overa l l way that the th i ng works. DICK: You st i l l ge t quest i ons f rom peop l e who don ' t know enough about i t who are steeped i n the o l d l ore but don ' t understand very much. They ask th i ngs l i ke " I f you des i gn a ch i p i n the Mead-Conway st ruc tured sty l e, how much per formance do you have to g i ve up : 20 percent, 30 percent?" I say tha t ' s a s i l l y quest i on. The quest i on i s how much per formance can you ga i n. They th i nk of the st ruc ture as be i ng an ar t i f i c i a l l i mi tat i on, wh i ch i t i s, but i t a l l ows you to do th i ngs you cou l dn ' t or wou l dn ' t do otherw i se. And you ge t to i nvent the st ruc ture, wh i ch means you make your prob l em eas i er not on l y a t the bot tom l eve l of hack i ng t rans i stors but a t the next l eve l up - - you can do crea t i ve des i gn and a l gor i thm p l ann i ng a t a h i gher l eve l and make i t easy to bu i l d the who l e th i ng and make i t e f f i c i ent. Instead of do i ng i t the o l d way where you draw many pages of schema t i cs and then have somebody t ry to force i t i nto the ch i p. Why can ' t we ge t both VLSI and h i gh speed a t the same t i me? CARVER : In 1956 when I was a student i t was thought tha t i f you make a b i gger t rans i stor i t a l ways got s l ower. Tha t was the common exper i ence of the day. So th i s one company sa i d i f I make a t i ny t rans i stor I can make i t faster, and i f I make a bunch of these l i t t l e t rans i stors and put them a l l i n para l l e l, why i sn ' t tha t faster? We l l i t i s faster. Because a l l the paras i t i cs tha t l oaded down the b i g t rans i stors aren ' t a proper ty of the i nd i v i dua l t rans i stor ; the rea l prob l em was tha t peop l e were sca l i ng th i ngs wrong to make the b i g t rans i stors. They were th i nk i ng about i t wrong. And these guys had f i gured i t out. They asked me to prove sc i ent i f i ca l l y tha t you don ' t have to ge t some th i ng tha t goes s l ower and takes more power i f you make i t b i gger. So I ana l yzed wha t was ac tua l l y go i ng on there and tha t ' s ac tua l l y wha t got me star ted th i nk i ng on th i s who l e t ra i n of sca l e - - the very f i rst consu l t i ng j ob I ever had. I t turns out tha t i t ' s the same k i nd of mi sconcept i on when peop l e assume some th i ng i s constant tha t i sn ' t when you sca l e.

21 Intervi ew : Mead 4 2 Fi le : 4 Carver.doe I f tha t ' s the case, then, can we put a number on the speed l i mi t of the Von Neumann computer? CARVER : We can do tha t very we l l. Le t ' s do a one-ch i p mach i ne. Th i ngs are sca l i ng i n such a way tha t the power /de l ay produc t i s ge t t i ng be t ter by some th i ng be tween the square and the cube of the sca l i ng fac tor. As you ge t c l ose to the l i mi ts you don ' t ge t the who l e cube l aw any more. So i f you take the mi n i mum-s i ze t rans i stors - - quar ter - mi cron - - you have ga te de l ay t i mes i n the 10 p i cosecond range. Peop l e have a l ready made c i rcu i ts w i th 30 p i coseconds and the l a test I heard was 18 p i coseconds i n submi cron CMOS. And i t ' s go i ng to ge t be t ter. So the speed l i mi t of one i so l a ted ga te not dr i v i ng anyth i ng w i l l be i n the 10 p i cosecond range ; see, the prob l em w i th the Von Neumann arch i tec ture i s they have these damn l ong w i res you have to dr i ve, wh i ch i s i nherent i n the l arge RAM conf i gura t i on - - so you need e i ther l ong w i res or many stages of de l ays. So you can say, we l l, wha t do I have to do to dr i ve tha t th i ng? We l l, the w i res i n terms of the techno l ogy undernea th, are ge t t i ng l onger because you st i l l want the w i re to go c l ear across the ch i p because you ' re bu i l d i ng a b i gger mach i ne ; on the other hand the t rans i stor i tse l f i s ge t t i ng be t ter ab l e to dr i ve, because the channe l l engths are shor ter and the submi cron t rans i stors are st ronger for a g i ven l ength. So when you add i t a l l up, the ac tua l speed ge ts be t ter about l i near l y w i th the sca l i ng. So f rom where we are now i n dev i ce s i ze - - say 1.25 mi cron - - tha t ' s a fac tor of f i ve f rom the.25-mi cron techno l ogy, and i t runs a t about 20 MHz. So you can l ook forward - - i f you ' re go i ng to bu i l d a mi crocomputer tha t f i l l s a ch i p a t a quar ter -mi cron and you have good process i ng and you ' ve done th i ngs sens i b l y, you can l ook forward to i n excess of 100 MHz c l ock ra tes for those th i ngs. Tha t ' s a l ready respec tab l e by the standards of peop l e l i ke Cray. And you can l ook forward to the same k i nd of comp l ex i t i es you have on those k i nds of mach i nes. R i ght now you have comp l ex i t i es l i ke you have on VAXes on one ch i p, there ' s no reason you can ' t have a comp l ex i ty on the order of a Cray on one ch i p by tha t t i me. And the power per un i t area wou l d be about the same as i t i s today. So the u l t i ma te speed of the Von Neumann mach i ne i s about ten t i mes wha t i t i s today, tha t i s, ten t i mes a Cray? CARVER : No. You 'd be j ust about ge t t i ng i n the range on one ch i p where Cray i s today by the t i me you do a l l tha t. The speed l i mi t on a s i ng l e ch i p i s about a Cray. Tha t ' s a good number to keep i n your head : you ' l l ge t about one Cray on a ch i p, i nc l ud i ng the c l ock ra te and everyth i ng, by the t i me you push ord i nary MOS dev i ces as far as they ' re go i ng to go. Tha t i s, i f you choose to make a Cray. Wh i ch i s an e l abora ted or extended vers i on of the Von Neumann arch i tec ture w i th vec tor process i ng. So we ' re ta l k i ng be tween 50 and 100 mega f l ops depend i ng on how we l l i ts done. And how much speedup w i l l we ge t f rom new ma ter i a l s l i ke ga l l i um arsen i de? A fac tor of ten more? CARVER : I doubt i t. F i rst, fundamenta l l y you can ' t make i t as sma l l as s i l i con. The reason ga l l i um arsen i de i s fast i s because the e l ec t ron mob i l i ty i s h i gh. The reason the mob i l i ty i s h i gh i s because the e f fec t i ve mass i s l ow. Th i s means the tunne l i ng d i stances are l onger. There ' s a d i rec t re l a t i onsh i p be tween e f fec t i ve mass and how far you can tunne l. S i nce tunne l i ng i s one of the th i ngs tha t rest r i c ts how sma l l you can make dev i ces, to the extent tha t tha t ' s the l i mi t i ng fac tor - - and i t does enter i n submi cron dev i ces - - you ' l l bump aga i nst i t sooner w i th ga l l i um arsen i de than you w i l l w i th s i l i con. Exac t l y how a l l tha t comes out i s not known r i ght now. Nobody has ye t done i t. Meanwh i l e peop l e are work i ng on some very fancy th i ngs ca l l ed super l a t t i ce st ruc tures wh i ch g i ve ext reme l y h i gh mob i l i t i es. But exac t l y how the dev i ce st ruc tures are go i ng to sca l e i s some th i ng we need to know. These are compound l ayers whose l a t t i ces ma tch and you can make the bandgap go up and down by a sor t of sta i rstep th i ng. Tha t makes a quantum e f fec t on the e l ec t rons. I t bas i ca l l y shapes the bandgap i n a n i ce way and a l ready McG i l l has worked out a theory for tha t and the measured sh i f t i n the photo-emi ss i on has turned out to agree exac t l y w i th h i s theory. Tha t wou l d be the HEMT dev i ce. Tha t may ge t you the fac tor of ten or even more i n terms of raw speed. But you can ' t take the numbers for i nd i v i dua l dev i ces and app l y them to l arge-sca l e c i rcu i ts because. For i nstance wha t ' s wrong w i th ga l l i um arsen i de techno l ogy r i ght now i s not the i nd i v i dua l dev i ces. They work be t ter than you wou l d have thought. They ' re be t ter than my f i rst pred i c t i ons, because there ' s a f l uke, there ' s a v i o l a t i on of Murphy ' s Law tha t happens. When an e l ec t ron goes i n under the ga te you 'd th i nk i t wou l d come up to sa tura ted ve l oc i ty - - wh i ch i s on l y a l i t t l e be t ter i n ga l l i um arsen i de than i n s i l i con - - but i t turns out they do a ba l l i st i c overshoot, so they ac tua l l y, for shor t channe l i ngs, go faster than the sa tura ted ve l oc i ty because they haven ' t star ted to have enough co l l i s i ons ye t. Tha t was a v i o l a t i on of Murphy ' s Law of the f i rst order. I t was wonder fu l. The rea l prob l em w i th tha t k i nd of techno l ogy i s tha t there ' s on l y one k i nd of dev i ce - - a dep l e t i on mode dev i ce. So you have to have the ga te vo l tage of oppos i te po l ar i ty f rom the dra i n vo l tage, wh i ch means you have to l eve l sh i f t be tween every stage. I t ' s hor r i b l e, j ust l i ke the o l d vacuum tube th i ngs. And i t turns out tha t ends up tak i ng more space than the c i rcu i t by a l ot. So i t

22 Intervi ew : Mead 4 3 Fi le : 4 Carver.doe i sn ' t a t a l l c l ear tha t the dens i t i es w i l l ever be compe t i t i ve w i th wha t you can do i n s i l i con. DICK : But for tha t same reason i t wou l d probab l y make poss i b l e very h i gh dens i ty, very fast memor i es. And you mi ght st i l l be ab l e to ge t a Cray-s i ze Von Neumann mach i ne on a ch i p because the memory i s where most of i t goes. In fac t as you everyth i ng up and the amount of g l oba l w i r i ng ge ts more and more domi nant, the amount of memory ge ts b i gger and b i gger, the power of the ar i thme t i c processor goes up a l i t t l e b i t and tha t ' s i t. So the b i ggest th i ng goes i nto more w i re, the next b i ggest th i ng goes i nto more memory, and the l east ga i n i s i n the processor. Th i s i s wha t peop l e have been do i ng. When you grow a Von Neumann mach i ne i n any techno l ogy tha t ' s wha t happens whe ther i t ' s on a ch i p or not. Cray i s a depar ture f rom tha t because they f i gured out how to put a l ot more i n the process i ng ; but i t ' s l ess Von Neumann- l i ke than other mach i nes. CARVER : Cray i s one of the peop l e who f rom day one rea l i zed tha t, we l l, he sa i d two th i ngs. He sa i d computer des i gn has two prob l ems. One i s how you ge t the hea t out. The second i s how you keep the th i ckness of the w i r i ng ma t w i th i n bounds. He thought the key to h i s des i gn was ge t t i ng the hea t out. We l l we can take care of mak i ng more comp l ex th i ngs w i th l ess hea t by j ust sca l i ng down the s i l i con. So tha t par t i sn ' t the key ; but the other par t i s, the w i r i ng management. And he was the on l y ma j or computer des i gner tha t I know of who spec i f i ca l l y twenty years ago i dent i f i ed the th i ckness of the w i r i ng ma t as the domi nant th i ng about computer des i gn. Everybody e l se was say i ng we l l you have to have the x-y- z i nst ruc t i on and there ' s th i s and tha t, and he was say i ng no, i t ' s the th i ckness of tha t w i r i ng ma t. W i l l there ever be a Von Neumann mach i ne w i th a one nanosecond c l ock. CARVER : There mi ght be. We a l ready got you to under ten nano-seconds w i th our l i t t l e ch i p a t a quar ter -mi cron. You cou l d do i t today w i th a sma l l enough Von Neumann mach i ne. A rea l s i mp l e mach i ne. You 'd come c l ose to tha t today us i ng submi cron CMOS. They ' ve a l ready demonst ra ted.8-mi cron CMOS w i th 35-p i cosecond ga tes. You cou l d no doubt make a cred i b l e one-nanosecond sma l l Von Neumann mach i ne. I f i t was on l y e i ght b i ts w i de and had on l y three reg i sters. Or i f you bu i l t i t l i ke Cray d i d and p i pe l i ne the he l l out of i t. And we cou l d cer ta i n l y make these b i t -ser i a l th i ngs to run tha t fast. Tha t ' s our mus i c ch i ps and D i ck ' s speech ch i p, where you do one b i t a t a t i me. Because the s i gna l s on l y have to go to the next stage i nstead of go i ng c l ear across the ch i p. Is RISC arch i tec ture re l evant on l y to Von Neumann arch i tec ture? CARVER : Yes, and wha t i t rea l l y means i s some th i ng they d i d ear l y on and forgot about l a ter. Peop l e used to bu i l d a l l mach i nes as reduced i nst ruc t i on se ts because they cou l dn ' t a f ford anyth i ng e l se. The best examp l e of tha t was the o l d Da ta Genera l Nova, wh i ch had a rea l s i mp l e i nst ruc t i on se t. Ever s i nce then they bu i l t i n ever more e l abora te i nst ruc t i on se ts, be l i ev i ng tha t i t was go i ng to make the i r computers run faster. And i t turns out tha t wha t l i mi ts your computer i s the l oad i nst ruc t i on and the store i nst ruc t i on and the i ndex ca l cu l a t i ons. I f you do those fast i t doesn ' t much ma t ter wha t e l se you do. And i t doesn ' t a t a l l ma t ter how fast you do ar i thme t i c. So peop l e f i na l l y went back and red i scovered the fac t tha t had been known for a l ong t i me by good computer des i gners : tha t i t doesn ' t rea l l y much ma t ter wha t you do w i th i nst ruc t i ons i f you ge t the bas i c ones rea l fast. So they sa i d why put i n a l l the others, why not j ust make them up out of these pr i mi t i ves? You have to have a comp i l er tha t ' s smar t enough to do a decent j ob of tha t. DICK: The other argument i s tha t the mach i nes w i th comp l i ca ted i nst ruc t i on se ts don ' t rea l l y ge t any advantage f rom them un l ess you have a comp i l er tha t ' s smar t enough to f i gure out when to use comp l i ca ted i nst ruc t i ons. W i th the RISC you may ge t be t ter per formance even w i th a very s i mp l e comp i l er. CARVER : But i n fac t nobody has ye t done the rea l exper i ment of ac tua l l y mak i ng a comp l e te system w i th a comp l e te opera t i ng system and everyth i ng on one of these RISC th i ngs and ac tua l l y honest - to-god put t i ng i t s i de by s i de w i th some other computer and g i ven i t the smoke test. Tha t ' l l no doubt happen but i t ' s i ncred i b l y l a te i n comi ng, because the f i gures peop l e quote are concoc ted examp l es ra ther than rea l honest - to-god th i ngs happen i ng. Sure l y there w i l l a l ways be a supercomputer, because there ' l l a l ways be the ten-mi l l i on-do l l ar mach i ne. CARVER : Oh but you know wha t i t ' s go i ng to be? I t ' s go i ng to be one of these ch i ps tha t has the Cray on i t and a

23 Intervi ew : Mead 4 4 Fi le : 4 Carver.doe bunch of memory and d i scs and a l l tha t, wh i ch w i l l ea t up a who l e l ot of money, and then i t ' s go i ng to be a box tha t has one k i nd of spec i a l purpose ch i p on i t wh i ch does the s i gna l process i ng th i ngs, and another card tha t has another k i nd of ch i p wh i ch does the geome t ry ca l cu l a t i ons, and another card w i th ch i ps tha t do wha tever e l se. When you ta l k about extend i ng the i nst ruc t i on se t... i t ' s l i ke r i ght now when you buy a mach i ne you ge t an ar i thme t i c acce l era tor and you p l ug i t i n and i t runs ten t i mes as fast do i ng f l oa t i ng-po i nt. So you ' l l have some th i ng tha t runs ten- thousand t i mes as fast do i ng s i gna l process i ng. And some th i ng tha t runs a mi l l i on t i mes as fast do i ng graph i cs render i ng. And some th i ng tha t runs a hundred- thousand t i mes as fast do i ng wha tever e l se you ' re do i ng. There ' l l be these cards wh i ch are these acce l era tors for spec i a l c l asses of prob l ems. And they ' l l p l ug i nto th i s th i ng. There w i l l a l ways be card cages and power supp l i es and fans (or cryogen i c coo l i ng systems) but the th i ng you p l ug i n there i s go i ng to have a god-awfu l amount of capab i l i ty compared w i th the l i t t l e poor w i zened Cray tha t ' s s i t t i ng on tha t one ch i p. DICK: The i nterest i ng prob l em i s to f i gure out how there w i l l be ten-mi l l i on-do l l ar mach i nes. I mean, wha t par t of tha t are go i ng to spend your money on. I be l i eve there w i l l st i l l be such mach i nes but they ' l l probab l y be domi na ted by head-per - t rack d i scs or some th i ng l i ke tha t. You ' l l spend n i ne out of the ten mi l l i on do l l ars ge t t i ng enough secondary storage and i /o bandw i dth tha t ' s fast enough to keep up w i th the th i ng. Any one of our l i t t l e ch i ps can genera te da ta a god-awfu l amount faster than any d i sc can soak i t up. We so l ve the prob l em by j ust put t i ng i t out on a l oudspeaker. We never store samp l es a t a l l, ever ; and i t may be tha t for some app l i ca t i ons you never wou l d need to store a l l tha t da ta. DICK: In the speech ana l ys i s prob l em, you come i n w i th some samp l ed speech, wh i ch may be a t a reasonab l y l ow ra te l i ke you ge t of f a te l ephone system, maybe be t ter qua l i ty. In the i ntermed i a te ca l cu l a t i ons you end up w i th p l aces i n your b l ock d i agram be tween the f i rst b l ock and the second b l ock where the da ta ra te may be 100 t i mes h i gher than wha t you star ted out w i th. There ' s no way you ' re go i ng to feed tha t back i nto your genera l purpose computer or put i t on d i sc or anyth i ng e l se. But i f you put spec i a l i zed stuf f out there to keep dea l i ng w i th i t unt i l you ' ve got ten c l ose to an answer, unt i l you ' ve nar rowed i t down to wha t word was sa i d, then you can throw away a l l tha t i ntermed i a te da ta. i t ' s a t remendous amount of da ta but you don ' t need to keep i t. You j ust l ook a t i t to ext rac t the next stuf f f rom i t and you throw i t away. Put i t i n the b i t -bucke t. Wha t you guys are do i ng i s amaz i ng. DICK: I t ' s happen i ng so fast tha t even new eng i neer i ng gradua tes aren ' t aware of the ha l f of i t. Any g i ven app l i ca t i on a l ways has a l i mi t i ng case. In i mag i ng, you don ' t need to compute more than wha t your d i sp l ay dev i ce can show. CARVER : There ' s no po i nt i n mak i ng a p i c ture faster than rea l t i me, any more than there i s i n me mak i ng samp l es faster than rea l t i me. Al vy Ray Smi th th i nks you need a comp l ex i ty of 80 mi l l i on po l ygons per f rame for photorea l na tura l phenomena. CARVER : Tha t ' s a J i m Ka j i ya measure : a comp l ex scene has more than one po l ygon per p i xe l. Ac tua l l y i t turns out there are some very i nterest i ng prob l ems assoc i a ted w i th scenes tha t are much more comp l ex than one po l ygon per p i xe l. Wh i ch i s the k i nd our guys dea l w i th. So l e t ' s say we reach some arb i t rary po i nt beyond wh i ch no more computat i on power i s needed, then you ' re say i ng tha t i t ' s probab l y go i ng to be poss i b l e to put tha t k i nd of power on a board and tha t ' s tha t. CARVER : You j ust use i t unt i l you ge t a be t ter i dea of how to do i t, be t ter a l gor i thms. The rea l i nterest i ng prob l em now i s the or i g i n of l i fe prob l em, r i ght? I t ' s the th i ng I ta l ked about w i th the "Newma th" company. They ' re work i ng the or i g i n of l i fe prob l em. They want to des i gn s i l i con a l gor i thms so they ' re bu i l d i ng th i ngs tha t ' l l he l p them do tha t - - and by the way they can se l l tha t too, because other peop l e w i l l want to make s i l i con a l gor i thms too - - peop l e who want to make be t ter graph i cs a l gor i thms, who ' s go i ng to he l p them? Because you ' ve got to f i gure out i f your i deas are r i ght be fore you go and make the be t ter graph i cs a l gor i thm. So there ' s a l ways go i ng to be room for a th i ng tha t ' s more genera l than the spec i f i c th i ng because somebody has to i nvent the next one. So you ' l l need a deve l opment system tha t ' s an eng i neer i ng work- stat i on for peop l e do i ng computer graph i cs, or speech or mus i c - - wh i ch i s not

24 Intervi ew : Mead 4 5 Fi le : 4 Carver.doe rea l l y a genera l purpose computer but i t more genera l than the f i na l s i l i con a l gor i thms i t w i l l produce. DICK: Th i s i s exac t l y the i dea tha t mot i va ted my arch i tec ture : make a mach i ne much more genera l than wha t I wanted to do. So tha t I cou l d exper i ment w i th a l gor i thms on tha t mach i ne. So tha t the next t i me around I ' l l know wha t I want to do and I can put i t i n s i l i con. So say we ' ve got the l i mi ts of wha t we need to do on one card of spec i a l i zed ch i ps, and I ' ve got my genera l purpose computer cont ro l l i ng i t. So you ' re say i ng tha t a t tha t po i nt a supercomputer wou l d be a system l i ke tha t but one wh i ch wou l d do a l l these th i ngs. CARVER : And a l ot of them. So tha t your i nst ruc t i ons, i nstead of be i ng a l oad i nst ruc t i on or a store i nst ruc t i on - - you st i l l have those - - but wha t you rea l l y do i s you say go Four r i er - t ransform th i s, go render th i s d i sp l ay l i st on the screen w i th the ray- t rac i ng board, go ana l yze th i s speech w i th th i s other board. But i n fac t a ne twork i n wh i ch those func t i ons are d i st r i buted, each node w i th a par t i cu l ar spec i a l ty, wou l d be equ i va l ent to the supercomputer. You cou l d synthes i ze a supercomputer by hav i ng sma l l processors on a ne twork. CARVER : I th i nk tha t th i ng you j ust sa i d i s go i ng to rep l ace the supercomputer as a concept : you have servers on a ne twork. You have a server tha t does the Four i er ana l ys i s and a server tha t does the graph i cs render i ng and so on, i nstead of th i nk i ng of i t as be i ng i n one box. Because then i t ' s j ust a much more conven i ent se tup. So i n fac t I don ' t see any grea t b i g b l ue boxes w i th g i ant power supp l i es w i th i n a few years. I j ust don ' t see them a t a l l. Because you can buy some th i ng tha t ' s got a thousand t i mes the capab i l i ty i n one of these l i t t l e boxes you hook on a ne twork. Tha t makes so much more sense. So bandw i dth i s emerg i ng as the ma j or bot t l eneck. CARVER : I t a l ways has been. I t ' s a ma j or oppor tun i ty tha t b i gger bandw i dths are ava i l ab l e now. F i bers not on l y car ry more, they go l onger d i stances. The ne tworks you ' re l i ke l y to f i nd i n a med i um-s i zed company or research l ab i s the ne twork tha t runs down the ha l l to the room where you ' ve got a l l your spec i a l i zed mach i nes. So you can use th i s power fu l mach i ne f rom your own of f i ce. Tha t k i nd of ne t - work i s prov i d i ng a l ot of power to the peop l e and you don ' t have to go out over the phone company to do any of tha t. Tha t ' s a dec i s i on tha t can be made by a sma l l company, not a b i g company. So wha t we ' re f i nd i ng i s tha t a l l of the de f i n i t i ons of these th i ngs are be i ng done by the peop l e who are bu i l d i ng the stuf f. And the government agenc i es and AT&T come a l ong l a ter and say, "Oh, gee! We d i dn ' t say i t was OK. " We l l you d i dn ' t have to say i t was OK, thank god ; somebody got a chance to do i t anyway. I f we were wa i t i ng around for a ne twork standard to emerge nobody wou l d have any ne tworks r i ght now. In fac t we ' ve had ne tworks s i nce the mi d-70s because peop l e j ust went of f and d i d them. When you l eave the bu i l d i ng, tha t ' s when the po l i t i cs star t. I t ' s hor r i b l e. Where are we now i n speech recogn i t i on i n re l a t i on to some th i ng tha t cou l d be put i nto most user i nter faces? Is th i s techno l ogy we ' ve j ust been ta l k i ng about l ead i ng to a so l ut i on of the user - i ndependent recogn i t i on dev i ce w i th i n ten years. DICK: Cer ta i n l y w i th i n ten years there ' s go i ng to be a l ot happen i ng. I t ' s a dangerous f i e l d to make pred i c t i ons about. I f you l ook ten or twenty years ago peop l e were mak i ng very opt i mi st i c pred i c t i ons and not do i ng the r i ght th i ngs. Wha t ' s happened dur i ng tha t t i me i s tha t, i n many cases peop l e have pre tended tha t the prob l em i s techno l ogy l i mi ted. So every t i me there ' s an advance i n IC techno l ogy they make the next be t ter th i ng, but i t ' s st i l l no good. CARVER : I t ' s j ust l i ke mi crocomputers. The i dea st ream has not got ten be t ter as fast as the techno l ogy has. So they make a much faster th i ng tha t ' s st i l l l ousy because i t doesn ' t have any new i deas i n i t. Tha t has happened a l ot. Just l i ke the mi croprocessor i s an i mage of the ug l y o l d computers of yesteryear. DICK: When you record w i th j ust one mi crophone you l ose a l ot of i nforma t i on. So wh i l e your two ears don ' t have any t roub l e hear i ng me, th i s one mi crophone may have more d i f f i cu l ty. Th i s i s one area where there ' s a c l ear d i rec t i on tha t ' s needed i n the speech recogn i t i on bus i ness - - th i ngs tha t l i sten to peop l e ta l k i ng need to have two