/* ĊĠĠĠ ĠThe ĠMIT ĠLicense Ġ( MIT ) ĊĊĠĠĠ ĠF TL - Compiler ĠCopyright Ġ( c ) Ġ 2 0 1 6 Ġth oma ĊĊĠĠĠ ĠPermission Ġis Ġhereby Ġgranted , Ġfree Ġof Ġcharge , Ġto Ġany Ġperson Ġobtaining Ġa Ġcopy ĊĠĠĠ Ġof Ġthis Ġsoftware Ġand Ġassociated Ġdocumentation Ġfiles Ġ( the Ġ" Software "), Ġto Ġdeal ĊĠĠĠ Ġin Ġthe ĠSoftware Ġwithout Ġrestriction , Ġincluding Ġwithout Ġlimitation Ġthe Ġrights ĊĠĠĠ Ġto Ġuse , Ġcopy , Ġmodify , Ġmerge , Ġpublish , Ġdistribute , Ġsublicense , Ġand / or Ġsell ĊĠĠĠ Ġcopies Ġof Ġthe ĠSoftware , Ġand Ġto Ġpermit Ġpersons Ġto Ġwhom Ġthe ĠSoftware Ġis ĊĠĠĠ Ġfurnished Ġto Ġdo Ġso , Ġsubject Ġto Ġthe Ġfollowing Ġconditions : ĊĊĠĠĠ ĠThe Ġabove Ġcopyright Ġnotice Ġand Ġthis Ġpermission Ġnotice Ġshall Ġbe Ġincluded Ġin Ġall ĊĠĠĠ Ġcopies Ġor Ġsubstantial Ġportions Ġof Ġthe ĠSoftware . ĊĊĠĠĠ ĠTHE ĠSOFTWARE ĠIS ĠPROVIDED Ġ" AS ĠIS ", ĠWITHOUT ĠWARRANTY ĠOF ĠANY ĠKIND , ĠEXPRESS ĠOR ĊĠĠĠ ĠIMPLIED , ĠINCLUDING ĠBUT ĠNOT ĠLIMITED ĠTO ĠTHE ĠWARRANTIES ĠOF ĠMERCHANTABILITY , ĊĠĠĠ ĠFITNESS ĠFOR ĠA ĠPARTICULAR ĠPURPOSE ĠAND ĠNONINFRINGEMENT . ĠIN ĠNO ĠEVENT ĠSHALL ĠTHE ĊĠĠĠ ĠAUTHORS ĠOR ĠCOPYRIGHT ĠHOLDERS ĠBE ĠLIABLE ĠFOR ĠANY ĠCLAIM , ĠDAMAGES ĠOR ĠOTHER ĊĠĠĠ ĠLIABILITY , ĠWHETHER ĠIN ĠAN ĠACTION ĠOF ĠCONTRACT , ĠTORT ĠOR ĠOTHERWISE , ĠARISING ĠFROM , ĊĠĠĠ ĠOUT ĠOF ĠOR ĠIN ĠCONNECTION ĠWITH ĠTHE ĠSOFTWARE ĠOR ĠTHE ĠUSE ĠOR ĠOTHER ĠDEALINGS ĠIN ĠTHE ĊĠĠĠ ĠSOFTWARE . Ċ */ Ċ package Ġcom . th omas . need ham . ft l . frontend . lexer Ċ Ċ import Ġcom . th omas . need ham . ft l . utils . Global Functions . TryParse Ċ import Ġcom . th omas . need ham . ft l . utils . Global Functions . Try Decode Ċ import Ġcom . th omas . need ham . ft l . frontend . lexer . Enum Token Types Ċ import Ġcom . th omas . need ham . ft l . frontend . lexer . Token Ċ Ċ /** Ċ Ġ* ĠClass Ġto Ġmanage Ġall Ġof Ġthe Ġtokens Ġthat Ġare Ġpredefined Ġwithin Ġthe Ġcompiler Ċ Ġ* Ġsuch Ġas ĠKey words , ĠTypes , ĠOperators , Ġetc . ĠAnd Ġalso Ġall Ġof Ġthe Ġtokens Ġproduced Ġby Ġthe ĠLex er Ċ Ġ* Ġwhen Ġlex ing Ġa Ġsource Ġfile Ċ Ġ* Ġ@ author ĠTom ĠNeed ham Ġ( 0 6 need ham t ) Ċ Ġ* Ġ@ see ĠLex er Ċ Ġ*/ Ċ class ĠToken Registry Ġ{ Ċ ĉ /** Ċĉ Ġ* ĠComp anion Ġobject Ġof ĠToken ĠRegistry Ġto Ġcontain Ġall Ġstatic Ġfunctions Ġand Ġvariables Ċĉ Ġ*/ Ċ ĉ comp anion Ġobject ĠStatic Data Ġ{ Ċĉ ĉ /** Ċĉĉ Ġ* ĠInitial isation Ġlambda Ġfor ĠArray < Token > Ċĉĉ Ġ*/ Ċĉ ĉ @ Jvm Static Ġval Ġtoken Array Init : Ġ( Int ) Ġ-> ĠToken Ġ= Ġ{ Ċĉĉ ĉ Token (" NULL ", ĠEnum Token Types . Types . UNKNOWN _ TOKEN ) Ċĉ ĉ } Ċ ĉ } ĊĊ ĉ /** Ċĉ Ġ* ĠMutable Ġlist Ġto Ġcontain Ġall Ġtokens Ġoutput ted Ġby Ġthe ĠLex er Ċĉ Ġ* Ġ@ see ĠLex er Ċĉ Ġ*/ Ċ ĉ val Ġregistered Tokens : ĠMutable List < Token > ĊĊ ĉ /** Ċĉ Ġ* ĠImmutable Ġlist Ġto Ġcontain Ġall Ġall Ġof Ġthe Ġtokens Ġthat Ġare Ġpredefined Ġwithin Ġthe Ġcompiler Ċĉ Ġ* Ġsuch Ġas ĠKey words , ĠTypes , ĠOperators , Ġetc . Ċĉ Ġ*/ Ċ ĉ val Ġpredefined Tokens : ĠArray < Token > Ġ= ĠarrayOf < Token >( Ċĉ ĉ Token (" class ", ĠEnum Token Types . Types . KEYWORD _ TOKEN ), Ċĉ ĉ Token (" interface ", ĠEnum Token Types . Types . KEYWORD _ TOKEN ), Ċĉ ĉ Token (" enum ", ĠEnum Token Types . Types . KEYWORD _ TOKEN ), Ċĉ ĉ Token (" singleton ", ĠEnum Token Types . Types . KEYWORD _ TOKEN ), Ċĉ ĉ Token (" array ", ĠEnum Token Types . Types . KEYWORD _ TOKEN ), Ċĉ ĉ Token (" static ", ĠEnum Token Types . Types . KEYWORD _ TOKEN ), Ċĉ ĉ Token (" final ", ĠEnum Token Types . Types . KEYWORD _ TOKEN ), Ċĉ ĉ Token (" abstract ", ĠEnum Token Types . Types . KEYWORD _ TOKEN ), Ċĉ ĉ Token (" override ", ĠEnum Token Types . Types . KEYWORD _ TOKEN ), Ċĉ ĉ Token (" while ", ĠEnum Token Types . Types . KEYWORD _ TOKEN ), Ċĉ ĉ Token (" for ", ĠEnum Token Types . Types . KEYWORD _ TOKEN ), Ċĉ ĉ Token (" if ", ĠEnum Token Types . Types . KEYWORD _ TOKEN ), Ċĉ ĉ Token (" else ", ĠEnum Token Types . Types . KEYWORD _ TOKEN ), Ċĉ ĉ Token (" switch ", ĠEnum Token Types . Types . KEYWORD _ TOKEN ), Ċĉ ĉ Token (" case ", ĠEnum Token Types . Types . KEYWORD _ TOKEN ), Ċĉ ĉ Token (" default ", ĠEnum Token Types . Types . KEYWORD _ TOKEN ), Ċĉ ĉ Token (" synchron ised ", ĠEnum Token Types . Types . KEYWORD _ TOKEN ), Ċĉ ĉ Token (" continue ", ĠEnum Token Types . Types . KEYWORD _ TOKEN ), Ċĉ ĉ Token (" break ", ĠEnum Token Types . Types . KEYWORD _ TOKEN ), Ċĉ ĉ Token (" true ", ĠEnum Token Types . Types . KEYWORD _ TOKEN ), Ċĉ ĉ Token (" false ", ĠEnum Token Types . Types . KEYWORD _ TOKEN ), Ċĉ ĉ Token (" null ", ĠEnum Token Types . Types . KEYWORD _ TOKEN ), Ċĉ ĉ Token (" func ", ĠEnum Token Types . Types . KEYWORD _ TOKEN ), Ċĉ ĉ Token (" do ", ĠEnum Token Types . Types . KEYWORD _ TOKEN ), Ċĉ ĉ Token (" var ", ĠEnum Token Types . Types . KEYWORD _ TOKEN ), Ċĉ ĉ Token (" let ", ĠEnum Token Types . Types . KEYWORD _ TOKEN ), Ċĉ ĉ Token (" return ", ĠEnum Token Types . Types . KEYWORD _ TOKEN ), Ċĉ ĉ Token (" this ", ĠEnum Token Types . Types . KEYWORD _ TOKEN ), Ċĉ ĉ Token (" call ", ĠEnum Token Types . Types . KEYWORD _ TOKEN ), Ċĉ ĉ Token (" goto ", ĠEnum Token Types . Types . KEYWORD _ TOKEN ), Ċĉ ĉ Token (" as ", ĠEnum Token Types . Types . KEYWORD _ TOKEN ), Ċĉ ĉ Token (" int ", ĠEnum Token Types . Types . TYPE _ TOKEN ), Ċĉ ĉ Token (" short ", ĠEnum Token Types . Types . TYPE _ TOKEN ), Ċĉ ĉ Token (" long ", ĠEnum Token Types . Types . TYPE _ TOKEN ), Ċĉ ĉ Token (" float ", ĠEnum Token Types . Types . TYPE _ TOKEN ), Ċĉ ĉ Token (" double ", ĠEnum Token Types . Types . TYPE _ TOKEN ), Ċĉ ĉ Token (" bool ", ĠEnum Token Types . Types . TYPE _ TOKEN ), Ċĉ ĉ Token (" byte ", ĠEnum Token Types . Types . TYPE _ TOKEN ), Ċĉ ĉ Token (" char ", ĠEnum Token Types . Types . TYPE _ TOKEN ), Ċĉ ĉ Token (" object ", ĠEnum Token Types . Types . TYPE _ TOKEN ), Ċĉ ĉ Token (" string ", ĠEnum Token Types . Types . TYPE _ TOKEN ), Ċĉ ĉ Token (" =", ĠEnum Token Types . Types . OPERATOR _ TOKEN ), Ċĉ ĉ Token ("+ ", ĠEnum Token Types . Types . OPERATOR _ TOKEN ), Ċĉ ĉ Token ("- ", ĠEnum Token Types . Types . OPERATOR _ TOKEN ), Ċĉ ĉ Token ("* ", ĠEnum Token Types . Types . OPERATOR _ TOKEN ), Ċĉ ĉ Token ("/", ĠEnum Token Types . Types . OPERATOR _ TOKEN ), Ċĉ ĉ Token ("% ", ĠEnum Token Types . Types . OPERATOR _ TOKEN ), Ċĉ ĉ Token ("+ =", ĠEnum Token Types . Types . OPERATOR _ TOKEN ), Ċĉ ĉ Token ("- =", ĠEnum Token Types . Types . OPERATOR _ TOKEN ), Ċĉ ĉ Token ("* =", ĠEnum Token Types . Types . OPERATOR _ TOKEN ), Ċĉ ĉ Token ("/ =", ĠEnum Token Types . Types . OPERATOR _ TOKEN ), Ċĉ ĉ Token ("% =", ĠEnum Token Types . Types . OPERATOR _ TOKEN ), Ċĉ ĉ Token (" ==", ĠEnum Token Types . Types . OPERATOR _ TOKEN ), Ċĉ ĉ Token (" ! =", ĠEnum Token Types . Types . OPERATOR _ TOKEN ), Ċĉ ĉ Token ("& ", ĠEnum Token Types . Types . OPERATOR _ TOKEN ), Ċĉ ĉ Token ("| ", ĠEnum Token Types . Types . OPERATOR _ TOKEN ), Ċĉ ĉ Token (" ~ ", ĠEnum Token Types . Types . OPERATOR _ TOKEN ), Ċĉ ĉ Token ("^ ", ĠEnum Token Types . Types . OPERATOR _ TOKEN ), Ċĉ ĉ Token (" << ", ĠEnum Token Types . Types . OPERATOR _ TOKEN ), Ċĉ ĉ Token ("> >", ĠEnum Token Types . Types . OPERATOR _ TOKEN ), Ċĉ ĉ Token ("> >> ", ĠEnum Token Types . Types . OPERATOR _ TOKEN ), Ċĉ ĉ Token ("& =", ĠEnum Token Types . Types . OPERATOR _ TOKEN ), Ċĉ ĉ Token ("| =", ĠEnum Token Types . Types . OPERATOR _ TOKEN ), Ċĉ ĉ Token (" ~ =", ĠEnum Token Types . Types . OPERATOR _ TOKEN ), Ċĉ ĉ Token ("^ =", ĠEnum Token Types . Types . OPERATOR _ TOKEN ), Ċĉ ĉ Token (" << =", ĠEnum Token Types . Types . OPERATOR _ TOKEN ), Ċĉ ĉ Token ("> > =", ĠEnum Token Types . Types . OPERATOR _ TOKEN ), Ċĉ ĉ Token ("> >> =", ĠEnum Token Types . Types . OPERATOR _ TOKEN ), Ċĉ ĉ Token (" (", ĠEnum Token Types . Types . LEFT _ PAREN _ TOKEN ), Ċĉ ĉ Token (" )", ĠEnum Token Types . Types . RIGHT _ PAREN _ TOKEN ), Ċĉ ĉ Token ("[ ", ĠEnum Token Types . Types . LEFT _ S QUARE _ PAREN _ TOKEN ), Ċĉ ĉ Token (" ]", ĠEnum Token Types . Types . RIGHT _ S QUARE _ PAREN _ TOKEN ), Ċĉ ĉ Token ("{ ", ĠEnum Token Types . Types . LEFT _ BRACE _ TOKEN ), Ċĉ ĉ Token (" }", ĠEnum Token Types . Types . RIGHT _ BRACE _ TOKEN ), Ċĉ ĉ Token ("\ n ", ĠEnum Token Types . Types . WH ITES PACE _ TOKEN ), Ċĉ ĉ Token ("\ r ", ĠEnum Token Types . Types . WH ITES PACE _ TOKEN ), Ċĉ ĉ Token ("\ t ", ĠEnum Token Types . Types . WH ITES PACE _ TOKEN ), Ċĉ ĉ Token (" Ġ", ĠEnum Token Types . Types . WH ITES PACE _ TOKEN ), Ċĉ ĉ Token (",", ĠEnum Token Types . Types . COMMA _ TOKEN ), Ċĉ ĉ Token (".", ĠEnum Token Types . Types . DOT _ TOKEN ), Ċĉ ĉ Token (" :", ĠEnum Token Types . Types . COL ON _ TOKEN ), Ċĉ ĉ Token (" ;", ĠEnum Token Types . Types . SE MIC OL ON _ TOKEN ), Ċĉ ĉ Token (" -> ", ĠEnum Token Types . Types . ARROW _ TOKEN ), Ċĉ ĉ Token ("", ĠEnum Token Types . Types . END _ OF _ FILE _ TOKEN ), Ċĉ ĉ Token ("\ u 0 0 0 0 ", ĠEnum Token Types . Types . END _ OF _ FILE _ TOKEN ) Ċ ĉ ) ĊĊ ĉ /** Ċĉ Ġ* ĠConstructor ĠFor Ġthe ĠToken Registry Ċĉ Ġ*/ Ċ ĉ constructor () Ġ{ Ċĉ ĉ registered Tokens Ġ= Ġmutable ListOf < Token >() Ċ ĉ } ĊĊ ĉ /** Ċĉ Ġ* ĠFunction Ġto Ġadd Ġa ĠToken Ġinto Ġthe ĠToken Registry Ċĉ Ġ* Ġ@ param Ġvalue ĠThe Ġstring Ġvalue Ġof Ġthe Ġtoken Ġto Ġbe Ġadded Ċĉ Ġ* Ġ@ return ĠWhether Ġthe ĠToken Ġwas Ġsuccessfully Ġadded Ġto Ġthe ĠToken Registry Ċĉ Ġ* Ġ@ see ĠToken Ċĉ Ġ*/ Ċ ĉ fun Ġregister Token ( value : ĠString ): ĠBoolean Ġ{ Ċĉ ĉ var Ġtemp : ĠToken ? Ġ= Ġpredefined Tokens . first OrNull Ġ{ Ġx Ġ-> Ġ// ĠIs ĠThe ĠToken ĠAlready Ġpresent Ġin Ġthe Ġregistry Ċĉĉ ĉ x . value Ġ== Ġvalue Ċĉ ĉ } Ċĉ ĉ if Ġ( temp Ġ!= Ġnull ) Ġ{ Ġ// Ġif Ġthe Ġtoken Ġis Ġalready Ġpresent Ċĉĉ ĉ registered Tokens . add ( temp ) Ġ// Ġjust Ġadd Ġit Ġto Ġthe Ġregistry Ġno Ġneed Ġto Ġresolve Ġit 's Ġtype Ċĉ ĉ } Ġelse Ġ{ Ġ// Ġif Ġthe Ġtoken Ġis Ġnot Ġalready Ġpresent Ċĉĉ ĉ if Ġ( Double . TryParse ( value )) Ġ// Ġis Ġit Ġa Ġvalid Ġnumeric Ġliteral Ċĉĉĉ ĉ temp Ġ= ĠToken ( value , ĠEnum Token Types . Types . NUMERIC _ LITERAL ) Ċĉĉ ĉ else Ġif ( Long . Try Decode ( value )) Ġ// Ġis Ġit Ġa Ġvalid Ġnon Ġbase Ġ 1 0 Ġnumeric Ġliteral Ċĉĉĉ ĉ temp Ġ= ĠToken ( value , ĠEnum Token Types . Types . NUMERIC _ LITERAL ) Ċĉĉ ĉ else Ġif Ġ( value . startsWith ("\" ") Ġ&& Ġvalue . endsWith ("\ "" )) Ġ// Ġis Ġit Ġa Ġvalid Ġstring Ġliteral Ċĉĉĉ ĉ temp Ġ= ĠToken ( value , ĠEnum Token Types . Types . STRING _ LITERAL ) Ċĉĉ ĉ else Ġif Ġ(! value . isNull OrEmpty () Ġ&& Ġ! value . isNull Or Blank ()) Ġ// Ġis Ġit Ġa Ġvalid Ġidentifier Ċĉĉĉ ĉ temp Ġ= ĠToken ( value , ĠEnum Token Types . Types . IDENTIFIER _ TOKEN ) Ċĉĉ ĉ else Ġ{ Ċĉĉĉ ĉ System . err . println (" ERROR : ĠFound ĠUnknown ĠToken : Ġ${ value }") Ċĉĉĉ ĉ // ĠWe Ġhave Ġfound Ġan Ġunknown Ġtoken ĠN . B ĠThis Ġshould Ġnever Ġhappen Ċĉĉĉ ĉ return Ġfalse Ċĉĉ ĉ } Ċĉĉ ĉ registered Tokens . add ( temp ) Ġ// Ġthe Ġtoken Ġhas Ġbeen Ġresolved Ġadd Ġit Ġto Ġthe Ġregistry Ċĉ ĉ } Ċĉ ĉ return Ġtrue Ċ ĉ } Ċ }