What is a dollar, currently? What is the definition of a dollar now without gold backing?  It seems that a dollar is now just an entry in a book somewhere in some bank.  Or it is the federal reserve note?  Or is it both?  I understand the concept of the monetary base.  But what I don't understand is how this works.  If a dollar is in part just an entry in a book, what stops some bank from simply claiming they have an infinite amount of dollars?  Or is there some central registry that claims to keep track of how many dollars each bank has? (This would seem absurd to me.  How then would foreign banks holding US dollars keep track of how many dollars they had?)  If there is no central registry, then how do you prevent people claiming to have dollars on their books?

I think explaining what a dollar is would go toward answering these questions.
A dollar is money which can be used in the United States to purchase goods and services and to repay debts. It can sometimes be used for these purposes in some other parts of the world. 

If a bank says it has an unlimited amount of dollars, then it will presumably be willing to lend them to customers, who will then want to spend them. 

The bank will then either have to give the customer dollar bills (Federal Reserve Notes which presumably it does not have in unlimited amounts) or be willing to transfer dollars to another bank on the instruction of its customer.

The other bank may then expect this to be settled.  Unless there are also flows into the first bank to offset the amount owed, it can only settle with dollar bills (but faces the same issue as before) or by instructing a third bank where it has an account to transfer dollars to the second bank.  This can continue up the chain until an instruction to transfer funds reaches the U.S. Federal Reserve Banks, where the top commercial banks hold their reserve deposits.  

Other banks will not be impressed with the first bank simply saying it has unlimited dollars if it does not have accounts with other banks with dollars to support the transfers.  Writing numbers in your own books does not change the numbers in other banks' books.  The constraints that this causes is one of the constraints on lending and so on the creation of money.