When are banks allowed to fail? If the Federal Reserve acts as a lender of last resort and steps in to prevent bank runs, then how do banks still fail?
Do they only step in if the bank is solvent but in emergency need of funds?
First, it is actually not Fed that prevents bank runs. Bank runs in the US are prevented by Federal Deposit Insurance Corporation (FDIC)  which insures deposits that people have at the bank up to \$250000. This is what prevents the bank runs as runs are created when people worry bank will fail and they will loose their deposits. This is because bank never has enough money to let all people withdraw them at the same time as loans are made long term while deposits can be withdrew at a depositors whim and consequently if people worry bank will fail they will all 'run' to their bank in order to make sure they get their money before bank fails and stops paying out deposits.
Second, generally and broadly speaking the literature agrees that there is some rationale for saving banks under two conditions (Hanson, Kashyap &amp; Stein 2011; Osinski Seal &amp; Hoogduin, 2013):

They are systemically important (their collapse would lead to collapse of other institutions).
They are only illiquid not insolvent.

The point 1 is considered to be more important than the second so even illiquid institutions might be considered to be worth saving if they are too systemically important.
Now Fed or other government bodies can't be always $100\%$ sure about how systemically important bank is or whether it is illiquid or only insolvent, but central banks around the world typically create and track some measures of systemic importance of various banking institutions. For example, Fed has whole dedicated unit/department just for researching systemic risk.  In addition they will also try to estimate whether bank is insolvent or just illiquid. Lastly, since no research can give you $100\%$ correct answer at the speed Fed usually has to act there is also some discretion involved in the decision process. So partially it also depends on what decision makers at Fed think should be done in each specific case.
However, if some banks are judged to be insolvent and not systemically important  they are routinely allowed fail. In fact as the data from FDIC show the total number of commercial banks that have to register for deposit insurance (basically any 'ordinary' bank) fell by almost 200 just between year 2018-2019.