To count on someone means you can rely on them. To count also means to determine the total number of something. The mathematician is inept, so unable to determine the number of his friends, or to rely on them. 
The inept mathematician couldn ' t count on his friends .