Can I make this assumption about the utility function? My question concerns a household's utility function U(c,l). We're told that the partial derivatives dU/dc and dU/dl > 0, and the second order partial derivatives are negative. Now, can I assume anything about the sign of the partial derivatives Ucl (or Ulc)?
No.

In general, based on the information you have, there are no results you can infer for the partial derivatives $U_{cl}$ / $U_{lc}$. In this context they could be anything.

As to what you are allowed to assume is a different story. You can assume anything in general, of course. Nevertheless, there is no way to check if your assumptions would be correct in this case.