Personally for me:

1) Mid afternoon - between straight after lunch and between 3:30 or 4pm - I have a theory that this is the point in which people have their biggest slump in terms of energy, and so this isn't the same level of brain capacity to deal with stressors, so you feel more a lot more overwhelmed

2) From around 9pm as your body is getting ready to wind down in preparation for sleep, I usually find that a similar thing happens as above.