So, what exactly is an 'ultra-cool' dwarf star? The TRAPPIST-1 system is around an ultra-cool dwarf star. I went looking for more information on that kind of star, and found very little. The Wikipedia article on it lengthened from a minimal stub to one paragraph in the 15 minutes from when I began the search, after the press conference on the discovery of 7 Earth-sized planets in orbit around TRAPPIST-1.

What kind of star is this?
Breaking the phrase down: 


Dwarf star - a term I will never understand - is used to describe relatively small, dim stars. Unfortunately, this encompasses most main-sequence stars, which are indeed dwarfs compared to large giants and supergiants.
Ultra-cool, as called2voyage already discussed, means that the star has an


  effective temperatures of less than 2,700 kelvin.


This is about half of the Sun's surface temperature. Spectroscopically, it means that these stars are of class M7-L8, i.e. really, really cool stars that just barely reach the threshold for nuclear fusion. Actually, some L dwarfs will never fuse hydrogen as normal stars do, and will become brown dwarfs. Therefore, "ultra-cool dwarf" doesn't necessarily refer to just main sequence stars.


Other characteristics in some cases (from Cruz et al.):


Lithium in some L-type objects, indicative of low temperatures, low masses, and young ages in those particular objects.
Very low surface gravities in late M-type dwarfs (though likely not as much in an M7 dwarf like TRAPPIST-1).
Very low metallicity in certain other L-type objects.


TRAPPIST-1 actually appears to be a rather massive ultra-cool dwarf, then, in comparison to some of these other objects.

Here's an annotated Hertzsprung-Russell diagram, with a box for ultra-cool dwarfs and a circle for where TRAPPIST-1 is, approximately:



Image from Wikipedia courtesy of user Saibo, under the Creative Commons Attribution-Share Alike 2.5 Generic license.