I have the original Fila version of Vibram Five Fingers (<a data-hook="product-link-linked" class="a-link-normal" href="/Fila-Men-s-Skele-Toes-2-0-Slip-On-Shoe/dp/B005PLGLSY/ref=cm_cr_arp_d_rvw_txt?ie=UTF8">Fila Men's Skele-Toes 2.0 Slip-On Shoe</a>-quite a bit cheaper, by the way, if you're on a budget), and I've been wearing them for quite a while. Due to a couple of injuries, I hadn't been running for some time, but I got these and another pair (<a data-hook="product-link-linked" class="a-link-normal" href="/Vivobarefoot-Men-s-Neo-Trail-Running-Shoe/dp/B0078J0ZGU/ref=cm_cr_arp_d_rvw_txt?ie=UTF8">Vivobarefoot Men's Neo Trail Running Shoe</a>) to start running again. They are incredibly light and comfortable, and they are very comfortable after wearing toe shoes. It's a very easy switch, back and forth.
They are made of a very breathable ballistic nylon. My feet sweat quite a bit, and my toe shoes can get pretty strong... I've been wearing these both for working out and for running around town (I ditched almost all my other athletic shoes), and they definitely do NOT smell bad. There are small channels of foam sewn into the collar and heel area, with synthetic suede overlays on the sides and around the toe area. Both add to the comfort and strength of this shoe. I used to work in manufacturing, and I can tell you that they really have a great design that makes the shoes comfortable and durable.
I don't buy the 'lugs tearing off' comment, either. I'm 6'5", and I weightlift, so I'm usually in the 270-280 range, and other than wearing down some of the tiny nubs on the lugs, mine are in great shape after several weeks of hard wear. If you take a close look at the sole, the lugs project out, but they are part of longer strips embedded in the sole. If they tore off, it would be pretty obvious that a whole strip of lugs came out of the shoe. Heads up on the Vibram sole. It is a very durable material, but I've found it to lose traction on wet, smooth surfaces (I slid around on a just-mopped tile floor, and I can find the greasy spots on a tile floor, too), but I used to have a couple of pair of boots with Vibram soles that had the same problem. There isn't a 'perfect' material for shoe soles, and these are designed for walking and running on trails.
Finally, I do NOT wear them barefoot. I always wear at least ankle socks, when I run. They are pretty roomy in the toe box, so if you don't wear socks, I'd suggest you order a size down.