Whatâs more, fried potatoes, such as French fries, have been associated (along with other foods) with a greater risk of type 2 diabetes, which is known to cause sexual health problems in both men and women. Sass says the type of oil used to fry the food is key.