Does segregation make economic sense? A central tenet of libertarianism is the notion of free association–being able to socialise and enter into contracts with people of your choosing, with the corollary of being able to refuse to do those things too. The obvious criticism of this would be that one could then refuse to sell to people based on their skin colour, sex, orientation, etc., which has a historical precedent pretty much all over the globe. Does this make economic sense, though, or does it represent a market failure?

To my non-economist eyes it looks pretty cut and dry. Say Alice has a widget store and decides to only sell to people in group A. Bob also has a widget store and sells to both groups A &amp; B. Surely Bob should have a significant economic advantage here, allowing him to lower prices and stay competitive with Alice? Even if some group A customers would refuse to shop at Bob's, presumably more would be attracted to Bob's store for its lower prices?1

Even if social or legal pressures meant everyone had to segregate, would this not present a gap in the market to be filled by group B store-owners, who would then sell to the underserved group Bs, with prices presumably more in line with group B incomes (if they are also economically disadvantaged compared to group A), and with the profits more likely to be redistributed amongst other group B businesses?

It's a cliche that the libertarian answer to everything is that 'the market will fix it', but this does seem like a sitation where that should happen, unless government pressure countered market forces (e.g. if it was made illegal for group Bs to even open their own stores or buy stock from wholesalers willing to sell to them). Would the libertarian counter-claim be, then, that the existence of economic segregation can only occur when backed by state force?

1 However, this paper does suggest that '[d]iscriminators are on average willing to forego 8 percent of their earnings to avoid a coworker of the other ethnic type'.
In many models, we assume that firms simply want to maximize profits, and there are no externalities. If these assumptions are true, then a free market will be racism-free, gender-blind etc, etc. However, this is not the case in the real world. People that own and operate firms not only want to maximize profits, often they also have all kinds of biases, including social biases like racism, sexism, etc; as well as cognitive biases, for example when they want to make the firm as big as possible even if that is not profitable. In all cases, this biases necessarily mean forgoing profits. For example the paper you cite. Ross Levine has other papers on the topic.

However, the question of segregation is broader than this. For some type of relationships, "segregation" can make economic sense. For example, if you have a common resource, like a lake that provides some fishes, limiting who has access to such lake to only people who take care of the lake and does not overfish it, can have clear advantages, and be socially efficient. 

In short, segregation can be the result of a friction, like cognitive or social biases. This is not a market failure, properly speaking, but it entails some inefficiencies, so in that sense, it does not make "economic sense". But you can also have segregation, being the optimal response to a situation that involves externalities (like in the example of the lake). This segregation can make "economic sense" in that it is individually rational, and it may or may not be socially efficient. 

Consider an example where it is likely not socially efficient. Suppose there is a big proportion of people who are racist, and you own a restaurant. Let's assume you are not racist and you only care about maximizing profits. It is perfectly reasonable to think that you will deny service to people based on race if that will make your racist costumers more likely to visit your restaurant. This segregation makes "economic sense" even though it might not be socially efficient in some way or desirable (the social welfare can be measured in many ways). Notice that I needed to assume that your other customers somehow have negative externalities from you selling to racial minorities.