What is the game theory of retaliatory trade tariffs? Suppose no tariffs, or negligible tariffs, exist between nations A and B, and suddenly A introduces substantive tariffs on certain products A imports from B. In theory, this could lead to B and A taking turns imposing tariffs on specific products they import from each other. Given the economic benefits of tariff-free trade, this scenario requires

(i) the optimum tariffs in one direction to depend on those in the other and

(ii) the relationship between these to bear in mind expectations of how (i) would lead to a feedback.

I'm sure economists have tried to quantify this (possibly with an eye to each nation's alternative sources for goods from third parties), but I know nothing about the relevant literature. So what do models predict?
Indeed it's difficult to account for all the real-life complexities, but the basic game-theory model of trade wars is prisoner's dilemma, e.g.


https://leadersatwork.northeastern.edu/management/trump-tariffs-and-the-prisoners-dilemma/



  The concept of the Prisoner’s Dilemma is important to a number of business school subjects, including economics and negotiations. The current Trump administration provides a rich opportunity to explore this concept in the context of important policy deliberations. [...]
  
  This also happens between countries and is playing itself out in many of President Trump’s recent actions involving tariffs. Bilateral agreements between the U.S. and its trading partners confront the “Prisoner’s Dilemma” when it comes to the issue of tariffs. Consider when the President said on March 8, 2018, “he” will impose tariffs on steel and aluminum imports within 15 days. Tariffs represent a situation where the “rational” decision would appear to be to raise tariffs on imports and hope the other side either doesn’t or cannot. The “rational” decision typically leads to retaliation as it does in many prisoner dilemma situations.
  
  Of course, President Trump doesn’t see tariffs as posing a prisoner’s dilemma type problem, at least in his public pronouncements: “When a country (USA) is losing many billions of dollars on trade with virtually every country it does business with, trade wars are good, and easy to win. Example, when we are down $100 billion with a certain country and they get cute, don’t trade anymore-we win big. It’s easy!” (March 2, 2018 Twitter)
  
  Not surprisingly, other countries threaten to retaliate. China, Canada, and the European Union have already said they would respond by enacting tariffs that could result in extensive American export losses affecting industries and farms that the President had claimed to protect. Most economists feel that a trade war is disruptive and will make the world as a whole poorer.
  
  Perhaps the President sees international trade as similar to the construction industry; i.e., as a forum where he can exert power and his will on the other party. What works in the construction industry where Trump gained his negotiation experience may not work in the very different world of international diplomacy. The President might have had leverage in dealing with contractors. But in international trade, the U.S. accounts for 9 percent of world exports and 14 percent of imports. This would not appear to be a position of domination.
  
  Of course, another characteristic of President Trump’s negotiation style is to stake out an extreme position (anchor) in order to both intimidate an opponent and to give room to make concessions or compromise, and this may be the case with his tariff pronouncements.



http://blogs.cornell.edu/info2040/2018/09/20/the-trade-war-between-the-u-s-and-china-a-prisoners-dilemma/



  In his [FT] article “Trade wars and the prisoners’ dilemma,” Gavyn Davies draws connections between this tit-for-tat tariff game and the classic example of the prisoners’ dilemma.
  
  Davies highlights the fact that, if the U.S. and China are truly engaged in a non-cooperative, prisoner’s-dilemma style game, they will likely end up in a “bad Nash equilibrium,” where both countries’ payoffs are worse than if they were to simply cooperate with each other. Specifically, assuming that each country continues to choose strategies that protect its best interests in a time of uncertainty, it’s likely that there will be 1-3% reduction in global output in the next several years.
  
  In its entirety, Davies’ article demonstrates that the game theory logic underlying the prisoner’s dilemma is omnipresent. Almost every economist and investor examining the impacts of the budding trade war has assumed that the two countries will continue to retaliate against one another without cooperation. Their models paint a dismal (albeit realistic) picture of this trade war’s global repercussions. Still, economists and investors might be better off employing the same game theory logic to illustrate the mutually beneficial benefits of cooperation. Perhaps the prisoner’s dilemma will help to show how cooperation, not just among inmates, but among countries can lead to better payoffs than self-preserving strategies.



https://www.bloombergquint.com/global-economics/game-theory-explains-why-the-u-s-china-trade-spat-is-worsening



  Experts surveyed by Bloomberg News lay out four scenarios for how the U.S. trade conflict with China could end: Both sides back down, which now seems unlikely in the short term; China blinks; the U.S. blinks; or both sides keep escalating.
  
  The “China blinks” scenario assumes President Xi Jinping won’t want to endure a downturn in the Chinese economy, which showed signs it underperformed in May. The “U.S. blinks” scenario assumes China calls Trump’s bluff, knowing how much he enjoys a strong economy, rising stock market, and the support of voters in farm states that China could target. 
  
  The fourth scenario, in which neither blinks, is the most damaging. “We’re not there yet, but it’s scary, because it seems like we’re on a path toward major conflict, and it’s hard to see the offramp,” says Michael Smart, managing director at Rock Creek Global Advisors LLC in Washington and a former international trade director on the National Security Council.
  
  The World Trade Organization was created to prevent exactly this kind of bluffing and brinkmanship. “Trade agreements are a way of escaping from a prisoner’s dilemma in which each country acting rationally is stuck doing something that is bad for it individually, but they can’t get out of it without a collective agreement,” says Dartmouth College economist Robert Staiger. 
  
  The Bagwell-Staiger Theory of Trade Agreements and the "Reciprocity Approach"


The WTO literature seldom fails to mention prisoner's dilemma (avoidance), e.g.:


  At the latest since the writings of David Ricardo and John Stuart Mill the welfare-enhancing qualities
  of international trade have been common wisdom in the realm of economics (Irwin, 1996). Reality,
  however, paints a different picture: Countries are often reluctant to open up their borders to foreign
  goods and services and unwilling to liberalize trade unilaterally. Yet, there are plenty of examples that
  countries have overcome what appears to be a mercantilist and protectionist stance and engage in
  bilateral or multilateral trade agreements. What is the rationale for such cooperation? Why do
  sovereign countries contract over trade issues?
  
  Starting with the seminal work of Harry Johnson (1953), economists have strived to address this
  question. The basic intuition that has guided much of the literature is that countries cooperate in an
  effort to constrain unilateral “beggar-thy-neighbour” policies. Trade protection by one country may
  reduce the welfare of trading partners; it provokes negative externalities, or spill-overs.
   The strategic
  set-up of a prisoners’ dilemma emerges. Excessive trade protection, albeit inefficient, becomes the
  dominant strategy for importing countries. The purpose of a trade agreement is to eliminate the
  inefficient restrictions on trade volumes that arise when policies are set unilaterally, and thereby offer
  governments a means to escape the prisoners’ dilemma.
  
  Theories of trade agreements based on externalities come in two variations.
   The best-known and
  most elaborate theory of trade cooperation is based on a terms-of-trade driven prisoners’ dilemma that
  (large) countries overcome by means of concluding an international trade contract. The terms-of-trade
  school contends that it is solely the ability of economically large countries to influence world prices
  for goods and services – and their potential to actively deteriorate other states’ terms-of-trade - that
  motivates governments to conclude trade agreements. Large countries realize that the unilateral
  setting of import tariffs creates inefficiencies that can be avoided by mutual trade cooperation. This
  theory is represented most prominently by Kyle Bagwell and Robert Staiger. Their research
  programme is the only one that formally integrates an explanation of why countries cooperate on trade
  matters with an explanation of how this can be done. In other words, it currently is the only approach
  in economics that can explain both the existence of the multilateral trading system and its architecture. 
  
  The political externalities strand of the literature, which gained prominence through the writings of
  Wilfred Ethier (2004; 2004a; 2006), is critical of the terms-of-trade approach. It alleges that “a trade
  agreement serves governments to get credit for the reduction in foreign trade barriers. International
  cooperation is not about the elimination of economic (world-price) externalities, but about political
  externalities. The latter arise when politicians in one country believe that their political status is
  directly affected by actions of politicians of another country” (Hauser and Roitinger, 2004: 652).
  
  Different as these rationales for trade agreements may be concerning possible policy implications,
  their basic intuition is identical: Both schools rely on a simple game set-up, where two or more
  rational players are faced with a prisoners’ dilemma situation. 


(Avoiding) prisoner's dilemma also applies to subsidies, not just tariffs:


  We begin our investigation of the role for retaliation by looking at export subsidies. Bagwell and Staiger
  (2001b) show how a subsidy agreement that limits government payments and avoids subsidy escalation
  can be of value to governments of exporting firms. Without such an agreement, each government is
  tempted to subsidize its exporters so as to create a competitive advantage in a third market – i.e., a
  prisoners' dilemma problem.


It's actually not clear to me if a more formal turn-based "beggar thy neighbor" rather than the simpler simultaneous version (prisoner's dilemma) has received much attention game-theory wise.

Since trade wars are back in fashion, perhaps we'll see more theoretical literature dealing with how they happen rather than how they are avoided...

The tit for tat strategy and the associated iterated prisoner's dilemma have been studied theoretically quite a bit; perhaps there's more literature relating them to trade wars...

Actually there is some literature from three decades ago how trade wars start and iterate, but it seems mired in controversy as far as modelling goes; the hypothesis  of rent-seeking by internal pressure groups adds another dimension to simpler model of the (external) trade war; this conflicting issue of modelling hasn't been fully resolved as of then:


  In Trade Wars, Professor John A.C. Conybeare, a political scientist,
  employs a game-theory approach in an attempt to formalize a predictive
  theory for when trade wars will occur.  While Conybeare praises the
  rent-seeking literature for rendering trade wars "more theoretically and
  empirically understandable," he finds the interest group explanation
  conveyed in that literature to be incomplete because it "has largely been
  confined to examining the interindustry structure of protection within
  countries and, with a few exceptions, has not made international comparisons." In other words, Conybeare takes the position that, while interest
  groups may set the internal political agenda within a particular country,
  no interest group explanation of trade wars can be complete without an
  understanding ofthe constraints facing that country due to its bargaining
  power vis-a-vis other nations. [...]
  
  In Chapter Five, he looks at the Anglo-Hanse trade wars to test his
  assertion that bilateral trade wars between large trading units will result
  in a prisoners' dilemma that will evolve into a cooperative equilibrium as
  the two sides learn to employ contingent retaliation. Unfortunately, four
  hundred years worth of trade-a sufficient time span to produce iterative
  opportunities for even the most exacting game theoretician-did not
  produce cooperation. By the author's own admission this was because
  "domestic rent-seeking in conjunction with a severe public good problem in the use of the tit-for-tat strategy of retaliation" caused "outbreaks
  of conflict to escalate."
  
  In Chapter Six, Professor Conybeare describes the Anglo-French
  trade wars beginning in 1664. These trade wars are used to illustrate the
  author's hypothesis that trade wars between two large countries should
  evolve into a cooperative interaction after some initial defection because
  such wars resemble iterative prisoners' dilemmas.3o Contrary to Professor Conybeare's general hypothesis, the trade wars lasted for over two
  hundred years and did not evolve into a cooperative equilibrium at the
  pace his model predicts. As with the Anglo-Hanse trade wars, the author
  is forced to admit that his own example "illustrates the ease with which
  the effects of iteration may be' disrupted." Once again, the cause of a
  protracted trade war can be traced to domestic rent seeking. [...]
  
  The Chicken Wars began because of Regulation 22, promulgated in
  July, 1962, which curbed the export of frozen chickens into EEC countries. The Chicken Wars brought the powerful agricultural lobby in
  France into direct conflict with the powerful agricultural lobby in the
  United States. Not surprisingly, France, the largest poultry producer in
  the EEC, was unwilling to embrace a free trade policy whose benefits
  would be spread among consumers throughout Europe. The benefits of
  protectionism were concentrated in France and the costs were borne by
  consumers throughout Europe. Indeed, the stakes were sufficiently high
  that France might have quit the EEC had not French agricultural interests been appeased.


So a fuller (game theory) model of trade wars seem to need to account for internal groups as different players rather than treat countries as single entities. I don't know if more progress on this has been made since that book.

A somewhat more recent paper (only two decades old) on the US-Canada wars from more than a century ago:


  The classic analysis of retaliation with optimal tariffs by Johnson (1954) is shown to
  have empirical validity. The pattern of US–Canadian tariff interactions in the period
  1868–70 shows evidence of a retaliatory adjustment process of the kind described by
  Johnson. Both countries perceive they have market power and therefore think they
  can gain from applying retaliatory optimal tariffs. In the short run, changes in the tariff
  level of each country are seen to be Granger-caused by changes in the tariff of the
  other. An interpretation of that result is that a tariff innovation in one country due to
  exogenous political economic forces in one country, say a highly influential protection-
  ist lobby, is met with retaliation by the partner country which seeks to reverse the
  initial increase. This leads to a tariff war. The Granger causality tests capture the short-
  run reactions that give credence to this story.
  
  What is more interesting, however, are the questions about the long-run steady-state
  solution after the retaliation war ends. In the long run, the USA maintains and even
  increases the tariff shock that it initiates, while the Canadian policy tends to back down
  from the tariff shock that it initiates. This is consistent with the Kennan–Reizman
  hypothesis that big countries win tariff wars, no matter who starts it. A second long-run
  result is that the Canadian tariff demonstrates a permanent change following a US
  tariff shock; that is, the cross-country effect is positive. On the other hand, in the long
  run, the US tariff returns to its own internal equilibrium (retaliation component is
  significantly no different from zero), unaffected by innovations in the Canadian tariff.
  This result is consistent with Chan’s (1991) model of bargaining which emphasizes
  the availability of outside options. Since the USA has greater outside options, it
  chooses not to participate in a costly retaliatory war. However, since Canada is highly
  trade-dependent on the USA, its outside options are limited and the best it can do is to
  retaliate in an attempt to bring down the US tariff increase.
  
  Interestingly, an additional inference that may be made from the study is not one
  about retaliation, but about internal political economic factors. While Canadian retali-
  ation does not succeed in reversing a US tariff increase due to political economic
  factors, US retaliation is shown to succeed in at least partially reversing a tariff
  shock caused by political economic factors. Hence, not only is the size and lower
  trade dependency of the USA vis-à-vis Canada beneficial in non-cooperative or bar-
  gaining games, but the internal political economic factors in the USA seem to be more
  stubborn in the face of retaliation than are political economic factors in Canada. It was
  perhaps in the best interest of Canada to establish the Canada–US Free Trade Agreement in which the political importance of the issue was far greater in Canada than in
  the USA.


I have to say however that Chan's model (although based on game theory) seems pretty obscure; the paper only has 10 citation in Google scholar. The Kennan-Riezman hypothesis (that big countries win trade wars) seems a lot more discussed, with a few hundred citations; but it doesn't seem primarily based on based on game theory; it uses an Edgeworth box
 analysis to draw its main conclusion as to the bargaining power in a trade war.

A highly-cited 1993 paper by Grossman and Helpman does attempt to model national-level interest groups: it includes not only the global country utility but also (domestic) political contributions in the government's utility function when doing trade negotiations.

I think this is still pretty limited, since it only considered campaign finance contributions as part of the latter. Potential for votes is not distributed among groups, but only modelled via the general country welfare utility function. But even this results in a pretty complicated analysis. Only by the 34th page in their paper they get to introduce the competitive model including foreign governments, which is not developed a lot. The paper notes/conclues the obvious trade-off that a national government has to face between industry lobby groups and general population welfare with respect to tariffs, and also (obviously)


  Tariff rates are highest in industries with the largest stakes in trade policy (i.e., greatest output) and in industries where foreign export supplies and mohme import demands are most inelastic. [...]
  
  Industry groups that are politically stronger than their counterparts abroad will secure exceptions from across-the board tariff cuts or from reductions in the levels of export promotion.


But practically all of this involves equilibrium analyses rather than much game theory; there are no (game) iterations involved in this paper. These two authors have an even more highly cited paper title "protection for sale" which elaborates on the costs borne by lobby/industry groups.