code: def average_sharded_losses(sharded_losses):
    
    losses = {}
    for loss_name in sorted(sharded_losses[0]):
      all_shards = [shard_losses[loss_name] for shard_losses in sharded_losses]
      if isinstance(all_shards[0], tuple):
        sharded_num, sharded_den = zip(*all_shards)
        mean_loss = (
            tf.add_n(sharded_num) / tf.maximum(
                tf.cast(1.0, sharded_den[0].dtype), tf.add_n(sharded_den)))
      else:
        mean_loss = tf.reduce_mean(all_shards)
  
      losses[loss_name] = mean_loss
    return losses
docs: Average losses across datashards.
  
    Args:
      sharded_losses: list<dict<str loss_name, Tensor loss>>. The loss
        can be a single Tensor or a 2-tuple (numerator and denominator).
  
    Returns:
      losses: dict<str loss_name, Tensor avg_loss>