code: def add_timing_signal_1d(x,
                           min_timescale=1.0,
                           max_timescale=1.0e4,
                           start_index=0):
    
    length = common_layers.shape_list(x)[1]
    channels = common_layers.shape_list(x)[2]
    signal = get_timing_signal_1d(length, channels, min_timescale, max_timescale,
                                  start_index)
    return x + common_layers.cast_like(signal, x)
docs: Adds a bunch of sinusoids of different frequencies to a Tensor.
  
    Each channel of the input Tensor is incremented by a sinusoid of a different
    frequency and phase.
  
    This allows attention to learn to use absolute and relative positions.
    Timing signals should be added to some precursors of both the query and the
    memory inputs to attention.
  
    The use of relative position is possible because sin(x+y) and cos(x+y) can be
    expressed in terms of y, sin(x) and cos(x).
  
    In particular, we use a geometric sequence of timescales starting with
    min_timescale and ending with max_timescale.  The number of different
    timescales is equal to channels / 2. For each timescale, we
    generate the two sinusoidal signals sin(timestep/timescale) and
    cos(timestep/timescale).  All of these sinusoids are concatenated in
    the channels dimension.
  
    Args:
      x: a Tensor with shape [batch, length, channels]
      min_timescale: a float
      max_timescale: a float
      start_index: index of first position
  
    Returns:
      a Tensor the same shape as x.