code: def calculate_generalized_advantage_estimator(
      reward, value, done, gae_gamma, gae_lambda):
    # pylint: disable=g-doc-args
    
    # pylint: enable=g-doc-args
  
    next_value = value[1:, :]
    next_not_done = 1 - tf.cast(done[1:, :], tf.float32)
    delta = (reward[:-1, :] + gae_gamma * next_value * next_not_done
             - value[:-1, :])
  
    return_ = tf.reverse(tf.scan(
        lambda agg, cur: cur[0] + cur[1] * gae_gamma * gae_lambda * agg,
        [tf.reverse(delta, [0]), tf.reverse(next_not_done, [0])],
        tf.zeros_like(delta[0, :]),
        parallel_iterations=1), [0])
    return tf.check_numerics(return_, "return")
docs: Generalized advantage estimator.
  
    Returns:
      GAE estimator. It will be one element shorter than the input; this is
      because to compute GAE for [0, ..., N-1] one needs V for [1, ..., N].