code: def nalu(x, depth, epsilon=1e-30, name=None, reuse=None):
    
    with tf.variable_scope(name, default_name="nalu", values=[x], reuse=reuse):
      x_shape = shape_list(x)
      x_flat = tf.reshape(x, [-1, x_shape[-1]])
      gw = tf.get_variable("w", [x_shape[-1], depth])
      g = tf.nn.sigmoid(tf.matmul(x_flat, gw))
      g = tf.reshape(g, x_shape[:-1] + [depth])
      a = nac(x, depth, name="nac_lin")
      log_x = tf.log(tf.abs(x) + epsilon)
      m = nac(log_x, depth, name="nac_log")
      return g * a + (1 - g) * tf.exp(m)
docs: NALU as in https://arxiv.org/abs/1808.00508.