What are the different approaches proposed for query expansion and reformulation in NLP?
Query Expansion and Reformulation Traditionally, query expansion methods based on pseudo relevance feedback utilize relevant context without external resources to expand queries (Rocchio, 1971; Jaleel et al., 2004; Lv and Zhai, 2010; Yu et al., 2021). Recent studies attempt to reformulate queries using generative models, relying on external resources such as search sessions (Yu et al., 2020) or conversational contexts (Lin et al., 2020; Vakulenko et al., 2021), or involve sample-inefficient reinforcement learning training (Nogueira and Cho, 2017). More recently, GAR (Mao et al., 2021a) explored the use PLMs for query expansion instead of external resources. A concurrent study (Liu et al., 2022) generates multiple expansions with beam search and filters and fuses the results, but EAR is aware of the BM25 retriever and could select more promising query expansions and run fewer BM25 retrievals.
Query Expansion Using Contextual Clue Sampling with Language Models
Query expansion is an effective approach for mitigating vocabulary mismatch between queries and documents in information retrieval. One recent line of research uses language models to generate query-related contexts for expansion. Along this line, we argue that expansion terms from these contexts should balance two key aspects: diversity and relevance. The obvious way to increase diversity is to sample multiple contexts from the language model. However, this comes at the cost of relevance, because there is a well-known tendency of models to hallucinate incorrect or irrelevant contexts. To balance these two considerations, we propose a combination of an effective filtering strategy and fusion of the retrieved documents based on the generation probability of each context. Our lexical matching based approach achieves a similar top-5/top-20 retrieval accuracy and higher top-100 accuracy compared with the well-established dense retrieval model DPR, while reducing the index size by more than 96%. For end-to-end QA, the reader model also benefits from our method and achieves the highest Exact-Match score against several competitive baselines.
[{"first":"Linqing","last":"Liu","middle":[]},{"first":"Minghan","last":"Li","middle":[]},{"first":"Jimmy","last":"Lin","middle":[]},{"first":"Sebastian","last":"Riedel","middle":[]},{"first":"Pontus","last":"Stenetorp","middle":[]}]
Generation-Augmented Retrieval for Open-Domain Question Answering
We propose Generation-Augmented Retrieval (GAR) for answering open-domain questions, which augments a query through text generation of heuristically discovered relevant contexts without external resources as supervision. We demonstrate that the generated contexts substantially enrich the semantics of the queries and GAR with sparse representations (BM25) achieves comparable or better performance than state-of-the-art dense retrieval methods such as DPR. We show that generating diverse contexts for a query is beneficial as fusing their results consistently yields better retrieval accuracy. Moreover, as sparse and dense representations are often complementary, GAR can be easily combined with DPR to achieve even better performance. GAR achieves state-of-the-art performance on Natural Questions and TriviaQA datasets under the extractive QA setup when equipped with an extractive reader, and consistently outperforms other retrieval methods when the same generative reader is used.
[{"first":"Yuning","last":"Mao","middle":[]},{"first":"Pengcheng","last":"He","middle":[]},{"first":"Xiaodong","last":"Liu","middle":[]},{"first":"Yelong","last":"Shen","middle":[]},{"first":"Jianfeng","last":"Gao","middle":[]},{"first":"Jiawei","last":"Han","middle":[]},{"first":"Weizhu","last":"Chen","middle":[]}]
Improving Query Representations for Dense Retrieval with Pseudo Relevance Feedback
Dense retrieval systems conduct first-stage retrieval using embedded representations and simple similarity metrics to match a query to documents. Its effectiveness depends on encoded embeddings to capture the semantics of queries and documents, a challenging task due to the shortness and ambiguity of search queries. This paper proposes ANCE-PRF, a new query encoder that uses pseudo relevance feedback (PRF) to improve query representations for dense retrieval. ANCE-PRF uses a BERT encoder that consumes the query and the top retrieved documents from a dense retrieval model, ANCE, and it learns to produce better query embeddings directly from relevance labels. It also keeps the document index unchanged to reduce overhead. ANCE-PRF significantly outperforms ANCE and other recent dense retrieval systems on several datasets. Analysis shows that the PRF encoder effectively captures the relevant and complementary information from PRF documents, while ignoring the noise with its learned attention mechanism.
[{"first":"HongChien","last":"Yu","middle":[]},{"first":"Chenyan","last":"Xiong","middle":[]},{"first":"Jamie","last":"Callan","middle":[]}]
Question Rewriting for Conversational Question Answering
Conversational question answering (QA) requires answers conditioned on the previous turns of the conversation. We address the conversational QA task by decomposing it into question rewriting and question answering subtasks, and conduct a systematic evaluation of this approach on two publicly available datasets. Question rewriting is designed to reformulate ambiguous questions, dependent on the conversation context, into unambiguous questions that are fully interpretable outside of the conversation context. Thereby, standard QA components can consume such explicit questions directly. The main benefit of this approach is that the same questions can be used for querying different information sources, e.g., multiple 3rd-party QA services simultaneously, as well as provide a human-readable interpretation of the question in context. To the best of our knowledge, we are the first to evaluate question rewriting on the conversational question answering task and show its improvement over the end-to-end baselines. Moreover, our conversational QA architecture based on question rewriting sets the new state of the art on the TREC CAsT 2019 dataset with a 28% improvement in MAP and 21% in NDCG@3. Our detailed analysis of the evaluation results provide insights into the sensitivity of QA models to question reformulation, and demonstrates the strengths and weaknesses of the retrieval and extractive QA architectures, that should be reflected in their integration.
[{"first":"Svitlana","last":"Vakulenko","middle":[]},{"first":"Shayne","last":"Longpre","middle":[]},{"first":"Zhucheng","last":"Tu","middle":[]},{"first":"Raviteja","last":"Anantha","middle":[]}]
Query Reformulation using Query History for Passage Retrieval in Conversational Search
Passage retrieval in a conversational context is essential for many downstream applications; it is however extremely challenging due to limited data resources. To address this problem, we present an effective multi-stage pipeline for passage ranking in conversational search that integrates a widely-used IR system with a conversational query reformulation module. Along these lines, we propose two simple yet effective query reformulation approaches: historical query expansion (HQE) and neural transfer reformulation (NTR). Whereas HQE applies query expansion, a traditional IR query reformulation technique, NTR transfers human knowledge of conversational query understanding to a neural query reformulation model. The proposed HQE method was the top-performing submission of automatic systems in CAsT Track at TREC 2019. Building on this, our NTR approach improves an additional 18% over that best entry in terms of NDCG@3. We further analyze the distinct behaviors of the two approaches, and show that fusing their output reduces the performance gap (measured in NDCG@3) between the manually-rewritten and automatically-generated queries to 4 from 22 points when compared with the best CAsT submission.
[{"first":"Sheng-Chieh","last":"Lin","middle":[]},{"first":"Jheng-Hong","last":"Yang","middle":[]},{"first":"Rodrigo","last":"Nogueira","middle":[]},{"first":"Ming-Feng","last":"Tsai","middle":[]},{"first":"Chuan-Ju","last":"Wang","middle":[]},{"first":"Jimmy","last":"Lin","middle":[]}]
Query Expansion Using Contextual Clue Sampling with Language Models
Despite the advent of dense retrieval approaches based on semantic matching for open-domain question answering such as DPR (Karpukhin et al., 2020), approaches based on lexical matching (e.g., BM25) remain important due to their space-efficiency and can serve as input to hybrid methods (Gao et al., 2021;Formal et al., 2021;.
Query Expansion Using Contextual Clue Sampling with Language Models
A core challenge for lexical retrieval is the vocabulary mismatch between the query and documents. Query expansion techniques dating back over half a century have proven effective in overcoming this issue (Salton, 1971). The expansion terms are traditionally precomputed from relevant corpora using pseudo-relevance feedback techniques (Salton, When is the new jurassic world game coming out?  Figure 1: Given the query, we first sample various contextual clues from the language model. Lexically similar outputs are grouped together and we only keep the output with the highest generation probability (Filtering). We then perform retrieval for each augmented query individually. All the retrieved documents are then ranked together in the fusion step. 1971; Robertson and Jones, 1976;Abdul-Jaleel et al., 2004). In recent work, GAR (Mao et al., 2021) explored removing the query expansion's reliance on an external corpus and instead used a large language model to generate a context.
Query Expansion Using Contextual Clue Sampling with Language Models
We argue that expansion needs to balance two key factors: (1) Diversity: Given the question, there can be multiple different reasoning paths (referred to as contextual clues) to reach the correct answer.
Query Expansion Using Contextual Clue Sampling with Language Models
(2) Relevance: Simply relying on a single generated context increases the risk of query drift, as the generated context could be semantically irrelevant or contain factual errors (Schütze et al., 2008). However, simply generating multiple contexts is prone to the hallucination problem -they can be unfaithful to the input or include false information (Tian et al., 2019;Maynez et al., 2020;Dziri et al., 2021). Thus, in this work, we wish to explore the question: How can we best generate a sufficiently rich set of contextual clues to answer a query?
Query Expansion Using Contextual Clue Sampling with Language Models
Our proposed solution ( Figure 1) overcomes these problems with two simple and efficient steps: Filtering and fusion. After sampling top-k outputs from the decoder of the fine-tuned language model, we first cluster these generated contextual clues based on their lexical distance. In each cluster, where highly similar contextual clues are grouped together, we only keep a single generated output with the highest generation probability. The filtering step effectively reduces potential factual errors and redundant close duplicates. The query is then individually augmented with each filtered contextual clue. We retrieve documents separately for every single augmented query. As the last step, all the documents are ranked together (fusion) with the generation probability from the integral contextual clue in the augmented query.
Query Expansion Using Contextual Clue Sampling with Language Models
We evaluate our approach on two established benchmarks: Natural Questions (Kwiatkowski et al., 2019) and TriviaQA (Lee et al., 2019). Our baseline model GAR (Mao et al., 2021) trails behind its dense retrieval counterpart DPR (Karpukhin et al., 2020) by a large margin when retrieving a small number of passages. We bridge this gap and outperform GAR by 3.1% and 2.9% on Top-5/Top-20 accuracy on the NQ dataset. Compared with DPR, our approach outperforms it by 0.6 and 1.0 points on Top-100 accuracy on the two datasets, while requiring 96% less index storage space. The accuracy can be further improved by 3.4% on the basis of DPR's performance when fusing the documents retrieved from DPR and our method together. Furthermore, our retrieval performance also successfully transfers to downstream question answering tasks, where our methods increase by 3.2% and 0.8% Exact Match score compared with the DPR and GAR retrieved documents.
Query Expansion Using Contextual Clue Sampling with Language Models
We employ a sequence-to-sequence model that takes the question as input and generates the contextual clues for the answer as target. As our model, we use BART-large (Lewis et al., 2020a), but note that the generator can be replaced with any other sequence-to-sequence model. Contextual clues are the sentences in a passage that contains the groundtruth answer to the question. These sentences are either extracted from the passage provided by the dataset (when available), or from the matching passage used as a reference for the retriever.
Query Expansion Using Contextual Clue Sampling with Language Models
At inference time, we first sample a diverse set of contextual clues from the fine-tuned model. Generally speaking, a single contextual clue can be broken into two main components; relational facts ("august 21, 2018") and contextual description ( "the game was released on"). Interestingly, we notice that many generations are identical in contextual descriptions, but inconsistent with the fact words (various dates, numbers or named entities). Previous works try to solve this inconsistency issue with an additional training loss (Elazar et al., 2021), adding a reasoning module (Nye et al., 2021), or by majority vote (Wang et al., 2022). Instead, we first cluster the contextual clues based on their edit distance. For most cases, contextual clues with the same contextual descriptions but varying relational facts, are grouped together in the same cluster. We then employ a simple filtering strategy for each cluster by keeping the top-ranked output with maximum generation probability, while discarding the rest outputs in the cluster. As a result, we could gather all possible reasoning paths to the answer, while reducing potential factual errors. As shown in Appendix A.2, the filtering strategy is crucial for the following retrieval step in terms of both retrieval efficiency (saves 70% for the retrieval process) and accuracy (consistently better than using full contextual clues).
Query Expansion Using Contextual Clue Sampling with Language Models
Defining the n generated and filtered contextual clues as
Query Expansion Using Contextual Clue Sampling with Language Models
by appending each individual context to it. Following GAR, we use BM25 as the backend for retrieval where it could be seen as a logical scoring model using a query encoder η q and passage encoder η d (Lin, 2022):
Query Expansion Using Contextual Clue Sampling with Language Models
where φ is a similarity function such as dot product or L2 distance. Note that we use c to denote the generated contexts and d to denote real passages in the corpus. To aggregate the retrieval results of different augmented queries, we perform retrieval individually for each augmented query and use the likelihood p(c i | q) of the generated context c i as the fusion weights. Therefore, the final retrieval score s f (q, d) for each question-passage pair is calculated as:
Query Expansion Using Contextual Clue Sampling with Language Models
(2) We finally re-sort the candidates according to the fusion scores and return the top-k passages for the next stage of answer extraction. 
Query Expansion Using Contextual Clue Sampling with Language Models
We finetune the BART-large model (Lewis et al., 2020a) for contextual clue generation. Given the question, we generate 100 candidate outputs from BART using beam search with beam size 100. We first cluster candidates using fuzzy string matching with the built-in difflib Python module. The similarity cutoff is set to 0.8 and any string pairs scoring less than the cutoff are not kept in the same group. On average, for each question, there are 24 contextual clues for NQ and 33 for TriviaQA after filtering. More details on processing the contextual clues are in the Appendix A.1 and A.2.
Query Expansion Using Contextual Clue Sampling with Language Models
For each contextual clue augmented query, we use the Pyserini  BM25 to retrieve top-1000 candidate passages. All the retrieved documents are then re-ranked according to Eq. (2). Further details are included in the Appendix A.3.
Query Expansion Using Contextual Clue Sampling with Language Models
Retriever Retrieval in open-domain QA is traditionally implemented with sparse vector space model BM25 (Robertson and Zaragoza, 2009), based on exact term matching. DPR (Karpukhin et al., 2020) implements retrieval by representing questions and passages as dense vectors. GAR (Mao et al., 2021) proposes to expand the query by adding relevant answers, the title of a passage and the sentence where the answer belongs. It also fuse the results from its own and from DPR (GAR+DPR). To make a fair comparison, we extend our generation target from the answer context only (Ours-single) to include both the answer and the passage title (Ours-multi). We also report the fusion results with DPR. SEAL (Bevilacqua et al., 2022) use BART model to generate ngrams then map to full passage with FM index.
Query Expansion Using Contextual Clue Sampling with Language Models
Reader DPR (Karpukhin et al., 2020) employs an extractive reader model based on BERT (Devlin et al., 2019) and predicts the answer span. RAG (Lewis et al., 2020b) combines the DPR dense retriever together with a BART answer generator, and jointly trains the two models end-to-end. FiD (Izacard and Grave, 2021) also uses DPR to retrieve relevant passages and the decoder attends over all the encoded passages to generate the final answer. For fair comparison, we evaluate the retrieval results of FiD, GAR, and SEAL on the same reader model, FiD-large which takes the question along with 100 top retrieved passages as input.
Query Expansion Using Contextual Clue Sampling with Language Models
We are interested in understanding the quality of the generated contextual clues. In Table 1, Top-1 returns the top sequence with the highest probability during beam search, while Full contexts contain all top-ranked 100 sequences. Filtered is the set of contextual clues after filtering. We report the ROUGE F-measure scores between the ground-truth and generated contextual clues on the NQ validation set. We also report the answer coverage rate, measured as the percentage of contextual clues that contain the answer.
Query Expansion Using Contextual Clue Sampling with Language Models
As shown in Table 1, rigorously increasing the number of generated candidates increases the ROUGE scores by at least 10% compared with only generating the top sequence, indicating it's more probable to capture the potential ground-truth answer context. The filtering strategy effectively reduces the size of candidate contexts while maintaining high coverage and diversity (less than 1% difference in ROUGE scores). Moreover, Full significantly increases the answer coverage rate by ∼ 17% compared with Top-1, suggesting that not only more semantics but also more fact words are captured in a larger sizes of candidates.
Query Expansion Using Contextual Clue Sampling with Language Models
In  the system implementation. The baseline models are reported in their open-sourced versions. We additionally compare with other memory-efficient neural retrieval models in the appendix A.4 and report retrieval time latency in the appendix A.5. Compared with other lexical retrieval models, our method significantly outperforms both GAR and SEAL, showing the effectiveness of extensively sampled contextual clues. We also find that Oursmulti consistently improves over Ours-single. We surmise that ground-truth answers serve as useful signals during retrieval and they are more likely to be covered when directly sampling answers. Most of the traditional lexical retrieval methods always trail behind dense retrieval by a large margin, as illustrated in Table 2. Surprisingly, our method even outperforms the DPR model by 0.6 and 1.0 points in terms of top-100 accuracy on two datasets, while requiring 96% less index storage space. For the purpose of pushing the limit of retrieval performance, we also show the accuracy of different lexical-based methods fused with DPR. Overall, our method fused with DPR achieves the highest accuracy across all baseline methods on both datasets. Table 3  ference between FiD and Ours is that FiD retrieves from DPR. Although ours-single is 0.2 points lower on Top-100 accuracy than that of DPR, the EM score of Ours-single is ∼ 2 points higher than FiD. It demonstrates even with relatively same top-k retrieval accuracy, that our approach could retrieve qualitatively better passages that are easier for the reader model to answer.
Query Expansion Using Contextual Clue Sampling with Language Models
We propose to narrow the lexical gap between the query and the documents by augmenting the query with extensively sampled contextual clues. To make sure the generated contextual clues are both diverse and relevant, we propose the strategy of context filtering and retrieval fusion. Our approach outperforms both the previous generation-based query expansion method and the dense retrieval counterpart with a much smaller index requirement. 
Query Expansion Using Contextual Clue Sampling with Language Models
Directly augmenting question with the full set of sampled contextual clues is a sub-optimal solution due to the following reasons: 1) Retrieval efficiency: After filtering, for each query, we only need to perform 70% less times of the retrieval. As a result, it's also saves the search space for the retrieval fusion step. 2) Retrieval accuracy: As shown in Table 4, the accuracy for the unfiltered contexts is consistently lower than that on the filtered contexts. We suppose it's due to removal of hallucinated facts contained in the contextual clues during filtering.
Query Expansion Using Contextual Clue Sampling with Language Models
We put all the passages belonging to the same question but different augmentation into a public pool after filtering duplicates. If a passage in the public pool does not appear in the top-1000 retrieval list of an augmented query, we use the minimum score of the augmented query's top-1000 list as the default score for the missing passage from that augmented query. We then average the retrieval score for each passage in the pool according to Eq.
Query Expansion Using Contextual Clue Sampling with Language Models
(2) and re-sort the order of the fused passages. For fair comparison with GAR (Mao et al., 2021), we additionally fine-tune an answer generation model and a title generation model. We perform the same fusion steps above for all three generation models, and we linearly interpolate their fusion results by searching the best weighting on the development set using Bayesian Optimization (Frazier, 2018).
Query Expansion Using Contextual Clue Sampling with Language Models
We compare our approach with other memory efficient neural retrieval models in Table 5.  show that the DPR could be furthered compressed to trade accuracy off against speed and storage. However, the accuracy of DPR could drop significantly if compressed to the same storage level of the lexical index. BPR (Yamada et al., 2021) integrates a learning-to-hash technique into DPR to represent the passage index using compact binary codes. The index size of BPR is slightly smaller than ours approach, but we achieve higher retrieval accuracy on both two datasets. We also include DrBoost (Lewis et al., 2021), a dense retrieval ensemble trained in stages. DrBoost outperforms ours approach on NQ dataset, while taking 6× times larger index size.
Query Expansion Using Contextual Clue Sampling with Language Models
Retrieval time latency is an important factor to consider for deployment. We list the latency time in Table 6. It is notable that the latency listed in the table is tested on CPU, since we use BM25 as our retrieval backend and it only requires CPU to run. Dense retrieval methods (e.g. DPR) normally is running on GPU devices, which only takes 456.9ms (Lewis et al., 2021) per query without other device specific speed-up techniques.    Table 6: Comparison on retrieval time latency, which is tested using 1 Intel Xeon CPU E5-2699 v4 @ 2.20GHz.
Generation-Augmented Retrieval for Open-Domain Question Answering
Open-domain question answering (OpenQA) aims to answer factoid questions without a pre-specified domain and has numerous real-world applications. In OpenQA, a large collection of documents (e.g., Wikipedia) are often used to seek information pertaining to the questions. One of the most common approaches uses a retriever-reader architecture (Chen et al., 2017), which first retrieves a small subset of documents using the question as the query and then reads the retrieved documents to extract (or generate) an answer. The retriever is crucial as it is infeasible to examine every piece of information in the entire document collection (e.g., millions of Wikipedia passages) and the retrieval accuracy bounds the performance of the (extractive) reader. * Work was done during internship at Microsoft Azure AI. 1 Our code is available at https://github.com/ morningmoni/GAR. Early OpenQA systems (Chen et al., 2017) use classic retrieval methods such as TF-IDF and BM25 with sparse representations. Sparse methods are lightweight and efficient, but unable to perform semantic matching and fail to retrieve relevant passages without lexical overlap. More recently, methods based on dense representations (Guu et al., 2020;Karpukhin et al., 2020) learn to embed queries and passages into a latent vector space, in which text similarity beyond lexical overlap can be measured. Dense retrieval methods can retrieve semantically relevant but lexically different passages and often achieve better performance than sparse methods. However, the dense models are more computationally expensive and suffer from information loss as they condense the entire text sequence into a fixed-size vector that does not guarantee exact matching (Luan et al., 2020).
Generation-Augmented Retrieval for Open-Domain Question Answering
There have been some recent studies on query reformulation with text generation for other retrieval tasks, which, for example, rewrite the queries to context-independent (Yu et al., 2020;Lin et al., 2020;Vakulenko et al., 2020) or well-formed  ones. However, these methods require either task-specific data (e.g., conversational contexts, ill-formed queries) or external resources such as paraphrase data (Zaiem and Sadat, 2019;) that cannot or do not transfer well to OpenQA. Also, some rely on timeconsuming training process like reinforcement learning (RL) (Nogueira and Cho, 2017; that is not efficient enough for OpenQA (more discussions in Sec. 2).
Generation-Augmented Retrieval for Open-Domain Question Answering
In this paper, we propose Generation-Augmented Retrieval (GAR), which augments a query through text generation of a pre-trained language model (PLM). Different from prior studies that reformulate queries, GAR does not require external resources or downstream feedback via RL as supervision, because it does not rewrite the query but expands it with heuristically discov-ered relevant contexts, which are fetched from PLMs and provide richer background information (Table 2). For example, by prompting a PLM to generate the title of a relevant passage given a query and appending the generated title to the query, it becomes easier to retrieve that relevant passage.
Generation-Augmented Retrieval for Open-Domain Question Answering
Intuitively, the generated contexts explicitly express the search intent not presented in the original query. As a result, GAR with sparse representations achieves comparable or even better performance than state-of-the-art approaches (Karpukhin et al., 2020;Guu et al., 2020) with dense representations of the original queries, while being more lightweight and efficient in terms of both training and inference (including the cost of the generation model) (Sec. 6.4).
Generation-Augmented Retrieval for Open-Domain Question Answering
Specifically, we expand the query (question) by adding relevant contexts as follows. We conduct seq2seq learning with the question as the input and various freely accessible in-domain contexts as the output such as the answer, the sentence where the answer belongs to, and the title of a passage that contains the answer. We then append the generated contexts to the question as the generationaugmented query for retrieval. We demonstrate that using multiple contexts from diverse generation targets is beneficial as fusing the retrieval results of different generation-augmented queries consistently yields better retrieval accuracy.
Generation-Augmented Retrieval for Open-Domain Question Answering
We conduct extensive experiments on the Natural Questions (NQ) (Kwiatkowski et al., 2019) and TriviaQA (Trivia) (Joshi et al., 2017) datasets. The results reveal four major advantages of GAR:
Generation-Augmented Retrieval for Open-Domain Question Answering
(1) GAR, combined with BM25, achieves significant gains over the same BM25 model that uses the original queries or existing unsupervised query expansion (QE) methods. (2) GAR with sparse representations (BM25) achieves comparable or even better performance than the current state-of-the-art retrieval methods, such as DPR (Karpukhin et al., 2020), that use dense representations. (3) Since GAR uses sparse representations to measure lexical overlap 2 , it is complementary to dense representations: by fusing the retrieval results of GAR and DPR, we obtain consistently better performance than either method used individually. (4) GAR outperforms DPR in the end-to-end QA performance (EM) when the same extractive reader is used: EM=41.8 (43.8 when combining with DPR) on NQ and 62.7 on Trivia, creating new state-ofthe-art results for extractive OpenQA. GAR also outperforms other retrieval methods under the generative setup when the same generative reader is used: EM=38.1 (45.3 when combining with DPR) on NQ and 62.2 on Trivia.
Generation-Augmented Retrieval for Open-Domain Question Answering
(1) We propose Generation-Augmented Retrieval (GAR), which augments queries with heuristically discovered relevant contexts through text generation without external supervision or time-consuming downstream feedback.
Generation-Augmented Retrieval for Open-Domain Question Answering
(2) We show that using generation-augmented queries achieves significantly better retrieval and QA results than using the original queries or existing unsupervised QE methods. (3) We show that GAR, combined with a simple BM25 model, achieves new state-of-the-art performance on two benchmark datasets in extractive OpenQA and competitive results in the generative setting.
Generation-Augmented Retrieval for Open-Domain Question Answering
OpenQA aims to answer factoid questions without pre-specified domains. We assume that a large collection of documents C (i.e., Wikipedia) are given as the resource to answer the questions and a retriever-reader architecture is used to tackle the task, where the retriever retrieves a small subset of the documents D ⊂ C and the reader reads the documents D to extract (or generate) an answer. Our goal is to improve the effectiveness and efficiency of the retriever and consequently improve the performance of the reader.
Generation-Augmented Retrieval for Open-Domain Question Answering
In GAR, queries are augmented with various heuristically discovered relevant contexts in order to retrieve more relevant passages in terms of both quantity and quality. For the task of OpenQA where the query is a question, we take the following three freely accessible contexts as the generation targets. We show in Sec. 6.2 that having multiple generation targets is helpful in that fusing their results consistently brings better retrieval accuracy. Context 1: The default target (answer). The default target is the label in the task of interest, which is the answer in OpenQA. The answer to the question is apparently useful for the retrieval of relevant passages that contain the answer itself. As shown in previous work (Roberts et al., 2020;Brown et al., 2020), PLMs are able to answer certain questions solely by taking the questions as input (i.e., closedbook QA). Instead of using the generated answers directly as in closed-book QA, GAR treats them as contexts of the question for retrieval. The advantage is that even if the generated answers are partially correct (or even incorrect), they may still benefit retrieval as long as they are relevant to the passages that contain the correct answers (e.g., cooccur with the correct answers). Context 2: Sentence containing the default target. The sentence in a passage that contains the answer is used as another generation target. Similar to using answers as the generation target, the generated sentences are still beneficial for retrieving relevant passages even if they do not contain the answers, as their semantics is highly related to the questions/answers (examples in Sec. 6.1). One can take the relevant sentences in the ground-truth passages (if any) or those in the positive passages of a retriever as the reference, depending on the trade-off between reference quality and diversity. Context 3: Title of passage containing the default target. One can also use the titles of relevant passages as the generation target if available. Specifically, we retrieve Wikipedia passages using BM25 with the question as the query, and take the page titles of positive passages that contain the answers as the generation target. We observe that the page titles of positive passages are often entity names of interest, and sometimes (but not always) the answers to the questions. Intuitively, if GAR learns which Wikipedia pages the question is related to, the queries augmented by the generated titles would naturally have a better chance of retrieving those relevant passages.
Generation-Augmented Retrieval for Open-Domain Question Answering
While it is likely that some of the generated query contexts involve unfaithful or nonfactual information due to hallucination in text generation (Mao et al., 2020) and introduce noise during re-trieval, they are beneficial rather than harmful overall, as our experiments show that GAR improve both retrieval and QA performance over BM25 significantly. Also, since we generate 3 different (complementary) query contexts and fuse their retrieval results, the distraction of hallucinated content is further alleviated.
Generation-Augmented Retrieval for Open-Domain Question Answering
After generating the contexts of a query, we append them to the query to form a generation-augmented query. 3 We observe that conducting retrieval with the generated contexts (e.g., answers) alone as queries instead of concatenation is ineffective because (1) some of the generated answers are rather irrelevant, and (2) a query consisting of the correct answer alone (without the question) may retrieve false positive passages with unrelated contexts that happen to contain the answer. Such low-quality passages may lead to potential issues in the following passage reading stage. If there are multiple query contexts, we conduct retrieval using queries with different generated contexts separately and then fuse their results. The performance of one-time retrieval with all the contexts appended is slightly but not significantly worse. For simplicity, we fuse the retrieval results in a straightforward way: an equal number of passages are taken from the top-retrieved passages of each source. One may also use weighted or more sophisticated fusion strategies such as reciprocal rank fusion (Cormack et al., 2009), the results of which are slightly better according to our experiments. 4 Next, one can use any off-the-shelf retriever for passage retrieval. Here, we use a simple BM25 model to demonstrate that GAR with sparse representations can already achieve comparable or better performance than state-of-the-art dense methods while being more lightweight and efficient (including the cost of the generation model), closing the gap between sparse and dense retrieval methods.
Generation-Augmented Retrieval for Open-Domain Question Answering
To further verify the effectiveness of GAR, we equip it with both extractive and generative readers for end-to-end QA evaluation. We follow the 3 One may create a title field during document indexing and conduct multi-field retrieval but here we append the titles to the questions as other query contexts for generalizability. 4 We use the fusion tools at https://github.com/ joaopalotti/trectools. reader design of the major baselines for a fair comparison, while virtually any existing QA reader can be used with GAR.
Generation-Augmented Retrieval for Open-Domain Question Answering
For the extractive setup, we largely follow the design of the extractive reader in DPR (Karpukhin et al., 2020). Let D = [d 1 , d 2 , ..., d k ] denote the list of retrieved passages with passage relevance scores D. Let S i = [s 1 , s 2 , ..., s N ] denote the top N text spans in passage d i ranked by span relevance scores S i . Briefly, the DPR reader uses BERT-base  for representation learning, where it estimates the passage relevance score D k for each retrieved passage d k based on the [CLS] tokens of all retrieved passages D, and assigns span relevance scores S i for each candidate span based on the representations of its start and end tokens. Finally, the span with the highest span relevance score from the passage with the highest passage relevance score is chosen as the answer. We refer the readers to Karpukhin et al. (2020) for more details. Passage-level Span Voting. Many extractive QA methods (Chen et al., 2017;Min et al., 2019b;Guu et al., 2020;Karpukhin et al., 2020) measure the probability of span extraction in different retrieved passages independently, despite that their collective signals may provide more evidence in determining the correct answer. We propose a simple yet effective passage-level span voting mechanism, which aggregates the predictions of the spans in the same surface form from different retrieved passages. Intuitively, if a text span is considered as the answer multiple times in different passages, it is more likely to be the correct answer. Specifically, GAR calculates a normalized score p(S i [j]) for the j-th span in passage d i during inference as follows:
Generation-Augmented Retrieval for Open-Domain Question Answering
. GAR then aggregates the scores of the spans with the same surface string among all the retrieved passages as the collective passage-level score. 5
Generation-Augmented Retrieval for Open-Domain Question Answering
For the generative setup, we use a seq2seq framework where the input is the concatenation of the question and top-retrieved passages and the target output is the desired answer. Such generative readers are adopted in recent methods such as SpanSe-qGen  and Longformer (Beltagy et al., 2020). Specifically, we use BART-large (Lewis et al., 2019) as the generative reader, which concatenates the question and top-retrieved passages up to its length limit (1,024 tokens, 7.8 passages on average). Generative GAR is directly comparable with SpanSeqGen ) that uses the retrieval results of DPR but not comparable with Fusion-in-Decoder (FID) (Izacard and Grave, 2020) since it encodes 100 passages rather than 1,024 tokens and involves more model parameters.
Generation-Augmented Retrieval for Open-Domain Question Answering
We conduct experiments on the open-domain version of two popular QA benchmarks: Natural Questions (NQ) (Kwiatkowski et al., 2019) and Trivi-aQA (Trivia) (Joshi et al., 2017). The statistics of the datasets are listed in Table 1. 
Generation-Augmented Retrieval for Open-Domain Question Answering
Following prior studies (Karpukhin et al., 2020), we use top-k retrieval accuracy to evaluate the performance of the retriever and the Exact Match (EM) score to measure the performance of the reader.
Generation-Augmented Retrieval for Open-Domain Question Answering
Top-k retrieval accuracy is defined as the proportion of questions for which the top-k retrieved passages contain at least one answer span, which is an upper bound of how many questions are "answerable" by an extractive reader.
Generation-Augmented Retrieval for Open-Domain Question Answering
Exact Match (EM) is the proportion of the predicted answer spans being exactly the same as (one of) the ground-truth answer(s), after string normalization such as article and punctuation removal.
Generation-Augmented Retrieval for Open-Domain Question Answering
For passage retrieval, we mainly compare with BM25 and DPR, which represent the most used state-of-the-art methods of sparse and dense retrieval for OpenQA, respectively. For query expansion, we re-emphasize that GAR is the first QE approach designed for OpenQA and most of the recent approaches are not applicable or efficient enough for OpenQA since they have task-specific objectives, require external supervision that was shown to transfer poorly to OpenQA, or take many days to train (Sec. 2). We thus compare with a classic unsupervised QE method RM3 (Abdul-Jaleel et al., 2004) that does not need external resources for a fair comparison. For passage reading, we compare with both extractive (Min et al., 2019a;Asai et al., 2019;Min et al., 2019b;Guu et al., 2020;Karpukhin et al., 2020) and generative (Brown et al., 2020;Roberts et al., 2020;Lewis et al., 2020a;Izacard and Grave, 2020) methods when equipping GAR with the corresponding reader.
Generation-Augmented Retrieval for Open-Domain Question Answering
Retriever. We use Anserini (Yang et al., 2017) for text retrieval of BM25 and GAR with its default parameters. We conduct grid search for the QE baseline RM3 (Abdul-Jaleel et al., 2004). Generator. We use BART-large (Lewis et al., 2019) to generate query contexts in GAR. When there are multiple desired targets (such as multiple answers or titles), we concatenate them with [SEP] tokens as the reference and remove the [SEP] tokens in the generation-augmented queries. For Trivia, in particular, we use the value field as the generation target of answer and observe better performance. We take the checkpoint with the best ROUGE-1 F1 score on the validation set, while observing that the retrieval accuracy of GAR is relatively stable to the checkpoint selection since we do not directly use the generated contexts but treat them as augmentation of queries for retrieval. Reader. Extractive GAR uses the reader of DPR with largely the same hyperparameters, which is initialized with BERT-base  and takes 100 (500) retrieved passages during training (inference). Generative GAR concatenates the question and top-10 retrieved passages, and takes at most 1,024 tokens as input. Greedy decoding is adopted for all generation models, which appears to perform similarly to (more expensive) beam search.
Generation-Augmented Retrieval for Open-Domain Question Answering
We evaluate the effectiveness of GAR in three stages: generation of query contexts (Sec. 6.1), retrieval of relevant passages (Sec. 6.2), and passage reading for OpenQA (Sec. 6.3). Ablation studies are mostly shown on the NQ dataset to understand the drawbacks of GAR since it achieves  better performance on Trivia.
Generation-Augmented Retrieval for Open-Domain Question Answering
Automatic Evaluation. To evaluate the quality of the generated query contexts, we first measure their lexical overlap with the ground-truth query contexts. As suggested by the nontrivial ROUGE scores in Table 3, GAR does learn to generate meaningful query contexts that could help the retrieval stage. We next measure the lexical overlap between the query and the ground-truth passage. The ROUGE-1/2/L F1 scores between the original query and ground-truth passage are 6.00/2.36/5.01, and those for the generation-augmented query are 7.05/2.84/5.62 (answer), 13.21/6.99/10.27 (sentence), 7.13/2.85/5.76 (title) on NQ, respectively. Such results further demonstrate that the generated query contexts significantly increase the word overlap between the queries and the positive passages, and thus are likely to improve retrieval results. 6
Generation-Augmented Retrieval for Open-Domain Question Answering
Context ROUGE-1 ROUGE-2 ROUGE-L  Table 3: ROUGE F1 scores of the generated query contexts on the validation set of the NQ dataset. 6 We use F1 instead of recall to avoid the unfair favor of (longer) generation-augmented query.
Generation-Augmented Retrieval for Open-Domain Question Answering
Case Studies. In Table 2, we show several examples of the generated query contexts and their ground-truth references. In the first example, the correct album release date appears in both the generated answer and the generated sentence, and the generated title is the same as the Wikipedia page title of the album. In the last two examples, the generated answers are wrong but fortunately, the generated sentences contain the correct answer and (or) other relevant information and the generated titles are highly related to the question as well, which shows that different query contexts are complementary to each other and the noise during query context generation is thus reduced.
Generation-Augmented Retrieval for Open-Domain Question Answering
Comparison w. the state-of-the-art. We next evaluate the effectiveness of GAR for retrieval. In Table 4, we show the top-k retrieval accuracy of BM25, BM25 with query expansion (+RM3) (Abdul-Jaleel et al., 2004), DPR (Karpukhin et al., 2020), GAR, and GAR +DPR.
Generation-Augmented Retrieval for Open-Domain Question Answering
On the NQ dataset, while BM25 clearly underperforms DPR regardless of the number of retrieved passages, the gap between GAR and DPR is significantly smaller and negligible when k ≥ 100. When k ≥ 500, GAR is slightly better than DPR despite that it simply uses BM25 for retrieval. In contrast, the classic QE method RM3, while showing Method NQ Trivia  Top-5 Top-20 Top-100 Top-500 Top-1000 Top-5 Top-20 Top-100 Top-500 Top- Table 4: Top-k retrieval accuracy on the test sets. All baselines are evaluated by ourselves and better than reported in Karpukhin et al. (2020). GAR helps BM25 to achieve comparable or better performance than DPR.
Generation-Augmented Retrieval for Open-Domain Question Answering
marginal improvement over the vanilla BM25, does not achieve comparable performance with GAR or DPR. By fusing the results of GAR and DPR in the same way as described in Sec. 3.3, we further obtain consistently higher performance than both methods, with top-100 accuracy 88.9% and top-1000 accuracy 93.2%. On the Trivia dataset, the results are even more encouraging -GAR achieves consistently better retrieval accuracy than DPR when k ≥ 5. On the other hand, the difference between BM25 and BM25 +RM3 is negligible, which suggests that naively considering top-ranked passages as relevant (i.e., pseudo relevance feedback) for QE does not always work for OpenQA. Results on more cutoffs of k can be found in App. A. Effectiveness of diverse query contexts. In Fig. 1, we show the performance of GAR when different query contexts are used to augment the queries. Although the individual performance when using each query context is somewhat similar, fusing their retrieved passages consistently leads to better performance, confirming that different generation-augmented queries are complementary to each other (recall examples in Table 2). Performance breakdown by question type. In Table 5, we show the top-100 accuracy of the compared retrieval methods per question type on the NQ test set. Again, GAR outperforms BM25 on all types of questions significantly and GAR +DPR achieves the best performance across the board, which further verifies the effectiveness of GAR.
Generation-Augmented Retrieval for Open-Domain Question Answering
Comparison w. the state-of-the-art. We show the comparison of end-to-end QA performance of extractive and generative methods in Table 6. Extractive GAR achieves state-of-the-art performance among extractive methods on both NQ and Trivia datasets, despite that it is more lightweight and computationally efficient. Generative GAR outper-   forms most of the generative methods on Trivia but does not perform as well on NQ, which is somewhat expected and consistent with the performance at the retrieval stage, as the generative reader only takes a few passages as input and GAR does not outperform dense retrieval methods on NQ when k is very small. However, combining GAR with DPR achieves significantly better performance than both methods or baselines that use DPR as input such as SpanSeqGen  and RAG (Lewis et al., 2020a). Also, GAR outperforms BM25 significantly under both extractive and generative se-
Generation-Augmented Retrieval for Open-Domain Question Answering
Extractive Hard EM (Min et al., 2019a) 28.1 50.9 -Path Retriever (Asai et al., 2019) 32.6 --ORQA  33.3 45.0 -Graph Retriever (Min et al., 2019b) 34.5 56.0 -REALM (Guu et al., 2020) 40.4 --DPR (Karpukhin et al., 2020) 41.5 57.9 -BM25 (ours) 37.7 60.1 -GAR 41.8 62.7 74.8 GAR +DPR 43.8 --Generative GPT-3 (Brown et al., 2020) 29.9 -71.2 T5 (Roberts et al., 2020) 36.6 60.5 -SpanSeqGen  42.2 --RAG (Lewis et al., 2020a) 44.5 56.1 68.0 FID (Izacard and Grave, 2020) 51  tups, which again shows the effectiveness of the generated query contexts, even if they are heuristically discovered without any external supervision. The best performing generative method FID (Izacard and Grave, 2020) is not directly comparable as it takes more (100) passages as input. As an indirect comparison, GAR performs better than FID when FID encodes 10 passages (cf. Fig. 2 in Izacard and Grave (2020)). Moreover, since FID relies on the retrieval results of DPR as well, we believe that it is a low-hanging fruit to replace its input with GAR or GAR +DPR and further boost the performance. 7 We also observe that, perhaps surprisingly, extractive BM25 performs reasonably well, especially on the Trivia dataset, outperforming many recent state-of-the-art methods. 8 Generative BM25 also performs competitively in our experiments. Model Generalizability. Recent studies (Lewis et al., 2020b) show that there are significant question and answer overlaps between the training and test sets of popular OpenQA datasets. Specifically, 60% to 70% test-time answers also appear in the training set and roughly 30% test-set questions have a near-duplicate paraphrase in the training set. Such observations suggest that many questions might have been answered by simple question or 7 This claim is later verified by the best systems in the NeurIPS 2020 EfficientQA competition (Min et al., 2021). 8 We find that taking 500 passages during reader inference instead of 100 as in Karpukhin et al. (2020) improves the performance of BM25 but not DPR. answer memorization. To further examine model generalizability, we study the per-category performance of different methods using the annotations in Lewis et al. (2020b).  As listed in Table 7, for the No Overlap category, GAR +DPR (E) outperforms DPR on the extractive setup and GAR +DPR (G) outperforms RAG on the generative setup, which indicates that better endto-end model generalizability can be achieved by adding GAR for retrieval. GAR +DPR also achieves the best EM under the Answer Overlap Only category. In addition, we observe that a closed-book BART model that only takes the question as input performs much worse than additionally taking topretrieved passages, i.e., GAR +DPR (G), especially on the questions that require generalizability. Notably, all methods perform significantly better on the Question Overlap category, which suggests that the high Total EM is mostly contributed by question memorization. That said, GAR +DPR appears to be less dependent on question memorization given its lower EM for this category. 9
Generation-Augmented Retrieval for Open-Domain Question Answering
GAR is efficient and scalable since it uses sparse representations for retrieval and does not involve time-consuming training process such as RL (Nogueira and Cho, 2017;. The only overhead of GAR is on the generation of query contexts and the retrieval with generationaugmented (thus longer) queries, whose computational complexity is significantly lower than other methods with comparable retrieval accuracy.
Generation-Augmented Retrieval for Open-Domain Question Answering
We use Nvidia V100 GPUs and Intel Xeon Platinum 8168 CPUs in our experiments. As listed in   Table 8, the training time of GAR is 3 to 6 hours on 1 GPU depending on the generation target. As a comparison, REALM (Guu et al., 2020) uses 64 TPUs to train for 200k steps during pre-training alone and DPR (Karpukhin et al., 2020) takes about 24 hours to train with 8 GPUs. To build the indices of Wikipedia passages, GAR only takes around 30 min with 35 CPUs, while DPR takes 8.8 hours on 8 GPUs to generate dense representations and another 8.5 hours to build the FAISS index (Johnson et al., 2017). For retrieval, GAR takes about 1 min to generate one query context with 1 GPU, 1 min to retrieve 1,000 passages for the NQ test set with answer/title-augmented queries and 2 min with sentence-augmented queries using 35 CPUs. In contrast, DPR takes about 30 min on 1 GPU.
Generation-Augmented Retrieval for Open-Domain Question Answering
In this work, we propose Generation-Augmented Retrieval and demonstrate that the relevant contexts generated by PLMs without external supervision can significantly enrich query semantics and improve retrieval accuracy. Remarkably, GAR with sparse representations performs similarly or better than state-of-the-art methods based on the dense representations of the original queries. GAR can also be easily combined with dense representations to produce even better results. Furthermore, GAR achieves state-of-the-art end-to-end performance on extractive OpenQA and competitive performance under the generative setup.
Generation-Augmented Retrieval for Open-Domain Question Answering
Potential improvements. There is still much space to explore and improve for GAR in future work. For query context generation, one can explore multi-task learning to further reduce computational cost and examine whether different contexts can mutually enhance each other when generated by the same generator. One may also sample multiple contexts instead of greedy decoding to enrich a query. For retrieval, one can adopt more advanced fusion techniques based on both the ranking and score of the passages. As the generator and retriever are largely independent now, it is also interesting to study how to jointly or iteratively optimize generation and retrieval such that the generator is aware of the retriever and generates query contexts more beneficial for the retrieval stage. Last but not least, it is very likely that better results can be obtained by more extensive hyper-parameter tuning. Applicability to other tasks. Beyond OpenQA, GAR also has great potentials for other tasks that involve text matching such as conversation utterance selection (Lowe et al., 2015;Dinan et al., 2020) or information retrieval (Nguyen et al., 2016;Craswell et al., 2020). The default generation target is always available for supervised tasks. For example, for conversation utterance selection one can use the reference utterance as the default target and then match the concatenation of the conversation history and the generated utterance with the provided utterance candidates. For article search, the default target could be (part of) the ground-truth article itself. Other generation targets are more taskspecific and can be designed as long as they can be fetched from the latent knowledge inside PLMs and are helpful for further text retrieval (matching). Note that by augmenting (expanding) the queries with heuristically discovered relevant contexts extracted from PLMs instead of reformulating them, GAR bypasses the need for external supervision to form the original-reformulated query pairs.
Generation-Augmented Retrieval for Open-Domain Question Answering
We show the detailed results of top-k retrieval accuracy of the compared methods in Figs. 2 and 3. GAR performs comparably or better than DPR when k ≥ 100 on NQ and k ≥ 5 on Trivia. We show in Table 9 the retrieval accuracy breakdown using the question-answer overlap categories. The most significant gap between BM25 and other methods is on the Question Overlap category, which coincides with the fact that BM25 is unable to conduct question paraphrasing (semantic matching). GAR helps BM25 to bridge the gap by providing the query contexts and even outperform DPR in this category. Moreover, GAR consistently improves over BM25 on other categories and GAR +DPR outperforms DPR as well.  Table 9: Top-100 retrieval accuracy by questionanswer overlap categories on the NQ test set.
Improving Query Representations for Dense Retrieval with Pseudo Relevance Feedback
Dense retrieval systems first encode queries and documents into a dense embedding space and then perform information retrieval by finding a query's nearest neighbors in the embedding space [16,20,27,33]. With the advancement of pre-trained language models [8,23], dedicated training strategies [16,33], and efficient nearest neighbor search [12,15], dense retrieval systems have shown effectiveness in a wide range of tasks, including web search [26], open domain question answering [18], and zero-shot IR [29]. Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s). CIKM '21, November 1-5, 2021, Virtual Event, QLD, Australia © 2021 Copyright held by the owner/author(s). ACM ISBN 978-1-4503-8446-9/21/11. https://doi.org /10.1145/3459637.3482124 Retrieval with dense, fully-learned representations has the potential to address some fundamental challenges in sparse retrieval. For example, vocabulary mismatch can be solved if the embeddings accurately capture the information need behind a query and maps it to relevant documents. However, decades of IR research demonstrates that inferring a user's search intent from a concise and often ambiguous search query is challenging [7]. Even with powerful pre-trained language models, it is unrealistic to expect an encoder to perfectly embed the underlying information need from a few query terms.
Improving Query Representations for Dense Retrieval with Pseudo Relevance Feedback
A common technique to improve query understanding in sparse retrieval systems is pseudo relevance feedback (PRF) [7,19,34], which uses the top retrieved documents from an initial search as additional information to enrich the query representation. Whether PRF information is used via query expansion [14,34] or query term reweighting [2], its efficacy has been consistently observed across various search scenarios, rendering PRF a standard practice in many sparse retrieval systems.
Improving Query Representations for Dense Retrieval with Pseudo Relevance Feedback
This work leverages PRF information to improve query representations in dense retrieval. Given the top retrieved documents from a dense retrieval model, e.g., ANCE [33], we build a PRF query encoder, ANCE-PRF, that uses a BERT encoder [8] to consume the query and the PRF documents to refine the query representation. ANCE-PRF is trained end-to-end using relevance labels and learns to optimize the query embeddings using the rich information from PRF documents. It reuses the document index from ANCE to avoid duplicating index storage.
Improving Query Representations for Dense Retrieval with Pseudo Relevance Feedback
In experiments on MS MARCO and TREC Deep Learning (DL) Track passage ranking benchmarks, ANCE-PRF is consistently more accurate than ANCE and several recent dense retrieval systems that use more sophisticated models and training strategies [24,36]. We also observe large improvements on DL-HARD [25] queries, a curated set to include complex search intents challenging for neural systems. To the best of our knowledge, ANCE-PRF is among the best performing first-stage retrieval systems on the highly competitive MARCO passage ranking leaderboard.
Improving Query Representations for Dense Retrieval with Pseudo Relevance Feedback
Our studies confirm that the advantages of ANCE-PRF reside in its ability to leverage the useful information from the PRF documents while ignoring the noise from irrelevant PRF documents. The PRF encoder allocates substantially more attention to terms from the relevant PRF documents, compared to those from the irrelevant documents. A case study shows that the encoder focuses more on PRF terms that are complementary to the query terms in representing search intents. These help ANCE-PRF learn better query embeddings that are closer to the relevant documents and improve the majority of testing queries. 1 
Improving Query Representations for Dense Retrieval with Pseudo Relevance Feedback
In dense retrieval systems, queries and documents are encoded by a dual-encoder, often BERT-based, into a shared embedding space [16,17,24,33]. Recent research in dense retrieval mainly focuses on improving the training strategies, especially the negative sampling part, including random sampling in batch [33], sampling from BM25 top negatives [9,20], sampling from an asynchronously [33] updated hard negatives index, constructing hard negatives using document index from an existing dense retrieval model [36], or reranking models [13,22]. Most dense retrieval systems encode a document using a constant number of embedding vectors [16,24,33], often one per document. There are also approaches using one vector per document token [17], similar to the interaction-based neural IR approaches [11,32]. In this work, we focus on models that only use one vector per document, whose retrieval efficiency is necessary for real production systems [33].
Improving Query Representations for Dense Retrieval with Pseudo Relevance Feedback
In recent research, PRF information has been leveraged by neural networks to combine feedback relevance scores [21], modify querydocument interaction using encoded feedback documents [1,3], or learn contextualized query-document interactions [35]. A parallel work [31] expands multi-vector query representations with feedback embeddings extracted using a clustering technique.
Improving Query Representations for Dense Retrieval with Pseudo Relevance Feedback
A typical dense retrieval system encodes query and document using a BERT-style encoder and then calculates the matching score ( , ) using simple similarity metrics:
Improving Query Representations for Dense Retrieval with Pseudo Relevance Feedback
where BERT and BERT respectively output their final layer [CLS] embeddings as the query and the document embeddings. Eq. (1) is fine-tuned using standard ranking losses and with various negative sampling techniques [16,24]. The initial retrieval system this work uses, ANCE, conducts negative sampling from an asynchronously updated document index [33]. ANCE-PRF leverages PRF documents retrieved by ANCE to enrich query representations. Given the top documents 1 , ..., from ANCE, ANCE-PRF trains a new PRF query encoder to output the query embedding prf :
Improving Query Representations for Dense Retrieval with Pseudo Relevance Feedback
It then conducts another retrieval with PRF embeddings:
Improving Query Representations for Dense Retrieval with Pseudo Relevance Feedback
The training uses the standard negative log-likelihood loss:
Improving Query Representations for Dense Retrieval with Pseudo Relevance Feedback
where + and − are embeddings of relevant and irrelevant documents. ANCE-PRF uses document embeddings from the initial dense retrieval model to avoid maintaining a separate document index for PRF. Therefore, only is newly learned. Eq. (4) trains the query encoder to identify the relevant PRF information using its Transformer attention. Specifically, the attention from the [CLS] embedding in the last layer of Eq. (2) to the th token of the input sequence is:
Improving Query Representations for Dense Retrieval with Pseudo Relevance Feedback
where and are the "query" vector and th input token's "key" vector of the th attention head [30]. Ideally, the PRF encoder should learn to yield ∑︁
Improving Query Representations for Dense Retrieval with Pseudo Relevance Feedback
where + are indexes of the meaningful tokens from the PRF documents, and − are those of the irrelevant PRF tokens. ANCE-PRF can be easily integrated with any dense retrieval models. With the document embeddings and index unchanged, the only computational overheads are one more query encoder forward pass (Eq. (2)) and one more nearest neighbor search (Eq. (3)), a minor addition to the dense retrieval process [33].
Improving Query Representations for Dense Retrieval with Pseudo Relevance Feedback
Next, we discuss the datasets, baselines, and implementation details.
Improving Query Representations for Dense Retrieval with Pseudo Relevance Feedback
Datasets. We use MS MARCO passage training data [26] which includes 530K training queries. We first evaluate on its dev set with 7k queries and also obtain the testing results by submitting to its leaderboard. MARCO's official metric is MRR@10.
Improving Query Representations for Dense Retrieval with Pseudo Relevance Feedback
We also evaluate the MARCO trained model on two additional evaluation benchmarks, TREC DL [5,6] and DL-HARD [25]. TREC DL [5,6] includes 43 labeled queries from 2019 and 54 from 2020 for the MARCO corpus. The official metric is NDCG@10 and Re-call@1K, the latter with label binarized at relevance point 2. Following Xiong et al. [33], we also report HOLE@10, the unjudged fraction of top 10 retrieved documents, to reflect the coverage of pooled labels on dense retrieval systems. DL-HARD [25] contains 50 queries from TREC DL that were curated to challenge neural systems in a prior TREC DL track. Its official metric is NDCG@10.
Improving Query Representations for Dense Retrieval with Pseudo Relevance Feedback
Baselines include BM25 [28], RM3 [14,19], a classical PRF framework in sparse retrieval. We also compare with several recent dense retrievers. ME-BERT [24] was trained with hard-negative mining [10], and is the only one that uses multi-vector document encoding. DE-BERT [24] is the single-vector version of ME-BERT. DPR [16] is trained with in-batch negatives. LTRe [36] generates hard negatives using document embeddings from an existing dense retrieval model. ANCE [33] uses hard negatives from asynchronously updated dense retrieval index using the latest model checkpoint.
Improving Query Representations for Dense Retrieval with Pseudo Relevance Feedback
Implementation Details. In training, we initialize query encoder from the ANCE FirstP model [33] 2 and kept the document embeddings from ANCE (and thus also the ANCE negative index) uncharged. All hyperparameters used in ANCE training are inherited in ANCE-PRF. All models are trained on two RTX 2080 Ti GPUs with per-GPU batch size 4 and gradient accumulation step 8 for 450K steps. We keep the model checkpoint with the best MRR@10 score on the MS MARCO dev set.
Improving Query Representations for Dense Retrieval with Pseudo Relevance Feedback
In inference, we first obtain ANCE top documents using Faiss IndexFlatIP and Eq. (1), feed them into the ANCE-PRF query encoder (Eq. (2)) for updated query embeddings, and run another Faiss search with Eq. (3) for final results.   Table 3: Results on DL-HARD [25]. We use the same symbols as in Table 1 for statistically significant improvements. 
Improving Query Representations for Dense Retrieval with Pseudo Relevance Feedback
In this section, we discuss our experimental results and studies. Table 1 includes overall retrieval accuracy on MS MARCO and TREC DL datasets. ANCE-PRF outperforms ANCE, its base retrieval system, on all datasets. On the challenging DL-HARD (Table 3), ANCE-PRF improves NDCG@10 By 9.3% over ANCE, indicating ANCE-PRF's advantage in queries challenging for neural systems. These results suggest that ANCE-PRF effectively leverages PRF information to produce better query embeddings. ANCE-PRF also helps retrieve relevant documents not recognized by ANCE, improving R@1K by about 5% on both TREC DL sets. ANCE-PRF rankings are significantly more precise than the sparse retrieval baselines with large margins across all datasets. RM3 achieves the best R@1K on both TREC DL sets, but its improvement is not as significant on DL-HARD.
Improving Query Representations for Dense Retrieval with Pseudo Relevance Feedback
ANCE-PRF also outperforms several strong dense retrieval baselines and produces the most accurate rankings on almost all datasets. While Luan et al. [24] discuss the theoretical benefits of higher dimensional dense retrieval as in ME-BERT, our empirical results show that a well-informed query encoder can achieve comparable results, while avoiding the computational and spatial overhead caused by using multiple vectors per document.
Improving Query Representations for Dense Retrieval with Pseudo Relevance Feedback
To understand the number of feedback documents ( ) needed for effective learning, we trained models using different and report the results in Table 2. We trained = 0 as a controlled experiment, which is equivalent to training ANCE for an extra 450K steps with fixed negatives.
Improving Query Representations for Dense Retrieval with Pseudo Relevance Feedback
Overall, we observe that models with > 0 are consistently better than ANCE ( = * ) and = 0, showing that ANCE-PRF effectively utilizes the given feedback relevance information. The Avg_Rel indicates that PRF documents at > 1 contain noisy relevance information, which is a known challenge for traditional PRF approaches [4]. Nevertheless, ANCE-PRF yields stable improvements over ANCE for = 1 to 5, demonstrating the model's robustness against noisy feedback from deeper .
Improving Query Representations for Dense Retrieval with Pseudo Relevance Feedback
In this group of experiments, we analyze the learned embeddings and attention in ANCE-PRF.
Improving Query Representations for Dense Retrieval with Pseudo Relevance Feedback
Embedding Space. Fig. 1(a) shows the distance during training between the ANCE-PRF query embedding and the embeddings of the original ANCE query, the relevant documents, and the irrelevant documents. We use MARCO dev in this study, in which about one out of the three PRF documents is relevant. In the embedding space, ANCE-PRF queries are closest to the original query and then the relevant documents, while further away from the irrelevant documents. ANCE-PRF's query embeddings effectively encode both the query and the feedback relevance information.
Improving Query Representations for Dense Retrieval with Pseudo Relevance Feedback
Learned Attention. We also analyze the learned attention on the relevant and the irrelevant PRF documents during training. We use TREC DL 2020 for this study as its dense relevance labels provide more stable observations. We calculate the average attention from the [CLS] token to each group ("relevant", "irrelevant", and "all") of PRF document (Eq. (5) & (6)), and plot them in Fig. 1(b)-1(d).  Fig. (a) shows the dot product between the ANCE-PRF query embedding and document embeddings at different training steps (x-axis). Fig. (b)-(d) are the cls_attention (y-axes) on "all", "relevant", and "irrelevant" feedback documents ranked at positions 1-3 in the initial retrieval. As training proceeds, ANCE-PRF pays more and more attention to the relevant PRF documents than the irrelevant ones, showing the effectiveness of its learning. Note that the original query always attracts the most attention from the PRF encoder, which is intuitive, as the majority of the search intent is to be determined by the query. The PRF information is to refine the query representation with extra information but not to invalidate it. ANCE-PRF wins on more queries and with larger margins. We also notice the PRF query encoder focuses more on terms that are complementary to the query. In the winning example, ANCE-PRF picks up terms explaining what "un fao" is and does not mistake "un" as "uno". On the other hand, ANCE-PRF may be misled by information appearing in multiple feedback documents. This is a known challenge for PRF because the correctness of information from multiple feedback documents is its core assumption [19]. In the losing example, "pattern their shotguns" occurs in multiple PRF documents, attracting too much attention to allow ANCE-PRF to make a better choice.
Improving Query Representations for Dense Retrieval with Pseudo Relevance Feedback
Existing dense retrievers learn query representations from short and ambiguous user queries, thus a query representation may not precisely reflect the underlying information need. ANCE-PRF addresses this problem with a new query encoder that learns better query representations from the original query and the top-ranked documents from a state-of-the-art dense retriever, ANCE.
Improving Query Representations for Dense Retrieval with Pseudo Relevance Feedback
Our experiments demonstrate that ANCE-PRF's effectiveness in refining query understanding and its robustness against noise from imperfect feedback. Our studies reveal that ANCE-PRF learns to distinguish between relevant and irrelevant documents. We show that ANCE-PRF successfully learns to identify relevance information with its attention mechanism. Its query encoder pays more attention to the relevant portion of the PRF documents, especially the PRF terms that complement the query terms in expressing the information need.
Improving Query Representations for Dense Retrieval with Pseudo Relevance Feedback
ANCE-PRF provides a straightforward way to leverage the PRF information in dense retrieval and can be used as a plug-in in embedding-based retrieval systems. We observe that simply leveraging the classic PRF information in the new neural-based retrieval regime leads to significant accuracy improvements, suggesting that more future research can be done in this direction.
Question Rewriting for Conversational Question Answering
Extending question answering systems to a conversational setting is a natural development that allows smooth interactions with a digital assistant [10]. This transition requires taking care of complex linguistic phenomena characteristic of a human dialogue, such as anaphoric expressions and ellipsis [29]. Such contextual dependencies refer back to the previous conversational turns, which are necessary to correctly interpret the question.
Question Rewriting for Conversational Question Answering
In this paper, we propose to address the task of conversational QA by decomposing it into two sub-tasks: (1) a question rewriting (QR) model that given previous conversational turns produces an explicit (contextually independent) question, which can then be used as an input to (2) a standard question answering (QA) model that can process explicit questions outside of the conversation context. Thereby, an explicit question serves as an intermediate output that connects QR with QA components. This setup offers a wide range of advantages in comparison with training a single conversational QA model end-to-end:
Question Rewriting for Conversational Question Answering
(1) Interpretability: Since contextual dependencies in a conversation are resolved explicitly by rewriting questions it becomes possible to separate errors that originate from incorrect context interpretation from the errors that stem from the questions answering phase. Our error analysis makes full use of this feature by investigating different sources of error and their correlation. (2) Reuse: Since QR produces an explicit question that can be answered by non-conversational QA models, we can leverage existing QA models and datasets without the need to re-design architectures to handle conversational context. Additionally, a QR model can be pre-trained once, and reused with a variety of standard QA models. (3) Modularity: QR setup makes it possible to query multiple remote 3rd-party APIs suitable for QA over different collections. This is a realistic scenario, in which information is distributed across a network of heterogenous nodes that do not share internal representations. Natural language provides a suitable communication protocol between these components [25].
Question Rewriting for Conversational Question Answering
Previous work on QR for conversational QA [8] evaluates QR performance using intrinsic quality of question rewrites. Text similarity metrics used for evaluation were already shown to be flawed on other generative tasks [17,34]. Consequently, there is no evidence that an improvement in a QR model, as evaluated by these metrics, will translate to better end-to-end performance in conversational QA. To the best of our knowledge, our work is the first to bridge this gap.
Question Rewriting for Conversational Question Answering
Two dimensions in which QA tasks vary are: the type of data source used to retrieve the answer (e.g., a paragraph, a document collection, or a knowledge graph); and the expected answer type (a text span, a ranked list of passages, or an entity). In this paper, we experiment with two variants of the QA task: retrieval QA, the task of finding an answer to a given natural-language question as a ranked list of relevant passages given a document collection; and extractive QA, the task of finding an answer to a given naturallanguage question as a text span within a given passage.
Question Rewriting for Conversational Question Answering
Though the two QA tasks are complementary to each other, in this paper we focus on the QR task and its ability to enable different types of QA models within a conversational setting. We use both retrieval and extractive QA tasks to examine the effect of the QR component on the end-to-end conversational QA performance.
Question Rewriting for Conversational Question Answering
Our main contribution is a systematic evaluation of the QR approach on the conversational QA task. We examine the effect of QR on two different QA architectures in parallel to demonstrate how the differences in the setup affect the QA performance and the types of errors. This analysis is performed to better understand the interaction between the components and their limitations which have implications on further integration.
Question Rewriting for Conversational Question Answering
The approach described in this paper outperforms all other published methods on both conversational QA datasets used for evaluation. Moreover, we show that QR performance metrics correctly predict the model which consistently performs better across both QA datasets. We also demonstrate how correlation in the performance metrics and question rewrites can be used as a tool for diagnosing QA models.
Question Rewriting for Conversational Question Answering
Conversational QA is an extension of the standard QA task that introduces contextual dependencies between the input question and the previous dialogue turns. Several datasets were recently proposed extending different QA tasks to a conversational setting including extractive [2,26], retrieval [5] and knowledge graph QA [3,12]. One common approach to conversational QA is to extend the input of a QA model by appending previous conversation turns [3,13,22]. Such approach, however falls short in case of retrieval QA, which requires a concise query as input to the candidate selection step, such as BM25 [21]. Results of the recent TREC CAsT track demonstrated that co-reference models are also not sufficient to resolve the missing context in the follow-up questions [5]. A considerable gap between the performance of automated rewriting approaches and manual human annotations call for new architectures that are capable of retrieving relevant answers from large text collection using conversational context.
Question Rewriting for Conversational Question Answering
In this paper we explore an alternative to training an end-to-end conversational QA model by separating it into QR and QA subtasks. We present the results of our experiments with the state-of-the-art text generation approach applied to resolve missing conversation context in the follow-up questions. QR is designed to handle conversation context and produce an equivalent question that no longer depends on the conversation context, i.e., with anaphoras and other contextual ambiguities resolved. The output of QR model can then be used by a standard QA model pretrained on non-conversational datasets, such as SQuAD [24], WikiQA [33] or TrecQA [30].
Question Rewriting for Conversational Question Answering
QR has been already shown effective for multi-turn chitchat and task-oriented dialogues [25,28]. These settings are dissimilar to ours since they use different evaluation criteria. Instead of measuring slot-matching and intent detection performance as in task-oriented dialogue systems or user engagement as in chit-chat, we evaluate open-domain QA. As we show in our experimental evaluation, this directly correlates with the QR performance. In this paper, we show that QR is well suited for open-domain conversational QA and demonstrate that QR helps to achieve superior performance on this task. The setup we propose also provides a convenient framework for measuring performance of the individual components.
Question Rewriting for Conversational Question Answering
Question rewriting for conversational QA was initially proposed by Elgohary et al., who released the CANARD dataset that contains human rewrites of questions from QuAC. However, the evaluation of the question rewriting approaches trained on this task was limited to the intrinsic metrics reported in the original paper [16]. No evaluation of the end-to-end performance using the generated question rewrites or the impact of the errors propagating from this stage to the question answering stage was reported to date. We close this research gap by training several question rewriting models and integrating them with the state-of-the-art question answering models based on different Transformer architectures. Finally, we show that the same QR model trained on the question rewrites for extractive QA can as well extend a standard retrieval QA model trained on the passage ranking task [19] to perform conversational QA on the TREC CAsT dataset.
Question Rewriting for Conversational Question Answering
QR allows to use existing retrieval and extractive QA models to conversational QA setting by introducing a question rewriting component that generates explicit questions interpretable outside of the conversation context (see Figure 1 for an illustrative example). To evaluate question rewriting for conversational QA on the endto-end task, we set up two different QA models independently from each other: one of them is designed for passage retrieval and one for answer extraction from a passage. This setup allows us to better examine performance of the individual components and analyze similarities and differences between retrieval and extractive models that can provide insights on the potential for a better integration of all three components together.
Question Rewriting for Conversational Question Answering
Given a conversation context C and a potentially implicit question Q, a question which may require the conversation context C to be fully interpretable, the task of a question rewriting (QR) model is to generate an explicit question Q ′ which is equivalent to Q under conversation context C and has the same correct answer A. We use a model for question rewriting, which employs a unidirectional Transformer decoder [23] for both encoding the input sequence and decoding the output sequence. The input to the  model is the question with previous conversation turns (we use 5 previous turns in our experiments) turned into token sequences separated with a special [SEP] token. The training objective is to predict the output tokens provided in the ground truth question rewrites produced by human annotators. The model is trained via teacher forcing approach, which is a standard technique for training language generation models, to predict every next token in the output sequence given all the preceding tokens. The loss is calculated as negative log-likelihood (cross-entropy) between the output distribution D ′ ∈ R |V | over the vocabulary V , and the onehot vector y Q ′ ∈ R |V | for the correct token from the ground truth: loss = −y Q ′ log D ′ . At training time the output sequence is shifted by one token and is used as input to predict all next tokens of the output sequence at once. At inference time, the model uses maximum likelihood estimation to select the next token from the final distribution D ′ (greedy decoding), as shown in Figure 2.
Question Rewriting for Conversational Question Answering
We further increase capacity of our generative model by learning to combine several separate distributions (D ′ 1 and D ′ 2 in Figure 2). The final distribution D ′ is then produced as a weighted sum of the intermediate distributions:
Question Rewriting for Conversational Question Answering
i ∈ R |V | we pass the last hidden state of the Transformer Decoder h ∈ R d through a separate linear layer for each intermediary distribution:
Question Rewriting for Conversational Question Answering
where W H i is the weight matrix and b H is the bias. For the weighting coefficients α i we use the matrix of input embeddings X ∈ R n×d , where n is the maximum sequence length and d is the embedding dimension, and the weights of the first attention head of the Transformer Decoder G ∈ R n×d put through a layer normalization function:
Question Rewriting for Conversational Question Answering
where all W are the weight matrices and b α i is the bias.
Question Rewriting for Conversational Question Answering
In the retrieval QA settings, the task is to produce a ranked list of text passages from a collection, ordered by their relevance to a given a natural language question [7,19]. We employ a state-of-the-art approach to retrieval QA, which consists of two phases: candidate selection and passage re-ranking. In the first phase, a traditional retrieval algorithm (BM25) is used to quickly sift through the indexed collection retrieving top-k passages ranked by relevance to
Question Rewriting for Conversational Question Answering
Layer Norm
Question Rewriting for Conversational Question Answering
Feed Forward
Question Rewriting for Conversational Question Answering
Where is … <sep> What is its GDP? <go> What is the GDP of Q' the input question Q ′ . In the second phase, a more computationallyexpensive model is used to re-rank all question-answer candidate pairs formed using the previously retrieved set of k passages. For reranking, we use a binary classification model that predicts whether the passage answers a question, i.e., the output of the model is the relevance score in the interval [0, 1]. The input to the re-ranking model is the concatenated question and passage with a separation token in between (see Figure 3 for the model overview). The model is initialized with weights learned from unsupervised pre-training on the language modeling (masked token prediction) task (BERT) [6]. During fine-tuning, the training objective is to reduce cross-entropy loss, using relevant passages and non-relevant passages from the top-k candidate passages.  Figure 3: Retrieval QA component includes two sequential phases: candidate selection (BM25) followed by passage reranking (Transformer Encoder).
Question Rewriting for Conversational Question Answering
The task of extractive QA is given a natural language question and a single passage find an answer as a contiguous text span within the given passage [24]. Our model for extractive QA consists of a Transformer-based bidirectional encoder (BERT) [6] and an output layer predicting the answer span. The input to the model is the sequence of tokens formed by concatenating a question and a passage separated with a special [SEP] token. The encoder layers are initialized with the weights of a Transformer model pre-trained on an unsupervised task (masked token prediction). The output of the Transformer encoder is a hidden vector T i for each token i of the input sequence. For fine-tuning the model on the extractive QA task, we add weight matrices W s , W e and biases b s , b e that produce two probability distributions over all the tokens of the given passage separately for the start (S) and end position (E) of the answer span. For each token i the output of the Transformer encoder T i is passed through a linear layer, followed by a softmax normalizing the output logits over all the tokens into probabilities:
Question Rewriting for Conversational Question Answering
The model is then trained to minimize cross-entropy between the predicted start/end positions (S i and E i ) and the correct ones from the ground truth (y S and y E are one-hot vectors indicating the correct start and end tokens of the answer span):
Question Rewriting for Conversational Question Answering
At inference time all possible answer spans from position i to position j, where j ≥ i, are scored by the sum of end and start positions' probabilities: S i + E j . The output of the model is the maximum scoring span (see Figure 4 for the model overview). 21% of the CANARD (QuAC) examples are Not Answerable (NA) by the provided passage. To enable our model to make No Answer predictions we prepend a special [CLS] token to the beginning of the input sequence. For all No Answer samples we set both the gold truth start and end positions spans to this token's position (0). Likewise, at inference time, predicting this special token is equivalent to a No Answer prediction for the given example. 
Question Rewriting for Conversational Question Answering
In the following subsections we describe the datasets used for training and evaluation, the set of metrics for each of the components, our baselines and details of the implementation.
Question Rewriting for Conversational Question Answering
We chose two conversational QA datasets for the evaluation of our approach: (1) CANARD, derived from Question Answering in Context (QuAC) for extractive conversational QA [2], and (2) TREC CAsT for retrieval conversational QA [5].
Question Rewriting for Conversational Question Answering
Following the setup of the TREC CAsT 2019, we use the MS MARCO Passage Ranking [19] and the TREC CAR [7] paragraph collections. After de-duplication, the MS MARCO collection contains 8.6M documents and the TREC CAR -29.8M documents. We evaluated on the test set with relevance judgements for 173 questions across 20 dialogues (topics).
Question Rewriting for Conversational Question Answering
CANARD [8] is built upon the QuAC dataset [2] by employing human annotators to rewrite original questions from QuAC dialogues into explicit questions. CANARD contains 40.5k pairs of question rewrites that can be matched to the original answers in QuAC. We use CANARD splits for training and evaluation. Each answer in QuAC is annotated with a Wikipedia passage from which it was extracted alongside the correct answer spans within this passage. We use the question rewrites provided in CANARD and passages with answer spans from QuAC. In our experiments, we refer to this joint dataset as CANARD for brevity. See Table 1 for the overview of the datasets. Since TREC CAsT is relatively small we use only CANARD for training QR. The same QR model trained on CANARD is evaluated on both CANARD and TREC CAsT. The model for retrieval QA is tuned on a sample from the MS MARCO passage ranking dataset, which includes relevance judgements for 12.8M query-passage pairs with 399k unique queries [20]. The model for extractive QA is pre-trained on MultiQA dataset, which contains 75k QA pairs from six standard QA benchmarks [9].
Question Rewriting for Conversational Question Answering
Mean average precision (MAP), mean reciprocal rank (MRR), normalized discounted cumulative gain (NDCG@3) and precision on the top-passage (P@1) evaluate quality of passage ranking. 1000 documents are evaluated per query with a relevance judgement value cut-off level of 2. We use F1 and Exact Match (EM) for extractive QA, which measure word token overlap between the predicted answer span and the ground truth. We also report accuracy for questions without answers in the given passage (NA Acc).
Question Rewriting for Conversational Question Answering
Our analysis showed that ROUGE recall calculated for unigrams (ROUGE-1 recall) correlates with the human judgement of the question rewriting performance (Pearson 0.69), which we adopt for our experiments as well. ROUGE [15] is a standard metric estimating lexical overlap, which is often used in text summarization and other text generation tasks. We also calculate question similarity scores using the Universal Sentence Encoder (USE) model [1] (Pearson 0.71).
Question Rewriting for Conversational Question Answering
The baselines were designed to challenge the need for a separate QR component by incorporating previous turns as direct input to custom QA components. Manual rewrites by human annotators provide the upper-bound performance for a QR approach and allows for an ablation study of the down-stream QA components.
Question Rewriting for Conversational Question Answering
Original. Original questions from the conversational QA datasets without any question rewriting.
Question Rewriting for Conversational Question Answering
Original + k-DT.. Our baseline approach for extractive QA prepends the previous k questions to the original question to compensate for the missing context. The questions are separated with a special token and used as input to the Transformer model. We report the results for k = {1, 2, 3}.
Question Rewriting for Conversational Question Answering
Original + k-DT*. Since in the first candidate selection phase we use BM25 retrieval function which operates on a bag-of-words representation, we modify the baseline approach for retrieval QA as follows. We select keywords from k prior conversation turns (not including current turn) based on their inverse document frequency (IDF) scores and append them to the original question of the current turn. We use the keyword-augmented query as the search query for Anserini [32], a Lucene toolkit for replicable information retrieval research, and if we use BERT re-ranking we concatenate the keyword-augmented query with the passages retrieved from the keyword-augmented query. We use the keywords with IDF scores above the threshold of 0.0001, which was selected based on a 1 million document sample of the MS MARCO corpus.
Question Rewriting for Conversational Question Answering
Human. To provide an upper bound (skyline), we evaluate all our models on the question rewrites manually produced by human annotators.
Question Rewriting for Conversational Question Answering
In addition to the baselines described above, we chose several alternative models for question rewriting of the conversational context: (1) co-reference resolution as in the TREC CAsT challenge; (2) PointerGenerator proposed in the related work for CANARD but not evaluated on the end-to-end conversational QA task [8]; (3) CopyTransformer extension of the PointerGenerator model that replaces the bi-LSTM encoder-decoder architecture with the same Transformer Decoder model as in ActionGenerator. All models, except co-reference, were trained on the train split of the CANARD dataset. Question rewrites are generated turn by turn for each dialogue recursively using already generated rewrites as previous turns. This is the same setup as in the TREC CAsT evaluation.
Question Rewriting for Conversational Question Answering
Co-reference. Anaphoric expressions in original questions are replaced with their antecedents from the previous dialogue turns. Co-reference dependencies are detected using a publicly available neural co-reference resolution model that was trained on the OntoNotes corpus [14]. 1 PointerGenerator. A sequence-to-sequence model for text generation with bi-LSTM encoder and a pointer-generator decoder [27].
Question Rewriting for Conversational Question Answering
CopyTransformer. The Transformer decoder, which, similar to pointer-generator model, uses one of the attention heads as a pointer [11]. The model is initialized with the weights of a pretrained GPT2 model [23,31] (Medium-sized GPT-2 English model: 24-layer, 1024-hidden, 16-heads, 345M parameters) and then finetuned on the question rewriting task. 2 ActionGenerator. The Transformer-based model described in Section 3.1. ActionGenerator is initialized with the weights of the pre-trained GPT2 model, same as in CopyTransformer.
Question Rewriting for Conversational Question Answering
Our retrieval QA approach is implemented as proposed in [20] using Anserini for the candidate selection phase with BM25 (top-1000 passages) and BERT LARG E for the passage re-ranking phase (Anserini + BERT). Both components were fine-tuned only on the MS MARCO dataset (k 1 = 0.82, b = 0.68). 3 We train several models for extractive QA on different variants of the training set based on the CANARD training set [8]. All models are first initialized with the weights of the BERT LARG E model pretrained using the whole word masking [6]. CANARD-H.. To accommodate input of the question rewriting models, we train a QA model that takes human rewritten question from the CANARD dataset as input without any additional conversation context, i.e., as in the standard QA task.
Question Rewriting for Conversational Question Answering
MultiQA → CANARD-H.. Since the setup with rewritten questions does not differ from the standard QA task, we experiment with pretraining the extractive QA model on the MultiQA dataset with explicit questions [9], using parameter choices introduced by Longpre et al. [18]. We further fine-tune this model on the target CANARD dataset to adopt the model to a different type of QA samples in CANARD (see Figure 6).
Question Rewriting for Conversational Question Answering
Our proposed approach, using question rewriting for conversational QA, consistently outperforms the baselines that use previous dialogue turns, in both retrieval and extractive QA tasks. The Point-erGenerator network that was previously proposed for the QR task in [8] is the weakest rewriting model according to the end-to-end QA results (MAP 0.100 F1 57.37). This result was not apparent from the QR evaluation metric reported in Table 2.
Question Rewriting for Conversational Question Answering
Passage re-ranking with BERT always improves ranking results (almost a two-fold increase in MAP, see Table 3). The best question rewriting model, ActionGenerator, when used together with BERT re-ranking and Anserini retrieval, sets the new state of the art on the TREC CAsT dataset with a 28% improvement in MAP and 21% in NDCG@3 over the top automatic run [4]. Keyword-based baselines (Original + k-DT*) prove to be very strong outperforming both Co-reference and PointerGenerator models on all three performance metrics. Both MRR and NDCG@3 are increasing with the number of turns used for sampling keywords, while MAP is slightly decreasing, which indicates that it brings more relevant results at the very top of the rank but non-relevant results also receive higher scores. In contrast, the baseline results for Anserini + BERT model indicate that the re-ranking performance for all metrics decreases if the keywords from more than 2 previous turns are added to the original question. Similarly, we observe a performance peak of the F1 measure at k = 2 in the extractive QA settings (see Table 4).  Training an extractive QA model on rewritten questions improves the performance even when the original questions are used as input at inference time (CANARD-H in Table 4). The tendency of the extractive model to hit correct answers even with incomplete information from ambiguous questions is also observable in Figure 7.  but not when it was explicitly reformulated by a human annotator (green and purple in the right plot) is almost absent in the retrieval model analysis (see also the sparse rows in Table 3). Pre-training on MultiQA improves performance of the extractive QA model. The style of questions in CANARD dataset is rather different from other QA tasks and the Figure 6 shows that even a little training data for CANARD helps to immediately boost the performance of a pre-trained model.
Question Rewriting for Conversational Question Answering
The precision-recall trade-off curve in Figure 5 shows that question rewriting performance is close to the performance achieved by manually rewriting implicit questions. Precision is decreasing rapidly and is below 0.1 when optimised for full recall. At the same time, the performance results of the extractive QA suggest that the model can not discriminate well the passages that do not have an answer to the question (71% accuracy on the human rewrites). Since the proportion of irrelevant passages produced by the retrieval component is twice the proportion of non-answerable questions in CANARD, the error rate is expected to increase when these components are combined. 
Question Rewriting for Conversational Question Answering
For every answer in the datasets we have three question formulations: (1) an original, possibly implicit, question (Original), and question rewrites that were produced either (2) by one of our QR models (we take the rewrites generated by the best QR model -ActionGenerator (QR) or (3) by a human annotator (Human). Table 5 allows us to better analyze the types of errors by grouping QA samples based on the performance of the retrieval QA component when given each of the specified question formulations. For example, the first row of the table indicates the number of samples where neither generated rewrite nor human rewrite were able to solicit the correct answer when measured by P@1, i.e., at the first position in the ranking. Therefore, all of these 49 error cases can be attributed to the retrieval QA component.
Question Rewriting for Conversational Question Answering
There are two anecdotal cases where our QR component was able to generate rewrites that helped to produce better ranking than the human-written questions. The first example shows that the re-ranking model does not handle paraphrases well. Original question: "What are good sources in food?", human rewrite: "What are good sources of melatonin in food?", model rewrite: "What are good sources in food for melatonin". In the second example the human annotator and our model chose different context to disambiguate the original question. Original question: "What about environmental factors?", human rewrite: "What about environmental factors during the Bronze Age collapse?", model rewrite: "What about environmental factors that lead to led to a breakdown of trade". Even though both model rewrites are not grammatically correct they solicited correct top-answers, while the human rewrites failed, which indicate flaws in the QA model performance.
Question Rewriting for Conversational Question Answering
More generally, assuming that humans always make correct question rewrites, we can attribute all cases in which they did not result in a correct answer as errors of the QA component (rows 1-4 in Tables 5-6). The next two rows 5-6 show the cases, where human rewrites succeeded but the model rewrites failed, which we consider   to be a likely error of the QR component. The last two rows are true positives for our model, where the last row combines cases where the original question was just copied without rewriting (numbers in brackets) and other cases when rewriting was not required. The majority of errors stem from the QA model: 29% of the test samples for retrieval and 55% for extractive estimated for P@1 and F1, comparing to 11% and 5% for QR respectively. It is a rough estimate since we can not tell whether the cases failing QA did not fail QR as well. Another interesting observation is that 10% of the questions in TREC CAsT were rewriten by human annotators that did not need rewriting to retrieve the correct answer. For CANARD the majority of questions (62%) can be correctly answered without question rewriting even when the questions are ambiguous.
Question Rewriting for Conversational Question Answering
Correlation between QR and QA metrics. To discover the interaction between QR and QA metrics, we discarded all questions that do not lead to correct answers for human rewrites (top 4 rows) and then measure correlation between ROUGE scores and P@1. There is a strong correlation for ROUGE = 1, i.e., when the generated rewrite is very close to the human one, but when ROUGE < 1 the answer is less predictable: even for rewrites that have a relatively small lexical overlap with the ground-truth (ROUGE ≤ 0.4) it is possible to retrieve a correct answer, and vice versa.
Question Rewriting for Conversational Question Answering
We explore this effect further by comparing differences in answers produced for human and model rewrites of the same question irrespective of the correct answer. Figure 8 demonstrates strong correlation between question similarity, as measured by ROUGE, and answer set similarity, as measured by recall of the answers produced for the human rewrites. Recall correlates with ROUGE more than precision. Points in the bottom right of this plot show sensitivity of the QA component, where similar questions lead to different answer rankings. The data points that are close the top center area indicate weakness of the QR metric as a proxy for the QA results: often questions do not have to be the same as the ground truth questions to solicit the same answers. The blank area in the top-left from the diagonal shows that a lexical overlap is required to produce the same answer set, which is likely due to the candidate filtering phase based on the bag-of-word representation matching. ROUGE and Jaccard in extractive QA show only weak correlation (Pearson 0.31). The extractive model is very sensitive to slight input perturbations and provides the same answer to very distinct questions. Correlation of the QA performance with USE metric is less than with ROUGE for both models and the outliers do not correspond to paraphrases. QA sensitivity. We showed that QA results can provide an estimate of the question similarity. However, this property is directly dependent on the ability of the QA component to match equivalent questions to the same answer. Alternative question rewrites allow us to evaluate robustness and consistency of the QA models. Our analysis indicates that small perturbations of the input question, such as anaphora resolution, has a considerable impact on the answer ranking, e.g., the pair of the original question: "Who are the Hamilton Electors and what were they trying to do?", and the human rewrite: "Who are the Hamilton Electors and what were the Hamilton Electors trying to do?" produce ROUGE = 1 but R@1000 = 0.33. We identified many cases in which inability of the QR component to generate apostrophes jeopardized relevance matching.
Question Rewriting for Conversational Question Answering
These results demonstrate the complexity of error estimation for this task. When asked for a human judgment of the quality of the generated rewrites independent from the QA results, little deviations may not seem important, but from the pragmatic point of view they have an impact on the overall performance of the end-toend system. However, such inconsistencies (typos and paraphrases) should be handled by the QA component, since they do not originate from the context interpretation errors but are inherent in the standalone questions as well.
Question Rewriting for Conversational Question Answering
QR completeness. The level of detail required to answer a particular question is often not apparent and depends on the collection. There are cases in which original questions without rewriting were already sufficient to retrieve the correct answers from the passage collection (see last row of the Table 5). For example, original question: "What is the functionalist theory?", human rewrite: "What is the functionalist theory in sociology?" However, in another question from the same dialogue, omitting the same word from the rewrite leads to retrieval of an irrelevant passage, since there are multiple alternative answers. This class of errors also corresponds to the variance evident from the Figure 8, since a one-word difference between two questions may have a very little effect on the answer ranking as well as a dramatic change in the question interpretation. This effect, however, interacts with the size and diversity of the collection content. Some of the questions were correctly answered even with underspecified questions, e.g., original question: "What are some ways to avoid injury?", human rewrite: "What are some ways to avoid sports injuries?", because of the collection bias. The idea behind the question rewriting approach is to learn patterns that correct for such semantic differences independent from the collection content, similar to how humans resolve such cases, based on their knowledge of language and the world.
Question Rewriting for Conversational Question Answering
Question rewriting (QR) is a challenging task that attempts to learn linguistic patterns that signal and resolve ambiguity in question formulation. The core idea to develop QR as a separate component is that the context understanding and question formulation are independent from the knowledge collection process, i.e. the way humans approach this by relying on their linguistic and world knowledge. Our experimental results show that human intuition about correct rewrites is sub-optimal but it allows to establish a mechanism that performs well across different datasets. We showed in an end-to-end evaluation that question rewriting is an effective method to extend existing question answering (QA) approaches to conversational settings, which establishes new state of the art on both retrieval and extractive conversational QA tasks on the TREC CAsT and CANARD datasets. The advantage of our approach is the ability to reuse existing non-conversational QA models for passage retrieval and extractive QA subtasks out-of-the-box. An end-to-end conversational QA system requires integration of all three components. Here, we focused exclusively on the integration of the first component responsible for the question formulation, and analyzed its effect on the other two components. By producing explicit representations of the question interpretation, question rewriting makes conversational QA results more explicable and the contribution of the individual components more transparent, which enable us to conduct a thorough performance analysis. Our analysis demonstrates sensitivity of both QA models to differences in question formulation, which calls for more adequate evaluation setups that are able to reflect model robustness.
Question Rewriting for Conversational Question Answering
We compared two QA models side by side and discovered major differences in their interaction with the QR component. The role of question rewriting is especially prominent in the case of retrieval QA where the candidate answer space is so large that any ambiguity in question formulation results in a very different answer. In contrast, the extractive QA model setup is optimized for recall and tends to produce answers for questions that are ambiguous or unanswerable given the passage. Since the accuracy of the retrieval component is twice as low as the proportion of answerable questions in the CANARD (QuAC) dataset, integration of the two QA components is likely to result in considerable error propagation from passage retrieval to the answer extraction phase. An important direction for future work is to design architectures that will be able to mitigate this negative effect.
Question Rewriting for Conversational Question Answering
It is important to note that the QR-QA architecture, which we employed to account for previous conversation turns, is generic enough to incorporate other types of context, which may incorporate a user model or an environment context obtained from multi-modal data (deictic reference). Experimental evaluation of QR-QA performance augmented with such auxiliary inputs is a promising direction for future work.
Query Reformulation using Query History for Passage Retrieval in Conversational Search
In recent years, the rise of machine learning techniques has accelerated the development of conversational agents such as smart speakers and digital personal assistants [38]. Therefore, conversational information seeking is both a timely and an important research area in which we seek to boost the ability of conversational assistant systems to satisfy users with information needs [15].
Query Reformulation using Query History for Passage Retrieval in Conversational Search
Understanding users' conversations is a challenging part of a generic conversational assistant system. The information needs of users in such a scenario-conversational question answering (ConvQA)-are typically colloquially expressed and contextually dependent. To make such a challenging task tractable, environmental settings are generally controlled to answer questions within a relevant document, under which several studies [24,37] have conducted conversational context modeling leading to progress in ConvQA benchmarks such as CoQA and QuAC [10,41]. * Contributed equally. What is a physician's assistant? 2
Query Reformulation using Query History for Passage Retrieval in Conversational Search
What are the educational requirements required to become one? 3
Query Reformulation using Query History for Passage Retrieval in Conversational Search
What does it cost? 4
Query Reformulation using Query History for Passage Retrieval in Conversational Search
What's the average starting salary in the UK? 5 What about in the US? 6 What school subjects are needed to become a registered nurse? 7
Query Reformulation using Query History for Passage Retrieval in Conversational Search
What is the PA average salary vs an RN? 8 What the difference between a PA and a nurse practitioner? 9
Query Reformulation using Query History for Passage Retrieval in Conversational Search
Do NPs or PAs make more? 10 Is a PA above a NP? 11 What is the fastest way to become a NP? 12 How much longer does it take to become a doctor after being an NP?
Query Reformulation using Query History for Passage Retrieval in Conversational Search
Another underlying scenario is open-domain ConvQA, in which answers are sought for given open-domain questions from one or more knowledge bases. This scenario makes the problem much more complex than those considered in previous ConvQA studies and significantly deteriorates the performance of QA systems [16]. In particular, for such a scenario, information retrieval (IR) in conversational search is naturally involved [8,38]. As a result, to facilitate generic open-domain ConvQA systems, conversational passage retrieval (ConvPR) plays a vital role in the whole systems, for which, however, little has been done in the literature.
Query Reformulation using Query History for Passage Retrieval in Conversational Search
There are currently two main challenges facing ConvPR: limited labeled data and ambiguous queries. First, even though neural networks have brought fruitful progress in natural language processing (NLP) [17,29,57] and IR [3,33], ConvPR remains challenging due to the limited amount of labeled data. To our best knowledge, at the current time, there is no reasonably-sized training dataset for ConvPR in contrast to other ad-hoc passage retrieval tasks, e.g., MS MARCO, TREC CAR [7,19]. Specifically, the conversational assistant track (CAsT) of the text retrieval conference (TREC) 2019 [25] only provides a total of 108 conversational user utterances in 13 topics with relevance judgments for model training, whereas MS MARCO and TREC CAR training sets contain 530k and 3M queries with relevant passages, respectively.
Query Reformulation using Query History for Passage Retrieval in Conversational Search
Second, ConvPR queries are usually ambiguous, due to commonly faced coreference and omission problems; therefore, it also requires tracking and understanding the information needs behind conversational user utterances, as users may ask questions referring to their past dialogues [38]. Table 1 shows an example of conversational user utterances in the CAsT training set. Observe that the second utterance contains one, denoting physician's assistant, showing the importance of coreference resolution in ConvPR. Also, the fifth utterance omits the contexts (average starting salary of physician's assistant) from previous dialogues, demonstrating the necessity to account for omissions in ConvPR. Clearly, without appropriate processing, raw user utterances are ambiguous queries for traditional IR systems, since it is hard to interpret them without context. The resultant need for tracking and understanding further increases the complexity beyond ad-hoc IR problems.
Query Reformulation using Query History for Passage Retrieval in Conversational Search
To address these two challenges, we propose a conversational multi-stage retrieval system with a conversational query reformulation (CQR) module. We build on competitive baselines in an existing IR toolkit for ad-hoc retrieval and take advantage of existing work on BERT-based re-ranking. To be clear, our primary focus is on conversational tracking and understanding. Inspired by research on query expansion and conversational query understanding, we propose two simple yet effective CQR approaches to address this problem, both of which inject context information into ambiguous user utterances for downstream IR systems.
Query Reformulation using Query History for Passage Retrieval in Conversational Search
Specifically, the first approach-historical query expansion (HQE)is a non-parametric model that applies query expansion techniques using context information. Neural transfer reformulation (NTR), the other approach, transfers knowledge of conversational query understanding by training a neural model to mimic how humans rewrite questions in a conversational context.
Query Reformulation using Query History for Passage Retrieval in Conversational Search
The contributions of this work are summarized as follows:
Query Reformulation using Query History for Passage Retrieval in Conversational Search
• We demonstrate the effectiveness of two conversational query reformulation approaches (HQE and NTR) stacked on top of a widely-used multi-stage search architecture. • We conduct a detailed analyses of the HQE and NTR approaches quantitatively and qualitatively, explaining their pros and cons. One variant of HQE was the best automatic submission to TREC 2019 CAsT, and NTR further improves it by 18% in NDCG@3. Since our work only exploits CAsT training data for hyperparameter tuning, it provides strong baselines for future studies.
Query Reformulation using Query History for Passage Retrieval in Conversational Search
In sum, this work demonstrates how to tackle ConvPR with limited training data, based on which we build simple but effective baselines for future IR research in conversational search.
Query Reformulation using Query History for Passage Retrieval in Conversational Search
Open-domain question answering (QA) systems return answers in response to user questions, both posed in natural language, from a broad range of domains [47,53]. An automatic open-domain QA system is often constructed with a pipeline: an IR model followed by a reading comprehension (RC) model to infer the answer from retrieved documents [8].
Query Reformulation using Query History for Passage Retrieval in Conversational Search
Despite the progress of QA system on RC models [17,29,45,57], few studies address retrieval [8,16,53]. Most QA research [26,40,50,56,59], including conversational QA studies [10,41], focuses on a restricted version of the open-domain QA problem posed in [8,18,20]: returning answers from a finite set of relevant documents-a relevant article [40,56] or multi-hop hyperlinked documents [58]. Our work instead addresses a research problem regarding retrieval, especially in the context of open-domain conversational questions, posed in natural language, in a sequence.
Query Reformulation using Query History for Passage Retrieval in Conversational Search
Multi-stage retrieval systems are comprised of a candidate generation process followed by one or more re-ranking stages to strike a balance between efficiency and effectiveness [6,11,49]. Multi-stage retrieval systems research includes feature extraction efficiency [5], dynamic cutoff depth [14], shard prediction [31], and joint cascade ranking optimization [3,22,33,35,51]. The foundation of our work is built on a competitive cascade pipeline proposed by [33] and [3]: BM25 candidate generation with BERT re-ranking, the effectiveness of which has been proved in representative IR datasets: Robust04, TREC CAR, and MS MARCO [3,7,19].
Query Reformulation using Query History for Passage Retrieval in Conversational Search
Query reformulation (QR) has proven effective in IR. For example, Xu and Croft [54] expand a query with terms from retrieved documents; Nogueira and Cho [32] further improve IR systems using reinforcement learning to reformulate the query. Note that although many previous studies focus on improving the performance of ad-hoc queries, we emphasize QR in a conversational context. Among QR studies, the most relevant works to ours are [21,42], both of which demonstrate the feasibility of deep learning for reformulating conversational queries. However, they only examine one facet of performance in terms of question-in-context rewriting. In this work, though, we practically apply and formally analyze a query-expansion-based method as well as a transfer learning method [23,36] under a full conversational IR pipeline.
Query Reformulation using Query History for Passage Retrieval in Conversational Search
Conversational search [38] covers a broad range of perspectives to facilitate an IR task in a conversational context: natural language interaction, cumulative clarification [4], feedback collection, and information needs profiling during conversations. In the literature, our work is closely related to that based on web search [1,2]; even so, our study differs from these in the following three ways. First, in our task, the user's information needs are expressed both colloquially and sequentially; thus, utterances include common natural language features beyond keyword queries, e.g., coreference, omission, and sentence semantics. Second, previous web search works involve in-domain model training, whereas this work represents a simple solution-only hyperparameter tuning. Finally, web search studies rely on user responses (e.g., clicks) as positive feedback, which can be viewed as implicit relevance without guidelines for consensus judgements used in our task. 1 
Query Reformulation using Query History for Passage Retrieval in Conversational Search
Conversational passage retrieval (ConvPR) is defined as an IR task in a conversational context. Given a sequence of conversational utterances u s = (u 1 , · · · , u i , u i+1 , · · · ) for a topic-oriented session s ∈ S, where S is the set of all dialogue sessions and u i stands for the i-th utterance (i ∈ N + ) in the session, which is formalized through turn i, the goal of this task is to find a set of relevant passages P i , for each turn's user utterance u i that satisfies the information needs in turn i with the context in previous turns u <i = (u 1 , · · · , u i−1 ).
Query Reformulation using Query History for Passage Retrieval in Conversational Search
Task scope To facilitate the ConvPR task and to provide a reusable, tractable dataset, the organizers of CAsT of TREC 2019 began with a selection of open-domain exploratory information needs I and provided a predefined set of topic-oriented sessions S I . 2 In addition, a passage collection C ⊇ P i was provided to retrieve candidate response passages for each turn in these sessions.
Query Reformulation using Query History for Passage Retrieval in Conversational Search
Under the CAsT setting, the utterances in the provided topicoriented sessions S I not only control the complexity of the task but also mimic features of "real" dialogues via the following properties:
Query Reformulation using Query History for Passage Retrieval in Conversational Search
• Utterance transitions are coherent between turns in a given topicoriented session. • Utterances are natural language questions, which are similar to the questions in the widely used Google Natural Questions dataset [27]. • Coreference and omission of natural language features in dialogues are included. • Turns depend only on previous utterances and not system responses. • Comparison between subtopics are introduced.
Query Reformulation using Query History for Passage Retrieval in Conversational Search
Conversational multi-stage retrieval system To reuse existing IR pipelines and benefit from the fine-tuned performance of relevance prediction models, a typical approach for ConvPR is to reformulate user utterances with their context into suitable queries and feed the reformulated queries into the pipelines.
Query Reformulation using Query History for Passage Retrieval in Conversational Search
For an IR system, let P (R = 1 | q, p) denote the probability conditioned on a query-passage pair (q, p), where R = 1 denotes that passage p ∈ C is relevant to query q (otherwise, R = 0). Currently, a mainstream method to facilitate IR is to further factorize P (R = 1 | q, p) into a multi-stage pipeline f θ • f ϕ ∝ P(R = 1 | q, p) as a trade-off between efficiency and effectiveness: f ϕ is a predefined non-parametric model such as Okapi BM25, the vector space model with TF-IDF, or variants of traditional IR models, and f θ stands for data-driven parametric models such as neural networks or other machine learning methods.
Query Reformulation using Query History for Passage Retrieval in Conversational Search
Likewise, for ConvPR, we factorize the probability of retrieving a relevant passage p ∈ P i for each turn i with an information set {u i , u <i } that comprises the utterances by turn i as
Query Reformulation using Query History for Passage Retrieval in Conversational Search
With this formulation, ConvPR can be approximated by separately maximizing the probabilities of (a) a relevance prediction model
Query Reformulation using Query History for Passage Retrieval in Conversational Search
Thus the goal of a query reformulation model is to reformulate a raw conversational user utterance u i in each turn i into a clear and informative query q i for the relevance prediction model [61].
Query Reformulation using Query History for Passage Retrieval in Conversational Search
As the judgments of relevant pairs in the training set from CAsT are sparse and very limited (see Table 2), we here focus on query reformulation methods and leave the burden of tuning a relevance prediction model to a known competitive pipeline-BM25 with BERT-in the large-scale passage ranking task [7,19,33].
Query Reformulation using Query History for Passage Retrieval in Conversational Search
Conversational query reformulation The goal of conversational query reformulation (CQR) is to obtain an informative query q i for each turn i for downstream relevance prediction models. Specifically, given an information set {u i , u <i } that includes the utterances by turn i, the tasks of CQR consist of the following two components: (a) filter out unnecessary information in {u i , u <i } and (b) construct informative input q i from the filtered information. Thus with CQR we seek a function q i = д ({u i , u <i }), the output of which (i.e., q i ) maximizes the probability in Eq. (1).
Query Reformulation using Query History for Passage Retrieval in Conversational Search
However, given the limited number of relevance labels, using supervised learning to construct a parametric function to maximize Eq. (1) is difficult. Therefore, we propose two label-free approximations for CQR as intuitive attempts. The first one is a non-parametric predefined model д ϕ (·) (see Section 3.3); the other is an off-the-shelf data-driven parametric model д θ (·) pretrained on other datasets under a transfer learning paradigm (Section 3.4). Note that both approaches only approximate д(·) with д ϕ (·) and д θ (·), respectively, due to the fact that the objective of д(·) in fact involves optimizing queries for an IR system.
Query Reformulation using Query History for Passage Retrieval in Conversational Search
In order to develop models for CQR with limited training data, we start with observing the characteristics of conversational user utterances.
Query Reformulation using Query History for Passage Retrieval in Conversational Search
Observation #1: Main topic and subtopic A session is centered around a main topic and the turns in the session dive deeper into several subtopics, each of which however only lasts a few turns. For instance, in Table 1, the main topic of the session is "physician's assistant" according to which Turns 2 and 3 discuss the subtopic of "educational requirements" while Turns 4 and 5 are related to the subtopic of "average starting salary. "
Query Reformulation using Query History for Passage Retrieval in Conversational Search
Observation #2: Degree of ambiguity The degree of ambiguity divides utterances into three categories. The first category includes utterances with clear implications, which can thus be treated as ad-hoc queries, such as Turns 1 and 6 in Table 1. The second category contains those starting a subtopic (e.g., Turns 2 and 4), and the last category is composed of most ambiguous utterances that continue a subtopic (e.g., Turns 3 and 5).
Query Reformulation using Query History for Passage Retrieval in Conversational Search
Based on the above observations, we propose two CQR methods: (1) Historical Query Expansion (HQE), a heuristic query expansion strategy; (2) Neural Transfer Reformulation (NTR), a data-driven approach transferring human knowledge to neural models from human annotated queries.
Query Reformulation using Query History for Passage Retrieval in Conversational Search
We first introduce HQE to heuristically capture the observations. Specifically, there are three main steps in HQE. For each utterance in a session, we (1) extract the main topic and subtopic keywords from the utterance; (2) measure the ambiguity of the utterance;
Query Reformulation using Query History for Passage Retrieval in Conversational Search
(3) perform query expansion for the ambiguous utterances with the main topic and subtopic keywords extracted from previous turns. We propose keyword extractor and query performance predictor modules to realize these three steps for constructing the non-parametric function д ϕ .
Query Reformulation using Query History for Passage Retrieval in Conversational Search
Given an utterance u i consisting of n(u i ) tokens, the utterance is represented as a tuple t 1 i , . . . , t
Query Reformulation using Query History for Passage Retrieval in Conversational Search
where t k i denotes the k-th token in u i . The aim of the KE is to compute the score of each token in the utterance so that the score indicates the importance of the token in the utterance. For each token, we propose leveraging the retrieval score of its most relevant document to characterize its importance in the utterance as
Query Reformulation using Query History for Passage Retrieval in Conversational Search
where R k i denotes the importance score of token t k i , and F KE (·) is the function to compute the relevance between a token and a passage p. The intuition behind this design is that the importance of a token can be judged from those documents that are highly relevant to it; that is, if a word is representative of its relevant documents, it is with high probability a keyword.
Query Reformulation using Query History for Passage Retrieval in Conversational Search
Given an utterance u i and a passage collection C, QPP measures the utterance's ambiguity. The literature demonstrates that the degree of query ambiguity is closely related to its ambiguity with respect to the collection of documents being searched [13,43,60]; thus, many metrics evaluate query ambiguity by analyzing retrieval scores. As we here are focused on providing an effective query expansion strategy for CQR rather than calculating the most accurate QPP, we keep the measurement for utterance ambiguity as simple as possible. Following the KE, we measure utterance ambiguity for u i as
Query Reformulation using Query History for Passage Retrieval in Conversational Search
where A i stands for the degree of utterance ambiguity and F QPP(·) estimates the relevance score between a passage and an utterance.
Query Reformulation using Query History for Passage Retrieval in Conversational Search
In our experiments, we set F QPP(·) (F KE (·)) as BM25 function. Note that the higher the A i score, the clearer the utterance u i .
Query Reformulation using Query History for Passage Retrieval in Conversational Search
Putting it all together. Algorithm 1 details the procedure of the proposed HQE, д ϕ ({u i , u <i }, C): keyword extraction (lines 3-8), query performance prediction (line 10), and query expansion (lines [11][12][13]. Note that R topic , R sub (where R topic > R sub ), η, and M are hyperparameters. Specifically, for each utterance u i in a session s ∈ S and a given passage collection C, HQE first extracts topic and subtopic keywords from u i and collects them in the keyword sets W topic and W sub , respectively. Then, QPP measures the clearness (ambiguity) of all u i for i > 1. Here η is the threshold to judge whether an utterance falls into the most ambiguous category. For all u i except the first utterance u 1 , HQE first rewrites u i by concatenating u i with the topic keyword sets W topic collected from u i and u <i . Moreover, if u i is ambiguous (i.e., A i < η), HQE further adds the subtopic keywords from previous M turns and turn i. We thus assume that the first utterance in a session is clear enough and that following utterances belong to the second or the most ambiguous category. Note that we concatenate W sub derived from previous M turns, as subtopic keywords last a few turns (see Observation #1). Also note that W sub includes the topic keywords in W topic , which ensures that topic keywords gain higher term weights than subtopic keywords in rewritten utterances.
Query Reformulation using Query History for Passage Retrieval in Conversational Search
Following a thought of a series of works regarding data-driven conversational query reformulation using neural networks [21,28,42,52], we also propose reformulating a raw utterance u i into a coreference-and-omission-free natural language question q NL i using neural transfer reformulation (NTR), which leverages neural networks to mimic and transfer patterns of how people rewrite questions in a conversational context.
Query Reformulation using Query History for Passage Retrieval in Conversational Search
We need these ingredients to use NTR to construct the parametric function д θ : (a) a large-scale, high-quality dataset of human generated q NL with source utterances and contexts; (b) an architecture to map an utterance and its conversational context into q NL ; (c) a dataset with enough diversity to cover open-domain exploratory information needs selected in the session sets of our interest S I .
Query Reformulation using Query History for Passage Retrieval in Conversational Search
Fortunately, open-domain QA research has produced QuAC [10]a diverse, large-scale dataset that contains conversational natural language questions of exploratory information needs-as well as CANARD [21], a derived conversational question-in-context rewriting dataset with human generated questions for QuAC questions.
Query Reformulation using Query History for Passage Retrieval in Conversational Search
Like CANARD and text summarization studies [21,46], we choose a sequence-to-sequence [9,48] (Seq2Seq) architecture to map variable length conversational contexts u <i and u i into q NL i . Without loss of generality, instead of using θ to represent the parametric function д θ that reformulates conversational queries optimized for an IR system in Eq. (1), we define a function parameterized bȳ θ taking input tokens (x 1 , . . . , x n ) of length n and output tokens (y 1 , . . . , y m ) of length m as
Query Reformulation using Query History for Passage Retrieval in Conversational Search
for this neural historical query reformulation task. A proxy for obtaining a particular set of parametersθ of this task under a configuration of a parametric function Seq2Seq(·,θ ) and a dataset from CANARD instead of CAsT is then
Query Reformulation using Query History for Passage Retrieval in Conversational Search
where [ū <i ∥ū i ] stands for the concatenation of a conversational context and an utterance of the i-th turn defined in the CANARD dataset with a separation token "|||" that indicates a boundary of utterances of different conversation turns. Finally, for CQR we adopt parameter and network architecture sharing as a simple strategy in transfer learning. Thus, after training on CANARD, we directly use the Seq2Seq model with its optimized parameter setθ to form our parametric model д θ (·) (i.e., θ =θ ) and directly use the model д θ (·) to reformulate q i from the information set {u i , u <i } of the CAsT dataset.  
Query Reformulation using Query History for Passage Retrieval in Conversational Search
Best CAsT entry This baseline is one of our submissions to TREC 2019 CAsT, which uses an earlier version of the proposed HQE method in a two-stage ConvPR system. This submission resulted in the best automatic run of the 41 runs from 21 teams.
Query Reformulation using Query History for Passage Retrieval in Conversational Search
Raw query A simple baseline that adopts the original queries without any query reformulation.
Query Reformulation using Query History for Passage Retrieval in Conversational Search
Concat Another baseline that concatenates each query with the queries in its previous M turns, where M is a hyperparameter. A variant of this method is to filter out certain types of words from the queries in the previous M turns before concatenation. Here we filter out words with POS tags other than adjective and noun, using spaCy as the POS tagger. 5 This variant is also applied to the proposed HQE.
Query Reformulation using Query History for Passage Retrieval in Conversational Search
Manual TREC organizers manually rewrote the originally ambiguous queries according to conversational context. 6 As the rewritten queries contain all of the information required to represent a
Query Reformulation using Query History for Passage Retrieval in Conversational Search
Information retrieval model settings As mentioned in Section 3.1, we implemented a two-stage information retrieval pipeline with BM25 retrieval (first stage) and BERT re-ranking (second stage). The parameters for the BM25 model were k 1 = 0.82 and b = 0.68, for which the number of retrieved passages was set to 1000. We used the Anserini toolkit [55] for corpus indexing and BM25 retrieval. The fine-tuned BERT model for the second stage re-ranking was provided by [33].
Query Reformulation using Query History for Passage Retrieval in Conversational Search
Query reformulation model settings The hyperparameters were selected by grid search on the CAsT training set (see Section 5.2.2 for more detail), whereas the neural models (i.e., LSTM and T5) were directly applied to rewrite the queries with beam search decoding after training on the CANARD dataset. The detailed settings of the neural models are described as follows. (a) LSTM (+Atten.): we adopted the bi-LSTM Seq2Seq model with attention, copy mechanism, and the same hyperparameter settings proposed in [21]. 7 (b) T5 [39]: 8 we used the T5-base model and its pretrained weights as the initialization and then fine-tuned it with the same hyperparameters used in [34]. 9 Evaluation For both stages, the results were evaluated by the overall ranking metric, mean average precision (MAP) at depth 1000, and the top-k ranking metrics NDCG@3 and NDCG@1. Note that NDCG@3 is the main metric used in CAsT. In addition, we report the values of recall at depth 1000 (R@1000) for first-stage retrieval. The evaluation was done using the TREC tool. 10 We also provide Win/Tie/Loss results based on R@1000 and MAP to show the number of queries whose performance was improved/unchanged/ deteriorated compared to manual query reformulation.
Query Reformulation using Query History for Passage Retrieval in Conversational Search
In this section, we first examine the effectiveness of the proposed HQE and NTR on the TREC-CAsT 2019 dataset; the results and analysis in terms of turn depths are also provided. Second, we study the impact of different query reformulation methods on passage re-ranking and provide the sensitivity analysis of our proposed HQE and NTR. Table 3 ("Full ranking" columns on right) lists the final results with the two-stage ConvPR approach on the TREC-CAsT evaluation set. The listed performance is from the re-ranked results based on the corresponding 1000 retrieved passages obtained in the first stage, the performance of which can be found in the same row. We note that all the query reformulation methods outperform the baseline with raw query; the naive Concat method serves as a competitive baseline. The proposed HQE and neural methods beat the best entry in TREC-CAsT 2019 and are only 4% to 5% below manual queries. In particular, the ad-hoc HQE (+POS) marginally Table 3: Performance on CAsT evaluation set. Win/Tie/Loss denotes the number of queries whose performance is improved/unchanged/deteriorated compared to manual query reformulation. The best results among single models for automatic query reformulation are in bold-faced. RRF denotes the reciprocal rank fusion of HQE (+POS) and NTR (T5).
Query Reformulation using Query History for Passage Retrieval in Conversational Search
Full ranking (BM25+BERT re-ranking) outperforms the best CAsT entry, which is from an earlier version of the HQE paper, in terms of MAP and NDCG@3 by 6% and 4%, respectively, while NTR (T5) significantly surpasses the best entry by 30% in MAP and 18% in NDCG@3.
Query Reformulation using Query History for Passage Retrieval in Conversational Search
Comparing the results of Concat with and without the POS filter suggests that using adjectives and nouns accurately extracts keywords from historical queries. Although the POS filter further improves the performance of HQE, the proposed HQE without such filtering still yields competitive performance, indicating the effectiveness of our keyword extraction module in HQE. However, for the neural models, the LSTM trained from scratch performs poorly; in contrast, the fine-tuned T5 delivers state-of-the-art performance, illustrating that the pretrained weights provide a satisfactory initialization for neural query reformulation models.
Query Reformulation using Query History for Passage Retrieval in Conversational Search
Also listed in Table 3 is the detailed Win/Tie/Loss performance comparison of each query with its manual counterpart. The "Raw query" results indicate that 55 out of 173 original queries in the dataset are clear enough for the full ranking task whereas the other 114 original queries are ambiguous and effectively rewritten manually; only 4 raw queries yield better performance than the manually rewritten ones. The proposed HQE (+POS) and NTR (T5) methods, in turn, successfully generate 30 better quality queries for full ranking compared to the manual ones. We also note that nearly 50% of NTR (T5) rewritten queries (i.e., 82 of 173) yield the same performance as the manually rewritten ones, demonstrating the effectiveness of transfer learning, where we directly fine-tune the T5 model on the CANARD dataset and conduct inference for queries (query written) in CAsT.
Query Reformulation using Query History for Passage Retrieval in Conversational Search
First stage retrieval with BM25 The effectiveness of the proposed query reformulation methods can also be observed from the results simply using the BM25 retriever in the first stage. As shown in Table 3 ("BM25" columns on left), the queries reformulated by HQE (+POS) and NTR (T5) both perform better than other baselines, leading to average performance improvement in terms of R@1000 over 70% and MAP around 20%. However, the Win/Tie/Loss comparison with manual queries shows the two methods improve the retrieval performance in a quite different way. Specifically, only less than 30% of the queries reformulated by NTR (T5) fail to beat their manual counterparts, which is far better than HQE (+POS) as it fails to rewrite 40% and 57% of queries regarding R@1000 and MAP, respectively. On the other hand, HQE (+POS) shows around 45 wins out of 173 queries, while only around 10 NTR (T5) rewritten queries beat the manual queries. It is surprising to find that these two methods achieve similar recall, but in a entirely different way, thus we also conduct detailed analysis in Section 6 to explore this.
Query Reformulation using Query History for Passage Retrieval in Conversational Search
Reciprocal rank fusion (RRF) [12] As HQE (+POS) and NTR (T5) improve the performance in a quite different way, we further fused the rank lists generated from HQE (+POS) and NTR (T5) with reciprocal rank fusion using the TREC tool. 11 The result is listed in the last row in Table 3. Observe that the fusion between the two lists in the first stage significantly outperforms the original ones and even yields performance comparable to manual queries, leading to more win than loss queries in terms of both R@1000 and MAP. Furthermore, the fusion from the two full ranking lists generates a better result with 0.348 and 0.536 performance in terms of MAP and NDCG@3, respectively.
Query Reformulation using Query History for Passage Retrieval in Conversational Search
Results by the turn depth Figure 1 compares the average recall and ranking performance of different reformulated queries in terms of the conversational turn depth. First, we observe that both the recall and ranking performance of raw queries (blue line) degrade abruptly after the first turn: conversational queries by nature become ambiguous as a dialogue moves forward. In contrast, HQE (+POS) and NTR (T5) yield stable recall performance over the turn depth with only slightly worse performance than the manual case. As for ranking performance, HQE (+POS) sees an obvious performance drop after the 7th turn in both stages, whereas NTR (T5) shows a slight performance decrease after the 8th turn, which suggests an advantage of NTR (T5) over HQE (+POS) in tracing deep conversation.
Query Reformulation using Query History for Passage Retrieval in Conversational Search
Summary We provide two strong query reformulation methods for conversational information retrieval: an ad-hoc HQE (+POS) and a neural NTR (T5) model. The experiments demonstrate that the reformulated queries effectively improve the performance of BM25 first-stage retrieval and BERT re-ranking. Furthermore, the two methods significantly outperform the best CAsT entry and achieve state-of-the-art performance for the CAsT full ranking task. Our analysis also shows that HQE (+POS) and NTR (T5) improve query reformulation from different perspectives and that the reciprocal rank fusion between the ranking lists from the two methods further leads to better performance.
Query Reformulation using Query History for Passage Retrieval in Conversational Search
Effects on re-ranking. The performance of full ranking does not fairly reflect the effects of each query reformulation method on BERT re-ranking, as it is also affected by the quality of the retrieved passages in the first stage. Therefore, we conducted another experiment to examine the effects solely of re-ranking. Specifically, we first retrieved the top 1000 passages using manual queries with BM25 and re-ranked the top 1000 passages using the reformulated queries via different query reformulation methods with BERT. In this setting, all the reformulation approaches had the same passage pool for re-ranking, ensuring a fair comparison.  Table 4 compares the results of passage re-ranking based on different query reformulation methods. Compared to the full ranking results, all query reformulation methods show better ranking results; NTR (T5)'s re-ranking performance especially closes with the manual case, with 0.353 in MAP and 0.554 in NDCG@3. Second to NTR (T5), both HQE (+POS) and Concat (+POS) obtain a 0.33 in MAP with over 30% of queries (50/173) beating the manual ones. HQE (+POS) and NTR (T5) show similar R@1000 and MAP performance using BM25 retrieval, but NTR (T5) yields significantly better BERT re-ranking results, perhaps because it generates queries more like natural language queries that are thus well-suited for the BERT re-ranker which was trained on natural language queries. Also note that although HQE (+POS) outperforms Concat (+POS) in top-k ranking performance (i.e., NDCG@3, NDCG@1), they have comparable overall ranking (MAP) performance in this task, which suggests that the proposed HQE outperforms Concat in full ranking (see Table 3) mainly due to the gain from first-stage BM25 retrieval. HQE (+POS) Figure 2(a) shows the grid search results in R@1000 and MAP. Specifically, we tune R topic , R sub , η and M for the optimal R@1000 and MAP separately. By fixing (η, R topic , R sub ) at the best R@1000, (10, 4.5, 3.5), and at the best MAP, (12, 4.0, 3.0), Figure 2(a) shows the grid search results in terms of various M.
Query Reformulation using Query History for Passage Retrieval in Conversational Search
We first note from Figure 2(a) that both R@1000 and MAP improve when M > 0, indicating that adding subtopic keywords from previous M turns is effective for query expansion. In addition, R@1000 and MAP see different trends on the grid search, with the best M = 5 and M = 1 for R@1000 and MAP, respectively, suggesting that the optimal query for BM25 search is different in terms of R@1000 and MAP. Thus, in the previous experiments, we generated HQE (and Concat) queries using the hyperparameters with the best R@1000 for BM25 first-stage retrieval and those with the best MAP for BERT re-ranking. 12 NTR (T5) We also analyze the sensitivity of beam width w in beam search decoding for NTR (T5) in Figure 2(b), where bars denote the BLEU scores -which is used for evaluating machine translated texts [44] and also served as a performance indicator in the work of Elgohary et al. [21] -(left y-axis) and lines denote the improvements of IR metrics compared to beam width w = 1 (right y-axis). Note that w stands for a number of partial sequences with highest probabilities we keep in order to find a single sequence with a limited-width bread first search in a context of sequential modeling. To determine the optimal hyperparameters for CAsT query inference, we consider the development (dev) set in CANARD and the training set of CAsT to choose w in the range of {1, 5, 10, 15, 20}. In specific, Figure 2(b) illustrates the BLEU score versus width in the CANARD dev set and R@1000 and MAP versus width in the CAsT training set. Observe that the best width, w = 10, achieves the highest BLEU (60.32) in the CANARD dev set, 13 whereas both R@1000 (+1.1 points) and MAP (+0.5 points) compared to w = 1 achieve the best performance at w = 5 in the training set of CAsT. To maintain query reformulation quality without hurting IR performance, we choose w = 10 in all of our experiments.
Query Reformulation using Query History for Passage Retrieval in Conversational Search
To further explore the distinct behaviors of HQE and NTR discovered in Sections 5, we present a study to unearth their differences from the following three perspectives:
Query Reformulation using Query History for Passage Retrieval in Conversational Search
(1) query characteristics from embedding space and pure texts;
Query Reformulation using Query History for Passage Retrieval in Conversational Search
(2) retrieval characteristics in terms of turn-depth-wise and session-wise aggregations; (3) a case study that illustrates the models' pros and cons.
Query Reformulation using Query History for Passage Retrieval in Conversational Search
Note that our analysis is based on the 20 sessions with relevance judgments in the CAsT evaluation set.
Query Reformulation using Query History for Passage Retrieval in Conversational Search
An intuitive way to illustrate the characteristics of conversational queries is to visualize their embedding using the BERT encoder. This intuition, which comes from the MS MARCO conversational search task, 14 is based on an assumption that utterances in the same conversation session are similar in the embedding space, as they are topic-oriented. We here leverage the BERT [17] model to project the reformulated queries-Raw query, HQE (+POS), NTR (T5), and Manual-into the embedding space and apply 2-dimensional tdistributed stochastic neighbor embedding (t-SNE) [30] on them altogether to make sure they are in the same embedding space. 15 Panels (a)-(d) in Figure 3 visualize their respective t-SNE embeddings, where color represents different session identifications (IDs) and the embedding size reflects turn depth.
Query Reformulation using Query History for Passage Retrieval in Conversational Search
From Figure 3 we note the following. First, the Raw query in panel (a) shows unclear boundaries between sessions, especially in the central region. This could be attributed to the ambiguity from coreferences and omissions in conversational utterances, as it is difficult to differentiate them without context. Second, Manual and NTR (T5) in panel (c) and (d) form more clear clusters between sessions than Raw query, suggesting their queries are more topicoriented. Furthermore, observe that NTR (T5) and Manual obtain similar embedding distributions, implying that the two models yield similar queries, thereby leading to the many Ties in Tables 3 and 4. Finally, HQE (+POS) in panel (b) forms clear clusters-queries in the same session heavily overlap, suggesting queries reformulated by HQE (+POS) are similar within the same session. 13 T5 achieves better BLEU than 51.37 of LSTM (+Atten.) in the dev set of CANARD [21]. 14 https://github.com/microsoft/MSMARCO-Conversational-Search 15 Note that we here follow the setup for building artificial conversational sessions from Bing search queries (see footnote 14 for details). To further attest the high similarity between NTR (T5) reformulated queries and Manual ones, we measure their query similarities quantitatively by pure text. Specifically, we compare query texts from different CQR methods with BLEU [44]. 16 Here, we take the Manual queries as the reference sentences and calculate BLEU scores for the other methods. As shown in Table 5, NTR (T5) queries yield the highest score, whereas HQE (+POS) queries have the lowest. Note that raw queries yield the medium score among all. These results not only validate the high query similarity between NTR (T5) and Manual observed in Figure 3 but also show that HQE (+POS) generated queries are markedly different from other methods.   
Query Reformulation using Query History for Passage Retrieval in Conversational Search
Above, we clarified the distinct behaviors of two CQR approaches from a query perspective. To further uncover the reasons behind the Wins and Ties of HQE (+POS) and NTR (T5) versus Manual queries in Table 3, we analyze the similarities of the retrieved sets when different CQR methods are adopted. In Figure 4, the sets retrieved by  BM25 are analyzed in a turn-depth-wise perspective in panel (a) and in a session-wise perspective in panel (b). Specifically, we consider the Jaccard similarity J (·) to quantitatively analyze the retrieved sets. Note that in Figure 4(a), the similarity for turn i is the averaged values of the Jaccard similarities between the (i − 1)-th and i-th turns over all sessions. Figure 4(b), in turn, takes the retrieved sets from Manual query as the reference sets to calculate relative (rel.) R@1000 and J (·, P Manual ) of NTR (T5) and HQE (+POS) versus Manual; then, a pair of average metrics (rel. R@1000, J (·)) over all turns in each session is illustrated as a point on the figure. We draw three conclusions from Figure 4. First, the Ties of Manual and NTR (T5) in Table 3 could be explained by the observations from panel (a) and (b) in Figure 4. As shown in panel (a), whereas the retrieved sets' similarities of NTR (T5) and Manual stay around 0.15 as the turns proceed, NTR (T5) also mainly centralizes around 0 on the y-axis in panel (b). Second, we conjecture the Wins of HQE (+POS) in Table 3 come along with the upper-left clustering in Figure 4(b); this could be due to the disparate behaviors observed in Figure 4(a)-HQE (+POS) tends to retrieve similar sets as the turns proceed. Finally, Figure 4 illustrates not only a significant gap between HQE (+POS) and NTR (T5) in panel (a) but also a clear boundary at 0.55 of the x-axis in panel (b). These observations suggest that the success of the fusion approach (RRF) could be attributable to the dissimilar behaviors of these two methods, which balance the biases from the two models [12]. Tables 6 and 7 present two examples from sessions 32 and 54 to showcase the pros and cons of HQE (+POS) and NTR (T5). The row under each turn's query texts also shows the BM25 retrieval performance (R@1000) of the four reformulation methods: Raw query, Manual, HQE (+POS), and NTR (T5). 17 Table 6 compares the reformulated queries about sharks and shows that the queries reformulated by NTR (T5) lose the context word shark after turn 6. Furthermore, from turns 7 to 10, NTR (T5) considers the context as makos rather than makos shark; hence, NTR (T5) is unlikely to retrieve passages with makos shark compared to HQE (+POS) and Manual. However, HQE (+POS) performs better in terms of R@1000 and the Win mainly due to the concatenation of the topic keyword shark. Especially in turn 6, HQE (+POS) significantly outperforms NTR (T5) and Manual, the main reason being that the words great white in NTR (T5) and Manual guide the BM25 model to retrieve documents with both great and white but not relevant to shark. This example also demonstrates that human rewriting queries are not always applicable.
Query Reformulation using Query History for Passage Retrieval in Conversational Search
On the other hand, HQE (+POS) can sometimes be too aggressive in injecting context into utterances. As shown in Table 7, HQE (+POS) emphasizes too much about "museum" when the subtopics have changed to reflecting pool in turn 6 and food (D.C. half smoke) in turn 7. On the contrary, the NTR (T5) mimics human to put adequate contexts in the utterances. For instance, as shown in the table, NTR (T5) puts Washington D.C. in turn 7 as sufficient contexts for BM25 model to understand the raw utterance. Moreover, take turn 5 as an example; NTR (T5) can sometimes address the context missing issue (i.e., adding the word Smithsonian) introduced by human writers, thereby making NTR (T5) outperform Manual query rewriting in few cases.
Query Reformulation using Query History for Passage Retrieval in Conversational Search
We present HQE and NTR, both conversational query reformulation methods stacked on a successful multi-stage IR pipeline. The effectiveness of our methods are attested by experiments on the CAsT benchmark dataset, the results of which suggest that the two methods have different advantages in fusing context information into conversational user utterances for downstream IR models. Finally, this work elevates the state of the art in CAsT benchmarks and provides simple but effectives baselines for future research.