How has cycle training been applied to improve the generation quality of transformer NLP models?
Cycle training (Zhu et al., 2017; Zhou et al.,
2016) relies on two models which are essentially
inverse transforms of each other that are used to
create “cycles”, which should return identical out-
put to the input given. There are two distinct forms
of cycle training. The first form (Zhou et al., 2016)
aims to learn to transform from one input form to
another, e.g., to learn rotations of a car in one im-
age to another. The second is the use of a “cycle
consistency loss” as an auxiliary loss to some other
task, e.g., in generative adversarial networks per-
forming style transfer on images (Zhu et al., 2017).
NLG typically relies on models which are auto-
regressive and non-differentiable. This precludes
the direct use of cycle consistency losses (Guo et al.,
2020; Pang and Gimpel, 2019; Iovine et al., 2022a).
Nonetheless, we can still use cycle training via an
alternating training strategy where we freeze one
model and train the other, and vice versa (Lample
et al., 2017; Pang and Gimpel, 2019). In this work,
we train solely using cycle consistency. Cycle train-
ing has been recently applied to language process-
ing tasks. In one text-to-text application, Iovine
et al. (2022b) use a similar unsupervised methodol-
ogy to perform bidirectional text transformations
for converting keyword search queries to natural
language questions, and vice versa. It has also been
used for Named Entity Recognition in the absence
of large annotated text (Iovine et al., 2022a). In
this case, one model extracts entities, and the in-
verse model creates text from those entities. The
approach is limited by the fact that there are many
ways to realize sentences with the same entities.
Put differently, there is no strong requirement of
cycle consistency, and this will become even more
apparent as we analyze the conditions under which
cycle training works well in data-to-text tasks
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
Named Entity Recognition (NER) is a crucial natural language understanding task for many down-stream tasks such as question answering and retrieval. Despite significant progress in developing NER models for multiple languages and domains, scaling to emerging and/or low-resource domains still remains challenging, due to the costly nature of acquiring training data. We propose CycleNER, an unsupervised approach based on cycle-consistency training that uses two functions: (i) sentence-to-entity – S2E and (ii) entity-to-sentence – E2S, to carry out the NER task. CycleNER does not require annotations but a set of sentences with no entity labels and another independent set of entity examples. Through cycle-consistency training, the output from one function is used as input for the other (e.g. S2E → E2S) to align the representation spaces of both functions and therefore enable unsupervised training. Evaluation on several domains comparing CycleNER against supervised and unsupervised competitors shows that CycleNER achieves highly competitive performance with only a few thousand input sentences. We demonstrate competitive performance against supervised models, achieving 73% of supervised performance without any annotations on CoNLL03, while significantly outperforming unsupervised approaches.
[{"first":"Andrea","last":"Iovine","middle":[]},{"first":"Anjie","last":"Fang","middle":[]},{"first":"Besnik","last":"Fetahu","middle":[]},{"first":"Oleg","last":"Rokhlenko","middle":[]},{"first":"Shervin","last":"Malmasi","middle":[]}]
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting
Users expect their queries to be answered by search systems, regardless of the query’s surface form, which include keyword queries and natural questions. Natural Language Understanding (NLU) components of Search and QA systems may fail to correctly interpret semantically equivalent inputs if this deviates from how the system was trained, leading to suboptimal understanding capabilities. We propose the keyword-question rewriting task to improve query understanding capabilities of NLU systems for all surface forms. To achieve this, we present CycleKQR, an unsupervised approach, enabling effective rewriting between keyword and question queries using non-parallel data.Empirically we show the impact on QA performance of unfamiliar query forms for open domain and Knowledge Base QA systems (trained on either keywords or natural language questions). We demonstrate how CycleKQR significantly improves QA performance by rewriting queries into the appropriate form, while at the same time retaining the original semantic meaning of input queries, allowing CycleKQR to improve performance by up to 3% over supervised baselines. Finally, we release a datasetof 66k keyword-question pairs.
[{"first":"Andrea","last":"Iovine","middle":[]},{"first":"Anjie","last":"Fang","middle":[]},{"first":"Besnik","last":"Fetahu","middle":[]},{"first":"Jie","last":"Zhao","middle":[]},{"first":"Oleg","last":"Rokhlenko","middle":[]},{"first":"Shervin","last":"Malmasi","middle":[]}]
Unsupervised Evaluation Metrics and Learning Criteria for Non-Parallel Textual Transfer
We consider the problem of automatically generating textual paraphrases with modified attributes or properties, focusing on the setting without parallel data (Hu et al., 2017; Shen et al., 2017). This setting poses challenges for evaluation. We show that the metric of post-transfer classification accuracy is insufficient on its own, and propose additional metrics based on semantic preservation and fluency as well as a way to combine them into a single overall score. We contribute new loss functions and training strategies to address the different metrics. Semantic preservation is addressed by adding a cyclic consistency loss and a loss based on paraphrase pairs, while fluency is improved by integrating losses based on style-specific language models. We experiment with a Yelp sentiment dataset and a new literature dataset that we propose, using multiple models that extend prior work (Shen et al., 2017). We demonstrate that our metrics correlate well with human judgments, at both the sentence-level and system-level. Automatic and manual evaluation also show large improvements over the baseline method of Shen et al. (2017). We hope that our proposed metrics can speed up system development for new textual transfer tasks while also encouraging the community to address our three complementary aspects of transfer quality.
[{"middle":[],"last":"Pang","first":"Richard Yuanzhe"},{"middle":[],"last":"Gimpel","first":"Kevin"}]
CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training
Two important tasks at the intersection of knowledge graphs and natural language processing are graph-to-text (G2T) and text-to-graph (T2G) conversion. Due to the difficulty and high cost of data collection, the supervised data available in the two fields are usually on the magnitude of tens of thousands, for example, 18K in the WebNLG dataset, which is far fewer than the millions of data for other tasks such as machine translation. Consequently, deep learning models in these two fields suffer largely from scarce training data. This work presents the first attempt to unsupervised learning of T2G and G2T via cycle training. We present CycleGT, an unsupervised training framework that can bootstrap from fully non-parallel graph and text datasets, iteratively back translate between the two forms, and use a novel pretraining strategy. Experiments on the benchmark WebNLG dataset show that, impressively, our unsupervised model trained on the same amount of data can achieve performance on par with the supervised models. This validates our framework as an effective approach to overcome the data scarcity problem in the fields of G2T and T2G.
[{"first":"Qipeng","last":"Guo","middle":[]},{"first":"Zhijing","last":"Jin","middle":[]},{"first":"Xipeng","last":"Qiu","middle":[]},{"first":"Weinan","last":"Zhang","middle":[]},{"first":"David","last":"Wipf","middle":[]},{"first":"Zheng","last":"Zhang","middle":[]}]
Unsupervised Machine Translation Using Monolingual Corpora Only
Machine translation has recently achieved impressive performance thanks to recent advances in deep learning and the availability of large-scale parallel corpora. There have been numerous attempts to extend these successes to low-resource language pairs, yet requiring tens of thousands of parallel sentences. In this work, we take this research direction to the extreme and investigate whether it is possible to learn to translate even without any parallel data. We propose a model that takes sentences from monolingual corpora in two different languages and maps them into the same latent space. By learning to reconstruct in both languages from this shared feature space, the model effectively learns to translate without using any labeled data. We demonstrate our model on two widely used datasets and two language pairs, reporting BLEU scores up to 32.8, without using even a single parallel sentence at training time.
[{"first":"Guillaume","last":"Lample","middle":[]},{"first":"Ludovic","last":"Denoyer","middle":[]},{"first":"Marc'Aurelio","last":"Ranzato","middle":[]}]
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
Named Entity Recognition (NER) is a core NLP task, being applied in Web search [4], conversational agents [35], and Relation Extraction [1], with increasing adoption in specialized domains like medicine [19,30], analysis of historical collections [9], etc.
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
NER approaches are typically trained in a fully supervised manner. The NER training data consists of token-level annotations, where each token is annotated according to a NER class taxonomy (e.g. PER, ORG, O, etc.). While there are significant annotation efforts for popular domains like news [23], annotations for specialized domains, e.g. medicine, are difficult to obtain due to: (i) annotator training being complex and time consuming, with domain knowledge acquisition being key (e.g. that "HKE6" is a Gene), and (ii) token-level annotations from diverse domains at scale being costly.
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
S-cycle E-cycle Generation Step
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
Training
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
Step Figure 1: Overview of the two CycleNER tasks.
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
Unsupervised NER methods can help alleviate some of these requirements. Existing unsupervised approaches are based on complex hand-crafted rules, or rely on entity information from preexisting lexicons or knowledge bases (e.g. [3,14]). However, it is challenging to create rules and lexicons for entities from special domains, and even harder for low-resource languages. In this paper, we propose CycleNER, an unsupervised approach for training NER models using unannotated sentences, and very small named entity samples that are independent from the sentences. CycleNER has two components: (i) sentence-to-entity (S2E), and (ii) entity-to-sentence (E2S). S2E extracts entities given a sentence, while E2S generates a sentence given input entities. S2E and E2S are implemented using sequence-to-sequence transformer architectures since this allows bi-directional conversion between sentences and entities. These two components are jointly trained via two cycles (as shown in Figure 1): (1) sentence learning cycle (S-cycle) and (2) entity learning cycle (E-cycle). Specifically, sentences are the input and output for the S-cycle, while entities are the same for the E-cycle. In each cycle, one component first generates the inputs and the other component is then trained using these generated inputs. In this way, two components are trained effectively and their representation spaces are aligned. Compared to the traditional NER training approach, cycle-consistent training used unlabeled data and is therefore unsupervised. This data can be much cheaper to obtain than traditional supervised datasets.
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
We experiment with several NER benchmarks, i.e. CoNLL03, WNUT17, OntoNotes 5.0, and BioCreative II BC2GM. We first verify that the sequence-to-sequence architecture, i.e. T5 [22], allows us to effectively extract entities from sentences. Since CycleNER implements NER by two-cycle training, which is different from the traditional token classification, we experiment with different training strategies. We evaluate the model by setting different numbers of sentences and entity examples, and compare it to SOTA NER results. Results show that our CycleNER can achieve competitive performance compared to supervised approaches. For example, when using a thousand entity examples, CycleNER can achieve 0.686 (72.7% of SOTA) and 0.613 (66% of SOTA) in CoNLL03 and OntoNotes 5.0, respectively. Our contributions are summarized as follows:
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
• We propose CycleNER, a novel unsupervised training approach.
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
• We implement CycleNER using two encoder-decoder architectures, E2S and S2E. • We study how to effectively train NER using CycleNER.
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
• We show through an ablation study that our approach increases performance as more sentence or entity examples are provided. • We compare our approach against several supervised and unsupervised baselines, achieving competitive results.
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
Supervised NER: NER is typically cast as a supervised learning task, with most of the state-of-the-art approaches relying on deep recurrent models [10,21], convolution based [16], or pre-trained transformer architectures [8,32]. These models achieve highly satisfactory NER performance on typical benchmarking datasets like CoNLL. Yet, the results achieved on CoNLL do not transfer across domains [17], and supervised data from target domains is necessary. While NER is typically performed through sequence labelling, recent approaches have cast this problem as a sequence-to-sequence (seq2seq) task. Zhu et al. [36] use a Bi-LSTM model to encode sentences and an LSTM+CRF to generate the output entities from an input sentence. Compared against several supervised baselines, the approach has proven to be effective. Similarly, Straková et al. [26], tackle the problem of nested NER, where for a token all possible entity labels are generated. Arguably, recent advances in pre-training seq2seq transformer models, like T5 [22] or BART [12], can be used to perform NER more effectively than recurrent models (LSTMs).
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
Unsupervised NER: Unsupervised approaches typically rely on hand-crafted rules and pre-obtained lexicons. Etzioni et al. [3] extract entities according to syntactic pattern matches. Zhang and Elhadad [34] propose an unsupervised approach that is applied on the medical domain. The approach first obtains seed entities from an external source, then identifies entities from sentences through chunking and using inverse document frequency. Similarly, Liang et al. [14] propose to generate distant labels using knowledge bases and use these labels to improve supervised NER training. Liu et al. [15] also use a knowledge base to train a NER model (KALM) by identifying whether a word in a sentence is from knowledge base or a general dictionary. While these methods highly rely on external knowledge bases, CycleNER aims at using very limited entity samples, without the need of external resources. In [28], a neural Hidden Markov Model (NeuralHMM) is proposed for token annotation. It estimates the probability distribution of the latent classes using the Baum-Welch algorithm. To do so, it relies on a set of lexical, morphological and syntactic features extracted using neural networks. Morphological features are extracted using CNN, whereas the sentence's context is captured using LSTMs. This approach is shown to be effective for POS tagging. Both POS tagging and NER can be cast as sequence labeling problems, and therefore we consider this approach as an unsupervised baseline.
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
Cycle-consistency Training: First introduced in neural machine translation (NMT) [11,18,24], cycle-consistency training has been applied to align the latent spaces of auto-encoders trained on different languages, such that for a few seed words and their corresponding translations, their representations are aligned. This allows to train NMT models without parallel datasets. Recently, Guo et al. [5] proposed CycleGT, which jointly learns graph-to-text and text-to-graph tasks. CycleGT is trained using non-parallel data, consisting of textual snippets and graph triples. To solve the multiple mapping problem between text and graph modalities, Guo et al. [6] propose a conditional variational auto-encoder to transform the surjective function into an implicit bijection.
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
Contrary to previous works, our approach does not require adversarial training or denoising auto-encoder strategies. Instead, we exploit pre-trained transformer models as a means to regularize the training of CycleNER. Furthermore, contrary to Mohiuddin and Joty [18], our cycle-consistency training does not use two latent spaces, but rather, the output of each models (S2E or E2S) is fed as input to each other to generate intermediate sentences or entity sequences.
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
Our approach transfers the intuitions from Lample et al. [10] and Guo et al. [5]. Unlike Guo et al. [5], we are not constrained on having different modalities in order to perform cycle-consistency training, since we treat NER as a text-to-text generation task.
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
We formulate CycleNER as an unsupervised cycle-consistency learning problem, outlining the intuition, the required data and tasks.
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
Cycle-consistency learning involves simultaneously learning a forward and inverse transformation of data. This approach can be applied to unannotated non-parallel data [37] to learn mapping functions in an unsupervised setting. We propose to apply this framework for NER, using non-parallel entities and sentences, and training generative models to transform sentences to entities, and vice versa.
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
Unsupervised NER Data: CycleNER relies on non-parallel data, i.e. a set of sentences S = {s 1 , . . . , s n }, where each sentence s i can mention zero or more entities, and a set of entity sequences Q={⟨e 1,1 . . . , e 1,k ⟩, . . . , ⟨e m,1 , . . . , e m,l ⟩}, where each sequence ⟨e i,1 , . . . , e i,k ⟩ consists of zero or more entities. Specifically, we use an entity sequence to represent entities contained in a sentence. The two sets S and Q are unannotated, i.e. S does not have entity
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
EU<sep> Organization <sep> German <sep> Miscellaneous <sep> British <sep> Miscellaneous Figure 2: Entity sequence q format from sentence s.
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
annotations and Q is an independent set of entity sequences. The only prerequisite is that the entity distribution appearing in S and Q have some overlap.
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
Training Tasks: Given S and Q, in CycleNER we define two main tasks in CycleNER:
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
(1) Sentence-to-Entity (S2E): For a sentence s ∈ S, S2E outputs an entity sequence q ′ . This represents the NER task in CycleNER, e.g.:
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
(2) Entity-to-Sentence (S2E): For an input entity sequence q ∈ Q, E2S generates an output sentence s ′ containing the entities in q, e.g.: Figure 3 shows an overview of CycleNER, where S2E and E2S are used to implement Sand E-cycles, which ensure cycle-consistency and allow us to train them in an unsupervised manner. Next, we describe in detail S2E and E2S, and the cycle-consistency training.
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
In cycle-consistency training S2E represents the inverse function of E2S, and vice versa. To meet this requirement, S2E and E2S should share the same input and output format, i.e. textual format. Moreover, the output should retain the information needed to easily reconstruct the input from it. In sequence labeling, a sentence is classified as a sequence of IOB tags, where the tags are neither in the same format as the input sentence, nor they retain any token information. Hence, sequence labeling cannot be used to implement S2E and E2S.
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
On the other hand, seq2seq can naturally fit this task, as it can be used to generate entity tokens (denoted as q ∈ Q) directly from a sentence (e.g. in [36]). First, s is passed through a seq2seq encoder, then the decoder generates a sequence q a token at a time. Since q contains important information about important tokens from s, it can be to do construct a sentence in E2S.
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
Entity Sequence Representation in CycleNER. As shown in Figure 2, a sentence can contain multiple entities. We propose to use an entity sequence, q, to represent multiple entities in a sentence in CycleNER. The entity sequence is required to contain all entity information to enable cycle-consistency training. For this, we first represent an entity as a combination of its surface form and entity class in textual form. One sentence can contain multiple entities, denoted as an entity sequence q. A special token, ⟨sep⟩, separates different entities in q, as well as the surface forms and class tokens within an entity e ∈ q. Figure 2 shows an example of the proposed format. In case multiple occurrences of the same entity are present in a sentence, q will contain the repeated entity as many times as it appears in the sentence. Moreover, the order of the entities in q reflects the order of them appearing in the sentence. This guarantees that S2E can extract correct entities from sentences. For example, given a sentence "The Amazon CEO pledged to donate to the Amazon rainforest preservation efforts", q should be "Amazon ⟨sep⟩ organization ⟨sep⟩ Amazon ⟨sep⟩ location". Furthermore, keeping the entity order simplifies the reconstruction of the original sentence during CycleNER.
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
The objective of cycle-consistency is to ensure that S2E and E2S functions establish an effective mapping between sentences and entities. We follow the principles of Iterative Back-Translation (IBT) [5,7] to incrementally train the two functions, namely the output of one function is used as input to train the other function. We denote with θ and ϕ the parameters of S2E and E2S, respectively. The training is divided into two main cycles: S-cycle and E-cycle.
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
S-cycle Training. In this cycle, we generate first a synthetic set of intermediate entity sequences by applying S2E to the input S, namely S2E(S) → Q ′ , highlighted by the red box in Figure 3. The synthetic training tuples ⟨S, Q ′ ⟩ are used to train E2S in a supervised manner. Specifically, in this cycle E2S is trained to generate synthetic sentences S ′ that are similar to S (highlighted in the blue box). For instance, in Figure 3 (a), in the first step S2E receives an input sentence s, and produces the output entity sequence q ′ . In the second step, E2S receives the synthetic q ′ and generates back a synthetic sentence s ′ that ideally should mimic the original input sentence s, as shown in the example below:
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
This concludes the S-cycle training, at the end of which, similar to Guo et al. [5] we compute the reconstruction loss, i.e. the average cross-entropy loss between the input S and the generated S ′ :
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
where p(·) and д(·) represent the real and predicted token probabilities. Whereas, |s | represents the sentence length, s i and s ′ i are the i-th token in s and s ′ , and |S | is the number of input sentences.
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
E-cycle Training. In this cycle, the input are entity sequences Q, which are pushed through E2S to generate a set of synthetic sentences S ′ , namely E2S(Q) → S ′ , cf. highlighted boxes in red in Figure 3 (b). The E-cycle trains S2E to generate the entity sequences Q ′ that are similar to Q. First, for an entity sequence q a synthetic sentence s ′ is generated via E2S, which is then fed into S2E to generate the synthetic q ′ , that is identical or highly similar to q:  Similar to S-cycle, we compute the reconstruction loss between the input Q and the synthetic Q ′ in the E-cycle:
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
where |q| and |Q | are the fixed sequence length (i.e. the number of surface form tokens in q) and total number of entity sequences. q i and q ′ i are the i-th token in q and q ′ . Note that since the entity sequence contains both the surface form and entity type, L θ (Q, Q ′ ) reflects the loss of both the surface form and entity type.
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
Joint training. The two cycles are conducted iteratively. Operationally, the following steps are executed in sequence for each batch of training data:
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
• S-cycle Step 1 (see Figure 3): a batch of synthetic sequences Q ′ is generated by S2E giving batch of training sentences S. • S-cycle Step 2: The parameters of E2S are trained using synthetic training tuples ⟨Q ′ , S⟩. • E-cycle Step 1: A batch of synthetic sentences S ′ is generated by E2S giving a batch of training entity sequences Q. • E-cycle Step 2: The parameters of S2E are trained using synthetic training tuples ⟨S ′ , Q⟩.
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
S2E leverages Q to recognize entities and their contexts in S, while E2S learns to generate plausible sentences using the entries in Q. The parameter spaces of the two tasks are trained to align each other, such that from s we can generate entity sequences q, and vice versa. Cycle-consistency enforces the reconstruction of the original input sentence s from the generated entity sequence q ′ , or generating back the entity sequence q from an output sentence s ′ . This ensures that S2E and E2S can learn quickly even in the beginning stage. Accordingly, E-and S-cycles are equally important for CycleNER.
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
In order to select the best iteration during training (stopping criterion), we use the loss of E-cycle on a development set. This choice is futher discussed and verified in Section 7.2. Finally, it is worth noting that during the S-cycle, the parameters θ are not updated, due to the fact that the generated synthetic outputs at the end of Step 1 are non-differentiable. Effectively, L ϕ can only be back-propagated to E2S. Similarly, during the E-cycle, ϕ is not updated, and L θ is only back-propagated to S2E. This is similar to previous applications of IBT [5].
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
We consider several NER benchmarking datasets across various domains to sample sentences and entity sequences for training Cy-cleNER. Table 1 shows an overview of the datasets. All the datasets are in English language. CoNLL [27] is a common NER benchmark dataset, consisting of textual snippets from the news domain. WNUT [2] contains textual snippets extracted from social media, e.g. Twitter. This dataset represents a challenging benchmark with novel and emerging entities. OntoNotes [31] contains mostly sentences from news, Web data and conversational speeches. Specifically, we use the version that includes a taxonomy of 18 NER classes. BioCreative II BC2GM [25] consists of sentences extracted from medical articles, with genes as the annotated named entities. This dataset is very challenging, given that the surface form of gene sequence entities can be highly complex.
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
Type 
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
To train the E2S function, we feed it named entity sequences as input. The length of the entity sequences should reflect the distribution of sequences in our sentences. We compare two methods of acquiring entity sequences. The first solution directly uses a small portion of entity sequences from the ground-truth, which ensures high data quality, and helps to test our approach in an ideal scenario. In the second solution, we select a small sample of entities  
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
CycleNER requires a set of sentences and a set of entity sequences. From the different NER datasets, we construct varying sets of training data, where we vary the number of sentences and entity sequences used for training. For the sake of clarity, we introduce a naming convention to distinguish the different dataset configurations. Namely, Dataset/10k/10k SQ , where the first portion represents the dataset, followed by the number of sentences and entity sequences in thousands, and the means by which we extract the entity sequences. Table 3 shows the different configurations we use for training and development data, where we vary the amount of sentences and entity sequences we use to train CycleNER.
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
This section describes the baselines, and the setup of our approach. Furthermore, we explain evaluation scenarios that we want to validate in our experimental evaluation.  
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
CycleNER: For our approach, S2E and E2S represent seq2seq models. We implement them using pre-trained T5 [22] models.
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
BERT: This represents a competitive supervised baseline, and can be considered as the upper bound for NER performance of unsupervised models. We fine-tune a pre-trained BERT [8] for NER. NeuralHMM: This represents an unsupervised baseline, which trains a neural Hidden Markov Model using sentences as input only. NeuralHMM requires only sentences for training, and it does not require entity information for training. Its output space is a set of latent classes, the number of which can be set as a learning parameter. Additionally, a mapping strategy is required to link the latent classes to entity classes. To do this, the co-occurrence between each latent class and entity class in the test set is measured, and the most frequently co-occurring latent class with an entity class is assigned. Lexical Matcher: We extract the named entities from the training set and their corresponding type, which then using a lexical matcher (surface form match) are used to identify entities in the test set. This represents a basic unsupervised baseline, and fails for entities with ambiguous surface forms. BERT-Matcher: We also train a weakly-supervised BERT model using external entity knowledge, a common approach in the literature. We employ a similar method as Meng et al. [17] to first generate gazetteer data (3.9m entities) for CoNLL and WNUT. We then apply the lexical matcher and the gazetteer data to create weakly-annotated parallel NER training data from CONLL and WNUT sentences. We train our NER models using BERT with these datasets. Specifically, we check whether a given sentence contains an entity entry from the gazetteer and create parallel entity annotation for the sentence. However, this gazetteer data is noisy, e.g.  Table 4: NER performance of supervised models for NER as a sequence generation task, compared against the SOTA for each dataset.
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
an entity entry can belong to multiple entity classes. Therefore, we do not use the gazetteer for CyclerNER.
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
We empirically assess CycleNER under the following scenarios: Scenario 1: Can the NER task be cast as a sequence generation task using our proposed sequence format (see Figure 2)?  (4) we use all the datasets, namely the different configurations (cf. Table 3). We measure NER performance using the micro-averaged F1 score.
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
In the first evaluation scenario, we assess how seq2seq models address NER. As mentioned in Section 4.1, given a sentence, the seq2seq model outputs the mentioned entities with their types. We mainly verify the suitability of casting NER as a sequence generation task, given that it is one of the fundamental principles in training CycleNER in an unsupervised manner. Table 4 shows the performance of a BiLSTM and T5 model trained on the entire dataset. The training is done in a supervised manner. Comparing to BiLSTM, we note that T5 has superior performance on both datasets 1 . This is attributed mainly to the sophisticated architecture of T5 and its extensive pre-trained knowledge, which allows it to better capture contextual information.
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
When comparing the T5 sequence model to the state-of-theart results, T5 achieves very similar results. For both CoNLL and WNUT, T5 has only a ∼3% drop in terms of F1. These results validate our hypothesis that NER can be cast as a sequence generation task and our entity sequence representation is effective.
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
In the standard supervised setting, training is stopped whenever a given loss function (e.g. cross-entropy) converges to a minimum. However, in CycleNER, training is done in an unsupervised manner. Hence, determining when the model, namely the two functions S2E and E2S, are fully trained is not trivial. To determine when training has converged, we analyze the relation of the two reconstruction losses that CycleNER minimizes: L θ from the E-cycle, and L ϕ from the S-cycle. As the two cycles are used to train the two functions, 1 For WNUT, BiLSTM produces poor results, hence, we omit the results from the table.  Table 5: F1 performance varying the number of training (a) entity sequences and (b) sentences. #S is the number of training sentences, #Q is the number of training entity sequences.
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
S2E and E2S, convergence in both cases is important to have stable and optimal performance for CycleNER.
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
We assess the best stopping criterion by training CycleNER separately on the CoNLL and WNUT datasets, and consider both entity sequence generation approaches (i.e. GQ and SQ). Namely, we use the following configurations for training: CoNLL/14k/1k GQ , CoNLL/14k/1k SQ , WNUT/3.4k/1k GQ , and WNUT/3.4k/1k SQ . The relatively small size of entity sequences allows us to quickly conduct experments. Figure 4 shows the reconstruction losses at different epochs for the S-(blue line) and E-(orange line) cycles on the development set (cf. Table 3). Alongside the loss values we plot the corresponding F1 scores. These high F1 scores (e.g. ∼0.8 for CONLL and ∼0.4 for WNUT) suggest that CycleNER is effective during training. Figure 4 shows a clear relationship between the loss computed in the E-cycle and F1. For CONLL, we obtain a high negative correlation as measured through Pearson's correlation coefficient, with ρ = −0.81 for GQ, and with ρ = −0.87 for SQ. For WNUT, we observe similar correlations, with a high negative correlation for GQ with ρ = −0.73, whereas a moderate correlation is observed for SQ with ρ = −0.59. Contrary to the E-cycle, there is no clear relationship between the S-cycle loss and F1, with an average correlation coefficient of ρ = 0.41 over CoNLL and WNUT, respectively for both GQ and SQ. This is because that the reconstructed sentence in the S-cycle can be different from the original one although they contain the same entities.
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
We conclude that the E-cycle loss can be used to to guarantee a optimal NER performance. Therefore, we use it as the stopping criterion for all subsequent tests described in Sections 7.3 and 7.4.
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
In this evaluation scenario, we assess the impact of the training data used for CycleNER. Table 5 shows the impact of training data for CoNLL and WNUT datasets, where we vary either both the number of sentences (S) or entity sequences (Q) used for training. Number of Entity Sequences. Table 5 (a) shows the performance of CycleNER when trained on a fixed number of sentences, namely 14k for CoNLL, and 3.4k for WNUT, while varying the number of entity sequences. At a high level we note the good performance of CycleNER using very small amounts of entity knowledge.
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
For CONLL, when using 100 GQ entity sequences, the model performs modestly, with F 1 = 0.619. However, when the number of entity sequences is increased to 1k, then we see a nearly 20% absolute improvement in terms of F1. Adding more entity sequences results in gradual performance increase, however, the rate of improvement is slightly lower. This finding is interesting, where for sentences that are of fairly similar text genres, with relatively little data, we can converge to the top performance using CycleNER. The performance of SQ is lower compared to GQ, and although more sequences enable better performance, this is not always the case, as it can be seen between 1k and 1.5k sequences. The difference between GQ and SQ can due to two main reasons. First, pre-trained embedding might not effectively capture the semantic meaning of an entity and find its similar ones. Second, given that CoNLL contains sentences from news corpora, the set of entities that co-occur in a sentence is often determined by newsworthiness factors, and are not correlated with entity relatedness solely.
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
Contrary to the CoNLL dataset, for WNUT, the performance difference between entity sequences generated according to GQ or SQ is marginal. A possible explanation for this difference w.r.t CoNLL, is that WNUT consists of text snippets coming from social media, and thus, the set of entities that co-occur in a sentence is much more diverse, and less controlled as in news media.
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
For both datasets, CycleNER is able to maintain reasonable effective NER performance with a small size of entity examples. In particular, CycleNER achieved 0.34 F1 (vs. T-NER's 0.585 [29]) using only 500 SQ for WNUT, where entities are from social media and are highly complex.
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
Number of Sentences. Table 5 (b) shows the performance of CycleNER when trained on a fixed set of entity sequences, namely 1k and 500, for CoNLL and WNUT, respectively, while at the same time varying the number of input sentences.
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
As in the case of varying entity sequences, adding sentences translates into better NER performance. For CONLL, the difference is nearly 2% absolute points improvement for GQ, and 7% for SQ sequences. While, in the case of WNUT, the improvements are with 5% and 7%, for GQ and SQ, respectively.
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
Although the gains are significant given the scale of the datasets, the improvements are more moderate than when varying entity sequences. This is intuitive, considering that in Table 5 (a) we show that with 1k entity sequences, the model achieves a performance that is close to its peak. At the same time, we note that the impact of additional sentences for training is much larger for SQ.
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
In summary, more data helps overall to improve the model's performance. While, when comparing additional sentences or entity sequences, we note that more entity sequences are more beneficial for CycleNER, which allows the model to better learn the NER task. Table 6 shows the results of the different NER approaches defined in Section 6.1. Apart from BERT and BERT-Matcher, which perform NER in the standard token classification setting, the rest of the approaches are unsupervised. The SOTA row reports the performance for each dataset from existing work. We also report the results obtained by KALM on the CoNLL dataset, as reported in [15].
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
It is worth noting that NeuralHMM is trained only using sentences, whereas CycleNER is trained with a fairly larger amount of entity sequences for SQ, while for GQ this amount is much smaller. More specifically, for CoNLL we use 1k GQ sequences (representing 7% of the total sequences), 1k for WNUT (29% of the original size), 5k for Ontonotes (4.3% of the original size), and 1k for BC2GM (8% of the original size).
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
Performance Comparison. In all cases, BERT achieves the best performance. This is intuitive given that it uses annotated data at the token level, and thus, the loss is optimized at the token level, allowing the model to achieve optimal performance. However, as mentioned in the motivation of this work, obtaining such annotated data is not always feasible and can be costly. On the contrary, BERT-Matcher (trained using gazetteer-based weakly-annotated data) performs worst. This is because the weakly-annotated data is too noisy.
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
When comparing the performance of CycleNER and BERT across datasets, we note that the differences are not high. For instance, when CycleNER is trained on CoNLL/2k/1k GQ , the difference is 3.5%, which when considered that our approach is unsupervised presents remarkable results. The gap is higher, with 17.4%, when we use CoNLL/14k/14k SQ for training. As noted in Section 7.3,  Table 6: Comparison of the performance of the different approaches. The first column in each dataset sub-table represents the dataset used to train the respective models. Refer to Table 3 for the datasets used by CycleNER and Lexical Matcher. For the supervised BERT baseline, we report the size of the parallel data.
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
co-occurrence of entities in the news domain is determined by the newsworthiness of entities. Hence, SQ sequences, may introduce infrequent sequences in news. We note similar observations across the different datasets, such as WNUT, OntoNotes, where the gap is 7%, and 13.5%, respectively. For both datasets, the gap between GQ and SQ entity sequences is much smaller, and in some cases SQ sequences allow the model to have better performance, as is the case for WNUT.
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
On the BC2GM dataset, the gap between BERT and CycleNER increases. This is due to the fact that the T5 model does not contain sufficient pre-trained knowledge, and hence cannot correctly recognize entity boundaries for training S2E and E2S. Often surface forms (e.g. "HKE6") of named entities in BC2GM are tokenized into many subword units, due to the fact that such tokens are not present in the pretraining corpus of T5, resulting in much poorer performance. In this case, we note that since BERT performs token level classification, although far from the state of the art performance, the model can better learn to predict the correct NER class for each token.
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
Finally, the performance of NeuralHMM and Lexical Matcher is quite poor, which emphasizes the difficulty of conducting this task in an unsupervised fashion. For many of the datasets, e.g. WNUT, OntoNotes, NeuralHMM performs extremely poorly, hence, the results are omitted from the table. Liu et al. [15] report that their unsupervised KALM model can achieve a score of F1=0.86 for CoNLL, while our CycleNER achieves a comparable score of F1=0.83. Considering that CycleNER does not leverage large knowledge bases, its performance is still competitive using only a small entity sample.
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
Unseen Entities. When trained on large data (e.g. OntoNotes), CycleNER learns to extract new patterns of named entities and new entity classes that are not annotated in the original ground-truth. Figure 5 shows the output of CycleNER, which predicts that "festival" is an event, "riverside park" as a location, and "black pearl wax apples" as a product. Despite the fact that these words were not annotated in the ground-truth, CycleNER determines that they are entities based on the context in which they appear, and on similar entities appearing in the ground-truth. This shows that unsupervised NER and seq2seq offers several advantages that can circumvent annotation qualities present in an NER dataset.  Error Analysis. The most frequent errors in CycleNER are related to span detection. This is especially the case when the input sentence contains no entities. In such scenarios, S2E attempts to generate entity sequences with at least one entity.
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
To better understand this type of behavior, we exclude from the test set all sentences that contain no entities and re-calculate the performance metrics. This results in a 4% absolute increase in terms of F1 for CoNLL, and 8-12% for the other three datasets. Comparatively, BERT as the best performing baseline, sees an increase of 3% for CoNLL and 2-5% for the rest of the datasets. This is a limitation in our work, as sentences without entities impact both S2E and E2S training. There are two possible directions to address this issue: First, for sentences without entities, S2E can be trained to generate a special token. One drawback of such an approach is to reconstruct back the input sentence in E2S, which would lack the context to map a single special token to all the possible sentences that have no entities. This prerequisite is described in Guo et al. [6] where a bijective mapping between the two modalities is necessary. We aim to solve this issue in future work.
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
We presented CycleNER, a novel unsupervised NER approach that uses cycle-consistency training to learn an effective mapping between sentences and entities in two cycles by training S2E and E2S. During the two cycles, the model learns how to reconstruct sentences and entities jointly and interactively. To our best knowledge, this is the first time that cycle-consistency has been applied for NER.
CycleNER: An Unsupervised Training Approach for Named Entity Recognition
Through extensive experimental evaluations, we confirm the effectiveness of CycleNER on four different datasets, covering different domains (BC2GM, medicine), and data quality (WNUT). To ensure optimal training, we devise a stopping criterion that relies on the loss of E-cycle, which on most of the datasets has a high negative correlation with the actual NER performance. We showed that CycleNER achieves competitive performance w.r.t supervised and baselines like BERT. On different datasets, such as CoNLL, Cy-cleNER reaches 72.7% performance of current state of the art. Since CycleNER only requires sentences and entity examples, it has advantages on recognizing emerging and difficult entities, compared to existing unsupervised baselines relying on external knowledege. In practice, CycleNER can be easily applied to develop an initial NER model and scale up the model by providing more entity examples.
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting
Information Retrieval and Question Answering (QA) systems aim to fulfill user requests, from traditional Web search to voice assistants. Users can issue queries in two forms (White et al., 2015): as keyword queries ("iphone 14 price") or as fullyformed natural language questions ("how much is an iPhone 14?"). Natural Language Understanding (NLU) components are expected to correctly interpret semantically equivalent queries, regardless of the form. Figure 1 (a) shows examples of queries with the same intent, but in different forms. 1 https://github.com/amzn/kqr How much is iPhone 13? What is the price of iPhone 13? How much does iPhone 13 cost?
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting
iPhone 13 cost, cost of iPhone 13 iPhone 13 price, price of iPhone 13 iPhone 13 offer, iPhone 13deal
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting
The prices for iPhone 13 vary according to storage: $799 for 128GB. Keyword-question rewriting for disambiguation.
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting
Work on question paraphrases has grown recently, with current approaches utilizing encoderdecoder architectures for generating questions (Hosking and Lapata, 2021). However, NLU systems that handle different input forms must support two important functionalities beyond questions:
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting
• Understanding: Correctly interpret different query forms (cf. Figure 1 (a)), and create equivalent semantic representations, regardless of the query form, style, or other lexical variations.
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting
• Expression: Rewrite queries into different forms (e.g. keywords into questions). This increases transparency by relaying the query interpretation back to the users, or to ask clarification questions (Kiesel et al., 2018) to ensure that their requests are correctly understood (cf. Figure 1 (b)).
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting
To address these, we introduce the keywordquestion rewriting task, which aims to rewrite questions into semantically equivalent keyword queries, and vice versa. This task enables training models for the above desiderata, allowing NLU systems to handle either keywords or questions. Further, this capability enables asking clarification questions about user's input query (Ding and Balog, 2018), (cf. Figure 1 (b)), by rewriting it into a question.
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting
While previous work has tackled the problem of rewriting keywords into questions (Zhao et al., 2011;Bhagat and Hovy, 2013), it focuses on unidirectional (keyword → question) generation using supervised datasets. We propose a bidirectional and unsupervised approach in this paper.
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting
Specifically, we propose CycleKQR, an approach based on cycle-consistent training (Lample et al., 2017;Iovine et al., 2022), which is an unsupervised approach that effectively maps two non-parallel sets of keywords and questions via two cycles, keywords-to-question (K2Q) and questionto-keywords (Q2K). These two cycles learn interactively from each other, and K2Q & Q2K are trained using reconstruction losses. Although unsupervised, CycleKQR is more robust since the its rewriting models are trained simultaneously, while a supervised approach has to train both models in two rounds. As there are no existing datasets for keyword-question rewriting we also construct and release a dataset of 66k keyword-question pairs.
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting
We focus on (i) intrinsic and (ii) extrinsic evaluation strategies. For intrinsic evaluation, we measure the bidirectional rewriting quality of competing approaches using human studies and automated metrics such as ROUGE and BLEU.
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting
Our extrinsic evaluation focuses on QA systems, where the ability to correctly interpret the user query is critical, and if necessary, query rewriting approaches are applied to ensure answer accuracy (Vakulenko et al., 2021;Elgohary et al., 2019). More concretely, we measure the downstream impact on QA performance using the rewritten keyword or question queries. 2 We assess the impact on two types of QA systems: open domain and Knowledge Base QA systems.
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting
Experiments show that QA systems perform poorly on query forms that they are not trained on. We show that CycleKQR can effectively rewrite between keyword and question queries without annotations, and at the same time improve QA performance. CycleKQR obtains QA performance improvements of up to 3% over a supervised baseline using the same neural architecture. This is due to the ability to cope with noisy input and target query pairs, given that CycleKQR computes a reconstruction loss in each cycle, thus allowing it more flexibility in the rewriting step, and avoiding overfitting on noisy pairs.
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting
In sum, we make the following contributions: • Introduce a keyword-question rewriting task for improved query understanding; • Propose CycleKQR, an unsupervised approach for keyword-question rewriting; • Release a publicly available dataset with 66k pairs for keyword-question rewriting.
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting
We define the task of rewriting keywords into question queries and vice-versa through two rewriting functions: (i) keywords-to-question (K2Q), and (ii) question-to-keywords (Q2K). Let Q be a set of natural language questions, and K a set of keyword queries. K2Q rewrites a keyword query k ∈ K into a question q ′ ∈ Q, i.e. K → Q. Similarly, its inverse function, Q2K, rewrites a question into its corresponding keyword query, Q → K. For example, given the keyword query "houston tx county", K2Q will output the question "What county is Houston TX in?". Conversely, given the question "Where was Martin Luther King born?", Q2K will output "martin luther king birthplace". Both K2Q and Q2K must preserve the semantic meaning of the original input into the generated output, i.e. both the input and output must convey the same information need.
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting
for Keyword-Question Rewriting Figure 2 shows an overview of our approach. We now describe how we use cycle-consistency to train K2Q and Q2K for keyword-question rewriting.
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting
Overview: For the K2Q and Q2K, CycleKQR uses two encoder-decoder models (Lewis et al., 2020) with parameters denoted as θ and ϕ. To train them, we follow Iterative Back-translation (Hoang et al., 2018), in which the output of each model is used to generate intermediate training data for the other. The core idea behind cycle-consistent training is to jointly train the K2Q and Q2K functions, allowing CycleKQR to align the latent spaces of keyword and question queries. In the following, we describe the two training cycles.
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting
The input data is the set of training questions q ∈ Q. For a question q, in the first step, called generation step (cf. Figure 2 (a)) we generate an intermediate keyword query by passing q as input to the Q2K model, generating
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting
where q ′ is supposed to be semantically similar to q. This completes the Q-cycle, where the training is guided by the reconstruction loss as the average cross-entropy between q and q ′ , that is backpropagated to the K2Q model:
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting
where, p and g represent the real and predicted token probabilities, |q| is the question length, |Q| is the total number of training questions, and q i and q ′ i are the i-th token of q and q ′ , respectively. K-Cycle: K-cycle represents the opposite direction of Q-cycle. For an input keyword query k, K2Q outputs the intermediate question q ′ = K2Q(k). Similarly, the ⟨k, q ′ ⟩ pair is used to train Q2K, which generates the back-translated keyword query k ′ = Q2K(q ′ ). Here too, the reconstruction loss is used to train Q2K model:
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting
where, |k| is the keyword query length, |K| is the number of keyword queries, and k i and k ′ i represent the i-th token in the original and back-translated keyword queries, respectively.
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting
Training: The two cycles are repeated iteratively. First, we run one iteration of Q-cycle with one batch of training questions, followed by an iteration of K-cycle using one batch of training keyword queries. This alternating process in CycleKQR is repeated until both models converge. Both cycles are instrumental in the learning process: Q-cycle trains K2Q to reconstruct a natural question q from a given keyword query k ′ , while K-cycle trains Q2K to generate a keyword query k from a question q ′ . Preliminary experiments confirm that this training process effectively trains K2Q and Q2K to generate the correct keyword and question queries even on the first iterations. As training continues, the distribution of the intermediate data generated by K2Q and Q2K becomes closer to that of the real keyword and question queries.
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting
Next, we describe our method for extracting the keyword-question rewriting dataset, namely how we construct the non-parallel sets of keyword and question queries used to train CycleKQR as well as the supervised pairs used for training the QA models and testing all competing approaches.
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting
Our approach relies on non-parallel keyword and question queries for training, which we extract from the MS-MARCO (Bajaj et al., 2018) and OR-CAS (Craswell et al., 2020) datasets, consisting of natural language questions (1M instances) and keyword queries (10M instances), respectively.
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting
We apply several filtering criteria to generate the training corpus. First, questions from MS-MARCO are filtered using a strategy proposed in (Dror et al., 2013), selecting questions that are syntactically correct, typically starting with a wh-word or an auxiliary verb, and contain at least five tokens. Next, keyword queries from the ORCAS dataset are filtered such that they do not start with a wh-word or auxiliary verb, and contain less than three words.
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting
To ensure that questions and keyword queries share a similar domain, we compute the cosine similarity across all pairs, and remove those with a similarity less than 0.6. Finally, we randomly sample two separate sets of 100k questions and 100k keyword queries, thus ensuring they are nonparallel. We call this the non-parallel corpus (cf .  Table 1), and we use it to train CycleKQR.
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting
As we are the first to propose the bidirectional rewriting of keyword and question queries, we collect a corpus of aligned keyword and question pairs, used to evaluate all the competing rewriting approaches. The parallel corpora come from two main domains: (i) Open-domain QA, and (ii) Knowledge Base QA. Additionally, these two domains represent our extrinsic evaluation strategy, which measures the impact of query rewriting on the respective QA systems.
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting
As in §5.1, we leverage MS-MARCO and ORCAS to construct a parallel corpus of keyword and question pairs. While questions in MS-MARCO are associated with an answer, keyword queries in OR-CAS are not. The answer is key in establishing an alignment between the keyword queries and questions that point to the same answer, hence, sharing the same information need.
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting
To align the keyword and question pairs with high accuracy, we propose the following steps.
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting
Filtering We apply the same filtering process described in §5.1, ensuring that questions are syntactically in the correct natural language question form, and similarly keywords do not contain natural language questions. The result is a set of 632k questions and 7M keyword queries. Question to Keyword Matching We next compute the similarity between each question and all keyword queries (Cer et al., 2018). Then, we associate a question to its closest matching keyword query. Due to the low semantic overlap between MS-MARCO questions and ORCAS queries, the extracted pairs are noisy. There is no guarantee that for each question, a keyword query with the same answer is always available. To avoid adding noisy matching pairs into the dataset, we filter pairs by setting a minimum semantic similarity threshold of 0.8, discarding all pairs with lower similarity. Since a natural language question usually expresses a clear request, matching from question to keyword query also helps remove ambiguous keyword queries. We obtain 62k keyword-question pairs, 38k of which have an answer in MS-MARCO's passage collection.
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting
Manual Alignment To validate and further improve the quality of the generated parallel data, we conduct a human evaluation. For 11k question and keyword pairs, we asked annotators whether the question and keyword query express the same information need. Results showed that 87% (9.1k) of the pairs were deemed to have the same information need. We filtered out the remaining 13% pairs, and the high-quality ones were then divided 3 into two equal sets for testing and validation. The remaining 27k pairs are used for training. We call this set MS-pairs as shown in Table 1.
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting
For our final dataset, we rely on WebQuestionsSP (Yih et al., 2016), a popular dataset for Knowledge Base QA, containing 4.7k question-answer pairs. Given its small size, we manually extract the keyword queries from the questions. Keyword extraction was done by proficient annotators with domain expertise, ensuring high data quality. For a given question, the annotators were asked to generate a keyword query expressing the same information need. The annotated data is split into 2k, 1k and 1.6k for training, validation, and test, denoted as QSP-pairs in Table 1. Note that all QSP-pairs are associated with answers.
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting
6 Experimental Setup
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting
We evaluate the different competing approaches based on two evaluation strategies.
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting
Intrinsic: Approaches are directly evaluated in terms of correctness and similarity of the rewritten keyword or question query compared to the groundtruth target, as measured through automated metrics such as ROUGE and BLEU, following a similar strategy to Ding and Balog (2018). Additionally, we carry out a small scale analysis using human evaluation to assess the preference of annotators on the rewritten queries for the competing approaches.
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting
Extrinsic: We measure the downstream impact of rewriting keyword and question queries into the appropriate form for a given QA system. We distinguish between two cases: (1) Question-based QA (QuA), for natural questions; and (2) Keywordbased QA (KeA) for keyword queries. The two cases represent common applications in real-world, e.g., Google and Alexa voice assistants answer natural questions, while many retrieval systems are keyword-based. Both QuA and KeA are treated as black boxes (the QA systems cannot be modified). This evaluation shows the utility of rewriting keyword and question queries into the appropriate form for a target QA, when compared to issuing keyword queries to QuA or questions to KeA. In terms of evaluation metrics, for the open-domain QA scenario, we report MRR and recall metrics, which are the standard evaluation metrics employed in the MS-MARCO Passage Ranking challenge (Choi et al., 2021), whereas for Knowledge Base QA, we report the F1 metric (Ye et al., 2021).
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting
For the extrinsic evaluation, we choose two state-ofthe-art QA systems as the basis for QuA and KeA. In the open-domain setting, we used the BM25 + BERT-base model (Liu et al., 2021), whereas for the Knowledge Base setting, we used RnG-KBQA (Ye et al., 2021), composed of a BERTbased ranker and a T5-based generation model.
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting
We train four QA systems:
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting
(1) open-domain QuA, trained using the complete question-answer pairs from MS-MARCO;
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting
(2) open-domain KeA, trained using 27k keyword-answer pairs from MSpairs; (3) Knowledge Base QuA and (4) KeA, trained using 2k question-answer and keywordanswer pairs from QSP-pairs.
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting
Our approach -OURS: We train two versions of CycleKQR, both using the T5 encoder-decoder model (Raffel et al., 2019) (2) combining non-parallel and QSP-pairs (a more realistic scenario), resulting in a level of noise that is comparable to the open-domain setting (OURS R ). The two configurations allow us to assess whether our approach benefits from additional training data despite higher noise.
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting
Supervised Baseline -SB: Here we train K2Q and Q2K without cycle-consistent training. This corresponds to a standard supervised approach, where each input is paired with an output. For this purpose, we use the questions and keyword queries from Non-parallel, which we automatically pair using the semantic similarity strategies described in §5.2. We follow the same strategy as in OURS and train three separate models: SB for open-domain QA, SB C and SB R for Knowledge Base QA.
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting
Semantic Similarity -SIM: It replaces questions with keyword queries and vice versa, following the same strategy we used to match questions to keyword queries (see §5.2). Since the MS-pairs test set was generated by pairing together highsimilarity questions and keyword queries, it is biased in favor of this baseline: for each question, it will simply retrieve the paired keyword query, and vice versa. To avoid this bias, we created an auxiliary test set specifically for evaluating SIM. For QuA we randomly sampled 112 keyword queries from ORCAS, and asked experts to manually assign an answer from the MS-MARCO passage corpus. Each query is matched to its most similar question from MS-MARCO. For KeA, we randomly sampled 7k different questions from MS-MARCO, and matched them to the keyword queries in OR-CAS. Differently from the data collection in §5, the pairs were not filtered in any way. The results obtained using this auxiliary set are labeled with the A subscript, e.g. OURS A and SIM A . Note that no auxiliary test set is needed for the Knowledge Base QA setting, since the QSP-pairs test set is generated using a different approach.
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting
Upper bound -UPPER: The QA system receives ground-truth queries in their correct form (ideal scenario), e.g., questions to QuA, keywordbased queries to KeA.
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting
Lower bound -LOWER: Contrary to UPPER, the QA system receives the query in an unfamiliar form, keyword queries to QuA, questions to KeA. 5 A subset of the NLTK stop word corpus was used. We kept wh-words, as they are important for query understanding.    Table 2 reports ROUGE and BLEU scores for the competing approaches calculated on the test sets of MS-pairs and QSP-pairs. Results show that K2Q is harder than Q2K. Both OURS and SB have the highest scores, with SB obtaining higher scores. The higher scores of SB are intuitive, given that the approach is trained in a supervised manner, and it is optimized to generate the target output query. Note that SB can potentially overfit on its training data, and can be affected by noisy pairs generated by the automatic matching process. This is not necessarily the case for OURS, due to its cycleconsistent training. However, automated natural language generation metrics such as ROUGE and BLEU, cannot capture possible lexical variability in rewriting, which may lead to lower scores.
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting
To verify this, through a user study we evaluated with domain experts the rewriting quality. Given an input query, the annotator is asked to choose the better rewrite in terms of semantic preservation and syntactic correctness among OURS and SB. 6 Table 3 shows the annotator preferences for the rewrites on 100 questions (50 each extracted from MS-pairs and QSP-pairs) and 100 keyword queries (same as above). On both QA domains, rewrites from OURS are significantly 7 preferred over SB.
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting
Results from this section validate RQ1. Table 2 showed that OURS achieves high automated metric scores, whereas Table 3 shows that our rewrites are significantly preferred by human annotators. Examples from our models are shown in Appendix A. Cycle-consistent training allows our K2Q model to keep the keyword query intent, while the weaklysupervised K2Q may change its meaning, due to the noisy training data.
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting
Q2K for KeA. Unlike for K2Q, here UPPER obtains lower scores than LOWER for both MRR and recall. Although OURS significantly improves QA performance over SB (increase of +10.4% for MRR@10), it does not show improvement over LOWER. Theoretically, the performance of UPPER should be always better than that of LOWER, since KeA is trained to fit keyword queries in both recall and MRR. Two factors can be attributed to this outcome: (1) Our KeA is trained using BERT, which interprets better questions even after being fine-tuned on keywords; 9 and (2) the 7 Measured per Wilcoxon signed-rank test with p = 0.034 8 Our MRR scores are lower than those in MS-MARCO leaderboard. For comparison, the pre-trained OpenMatch model (state-of-the-art) obtains MRR@10=0.235 on MS-pairs test set. We conclude that the performance difference is due to the use of a different test set. 9 We tried to verify this by training an LSTM model, however, it did not obtain reasonable rewriting performance. test data may contain noise that prevents us from obtaining accurate results. Despite this, Table 4 shows that OURS outperforms all other baselines and LOWER in recall.
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting
We also compare our approach against a simple strategy based on semantic similarity (SIM), shown as OURS A and SIM A in Table 4. OURS outperforms SIM since SIM heavily relies on having the correct question/keyword for each input. When such a question/keyword query is not available, SIM will inevitably fail. The poor performance of SIM also confirms the low overlap between MS-MARCO questions and ORCAS keyword queries discussed in §5.2: despite the size of MS-MARCO and ORCAS, and the effectiveness of USE embeddings, a semantically-equivalent question/keyword query is often missing. This also justifies the filtering and human annotation steps applied to ensure the quality of the MS-pairs test set.
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting
Open-domain QA Rewriting Examples. Given the keyword query "student loans without a cosigner" shown in Table 5 (a), OURS can reformulate it into "how to get student loans without a cosigner?".
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting
The rewritten question improves the ranking of the correct passage from the 14th to 7th rank, matching UPPER. SB on the other hand, generates the question "how do you get a cosigner on student loans?", which lowers performance as the meaning of the request has been changed.
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting
For the question "how to eject disc from hp laptop?" in Table 5 (b), OURS generates "eject disc from hp laptop". This simple transformation improves the ranking of the correct answer from 17th to 6th position, while SB removed important information (brand of the laptop).
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting
Here, we show the QA performance on two settings, which vary on the data used to train OURS and SB:
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting
(1) clean setting, where the models (OURS C and SB C ) are trained using 2k training QSP-pairs, and (2) realistic scenario, which additionally adds 100k keyword and question queries from the open domain QA for training OURS R and SB R . Note that this realistic setting contains noise.
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting
K2Q for QuA. We again observe that UPPER has much higher performance, with F1=0.762 compared to LOWER with F1=0.698. It further confirms that an unfamiliar query form causes low QA performance. All OURS models    Table 6: Knowledge Base QA results. The best results are highlighted in bold. * means that at least one of our approaches scores significantly higher than this baseline. perform close to UPPER and better than SIM and LOWER. In both clean and realistic settings, we observe that OURS performs better than SB, with a 2.7% relative improvement in the latter.
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting
Q2K for KeA. Similarly, we obtain positive results in KeA. In the clean setting, OURS C obtains F1=0.732, and outperforms all baselines, except for SB C , which obtains F1=0.736 (insignificant difference). This is intuitive, given that SB C has access to accurate paired training data. On the other hand, OURS R performs better than the supervised baseline with F1=0.716 compared to SB R , which obtains F1=0.685, a 7.5% relative improvement. This indicates that our approach is more resilient to noise compared to the supervised one. This is because the objective of cycle-consistency training is to learn the semantic representations of questions and keyword queries by reconstructing them from noisy data, rather than comparing against a specific target output.
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting
Rewriting in Knowledge Base QA. Table 7 (a) shows an example query, "emily osment highschool", and its rewrites. OURS infers the question "where did emily osment go to highschool?", matching the performance of SB C .
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting
On the other hand, the noisy training data caused SB R to change the intent of the input, while SIM failed to find a matching question from MS-MARCO. Table 7 (b) shows the example question "who owns chrysler corporation 2011?", which is correctly understood by OURS, rewriting it into the keywords "chrysler corporation owner 2011". This example confirms that Q2K is a non-trivial task, differing from simple keyword extraction: the rewritten keywords are not simply a subset of the input, but a coherent query in which linguistic expressions must also be converted into appropriate keywords.
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting
Finally, the results in §7.2.2 and §7.2.1 validate RQ2, showing that rewriting queries into their appropriate form has a significant impact on QA performance. Furthermore, rewritten queries from OURS trained based on cycle-consistent training outperforms strong baselines such as SB.
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting
We introduced the bidirectional keyword-question rewriting task, which improves QA performance by rewriting queries into the desired form for a given QA system. Furthermore, we presented Cy-cleKQR, which learns the K2Q and Q2K functions simultaneously through cycle-consistent training in an unsupervised manner. CycleKQR allows for the two rewriting functions to be less susceptible to noise coming from unsupervised data, and can be easily adapted to different QA systems and scenarios, for which non-parallel data is available.  We carried out a detailed evaluation of different competing rewriting approaches for two QA scenarios (Open Domain and Knowledge Base), assessing their performance through an intrinsic and extrinsic evaluation. Experimental results show that rewriting queries into the correct form improves QA performance, and that CycleKQR is able to provide highly accurate rewrites, which retain the original query intent and improve the ranking of the correct answer for the underlying QA system.
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting
Finally, we contributed a keyword-question rewriting dataset, consisting of a total of 66k keyword-question pairs which can be used to facilitate research on keyword-question rewriting.
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting
A notable limitation of this work is that we do not explicitly investigate how to handle ambiguous keywords (vague or broad queries that could be mapped to multiple different questions) and paraphrases (Q-K pairs with equivalent meaning but different surface forms). Both of these are active research areas in the community. We addressed this with the simplifying assumption introduced in §5.1 that sets a minimum semantic similarity threshold between our Q and K data, as determined by an existing sentence encoder. We believe that this is a key future direction in this line of research. The ambiguity issues requires the detection of such keyword queries, and specific mechanisms to generate diverse question interpretations. The paraphrasing problem can be addressed by diversifying the non-parallel data selection process. Furthermore, our extrinsic evaluation focused on K2Q for QA under the assumption that bidirectional cycle training leads to improved K2Q performance, and we did not include a specific extrinsic application of Q2K. One example of such an application is query expansion for questions, where the original question can be augmented with additional keywords coming from a Q2K model that uses diversity-based or lexically-constrained sampling to generate keywords that are not in the question itself. Due to space limitations, we leave the exploration of this direction for future work.
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting
In the current work we did not consider a supervised cycle-training setting. While the unsupervised setting allows us to leverage non-parallel data, we believe that supervised fine-tuning (even in a few-shot setting) could deliver further improvements. We leave this for future work.
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting
Finally, for deploying our proposed approach in real-world scenarios (e.g. in a search engine or voice assistant), we must consider the task of how to decide when to apply either Q2K or K2Q to an input. This is relevant to our bidirectional approach, as well as previous work on unidirectional transformations. We did not discuss this component in the paper for reasons of space, but we do not believe that this is a significant challenge. The most straightforward approach is to employ a question classifier to determine if an input is a fully-formed question, and use this information to decide whether to apply Q2K or K2Q. Such an approach is common in practical settings, and other heuristics such as input length are also commonly used. We plan to further explore this area, including empirical evaluation of binary classification of K vs. Q, as part of future work.     
Unsupervised Evaluation Metrics and Learning Criteria for Non-Parallel Textual Transfer
We consider textual transfer, which we define as the capability of generating textual paraphrases with modified attributes or stylistic properties, such as politeness (Sennrich et al., 2016a), sentiment (Hu et al., 2017;Shen et al., 2017), and formality (Rao and Tetreault, 2018). An effective transfer system could benefit a range of user- § Work completed while the author was a student at the University of Chicago and a visiting student at Toyota Technological Institute at Chicago. facing text generation applications such as dialogue (Ritter et al., 2011) and writing assistance (Heidorn, 2000). It can also improve NLP systems via data augmentation and domain adaptation.
Unsupervised Evaluation Metrics and Learning Criteria for Non-Parallel Textual Transfer
However, one factor that makes textual transfer difficult is the lack of parallel corpora. Advances have been made in developing transfer methods that do not require parallel corpora (see Section 2), but issues remain with automatic evaluation metrics.  used crowdsourcing to obtain manually-written references and used BLEU (Papineni et al., 2002) to evaluate sentiment transfer. However, this approach is costly and difficult to scale for arbitrary textual transfer tasks.
Unsupervised Evaluation Metrics and Learning Criteria for Non-Parallel Textual Transfer
Researchers have thus turned to unsupervised evaluation metrics that do not require references. The most widely-used unsupervised evaluation uses a pretrained style classifier and computes the fraction of times the classifier was convinced of transferred style (Shen et al., 2017). However, relying solely on this metric leads to models that completely distort the semantic content of the input sentence. Table 1 illustrates this tendency.
Unsupervised Evaluation Metrics and Learning Criteria for Non-Parallel Textual Transfer
We address this deficiency by identifying two competing goals: preserving semantic content and producing fluent output. We contribute two corresponding metrics. Since the metrics are unsupervised, they can be used directly for tuning and model selection, even on test data. The three metric categories are complementary and help us avoid degenerate behavior in model selection. For particular applications, practitioners can choose the appropriate combination of our metrics to achieve the desired balance among transfer, semantic preservation, and fluency. It is often useful to summarize the three metrics into one number, which we discuss in Section 3.3.
Unsupervised Evaluation Metrics and Learning Criteria for Non-Parallel Textual Transfer
We also add learning criteria to the framework of Shen et al. (2017) to accord with our new metrics. We encourage semantic preserva-tion by adding a "cyclic consistency" loss (to ensure that transfer is reversible) and a loss based on paraphrase pairs (to show the model examples of content-preserving transformations). To encourage fluent outputs, we add losses based on pretrained corpus-specific language models. We also experiment with multiple, complementary discriminators and find that they improve the trade-off between post-transfer accuracy and semantic preservation.
Unsupervised Evaluation Metrics and Learning Criteria for Non-Parallel Textual Transfer
To demonstrate the effectiveness of our metrics, we experiment with textual transfer models discussed above, using both their Yelp polarity dataset and a new literature dataset that we propose. Across model variants, our metrics correlate well with human judgments, at both the sentencelevel and system-level.
Unsupervised Evaluation Metrics and Learning Criteria for Non-Parallel Textual Transfer
Prior work in automatic evaluation of textual transfer has focused on post-transfer classification accuracy ("Acc"), computed by using a pretrained classifier to measure classification accuracy of transferred texts (Hu et al., 2017;Shen et al., 2017). However, there is a problem with  relying solely on this metric. Table 1 shows examples of transferred sentences at several points in training the model of Shen et al. (2017). Acc is highest very early in training and decreases over time as the outputs become a stronger semantic match to the input, a trend we show in more detail in Section 6. Thus transfer quality is inversely proportional to semantic similarity to the input sentence, meaning that these metrics are complementary and difficult to optimize simultaneously. We also identify a third category of metric, namely fluency of the transferred sentence, and similarly find it to be complementary to the first two. These three metrics can be used to evaluate textual transfer systems and to do hyperparameter tuning and early stopping. In our experiments, we found that training typically converges to a point that gives poor Acc. Intermediate results are much better under a combination of all three unsupervised metrics. Stopping criteria are rarely discussed in prior work on textual transfer.
Unsupervised Evaluation Metrics and Learning Criteria for Non-Parallel Textual Transfer
We now describe our proposals. We validate the metrics with human judgments in Section 6.3.
Unsupervised Evaluation Metrics and Learning Criteria for Non-Parallel Textual Transfer
This metric was mentioned above. We use a CNN (Kim, 2014) trained to classify a sentence as being from X 0 or X 1 (two corpora corresponding to different styles or attributes). Then Acc is the percentage of transferred sentences that are classified as belonging to the transferred class.
Unsupervised Evaluation Metrics and Learning Criteria for Non-Parallel Textual Transfer
Semantic Similarity ("Sim"): We compute semantic similarity between the input and transferred sentences. We embed sentences by averaging their word embeddings weighted by idf scores, where idf(q) = log(|C| · |{s ∈ C : q ∈ s}| −1 ) (q is a word, s is a sentence, C = X 0 ∪ X 1 ). We use 300-dimensional GloVe word embeddings (Pennington et al., 2014). Then, Sim is the average of the cosine similarities over all original/transferred sentence pairs. Though this metric is quite simple, we show empirically that it is effective in capturing semantic similarity. Simplicity in evaluation metrics is beneficial for computational efficiency and widespread adoption. The quality of transfer evaluations will be significantly boosted with even such a simple metric. We also experimented with METEOR (Denkowski and Lavie, 2014). However, given that we found it to be strongly correlated with Sim (shown in supplemental materials), we adopt Sim due to its computational efficiency and simplicity.
Unsupervised Evaluation Metrics and Learning Criteria for Non-Parallel Textual Transfer
Different textual transfer tasks may require different degrees of semantic preservation. Our summary metric, described in Section 3.3, can be tailored by practitioners for various datasets and tasks which may require more or less weight on semantic preservation.
Unsupervised Evaluation Metrics and Learning Criteria for Non-Parallel Textual Transfer
Fluency ("PP"): Transferred sentences can exhibit high Acc and Sim while still being ungrammatical. So we add a third unsupervised metric to target fluency. We compute perplexity ("PP") of the transferred corpus, using a language model pretrained on the concatenation of X 0 and X 1 . We note that perplexity is distinct from fluency. However, certain measures based on perplexity have been shown to correlate with sentence-level human fluency judgments (Gamon et al., 2005;Kann et al., 2018). Furthermore, as discussed in Section 3.3, we punish abnormally small perplexities, as transferred texts with such perplexities typically consist entirely of words and phrases that do not result in meaningful sentences. Our summary metric, described in Section 3.3, can be tailored by practitioners for various datasets and tasks which may require more or less weight on semantic preservation.
Unsupervised Evaluation Metrics and Learning Criteria for Non-Parallel Textual Transfer
It is often useful to summarize multiple metrics into one number, for ease of tuning and model selection. To do so, we propose an adjusted geometric mean (GM) of a generated sentence q:
Unsupervised Evaluation Metrics and Learning Criteria for Non-Parallel Textual Transfer
where t = (t i ) i∈ [4] , and [·] + = max(·, 0). Note that as discussed above, we punish abnormally small perplexities by setting t 4 .
Unsupervised Evaluation Metrics and Learning Criteria for Non-Parallel Textual Transfer
When choosing models, different practitioners may prefer different trade-offs of Acc, Sim, and PP. As one example, we provide a set of parameters based on our experiments: t = (63, 71, 97, −37). We sampled 300 pairs of transferred sentences from a range of models from our two different tasks (Yelp and literature) and asked annotators which of the two sentences is better. We denote a pair of sentences by (y + , y − ) where y + is preferred. We train the parameters t using the following loss:
Unsupervised Evaluation Metrics and Learning Criteria for Non-Parallel Textual Transfer
In future work, a richer function f (Acc, Sim, PP) could be learned from additional annotated data, and more diverse textual transfer tasks can be integrated into the parameter training.
Unsupervised Evaluation Metrics and Learning Criteria for Non-Parallel Textual Transfer
The textual transfer systems introduced below are designed to target the metrics. These system variants are also used for metric evaluation. Note that each variant of the textual transfer system uses different components described below.
Unsupervised Evaluation Metrics and Learning Criteria for Non-Parallel Textual Transfer
Our model is based on Shen et al. (2017). We define y ∈ R 200 and z ∈ R 500 to be latent style and content variables, respectively. X 0 and X 1 are two corpora containing sentences x (i) 0 and x (i) 1 respectively, where the word embeddings are in R 100 . We transfer using an encoder-decoder framework. The encoder E : X × Y → Z (where X , Y, Z are sentence domain, style space, and content space, respectively) is defined using an RNN with gated recurrent unit (GRU; Chung et al., 2014) cells. The decoder/generator G : Y × Z → X is defined also using a GRU RNN.
Unsupervised Evaluation Metrics and Learning Criteria for Non-Parallel Textual Transfer
We use x to denote the style-transferred version of x. We want x Shen et al. (2017) used two families of losses for training: reconstruction and adversarial losses. The reconstruction loss solely helps the encoder and decoder work well at encoding and generating natural language, without any attempt at transfer:
Unsupervised Evaluation Metrics and Learning Criteria for Non-Parallel Textual Transfer
The loss seeks to ensure that when a sentence x t is encoded to its content vector and then decoded to generate a sentence, the generated sentence should match x t . For their adversarial loss, Shen et al. (2017) used a pair of discriminators: D 0 tries to distinguish between x 0 and x 1 , and D 1 between x 1 and x 0 . In particular, decoder G's hidden states are aligned instead of output words.
Unsupervised Evaluation Metrics and Learning Criteria for Non-Parallel Textual Transfer
where k is the size of a mini-batch. D t outputs the probability that its input is from style t where the classifiers are based on the convolutional neural network from Kim (2014). The CNNs use filter n-gram sizes of 3, 4, and 5, with 128 filters each. We obtain hidden states h by unfolding G from the initial state (y t , z (i) t ) and feeding in x (i) t . We obtain hidden states h by unfolding G from (y 1−t , z (i) t ) and feeding in the previous output probability distributions.
Unsupervised Evaluation Metrics and Learning Criteria for Non-Parallel Textual Transfer
We use a "cyclic consistency" loss (Zhu et al., 2017) to encourage already-transferred sentences to be able to be recovered by transferring back again. This loss is similar to L rec except we now transfer style twice in the loss. Recall that we seek to transfer x t to x t . After successful transfer, we expect x t to have style y 1−t , and x t (transferred back from x t ) to have style y t . We want x t to be very close to the original untransferred x t . The loss is defined as
Unsupervised Evaluation Metrics and Learning Criteria for Non-Parallel Textual Transfer
To use this loss, the first step is to transfer sentences x t from style t to 1 − t to get x t . The second step is to transfer x t of style 1 − t back to t so that we can compute the loss of the words in x t using probability distributions computed by the decoder. Backpropagation on the embedding, encoder, and decoder parameters will only be based on the second step, because the first step involves argmax operations which prevent backpropagation. Still, we find that the cyclic loss greatly improves semantic preservation during transfer.
Unsupervised Evaluation Metrics and Learning Criteria for Non-Parallel Textual Transfer
While L rec provides the model with one way to preserve style (i.e., simply reproduce the input), the model does not see any examples of stylepreserving paraphrases. To address this, we add a paraphrase loss very similar to losses used in neural machine translation. We define the loss on a sentential paraphrase pair u, v and assume that u and v have the same style and content. The loss is the sum of token-level log losses for generating each word in v conditioned on the encoding of u:
Unsupervised Evaluation Metrics and Learning Criteria for Non-Parallel Textual Transfer
For paraphrase pairs, we use the ParaNMT-50M dataset (Wieting and Gimpel, 2018). 1
Unsupervised Evaluation Metrics and Learning Criteria for Non-Parallel Textual Transfer
We attempt to improve fluency (our third metric) and assist transfer with a loss based on matching a pretrained language model for the target style. The loss is the cross entropy (CE) between the probability distribution from this language model and the distribution from the decoder:
Unsupervised Evaluation Metrics and Learning Criteria for Non-Parallel Textual Transfer
where l t,i and g t,i are distributions over the vocabulary defined as follows:
Unsupervised Evaluation Metrics and Learning Criteria for Non-Parallel Textual Transfer
where · stands for all words in the vocabulary built from the corpora. When transferring from style t to 1 − t, l t,i is the distribution under the language model p LM 1−t pretrained on sentences from style 1 − t and g t,i is the distribution under the decoder G. The two distributions l t,i and g t,i are over words at position i given the i − 1 words already predicted by the decoder. The two style-specific language models are pretrained on the corpora corresponding to the two styles. They are GRU RNNs with a dropout probability of 0.5, and they are kept fixed during the training of the transfer network.
Unsupervised Evaluation Metrics and Learning Criteria for Non-Parallel Textual Transfer
Note that each of the textual transfer system variants uses different losses or components described 1 We first filter out sentence pairs where one sentence is the substring of another, and then randomly select 90K pairs. in this section. To create more variants, we add a second pair of discriminators, D 0 and D 1 , to the adversarial loss to address the possible mode collapse problem (Nguyen et al., 2017). In particular, we use CNNs with n-gram filter sizes of 3, 4, and 5 for D 0 and D 1 , and we use CNNs with n-gram sizes of 1, 2, and 3 for D 0 and D 1 . Also, for D 0 and D 1 , we use the Wasserstein GAN (WGAN) framework (Arjovsky et al., 2017). The adversarial loss takes the following form:
Unsupervised Evaluation Metrics and Learning Criteria for Non-Parallel Textual Transfer
is sampled for each training instance. The adversarial loss is based on Arjovsky et al. (2017), 2 with the exception that we use the hidden states of the decoder instead of word distributions as inputs to D t , similar to Eq. (3).
Unsupervised Evaluation Metrics and Learning Criteria for Non-Parallel Textual Transfer
We choose WGAN in the hope that its differentiability properties can help avoid vanishing gradient and mode collapse problems. We expect the generator to receive helpful gradients even if the discriminators perform well. This approach leads to much better outputs, as shown below.
Unsupervised Evaluation Metrics and Learning Criteria for Non-Parallel Textual Transfer
We iteratively update (1) θ D 0 , θ D 1 , θ D 0 , and θ D 1 by gradient descent on L adv 0 , L adv 1 , L adv 0 , and L adv 1 , respectively, and (2) θ E , θ G by gradient descent on L total = λ 1 L rec + λ 2 L para + λ 3 L cyc + λ 4 L lang −λ 5 (L adv 0 +L adv 1 )−λ 6 (L adv 0 +L adv 1 ). Depending on which model is being trained (see Table 2), the λ i 's for the unused losses will be zero. More details are shown in Section 5. The appendix shows the full algorithm.
Unsupervised Evaluation Metrics and Learning Criteria for Non-Parallel Textual Transfer
Yelp sentiment. We use the same Yelp dataset as Shen et al. (2017), which uses corpora of positive and negative Yelp reviews. The goal of the transfer task is to generate rewritten sentences with similar content but inverted sentiment. We use the same train/development/test split as Shen et al. (2017). The dataset has 268K, 38K, 76K positive training, development, and test sentences, respectively, and 179K/25K/51K negative sentences. Like Shen et al. (2017), we only use sentences with 15 or fewer words.
Unsupervised Evaluation Metrics and Learning Criteria for Non-Parallel Textual Transfer
Literature. We consider two corpora of literature. The first corpus contains works of Charles Dickens collected from Project Gutenberg. The second corpus is comprised of modern literature from the Toronto Books Corpus (Zhu et al., 2015). Sentences longer than 25 words are removed. Unlike the Yelp dataset, the two corpora have very different vocabularies. This dataset poses challenges for the textual transfer task, and it provides diverse data for assessing quality of our evaluation system. Given the different and sizable vocabulary, we preprocess by using the named entity recognizer in Stanford CoreNLP  to replace names and locations with -PERSON-and -LOCATION-tags, respectively. We also use byte-pair encoding (BPE), commonly used in generation tasks (Sennrich et al., 2016b). We only use sentences with lengths between 6 and 25. The resulting dataset has 156K, 5K, 5K Dickens training, development, and testing sentences, respectively, and 165K/5K/5K modern literature sentences.
Unsupervised Evaluation Metrics and Learning Criteria for Non-Parallel Textual Transfer
Section 4.6 requires setting the λ weights for each component. Depending on which model is being trained (see Table 2), the λ i 's for the unused losses will be zero. Otherwise, we set λ 1 = 1, λ 2 = 0.2, λ 3 = 5, λ 4 = 10 −3 , λ 5 = 1, λ 6 = 2 −ep where ep is the number of epochs. For optimization we use Adam (Kingma and Ba, 2014) with a learning rate of 10 −4 . We implement our models using Tensor-Flow (et al., 2015). 3 Code is available via the first author's webpage yzpang.me.
Unsupervised Evaluation Metrics and Learning Criteria for Non-Parallel Textual Transfer
For the pretrained classifiers, the accuracies on the Yelp and Literature development sets are 0.974 and 0.933, respectively. For language models, the perplexities on the Yelp and Literature development sets are 27.4 and 40.8, respectively.
Unsupervised Evaluation Metrics and Learning Criteria for Non-Parallel Textual Transfer
6 Results and Analysis 6.1 Analyzing Metric Relationships     Table 4: Manual evaluation results (%) using models from Table 2 (i.e., with roughly fixed Acc). > means "better than". ∆ Sim = Sim(A) − Sim(B), and ∆ PP = PP(A) − PP(B) (note that lower PP generally means better fluency). Each row uses at least 120 sentence pairs. A cell is bold if it represents a model win of at least 10%.
Unsupervised Evaluation Metrics and Learning Criteria for Non-Parallel Textual Transfer
trends. The figures show trajectories of statistics on corpora transferred/generated from the dev set during learning. Each two consecutive markers deviate by half an epoch of training. Lowerleft markers generally precede upper-right ones.
Unsupervised Evaluation Metrics and Learning Criteria for Non-Parallel Textual Transfer
In Figure 1(a), the plots of Sim by error rate (1 − Acc) exhibit positive slopes, meaning that error rate is positively correlated with Sim. Curves to the upper-left corner represent better trade-off between error rate and Sim. In the plots of PP by Sim in Figure 1(b), the M0 curve exhibits large positive slope but the curves for other models do not, which indicates that M0 sacrifices PP for Sim. Other models maintain consistent PP as Sim increases during training.
Unsupervised Evaluation Metrics and Learning Criteria for Non-Parallel Textual Transfer
Annotators were shown the untransferred sentence, as well as sentences produced by two models (which we refer to as A and B). They were asked to judge which better reflects the target style (A, B, or tie), which has better semantic preservation of the original (A, B, or tie), and which is more fluent (A, B, or tie). Results are shown in Table 4. Overall, the results show the same trends as our automatic metrics. For example, on Yelp, large differences in human judgments of semantic preservation (M2>M0, M7>M0, M7>M2) also show the largest differences in Sim, while M6 and M7 have very similar human judgments and very similar Sim scores.
Unsupervised Evaluation Metrics and Learning Criteria for Non-Parallel Textual Transfer
We describe a human sentence-level validation of our metrics in Table 5.
Unsupervised Evaluation Metrics and Learning Criteria for Non-Parallel Textual Transfer
To validate Acc, human annotators were asked to judge the style of 100 transferred sentences (sampled equally from M0, M2, M6, M7). Note that it is a binary choice question (style 0 or style 1  without "tie" option) so that human annotators had to make a choice. We then compute the percentage of machine and human judgments that match. We validate Sim and PP by computing sentence-level Spearman's ρ between the metric and human judgments (an integer score from 1 to 4) on 150 generated sentences (sampled equally from M0, M2, M6, M7). We presented pairs of original sentences and transferred sentences to human annotators. They were asked to rate the level of semantic similarity (and similarly for fluency) where 1 means "extremely bad", 2 means "bad/ok/needs improvement", 3 means "good", and 4 means "very good." They were also given 5 examples for each rating (i.e., a total of 20 for four levels) before annotating. From Table 5, all validations show strong correlations on the Yelp dataset and reasonable correlations on Literature.
Unsupervised Evaluation Metrics and Learning Criteria for Non-Parallel Textual Transfer
We validate GM by obtaining human pairwise preferences (without the "tie" option) of overall transfer quality and measuring the fraction of pairs in which the GM score agrees with the human preference. Out of 300 pairs (150 from each dataset), 258 (86%) match.
Unsupervised Evaluation Metrics and Learning Criteria for Non-Parallel Textual Transfer
The transferred sentences used in the evaluation are sampled from the development sets produced by models M0, M2, M6, and M7, at the accuracy levels used in Table 2. In the data preparation for the manual annotation, there is sufficient randomization regarding model and textual transfer direction.
Unsupervised Evaluation Metrics and Learning Criteria for Non-Parallel Textual Transfer
Cyclic Consistency Loss. We compare the trajectories of the baseline model (M0) and the +cyc model (M2). Table 2 and Figure 1 show that under similar Acc, M2 has much better semantic similarity for both Yelp and Literature. In fact, cyclic consistency loss proves to be the strongest driver of semantic preservation across all of our model configurations. The other losses do not constrain the semantic relationship across style transfer, so we include the cyclic loss in M3 to M7.
Unsupervised Evaluation Metrics and Learning Criteria for Non-Parallel Textual Transfer
Paraphrase Loss. Table 2 shows that the model with paraphrase loss (M1) slightly improves Sim over M0 on both datasets under similar Acc. For Yelp, M1 has better Acc and PP than M0 at comparable semantic similarity. So, when used alone, the paraphrase loss helps. However, when combined with other losses (e.g., compare M2 to M4), its benefits are mixed. For Yelp, M4 is slightly better in preserving semantics and producing fluent output, but for Literature, M4 is slightly worse. A challenge in introducing an additional paraphrase dataset is that its notions of similarity may clash with those of content preservation in the transfer task. For Yelp, both corpora share a great deal of semantic content, but Literature shows systematic semantic differences even after preprocessing.
Unsupervised Evaluation Metrics and Learning Criteria for Non-Parallel Textual Transfer
Language Modeling Loss. When comparing between M2 and M3, between M4 and M5, and between M6 and M7, we find that the addition of the language modeling loss reduces PP, sometimes at a slight cost of semantic preservation.
Unsupervised Evaluation Metrics and Learning Criteria for Non-Parallel Textual Transfer
If we want to compare the models using one single number, GM is our unsupervised approach. We can also compute BLEU scores between our generated outputs and human-written gold standard outputs using the 1000 Yelp references from . For BLEU scores reported for the methods of , we use the values reported by Yang et al. (2018b). We use the same BLEU implementation as used by Yang et al. (2018b), i.e., multi-bleu.perl. We compare three models selected during training from each of our M6 and M7 settings. We also report posttransfer accuracies reported by prior work, as well  Table 6: Results on Yelp sentiment transfer, where BLEU is between 1000 transferred sentences and human references, and Acc is restricted to the same 1000 sentences. Our best models (right table) achieve higher BLEU than prior work at similar levels of Acc, but untransferred sentences achieve the highest BLEU. Acc * : the definition of Acc varies by row because of different classifiers in use. Other results from  are not included as they are worse. our own computed Acc scores for M0, M6, M7, and the untransferred sentences. Though the classifiers differ across models, their accuracy tends to be very high (> 0.97), making it possible to make rough comparisons of Acc across them.
Unsupervised Evaluation Metrics and Learning Criteria for Non-Parallel Textual Transfer
BLEU scores and post-transfer accuracies are shown in Table 6. The most striking result is that untransferred sentences have the highest BLEU score by a large margin, suggesting that prior work for this task has not yet eclipsed the trivial baseline of returning the input sentence. However, at similar levels of Acc, our models have higher BLEU scores than prior work. We additionally find that supervised BLEU shows a trade-off with Acc: for a single model type, higher Acc generally corresponds to lower BLEU.
Unsupervised Evaluation Metrics and Learning Criteria for Non-Parallel Textual Transfer
We proposed three kinds of metrics for nonparallel textual transfer, studied their relationships, and developed learning criteria to address them. We emphasize that all three metrics are needed to make meaningful comparisons among models. We expect our components to be applicable to a broad range of generation tasks.
CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training
Knowledge graphs are a popular form of knowledge representation and central to many critical natural language processing (NLP) applications. One of the most important tasks, graph-to-text (G2T), aims to produce descriptive text that verbalizes the graphical data. For example, the knowledge graph triplet "(Allen Forest, genre, hip hop), (Allen Forest, birth year, 1981)" can be verbalized as "Allen Forest, a hip hop musician, was born in 1981." This has wide real-world applications, for instance, when a digital assistant needs to translate some structured information (e.g., the properties of the restaurant) to the human user. Another important task, text-to-graph (T2G), is to extract structures in the form of knowledge graphs from the text, so that all entities become nodes, and the relationships among entities form edges. It can help many downstream tasks, such as information retrieval and reasoning. The two tasks can be seen as a dual problem, as shown in Figure 1.
CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training
However, most previous work has treated them as two separate supervised learning problems, for which the data annotation is very expensive. Therefore, both fields face the challenge of scarce parallel data. All current datasets are of a much smaller size than what is required to train the model To circumvent the limitations of scarce supervised data, we present the first attempt to formulate both tasks in a cycle training framework, and also in the unsupervised manner with fully non-parallel datasets (as shown in Figure 1). The technical difficulty lies in the different modality of text and graphs, which can be intractable in a joint learning setting, making our problem more challenging than existing unsupervised image-to-image or text-to-text translation. We contribute an effective learning framework, CycleGT, which is iteratively trained with two cycle losses, and we also propose a task-specific warm up strategy accordingly.
CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training
We validate CycleGT on the widely used dataset WebNLG [9]. Our model achieves impressive performance that is comparable to the supervised methods which are trained on the same data but given supervised alignment of text and graphs. The high performance of CycleGT indicates that it is an effective approach to address the data scarcity problem in the fields of both G2T and T2G. Consequently, CycleGT can pave the way for scalable, unsupervised learning, and benefit future research in both fields. We summarize the differentiating factors of CycleGT in Table 1.
CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training
Our Approach
CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training
Prerequisite Aligned text-graph pairs (expensive) Non-parallel text and graph data (cheap) Method nature Supervised Unsupervised Efficiency
CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training
Learn G2T and T2G separately Learn G2T and T2G together Potential
CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training
Limited by the supervised data Unbounded due to large unsupervised data Table 1: Contributions of our approach compared with the previous works.
CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training
For unsupervised graph-to-text and text-to-graph generation, we have two non-parallel datasets:
CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training
The constraint is that the graphs and text contain the same distribution of latent content z, but are different forms of surface realization. Their marginal log-likelihood can be formulated with the shared latent content z:
CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training
Our goal is to train two models in an unsupervised manner: G2T that generates text based on the graph, and T2G that produces the graph based on text. In the context of this paper, as the definitions of the two tasks are valid, G2T and T2G are inverse functions of each other.
CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training
Denote the parameters of G2T as θ, and parameters of T2G as ϕ. Now suppose there exists an unseen ground truth distribution D Pair (e.g., as in the test set), where each paired text and graph (t, g)
CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training
share the same content. Then the ideal objective would be to maximize the log-likelihood of θ and ϕ over the text and graph pairs (t, g) ∼ D Pair :
CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training
The major challenge of our task is that the ground truth distribution D Pair is not available as training data. Instead, only the text corpus D T and graph dataset D G are observed separately without alignment. So our solution is to approximate the learning objective in Eq. (1) through non-parallel text and graph data. The alternative method we derive, namely CycleGT, will be introduced next.
CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training
In this section, we will first discuss the iterative back translation training strategy of CycleGT in Section 3.1, then introduce the G2T and T2G components in Section 3.2 and 3.3, respectively. Finally, we will introduce an additional component, warm start for the iterative training in Section 3.4.
CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training
Back Translation (BT) In NLP, back translation is first proposed for machine translation where a sentence in the source language (e.g., English) should be able to first translate to the target language (e.g., French) and then again translated "back" to the source language (e.g., English). The source sentence and the "back-translated" sentence should be aligned to be the same.
CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training
The essence of back translation is that a variable x and a bijective mapping function f (·) should satisfy x = f −1 (f (x)), where f −1 is the inverse function of f . In our case, G2T and T2G are inverse functions of each other, because one transforms graph to text and the other converts text to graph. Specifically, we align each text with its back-translated version, and also each graph with its back-translated version:
CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training
An equivalent way to interpret Eq. (2) and (3) is that due to the inavailability of paired text and graphs, we approximate D Pairs withD Pairs , a synthetic set of text-graph pairs generated by the two models. D Pairs consists of (t, T2G ϕ (t)) and (G2T θ (g), g) for every t ∈ D T , g ∈ D G , leading to
CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training
As such, the sum of Eq. (2) and (3) reasonably approximates the log likelihood in Eq. (1).
CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training
Note that J (θ, ϕ) = L CycT (θ) + L CycG (ϕ) holds whenD Pair has the same distribution as D Pair . In our framework, we iteratively improve the G2T and T2G models using an iterative back translation (IBT) training scheme, with the goal of reducing the discrepancy between the distribution of D Pair andD Pair . Specifically, we repeatedly alternate the optimization of the two cycles described by Eq. (4) and 5 over the corresponding θ or ϕ.
CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training
Non-Differentiability and Heterogeneity There are two challenging aspects of our problemspecific use of IBT, even with the two established cycle losses above. The first is non-differentiability, which constitutes a fundamental difference between our model and the line of work represented by CycleGAN [60]. For our cycle losses L CycT and L CycG , the intermediate model outputs are non-differentiable. For example, in the G-Cycle (graph→text→graph), the intermediate text is decoded in a discrete form to natural language. Hence, the graph-to-text part G2T θ will not be differentiable, and the final loss can only be propagated to the latter part, text-to-graph T2G ϕ . Hence, when alternatively optimizing the two cycle losses, we first fix ϕ to optimize θ for the T-Cycle, and then fix θ to optimize ϕ for the G-Cycle. Note that an analogous non-differentability issue is shared by unsupervised NMT works [23,1]; however, these methods have other significant differences from our approach as discussed below.
CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training
The second challenge faced by CycleGT is that text and graphs are of different modalities. Empirical studies (such as unsupervised NMT [23,1]) have shown that a pure text-to-text cycle can work by combining all the following into training: (1) shared embeddings between the encoder and decoder (because both are text), (2) denoising autoencoder, (3) cycle training, (4) adversarial loss, and (5) warm up strategies such as word-to-word lookup by a dictionary, or pretraining by language modeling on large text. However, UMT systems fail to work when having imperfect domain alignment between the source and target text [31], not to mention the challenging cross-modality problem we are dealing with. Obviously, when dealing with text and graphs, (1) and (5) cannot be applied. Moreover, (4) can be very cumbersome, and sometimes exhibits convergence issues. The remaining (2) is useless if only end-to-end cycle training is used. We demonstrate that (4) even when combined with (2) is ineffective through experiments in Section 4. In this work, we adopt a more streamlined approach suitable for the graph-to-text and text-to-graph applications.
CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training
We introduce the G2T component that we use in Eq. (2) and (3). The model G2T : G → T takes as input a graph g and generates a text sequence t that is a sufficient description of the graph.
CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training
Given a connected, directed graph G = (V, E), we seek its hidden representation h. Specifically, we introduce a root node that is connected to all other nodes, and use its final embedding h to represent the graph. We use Graph Attention Networks (GAT) [51] to calculate the representation of each node by aggregating the information from its neighbors:
CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training
where h l+1 i is the embedding of node i at layer l + 1, N i is the set of its neighboring nodes, h l j is the embedding of the neighbor node j at the previous layer l, and α ij is the attention between the two nodes by a single-layer feedforward neural network.
CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training
In this way, we represent the graph by the hidden embedding of the root node in the last layer, and overall we obtain h = GAT(V, E) .
CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training
Based on the encoded graph, the generation of each word w i at position i in the text takes as input all previously generated words w <i and the graph h:
CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training
where l i is the hidden state of the RNN decoder at step i, c i denotes the contextual information aggregated using attention mechanism, and m is an MLP layer.
CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training
Training of G2T aims to find the optimal parameters θ * to correctly encode the graph and decode it to the sentence. We use the maximum likelihood estimation:
CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training
We then introduce the remaining component, the T2G model. The function T2G : T → G takes as input a text sequence t and extracts its corresponding graph g , whose nodes are the entities and edges are the relations between each two entities. As generating both the entities and relations of the graph in a differentiable way graph is intractable in NLP, we use an off-the-shelf entity extraction model NER(t) that identifies all entities in the text with high accuracy [39]. We then predict the relations between every two entities to form the edges in the graph.
CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training
We first obtain the embeddings of every entity v i ∈ NER(t) by average pooling the contextualized embedding of each word w j in the entity term:
CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training
where enc encodes the embedding of w j by its preceding context w <j and succeeding content w >j .
CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training
Based on the entity embeddings, we derive each edge of the graph by a multi-label classification layer C. C takes in the two vertices of the edge and predicts the edge type, which includes the "no-relation" type, and the set of possible relations of entities:
CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training
T2G training aims to find the optimal parameters ϕ * that correctly encodes the text and predicts the graph, by maximum likelihood estimation:
CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training
As the IBT in Section 3.1 requires some initialization for the G2T and T2G models to start with, we propose a novel pretraining strategy to facilitate the warm start of IBT.
CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training
Pretraining from entities We conduct a novel pretraining to bootstrap T2G and G2T from only entity information. Specifically, we will learn an entity-to-text function whose parameters θ E2T will be used to initialize the G2T model, and an entity-to-graph model whose parameters ϕ E2G will be used to initialize the T2G model:
CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training
The training scheme of our model CycleGT with warm start is illustrated in Algorithm 1.
CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training
Require: The text dataset DT = {ti} N i=1 , graph dataset DG = {gj} M j=1 , entity recognition function NER, number of steps for different stages N1, N2, model G2T θ (·), and model T2Gϕ(·).
CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training
1: Initialize θ and ϕ with the optimized θE2T and ϕE2G by Eq. (6) and (7), respectively. 2: Generate the initial synthetic corpusDPair by G2T θ and T2Gϕ. 3: for step in 0..N1 do 4:
CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training
Update θ and ϕ according to supervised training onDPair by Eq. (4) and (5). 5: end for 6: for step in 0..N2 do 7:
CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training
Update θ by minimizing the cycle loss LCycT in Eq. (2). 8:
CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training
Update ϕ by minimizing the cycle loss LCycG in Eq. (3). 9: end for Initial accuracy by pretraining As the pretraining can potentially serve to warm start the iterative cycle training framework, we are interested in the best accuracy that can be achieved by such initialization.
CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training
The best accuracy for G2T θE2T is |EntSet| |DT| , and for T2G ϕE2G is |EntSet| |DG| , where |EntSet| is the number of possible entity sets that correspond to a graph or text. (See proof in Appendix A.)
CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training
The ratio |EntSet| |DX| is the inverse of the average number of X (text or graph) that has the same entity set in the dataset D X . It can also be interpreted as how much extra information the unknown relations will provide to the graph or text generation, apart from the known entity set.
CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training
From Lemma 1, it naturally follows that if each text or graph corresponds to a distinct entity set, namely |EntSet| |DX| = 1, then the upper bound of the accuracy provided by the pretraining will trivially be 100%. And for actual datasets, |EntSet| |D X | will be reasonably large, because the number of all possible entities |V | will generally increase with the size of the dataset |D X |. Also there is bias in real world knowledge expressions which usually have a consented relation between two given entities. For example, if the entity set is (Barack Obama, Michelle Obama), then the corresponding graph is highly likely to be (Barack Obama, spouse of, Michelle Obama). As the text and graph have a one-to-one mapping, the number of corresponding text expressions cannot be too large.
CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training
WebNLG Dataset Our main experiment uses the WebNLG dataset [9], which is widely used for graph-to-text generation. 2 Each graph consists of 2 to 7 triples extracted from DBPedia, and the text is collected by asking crowd-source workers to describe the graphs. We follow the preprocessing steps in [36] to obtain the text-graph pairs (with entity annotation) for 13,036 training, 1,642 validation, and 4,928 test samples. To test unsupervised models and baselines, we construct a non-parallel version of the training and validation sets by separating all the text in the dataset to form a text corpus, and all the graphs to build a graph dataset. We ensure that the order within the text and graph datasets are shuffled so that the data is fully non-parallel.
CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training
Setup We instantiate our CycleGT framework with the T2G architecture from [20] implemented by the DGL library [54], and G2T architecture based on Long Short Term Memory networks (LSTMs). We evaluate both our base model CycleGT Base and warm start model CycleGT Warm .
CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training
For unsupervised baselines, note that random guess almost gives 0% accuracy, and overall it is very difficult to get any unsupervised model working. Despite the difficulty, we develop (1) a non-trivial unsupervised baseline, CrossAlignment, using an autoencoder and adversarial training to unsupervisedly learn the alignment of the graph and text. As the performance of our model is very strong, we also compare with supervised systems (2) T2G and (3) G2T using the original supervised training data. Addition from that, we also quote performance of other state-of-the-art supervised models, including (4) Melbourne, the best supervised system submitted to the WebNLG challenge [9], which uses an encoder-decoder architecture with attention, (5) StrongNeural [36] which improves the encoder-decoder model, (6) BestPlan [36] with uses a special entity ordering algorithm before neural text generation. We also use a strong T2G model (7) OnePass, a BERT-based model which achieves state-of-the-art performance on T2G [53]. More details about the baselines are in Appendix B.
CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training
For the hyperparameters, note that we aim at a fair comparison, so the hyperparameters of the overlapped modules across different models are set to the same. For example, the supervised G2T has the same hyperparameters as the G2T component in our CycleGT, and CycleGT Warm shares the same configurations with CycleGT Base except the warm start training. Our implementation details and the link to our code are provided in Appendix C.
CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training
For G2T evaluation, We adopt the metrics from [36], i.e., BLEU [38], Meteor [3], ROUGE L [27] and CIDEr [50], to measure the closeness of the reconstructed paragraph (model output) to the input paragraph. 3 Briefly, they measure the n-gram recall and precision between the model outputs and the (ground-truth) references. For T2G evaluation, we evaluate the micro and macro F1 scores of the relation types of the edges, following the standard practice in relation extraction [34,59].
CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training
Main Results From the results in Table 2, we find that even with no pairing information between text and graphs, our best unsupervised model show impressive performance, very close to supervised models trained on the parallel graph and text data. For G2T generation, our CycleGT base model can achieve 46.2 BLEU scores, which is very close to the best supervised performance 47.4, and on par with the 45.8 BLEU scores by the supervised G2T model. Our CycleGT also outperforms the unsupervised baseline, CrossAlignment, by 37 BLEU scores. For T2G, our CycleGT achieves a micro F1 of 61.2%, which outperforms the 60.0% by the supervised T2G which has the same hyperparameters as our T2G component. There is still some performance gap between our best micro F1 score and that of the state-of-the-art system, OnePass. However, on macro F1, our performance  is on par with all supervised models. Note that CrossAlignment works decently on text generation, echoing with its reported performance on other text-based tasks [46], but works very poorly on graph extraction. Overall, our model has comparable performance to the supervised models. And as our model generates a pseudo-parallel dataset at each iteration, some data augmentation effect can be observed, making our model sometimes better than the supervised models with the same architecture.
CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training
Base and warm start CycleGT by training epochs We plot the performance of several models by training epochs in Figure 2, including the supervised G2T and T2G, CycleGT Base , and CycleGT Warm , As CycleGT Warm has been pretrained, so its start-off performance when entering the training process is higher than the other two models. The warm up strategy is especially useful on the T2G task, constantly outperforming the supervised and CycleGT Base model at all epochs. Our other model CycleGT Base is also competitive, -although it scores the lowest in the beginning, its performance steadily increases as training goes on, and finally exceeding the supervised models on both T2G and G2T, as shown by numbers in Table 2.   Table 2 compared supervised models and our unsupervised method on the same size of data. However, the advantage of the unsupervised method lies in its potential, as unsupervised data can be much more easier to obtain than elaborately annotated supervised data. So a practical comparison is whether unsupervised models with more data can exceed the supervised models by an even larger margin? We answer this question by first only providing 25% of the WebNLG data to train the supervised G2T and T2G models. Denote this training size as 1 unit. We then collect the performance of our CycleGT Base trained on non-parallel data with sizes of 1, 2, 3, and 4 units. As is seen in Figure 3, the performance of the supervised model is fixed due to the limited training set, whereas our unsupervised method has an increasing performance as the non-parallel data size rises. When the size of the non-parallel data reaches 2 times the supervised data size, the unsupervised performance exceeds the limited supervised performance by +22.35% F1 and +13.34 BLEU. When the non-parallel becomes 4 times the size of parallel data, the unsupervised performance gets even better, +34.07% and +18.17 BLEU over the supervised models. This indicates that our model gets more powerful when the number of non-parallel data exceeds that of the parallel data, which is very practical to achieve.
CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training
Scores by the number of entities or words per sentence We plot the correlation between the performance of CycleGT with two factors: (1) the number of entities per sample, which represents the size of the graph, shown in Figure 4, and (2) the number of words in the text, which represents the "size" of the text, shown in Figure 5. For each plot, we cluster the outputs into six groups according to their x values. Each point shows the average (x, y) of that complexity group. This analysis will mainly focus on the G2T quality. We evaluate by eBLEU, an improved criterion from the BLEU score which turns each entity into a symbol instead of multiple words that will cause inflation of the BLEU score. From Figure 4, we can see that as there are more entities per graph, the model performance decreases almost linearly in the beginning and plateaus at a certain value. From Figure 5, we can see that the eBLEU also decreases as the number of words in the ground-truth sentence gets larger. 
CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training
We made the first attempt of a cycle learning framework for both text-to-graph and graph-to-text generation in an unsupervised way. Experiment results validated that our model achieves comparable results to supervised models, given the same number of but unsupervised data. Further analysis demonstrated that data complexity largely correlates with the performance, and decomposing into simpler data samples will improve some aspects of the performance.
CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training
Our paper provides an unsupervised solution to two tasks: (1) T2G -extracting the latent knowledge graphs from text, and (2) G2T -generating the description for a given graph. The positive and negative impacts of our paper are both tied to its wide applications.
CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training
Our model enables knowledge extraction from text without any supervised annotation, so people can use it on unlimited web texts, and easily mine the knowledge graphs. This may facilitate parties who are using it for a good purpose such as social science research to survey the change of popular knowledge graphs in online text. But such a function can also be abused when people want to exploit the knowledge behind people's social media text, and take advantage of the mined graphs. This issue is common to all text-to-data technologies, and as such needs privacy and ownership enforcement.
CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training
The G2T function can facilitate AI for customer service, because it enables automatic text generation based on the some computer-retrieved graph information. When the user asks a question, the computer first locates answer in a part of its knowledge base, and then G2T can be applied to automatically generate natural language to convey the information. It may also be used in a negative way to make automatic fraud easier. The issue here is common to all generative technologies where proper governance must installed to prevent abuse. 
CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training
For our main results (such as Table 2), we randomly shuffled the originally parallel dataset so that alignment information is eliminated. Such a setting follows the practice in non-parallel text style transfer [25,16], and, more importantly, it can compare with supervised models trained on the same number of data. In the meantime, we understand that some work in unsupervised MT uses another setting which selects half of the parallel dataset to get text in one language (e.g, English), and uses the compliment set to get text in the other language (e.g., French) [23,14]. Therefore, we conducted a supplementary experiment using such a half-half setting. Note that for the supervised models, we use only half of the dataset, so that all models use the same amount of data.
CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training
The results comparable to Table 2 is in Table 3, where we can see that for G2T, our model still outperforms the unsupervised baseline CrossAlignment by a substantial margin. Comparing with the supervised models, our models are a bit lower on G2T, but stronger on T2G.  Table 3: T2G and G2T performance of supervised and unsupervised models on WebNLG.
Unsupervised Machine Translation Using Monolingual Corpora Only
Thanks to recent advances in deep learning (Sutskever et al., 2014;Bahdanau et al., 2015) and the availability of large-scale parallel corpora, machine translation has now reached impressive performance on several language pairs . However, these models work very well only when provided with massive amounts of parallel data, in the order of millions of parallel sentences. Unfortunately, parallel corpora are costly to build as they require specialized expertise, and are often nonexistent for low-resource languages. Conversely, monolingual data is much easier to find, and many languages with limited parallel data still possess significant amounts of monolingual data.
Unsupervised Machine Translation Using Monolingual Corpora Only
There have been several attempts at leveraging monolingual data to improve the quality of machine translation systems in a semi-supervised setting (Munteanu et al., 2004;Irvine, 2013;Irvine & Callison-Burch, 2015;Zheng et al., 2017). Most notably, Sennrich et al. (2015) proposed a very effective data-augmentation scheme, dubbed "back-translation", whereby an auxiliary translation system from the target language to the source language is first trained on the available parallel data, and then used to produce translations from a large monolingual corpus on the target side. The pairs composed of these translations with their corresponding ground truth targets are then used as additional training data for the original translation system.
Unsupervised Machine Translation Using Monolingual Corpora Only
Another way to leverage monolingual data on the target side is to augment the decoder with a language model (Gulcehre et al., 2015). And finally, Cheng et al. (2016); He et al. (2016a) have proposed to add an auxiliary auto-encoding task on monolingual data, which ensures that a translated sentence can be translated back to the original one. All these works still rely on several tens of thousands parallel sentences, however.
Unsupervised Machine Translation Using Monolingual Corpora Only
Previous work on zero-resource machine translation has also relied on labeled information, not from the language pair of interest but from other related language pairs (Firat et al., 2016;Johnson et al., 2016;Chen et al., 2017) or from other modalities (Nakayama & Nishida, 2017;Lee et al., 2017). The only exception is the work by Ravi & Knight (2011); Pourdamghani & Knight (2017), where the machine translation problem is reduced to a deciphering problem. Unfortunately, their method is limited to rather short sentences and it has only been demonstrated on a very simplistic setting comprising of the most frequent short sentences, or very closely related languages.  (autoencoding): the model is trained to reconstruct a sentence from a noisy version of it. x is the target, C(x) is the noisy input,x is the reconstruction. Right (translation): the model is trained to translate a sentence in the other domain. The input is a noisy translation (in this case, from source-to-target) produced by the model itself, M , at the previous iteration (t), y = M (t) (x). The model is symmetric, and we repeat the same process in the other language. See text for more details.
Unsupervised Machine Translation Using Monolingual Corpora Only
In this paper, we investigate whether it is possible to train a general machine translation system without any form of supervision whatsoever. The only assumption we make is that there exists a monolingual corpus on each language. This set up is interesting for a twofold reason. First, this is applicable whenever we encounter a new language pair for which we have no annotation. Second, it provides a strong lower bound performance on what any good semi-supervised approach is expected to yield.
Unsupervised Machine Translation Using Monolingual Corpora Only
The key idea is to build a common latent space between the two languages (or domains) and to learn to translate by reconstructing in both domains according to two principles: (i) the model has to be able to reconstruct a sentence in a given language from a noisy version of it, as in standard denoising auto-encoders (Vincent et al., 2008). (ii) The model also learns to reconstruct any source sentence given a noisy translation of the same sentence in the target domain, and vice versa. For (ii), the translated sentence is obtained by using a back-translation procedure (Sennrich et al., 2015), i.e. by using the learned model to translate the source sentence to the target domain. In addition to these reconstruction objectives, we constrain the source and target sentence latent representations to have the same distribution using an adversarial regularization term, whereby the model tries to fool a discriminator which is simultaneously trained to identify the language of a given latent sentence representation (Ganin et al., 2016). This procedure is then iteratively repeated, giving rise to translation models of increasing quality. To keep our approach fully unsupervised, we initialize our algorithm by using a naïve unsupervised translation model based on a word by word translation of sentences with a bilingual lexicon derived from the same monolingual data (Conneau et al., 2017). As a result, and by only using monolingual data, we can encode sentences of both languages into the same feature space, and from there, we can also decode/translate in any of these languages; see Figure 1 for an illustration.
Unsupervised Machine Translation Using Monolingual Corpora Only
While not being able to compete with supervised approaches using lots of parallel resources, we show in section 4 that our model is able to achieve remarkable performance. For instance, on the WMT dataset we can achieve the same translation quality of a similar machine translation system trained with full supervision on 100,000 sentence pairs. On the Multi30K-Task1 dataset we achieve a BLEU above 22 on all the language pairs, with up to 32.76 on English-French.
Unsupervised Machine Translation Using Monolingual Corpora Only
Next, in section 2, we describe the model and the training algorithm. We then present experimental results in section 4. Finally, we further discuss related work in section 5 and summarize our findings in section 6.
Unsupervised Machine Translation Using Monolingual Corpora Only
In this section, we first describe the architecture of the translation system, and then we explain how we train it.
Unsupervised Machine Translation Using Monolingual Corpora Only
The translation model we propose is composed of an encoder and a decoder, respectively responsible for encoding source and target sentences to a latent space, and to decode from that latent space to the source or the target domain. We use a single encoder and a single decoder for both domains (Johnson et al., 2016). The only difference when applying these modules to different languages is the choice of lookup tables.
Unsupervised Machine Translation Using Monolingual Corpora Only
Let us denote by W S the set of words in the source domain associated with the (learned) words embeddings Z S = (z s 1 , ...., z s |W S | ), and by W T the set of words in the target domain associated with the embeddings Z T = (z t 1 , ...., z t |W T | ), Z being the set of all the embeddings. Given an input sentence of m words x = (x 1 , x 2 , ..., x m ) in a particular language , ∈ {src, tgt}, an encoder e θenc,Z (x, ) computes a sequence of m hidden states z = (z 1 , z 2 , ..., z m ) by using the corresponding word embeddings, i.e. Z S if = src and Z T if = tgt; the other parameters θ enc are instead shared between the source and target languages. For the sake of simplicity, the encoder will be denoted as e(x, ) in the following. These hidden states are vectors in R n , n being the dimension of the latent space.
Unsupervised Machine Translation Using Monolingual Corpora Only
A decoder d θ dec ,Z (z, ) takes as input z and a language , and generates an output sequence y = (y 1 , y 2 , ..., y k ), where each word y i is in the corresponding vocabulary W . This decoder makes use of the corresponding word embeddings, and it is otherwise parameterized by a vector θ dec that does not depend on the output language. It will thus be denoted d(z, ) in the following. To generate an output word y i , the decoder iteratively takes as input the previously generated word y i−1 (y 0 being a start symbol which is language dependent), updates its internal state, and returns the word that has the highest probability of being the next one. The process is repeated until the decoder generates a stop symbol indicating the end of the sequence.
Unsupervised Machine Translation Using Monolingual Corpora Only
In this article, we use a sequence-to-sequence model with attention (Bahdanau et al., 2015). The encoder is a bidirectional-LSTM which returns a sequence of hidden states z = (z 1 , z 2 , ..., z m ). At each step, the decoder, which is an LSTM, takes the previous hidden state, the current word and a context vector given by a weighted sum over the encoder states.
Unsupervised Machine Translation Using Monolingual Corpora Only
We consider a dataset of sentences in the source domain, denoted by D src , and another dataset in the target domain, denoted by D tgt . These datasets do not correspond to each other, in general. We train the encoder and decoder by reconstructing a sentence in a particular domain, given a noisy version of the same sentence in the same or in the other domain.
Unsupervised Machine Translation Using Monolingual Corpora Only
At a high level, the model starts with an unsupervised naïve translation model obtained by making word-by-word translation of sentences using a parallel dictionary learned in an unsupervised way (Conneau et al., 2017). Then, at each iteration, the encoder and decoder are trained by minimizing an objective function that measures their ability to both reconstruct and translate from a noisy version of an input training sentence. This noisy input is obtained by dropping and swapping words in the case of the auto-encoding task, while it is the result of a translation with the model at the previous iteration in the case of the translation task. In order to promote alignment of the latent distribution of sentences in the source and the target domains, our approach also simultaneously learns a discriminator in an adversarial setting. The newly learned encoder/decoder are then used at the next iteration to generate new translations, until convergence of the algorithm. At test time and despite the lack of parallel data at training time, the encoder and decoder can be composed into a standard machine translation system.
Unsupervised Machine Translation Using Monolingual Corpora Only
Training an autoencoder of sentences is a trivial task, if the sequence-to-sequence model is provided with an attention mechanism like in our work 1 . Without any constraint, the auto-encoder very quickly learns to merely copy every input word one by one. Such a model would also perfectly copy sequences of random words, suggesting that the model does not learn any useful structure in the data. To address this issue, we adopt the same strategy of Denoising Auto-encoders (DAE) (Vincent et al., 2008)), and add noise to the input sentences (see Figure 1-left), similarly to Hill et al. (2016). Considering a domain = src or = tgt, and a stochastic noise model denoted by C which operates on sentences, we define the following objective function:
Unsupervised Machine Translation Using Monolingual Corpora Only
wherex ∼ d(e(C(x), ), ) means thatx is a reconstruction of the corrupted version of x, with x sampled from the monolingual dataset D . In this equation, ∆ is a measure of discrepancy between the two sequences, the sum of token-level cross-entropy losses in our case.
Unsupervised Machine Translation Using Monolingual Corpora Only
Noise model C(x) is a randomly sampled noisy version of sentence x. In particular, we add two different types of noise to the input sentence. First, we drop every word in the input sentence with a probability p wd . Second, we slightly shuffle the input sentence. To do so, we apply a random permutation σ to the input sentence, verifying the condition ∀i ∈ {1, n}, |σ(i) − i| ≤ k where n is the length of the input sentence, and k is a tunable parameter. In our experiments, both the word dropout and the input shuffling strategies turned out to have a critical impact on the results, see also section 4.5. Using both strategies at the same time is what gave us the best performance. In practice, we found p wd = 0.1 and k = 3 to be good parameters.
Unsupervised Machine Translation Using Monolingual Corpora Only
The second objective of our approach is to constrain the model to be able to map an input sentence from a the source/target domain 1 to the target/source domain 2 , which is what we are ultimately interested in at test time. The principle here is to sample a sentence x ∈ D 1 , and to generate a corrupted translation of this sentence in 2 . This corrupted version is generated by applying the current translation model denoted M to x such that y = M (x). Then a corrupted version C(y) is sampled (see Figure 1-right). The objective is thus to learn the encoder and the decoder such that they can reconstruct x from C(y). The cross-domain loss can be written as:
Unsupervised Machine Translation Using Monolingual Corpora Only
where y = M (x) and ∆ is again the sum of token-level cross-entropy losses.
Unsupervised Machine Translation Using Monolingual Corpora Only
Intuitively, the decoder of a neural machine translation system works well only when its input is produced by the encoder it was trained with, or at the very least, when that input comes from a distribution very close to the one induced by its encoder. Therefore, we would like our encoder to output features in the same space regardless of the actual language of the input sentence. If such condition is satisfied, our decoder may be able to decode in a certain language regardless of the language of the encoder input sentence.
Unsupervised Machine Translation Using Monolingual Corpora Only
Note however that the decoder could still produce a bad translation while yielding a valid sentence in the target domain, as constraining the encoder to map two languages in the same feature space does not imply a strict correspondence between sentences. Fortunately, the previously introduced loss for cross-domain training in equation 2 mitigates this concern. Also, recent work on bilingual lexical induction has shown that such a constraint is very effective at the word level, suggesting that it may also work at the sentence level, as long as the two monolingual corpora exhibit strong structure in feature space.
Unsupervised Machine Translation Using Monolingual Corpora Only
In order to add such a constraint, we train a neural network, which we will refer to as the discriminator, to classify between the encoding of source sentences and the encoding of target sentences (Ganin et al., 2016). The discriminator operates on the output of the encoder, which is a sequence of latent vectors (z 1 , ..., z m ), with z i ∈ R n , and produces a binary prediction about the language of the encoder input sentence: p D (l|z 1 , ..., z m ) ∝  The discriminator is trained to predict the language by minimizing the following cross-entropy loss:
Unsupervised Machine Translation Using Monolingual Corpora Only
where (x i , i ) corresponds to sentence and language id pairs uniformly sampled from the two monolingual datasets, θ D are the parameters of the discriminator, θ enc are the parameters of the encoder, and Z are the encoder word embeddings.
Unsupervised Machine Translation Using Monolingual Corpora Only
The encoder is trained instead to fool the discriminator:
Unsupervised Machine Translation Using Monolingual Corpora Only
with j = 1 if i = 2 , and vice versa.
Unsupervised Machine Translation Using Monolingual Corpora Only
Final Objective function The final objective function at one iteration of our learning algorithm is thus: L(θ enc , θ dec , Z) =λ auto [L auto (θ enc , θ dec , Z, src) + L auto (θ enc , θ dec , Z, tgt)]+ λ cd [L cd (θ enc , θ dec , Z, src, tgt) + L cd (θ enc , θ dec , Z, tgt, src)]+ λ adv L adv (θ enc , Z|θ D )
Unsupervised Machine Translation Using Monolingual Corpora Only
where λ auto , λ cd , and λ adv are hyper-parameters weighting the importance of the auto-encoding, cross-domain and adversarial loss. In parallel, the discriminator loss L D is minimized to update the discriminator.
Unsupervised Machine Translation Using Monolingual Corpora Only
In this section we describe the overall training algorithm and the unsupervised criterion we used to select hyper-parameters.
Unsupervised Machine Translation Using Monolingual Corpora Only
The final learning algorithm is described in Algorithm 1 and the general architecture of the model is shown in Figure 2. As explained previously, our model relies on an iterative algorithm which starts from an initial translation model M (1) (line 3). This is used to translate the available monolingual To jump start the process, M (1) simply makes a word-by-word translation of each sentence using a parallel dictionary learned using the unsupervised method proposed by Conneau et al. (2017), which only leverages monolingual data.
Unsupervised Machine Translation Using Monolingual Corpora Only
Algorithm 1 Unsupervised Training for Machine Translation 1: procedure TRAINING(D src , D tgt , T ) 2:
Unsupervised Machine Translation Using Monolingual Corpora Only
Infer bilingual dictionary using monolingual data (Conneau et al., 2017) 3:
Unsupervised Machine Translation Using Monolingual Corpora Only
M (1) ← unsupervised word-by-word translation model using the inferred dictionary 4:
Unsupervised Machine Translation Using Monolingual Corpora Only
for t = 1, T do The intuition behind our algorithm is that as long as the initial translation model M (1) retains at least some information of the input sentence, the encoder will map such translation into a representation in feature space that also corresponds to a cleaner version of the input, since the encoder is trained to denoise. At the same time, the decoder is trained to predict noiseless outputs, conditioned on noisy features. Putting these two pieces together will produce less noisy translations, which will enable better back-translations at the next iteration, and so on so forth.
Unsupervised Machine Translation Using Monolingual Corpora Only
In order to select hyper-parameters, we wish to have a criterion correlated with the translation quality. However, we do not have access to parallel sentences to judge how well our model translates, not even at validation time. Therefore, we propose the surrogate criterion which we show correlates well with BLEU (Papineni et al., 2002), the metric we care about at test time.
Unsupervised Machine Translation Using Monolingual Corpora Only
For all sentences x in a domain 1 , we translate these sentences to the other domain 2 , and then translate the resulting sentences back to 1 . The quality of the model is then evaluated by computing the BLEU score over the original inputs and their reconstructions via this two-step translation process. The performance is then averaged over the two directions, and the selected model is the one with the highest average score.
Unsupervised Machine Translation Using Monolingual Corpora Only
Given an encoder e, a decoder d and two non-parallel datasets D src and D tgt , we denote M src→tgt (x) = d(e(x, src), tgt) the translation model from src to tgt, and M tgt→src the model in the opposite direction. Our model selection criterion M S(e, d, D src , D tgt ) is: Figure 3 shows a typical example of the correlation between this measure and the final translation model performance (evaluated here using a parallel dataset).
Unsupervised Machine Translation Using Monolingual Corpora Only
In this section, we first describe the datasets and the pre-processing we used, then we introduce the baselines we considered, and finally we report the extensive empirical validation proving the effectiveness of our method. We will release the code to the public once the revision process is over.
Unsupervised Machine Translation Using Monolingual Corpora Only
In our experiments, we consider the English-French and English-German language pairs, on three different datasets.
Unsupervised Machine Translation Using Monolingual Corpora Only
WMT'14 English-French We use the full training set of 36 million pairs, we lower-case them and remove sentences longer than 50 words, as well as pairs with a source/target length ratio above 1.5, resulting in a parallel corpus of about 30 million sentences. Next, we build monolingual corpora by selecting the English sentences from 15 million random pairs, and selecting the French sentences from the complementary set. The former set constitutes our English monolingual dataset. The latter set is our French monolingual dataset. The lack of overlap between the two sets ensures that there is not exact correspondence between examples in the two datasets.
Unsupervised Machine Translation Using Monolingual Corpora Only
The validation set is comprised of 3,000 English and French sentences extracted from our monolingual training corpora described above. These sentences are not the translation of each other, and they will be used by our unsupervised model selection criterion, as explained in 3.2. Finally, we report results on the full newstest2014 dataset.
Unsupervised Machine Translation Using Monolingual Corpora Only
WMT'16 English-German We follow the same procedure as above to create monolingual training and validation corpora in English and German, which results in two monolingual training corpora of 1.8 million sentences each. We test our model on the newstest2016 dataset.
Unsupervised Machine Translation Using Monolingual Corpora Only
The task 1 of the Multi30k dataset (Elliott et al., 2016) has 30,000 images, with annotations in English, French and German, that are translations of each other. We consider the English-French and English-German pairs. We disregard the images and only consider the parallel annotations, with the provided training, validation and test sets, composed of 29,000, 1,000 and 1,000 pairs of sentences respectively. For both pairs of languages and similarly to the WMT datasets above, we split the training and validation sets into monolingual corpora, resulting in 14,500 monolingual source and target sentences in the training set, and 500 sentences in the validation set.
Unsupervised Machine Translation Using Monolingual Corpora Only
Word-by-word translation (WBW) The first baseline is a system that performs word-by-word translations of the input sentences using the inferred bilingual dictionary (Conneau et al., 2017). This baseline provides surprisingly good results for related language pairs, like English-French, where the word order is similar, but performs rather poorly on more distant pairs like English-German, as can be seen in Table 1.
Unsupervised Machine Translation Using Monolingual Corpora Only
Word reordering (WR) After translating word-by-word as in WBW, here we reorder words using an LSTM-based language model trained on the target side. Since we cannot exhaustively score every possible word permutation (some sentences have about 100 words), we consider all pairwise swaps of neighboring words, we select the best swap, and iterate ten times. We use this baseline only on the WMT dataset that has a large enough monolingual data to train a language model.
Unsupervised Machine Translation Using Monolingual Corpora Only
Oracle Word Reordering (OWR) Using the reference, we produce the best possible generation using only the words given by WBW. The performance of this method is an upper-bound of what any model could do without replacing words.  Table 1: BLEU score on the WMT and Multi30k-Task1 datasets using greedy decoding.
Unsupervised Machine Translation Using Monolingual Corpora Only
We finally consider exactly the same model as ours, but trained with supervision, using the standard cross-entropy loss on the original parallel sentences.
Unsupervised Machine Translation Using Monolingual Corpora Only
To implement our baseline and also to initialize the embeddings Z of our model, we first train word embeddings on the source and target monolingual corpora using fastText (Bojanowski et al., 2017), and then we apply the unsupervised method proposed by Conneau et al. (2017) to infer a bilingual dictionary which can be use for word-by-word translation.
Unsupervised Machine Translation Using Monolingual Corpora Only
Since WMT yields a very large-scale monolingual dataset, we obtain very high-quality embeddings and dictionaries, with an accuracy of 84.48% and 77.29% on French-English and German-English, which is on par with what could be obtained using a state-of-the-art supervised alignment method (Conneau et al., 2017).
Unsupervised Machine Translation Using Monolingual Corpora Only
On the Multi30k datasets instead, the monolingual training corpora are too small to train good word embeddings (more than two order of magnitude smaller than WMT). We therefore learn word vectors on Wikipedia using fastText 2 .
Unsupervised Machine Translation Using Monolingual Corpora Only
Discriminator Architecture The discriminator is a multilayer perceptron with three hidden layers of size 1024, Leaky-ReLU activation functions and an output logistic unit. Following Goodfellow (2016), we include a smoothing coefficient s = 0.1 in the discriminator predictions.
Unsupervised Machine Translation Using Monolingual Corpora Only
Training Details The encoder and the decoder are trained using Adam (Kingma & Ba, 2014), with a learning rate of 0.0003, β 1 = 0.5, and a mini-batch size of 32. The discriminator is trained using RMSProp (Tieleman & Hinton, 2012) with a learning rate of 0.0005. We evenly alternate between one encoder-decoder and one discriminator update. We set λ auto = λ cd = λ adv = 1. Table 1 shows the BLEU scores achieved by our model and the baselines we considered. First, we observe that word-by-word translation is surprisingly effective when translating into English, obtaining a BLEU score of 16.77 and 10.09 for fr-en on respectively Multi30k-Task1 and WMT datasets. Word-reordering only slightly improves upon word-by-word translation. Our model instead, clearly outperforms these baselines, even on the WMT dataset which has more diversity of topics and sentences with much more complicated structure. After just one iteration, we obtain a BLEU score of 27.48 and 12.10 for the en-fr task. Interestingly, we do even better than oracle reordering, suggesting that our model not only reorders but also correctly substitutes some words. After a few iterations, our model obtains BLEU of 32.76 and 15.05 on Multi30k-Task1 and WMT datasts for the English to French task, which is rather remarkable. Similar observations can be made for the other language pairs we considered. Comparison with supervised approaches Here, we assess how much labeled data are worth our two large monolingual corpora. On WMT, we trained a standard supervised model on both language pairs, using various amounts of parallel data. Figure 4-right shows the resulting performance. Our unsupervised approach obtains the same performance than a supervised model trained on about 100,000 parallel sentences, which is impressive. Of course, adding more parallel examples allows the supervised approach to outperform our method, but the good performance of our unsupervised method suggests that it could be very effective for low-resources languages where no parallel data are available. Moreover, these results open the door to the development of semi-supervised translation models, which will be the focus of future investigation. Figure 4-left illustrates the quality of the learned model after each iteration of the learning process in the language pairs of Multi30k-Task1 dataset, other results being provided in Table 1. One can see that the quality of the obtained model is high just after the first iteration of the process. Subsequent iterations yield significant gains although with diminishing returns. At iteration 3, the performance gains are marginal, showing that our approach quickly converges. Table 2 shows examples of translations of three sentences on the Multi30k dataset, as we iterate. Iteration 0 corresponds to the word-by-word translation obtained with our cross-lingual dictionary, which clearly suffers from word order issues. We can observe that the quality of the translations increases at every iteration.
Unsupervised Machine Translation Using Monolingual Corpora Only
We perform an ablation study to understand the importance of the different components of our system. To this end, we have trained multiple versions of our model with some missing components: the discriminator, the cross-domain loss, the auto-encoding loss, etc. Table 3 shows that the best performance is obtained with the simultaneous use of all the described elements. The most critical component is the unsupervised word alignment technique, either in the form of a back-translation dataset generated using word-by-word translation, or in the form of pretrained embeddings which enable to map sentences of different languages in the same latent space.
Unsupervised Machine Translation Using Monolingual Corpora Only
On the English-French pair of Multi30k-Task1, with a back-translation dataset but without pretrained embeddings, our model obtains a BLEU score of 25.29 and 26.10, which is only a few points below the model using all components. Similarly, when the model uses pretrained embeddings but no back-translation dataset (when λ cd = 0), it obtains 25.44 and 27.14. On the other hand, a model that does not use any of these components only reaches 8.78 and 9.15 BLEU.
Unsupervised Machine Translation Using Monolingual Corpora Only
Source un homme est debout près d' une série de jeux vidéo dans un bar . Iteration 0 a man is seated near a series of games video in a bar . Iteration 1 a man is standing near a closeup of other games in a bar . Iteration 2 a man is standing near a bunch of video video game in a bar . Iteration 3 a man is standing near a bunch of video games in a bar . Reference a man is standing by a group of video games in a bar .
Unsupervised Machine Translation Using Monolingual Corpora Only
une femme aux cheveux roses habillée en noir parleà un homme . Iteration 0 a woman at hair roses dressed in black speaks to a man . Iteration 1 a woman at glasses dressed in black talking to a man . Iteration 2 a woman at pink hair dressed in black speaks to a man . Iteration 3 a woman with pink hair dressed in black is talking to a man . Reference a woman with pink hair dressed in black talks to a man .
Unsupervised Machine Translation Using Monolingual Corpora Only
une photo d' une rue bondée en ville . Iteration 0 a photo a street crowded in city . Iteration 1 a picture of a street crowded in a city . Iteration 2 a picture of a crowded city street . Iteration 3 a picture of a crowded street in a city . Reference a view of a crowded city street .  Table 3: Ablation study on the Multi30k-Task1 dataset.
Unsupervised Machine Translation Using Monolingual Corpora Only
The adversarial component also significantly improves the performance of our system, with a difference of up to 5.33 BLEU in the French-English pair of Multi30k-Task1. This confirms our intuition that, to really benefit from the cross-domain loss, one has to ensure that the distribution of latent sentence representations is similar across the two languages. Without the auto-encoding loss (when λ auto = 0), the model only obtains 20.02, which is 8.05 BLEU points below the method using all components. Finally, performance is greatly degraded also when the corruption process of the input sentences is removed, as the model has much harder time learning useful regularities and merely learns to copy input data.
Unsupervised Machine Translation Using Monolingual Corpora Only
A similar work to ours is the style transfer method with non-parallel text by Shen et al. (2017). The authors consider a sequence-to-sequence model, where the latent state given to the decoder is also fed to a discriminator. The encoder is trained with the decoder to reconstruct the input, but also to fool the discriminator. The authors also found it beneficial to train two discriminators, one for the source and one for the target domain. Then, they trained the decoder so that the recurrent hidden states during the decoding process of a sentence in a particular domain are not distinguishable according to the respective discriminator. This algorithm, called Professor forcing, was initially introduced by Lamb et al. (2016) to encourage the dynamics of the decoder observed during inference to be similar to the ones observed at training time.
Unsupervised Machine Translation Using Monolingual Corpora Only
Before that, Hu et al. (2017) trained a variational autoencoder (VAE) where the decoder input is the concatenation of an unstructured latent vector, and a structured code representing the attribute of the sentence to generate. A discriminator is trained on top of the decoder to classify the labels of generated sentences, while the decoder is trained to satisfy this discriminator. Because of the nondifferentiability of the decoding process, at each step, their decoder takes as input the probability vector predicted at the previous step.
Unsupervised Machine Translation Using Monolingual Corpora Only
Perhaps, the most relevant prior work is by He et al. (2016b), who essentially optimizes directly for the model selection metric we propose in section 3.2. One drawback of their approach, which has not been applied to the fully unsupervised setting, is that it requires to back-propagate through the sequence of discrete predictions using reinforcement learning-based approaches which are notoriously inefficient. In this work, we instead propose to a) use a symmetric architecture, and b) freeze the translator from source to target when training the translator from target to source, and vice versa. By alternating this process we operate with a fully differentiable model and we efficiently converge.
Unsupervised Machine Translation Using Monolingual Corpora Only
We presented a new approach to neural machine translation where a translation model is learned using monolingual datasets only, without any alignment between sentences or documents. The principle of our approach is to start from a simple unsupervised word-by-word translation model, and to iteratively improve this model based on a reconstruction loss, and using a discriminator to align latent distributions of both the source and the target languages. Our experiments demonstrate that our approach is able to learn effective translation models without any supervision of any sort.