Thats a unique idea.You should consider the following idea to enhance your idea so that it would impact uniquely: Encoder-decoder networks are popular for modeling sequences probabilistically in many applications. These models use the por of the Long Short-Term Memory (LSTM) architecture to capture the full dependence among variables, unlike earlier models like CRFs that typically assumed conditional independence among non-adjacent variables. Hover in practice encoder-decoder models exhibit a bias towards short sequences that surprisingly gets worse with increasing beam size. #R##N#  show that such phenomenon is due to a discrepancy beten the full sequence margin and the per-element margin enforced by the locally conditioned training objective of a encoder-decoder model. The discrepancy more adversely impacts long sequences, explaining the bias towards predicting short sequences. #R##N#For the case where the predicted sequences come from a closed set,  show that a globally conditioned model alleviates the above problems of encoder-decoder models. From a practical point of view, our proposed model also eliminates the need for a beam-search during inference, which reduces to an efficient dot-product based search in a vector-space.
['Pavel Sountsov', 'Sunita Sarawagi']
I am writing a research paper on a idea Length bias in Encoder Decoder Models and a Case for Global Conditioning
[{"from": "human", "value": "I am writing a research paper on a idea Length bias in Encoder Decoder Models and a Case for Global Conditioning"}, {"from": "gpt", "value": "Thats a unique idea.You should consider the following idea to enhance your idea so that it would impact uniquely: Encoder-decoder networks are popular for modeling sequences probabilistically in many applications. These models use the por of the Long Short-Term Memory (LSTM) architecture to capture the full dependence among variables, unlike earlier models like CRFs that typically assumed conditional independence among non-adjacent variables. Hover in practice encoder-decoder models exhibit a bias towards short sequences that surprisingly gets worse with increasing beam size. #R##N#  show that such phenomenon is due to a discrepancy beten the full sequence margin and the per-element margin enforced by the locally conditioned training objective of a encoder-decoder model. The discrepancy more adversely impacts long sequences, explaining the bias towards predicting short sequences. #R##N#For the case where the predicted sequences come from a closed set,  show that a globally conditioned model alleviates the above problems of encoder-decoder models. From a practical point of view, our proposed model also eliminates the need for a beam-search during inference, which reduces to an efficient dot-product based search in a vector-space."}]