Thats a unique idea.You should consider the following idea to enhance your idea so that it would impact uniquely: Effective image and sentence matching depends on how to ll measure their global visual-semantic similarity. Based on the observation that such a global similarity arises from a complex aggregation of multiple local similarities beten pairwise instances of image (objects) and sentence (words),  propose a selective multimodal Long Short-Term Memory network (sm-LSTM) for instance-aware image and sentence matching. The sm-LSTM includes a multimodal context-modulated attention scheme at each timestep that can selectively attend to a pair of instances of image and sentence, by predicting pairwise instance-aware saliency maps for image and sentence. For selected pairwise instances, their representations are obtained based on the predicted saliency maps, and then compared to measure their local similarity. By similarly measuring multiple local similarities within a few timesteps, the sm-LSTM sequentially aggregates them with hidden states to obtain a final matching score as the desired global similarity. Extensive experiments show that our model can ll match image and sentence with complex content, and achieve the state-of-the-art results on two public benchmark datasets.
['Huang Y', 'Wei Wang', 'Liang Wang']
I am writing a research paper on a idea Instance-aware Image and Sentence Matching with Selective Multimodal LSTM
[{"from": "human", "value": "I am writing a research paper on a idea Instance-aware Image and Sentence Matching with Selective Multimodal LSTM"}, {"from": "gpt", "value": "Thats a unique idea.You should consider the following idea to enhance your idea so that it would impact uniquely: Effective image and sentence matching depends on how to ll measure their global visual-semantic similarity. Based on the observation that such a global similarity arises from a complex aggregation of multiple local similarities beten pairwise instances of image (objects) and sentence (words),  propose a selective multimodal Long Short-Term Memory network (sm-LSTM) for instance-aware image and sentence matching. The sm-LSTM includes a multimodal context-modulated attention scheme at each timestep that can selectively attend to a pair of instances of image and sentence, by predicting pairwise instance-aware saliency maps for image and sentence. For selected pairwise instances, their representations are obtained based on the predicted saliency maps, and then compared to measure their local similarity. By similarly measuring multiple local similarities within a few timesteps, the sm-LSTM sequentially aggregates them with hidden states to obtain a final matching score as the desired global similarity. Extensive experiments show that our model can ll match image and sentence with complex content, and achieve the state-of-the-art results on two public benchmark datasets."}]