Multimodal deep learning: A Comparison between LSTM and Transformers for Image captioning

Описание: Image captioning is the process of generating a textual description of images, which integrates both computer vision and natural language processing. Approaches based on encoder-decoder architectures have been recently proposed to solve image captioning problems. The main objective of this paper is to conduct a comparative study between the two most widely used approaches for natural language processing tasks, namely, LSTMs and Transformers. We used the Flickr8k dataset as input images. Regarding image feature extraction, we used the VGG16 model. To evaluate the obtained descriptions generated by the models, the BLEU score metric is used to measure the performance of both models. The latter were able to generate grammatically correct and expressive captions.