Image Captioning

شرح پروژه

This part involves developing an encoder-decoder based architecture to perform image
captioning on the Flickr30K dataset. The Flickr30K dataset contains contains 31,000 images
collected from Flickr, together with 5 reference sentences provided by human annotators.
a) Construct a baseline image captioning model employing a pre-trained CNN (e.g.,
VGG16) for image feature extraction and a simple RNN for caption generation. Do not
forget to use the previously discussed good practices in training an artificial neural
network.
b) What scores did use in the previous part? Explain BLUE, METEOR, CIDEr and ROUGE
scores and evaluate baseline performance using BLEU and ROUGE scores to set a
benchmark for subsequent improvements.
c) To address the limitations of the baseline RNN’s capacity for long-term dependencies,
upgrade the decoder network to an LSTM. Measure any performance improvements
against the baseline model using the same evaluation metrics.
d) Enhance the LSTM-based model by incorporating a soft attention mechanism, allowing
the model to focus on different regions of an image when generating each word in the
caption. Compare performance improvements over previous models both quantitatively
and qualitatively.
e) Bonus: Implement pre-trained word embeddings such as GloVe or Word2Vec within the
decoder to enrich the semantic understanding of captions. Evaluate the impact of these
pre-trained embeddings on the model’s ability to generate coherent and contextually
relevant captions, especially considering the limited vocabulary size of the Flickr30K
dataset 

حتما در pytorch و با گزارش کار 

مهارت ها و تخصص های مورد نیاز

درباره کارفرما

Melod

عضویت ده ماه پیش

5 پروژه ثبت شده ،

0 پروژه در حال انجام ،

0 پروژه آماده دریافت پیشنهاد ،

نرخ پذیرش پیشنهاد 40%