IRES Project: Dynamic Finetuning Of Multi-Task Multi-Modal Models

Over summer 2024, I got the chance to study at INAOE, a research university near Puebla, Mexico, in partnernship with University Of Houston as an IRES student. My partner and I worked on applying curriculum learning style techniques to finetune a Multi-Task Multi-Modal Model for Comic Mischief Detection. To learn more, see our final report here. In the report we discuss: the problem of Comic Mischief Detection; the multitask multimodal HICCAP model (pictured below); the difficulty of multiask learning; a survey of methods for training a multitask model; and our experimental results.

Oh, hello there. If you wish to learn more, please see the report linked above. While you’re here, here’s a picture of the volcano Popocatépetl erupting that I took from the window where I worked while at INAOE.