Abstract
Compared to laborious pixel-level annotations, scribbles, bounding boxes, and points are much more efficient in salient object detection (SOD). However, the annotation cost of these forms linearly increases with the number of salient objects in the image, which is not ideal for real-world scenarios. To address this issue, we propose a novel annotation form called oversize bounding box (OBB), i.e., a box that encompasses all salient objects without the need for tight enclosure. It has two characteristics: (1) All pixels outside the box are from the background. (2) A subset of the pixels inside the box belongs to salient objects. Therefore, the core issue is how to highlight integral and accurate object regions as pseudo-labels. Inspired by the powerful visual understanding and vision-language correlations demonstrated by large multimodal models, we devise the first multimodal model-based pseudo-label generation method for SOD. It utilizes MiniGPT-4 to generate descriptions of salient objects as text prompts for CLIP, combined with the CAM-based technique and proposed refinement algorithm based on the multi-head self-attention and superpixel to activate the object regions. Then we use OBB to further correct the activation regions based on its two characteristics. Given the potential errors of pseudo-labels within the box, we propose a center-pixel-based cross entropy loss, as activated pixels closer to the center are generally more reliable. Moreover, we establish an OBB-supervised training dataset by relabeling the DUTS dataset. Extensive experiments on six benchmarks demonstrate that our method achieves the state-of-the-art performance.