Fourier-modulated CLIP for zero-shot vehicle counting.

Vehicle counting is a critical task in traffic monitoring and management. Although large vision-language models have achieved significant progress in zero-shot text-image matching, adapting them for vehicle counting remains a challenging task. To tackle this issue, we propose FCLIP-VC, a Fourier-modulated Contrastive Language-image Pre-training (CLIP) for zero-shot vehicle counting. First, a Vision Transformer (ViT)-based CLIP model is employed to encode image features for precise vehicle localization. Additionally, we introduce a learnable visual prompt to avoid tuning global parameters, allowing the pre-trained CLIP knowledge to be effectively applied to dense prediction tasks. Next, for patch-level embedding of images, we introduce a patch-language contrast loss to optimize the effectiveness of the model in capturing fine-grained features of images. Then, we propose a patch-language Discrete Fourier transform (DFT) interaction module, leveraging DFT to extract multi-scale image features in the frequency domain, which improves the ability of the model to handle diverse vehicle sizes and complex traffic environments. Finally, a content-aware density map decoder is designed to generate accurate density map predictions through multi-layer convolution and progressive upsampling. Extensive experiments demonstrate that our proposed FCLIP-VC achieves state-of-the-art accuracy in zero-shot vehicle counting.