Ishan Dave

I am a fifth-year Ph.D. student in the Center for Research in Computer Vision (CRCV), University of Central Florida (UCF), advised by Prof. Mubarak Shah. .

Email  /  Google Scholar  /  Github  /  LinkedIn  /  CV (it's a public version, for a detailed version feel free to ask!)

Updates

  Oct'24: Successfully defended my Ph.D dissertation!
  Aug'24: SPAct Patent Approved! my first patent as the primary inventor 💥
  Jul'24: 2 First author papers accepted at ECCV 2024: Oral presentation! (Top 3% papers)💥💥
  Jun'24: Selected as Outstanding Reviewer of CVPR 2024! (top 2% among 10,000 reviewers)🥇
  May'24: Started internship at Apple, Cupertino, CA
  Dec'23: A First author paper "No More Shortcuts" accepted to AAAI 2024 💥
  Jul'23: A First author paper "Event-TransAct" accepted at IROS 2023 💥
  Jul'23: A paper "TeD-SPAD" accepted at ICCV 2023
  May'23: Started summer internship at Adobe, San Jose, CA
  Mar'23: A First author paper "TimeBalance" accepted to CVPR 2023 💥
  Jan'23: A paper "TransVisDrone" accepted at ICRA 2023
  May'22: Started summer internship at Adobe, USA (remote- Florida)
  Mar'22: A First author paper "TCLR" accepted to CVIU 2022 💥
  Mar'22: A First author paper "SPAct" accepted to CVPR 2022 💥
  Jan'21: Our Gabriella paper has been awarded the best scientific paper award at ICPR 2020

• Enhanced stable diffusion models for image editing by leveraging vision-language multimodal foundation models.
• Trained diffusion models on a large-scale, high-resolution dataset of 10M samples.
• Reproduced and outperformed state-of-the-art image editing methods using a novel approach, achieving superior results.

• Enhanced the fine-grained capabilities of existing video retrieval methods.
• Worked on large-scale video galleries with millions of samples.
• Filed a patent and had a paper accepted at ECCV 2024.

• Developed a novel self-supervised video representation framework by reformulating temporal self-supervision as frame-level recognition tasks and introducing an effective augmentation strategy to mitigate shortcuts.
• Achieved state-of-the-art performance on 10 video understanding benchmarks across linear classification (Kinetics400, HVU, SSv2, Charades), video retrieval (UCF101, HMDB51), and temporal correspondence (CASIA-B).
• Published a paper at AAAI 2024.

I have a broad interest in computer vision and machine learning. My primary research focuses on video understanding: self/semi supervised learning and action recognition. My recent research also includes enhancing the fine-grained video understanding of the large foundational models and improving multi-modal generative AI for image editing applications. I have also worked on various robotics-related vision tasks like event-camera-based action recognition and drone-to-drone detections from videos.
Below is a selected list of my works (in chronological order), representative papers are highlighted.

Temporal video alignment synchronizes key events like object interactions or action phase transitions in two videos, benefiting video editing, processing, and understanding tasks. Existing methods assume a given video pair, limiting applicability. We redefine this as a search problem, introducing Alignable Video Retrieval (AVR), which identifies and synchronizes well-alignable videos from a large collection. Key contributions include DRAQ, a video alignability indicator, and a generalizable frame-level video feature design.

We introduce Alignability-Verification-based Metric learning for semi-supervised fine-grained action recognition. Using dynamic time warping (DTW) for action-phase-aware comparison, our learnable alignability score refines pseudo-labels of the video encoder. Our framework, FinePseudo, outperforms prior methods on fine-grained action recognition datasets. Additionally, it demonstrates robustness in handling novel unlabeled classes in open-world setups.

We propose a Domain Adaptive Contrastive objective to bridge the gap between High and Low Cost Microscopes. On the publicly available large-scale M5 dataset, our proposed method shows a significant improvement of 16% over the state-of-the-art methods in terms of the mean average precision metric (mAP), provides a 21× speed-up during inference, and requires only half as many learnable parameters as the prior methods.

We demonstrate experimentally that our more challenging frame-level task formulations and the removal of shortcuts drastically improve the quality of features learned through temporal self-supervision. Our extensive experiments show state-of-the-art performance across 10 video understanding datasets, illustrating the generalization ability and robustness of our learned video representations.

We propose TeD-SPAD, a privacy-aware video anomaly detection framework that destroys visual private information in a self-supervised manner. In particular, we propose the use of a temporally-distinct triplet loss to promote temporally discriminative features, which complements current weakly-supervised VAD methods.

We propose a video transformer-based framework for event-camera based action recognition, which leverages event-contrastive loss and augmentations to adapt the network to event data. Our method achieved state-of-the-art results on N-EPIC Kitchens dataset and competitive results on the standard DVS Gesture recognition dataset, while requiring less computation time compared to competitive prior approaches.

We propose a student-teacher semi-supervised learning framework, where we distill knowledge from a temporally-invariant and temporally-distinctive teacher. Depending on the nature of the unlabeled video, we dynamically combine the knowledge of these two teachers based on a novel temporal similarity-based reweighting scheme. State-of-the-art results on Kinetics400, UCF101, HMDB51.

We propose a simple yet effective framework, TransVisDrone, that provides an end-to-end solution with higher computational efficiency. We utilize CSPDarkNet-53 network to learn object-related spatial features and VideoSwin model to improve drone detection in challenging scenarios by learning spatio-temporal dependencies of drone motion.

For the first time, we present a novel training framework that removes privacy information from input video in a self-supervised manner without requiring privacy labels. We train our framework using a minimax optimization strategy to minimize the action recognition cost function and maximize the privacy cost function through a contrastive self-supervised loss.

We propose a new temporal contrastive learning framework for self-supervised video representation learning, consisting of two novel losses that aim to increase the temporal diversity of learned features. The framework achieves state-of-the-art results on various downstream video understanding tasks, including significant improvement in fine-grained action classification for visually similar classes.

We propose a realtime, online, action detection system which can generalize robustly on any unknown facility surveillance videos. We tackle the challenging nature of action classification problem in various aspects like handling the class-imbalance training using PLM method and learning multi-label action correlations using LSEP loss. In order to improve the computational efficiency of the system, we utilize knowledge distillation.

Gabriella consists of three stages: tubelet extraction, activity classification, and online tubelet merging. Gabriella utilizes a localization network for tubelet extraction, with a novel Patch-Dice loss to handle variations in actor size, and a Tubelet-Merge Action-Split (TMAS) algorithm to detect activities efficiently and robustly.

Feel free to steal this website's source code. Do not scrape the HTML from this page itself, as it includes analytics tags that you do not want on your own website — use the github code instead. Also, consider using Leonid Keselman's Jekyll fork of this page.