Research Projects


Visualize Your Story



avatar
UNDER REVIEW: Story illustration is the task of illustrating a natural language story with a coherent sequence of images. We propose a more generalized task: Many-to-Many Story Illustration, i.e. automatic visualization of a textual story by a coherent sequence of images of any length. We introduce a novel many-to-many dataset created by aligning natural language descriptions with corresponding coherent sequence of images sampled from video clips. An end-to-end encoder-decoder neural architecture is proposed that sequentially retrieves a coherent sequence of images given an input story. User studies show the applicability of the proposed task and dataset and reveal that the illustrations generated by the proposed model are comparable to the ground truth. Additionally, evaluation on an existing one-to-one storytelling dataset shows the model’s generalization capability.



GitEvolve



avatar
GitEvolve is a multi-task sequential deep network for simulation of future github events given past events for a particular repository. Each event is characterized by a 3-tuple including type of the event, user cluster id and the time stamp of the event. The three tasks are trained simultaneously. Social structure of Github is further modelled by automatically learning graph based representation for each repository. The effectiveness of the proposed technique is evaluated using an array of metrics.




Show Me a Story



avatar
Story Illustration is the problem of retrieving/generating a sequence of images, given a natural language story as input. We propose a hierarchical GRU network that learns a representation for the input story and use it to retrieve an ordered set of images from a dataset. In its core, the model is designed to explicitly model coherence between sentences in a story optimized over sequential order embedding based loss function. The performance is qualitatively and quantitatively evaluated.

Anti-Forensic Enhancement



avatar
Digital images can be convincingly edited using image editing tools. In order to identify such image pro-cessing operations, various forensic techniques have been proposed. In response, anti-forensic operationsdesigned as counter-measures have been devised. We propose an anti-forensic technique tocounter spatial domain forensic detectors and demonstrate its accuracy on popular image manipulation operations such as median filtering and contrast enhancement. Through a series of experiments, we prove that the proposed algorithm canseverely degrade the performance of median filtering and contrast enhancement detectors. The proposedalgorithm also outperforms popular anti-forensic algorithms.

Image Filtering Detection



avatar
Smart image editing and processing techniques make it easier to manipulate an image convincingly and also hide any artifacts of tampering using operations like filtering, compression and/or format conversion to suppress forgery artifacts. We propose an algorithm to detect if a given image has undergone filtering based enhancement irrespective of the format of image or the type of filter applied using spatial domain quantization noise.