Abstract
In this paper we present Jarvis, a multimodal explorer and navigation system for biocuration data, from both curated sources and text-derived datasets. This system harnesses voice and haptic control for a bioinformatic research context, specifically manipulation of data visualizations such as heatmaps and word clouds showing related terms in the dataset. We combine external speech systems with Clustergrammer [1] for the generation of bioinformatic queries, the BoB interface [2] for answering queries in that domain, and the VoxML framework
[12] for manipulating the results and semantic grounding. We deploy the resulting system to iOS on an iPad for use by researchers over a test dataset of gene expression in tumor samples. The intent is to integrate multimodal control (here voice and haptics), so as to facilitate interaction with and analysis of data, taking advantages of using both modalities.
