Dylan Cashman

Advancements in accessibility technologies such as low-cost swell form printers or refreshable tactile displays promise to allow blind or low-vision (BLV) people to analyze data by transforming visual representations directly to tactile representations. However, it is possible that design guidelines derived from experiments on the visual perception system may not be suited for the tactile perception system. We investigate the potential mismatch between familiar visual encodings and tactile perception in an exploratory study into the strategies employed by BLV people to measure common graphical primitives converted to tactile representations. First, we replicate the Cleveland and McGill study on graphical perception using swell form printing with eleven BLV subjects. Then, we present results from a group interview in which we describe the strategies used by our subjects to read four common chart types. While our results suggest that familiar encodings based on visual perception studies can be useful in tactile graphics, our subjects also expressed a desire to use encodings designed explicitly for BLV people. Based on this study, we identify gaps between the perceptual expectations of common charts and the perceptual tools available in tactile perception. Then, we present a set of guidelines for the design of tactile graphics that accounts for these gaps. Supplemental material is available at https://osf.io/3nsfp/?view_only=7b7b8dcbae1d4c9a8bb4325053d13d9f.

In this interactive article, we present an interactive game that represents the types of tasks solved by machine learning algorithms. We use this game to motivate the definition of Probably Approximately Correct (PAC) learning, illustrating a proof of PAC learnability for Empirical Risk Minimization (ERM). Then, we show three types of vulnerabilities of ERM that often occur in applied machine learning - domain mismatch, dependencies in data, and an incorrect model class. We conclude by arguing for the need for visualization to identify these issues in a modeling dataset.

Findings Assoc. Comput. Linguistics: EMNLP 2024 15447-15459 (2024) Large language model (LLM) agents show promise in an increasing number of
domains. In many proposed applications, it is expected that the agent reasons
over accumulated experience presented in an input prompt. We propose the OEDD
(Operationalize Experience Despite Distraction) corpus, a
human-annotator-validated body of scenarios with pre-scripted agent histories
where the agent must make a decision based on disparate experiential
information in the presence of a distractor. We evaluate three state-of-the-art
LLMs (GPT-3.5 Turbo, GPT-4o, and Gemini 1.5 Pro) using a minimal
chain-of-thought prompting strategy and observe that when (1) the input context
contains over 1,615 tokens of historical interactions, (2) a crucially
decision-informing premise is the rightful conclusion over two disparate
environment premises, and (3) a trivial, but distracting red herring fact
follows, all LLMs perform worse than random choice at selecting the better of
two actions. Our code and test corpus are publicly available at:
https://github.com/sonnygeorge/OEDD .

Dimensionality reduction is used as an important tool for unraveling the complexities of high-dimensional datasets in many fields of science, such as cell biology, chemical informatics, and physics. Visualizations of the dimensionally-reduced data enable scientists to delve into the intrinsic structures of their datasets and align them with established hypotheses. Visualization researchers have thus proposed many dimensionality reduction methods and interactive systems designed to uncover latent structures. At the same time, different scientific domains have formulated guidelines or common workflows for using dimensionality reduction techniques and visualizations for their respective fields. In this work, we present a critical analysis of the usage of dimensionality reduction in scientific domains outside of computer science. First, we conduct a bibliometric analysis of 21,249 academic publications that use dimensionality reduction to observe differences in the frequency of techniques across fields. Next, we conduct a survey of a 71-paper sample from four fields: biology, chemistry, physics, and business. Through this survey, we uncover common workflows, processes, and usage patterns, including the mixed use of confirmatory data analysis to validate a dataset and projection method and exploratory data analysis to then generate more hypotheses. We also find that misinterpretations and inappropriate usage is common, particularly in the visual interpretation of the resulting dimensionally reduced view. Lastly, we compare our observations with recent works in the visualization community in order to match work within our community to potential areas of impact outside our community. By comparing the usage found within scientific fields to the recent research output of the visualization community, we offer both validation of the progress of visualization research into dimensionality reduction and a call for action to produce techniques that meet the needs of scientific users.

Presenting a predictive model's performance is a communication bottleneck that threatens collaborations between data scientists and subject matter experts. Accuracy and error metrics alone fail to tell the whole story of a model - its risks, strengths, and limitations - making it difficult for subject matter experts to feel confident in their decision to use a model. As a result, models may fail in unexpected ways or go entirely unused, as subject matter experts disregard poorly presented models in favor of familiar, yet arguably substandard methods. In this paper, we describe an iterative study conducted with both subject matter experts and data scientists to understand the gaps in communication between these two groups. We find that, while the two groups share common goals of understanding the data and predictions of the model, friction can stem from unfamiliar terms, metrics, and visualizations - limiting the transfer of knowledge to SMEs and discouraging clarifying questions being asked during presentations. Based on our findings, we derive a set of communication guidelines that use visualization as a common medium for communicating the strengths and weaknesses of a model. We provide a demonstration of our guidelines in a regression modeling scenario and elicit feedback on their use from subject matter experts. From our demonstration, subject matter experts were more comfortable discussing a model's performance, more aware of the trade-offs for the presented model, and better equipped to assess the model's risks - ultimately informing and contextualizing the model's use beyond text and numbers.

Projection techniques are often used to visualize high-dimensional data, allowing users to better understand the overall structure of multi-dimensional spaces on a 2D screen. Although many such methods exist, comparably little work has been done on generalizable methods of inverse-projection - the process of mapping the projected points, or more generally, the projection space back to the original high-dimensional space. In this article we present NNInv, a deep learning technique with the ability to approximate the inverse of any projection or mapping. NNInv learns to reconstruct high-dimensional data from any arbitrary point on a 2D projection space, giving users the ability to interact with the learned high-dimensional representation in a visual analytics system. We provide an analysis of the parameter space of NNInv, and offer guidance in selecting these parameters. We extend validation of the effectiveness of NNInv through a series of quantitative and qualitative analyses. We then demonstrate the method's utility by applying it to three visualization tasks: interactive instance interpolation, classifier agreement, and gradient visualization.

In a 2011 article in this journal entitled "Whites See Racism as a Zero-Sum Game That They Are Now Losing" ( , 215-218), Norton and Sommers assessed Black and White Americans' perceptions of anti-Black and anti-White bias across the previous 6 decades-from the 1950s to the 2000s. They presented two key findings: White (but not Black) respondents perceived decreases in anti-Black bias to be associated with increases in anti-White bias, signaling the perception that racism is a zero-sum game; White respondents rated anti-White bias as more pronounced than anti-Black bias in the 2000s, signaling the perception that they were losing the zero-sum game. We collected new data to examine whether the key findings would be evident nearly a decade later and whether political ideology would moderate perceptions. Liberal, moderate, and conservative White (but not Black) Americans alike believed that racism is a zero-sum game. Liberal White Americans saw racism as a zero-sum game they were winning by a lot, moderate White Americans saw it as a game they were winning by only a little, and conservative White Americans saw it as a game they were losing. This work has clear implications for public policy and behavioral science and lays the groundwork for future research that examines to what extent racial differences in perceptions of racism by political ideology are changing over time.

Visual exploration of large multi-dimensional datasets has seen tremendous progress in recent years, allowing users to express rich data queries that produce informative visual summaries, all in real time. Techniques based on data cubes are some of the most promising approaches. However, these techniques usually require a large memory footprint for large datasets. To tackle this problem, we present NeuralCubes: neural networks that predict results for aggregate queries, similar to data cubes. NeuralCubes learns a function that takes as input a given query, for instance, a geographic region and temporal interval, and outputs the result of the query. The learned function serves as a real-time, low-memory approximator for aggregation queries. Our models are small enough to be sent to the client side (e.g. the web browser for a web-based application) for evaluation, enabling data exploration of large datasets without database/network connection. We demonstrate the effectiveness of NeuralCubes through extensive experiments on a variety of datasets and discuss how NeuralCubes opens up opportunities for new types of visualization and interaction.

Statistical regression methods can help pharmaceutical organizations improve the quality of their pharmacovigilance by predicting the expected quantity of adverse events during a trial. However, the use of statistical techniques also changes the risk profile of any downstream tasks, due to bias and noise in the model's predictions. That risk profile must be clearly understood, documented, and communicated across many different stakeholders in a highly regulated environment. Aggregated performance metrics such as explained variance or mean average error fail to tell the whole story, making it difficult for subject matter experts to feel confident in deciding to use a model. In this work, we describe guidelines for communicating regression model performance for models deployed in predicting adverse events. First, we describe an interview study in which both data scientists and subject matter experts within a pharmaceutical organization describe their challenges in communicating and understanding regression performance. Based on the responses in this study, we develop guidelines for which visualizations to use to communicate performance, and use a publicly available trial safety database to demonstrate their use.

Most visual analytics systems assume that all foraging for data happens before the analytics process; once analysis begins, the set of data attributes considered is fixed. Such separation of data construction from analysis precludes iteration that can enable foraging informed by the needs that arise in-situ during the analysis. The separation of the foraging loop from the data analysis tasks can limit the pace and scope of analysis. In this paper, we present CAVA, a system that integrates data curation and data augmentation with the traditional data exploration and analysis tasks, enabling information foraging in-situ during analysis. Identifying attributes to add to the dataset is difficult because it requires human knowledge to determine which available attributes will be helpful for the ensuing analytical tasks. CAVA crawls knowledge graphs to provide users with a a broad set of attributes drawn from external data to choose from. Users can then specify complex operations on knowledge graphs to construct additional attributes. CAVA shows how visual analytics can help users forage for attributes by letting users visually explore the set of available data, and by serving as an interface for query construction. It also provides visualizations of the knowledge graph itself to help users understand complex joins such as multi-hop aggregations. We assess the ability of our system to enable users to perform complex data combinations without programming in a user study over two datasets. We then demonstrate the generalizability of CAVA through two additional usage scenarios. The results of the evaluation confirm that CAVA is effective in helping the user perform data foraging that leads to improved analysis outcomes, and offer evidence in support of integrating data augmentation as a part of the visual analytics pipeline.