HapLearn Iteration 2

Group Members: Ahmed, Karthik, Rayan, Kattie

Objective

Our overall project is aimed towards Haptic Data Visualization (HDV). In iteration 1, we experimented with a simple bar chart using vibrotactile and auditory feedback. However, in this iteration, we want to experiment with more complex graph type i.e. scatterplot with multiple dimensions and assist user in graph exploration using Haptics.

Motivation

The motivation behind choosing a scatterplot is to see whether a complex graph such as above can be represented using Haptic feedback. We acknowledge that the animated graph aptly represents a lot of quantitative information and is probably the gold standard. Therefore, our motivation is not to compete against it but to see how far can haptic elements represent some of the additional quantitative information with a static scatter plot.

Approach

For this iteration, three of our group members including me had finally received their Haply devices. Therefore, we decided to pivot towards incorporating it into our project. Using the Haply, we wanted to experiment with navigation and haptic feedback while exploring the scatterplot. Therefore, we had following sub-tasks and division of tasks.

1. Construction of scatterplot
2. Adding navigation to a particular point (Rayan)
3. Map haptic feedback using Haply to a point’s value (Kattie and Ahmed)
4. Map vibrotactile feedback using actuators to the graph (Karthik)

Scatterplot

For creating the scatterplot, we started with constructing a scatterplot using Fisica library. However, as we added more points we faced issues in portraying overlapping points and controlling the size of the points. Therefore, we used the Grafica library available in processing. It takes in a csv file to add points to the chart and compiles a pretty scatterplot.

Navigation

From the scatterplot above, we can see that it can be difficult to navigate to a specific point as there is a lot of overlap between points. To solve this issue, we took inspiration from the PID lab 4 in this course and mapped the centers of these points to the target position. Therefore, by selecting a country, the target location changes and the haply moves to the respective country. This process required creating custom functions to map the coordinates of the points to the target location.

Navigation approach

This was a very challenging task. After creating the scatterplot above we wanted to add navigation component in it. We took inspiration from the PID Lab 4 and decided to use the P and D component. We decided not to use I component as we wanted a quick snapping action. Using a value of P=0.05, D=0.99, we were able to create an optimal navigation mechanism. As an example, we added two countries in the bottom i.e. United States and Somalia. In addition, we also added a text box where the user can type in the name of his desired country and the haply takes the user the respective point. The following is the final navigation sketch.

Navigation sketch

Haptic Feedback

In this step, we wanted to encode a haptic feedback when the user hovers over a specific point using Haply. We decided to use damping property of Fisica objects to represent this feedback. I started off to create circular objects using the Fisica library. My goal was to add damping that is proportionate to a value in that point (eg GDP). Meanwhile, Kattie worked on adding bumpiness/vibrations to represent densely populated areas. This could allow us to represent an extra dimension via Haptic feedback.

code for plotting few points on scatterplot

code for incorporating damping in the data points

Sketch with Damping to represent Friction

Through this, I was able to create different damping effects. While navigating through the point, I felt some resistance. However, it was not clearly distinguishable for different points. But that was since I was not using the actual data set and there was no visual aid to indicate the values encoded by the damping factor.

Bumpiness and stickiness when in crowded area

In the above sketch, Kattie was able to add bumpiness and stickiness in the crowded regions. We felt that this can be useful to uniquely represent a cluster of countries or even continents.

The next step was integrating this component in the navigation component created in the previous step. My goal was to dynamically create an invisible Fisica circle at the target location. So that when the Haply takes the user to specific point, he can experience some friction proportional to a value of that point. This turned out to be very challenging as there were integration issues between Fisica objects (invisible FCircle) and Grafica objects (graph). The Fisica objects were not being instantiated in the sketch.

Vibrotactile Feedback

To explore representing data points using vibrotactile feedback, Karthik used vibrotactile actuators. He was able to represent the direction of the mean in the graph by changing the amplitude of the vibration. So, moving away from the mean gave subtler vibrations and moving towards the mean gave stronger vibrations. We felt that this was helpful in observing the important statistic of mean without any visual aid but for representing many points it would be too noisy for the user. Ideally, I want to combine these actuators with my Haply and experiment with both modalities, but since my haply was damaged already once so I am wary of this now.

Personal Contribution

Specifically, I was tasked with the exploration and integration of texture on top of the navigation sketch. I was able to map a proportionate damping factor on each point using Fisica objects. It worked for smaller number of points but for several points it turned out to be infeasible. Using this, I wanted to instantiate an invisible Fisica circle in the navigation sketch and texture to it. But due to integration issues between Fisica objects and Grafica objects, I was not able to add it to the navigation sketch.

Reflection

There was a steep learning curve since, as we had to pivot towards using Haply and I had received mine late. The built in examples for the Fisica library and labs helped a lot in getting started. I feel that our navigation sketch is great for quickly jumping to a point in a very densely populated scatter chart so it can be counted a success. Similarly, locating mean with the vibrotactile feedback was a success. The texture mapping via damping was also good but only for smaller number of points. Also, as I couldn’t integrate it to the navigation sketch so it can be counted as a failure.

Next Iteration

For the next iteration, I would want to build on our navigation sketch and try to add different textures without the Fisica objects as Fisica took a lot of time without any success. Using this, we want to represent extra dimensions such as GDP on top of an existing scatterplot.

References

S. Paneels and J. C. Roberts, “Review of Designs for Haptic Data Visualization,” in IEEE Transactions on Haptics, vol. 3, no. 2, pp. 119–137, April-June 2010, doi: 10.1109/TOH.2009.44.

Broken Haply wire (Luckily I was able to get soldered from hardware store) It was scary as the socket was made of plastic and I was afraid that it would melt the socket as well.