University of Pittsburgh

ContactDB: Analyzing and Predicting Grasp Contact via Thermal Imaging

Associate Professor, Georgia Institute of Technology and Staff Scientist, Argo AI
Friday, February 8, 2019 - 12:30pm - 1:30pm

Grasping and manipulating objects is an important human skill. Since contact between hand and object is fundamental to grasping, measuring it can lead to important insights. However, observing contact through external sensors is challenging because of occlusion and the complexity of the human hand. We present ContactDB, a novel dataset of contact maps for household objects that captures the rich hand-object contact during grasping, enabled by use of a thermal camera. Participants in our study grasp 3D printed objects with a post-grasp functional intent. ContactDB includes 3750 3D meshes of 50 household objects textured with contact maps and 375K frames of synchronized RGB-D+thermal images. To the best of our knowledge, this is the first large-scale dataset that records detailed contact maps for functional human grasps. Analysis of this data shows the influence of functional intent and object size on grasping, the tendency to touch/avoid "active areas" on the object surface, and the importance of palm and lower finger contact. Finally, we learn to predict diverse contact patterns for unseen objects by using state-of-the-art image translation and 3D convolution algorithms.

Bio: James Hays an associate professor of computing at Georgia Institute of Technology since fall 2015. Since 2017, James also work with Argo AI to create self-driving cars. Previously, James was the Manning assistant professor of computer science at Brown University. James received his Ph.D. from Carnegie Mellon University and was a postdoc at Massachusetts Institute of Technology. His research interests span computer vision, computer graphics, robotics, and machine learning. He studies fundamental computer vision problems such as object detection and place recognition and their applications to domains such as robotics, biology, and graphics. James's research often involves finding new data sources to exploit (e.g. geotagged imagery) or creating new datasets where none existed (e.g. human sketches).  James is the recipient of the NSF CAREER award and Sloan Fellowship.

Copyright 2009–2019 | Send feedback about this site