Machine Learning for Tangible Effects: Natural Language Processing for Uncovering the Illicit Massage Industry & Computer Vision for Tactile Sensing

I explore two questions in this thesis: how can computer science be used to fight human trafficking? And how can computer vision create a sense of touch? I use natural language processing (NLP) to monitor the United States illicit massage industry (IMI), a multi-billion dollar industry that offers not just therapeutic massages but also commercial sexual services. Employees of this industry are often immigrant women with few job opportunities, leaving them vulnerable to fraud, coercion, and other facets of human trafficking. Monitoring spatiotemporal trends helps prevent trafficking in the IMI. By creating datasets with three publicly-accessible websites: Google Places, Rubmaps, and AMPReviews, combined with NLP techniques such as bag-of-words and Word2Vec, I show how to derive insights into the labor pressures and language barriers that employees face, as well as the income, demographics, and societal pressures affecting sex buyers. I include a call-to-action to other researchers given these datasets. I also consider how to creating synthetic financial data, which can aid with counter-trafficking in the banking sector. I use an agent-based model to create both tabular and payee-recipient graph data. I then consider the role of computer vision in making tactile sensors. I report on a novel sensor, the Digger Finger, that adapts the Gelsight sensor to finding objects in granular media. Changes include using a wedge shape to facilitate digging, replacing the internal lighting LEDs with fluorescent paint, and adding a vibrator motor to counteract jamming. Finally, I also show how to use a webcam and a printed reference marker, or fiducial, to create a low-cost six-axis force-torque sensor. This sensor is up to a hundred times less expensive than commercial sensors, allowing for a wider range of applications. For this and earlier chapters I release design files and code as open source.

Digger Finger: GelSight Tactile Sensor for Object Identification Inside Granular Media

In this paper we present an early prototype of the Digger Finger that is designed to easily penetrate granular media and is equipped with the GelSight sensor. Identifying objects buried in granular media using tactile sensors is a challenging task. First, particle jamming in granular media prevents downward movement. Second, the granular media particles tend to get stuck between the sensing surface and the object of interest, distorting the actual shape of the object. To tackle these challenges we present a Digger Finger prototype. It is capable of fluidizing granular media during penetration using mechanical vibrations. It is equipped with high resolution vision based tactile sensing to identify objects buried inside granular media. We describe the experimental procedures we use to evaluate these fluidizing and buried shape recognition capabilities. A robot with such fingers can perform explosive ordnance disposal and Improvised Explosive Device (IED) detection tasks at a much a finer resolution compared to techniques like Ground Penetration Radars (GPRs). Sensors like the Digger Finger will allow robotic manipulation research to move beyond only manipulating rigid objects.

Low-Cost Fiducial-based 6-Axis Force-Torque Sensor

Commercial six-axis force-torque sensors suffer from being some combination of expensive, fragile, and hard-to-use. We propose a new fiducial-based design which addresses all three points. The sensor uses an inexpensive webcam and can be fabricated using a consumer-grade 3D printer. Open-source software is used to estimate the 3D pose of the fiducials on the sensor, which is then used to calculate the applied force-torque. A browser-based (installation free) interface demonstrates ease-of-use. The sensor is very light and can be dropped or thrown with little concern. We characterize our prototype in dynamic conditions under compound loading, finding a mean $R^2$ of over 0.99 for the $F_x, F_y, M_x$, and $M_y$ axes, and over 0.87 and 0.90 for the $F_z$ and $M_z$ axes respectively. The open source design files allow the sensor to be adapted for diverse applications ranging from robot fingers to human-computer interfaces, while the sdesign principle allows for quick changes with minimal technical expertise. This approach promises to bring six-axis force-torque sensing to new applications where the precision, cost, and fragility of traditional strain-gauge based sensors are not appropriate. The open-source sensor design can be viewed at http://sites.google.com/view/fiducialforcesensor.