This work was completed as a final project for Bart Selman’s CS 6700: Advanced Artificial Intelligence graduate course at Cornell. For this project, I designed and fabricated a soft touch sensor using an elastomeric foam with embedded infrared (IR) LEDs and receivers. The soft touch sensor identifies the location and depth of a press into the foam. I developed a testing apparatus to collect training data by probing the sensor at known positions and collecting data from the IR receivers. For each press, eight distinct patterns are applied to 32 LEDs and the voltages across 32 phototransistors are recorded, yielding 256 training features. I then used random forests to generate a model to predict the probe’s location. First, I used a classification model to determine whether the probe is pressing the sensor (with >98% accuracy). For cases where the probe is pressing the foam, the root-mean-square error (RMSE) of the predicted location is <1.6mm, and the RMSE of the predicted depth is <0.6mm.

I used an Arduino DUE microcontroller to drive 32 IR LEDs (TSML1020; Vishay Semiconductors) and read voltages across each of the IR phototransistors (TSML1020; Vishay Semiconductors). I used two multiplexers (CD74HC4067; Texas Instruments) to interface with the phototransistors so that two analog input pins on the Arduino DUE can measure the voltage across each of the 32 phototransistors. The schematic below shows the design of five custom circuit boards that integrate the electronics for the soft touch sensor. Each of the four edge pieces hosts eight LEDs and eight phototransistors. These edge pieces attach to the connector board using right angle headers. The connector board also hosts the multiplexer breakout boards and Arduino DUE.

A laser cut acrylic enclosure houses the electronics and elastomeric foam for light transmission. The acrylic surface also procides a rigid surface under the foam that ensures reproducible results as the foam is deformed.The outer housing has an opening to allow connections to the USB port on the Arduino DUE. The elastomeric foam is a 12.5mm thick polyurethane sheet (8643K502; McMaster-Carr). I cut the foam cut slightly larger than the opening so that compression holds the foam in place. I covered the foam layer with a room temperature vulcanization (RTV) silicone blend (Ecoflex-30; Smooth-On Inc.). I mixed a red-colored pigment (Silc Pig; Smooth-On Inc.) with the silicone to prevent light transmission, minimizing the effect of ambient light on the soft sensor.

For this project, I performed all machine learning using the caret Package [6] built for R. Prior to any data analysis, I randomly sampled the recorded data and reserved 20% as a testing set. For an initial proof of concept, I used a random forest algorithm. I used 500 trees for each model and the number of variables available for splitting at each tree node (mtry) was tuned using five-fold cross-validation.

The first model I developed is a classification of whether or not the sensor is being touched. Each of the 5120 data samples are labeled with a Boolean state for touching or not. I randomly reserved 20% of the data as a fixed testing set. Of the remaining training set, I created partitions ranging from 1%-100% of the training data. I then used random forests to build a classifier to predict whether the device is being touched. Classification accuracy results are shown below. Using just 20% (819 samples) of the training data, a classification model achieves an accuracy of 95.1%. Using the full training set (4096 samples), the classification accuracy increases to 98.2%.

After developing a model for classifying touch, the next step was to build a model to predict the location and depth of the touch. As described for the touch model, I reserved 20% of the available data as a testing set. For this regression problem, I used data for which the Z-depth is 4, 6, 8 or 10mm. This provided a testing set of 816 samples, and an available training set of up to 3280 samples. I varied the size of the training set and fit models for X, Y, and Z location using each subset of the data. The resulting root-mean-square error (RMSE) and training time are shown below. Using just 40% (1312 samples) of the training data, the RMSE for the X and Y location predictions is less than 3mm and the RMSE for the z depth prediction is less than 0.76mm. With the full training set (3280 samples), the RMSE for the X and Y location predictions is less than 1.6mm, and the RMSE for the Z depth prediction is 0.59mm.

The predictions for X, Y, and Z locations are shown below. Each set of predictions is based on models fit using the full training sets. In each figure, the top plot shows the predicted values for each location and the bottom plot shows the difference between predicted and actual values. For the X predictions, the data is split into two sets because the prediction accuracy dramatically decreases for X > 55mm. For X <= 55mm, the magnitude of the prediction error tends to be less than 0.5mm, but increases to nearly 20mm near the right edge of the touch sensor.

For the Y predictions, the magnitude of prediction error tends to be less than 1mm. The outliers, however, then to be skewed towards the center of the touch sensor. Outliers near the bottom of the touch sensor have positive error, while those near the top tend to have negative error. This same trend is evident for the Z predictions. Most predictions for Z however, are within 0.5mm of the nominal value.

These initial results show that elastomeric foams coupled with infrared LEDs and phototransistors are a promising technology for creating a soft touch sensor. The accuracy of the preliminary tests are sufficient for many human robot interaction tasks; the RMSE of the predicted location is <1.6mm and the RMSE of the predicted depth is <0.6mm. More sophisticated machine learning algorithms and more training data could further improve these results. Moreover, much of the error in mean values came from outliers. Averaging multiple samples prior to input into the trained model may mitigate the effect of these
outliers. With these improvements, this soft foam touch sensor may imbue robots with a sense of touch rivalling that of humans. This device is of particular interest in robotics due to its flexible form factor and low cost. The elastomeric foams used in this device are easily cast into 3D shapes to form a tactile skin for a soft robot. Furthermore, this initial prototype cost approximately $150, including the cost of custom PCB fabrication. The actual sensors and emitters cost less than $1 each, even when bought in quantities of one.

While this project only tested scenarios with a single press into the foam actuator, it is likely that the device is capable of multi-touch operation. This device may also be suitable for integration into stretchable applications (e.g. soft robotics) where the sensor is stretching in addition to being pressed. More advanced training routines would be needed to decouple device strain from external pressing. Decoupling strain and external stimuli would enable significant improvements in the closed-loop feedback control of soft robotic systems.

[1] Mac Murray et al. (2015) Poroelastic foams for simple fabrication of complex soft robots. Advanced Materials.

[2] Sugiura et al. (2011) FuwaFuwa: Detecting shape deformation of soft objects using directional photoreflectivity measurement. UIST.

[3] Processing: https://processing.org/

[4] G-code: https://en.wikipedia.org/wiki/G-code

[5] GRBL: http://bengler.no/grbl

[6] caret Package: http://topepo.github.io/caret/index.html