Improving performances of data gloves based on bend sensors

Giovanni Saggio Department of Electronic Engineering, University of Rome Tor Vergata; Giuseppe Latessa Department of Electronic Engineering, University of Rome Tor Vergata; Stefano Bocchetti Department of Electronic Engineering, University of Rome Tor Vergata; Carlo Alberto Pinto Department of Computer Engineering, University of Rome Tor Vergata; Dave Beck Director of Engineering Flexpoint Sensor Systems

ABSTRACT

Data gloves are of main importance when it is necessary to measure finger static and dynamic postures of human hand. An advantageous cost to reliability ratio to realize data gloves is adopting bend sensors to measure each finger joints.

We validated our novel array configurations by means of standard measurement procedure but with some minor differences to overcome recognized problems. Obtained results are encouraging.

1. INTRODUCTION

Data gloves can find applications in really many fields regarding social, medical, work, sport and entertainment aspects.

In the social area the data glove can be applied for sign language recognition (Kuroda et al., 2004; Mehdi and Khan, 2002), as alternative to the actual pc input devices (de la Hamette, 2002; Kolsch and Tur, 2002), as a tool in domestic and remote assistance, as an appliance to design ergonomic devices.

The medical field reports advantages for patient motor therapy, rehabilitation (Morrow et al., 2006) and tele-rehabilitation (Heuser et al., 2007), post-surgical evaluation, tele-operations (Hong and Tan, 1989), estimation of functional assessment (Gentner and Classen, 2009; Micera et al., 2003; Simone et al., 2007) or disability. But also doctors can take advantages of their education by manipulating 3D virtual anatomic parts (Székely and Satava, 1999) or being trained for virtual surgery (Satava and Jones, 1998).

Thanks to data gloves, workers can be skilled, simulating the consequences of their manipulations in a virtual setting or in hazardous environments for safety purposes, professional staff can be formed such as soldiers (Yao and Zhang, 2006), astronauts, firefighters, etc., their actions (Micera et al., 2002) or the ergonomics of their environment can be evaluated, programmers can be aided with automatic programming tools (Biggs and MacDonald, 2003), people can be helped in remote apparatus control, can be supported in gesture recognitions, can be assisted in design and manufacturing, even before the actual construction of goodies, by means of interactions with computer generated environments.

A data glove can be a useful tool for the sport field where perfect hand static and dynamic posture are essential to obtain the requested goal (in golfing, cricketing, swimming, ..), so hand posture measurement registration and further data analysis can furnish important elements for physical performance evaluations.

The entertainment field can utilize the data glove for gaming and videogaming applications, for computer generating characters (Damasio and Musse, 2002), for multimedia, for art (Keefe et al, 2001) and music appliances (Mulder, 2000) (playing a virtual instrument, sound compositing or sculpting) since to a single hand gesture can be associated an event or a musical note or a chord.

2. DATA GLOVE REALIZATION

Data glove can be realized with the sensor part based on different principles:

• Optic: cameras with or without reference markers (Degeorges et al., 2005), fibers, photocells based systems;
• Magnetic: Hall effect (Villella et al., 2004), inductcoders (Kuroda et al., 2004);
• Physic: pressure (Karlsson et al., 1998), ultrasound (Hahn et al., 1995);
• Electric: potentiometers (Zurbrügg, 2003), capacitances;
• etc..

Among all the possibilities, the utilization of bend sensors, capable of changing their resistance value when bent, can assure an advantageous cost to reliability ratio. Bend sensors utilization for data glove applications have already been reported. Williams et al. (2000) placed flexion sensors over the dorsal aspects of the distal interphalangeal (DIP), proximal interphalangeal (PIP), and metacarpophalangeal (MCP) joints of the fingers and in the gussets of the glove. Noaman et al. (2008) attached flexion sensors on a hand exoskeleton structure used to facilitate putting on and taking off the glove. Simone et al. (2007) realized individual lycra sleeves for each joint to be monitored, and each sleeve contained a bend sensor encased in a thin plastic sheath. Gentner and Classen (2009) used two sewn layers of Lycra inserting the sensor between and utilizing this configuration for each finger joint.

3. SENSOR ARRAY

All these mentioned sensor exploitations as literature reports are interesting from different points of view. Anyway all of them have as a minimum a common denominator the fact that each sensor is fully independent and act separately from each other. This aspect presents some advantages but some drawbacks too.

Here we propose a different architecture with sensors disposed in an array configuration. The design has been made in collaboration with the Flexpoint Sensor Systems Inc. (www.flexpoint.com). In our design three sensors are placed adjacent on the same substrate, as depicted in fig. 1.

Figure 1: the sensor array configuration

We placed each array onto the dorsal part of each finger, every sensor on every joint, to realize our complete data glove, named Hiteg-glove, stands Hiteg (Health Involved Technical Engineering Group) our group name.

The array was mounted on a commercial glove made by a mix of Lycra and cotton materials with a reduced elasticity. The glove was comfortable enough during donning, doffing and use, as reported by users.

The array was designed in a way that sensors which measure the proximal interphalangeal (PIP in fig. 2) and the metacarpophalangeal (MCP) joints have the same resistance value when unbent (finger flat position) while the sensor on the distal interphalangeal joint has the half of that resistance value. This was a helpful expedient in designing the conditioning electronic circuitry.

Figure 2: representation of distal interphalangeal, proximal interphalangeal, metacarpophalangeal joints of human hand

Regarding the sensor utilized to measure postures of the metacarpophalangeal joint, a sort of slot was realized in the central part of its
longest dimension (see fig. 1), so to insert in it a tip previously fixed to the glove (see fig. 3) and to obtain an array sliding movement constrained into a predefined rail.

The array’s edge, in correspondence with the finger nail, was fixed to the glove. The array was then not inserted in a closed sleeve but in a open pocket a bit wider but not longer than the array itself. When finger flexed, the pocket’s open end allowed free sliding movements for the array maintained aligned with the finger thanks to the tip inserted into the slot of the array. Because of the sliding mechanism, the part of the sensor being flexed changes according to the amount of bending, as schematized in fig. 3. This can become an interesting fundamental aspect to trade on in next future in order to realize sensors with non uniform geometries so to obtain a desired pre-imposed electrical resistance variation vs. flexion force function.

Figure 3: the sensor is flexed in different sections, according to the amount of bending (a) 0° of bending, (b) 30° of bending, (c) 90° of bending for the metacarpophalangeal joint

4. ARRAY ADVANTAGES

We experienced some advantages in utilizing sensors in array configuration, meaning three sensors on a single substrate, used to measure the three joints of the same finger:

• One single substrate assures sensors to be kept always aligned with each other; otherwise sensors with the usual physical separation can produce inter misalignment during the glove usage
• The array is guaranteed to always remain aligned with the respective finger thanks to the predesigned rail configuration
• All the electrical contacts can be grouped in one tip of the array, so greatly reducing the problem of tangling of all the electrical wire to be connected to the external circuitry
• The array assures a single electrical mass for three different sensors, so the reference potential is exactly the same for all and electrical potential shifts from the reference value are avoided
• One array design can be easily adopted for all the fingers since it is sufficient to scale the design according to the finger sizes.

5. VALIDATION

We validated our novel array configurations by mean of the standard measure procedure. As a reference test method we adopted the generally accepted one proposed by Wise et al. (1990) and expanded by Dipietro et al. (2003), as further re-arranged by Simone et al. (2007), but with some minor differences to overcome recognized problems.

The tests were performed on six healthy individuals, four men and two women, aged 23-29. All of them were right-handed as determined by the Edinburgh Handedness Inventory (Oldfield, 1971) and had normal hand function. We used only a single version of the glove for all subjects, that is a M size which fit quite well each subject tested, except for subject 3 having a hand size slightly larger and subject 6 with a hand size slightly smaller respect to size M. The glove was placed on the dominant right hand for all. Before performing the predetermined tasks, all people were asked to execute some random movements for minutes so to become confident with the data glove, with the advantage of a visual feedback of a hand avatar reproducing the same movements on a pc screen. Customized plaster molds (see Fig. 4) were created individually for each subject, in a way that the hand joints could bent forming from 10° to 60° angles, depending on the particular joint and subject.

Figure 4: (a) two different molds, (b) hand positioned on the mold

Test steps can be summarized as:

Test A – Mold grip and glove on between data acquisition: The subjects, previously trained, were asked to hold (not to clench) the mold for 6 s and to release the mold placing the hand in a preimposed flat position on a desk for additional 6 s (this corresponds to 1 trial, for which the X-th data is acquired, averaging at least 130 measures), cycling 10 times (the average of all the ten X data forms 1 data block) without removing the glove. The forearm was in a prone-supine and the wrist in a neutral position. The procedures were repeated 10 times until obtaining 10 data blocks in total.

Test B – Mold grip and glove off between data acquisition: differing from test A the subjects were asked to take the glove off between each cycle, to evaluate donning and doffing effects on the measurement process

Test C – Hand flat and glove on between data acquisition: The subjects were asked to put the hand flat on a desk with the wrist fixed in a neutral position while the forearm pronated. Then the subjects had to clench the hand lightly in maximum flexion and to return it to the flat position. Every action for the standard duration of 6 s (1 trial) and cycling 10 times to form 1 data block always without removing the glove. The procedures were again repeated 10 times until obtaining 10 data blocks in total.

Test D – Hand flat and glove off between data acquisition: differing from test C the subjects were asked to take the glove off between each cycle During the tests we voluntarily utilized for the conditioning electronic circuitry a wired arrangement to be confident to not add eventual errors due to the wireless transmission system.

6. PROPOSED DIFFERENCES FROM THE STANDARD TEST METHOD

Differing from the reference test method, we realized the form of the molds with the aim to assure a comfortable closing hand position (see fig. 4) rather than arrange a roughly cylindrical aspect. In such a way we could avoid the Dipietro et al.’s (2003) observed problem that changes in grip force affected measured values, since no force is necessary to keep the hand on the position imposed by the mold. In such a manner it was also not necessary to compensate recommending the subjects to grip the mold with as low of a force as possible (Simone et al., 2007).

Again as a minor difference with the reference test method, the subjects had feedbacks of their movements via a virtual hand avatar on a computer screen reproducing the same movements, and an automatic beep informed when to change the hand position.

7. RESULTS AND COMMENTS

For the j-th data block and the k-th sensor, we calculated the range , its average value and the standard deviation SD values. The results were automatically obtained thanks to an acquisition software. In fig. 5 is reported an example of a typical data block in terms of Digital Volts (DV) vs number of samples. There are 14 degrees of freedom corresponding to hand joints interested in the flex-extension movements.

Figure 5 : data block in terms of Digital Volts (DV) vs number of samples

The acquired data blocks were then automatically converted in the values defined in order to evaluate the glove repeatability.

To overcome the problem of no meaningful measures acquired during the transition times, we eliminated with an automatic filtering procedure the 5% of values at the very begin and the very end of each trial. Indeed this choice was less stringent with respect to others reported in literature (Simone et al., 2007).

Table 1: Rk and SD values obtained from the tests

Stands the obtained measured results reported in tab. 1 and graphically represented in fig. 6 and fig. 7, we improved the performances of the data glove based on bend sensors with respect to the ones reached in literature. Let’s consider, for instance, the measures concerning Tests A and B, for which we registered an average RK=4.84°±1.34° and SD=1.6°±0.28° values, very interesting if compared to the meaning reported values RK=6.63°±1.86°, SD=2.10°±0.56° [Gentner, 2009] and RK=8.42°±1.35°, SD=2.78°±0.25° (Dipietro et al., 2003). Again an improvement was registered regarding the Tests C and D measurements since we obtained the values of RK=2.08°±0.5°, SD=0.56°±0.16° compared to RK=3.29°±1.29°, SD=1.07°±0.42° (Gentner and Classen, 2009). If we consider a more stringent filtering procedure, eliminating more than only our 5% of values as previous work suggests (Simone et al., 2007), we obtain for Tests A and B the values of RK=4.76°±1.34° and SD=1.58°±0.3° and for Tests C and D the values of RK=2.0°±0.65°, SD=0.5°±0.17° so even with a small further improvement.

With respect to the average, slightly different was the behavior of subject 3 (man), who obtained results just a little worst than the others. This was due to his hand size a bit larger than our standard M glove size, so he experienced some difficulties in closing the hand, when fingers assumed high angular bending degrees.

Differing from previously reported results (Dipietro et al., 2003), we experienced no meaningful differences between men and women tests, being the SD averaged value for men 0.98 and for women 1.29.

Figure 6: Range values for all the subjects and all the tests

Analyzing the fig. 6 it appears evident how the Test B had relatively worst results, while the data glove performances were better for Test C. This can demonstrate how our rail system repositioned quite well the array to flat arrangement with the hand returning to flat posture.

Figure 7: SD values for all the subjects and all the tests

The fig. 8 shows the SD average values obtained for each finger of all the subjects. As a comparison our data glove is referred to the most interesting ones reported in literature, i.e. the WV Glove by Gentner (2009) and the Human Glove by Dipietro (2003). The results are really quite encouraging and we can underline how the results can be even further improved if we do not consider the thumb values. In fact we adopted the array configuration with three sensors even for this finger, but this is not the ideal occurrence. Probably it would be better to adopt an array made of two sensors plus one sensor in a different position, but this aspect will be investigated in the future.

Figure 8: comparisons of the SD finger values among three different data gloves (Hiteg, WV, Shadow Monitor)

8. CONCLUSIONS

We demonstrated how the novel sensor array here proposed can be successfully exploited to realize data gloves with improved performances. The introduction of the array configuration demonstrated to represent an interesting improvement in accuracy and repeatability of data glove measurements. This is mostly due to the already discussed advantages (see section 4) which the array configuration can assure with respect to the standard single sensor layout. In particular the single substrate for the three sensors placed on the three joints of one finger guarantees the avoid misalignment among sensors while the rail configuration assures always array-finger alignment maintenance. Our work attests also a high correlation between Rk and SD parameters as just previously reported (Wise et al., 1990; Gentner and Classen, 2009).

As a final nice consideration, since the array realizes a tidier data glove, according to the Birkhoff’s (1933) curious speculative work, we can state to have increased the aesthetic value of our previous works (Saggio et al., 2009).

REFERENCES

• Biggs G, MacDonald B. A survey of robotic programming systems. Australas. Conf. Robot. Autom., Brisbane, Australia, 2003
• Birkhoff GD. Aesthetic Measure. Cambridge, Harvard University Press, 1933
• Damasio FW, Musse SR. Animating virtual humans using hand postures. in Proc. Brazilian Symp. Comput. Graph. Image Process., 2002, p. 437
• de la Hamette P, Lukowicz P, Tröster G, Svoboda T. Fingermouse: A Wearable Hand Tracking System. UBICOMP conference (proceedings) in 2002
• Degeorges R, Parasie J, Mitton D, Imbert N, Goubier JN, Lavaste F. Three-dimensional rotations of human three-joint fingers: an optoelectronic measurement. Preliminary results. Surg Radiol Anat (2005) 27: 43–50
• Dipietro L, Sabatini AM, Dario P. Evaluation of an instrumented glove for hand movement acquisition. J Rehabil Res Dev 2003;40(2):179–90
• Gentner R and Classen J. Development and evaluation of a low-cost sensor glove for assessment of human finger movements in neurophysiological settings. Journal of Neuroscience Methods 178 (2009) 138–147
• Hahn P, Krimmer H, Hradetzky A, Lanz U. Quantitative analysis of the linkage between the interphalangeal joints of the index finger. Journal of Hand Surgery (British and European Volume).1995; 20: 696-699
• Heuser A, Kourtev H, Winter S, Fensterheim D, Burdea G, Hentz V, Forducey P. Telerehabilitation using the Rutgers Master II glove following carpal tunnel release surgery: Proof-of-concept. IEEE Trans. Neural Syst. Rehabil. Eng., vol. 15, no. 1, pp. 43–49, Mar. 2007
• Hong J and Tan X. Calibrating a VPL Data Glove for teleoperating the Utah/MIT hand. in Proc. IEEE Int. Conf. Robot. Autom., 1989, vol. 3, pp. 1752–1757
• Karlsson N, Karlsson B, Wide P. A glove equipped with finger flexion sensors as a command generator used in a fuzzy control system. In Proceedings of the IEEE Instrumentation & Measurement Technology Conference 18–21 May 1998 St Paul, Minnesota, USA; 1998:441-445
• Keefe DF, Feliz DA, Moscovich T, Laidlaw DH, LaViola JJ. Cave painting: A fully immersive 3D artistic medium and interactive experience. in Proc. Symp. Interactive 3D Graph., Mar. 2001, pp. 85–93
• Kolsch M, Turk M. Keyboards without keyboards: A survey of virtual keyboards. Univ. California, Berkeley, Tech. Rep. 2002-21, 2002
• Kuroda T, Tabata Y, Goto A, Ikuta H, Murakami M. Consumer price data-glove for sign language recognition. Proc. 5th Intl Conf. Disability, Virtual Reality & Assoc. Tech., Oxford, UK, 2004
• Mehdi SA, Khan YN. Sign language recognition using sensor gloves. in Proc. Int. Conf. Neural Inf. Process., 2002, vol. 5, pp. 2204–2206
• Micera S, Dario P, Posteraro F. On the analysis of hand synergies during grasping in weightlessness. in Proc. Life Space Life Earth, Eur. Symp. Life Sci. Res. Space, Annu. Int. Gravitational Physiol. Meeting, 2002, pp. 233–234
• Micera S, Cavallaro E, Belli R, Zaccone F, Guglielmelli E, Dario P, Collarini D, Martinelli B, Santin C, Marcovich R. Functional assessment of hand orthopedic disorders using a sensorized glove: Preliminary results. in Proc.
  IEEE Int. Conf. Robot. Autom., 2003, vol. 2, pp. 2212–2217
• Morrow K, Docan C, Burdea G. Low-cost virtual rehabilitation of the hand for patients post-stroke. in Proc. Int. Workshop Virtual Rehabil., 2006, pp. 6–10
• Mulder A. Towards a choice of gestural constraints for instrumental performers. in Trends in Gestural Control of Music. Paris, France: IRCAM, 2000
• Noaman NM, Ajel AR, Issa AA. Design and implementation of a DHM glove using variable resistors sensors. Journal of Artificial Intelligence 1 (1): 44-52, 2008
• Oldfield RC. The assessment and analysis of handedness: the Edinburgh Inventory. Neuropsychologia 9: 97–113, 1971
• Saggio G, De Sanctis M, Cianca E, Latessa G, De Santis F, Giannini F. Long Term Measurement of Human Joint Movements for Health Care and Rehabilitation Purposes. Wireless Vitae09 – Wireless Communications, Vehicular Technology, Information Theory and Aerospace & Electronic Systems Technology, Aalborg (Denmark), 17-20 May, 2009 – pp. 674-678
• Satava RM, Jones SB. Current and future applications of virtual reality for medicine. Proc. IEEE, vol. 86, no. 3, pp. 484–489, Mar. 1998
• Simone LK, Sundarrajan N, Luo X, Jia Y, Kamper DG. A low cost instrumented glove for extended monitoring and functional hand assessment. J Neurosci Methods 2007;160:335–348
• Székely G, Satava RM. Virtual reality in medicine. BMJ 1999;319;1305
• Xu D, Yao W, Zhang Y. Hand gesture interaction for virtual training of SPG. in Proc. Int. Conf. Artif. Reality Telexistence, 2006, pp. 672–676
• Villella AP, Salsedo BF, Bergamasco CM. A new dataglove based on innovative goniometric sensors. Proceedings of the Wokshop on Modelling and Motion Capture Techniques for Virtual Environments, Zermatt, CH, December 9-11, 2004
• Williams NW, Penrose JMT, Caddy CM, Barnes E, Hose DR, Harley P. A goniometric glove for clinical hand assessment. Journal of Hand Surgery (British and European Volume, 2000) 25B: 2: 200-207
• Wise S, Gardner W, Sabelman E, Valainis E, Wong Y, Glass K, et al. Evaluation of a fiber optic glove for semi-automated goniometric measurements. J Rehabil Res Dev 1990;27(4):411–24
• Zurbrügg T. Dynamic Grasp Assessment for Smart Electrodes (GRASSY). Semester Thesis. ETH Zurich (Swiss Federal Institute of Technology), Department of Information Technology and Electrical Engineering 2003