Neuromorphic Sensor Based on FSR

Preprint

Article

Neuromorphic Sensor Based on FSR

Altmetrics

Downloads

Views

Comments

A peer-reviewed article of this preprint also exists.

This version is not peer-reviewed

Submitted:

18 April 2024

Posted:

18 April 2024

You are already at the latest version

Alerts

Abstract

This work introduces a neuromorphic sensor based on force-sensing resistors (FSR) and spiking neurons for robotic systems. The proposed sensor integrates the FSR in the schematic of spiking neuron in order to make the sensor to generate spikes with frequency that depends on the applied force. The performance of the proposed sensor is evaluated for the control of a SMA-actuated ro-botic finger by monitoring the force during steady state when the finger pushes on a tweezers. For comparison purposes, we performed similar evaluation when the SNN receives input from a widely used compression load cell (CLC). The results show that the proposed FSR based neuro-morphic sensor has very good sensitivity to low forces and the function between the spiking rate and the applied force is continuous with good variation range. However, when compared to the CLC, the proposed sensor has lower linearity of the response, but it benefits from two orders of magnitude lower power consumption. These aspects imply that the neuromorphic sensor based on FSR are useful for bioinspired control of humanoid robotics representing a low power and low cost alternative the widely used sensors.

Keywords:

Subject: Engineering - Bioengineering

1. Introduction

The spiking neural networks represents the third generation of neural networks which introduces time in information processing and learning. The SNN are based on spiking neurons that model rigorously the physiology of the neural cells in the brain. The hardware implementation of the SNN benefits from low power consumption, high reliability and very good signal to noise ratio, which makes SNN a good candidate for the implementation of control units of robotic systems. In robotic systems, the motion is of high importance because it provides the robots with the ability to interact mechanically with the environment. Typically, the motion is performed by electric, pneumatic or hydraulic actuators , while the control of motion is based on the feedback from neuromorphic sensors. This type of sensors includes a sensing component and the additional electronic circuits that converts the measured parameter into trains of impulses with parameter dependent frequency [1]. There are several methods to implement the sensing components that are based on ferromagnetic powder in a resin [2] or on Euler-Bernoulli bending rods. The later method is used to implement multi-point fine sensing using the principle of whiskers in living organisms [3]. An advanced method to implement the sensing components use piezo-electric field effect transistors FET that converts the physical measure into electrical current [4]. In neuromorphic sensors, this current is used by additional analogue hardware to generate spikes with corresponding frequency [5]. Another type of tactile sensor with spiking output includes one organic transistor for sensing and another transistor for pulse generation [6].

However, these types of sensing components are expensive or has limited availability. In contrast, force sensing resistor (FSR) is cheap and easy to use making this type of component suitable for a wide range of applications starting from implementation of wearable sensors in gaming [7], to prostheses for human in biomedical engineering [8] and for monitoring cardiorespiratory parameters during sleep [9].

Force Sensing Resistors

FSR uses polymer thick film technology to convert the force into resistance which varies in the opposite direction with the applied force. The FSR is built of two layers of flexible polyester film substrate, each with a layer of conductive silver ink. Between the layers there is a compressible polymer which responds to the applied force by increasing the conductance [8]. With no pressure the sensor behaves as an open switch due to infinite resistance, and when the sensor is pressed the drop of resistance is significant for equivalent mass below 100 grams. The variation per unit mass reduces following the power law while the compression mass increases to several kilograms. With proper design of the sensing component, the repeatability of the measurements for one sensor is below ±2% showing good reliability of the FSR [10].

Compression Load Cells

A CLC converts the applied force into voltage being based on strain gauge (SG) which represents the measuring component of the load cell. The strain gauge is a small grid consisting of wires made of an alloy of copper and nickel which are sensitive to the deformation of the load cell surface. This changes the resistance of the SG and with proper electronic circuit the force is converted into voltage. The circuit is based on Wheatstone bridge which connects multiple SGs and detects of deformation as a change in electrical signal [11]. The CLC benefits from high linearity and repeatability of the response even at extremely high load capacities but comes at higher cost than the FSR.

SMA Actuators

Among the electric actuators the shape memory alloy (SMA) represents an elegant alternative to the widely used motors because of their small size, low weight and significant force-weight ratio [12]. The SMA is a metallic material which changes its state when heated from martensite phase when cold to austenite phase which represents the memorized shape. During the austenite phase which is determined by higher temperature the actuator shortens its length with a significant force. For example, an actuator made of nitinol with 0.006 inch diameter can pull up a mass of 320 grams according to the technical data. Also, the lifetime of SMA is of order of 10⁶ actuation cycles implying that these actuators have significant reliability. However there are some disadvantages of these actuators such as the nonlinearity due to the difference between the contraction and respectively relaxation temperatures. The contraction speed and the initial length are influenced by the environmental temperature, that affects also the relaxation time when no auxiliary cooling method is used. In addition, the change in length during contraction is about 4% which increases the length of necessary actuators for long displacements.

Despite these disadvantages, the SMA benefits from the same actuation principle and are silent in operation as the natural muscles being suitable for actuation of the bioinspired systems. In addition, the spiking neurons are able to control directly the contraction of SMA actuators in a biomimetic manner by the firing rate [13,14].

The Proposed Sensor

In this work we proposed a neuromorphic sensor based on FSR (NS_FSR) that includes a spiking neuron implemented in analogue hardware which integrates the FSR in the SOMA. Compared with the existing neuromorphic pressure sensors [5,13] the proposed NS_FSR benefits from simplified circuit over CLC based sensor because the sensing device is integrated in the neuron and has significantly lower cost than the FET based ones. The NS_FSR is validated in a robotic application where SNN controls the force of an anthropomorphic finger actuated by SMA. The behavior of the finger is evaluated when the FSR based sensor and the CLC are used independently.

The remainder of the paper is organized as follows: Section 2 details the schematics of the SOMA with FSR and the structure of the SNN for evaluation of the NS_FSR and CLC in robotic finger control together with the experimental setup. The results are presented in the Section 3 followed by the conclusions that focuses on the advantages of the proposed sensor and the future work.

2. Materials and Methods

The proposed sensor is based on an electronic spiking neuron implemented in analogue hardware which obtained good results for the control of SMA actuated anthropomorphic fingers [13,14].

Electronic Neuron

This neuron model includes one input module denoted SOMA for integration of the incoming pulses and activates the synapses (SYN) when the activation threshold is detected. The SYNs stores the synaptic weights and generates excitatory or inhibitory spikes which intensity depends on the weights. In addition, SYNs model the Hebbian learning mechanisms that makes the neuron to respond to temporal coincidence of incoming stimuli.

The Neuromorphic Sensor

The schematic of the SOMA is presented in Figure 1. The capacitor

C_{M}

integrates the incoming spikes while the transistor

T_{T H}

detects the neuron activation threshold

T_{T H}

and activates the synapses using another transistor

T_{A C T}

. When

V_{T H}

is reached the SOMA activates for a fixed period of time

t_{A C T}

while the synapses to generate a spike. The SOMA can be activated by incoming pulses that are integrated by C_M of by a continuous potential

V_{E}

. In this case, the activation rate of the SOMA, and consequently of the neuron, depends on

V_{E}

and

R_{E}

. Thus, the firing rate of the neuron

f_{N}

varies with

V_{E}

when

R_{E}

is fixed, or with

R_{E}

when the input potential is constant.

Starting from this characteristic of the electronic neuron, the neuromorphic sensor uses the FSR instead of

R_{E}

and a fixed potential

V_{F S R}

to activate the neuron. The value of

V_{F S R}

is chosen empirically to increase the range of the spiking rates generated by the SOMA while keeping its sensor’s sensitivity high. In order to avoid the use of an additional power supply,

V_{F S R}

is adapted to the VDD through the auxiliary circuit AUX which is highlighted in blue in Figure 1. The synapses driven by this modified SOMA generate spikes which frequency depends on the applied force on the FSR.

The Structure of the SNN

The neuromorphic sensor NS_FSR is tested in a control application where the force applied on the tip of an anthropomorphic finger is regulated by a SNN which structure is shown in Figure 2a. This neural network drives the SMA wire that actuates the finger, and receives information from the sensor. The input layer of the SNN includes several excitatory neurons

E_{1,4}

that activates the motor neurons

M_{1,2}

that builds the output layer. The motor neurons are inhibited by the NS_FSR through the inhibitory synapses IS₁ and IS₂ as in Figure 2a. For evaluation of the SNN behavior with the load cell the NS_FSR is replaced by the CLC with an inhibitory neuron and the circuit POT as in Figure 2b.

The motor neurons stimulates through SMA driver the actuators which contraction force is determined directly by the neurons’ spiking rate. The frequency of the motor neurons depends on the resultant activity of NS_FSR and

E_{1,4}

which frequency

f_{E}

can be varied by adjusting the potential

V_{E}

. Therefore, considering that firing rate of motor neuron increases with the frequency of excitatory neurons, the force of the SMA actuator can be set using

V_{E} .

Experimental Setup

We validated the proposed sensor using an anthropomorphic robotic finger that is flexed by a SMA actuator and has two force sensors on the finger’s tip, see Figure 3. The actuator is implemented 0.006” SMA wires type Flexinol LT (produced by Dynalloy, inc) that can reach up to 4% stroke and 321g force when a maximum of 410 mA heats the wire. For the reported experiments, the length of the actuator is 115 cm that is powered at

V_{C C} = 24 V .

We compared the performances of a compression load cell type FS2050 (produced by TE Connectivity) and the force sensing resistor type FSR03CE. In order to reduce the influence of the contact area on the FSR response and other surface related errors [15] we glued a metal disk with diameter of 20 mm in front of the sensor. The system is evaluated by measuring the output of the CLC and the neuron’s frequency when during steady state when the finger pushes on an elastic tweezers. Note that the CLC is below the FSR in order to evaluate the behaviors of the finger with the two sensors in the same conditions. During the experiments the FSR based sensor and CLC are not connected to the input of the SNN in the same time implying that the use of two sensors for feedback is exclusive. However, the information about the finger’s force is obtained by reading the analogue output of the CLC even if the NS_FSR feeds the SNN. In order to deepen the comparison between the two sensors we determined the energy consumptions for the NS_FSR and respectively, of the CLC by measuring the electrical current that is used by the two circuits.

Figure 4 presents the diagram of the experimental setup that includes SNN, the SMA driver and the two sensors that are fixed on the finger’s tip.

The CLC is connected through a potentiometer POT to one inhibitory neuron included in the inhibitory area of the SNN that is shown in green. The FSR uses a SOMA, shown in dark blue, to drive the corresponding inhibitory synapses that is also integrated in the inhibitory area.

3. Results

The experimental work for this research is split in two phases that are focused on i) the evaluation of the NS_FSR and of CLC response off system, and ii) the validation of NS_FSR in robotic finger control.

i) Sensors response with the load mass

During the first experimental phase we determined the variation of the firing rate

f_{E}

of the SOMA with the input voltage

V_{E}

followed by evaluation of the response of CLC and NS_FSR with the load mass

m_{L}

. As presented in Figure 5a,b the activation rate

f_{E}

of the SOMA varies linearly with

V_{E}

when

R_{E}

is fixed, and similarly, the CLC out increases linearly with

m_{L}

. This behavior implies that the spiking rate of the inhibitory neuron is proportional with the load mass when the load cells is used as a force sensor.

For evaluation of the CLC’s response, we used 14 disks of 3.5 grams that are stacked sequentially above the sensor. For triggering the activity of the inhibitory neuron with the first disk we added an offset mass of 7.5 grams implying that the minimum mass that is used in this experimental phase is 11 grams. For one set of measurements we read the CLC out when the mass varied between 11 to 56 grams with the resolution of 3.5 grams, and this experiment is repeated 5 times. The obtained results are given in Figure 5b which shows that the output of the CLC is proportional with

m_{L}

Similarly, for evaluation of the FSR based neuromorphic sensor NS_FSR we performed 5 sets of measurements when the load mass ranges between 11 and 56 grams, see Figure 6. For this sensor we measured the output frequency

f_{N S}

of the neuron that is directly related to the FSR value and consequently to the applied force.

Note that the variation of the frequency

Δ f_{N S}

decreases with

{Δ m}_{L}

showing that the spiking rate generated by the neuromorphic sensor increases following a logarithmic-like function with

m_{L}

mainly above 25 grams. This behavior is a consequence of the power law that describes the resistance-force function of the FSR [11].

ii) Control of the robotic finger

For the second experimental phase, we implemented a robotic finger that is actuated by SMA and includes both force sensors on the distal phalange. In this setup, we evaluated the ability of this robotic finger to push and hold one tip of an elastic tweezers while the other tip is blocked. The contraction force of the actuator is adjusted by changing the input voltage

V_{E}

that activates the excitatory neurons (EN) at constant frequency. For having a clear view of the SNN activity, Figure 7 shows an example of the pulses generated by the SOMAs during activation of the neurons. On these diagrams that are recorded using the oscilloscope, the blue signal illustrates the activity of motor neurons (MN) that integrates the activity of EN (green signal) and of the NS_FSR (magenta signal).

In Figure 8 the force of the SMA actuator varies due to the intermittent activity of the motor neurons implying that for lower frequencies of the EN the regulatory performance of the SNN decreases. In contrast, for higher frequencies of the EN the variation of the finger’s force during regulatory regime is insignificant.

Figure 9a presents the variation of the CLC out with the frequency of the excitatory neurons when this CLC drives the inhibitory neurons as in Figure 2b.

The frequency

f_{E}

of EN is set to several values between 350 and 550 Hz with resolution of 25 Hz. The limits of this variation interval are chosen empirically to reduce the oscillation of the finger force for lower

f_{E}

and to reach maximum contraction force for higher

f_{E}

. Similarly, the variation of

V_{F}

with

f_{E}

is given in Figure 9b when the proposed FSR based neuromorphic sensor contributes to the regulation of the motor neurons’ activity. Taking into account the linearity of the load cell’s response, in this case the CLC is also used to measure the force of the finger on the tweezer’s tip.

Taking into account that power consumption represents an important parameter of the neuromorphic systems, besides analysis of the SNN activity we measured the maximum electrical current used by both sensors for the maximum force. The obtained values are ~4 mA at 5V for the CLC and about 80 µA at 1.6 V for the NS_FSR, showing that the power consumption of the neuromorphic sensor is more than 150 times lower than the power needed for CLC to operate. In addition, lower pressure increases the resistance of the FSR and the power consumption of the NS_FSR implying that higher sensitivity reduces the consumed energy even further.

5. Conclusions

In this work we proposed a neuromorphic sensor based on an electronic spiking neuron which includes a cheap force sensing resistor. The frequency of the output spikes that are generated by the sensor varies with the FSR value and consequently with the applied force. The validation of the proposed sensor was performed in two phases when we measured directly the spiking frequency when load masses with known values are placed on the sensor, and, for the second phase, we integrate the sensor in a SMA actuated robotic finger and measured the force during steady state when the finger pushes on an elastic tweezers.

The evaluation of the sensor response shows that the output of the CLC is proportional with the load mass and consequently with the applied force implying that the output of the neuron driven by the CLC is also linear. On the other hand, the variation of the output frequency of NS_FSR with the load mass follows a logarithmic-like function that is determined by the nonlinearity of the force-resistance function of the FSR. The results of the second experimental phase shows that despite the fact that the NS_FSR has lower linearity than the CLC, this FSR based sensor can be used to discriminate a wide range of applied forces. In addition, the power consumption of the NS_FSR is significantly lower than that of the CLC. This advantage makes the proposed sensor useful not only for implementation of bioinspired control systems in robotics, but also for low power electronics based on neuromorphic hardware.

For future work we intend to use the proposed sensor for a robotic hand with five fingers and to reduce the power consumption of the electrical circuit that implements the spiking neuron. In addition, we will design structures of spiking neurons that compensates for the errors of the FSR.

Author Contributions

Conceptualization, A.B, and M.H.; methodology, M.H.; validation, M.H. and A.B.; formal analysis, A.B. and M.H.; investigation, M.H.; data curation: M.H., writing—original draft preparation, M.H.; writing—review and editing, M.H. and A.B.; supervision, A.B.; All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by the Faculty of Automatic Control and Computer Engineering.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest.

Abbreviations

The following abbreviations are used in this manuscript:

SG Strain gauge

SOMA Neuron’s Soma

SYN Synapse

NS neuromorphic sensor

CLC Compression load cell

FSR force sensing resistor

SMA Shape memory alloy

SNN Spiking neural network

SN Spiking neuron

References

Zeng, M.; He, Y.; Zhang, C.; Wan, Q. Neuromorphic Devices for Bionic Sensing and Perception. Front. Neurosci. 2021, Sec. Neural Technology. 2021; 15. [Google Scholar]
Nagahama, S.; Migita, K.; Sugano, S. Soft Magnetic Powdery Sensor for Tactile Sensing. Sensors 2019, 19, 2677. [Google Scholar] [CrossRef] [PubMed]
Merker, L.; Fischer Calderon, S.J.; Scharff, M.; Alencastre Miranda, J.H.; Behn, C. Effects of Multi-Point Contacts during Object Contour Scanning Using a Biologically-Inspired Tactile Sensor. Sensors 2020, 20, 2077. [Google Scholar] [CrossRef] [PubMed]
Yogeswaran, N.; Navaraj, W. T.; Gupta, S.; Liu, F.; Vinciguerra, V.; Lorenzelli, L.; Dahiya, R. Piezoelectric graphene field effect transistor pressure sensors for tactile sensing. Appl. Phys. Lett. 2018, 113, 014102. [Google Scholar] [CrossRef]
Birkoben, T.; Winterfeld, H.; Fichtner, S.; Petraru, A.; Kohlstedt, H. A spiking and adapting tactile sensor for neuromorphic applications. Sci. Rep. 2020, 10, 17260. [Google Scholar] [CrossRef] [PubMed]
Zang, Y.; Shen, H.; Huang, D.; Di, C.; Zhu, D. A Dual-Organic-Transistor-Based Tactile-Perception System with Signal-Processing Functionality. Adv. Mater. 2017, 29, 1606088. [Google Scholar] [CrossRef] [PubMed]
Dontha, B.; Swearingen, K.; Swearingen, S.; Thrane, S.E.; Kiourti, A. Wearable Sensors Based on Force-Sensitive Resistors for Touch-Based Collaborative Digital Gaming. Sensors 2022, 22, 342. [Google Scholar] [CrossRef] [PubMed]
Swanson, E.C.; Weathersby, E.J.; Cagle, J.C.; Sanders, JE. Evaluation of Force Sensing Resistors for the Measurement of Interface Pressures in Lower Limb Prosthetics. J Biomech Eng. 2019, 141, 1010091–10100913. [Google Scholar] [CrossRef] [PubMed]
Haghi, M.; Asadov, A.; Boiko, A.; Ortega, J.A.; Martínez Madrid, N.; Seepold, R. Validating Force Sensitive Resistor Strip Sensors for Cardiorespiratory Measurement during Sleep: A Preliminary Study. Sensors 2023, 23, 3973. [Google Scholar] [CrossRef] [PubMed]
Yaniger, S. I., Force Sensing Resistors: A Review of the Technology, Electro International, 1991, New York, NY, USA, pp. 666-668.
Ho, M.H.; Wang, P. N.; Yeh, J.P.; Wu, B.H., Design and Analysis of Compression Low Profile Load Cells, Proc of the 2017 2nd International Conference on Advances in Materials, Mechatronics and Civil Engineering, 2017, Guangzhou, China, 261 – 264.
Moallem, M.; Tabrizi, V. A. Tracking Control of an Antagonistic Shape Memory Alloy Actuator Pair. IEEE Trans. Control Syst 2009, 17, 184–190. [Google Scholar] [CrossRef]
Hulea, M.; Uleru, G.I.; Caruntu, C.F. Adaptive SNN for Anthropomorphic Finger Control. Sensors 2021, 21, 2730. [Google Scholar] [CrossRef] [PubMed]
Uleru, G.-I.; Hulea, M.; Burlacu, A. Bio-Inspired Control System for Fingers Actuated by Multiple SMA Actuators. Biomimetics 2022, 7, 62. [Google Scholar] [CrossRef] [PubMed]
Gutiérrez Velásqueza, E.; Gómezb, L.; Paredes-Madrida, V.; Coloradoc, H. Error compensation in force sensing resistors. Sensing and Bio-Sensing Research 2019, 26, paper: 100300. [Google Scholar]

Figure 1. Schematic of the neuromorphic sensor based on FSR including one SOMA and two inhibitory synapses IS₁, IS_2, as well as the additional components for adjusting the generated frequency range.

Figure 2. (a) The structure of the SNN used for SMA control using the FSR-based neuromorphic sensor (NS_FSR); (b) The circuit that replaces the NS_FSR when the compression load cell is used to sense force.

Figure 3. The structure of the finger with the FSR and CLC on the finger’s tip.

Figure 4. the structure of the system including the SNN that is connected to the sensors and SMA driver.

Figure 5. (a) The influence of the input voltage on the activation rate of the SOMA, and (b) the variation of the CLC output with the mass that is placed above the sensor.

Figure 6. The response of the FSR based neuromorphic sensor with the load mass.

Figure 7. The neurons activity during steady state for different activation rates of the neurons 500 Hz when

V_{E} = 5.75 V

Figure 7. The neurons activity during steady state for different activation rates of the neurons 500 Hz when

V_{E} = 5.75 V

Figure 8. The neurons activity during steady state for different activation rates of the neurons 375 Hz when

V_{E} = 4.5 V

Figure 8. The neurons activity during steady state for different activation rates of the neurons 375 Hz when

V_{E} = 4.5 V

Figure 9. The variation of the CLC output with the spiking rate of the excitatory neurons when: (a) the CLC activates one inhibitory neuron; (b) the NS_FSR regulates the activity of the motor neurons.

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.

MDPI Initiatives

Important Links

Choose an area of interest and we will send you notifications of new preprints at your preferred frequency.

Disclaimer

Neuromorphic Sensor Based on FSR

Abstract

1. Introduction

Force Sensing Resistors

Compression Load Cells

SMA Actuators

The Proposed Sensor

2. Materials and Methods

Electronic Neuron

The Neuromorphic Sensor

The Structure of the SNN

Experimental Setup

3. Results

i) Sensors response with the load mass

ii) Control of the robotic finger

5. Conclusions

Author Contributions

Funding

Institutional Review Board Statement

Informed Consent Statement

Data Availability Statement

Conflicts of Interest

Abbreviations

References

MDPI Initiatives

Important Links

Subscribe