These are our application of neural networks to solving aerospace problems using embedded processors receiving data directly from sensors. Some of these projects were created before the widespread use of LSTM’s and Transformers with LLM’s. We are testing proprietary solutions using the new methods to reduce the labor of supervised learning and to increase the accuracy of our systems.
When we use the term Artificial Intelligence (AI) a neural network is involved
The neural network is a machine learning approach to failure identification that does not necessarily require more and more complex analysis, but instead uses prior operations experience to prevent inadvertent catastrophic surprises as engineering development continues. We call the method Imminent Fault Identification (IFI).
AI applied in this way can augment traditional analytical approaches to Failure Modes and Effects Analysis (FEMA) and Fault Detection Identification Reconfiguration (FDIR) for complex engineering systems.
Using the prior history it continually predicts what the system will be doing in the next data cycle. Any discrepancy is an output we call “Out of Family”.
IFI introduces a fundamentally more sophisticated real time diagnosis method, without needing to precisely define complex physical models, by utilizing both “good” and “bad” previous test or flight experience data.
Mars Vehicle Holistic Imminent Failure Identification
A holistic artificially intelligent Imminent Failure Identification (IFI) system of systems is proposed for a Mars transit vehicle on a crewed 520 day mission that is maintenance focused. The holistic IFI uses machine learning to compare learned spacecraft system performance with real time behavior. Each system IFI signals when its components are degrading to an “Out of Family” outlier condition. Images courtesy of Paragon Space Development Corporation
Holistic IFI goes far beyond traditional system monitoring and fault detection method relying on the system engineer’s best guess as to which parameters to measure, and what to do when they are out of lits. Instead, holistic IFI adds a self awareness quality to the whole spacecraft, which could be expanded to not only monitor system hardware based on experience before launch, but also learn its own performance after launch. Many light minutes away from Mission Control, out of reach of other vehicles, holistic IFI can be tuning systems and directing the crew to improve or modify the systems as the mission progresses, or environments change.
Rocket motor testing
Foale Aerospace created the first software implementation of IFI in 2014. It was initially provided to Whittinghill Aerospace while testing rocket motors. The purpose was to identify possible anomalous circumstances. It was successful in learning the rocket motor performance and the ability to indicate out-of-family conditions. Image courtesy of Whittinghill Aerospace.
A rocket motor is the most stringent safety critical system with which we have tested IFI, however the method is easily applied to any system, complex, or simple, with or without well understood physical models available.
Lunar Rover fault management and hazard avoidance
In 2021 we proposed to NASA to integrate IFI into an Astrobotic CubeRover “mule” for demonstration of IFI fault management and surface hazard identification on a real space vehicle system. The Astrobotic CubeRovers are space qualified for launch to the Moon and operations on the Lunar surface. The proposal built on experience using IFI on an experimental rover moving over lunar simulant. Image courtesy of Astrobotic
Microphones mounted next to wheel bearings, and a dragging microphone, together with motor current sensors and accelerometers were able to characterize the type of surface over which the rover was moving. One neural network learned the system to produce out-of-family signals, which input to a classifier to identify the lunar surface. IFI detected anomalies such as rocks stuck in wheel treads, high instantaneous motor currents, vehicle tilting and wheel slips. Surface type was identified correctly.
Solar Pilot Guard – Pilot loss of control (LOC) prevention
An application of a neural network classifier was used to address pilot fatalities caused by loss of control system (LOC). It works in conjunction with a microphone and accelerometer mounted on the tail, communicating over Bluetooth with a display in the cockpit. It provides audible commands to the pilot to prevent LOC, as well anticipating the possible onset of flutter in the tail stabilizer.
The three box IFI system is a precursor to our concept of Holistic IFI outlined above for a Mars mission vehicle. Foale Aerospace was awarded a prize for its Solar Pilot Guard entry to the 2017 Experimental Aircraft Association Founders Innovation Prize competition using the classifier neural network and Mathematica. We were subsequently awarded third prize in the finals of the 2021 for Solar Pilot Guard 2. That system used Digital Signal Processing (DSP) filters instead of a neural network classifier. The wing sensor is described here. Contact us to understand why we went to DSP instead of AI.
Bosch SoundSee listening for malfunctions on the ISS
The SoundSee technology for the ISS was developed by Bosch and Astrobotic, a U.S.-based company specializing in space robotics. Michael Foale supported the team with tests on Earth. The SoundSee system is mounted on Astrobee, a cube-shaped mini robot developed by NASA that floats autonomously through the ISS. From this vantage point, the SoundSee microphones are expected to continuously record the operating noises of the machinery and equipment on board the space station. Image courtesy Bosch
We have worked with Bosch developing AI to learn and identify different categories of sounds in multiple environments. See some of the cool stuff we have done with Bosch.
https://www.bosch.com/stories/life-on-the-international-space-station/
Drone package delivery safety
We won 1st prize in the America’s in 2022 as part of the Microsoft/Altera/Intel/Analog Devices annual InnovateFPGA competition. Our entry was “Drone package delivery safety in turbulent atomospheric conditions in confined areas like cities”. We came 4th in the World competition. The goal of the competition was to come up with a technology use of FPGA’s and Microsoft Azure that would lead to reduced global carbon emissions.
Our idea was that a system of AI controlled small drone Scouts could lead the way for larger heavily loaded cargo delivery drones. The Scouts would sense bad wind conditions around sky-scrapers or other hazardous weather situations. When bad conditions for the cargo drones are detected, the Azure controlling AI would redirect the cargo drones.
The Scout incorporated similar AI to our Solar Pilot Guard for gliders and light aircraft, incorporating an FPGA to using DSP detection of upsets. A very light scout drone is more susceptible to the atmospheric weather environment than a heavy drone, but is able to recover from upsets caused by wind gusts and downdrafts that would compromise cargo drone flight safety.
Non-Aerospace Applications of AI and IFI
3D Printing Imminent Fault Identification
3D printing is remarkable and effective when it works, but many factors such as tool temperature, bed temperature, mechanical accelerations, filament anomalies can cause a failed print. The IFI Print system had an image autoencoder, sound autoencoder, accelerometers and temperature sensors to train an LSTM neural network to recognize a successful print.
IFI Print was able to spot when the print had failed, crossing a normalized out of family threshold of 0.35. The print began normally, but successive ‘partial’ failures were introduced. At failure annunciation the print filament was no longer being extruded.
Happy Cat
Happy Cat is a cuddly toy people want to buy because it looks cute, and it helps sooth and calm a person down, for example, before sleep. If you are worried about an elderly person whose health is in question, this cuddly toy can be placed on their chest and issue an alert via SMS or social media if their breathing changes.
Happy Cat will incorporate our reduced training IFI inside its body. Happy Cat senses the person’s rhythms, learns them, and purrs or meows, depending on how it has learned the persons state.
Prototype Happy Cat parts cost about $80 purchased separately.
Other IFI applications – A Wind Turbine and Earthquakes
Applying IFI to a wind turbine was straight forward. IFI used a simple prediction neural network learning only accelerations at the rotor head. Resonances and abrupt wind changes were easily identified using IFI as wind turbine shutdown criteria.
Earthquake prediction was a very different proposition, and the data we were given as part of an online machine learning competition could not be properly learned by any neural network that would fit into a desktop computer RAM. Maybe that was a limitation of the data we were given.