Neurofire is an Australian startup company that aims to exploit the advantages of Spiking Neural Networks (SNNs) over standard Artificial Neural Networks (ANNs).
Spiking Neural Networks have been extensively studied. In general, they are closer to how the brain operates than ANNs, transmitting discrete events rather than continuous signals as used in ANNs. Because of this event based behaviour, SNNs can operate at much lower power levels, an issue that is starting to get prominence as cloud service providers deal with the ever increasing power requirements of large scale AI.
Spiking Neural Networks have another interesting feature and that is their ability to learn without the need for back-propagation -- this is where a significant speedup in training can be delivered. Neurofire’s SNN networks are trained with Spike Time Dependent Plasticity (STDP), Reinforcement STDP and Spike Forcing techniques and typically require less data than normal ANN training.
SNNs are known for being sparse, both in the number of connections between neurons and in the activation rates of those neurons. Handling sparsity of connection during training by taking advantage of sparse matrix processing is yet another advantage that improves training time. ANN networks are typically fully connected during training and then pruned for inference and hence can’t use this technique in training.
SNNs suffer from a phenomenon known as Information Propagation Failure (IPF) where SNN networks become unstable as the size of the network grows. Based on a four year research program funded by the Australian Department of Defence, and led by Professor Peter Stratton (a founder of Neurofire), this issue has been solved and is a key pillar of Neurofire’s competitive barrier to entry. In a nutshell, other SNN companies cannot deliver SNNs of the size and training speed required to be useful in large applications, like LLMs.
Neurofire’s IP is a combination of Peter’s research which is in the public domain, patents held by Neurofire and trade secrets. We are very confident of the intellectual property moat that this gives Neurofire.
Neurofire has been operating in bootstrap mode for the last 12 months. We are now moving towards our first investment round. If you are interested in discussing investment options please contact us.