Matt Mierzwinski

Your AI model lacks of neuroplasticity

neuromorphic-networks

What are the limitations of AI currently developed by the biggest companies like Meta, Google, or OpenAI? Since I started developing applications and APIs utilizing various machine learning models to predict local weather, recognize handwriting and translate it to LaTeX, solve mathematical equations, predict cryptocurrency trends, and auto buy/sell BTC, ETH, and others, I have mainly focused on static models. These models always required fine-tuned data to be trained—better data meant a more accurate model. However, better data also meant larger datasets to boost model accuracy. This accuracy has been achieved by constantly retraining the model and iterating on it, only to learn the next day that optimization parameters can be fine-tuned further or that my dataset is missing some critical aspect the model should address.

This is a constant loop of feedback and improvement that runs indefinitely, making maintenance a significant burden—hence the phrase, “AI is not a small project for one person to do.” It’s also not cheap, due to the hardware requirements needed to use that model effectively later on. This is because, in the classical approach, an AI model is static—generated during the training process and does not evolve over time. It’s not capable of learning from provided input when the model is being utilized. It lacks evolution; it lacks the neuroplasticity that our brains naturally have.

Think about it—each time we learn something new, we don’t change our brain, or we don’t put it on the shelf to retrain it in the next 3-4 days, but modern AI does exactly that. It’s a cost-inefficient and resource-hungry process that iterates possibly forever.

Neural Networks, Machine Learning, Deep Learning, and Neuromorphic Computing

Neural Networks: Modern neural networks are inspired by the perceptron but have evolved significantly. They include deep learning architectures with multiple layers (deep neural networks) that can learn complex patterns from vast amounts of data. These networks use tremendous amounts of memory and processing power to adjust weights and optimize neural networks, utilizing techniques like backpropagation.

Machine Learning (ML): Machine learning encompasses a broad range of algorithms that enable computers to learn from and make predictions or decisions based on data. It includes supervised learning, unsupervised learning, and reinforcement learning. ML models can be as simple as linear regression or as complex as deep learning networks.

Deep Learning (DL): Deep learning is a subset of machine learning that uses neural networks with many layers (deep neural networks). It has driven significant advancements in AI, enabling breakthroughs in image recognition, natural language processing, and more. However, deep learning models are typically static and require extensive retraining with new data to improve performance.

Neuromorphic Computing: Neuromorphic computing aims to mimic the human brain’s architecture and functionality. Research in neuromorphic hardware, like IBM’s TrueNorth and Intel’s Loihi, shows promise in creating more efficient and adaptive AI systems. These systems process information more like the human brain, using spiking neural networks and event-driven processing, allowing for continuous learning and adaptation without the need for extensive retraining.


Perceptron

When we look at the beginnings of machine learning and the Perceptron model introduced by Frank Rosenblatt in the late 1950s, we don’t see a static model. Instead, we see attempts to replicate the human brain in a more natural way, based on patterns well-known in biology. Rosenblatt’s Perceptron was inspired by the neural structure of the human brain and aimed to mimic how neurons process information.

Currently, AI is a simplification of this model because it “emulates” the idea of the perceptron using tremendous amounts of memory and processing power to adjust weights and optimize neural networks. Activation functions, inputs, outputs, weights, backpropagation, etc., are all put into an “emulated” solution that works to some degree, but is slow and memory-hungry.

How to Solve the Problem

Retraining the model and keeping it static is a kill-switch that can be used if something goes wrong in the future. However, it’s also slow and costly, burning a company’s budget faster than someone at Nvidia can say “AI HPC” (and they say that a lot). A solution to this problem could be neuromorphic code and later neuromorphic chips.

Neuromorphic AI models are interesting because they learn on the fly from feedback when input data is present. This dramatically reduces the feedback loop:

  • Input data
  • Process by paths (no layered architecture)
  • Validate if this input is known (result is non-zero at the decoder)
  • If the input is not known, ask for clarification on what to do with that input.
  • Update weights of the model.
  • Repeat.

This path-based neuromorphic AI architecture is more natural to how humans think and learn. We receive information, try to process it (understand it), and if we don’t know, we ask for clarification or help. It’s a more human-like way of thinking and learning new things because we are built to adapt. Your models should too.

The Future of Neuromorphic Architecture

With neuromorphic architecture—either built in software or hardware—we can constantly train the model with every input it does not understand, much like a child learning to talk. Step by step, starting with simple cases and then building more complex scenarios.

Neuromorphic architecture is currently under development by various universities and companies worldwide and has the potential to revolutionize AI and how we interact with it. Notable efforts include IBM’s TrueNorth and Intel’s Loihi chips, which are designed to process information in ways that mimic the human brain. It’s also an effective way of building small enough solutions that they can fit in the palm of your hand—your mobile phone—providing specific functionality. They are not like large language models (LLMs); they are not large but specialized, so they don’t need to be. If you need specific functionalities, you can always build new ones or connect two or three models working together in a processing pipeline—why not?

The question is not “if” neuromorphic architecture will empower humanity with specialized AIs but rather “when” it will, and I think we’re closer than some expect.
Conclusion

The evolution of AI towards models with neuroplasticity-like features is a critical area of research. Neuromorphic computing and adaptive learning represent significant steps towards creating AI that can learn and adapt like the human brain. As this field progresses, it has the potential to transform how we interact with and utilize AI, making it more efficient, adaptive, and integrated into our daily lives.

Source & references