TL;DR – AI is still held back by the same thing it’s always been restrained by… compute power. Even though we can muster more now than ever before, it will cost you… a lot. Training the latest and greatest models at scale is incredibly expensive and the companies with the resources to train them have no real incentive to reduce price. So what’s a machine learning programmer to do? Keep calm and use “the edge”.
A (really) Brief History
The concept of artificial intelligence has been around in various forms, and especially in science fiction, for a very long time. It wasn’t until the 1950s that Alan Turing formally manifested the concept of how to train and test machine intelligence in his paper “Computing Machinery and Intelligence”. Less than a decade later, thanks to a computer scientist, a programmer and an economist (yes, an economist), the first program- Logic Theorist-that replicated human problem-solving skills was born in 1956.
Since then, artificial intelligence has been honed slowly, in fits and spurts. Advances were touch and go, largely limited by the compute power of those times.
Fast Forward to Now…
…I told you it would be a brief history. It’s brief for one reason- artificial intelligence is still hindered by the same issue… raw compute power- well, and storage.
There was a brief time where the learning algorithms, compute power and cost all matched up quite well and the playing field was wide open. This led to some great advancements in the field. However, a combination of conditions – Moore’s Law beggining to slow while storage and data continue to amass- have created a perfect storm to bring cost back into the equation.
The latest algorithms in AI training require massive amounts of data (and a place to store it), compute and time in order to work, and all of that costs. Only wildly well-funded companies can afford the storage, compute power and data collection to properly train these models.
With price tags in the millions, the latest and greatest in AI seems limited to the upper eschelon of the tech and business world.
But then, that shouldn’t really be a surprise to anyone.
Find any booming industry and you’ll see how people have managed to monetize the resources it takes to execute. AI is no different. As with any product whose resources (or at least the volume of resources required) are limited, the cost will invariably reflect the scarcity so long as the demand is high… and the demand is high. So organizations with the means to train models have no real reason to reduce their multi-million dollar price tags.
What About the Rest of Us?
So does this mean your home project is doomed? Is there a paywall to learning this field? Is your startup now doomed to wither and die on the vine?
No- not really. In fact, due to the cost, the field of AI is finally thinking creatively again.
While home projects and proof of concept work can utilize existing, less than cutting edge algorithms to move forward, we are also seeing a paradigm shift. Humans seem to default to the idea that “more is better”. Call it a symptom of rampant consumerism, a genetic vestigial urge to hoard resources from our nomadic ancestors or simply the fact that it’s easier to get bigger rather than better.
Whatever you blame the condition on, the current price for cutting edge AI training has shifted work toward refinement rather than resource collection. There are now teams of computer scientists working on reducing the footprint of data models, pruning out inefficient processes that will reduce resource requirements.
There is also a lot of work going on to modularize the workloads for AI, pushing some of the processing out to “the edge” (IoT devices) using tools like tinyML. Additionally, there is work being done on the hardware end of the field, creating specialized chips to deal with some of the more intensive processes in an efficient way.
The combination of these efforts combined with companies like Mosaic ML that plan to open-source a great deal of their technology should infuse the industry with the required freedom and openness to continue making advancements despite the current paywall.
So don’t worry- you, too, can still contribute to the eventual enslavement of humanity by our AI overlords!
Happy tinkering!
