AMD Buys Taalas, Which Embeds AI Models’ Weight Parameters In Hardware

Chipmaker AMD has bought Taalas, a Canadian startup which specialises in embedding neural network weights on silicon. Their chips can run Llama 3.1–8B at >16,000 tokens/sec, which is significantly faster than GPUs. While they haven’t revealed how much they paid, they intend to integrate this tech into their Instinct accelerators. Similarly, Google is working on something called Gemini.

Each of these chips has a single embedded model (which cannot be reprogrammed), however if you have a service which requires a single task to be completed (such as content moderation, text-to-speech or embedded AI agents) and you want it to be performed quickly (>16,000 tokens/sec per user), then these chips will revolutionise your business model. This represents another step toward dedicated hardware which is tailored specifically to a single AI model.