Soofi S: Germany’s fully open model with 70B quality at a cost of only 3B compute

This model was developed by a German consortium led by KI Bundesverband and supported by Germany’s Federal Ministry of Economic Affairs. Participants were the Fraunhofer Institutes, DFKI, TU Darmstadt, Ellamind and Merantix Momentum.

It’s based on a hybrid Mamba-Transformer architecture, which has a total of 31.6 billion parameters, but only 3.2 billion active params per token. This allows for similar computational requirements compared to a 3B model but results in much better performance.

It was trained on the Deutsche Telekom Industrial AI Cloud in Munich using up to 512 Nvidia B200 GPUs, 253,000 GPU hours and approximately 27 trillion tokens (March–May). The training set consisted of HPLT, the Commons corpus and the commercial Genios corpus (1.3%).

Performance: Best performance for fully open models on English and German benchmarks (OLMo 3 32B and Apertus 70B) HumanEval: 73.8% German MBPP: 84.2% Weaknesses: German math and factual recall

Weights, checkpoints, code and data manifest are made public. Project conforms to Open Source AI Definition 1.0 but fails stricter European requirements due to license dependency on Genios.