DNA models

Carbon-X release combines DNA sequences and natural language

LoubnaBenAllal1 introduced Carbon-X as a 14B model trained on DNA and text that can answer questions about sequences and mutations or generate DNA. _lewtun said that it beat every open language model tested on DNA benchmarks and performed within a point of GPT-6.1 Sol at more than 1,000 times lower inference cost.

_lewtun said that training used 300B tokens of interleaved DNA and text, followed by supervised fine-tuning and reinforcement learning without distillation. The account described a hybrid tokenizer using approximately three times fewer DNA tokens than BPE and said that the Apache 2.0 model runs locally with llama.cpp. ClementDelangue framed the open release as a way for researchers to study the model's capabilities, limitations and risks.