YecoAI

Yeco Flash Mini-IT v1

118.1M-parameter Italian micro-LLM, trained for under €15, ~200 tok/s on CPU. Beats Minerva-350M (3× its size) on BoolQ-IT.

We present Yeco Flash Mini-IT v1, a 118.1M-parameter decoder-only language model for Italian that proves a contested point: sovereign, useful language models no longer require frontier budgets. Trained end to end for less than €15 of H100 time, Flash Mini-IT implements a custom 32k BPE tokenizer engineered for Italian morphology and post-SFT behavior that eliminates the failure modes typical of micro-scale models. On the official ITA-Bench suite (SapienzaNLP), it answers BoolQ-IT at 62.5%, above Minerva-350M (60.7%), a model 3× its size with roughly 70B pretraining tokens behind it. The model, its tokenizer, and its full training pipeline are released under Apache 2.0 with a complete model card, including the limitations.

Motivation: the small-model gap in Italian

Italian occupies an awkward position in the open-model landscape. The largest multilingual checkpoints treat it as one tail among a hundred; the few Italian-specialized models (MINERVA, Fauno, Camoscio) sit at 350M-7B parameters with limited documentation of their training economics. Meanwhile, the cost of a single frontier pretraining run exceeds the entire annual budget of most Italian research groups. The result is a gap: no reproducible, fully documented pipeline showing what a minimal Italian LLM costs and how far it gets.

Flash Mini-IT was designed as exactly that reference point. Every design decision, tokenizer, architecture, data mixture, training schedule, was made to maximize Italian-language quality per euro of compute, and every number in this article is measured on public benchmarks rather than estimated. Where the model loses, we say so: the model card reports full per-task results including the tasks where Minerva-350M clearly wins.

Architecture and tokenizer

The backbone is a GPT-2-style decoder-only transformer: 12 layers, hidden dimension 768, 12 attention heads, pre-LN normalization, and weight tying between input embeddings and the output projection. On top of the standard attention computation we add a learned context gate, a sigmoid-modulated scaling of attention scores, which stabilizes training at this scale and improved early validation loss in ablation runs.

The tokenizer is where most of the Italian-specific engineering lives. A byte-pair encoding vocabulary of 32,000 tokens was trained from scratch on Italian Wikipedia, replacing GPT-2's byte-level BPE whose merges were optimized for English. The resulting tokenizer has a fertility of 1.53 tokens per word, a 34% reduction against GPT-2 on the same Italian corpus. Fewer tokens per word means longer effective context, less attention computation, and faster generation, a compounding advantage that shows up in every downstream metric per unit of compute.

Training: two phases, under €15 total

Training proceeded in two phases, both on a single NVIDIA H100 rented on Modal.

Phase 1, pretraining

  • Corpus: 1.25B tokens of Italian Wikipedia, packed at the model's effective sequence length.
  • Schedule: 10,000 optimizer steps, AdamW, cosine decay.
  • Final validation loss: 2.8535.
  • Cost: approximately $8.75 of H100 time.

Phase 2, supervised fine-tuning (V2 checkpoint)

  • Data: 310,809 examples, real Italian instruction data (fine-instructions-ita-70k, alpaca-gpt4-it, evol-instruct-it), math with chain-of-thought (orca-math-decompositions IT), QA (SQuAD-IT), chat (oasst2-it-top1, italian_conversations), ~500 paraphrased self-identity variants, and 140k synthetic math/logic examples with CoT, half with direct answers.
  • Key fix: loss masking on the assistant tokens only, which eliminated the input-echo failure mode of the base checkpoint (the model used to repeat the question before answering).
  • Schedule: 28,950 steps (3 epochs, batch 32, cosine 1e-4 → 1e-5).
  • Final answer-only validation loss: 1.9585.
  • Cost: 0.95 H100-hours, approximately $4.
Total training cost
€15
-34%Single H100, 1.25B pretraining tokens plus SFT
Tokenizer fertility
1.53 tok/word
-34%34% better than a GPT-2 style tokenizer
BoolQ-IT
62.5
+1.8%Beats Minerva-350M (60.7) at 1/3 the size

Evaluation: measured, not estimated

All benchmark numbers come from the official ITA-Bench datasets (sapienzanlp/arc_italian, hellaswag_italian, boolq_italian, truthful_qa_italian, gsm8k_italian), evaluated with the official ITA-Bench harness for LLM evaluation on Italian tasks: accuracy in percent, zero-shot protocol, 4k context window, results averaged over 3 runs. Emma-5 550M numbers come from the same setup; Minerva-350M figures are from its paper (see the note below the table). These are measured numbers, not projections.

Reading the numbers honestly: Flash Mini-IT takes BoolQ (62.5%, the model’s best score of the suite, ahead of Minerva-350M’s 60.7% despite Minerva’s 3× size and ~70B pretraining tokens), while Minerva-350M stays ahead on ARC-Challenge (24.6% vs 23.0%), HellaSwag, TruthfulQA and GSM8K, and on the five-task average (42.5% vs 36.1%). The chat version (SFT v3) also stays at a 36.1% average. The value of SFT shows in behavior rather than in these scores: a reliable question → answer → stop format, elimination of question echo, and structured chat answers.

  • ARC-Challenge: 23.0% (Emma-5 550M: 22.0%, Minerva-350M: 24.6%)
  • HellaSwag: 26.0% (Emma-5: 31.5%, Minerva: 32.6%)
  • BoolQ: 62.5% (Emma-5: 39.0%, Minerva: 60.7%), Flash Mini-IT wins, best score of the suite
  • TruthfulQA: 34.5% (Emma-5: 21.0%, Minerva: 46.5%)
  • GSM8K: 34.5% (Emma-5: 39.0%, Minerva: 48.2%)
  • Five-task average: 36.1% (Emma-5: 30.5%, Minerva: 42.5%)
ITA-Bench: Flash Mini-IT vs Emma-5 vs Minerva-350M
Accuracy in percent on five Italian tasks, higher is better
YecoAI
ITA-Bench harness, zero-shot protocol, 4k context window, average of 3 runs. Minerva-350M: figures from its paper.
Full ITA-Bench results
Accuracy %, zero-shot, 4k context, average of 3 runs
TaskYeco Flash Mini-IT 118MEmma-5 550MMinerva-350M
ARC-Challenge23.022.024.6
HellaSwag26.031.532.6
BoolQ62.539.060.7
TruthfulQA34.521.046.5
GSM8K34.539.048.2
Five-task average36.130.542.5

Minerva-350M figures are from its paper (full N, possibly different few-shot): same benchmark and scoring, different N. The Minerva paper may also use a different evaluation protocol (number of few-shot examples, prompt format, score normalization), so part of the gap may come from the protocol rather than from the models. Baseline comparisons are therefore indicative, not certified head-to-head runs: only a run of Minerva-350M under our exact protocol would settle the comparison. On generative behavior, the post-SFT model reliably produces input → response → stop, has lost the echo failure ("What is Windows → windows ["), and produces sensible lists and prose on general-knowledge questions. Weaknesses are documented rather than hidden: self-identity is inconsistent, factual recall at 118M parameters / 1.25B tokens invents specifics, and the model imitates the form of chain-of-thought reasoning while failing the arithmetic.

Usage and reproducibility

The SFT prompt format is a two-line template, [CHAT] Utente: {question} / Assistente:, with generation stopped on a fixed four-token id sequence ([END]). The final checkpoint (models/best_v2.pt), the tokenizer (models/it_bpe_32k.json), the full Modal training pipeline (train_v2.py), the data preparation scripts (scripts/prepare_v2_datasets.py), and the benchmark harness (run_real_itabench.py) ship together under Apache 2.0. A chat GUI (chat_gui.py) is included for interactive evaluation. One sharp edge is documented for reproducers: the special chat tokens are not in the vocabulary file, do not add them with add_special_tokens, as ids ≥ 32768 will crash the model.

Limitations

  • 118M parameters: no complex reasoning; arithmetic is unreliable.
  • 1.25B pretraining tokens: limited encyclopedic knowledge; frequent hallucination of specifics.
  • No RLHF/DPO stage; alignment comes from SFT data only.
  • Monolingual: the tokenizer is optimized exclusively for Italian.
  • Special chat tokens live as sub-token sequences, not vocabulary entries (see usage note).

Next steps

The model card identifies three levers, in order of expected impact: continued pretraining on 2-3B further tokens (Wikipedia + packed QA, estimated $4-6), the only lever that moves ARC/HellaSwag/TruthfulQA; more aggressive self-identity data plus light DPO; and a full lm-evaluation-harness run on complete datasets for certified numbers. Flash Mini-IT is intentionally a living checkpoint: every result above will be superseded as the series trains, and the model card is versioned to make each step auditable. The model is part of the Yeco LLM Series, which brings together all YecoAI models.

A fully documented Italian LLM, tokenizer to benchmark, for the price of two pizzas. That is the reproducible floor the sovereign-AI debate was missing.

Related content