A measured place to start

Same computation. Then compare.

LoRA can be expressed as an ARTI Formula. First check the same computation and weights across implementations. Separately inspect the earlier English/Chinese experiment, which changed the formula and mounting position.

Check the same LoRA ↓Run it yourself ↗

ONE FORMULA, THREE IMPLEMENTATIONS

LoRA is a Formula program.

A standard low-rank update is part of ARTI’s program vocabulary. Match that update before judging the implementation.

y = Wx + (α / r) · B(Ax)

EM Answers matching a reference under the task scorer. F1 Overlap between generated and reference answer text.

The same trained weights · seeds 17 / 29 / 43

SQuAD 1.1

PEFT LoRA

EM ↑: 56.77 ± 1.85
F1 ↑: 64.22 ± 1.50

ARTI Formula LoRA

EM ↑: 56.77 ± 1.85
F1 ↑: 64.22 ± 1.50

CMRC2018

PEFT LoRA

EM ↑: 46.88 ± 1.95
F1 ↑: 70.77 ± 1.40

ARTI Formula LoRA

EM ↑: 46.88 ± 1.95
F1 ↑: 70.77 ± 1.40

The existing PEFT A/B matrices are bound to the native LoRA Formula. The same Qwen base, final two v_proj layers, rank 32, alpha 32, zero dropout, no adapter bias and precision are retained. The host stays BF16; adapter arithmetic uses FP32.

The same trained weights · seeds 17 / 29 / 43
Task	Implementation	EM ↑	F1 ↑
SQuAD 1.1	PEFT LoRA	56.77 ± 1.85	64.22 ± 1.50
SQuAD 1.1	ARTI Formula LoRA	56.77 ± 1.85	64.22 ± 1.50
CMRC2018	PEFT LoRA	46.88 ± 1.95	70.77 ± 1.40
CMRC2018	ARTI Formula LoRA	46.88 ± 1.95	70.77 ± 1.40

All 1,536 heldout generated answers match their recorded PEFT counterparts. These are imported trained weights, not independent ARTI training runs. The score demonstrates the same program behavior for this profile.

Training semantics: For each language and seed, three actual reading-task updates are repeated from the same weights in PEFT, handwritten PyTorch and ARTI. Maximum loss / parameter-gradient / updated-parameter differences: 0 / 0 / 0.
Program declaration: The native builder expands the update into two contractions, a scale and an addition. Its JSON declaration is reconstructed through FormulaProgram.from_dict before execution.

Python / declare the LoRA program

from arti import mechanisms as api

program = api.build_lora_program(
    input_dim=1024, output_dim=1024, rank=32,
    source_ref="arti/formula-operand-bank@1",
    dtype="float32",
    contract_accumulation_dtype="float32",
    pointwise_accumulation_dtype="float32",
)

The program takes x, the base projection and alpha/r as inputs; A and B are Bank operands. The runnable script shows their bindings and the model boundary.

Runnable Formula / LoRA check ↓Protocol and results ↓Every paired answer ↓Native builder source ↗

Actual generated answers

Two tasks. Different program choices.

Measured / 2026-10-12

English reading

SQuAD 1.1

Follow the practical case ↗

EM Answers matching a reference under the task scorer. F1 Overlap between generated and reference answer text.

SQuAD 1.1 · 256 fixed questions

Frozen base

Trainable values · 0

EM ↑: 55.08
F1 ↑: 62.56

SiLU residual · PyTorch

Trainable values · 131,072 · 512.4 KiB

EM ↑: 55.60 ± 0.81
F1 ↑: 62.43 ± 0.54

LoRA · PEFT

Trainable values · 131,072 · 512.5 KiB

EM ↑: 56.77 ± 1.85
F1 ↑: 64.22 ± 1.50

SiLU residual · ARTI

Trainable values · 131,072 · 545.4 KiB

EM ↑: 55.60 ± 0.81
F1 ↑: 62.43 ± 0.54

SQuAD 1.1 · 256 fixed heldout questions
Formula / implementation	Parameters	EM ↑	F1 ↑	Weight files
Frozen baseone frozen-base run	0	55.08—	62.56—	—
SiLU residual · PyTorch3 random seeds	131,072	55.60± 0.81	62.43± 0.54	512.4 KiB
LoRA · PEFT3 random seeds	131,072	56.77± 1.85	64.22± 1.50	512.5 KiB
SiLU residual · ARTI3 random seeds	131,072	55.60± 0.81	62.43± 0.54	545.4 KiB

EM and F1 are percentages; higher is better. Mean ± sample standard deviation over seeds 17 / 29 / 43; frozen base measured once.

Chinese reading

CMRC2018

Follow the practical case ↗

EM Answers matching a reference under the task scorer. F1 Overlap between generated and reference answer text.

CMRC2018 · 256 fixed questions

Frozen base

Trainable values · 0

EM ↑: 46.88
F1 ↑: 71.71

SiLU residual · PyTorch

Trainable values · 131,072 · 512.4 KiB

EM ↑: 48.31 ± 0.23
F1 ↑: 72.15 ± 1.23

LoRA · PEFT

Trainable values · 131,072 · 512.5 KiB

EM ↑: 46.88 ± 1.95
F1 ↑: 70.77 ± 1.40

SiLU residual · ARTI

Trainable values · 131,072 · 545.4 KiB

EM ↑: 48.31 ± 0.23
F1 ↑: 72.15 ± 1.23

CMRC2018 · 256 fixed heldout questions
Formula / implementation	Parameters	EM ↑	F1 ↑	Weight files
Frozen baseone frozen-base run	0	46.88—	71.71—	—
SiLU residual · PyTorch3 random seeds	131,072	48.31± 0.23	72.15± 1.23	512.4 KiB
LoRA · PEFT3 random seeds	131,072	46.88± 1.95	70.77± 1.40	512.5 KiB
SiLU residual · ARTI3 random seeds	131,072	48.31± 0.23	72.15± 1.23	545.4 KiB

EM and F1 are percentages; higher is better. Mean ± sample standard deviation over seeds 17 / 29 / 43; frozen base measured once.

Weight-file size includes inference tensors and ARTI graph-state sidecars, excluding tokenizer, JSON/TOML declarations and training checkpoints. This comparison reports answer quality and saved artifact size; it makes no speed or hardware-efficiency claim.

What the earlier experiment measured

The SiLU residual and the v-projection LoRA have different scores on these fixed tasks. Formula, activation and mounting position changed together. These differences cannot identify a framework effect or imply that ARTI cannot express the LoRA computation.

ARTI and the handwritten PyTorch SiLU program produced the same answers. That is an implementation check for this one formula. PEFT’s LoRA row is a separate program choice; its measured outputs remain available for inspection.

MORE THAN A SCORE

Know what you are choosing.

PyTorch

Write the function yourself.

A rank-32 down → SiLU → up → residual module after the last two MLPs. It matches ARTI’s function, initial values and data order. The runner saves weights; module reconstruction lives in its Python source.

PEFT LoRA

Choose a familiar low-rank adapter.

Rank 32 / alpha 32 on the last two square value projections, no bias or dropout. PEFT saves LoRA weights and mounting configuration. Its linear update has a different placement and activation from the residual program.

PEFT LoRA ↗

ARTI Formula

Keep the computation with the values.

The nonlinear residual is a typed FormulaProgram with Bank operands and named input/output resources. Export retains computation and trainable values. The built-in loader reconstructs this program from the asset.

Public Qwen recipe ↗

Verified beyond training.

Cold load: 512 heldout answers reproduced from saved ARTI assets in a separate process.
Controlled continuation: A 3-update continuation matched a save/reload/continue path in parameters and data cursor.

These local checks qualify the named public Qwen Formula profile. PyTorch and PEFT can also use training lifecycle tools; this harness does not claim they lack recovery.

THE RECORDED PROTOCOL

Keep the experiment inspectable.

Base & precision: Qwen/Qwen3-0.6B, immutable revision c1899de…e47ca. Frozen BF16 host and FP32 adapters. PyTorch matches ARTI’s BF16 initialization round trip and shared FP32 input conversion.
Data & split: 512 train, 128 validation and 256 heldout eligible questions per language. Context-first SHA256 order; validation contexts removed from train. Public dev supplies heldout. Original IDs and independent references are retained.
Eligibility: Complete native chat and answer ≤384 tokens; first answer ≤32 tokens. No silent truncation. These are short-context subsets, especially for Chinese; exclusions are part of the record.
Optimization: AdamW, lr 0.001, weight decay 0.01, clip 1. Batch 8, 200 updates, constant LR. Same length-grouped data order per seed. Last checkpoint; no heldout tuning.
Supervision: Only the final assistant answer through native EOS is labeled. User, system and padding are masked. Native template uses enable_thinking=false. Real EOS stays supervised even when PAD equals EOS.
Generation & scoring: Greedy generation ≤32 new tokens, identical left padding and EOS. SQuAD uses the official answerable EM / token F1 core, maximum over references. CMRC uses official Python 3 v6 mixed segmentation / longest-common-substring F1. Training loss is diagnostic.
Reproduction dependencies: Torch 2.11.0 · Transformers 5.13.0 · PEFT 0.19.1 · ARTI 3.3.0

Fixed public-dev slices, not official hidden-test leaderboard results or production SLOs.

See every eligibility count

SQuAD 1.1

Source	Total	Eligible	Too long	Answer >32
train	87,599	84,688	2,707	204
dev	10,570	10,169	394	7

CMRC2018

Source	Total	Eligible	Too long	Answer >32
train	10,142	4,380	5,181	581
dev	3,219	1,422	1,762	35

OPEN THE RECORD

Take the script. Check the answers.

Follow either practical case to install the pinned dependencies. Then run all methods and seeds locally. Every run reuses the first download.

Terminal / complete comparison

python reading_qa.py all --work reading-qa-work
python reading_verify.py --work reading-qa-work
python formula_lora.py --work reading-qa-work

Same-LoRA implementation checkformula_lora.py ↓Reproduction scriptreading_qa.py ↓Reload & resume checkreading_verify.py ↓Results and evaluation protocolJSON ↓Every English answer2,560 rows · CSV ↓Every Chinese answer2,560 rows · CSV ↓

Each CSV contains one frozen-base run plus nine trained runs ×256 questions. Dataset excerpts and references retain CC BY-SA 4.0 attribution. Qwen model weights are downloaded separately under Apache 2.0.

SQuAD ↗CMRC2018 ↗ARTI 3.3.0 ↗