TensorCode

Trainable models are ordinary functions in your program.

TensorCode is a Python and TypeScript library. You call encode, classify, score or decode where you would call any function, branch on what they return, and train them from the values your program produced. Written code and trained models share one program, one trace and one saved artifact.

Docs GitHub ↗

A model's output is a value you branch on, and a rule you write runs before the model.

classify(encode(text)) returns a label. The if in front of it is ordinary Python, so a ticket that mentions a lawyer goes to legal before the model sees it.

def route(text):
    if "lawyer" in text: return "legal"
    return classify(encode(text)).value

branch.py

text "lawyer" in text "legal" encode classify "billing"

writtenlearned

Supervising one value trains exactly the operations that produced it.

Inside trace() each value records the operation that produced it. t.supervise(guess, labels, source=...) attaches labels to one value and names where they came from. Trainer.from_ops then updates encode and route and nothing else. Ordinary Python between operations is not made differentiable.

with trace() as t:
    guess = route(encode(tickets))
t.supervise(guess, labels, source="reviewer")
trainer = Trainer.from_ops({"encode": encode,
                            "route": route})
trainer.fit([t], epochs=50)

hello.py

tickets encode route guess labels supervise gradient

Gradients propagate through ordinary Python functions, so you can differentiate the output of arm(pose(target)) against target and train pose with no joint-angle labels.

Arm is forward kinematics for three links, written in plain tensor code and marked replayable. pose is a learned network from a target position to three joint angles. The loss is the distance between the fingertip and the target, and the gradient reaches pose through Arm. After 400 epochs on six targets, the fingertip lands within 0.13 m of three positions it never trained on.

# written: forward kinematics, three links
class Arm(Operation):
    replayable = True
    def forward(self, angles, *, context=None):
        x = y = heading = 0.0
        for L, a in zip(LINKS, angles.unbind(1)):
            heading = heading + a
            x = x + L * cos(heading)
            y = y + L * sin(heading)
        return stack([x, y], -1)

pose = Decode({"architecture": "mlp",
               "output_dimensions": 3, ...})
with trace() as t:
    tip = arm(pose(targets))
t.supervise(tip, goals, loss="mse")

reach.py

target arm, three links

A written mask decides which options exist, and a learned score ranks the ones that do.

allowed is a dict of conditions you write. Decide picks the best-scoring action whose condition holds, so a masked action is never chosen, and changing a condition changes the choice without retraining. score was trained on eight postmortems. Below: the incident "errors after the deploy an hour ago" during peak hours, 20 minutes after a deploy.

allowed = {
    "restart":  not peak_hours,
    "rollback": deploy_age < 60,
    "scale":    budget_left > 0,
    "page":     True,
}
mask = tensor([allowed[a] for a in ACTIONS])
options = CandidateSet(encode(incident),
    Latent(actions.tensor, space, mask=mask),
    ACTIONS)
choice = decide(score(options)).identity

rules.py

  1. restartmasked, peak hours
  2. rollbackchosen
  3. scale
  4. page

learned scorewritten mask

A bounded numerical function compiles to a circuit that is checked on every input.

tensorcode.program.compile turns a function over declared integer ranges into a circuit. verify runs all 256 inputs of an int32(min=-128, max=127) against a reference. Parameters you declare are the only values training changes; with hybrid semantics a comparison keeps its hard value forward and takes a sigmoid surrogate backward. In the source checkout.

from tensorcode.program import compile, int32

src = "def choose(a): return 10 if a > 4 else 12"
choose = compile(src, inputs={
    "a": int32(min=-128, max=127)})
reference = lambda a: 10 if a > 4 else 12
checked = choose.verify(reference)
assert checked["passed"]
assert checked["cases"] == 256

Compiled programs

One tensor algorithm runs as a NumPy program, a PyTorch graph or a scheduled attention machine, and the three are compared case by case.

tensorcode.algorithm.compile parses the source and never executes it. "alg0", "alg1" and "alg2" are the three executors. compare reports the maximum error and the first counterexample over the cases you supply, and a hard attention machine converts back to the symbolic program. In the source checkout.

from tensorcode.algorithm import compile, compare

src = '''def policy(position, target, gain):
    return {"torque": gain * (target - position)}
'''
policy = compile(src, inputs={"position": (20,),
                              "target": (20,)},
                 parameters={"gain": 30.0})
numpy = policy.compile("alg0")
attention = policy.compile("alg2")
assert compare(numpy, attention, cases)["passed"]

Algorithm substrates

algorithm.json NumPy PyTorch attention machine compare

A workload you send to a large model can be distilled into a small program that is scored on cases its author never saw.

distill has an authoring model write one bounded function from your inputs and the large model's answers, compiles it without executing it, scores it with feedback on half the cases and once on the hidden half. A Router serves the program and escalates inputs it does not claim; repair folds those back in. Measured so far with recorded author replies only. In the source checkout.

from tensorcode.distill import distill

result = distill(task,
                 inputs={"features": (12,)},
                 workload=rows, targets=answers,
                 author=model, holdout=0.5)
print(result.report["acceptance"])
router = result.router(large_model)

Distilled programs

Programs save as JSON and safetensors, with no code, and load in Python, TypeScript and the browser.

save_pretrained writes configuration and weights; from_pretrained reloads them from disk or the Hugging Face Hub, and loading never evaluates source. Chatbot, Investigator, Decision, Planner and Scene are complete models saved the same way. Compiled programs also run in the browser on WebAssembly or WebGPU.

path = "./investigator"
model.save_pretrained(path)
restored = Investigator.from_pretrained(path)

Quickstart

config.json weights.safetensors Python TypeScript browser: WASM or WebGPU

On a simulated Unitree G1, a written controller placed 23 of 24 parcels; seven policies trained on its demonstrations placed none.

Unitree G1 with Dex3-1 hands in MuJoCo, rendered in Cycles; the scenery is not simulated. Every run is published with its failures. Every run ↗

Install.

Python 3.11 or newer, or Node.js 20.16 or newer. hello.py below trains a classifier on six tickets and routes a new one; it runs offline on a CPU in a few seconds. The library is at 0.4 alpha, and APIs may change between alphas.

from tensorcode import trace, training
from tensorcode.ops.vec import Classify
from tensorcode.ops.vec import VocabularyEncoder

tickets = ["refund my order", "charged twice",
           "the app crashed", "error on upload",
           "forgot my password", "cannot log in"]
labels = ["billing", "billing", "technical",
          "technical", "account", "account"]
words = sorted({w for t in tickets
                for w in t.split()})

space = {"name": "tickets", "dimensions": 8}
encode = VocabularyEncoder(
    {"vocabulary": words, "dimensions": 8,
     "output_space": space})
routes = ["billing", "technical", "account"]
route = Classify({"architecture": "linear",
                  "input_space": space,
                  "labels": routes})

with trace() as t:
    guess = route(encode(tickets))
t.supervise(guess, labels, source="me")
trainer = training.Trainer.from_ops(
    {"encode": encode, "route": route}, lr=0.5)
trainer.fit([t], epochs=50)

# technical
print(route(encode("the upload crashed")).value)

hello.py · prints technical