What Can Run Frontier-Competitive AI on a Low-Power CPU?

1. What 'Frontier-Competitive on a Low-Power CPU' Actually Requires

The query asks a deceptively high-bar question: what lets a CPU-only low-power device match or beat frontier-model quality on useful tasks? The honest reading — reinforced by the corpus across all eleven paradigms — is that the term survives only when you name a narrow useful task where a frontier LLM is energy-overkill. The query itself licenses this: it states that beating frontier on a narrow useful task counts as a win, provided the task is named explicitly. The phrase match or beat frontier-model quality therefore does not mean generating text like GPT-5 at lower cost, but delivering the quality the user actually needs, at a fraction of the energy, on a specific named task with a 50 KB classifier or a small in-RAM datastore.

The decisive evidence norm is the filter that separates demonstrated wins from advocacy. A measurement counts only if it names the hardware, the task, and the metric; energy claims in joules-per-token or tokens-per-watt outrank latency-only claims. Simulated, FPGA, ASIC, or neuromorphic numbers must be flagged as such and never reported as CPU measurements. The solo-builder envelope is a 24 GB Apple-Silicon Mac plus roughly $100 of GPU credit total; anything needing a cluster, an H100, or datacenter hardware is out of scope.

Apply that norm across all eleven paradigms and a clean split emerges. Three paradigms carry admissible CPU evidence on a named narrow task: Tsetlin Machines (interpretable classification), retrieval-dominant tiny models (small-corpus factual recall), and HDC/VSA plus the NysHD kernel bridge (structured/relational classification). The rest fail the norm in one of three ways — hardware-conditional (spiking neural networks), rebranded-excluded (MatMul-free / analog-in-software collapses to the excluded ternary transformer), or unmeasured (non-backprop training, energy-based models / Hopfield, cellular automata, program synthesis, symbolic distillation). This report argues that split is the answer, not a pessimistic read of an under-explored field.

A second filter sits underneath the first and must be stated up front. The approaches this lab already built and measured — transformer+quantization (GGUF/ternary/BitNet), MoE expert streaming from SSD, linear attention / Gated DeltaNet hybrids, Rust/candle engines, and retrofit sparse-memory finetuning — are excluded by the query itself and appear here only as baselines, never as recommendations. The excluded ternary transformer is not a footnote: it is the unacknowledged benchmark every non-excluded paradigm is measured against. On CPU the only fast ternary engine is bitnet.cpp, which reaches 2.37×–6.17× x86 and 1.37×–5.07× ARM speedups over full-precision baselines ^[1], ^[2]. It is the quality-per-joule frontier that the non-excluded paradigms are trying to beat on their narrow tasks, and reproducing it is the control every other test must clear. The single most important practical takeaway of the whole survey is that this control is almost never measured in the literature — the field-wide energy blind spot documented in the next sections hits the three "winners" as much as the "traps."

2. Tsetlin Machines

A Tsetlin Machine (TM) is a learning architecture built from many small teams of boolean automata that learn interpretable logical clauses (e.g. feature 3 AND NOT feature 7) rather than dense floating-point weights. Because inference reduces to boolean AND/XOR/accumulate over integer counters, it runs natively on CPUs and even on sub-100 MHz microcontrollers with no matmul unit. That mechanical simplicity is the source of its efficiency story.

The strongest measured efficiency result is on text classification: a Fuzzy Pattern Tsetlin Machine reaches 90.15% IMDb accuracy with a single clause per class, a 50 KB footprint, and 34.5 million predictions per second (51.4 GB/s effective throughput), training up to 316× faster than a Coalesced Tsetlin Machine ^[3]. The largest useful CPU win is interpretable text/tabular classification at classical-ML parity (~90% IMDb, ~20 Newsgroups), where the competing "frontier" is an LLM that is overkill on energy — not a frontier LLM that TM beats on quality. Early TM formulations already framed the method as learning human-interpretable rules for high-dimensional data ^[4], and the original game-theoretic bandit formulation underpins its training ^[5].

What breaks at scale is the semantic ceiling. The field's own 2025–26 work states plainly that a plain TM "captures little semantic information" and that earlier embedding bridges "miss contextual meaning," motivating LM-hybridization that reaches only "competitive with BERT" rather than above it ^[6]. Variable and structured input needs graph message passing (GraphTM) ^[7], embedding-based word representations ^[8], and state-space reasoning-by-elimination extensions ^[9]; the only sequence-generation TM benchmark is UCR time-series, not language ^[10].

The break is three explicit ceilings: a semantic ceiling (a plain TM captures little distributional meaning), a flat-fixed-length-input ceiling (vanilla TM is confined to concise flat AND-rules over fixed-length bit vectors), and a no-generation ceiling. The last is quantifiable — even reasoning-extended TM cannot escape the ~90% band: Reasoning-by-Elimination (2024) tops out at 90.62% IMDb ^[9], no leap over 2018's ~90% band eight years on — and matches no LLM on open-ended generation, QA, or multi-step reasoning. Zero measurements in the corpus match an LLM on open-ended generation, QA, or multi-step reasoning. The LM-hybrid pivot is empirical proof that standalone TM cannot represent distributional semantics, and even the distillation path only imports knowledge from a neural net rather than surpassing one ^[11].

For a solo builder the cost is essentially free on CPU: reproduce the IMDb/20 Newsgroups win (Tests 1+3) in hours; an optional hybrid-ceiling probe with TinyBERT features spends only ~$30 of GPU. The decisive caveat: no peer-reviewed measurement of TM energy in joules on a stock low-power CPU exists in this corpus. Every energy number is on bespoke silicon — FPGA (Dynamic TM FPGA: 2.54× GOP/s/W and 6× less power, both flagged), a custom RISC-V subset (29.7× energy reduction, 88.18% CIFAR-2 vs 60% for a binary neural network, 98% less execution time — all flagged), FlexIC/ASIC (FlexIC is a 6800-NAND2-gate custom IC), or on-chip training accelerators ^[12], ^[13], ^[14]. The throughput claim is itself grounded by a concrete training ratio: FPTM trains in 45 s, 316× faster than the 4 h Coalesced TM ^[3].

None of these is a commodity CPU. The throughput number is real and impressive; the joules-per-token number is an audited absence.

3. Hyperdimensional Computing and Vector-Symbolic Architectures

Hyperdimensional computing (HDC), also called vector-symbolic architectures (VSA), represents every symbol, feature, or datum as a high-dimensional (typically 10,000-bit) random vector and performs computation by bundling (element-wise majority or addition), binding (XOR or Hadamard), and permutation ^[15], ^[16]. It is the most CPU-native non-mainstream paradigm in this survey: operations are bitwise XOR/permute/bundle, trivially parallel across CPU cores with no matmul, and robust to bit-flips and hardware noise. HDC has even been framed as a programming paradigm for oscillatory computing, reinforcing its hardware-friendliness ^[17].

Measured CPU efficiency is solid for classification. HyperEmbed shows 6.18× memory reduction and 4.62× training / 3.84× test speedups on NLP classification tasks at on-par F1 ^[18]. The genuine CPU win is NysHD — the Nyström kernel bridge — which lifts HDC to structured-object classification with +11% average on graph and +17% on string tasks, +38% specifically on ENZYMES, and matches/beats DGCNN/GIN/GIUNet by 3–6% average on graph classification; it is 52% faster than a graph convolutional network (GCN) on an Intel i5-11400 CPU (only large fine-tunes spilled to an RTX 3050) ^[19], ^[20]. Federated HDC variants confirm the resource story holds under distributed, resource-limited settings ^[21]. None of this touches generation or next-token prediction.

What breaks at scale is a provable theoretical capacity wall. The representational capacity of VSA bundling is hard-bounded by dimensionality and the bundling operator (Clarkson et al. give explicit bounds on the dimensions required to test set membership or estimate set-intersection sizes) ^[22]. THDC's fix — replacing static random hypervectors with backprop-trained embeddings and a one-layer binary readout, cutting dimensionality 10,000→64 — upgrades the encoder (data→hypervector fidelity) while leaving the downstream bundle→class-prototypes→nearest-class-mean readout — where the capacity bound lives — untouched ^[23]. The load-bearing negative: after 15+ years of edge-ML interest, zero generative-HDC results with numbers on any language-model benchmark exist in the corpus — a strong negative signal, not a gap.

A solo builder reproduces NysHD graph/string classification in minutes for free; a decisive sequence-task probe with a $100 GPU baseline (BitNet or tiny-transformer BPC comparison) is the only spend. As with TM, the energy advantage is real for classification but unmeasured for generation — every published number is classification train/inference, with no joules/token or generation latency reported.

4. Energy-Based Models and Modern Hopfield Networks

An energy-based model (EBM) learns a scalar energy function where low energy means "plausible," and inference finds low-energy states by gradient descent or Markov-Chain Monte-Carlo (MCMC) sampling. Modern Hopfield networks are a dense associative-memory variant whose update rule, in the 2020 reformulation, is mathematically equivalent to transformer attention. That equivalence is the key fact for this query: a Hopfield/attention layer is not an independent CPU-efficiency win, it is a different way to compute the same operation ^[24].

Where Hopfield layers do win is small/structured classification — immune-repertoire classification, drug-design tasks, and UCI small-classification benchmarks — where they set new state-of-the-art ^[24]. EBMs additionally excel at out-of-distribution detection and adversarial robustness, which is a genuine edge-task niche ^[25]. The query's four-field contract demands a measured CPU quality-vs-energy figure for every paradigm; for EBM/Hopfield the honest answer is an audited absence. No corpus source reports a CPU joules-per-token (or per-sample) number for a Hopfield/EBM at any demonstrated task — the immune-repertoire, drug-design, and UCI SOTA figures are accuracy-only and hardware-unnamed, and the modern-Hopfield "win" is structured classification, not language-model-scale CPU energy ^[24]. Like SNN and non-backprop, this family earns a None — audited absence of stock-CPU joules/token flag rather than a measured tradeoff.

But EBM inference relies on iterative MCMC/Langevin sampling — many steps per query — which is expensive on a CPU at serving time and inherits the same per-step cost as any sampling-based model ^[25]. The modern Hopfield layer's equivalence to multi-head attention means it inherits transformer-class CPU behavior rather than introducing a separately efficient compute path: it is a different derivation of the same operation, not a cheaper one ^[24].

What breaks at scale is the convergence cost. The Langevin/MCMC inner loop scales with the number of sampling steps multiplied by model size, and on a CPU the step-by-step cost dominates serving latency; an EBM also demands a carefully tuned energy function and a negative-sampling scheme, which is why the literature's EBM wins cluster on OOD/robustness rather than on language quality. There is no corpus source reporting CPU joules-per-token at LLM scale for Hopfield/EBM. They are niche structured-classification tools, not a path to frontier language quality. For a solo builder the concrete probe is a free SOTA reproduction on UCI/immune-repertoire with a single named commodity CPU (the Mac) and a powermetrics energy read — the $100 is better spent on the three admissible paradigms, but this control is cheap to run. The audited absence here is the same one that recurs across the field: no measured commodity-CPU energy figure at useful scale. The same fixed-point settling cost haunts this family's non-backprop cousins — Equilibrium Propagation's two-phase equilibrium (free + target-nudged) requires more serial compute than one forward + one backward pass, so EP is not an efficiency win over backprop ^[26].

5. Spiking Neural Networks on Commodity CPU

A spiking neural network (SNN) replaces continuous activations with discrete, sparse spikes over time, promising event-driven computation: a neuron fires only when its input crosses a threshold, so in principle most of the network stays idle. The promise is real on the right hardware — but the right hardware is not a von Neumann CPU.

The corpus shows zero admissible measurements of an SNN energy or quality win on a named commodity CPU. SpikeGPT trains a 216M-parameter generative SNN that is "competitive with non-spiking models on tested benchmarks," but its 20× operations claim is conditioned on neuromorphic hardware that can exploit sparse event-driven activations, and the model is GPU-trained ^[27]. The Loihi 2 result — 3× higher throughput and 2× less energy versus transformer-based LLMs on an edge GPU — is measured on neuromorphic hardware: Intel silicon, validated on GPU, not a CPU ^[28]. On FPGA, SNNs give little or no advantage at small scale (MNIST), and the trend reverses only at CIFAR-10, with a small, task-dependent edge (measured on FPGA) ^[29]. The only commodity-CPU SNN result (Spiffy, Raspberry Pi, MNIST 92%, 0.45 ms inference, 0.9 ms train) is latency-only with no energy and no ANN baseline; the only commodity-edge SNN study (Ev-Edge) explicitly reports inferior SNN/ANN performance on commodity CPU/GPU/NPU due to event-stream↔hardware mismatch, with no energy figure.

The mechanism matters. On a von Neumann CPU an equal-parameter SNN is, by construction, strictly slower and more energy-hungry than the equivalent ANN, because the sparsity that justifies SNNs is realized in silicon event-driven logic, not in software — the energy gap is three load-bearing mechanisms, not hand-waving. - Silicon-only sparse-MAC rebate: a silent synapse draws ~no current on neuromorphic hardware, but on a CPU every weight still executes as a full FMAC with no per-synapse power gating and no synaptic-accumulator primitive, so binarity saves memory bandwidth, not ALU work. - Temporal-coding compute multiplier: SNNs unfold over T timesteps (SpikeGPT uses 1–4 steps; SpikeLM deliberately raises firing rate), so wall-clock and joules scale ~T relative to a single forward pass. - Neuron-model overhead: LIF dynamics plus surrogate-gradient/BPTT add a fixed factor per layer. The Inherent-Redundancy consequence compounds all three — real firing density far exceeds the sparsity assumption, so more neurons are needed to match ANN accuracy, raising the energy floor further rather than lowering it ^[30]. There is no software workaround; the recent software stack (SpikeLM elastic bindings ^[31], SpikingBERT distillation ^[32], extreme quantization ^[33]) narrows the quality gap but does not change the CPU energy verdict.

For a solo builder, a free Mac CPU energy shoot-out (SpikeGPT vs RWKV/transformer at matched perplexity, measured with powermetrics) is worthwhile only to confirm the negative — it predicts no win. SNN is a bet on the neuromorphic-hardware ecosystem maturing, which the query explicitly excludes.

6. Program Synthesis and Neurosymbolic Hybrids

Program synthesis learns programs in a domain-specific language (DSL) that satisfy input-output examples; neurosymbolic hybrids combine neural perception with symbolic reasoning. DreamCoder, the canonical system, uses wake-sleep program learning to rediscover modern functional programming, vector algebra, and Newton's and Coulomb's laws, producing interpretable and transferable knowledge ^[34]. AbstractBeam improves bottom-up synthesis with library learning ^[35]; systems like LEAPS learn program-abstraction and search heuristics from language ^[36]. These methods connect to abstract-reasoning benchmarks where induction-plus-transduction hybrids show promise ^[37].

What this family wins at today is narrow deterministic reasoning, domain-specific DSLs, and exact logic with zero hallucination — a genuine frontier gap, since generative LLMs still hallucinate on exact reasoning. It sits naturally alongside the broader compact-model literature: surveys of compact language models confirm that the energy and footprint wins in this space come from architectural restraint and task scoping, not from matching frontier generation ^[38], which is exactly the niche program synthesis occupies. What breaks at scale is combinatorial search explosion on open-domain tasks; it is a specialist for exact reasoning, not a general CPU language-model substitute. There is no CPU energy measurement at frontier scale in the corpus. For a solo builder this is worthwhile only if the target task is exactly a reasoning/DSL problem; for the general "frontier-competitive on CPU" goal it is out of scope as a first bet, and the budget is better spent elsewhere. The audited absence of any measured CPU joules/token for program synthesis at useful scale is itself a finding — the paradigm is cited for capability, not for an energy demonstration.

7. Retrieval-Dominant Tiny Models

A retrieval-dominant language model keeps a small parametric model but answers by fetching from a nearest-neighbor datastore of (context, next-token) pairs. The canonical kNN-LM reaches state-of-the-art WikiText-103 perplexity of 15.79 (a 2.9-point improvement) with no extra training, and the 103M-entry datastore build is CPU-admissible at roughly 2 hours on a single CPU, ~25 minutes for 250K validation tokens over 1024 retrieved keys ^[39], ^[40]. Quality scales monotonically with datastore size up to a 1.4-trillion-token store (MassiveDS), and a smaller model augmented with a large datastore outperforms a larger LM-only model on knowledge-intensive tasks ^[41], ^[42]. Adaptive variants extend this to machine translation ^[43], and retrieval-augmented generation surveys confirm the pattern across tasks ^[44].

This is the largest admissible CPU win and it measures the real per-watt knee: the task it wins today is factual recall over a personal or domain corpus, exactly where a frontier LLM is energy-overkill. What breaks at scale is not a quality wall but a memory-bandwidth-and-footprint wall. The datastore is physically huge — 103M entries × 1024-dim × 4 bytes ≈ 412 GB uncompressed — and already exceeds CPU RAM at 100M tokens, forcing FAISS compression (IVF+PQ) or NVMe spilling. The wall is auditable from the physics: product-quantized keys cost ~32–64 bytes each, so 1B keys ≈ 32–64 GB and 1.4T tokens ≈ ~90 TB, and the index cannot live in CPU RAM beyond a few hundred million keys. The real cost is per-query bandwidth, not capacity — at 1B keys, touching even 1% of a 32-byte/key index is 10M × 32 B = 320 MB per query, ≈32 GB/s at 100 queries/s, which exceeds the DRAM budget (~50–100 GB/s) well before the tiny LM's compute saturates, while NVMe random-access (3–7 GB/s sequential) is far worse.

For a 24 GB Mac the per-watt crossover lands around 10⁸–10⁹ compressed keys (confidence ≈0.5 on the exact ceiling, since the corpus gives only latency, not a measured per-watt crossover with named hardware); beyond it a denser parametric model (the excluded ternary transformer) becomes per-watt superior. Critically, no paper reports energy-per-token as a function of database size on CPU for any retrieval-dominant model — an audited absence.

For a solo builder the cost is ~$100 of GPU to embed and index the corpus; the CPU-side measurement (FAISS IVF+PQ, powermetrics energy-per-token) is free and is the decisive experiment. The builder should test the bandwidth knee, not assume unlimited scaling — because the quality curve and the per-watt curve are different curves, and the corpus measures the former, not the latter.

8. Cellular Automata and Neural Cellular Automata

A neural cellular automaton (NCA) learns a tiny local update rule (≈8.3K parameters) applied per-cell over a 3×3 neighborhood, asynchronously and parallelizable across the grid — no global weights, no attention. This makes it absurdly cheap per cell: a single cell update is about 10K multiply-add operations and does not need to be synchronized across the grid ^[45], ^[46].

What it wins at today is compact spatial/pattern generation, regeneration, and image restoration — morphogenesis tasks where local rules suffice. What breaks at scale is the sequence/symbolic reasoning wall; NCA struggle with ordered, discrete symbol manipulation and are a morphology specialist, not a language-model path. There is no CPU energy measurement at frontier scale in the corpus (only the ~8.3K-param footprint and async-update property), and for the framed goal it is not a first bet. The audited absence of any measured CPU joules/token for NCA at useful scale is a finding, not a gap — the paradigm is a specialist whose CPU cost is plausibly tiny but unverified.

9. Forward-Forward and Non-Backprop Training

"Non-backprop" covers a family of learning rules that avoid the backward pass: Forward-Forward (FF) replaces forward/backward with two forward passes (positive/negative); Equilibrium Propagation (EP) learns energy-based models with one neural phase and one nudging phase; Direct Feedback Alignment (DFA) trains deep nets with random feedback weights. They are motivated by biological plausibility and the hope of local, hardware-cheaper on-device learning.

Measured CPU quality-vs-energy is essentially absent. Hinton's own Forward-Forward paper is "preliminary," demonstrated "on a few small problems" (image data, no language, no CPU energy) ^[47]. The Predictive Forward-Forward variant and emergent sparse cortical-like representations "work as well as backprop" only on small images ^[48], ^[49]. The decisive counter: the emergent sparse representations "can emerge in networks trained with Backpropagation on the same objective," so FF is not a unique quality win ^[49]. The one concrete non-backprop quality floor is Direct Feedback Alignment, which reaches 1.45% error on permutation-invariant MNIST (with dropout) — explicitly "almost as good" as backprop, not better ^[50]. Equilibrium Propagation computes correct gradients via two phases (free + target-nudged) that require settling to equilibrium — more serial compute than one forward + one backward pass, so EP carries the same fixed-point convergence cost as EBMs and is not an inference-cost win ^[26], ^[50].

What breaks at scale is the absence itself: every corpus source fails the hardware+task+metric norm for CPU energy, and none reports non-backprop beating backprop on a useful (let alone language) task. For a solo builder the ~$100 GPU is better spent on the three admissible bets; non-backprop remains a research curiosity for the framed goal. The audited absence of any measured commodity-CPU joules/token or useful-task quality win for non-backprop training is a central negative finding — and notably, it is an absence of measurement, not proof of impossibility, which is exactly why the builder's own falsification experiment matters.

10. Analog and In-Memory-Inspired Algorithms in Software

"Analog-in-software" covers algorithms that mimic analog or in-memory computing (IMC) — such as matmul-free language modeling and reservoir-computing variants — inside ordinary software. The flagship "Scalable MatMul-free LM" reaches 2.7B parameters with >10× inference-memory savings and a "4× throughput / 10× less energy" claim ^[51]. The catching fact: that efficiency is built on the ternary BitNet b1.58 backbone — which is exactly the excluded transformer+quantization baseline. The 10× energy figure is on a multi-chip neuromorphic system (not a CPU measurement) ^[51]. The genuinely CPU-attributable MatMul-free speedup is bitnet.cpp at 2.37×–6.17× x86 — which is the excluded line, period ^[1], ^[2].

The reservoir-computing variant shows real but marginal gains — −19% parameters, −9.9% training time, −8.0% inference time — sitting on top of the same excluded ternary backbone ^[52]. In-memory HDC computes in-place on 760,000 phase-change memory devices — genuine analog hardware — but that advantage is lost on a von Neumann CPU, where it degenerates to integer XOR/binding ^[53].

What breaks at scale is the rebrand wall: the family is mostly the excluded ternary transformer, and no analog member reaches CPU language-model scale. For a solo builder, reproducing bitnet.cpp as a control is free and worthwhile — as a baseline to beat, not as a recommendation. The one legitimate experiment is a reservoir-vs-bitnet.cpp A/B on the Mac (~$100 GPU) to settle whether the residual variant is a distinct efficiency axis; the corpus says it is not.

11. Extreme Contextual and Dynamic Sparsity

Dynamic activation sparsity predicts, per input, which subset of neurons/attention heads to activate, so only 20–30% of the network runs per token. Deja Vu cuts OPT-175B inference latency >2× vs FasterTransformer and >6× vs Hugging Face, with no quality loss ^[54]. PowerInfer shows consumer-grade serving gains ^[55]; ProSparse and SVD sparsity predictors push intrinsic activation sparsity ^[56], ^[57]; the Sparsity Law characterizes the scaling behavior ^[58], and universal sparsity properties are now documented ^[?].

This is real and CPU-relevant — immediate latency/energy gains without changing quality — but it sits on the excluded transformer backbone and requires custom execution kernels. The measurement is GPU latency, not CPU joules, and no admissible CPU joules-per-token number names hardware+task+metric for the sparsity-only win. Because the sparsity gain is already captured by the excluded ternary-transformer baseline, it is out of scope as a recommendation and appears here only as the clear demonstration that the excluded line is where the real CPU efficiency lives. The audited absence: no CPU joules/token measurement isolates the sparsity-only contribution from the transformer it modifies.

12. Distillation to Symbolic Rules

Symbolic distillation extracts a neural network's behavior into decision trees, propositional logic rules, or state machines. The result is 100% interpretable with zero-latency CPU inference for narrow tabular/rule tasks, and methods range from eclectic rule extraction to systematic surveys of feedforward rule-extraction techniques ^[59], ^[60], ^[61]. Tsetlin Machines can themselves implement knowledge distillation into interpretable rule sets ^[11].

What it wins at today is narrow rule/decision tasks where auditability is the product — a frontier-grade property (guaranteed-sane reasoning) that no black-box LLM offers, especially in regulated settings. What breaks at scale is the fidelity wall: distilled rules drop sharply in accuracy on complex multimodal or language reasoning, and no symbolic-distilled system in the corpus matches a frontier LLM on a language/reasoning benchmark. There is no CPU energy measurement at frontier scale. It is a trustworthy specialist, not a path to frontier language quality, and not a first bet for the framed goal. The audited absence: no corpus system demonstrates symbolic distillation matching frontier LLM quality on a language/reasoning benchmark with a CPU energy measurement.

13. Cross-Paradigm Comparison

The table below consolidates the four fixed fields per paradigm and adds the column the evidence actually demands: the CPU energy-measurement status. A measurement counts only if it names hardware + task + metric; simulated/neuromorphic/FPGA numbers are flagged and never reported as CPU measurements.

Paradigm	Measured CPU quality-vs-energy (named task)	Task it wins today	What breaks at scale (wall type)	Solo-builder test cost	CPU Energy Measurement Status
Tsetlin Machines	90.15% IMDb, 50 KB, 34.5 M pred/s (throughput, not joules)	Interpretable text/tabular classification	Semantic wall (needs LM hybrid)	Free (Tests 1+3); ~$30 GPU optional	None — audited absence of stock-CPU joules
HDC/VSA + NysHD	6.18× mem, 4.62× train / 3.84× test; NysHD +11% graph / +17% string, 52% faster than GCN	Structured/relational classification	Capacity wall (provable, mitigable only for structured tasks)	Free repro; $100 GPU baseline	Latency/speed only (no joules)
Retrieval-dominant	15.79 WikiText-103 PPL (−2.9); ~2h/CPU build	Small-corpus factual recall	Bandwidth wall (~10⁸–10⁹ keys)	~$100 GPU embed+index	None — audited absence of joules/DB-size
EBM / Hopfield	Structured-class SOTA; attention-equivalent layer	Structured classification, OOD	Convergence wall (MCMC cost)	Cheap probe only	None at LLM scale
Spiking NN	None admissible on commodity CPU	—	Hardware wall (CPU strictly slower/hungrier)	Free shoot-out (predicts negative)	Neuromorphic/FPGA only — not a CPU measurement
Program synthesis	None at scale	Narrow exact reasoning, zero hallucination	Search wall (combinatorial)	N/A unless DSL task	None
Cellular automata / NCA	~8.3K-param async rule	Spatial/pattern, regeneration	Sequence/symbolic wall	Cheap; not a first bet	None
Forward-Forward / non-backprop	Small-vision only; no CPU joules	—	Blind-spot wall (unmeasured)	~$100 better elsewhere	None
Analog / MatMul-free	`bitnet.cpp` 2.37×–6.17× x86 = excluded baseline	—	Rebrand wall (excluded ternary)	Free control + ~$100 A/B	10× on neuromorphic hardware — not CPU; baseline excluded
Extreme sparsity	GPU latency >2× (no quality loss)	—	Transformer-retrofit wall	Out of scope	None admissible
Symbolic distillation	Zero-latency on rule tasks	Narrow rule/audit tasks	Fidelity wall	Cheap; not a first bet	None

The table exposes the central methodological finding. Five of the eleven paradigms have no stock-CPU joules-per-token measurement at useful scale; the measurements that exist are either simulated/neuromorphic/FPGA (not CPU measurements), the excluded ternary-transformer baseline, latency-only, or absent. The three admissible paradigms (TM, HDC+NysHD, retrieval-dominant) carry admissible quality wins on named CPU tasks with throughput/latency metrics, but even they lack a stock-CPU joules/token number — the blind spot hits the winners too. So the verdict almost nothing beats the excluded ternary transformer on CPU quality-per-joule is itself under-measured: we lack the energy curves to prove it, which is precisely why the solo builder's first experiment is to measure, not to assume.

The wall taxonomy in the final column is the practical takeaway. Each paradigm fails at scale for a different reason — distinct failure modes, not the same problem at larger scale:

Semantic (TM)
Capacity (HDC)
Bandwidth (retrieval)
Convergence (EBM)
Hardware (SNN)
Search (program synthesis)
Sequence (NCA)
Measurement (non-backprop)
Rebrand (MatMul-free)
Retrofit (extreme sparsity)
Fidelity (symbolic distillation) These are not the same problem at different scales; they are distinct failure modes that require distinct breakthroughs. A solo builder choosing "what to test first" is implicitly choosing which wall they are willing to hit, and the decision tree in the next section matches each task profile to the wall it can tolerate.

14. What a Solo Builder Should Test First

The recommendation resolves into a decision tree keyed to the builder's task profile and which wall they are willing to hit — because the six "test-first" experiments probe incommensurable hypotheses and the $100 / 24 GB envelope cannot run them all as leads.

Real-time interpretable classification on a tiny footprint → test Tsetlin Machines first (Tests 1+3 free, decisive in hours). Wall hit: semantic.
Factual retrieval over a personal/domain corpus ≤10⁸ tokens (≈10⁸ compressed keys; the 10⁹ figure is the upper tail) → test retrieval-dominant first (~$100 GPU embed+index; measures the real per-watt knee). Wall hit: bandwidth.
Graph/string/relational classification on CPU → test HDC + NysHD first (free reproduction, then $100 baseline). Wall hit: capacity.
Falsify the "analog-in-software rebrand" claim → test MatMul-free reservoir vs bitnet.cpp control (~$100 GPU). Wall hit: rebrand.
Measure the hardware gap → test SNN CPU energy shoot-out (free on Mac, predicts negative). Wall hit: hardware.
Do NOT lead with non-backprop training — the evidence says spend the GPU credit elsewhere.

Test 0 — the control every other test must beat: reproduce the excluded ternary-transformer baseline, bitnet.cpp on the Mac CPU with powermetrics, so every admissible paradigm is measured against the actual CPU quality-per-joule frontier ^[1].

Committed recommendation — one first bet, one second bet:

FIRST BET: retrieval-dominant tiny model over a small in-RAM personal/domain corpus (≤10⁸ tokens, ≈10⁸ compressed keys with a 10⁹ upper tail). It is the largest admissible win, the only experiment that measures the real per-watt knee, and it fits the $100 envelope exactly. Start with a domain corpus (codebase, notes, documentation), embed with a small model on GPU, build a FAISS IVF+PQ index, and measure quality-vs-energy-per-token on the Mac. The bandwidth wall is where the finding lives.
SECOND BET: Tsetlin Machines for interpretable real-time classification. Free to reproduce, decisive in hours, unbeatable for the interpretability niche, and the cleanest demonstration that a 50 KB model can match classical-ML parity where a frontier LLM is overkill.
THIRD PROBE: HDC + NysHD for graph/string/relational classification — free to reproduce, then a $100 baseline comparison.
Explicitly NOT first bets: SNN (hardware wall), MatMul-free/analog (rebrand wall), non-backprop (unmeasured). Run them as free/cheap falsification experiments against the bitnet.cpp control, not as leads.

Opinionated Synthesis

The two opposing readings of this corpus — only three paradigms win, the rest are traps versus the verdict is premature, the evidence norm is too strict — are not really in conflict. They disagree about rhetoric, not evidence. The first reading is correct that only three paradigms (Tsetlin Machines, HDC+NysHD, retrieval-dominant) carry admissible CPU evidence on a named narrow task, and that spiking neural networks, MatMul-free/analog-in-software, and non-backprop training fail the evidence norm by hardware-dependence, rebranding, or non-measurement. The second reading is correct that the failure is often a tooling and measurement gap rather than a physics verdict, and that the field-wide energy blind spot hits the "admissible three" as much as the "traps" — none has a stock-CPU joules-per-token number either. The synthesis that survives both is that the right paradigm depends on the builder's task profile and which wall they are willing to hit: semantic (TM), bandwidth (retrieval), capacity (HDC), hardware (SNN), rebrand (MatMul-free).

The practical upshot for a solo builder with a 24 GB Mac and ~$100 of GPU credit is unambiguous. Lead with retrieval-dominant (first bet) and Tsetlin Machines (second bet), treat HDC+NysHD as the third probe, run SNN / MatMul-free / non-backprop as falsification experiments against the bitnet.cpp control, and — above all — measure CPU joules-per-token themselves, because the literature will not do it for them. Beating frontier on a low-power CPU is real but named, narrow, and wall-bound; the three admissible paradigms are where to look, the excluded ternary transformer is the yardstick they must clear, and the published energy numbers are the gap to fill, not the proof to trust.

Looking forward, the most under-exploited opportunity is not a new architecture but a measurement discipline. The corpus shows a systematic absentee: nearly every paradigm reports latency and quality, almost none reports energy on a named commodity CPU. A solo builder who sets up a reproducible powermetrics harness — ideally anchored to a standardized edge benchmark such as MLPerf Tiny so results are comparable to the wider field ^[62] — and publishes joules-per-token curves for even two or three of these paradigms on a 24 GB Mac would produce evidence the field currently lacks. That builder would also be the first to know, concretely, whether any non-excluded approach actually crosses the excluded ternary-transformer's quality-per-joule line on a useful task. That is the experiment the whole survey points toward, and it is one a single person can run this quarter. The honest frontier of "frontier-competitive CPU AI" is therefore less a question of which exotic paradigm wins and more a question of who will first bother to measure the one number that settles it.