When Small LLMs Beat the Swarm: How Splitting Data Into Sections Gave a Weak Model a 4x Boost

Why small models are needed, but not in swarms

A department head at a design organization with 14 years of experience spent the spring trying to build an emergent swarm out of weak local models. The experiment was called Mycelium — inspired by work on dissipative structures and populations of models that distribute roles and negotiate a shared language on their own. But the swarm never took off: on 200 tasks from MBPP it solved 29%, versus 52% for plain best-of-N, where models simply get several attempts. Along the way, though, he discovered that one idea borrowed from design-bureau practice carries a colossal load.

All experiments ran on a GTX 1660 SUPER with 6 GB of memory. The author was forced to use genuinely tiny models: qwen2.5-coder-1.5b, qwen3-1.7b, llama-3.2-1b, internvl3-2b, gemma-3-1b, and smollm2-1.7b. Memory constraints meant keeping several models loaded at once and running them through thousands of calls. AI agents wrote all the experiment code; the author set the tasks and scrutinized the numbers.

A design-bureau structure instead of a swarm

The idea came from the story of Albert Kahn, the American architect who designed factories for Ford. Between 1929 and 1932, his firm designed around 500 factories for the USSR, including the Stalingrad Tractor Plant. Soviet design institutes are said to have grown out of his Moscow bureau. The author lives inside that organizational structure every day.

The chief project engineer splits an object into sections: heating and ventilation, electrical, construction, automation — each handled by its own department. A finished section is checked within its department (an intra-discipline review). Then sections are cross-checked against each other at the interfaces (an inter-discipline review) — one department has a pump, the construction team has to put a foundation under it, and the two have to match.

Disputed points go to the chief specialist. At the end, format control checks the paperwork, and only then do the drawing sets go to the client. No one holds the whole plant in their head — it simply doesn't fit.

A small model given 18 records and asked to find the right ones is in the same position.

What was tested

Across 40 test tasks, the models had to find all transactions matching a given category AND region within a set of 18 records. Between 2 and 5 records matched, with 2–4 decoys nearby — records with the right category but the wrong region, and vice versa. The model had to output ONLY a comma-separated list of matching IDs, with no explanations, and name exactly the right set — no extras, no omissions.

Models used, in GGUF format: qwen2.5-coder-1.5b-instruct (Q4_0), llama-3.2-1b-instruct (Q4_0), qwen3-1.7b (Q4_0), internvl3-2b (Q4_K_M), gemma-3-1b-it (Q5_K_S), smollm2-1.7b-instruct (Q4_K_M). Answers for the full list were pulled from disk from earlier runs to eliminate generation variance. In total, 720 calls, 614 seconds on the GTX 1660 SUPER.

Results: splitting into sections

Even without splitting, the poor distribution across positions was already visible: models found the first record 61% of the time, the last one only 21% of the time. The author initially assumed that toward the end of the list the model simply says "no" more often, but checking proved him wrong. By the end, the model becomes only slightly stingier (0.21 versus 0.50 at the start), but it distinguishes similar rows far worse. It finds fewer correct matches (0.30 versus 0.50), and false positives among non-matching records increase (0.16 versus 0.14).

The 18 records were then cut by a script into 3 blocks of 6 consecutive records each — order within a block was preserved. Each of the six models read each block separately. If a model named an ID that wasn't in its block, that counted as a hallucination, and the code discarded such an answer (only 5 hallucinations were caught across the entire run). A vote for a record only counted from the block it actually belonged to.

Results for exact match (the model named exactly the right set):

| Model | Without splitting | With blocks of 6 | Gain | |--------|-----------|-----------------|----------| | qwen2.5-coder-1.5b | 17.5% | 70% | +52.5 pts | | llama-3.2-1b | 40% | 72.5% | +32.5 pts | | internvl3-2b | 57.5% | 82.5% | +25 pts | | qwen3-1.7b | 72.5% | 85% | +12.5 pts | | gemma-3-1b | 0% | 5% | +5 pts | | smollm2-1.7b | 0% | 0% | 0 pts |

The weaker the model, the bigger the gain. The gap between the start and end of the list shrank from 20 points to 2 points. On the full list, recall at the end was 0.30; with splitting, it rose to 0.57.

The myth of emergence

The main goal was to find emergence — a case where the combined answer is correct even though no single model got it right on its own. Under the original, flawed comparison, that happened on 34 of 40 tasks. The author nearly opened the champagne, then looked more closely at what was being compared to what. On one side was the whole pipeline: blocks, review, voting, assembly. On the other was a single model given all 18 records at once. The conditions differ, so that comparison means nothing.

Under a fair comparison (running the best model, qwen3-1.7b, through the same block-based pipeline), it scores 0.850. All six models together score 0.900. An oracle that counts a success if "at least one of the six got it right" scores 0.975. Strict emergence came out at zero: the combined answer never went beyond what at least one model could already do on its own. The swarm added just 5 points over the best single model.

Out of seven design-bureau elements, only one worked

Only the split into sections carried any real weight. What each element contributed: the chief engineer splitting the object into sections (dividing 18 records into 3 blocks of 6) — nearly everything: the gap went from 0.20 to 0.

02, exact-match accuracy went from 0.75 to 0.90.

The executors (6 models, one per block) — +0.05 on top of the best single model. Intra-discipline review (an ID not from its own block gets rejected) — caught 5 hallucinations.

Inter-discipline review (cross-checking at block interfaces) — blocks are independent, so there was nothing to cross-check. Format control (checking output format) — no errors found. The chief specialist (a targeted question about a single row) — not needed.

Revisions (another pass when the answer is uncertain) — not needed.

The practical takeaway: cut small models' input into pieces when feeding them a batch of documents or a long list. In this experiment, the optimum was 6 records per block — the first 6 positions were found most reliably. Splitting by tokens is almost free: three chunks of a third each give the same total volume, plus three repeats of the prompt header. Three times as many calls, but each one is shorter. A swarm of six models on top of that costs six times as much and gains 5 points. The author doesn't endorse that trade-off, but the tool itself — splitting into sections — works.