Hybrid Paper Review Architecture Comparison — Run 001

Status · INCONCLUSIVE

Completion date

July 23, 2026

Research program

Review, Verification, and Accountability

Research question

Can a source-gated, two-call model review produce a more useful and evidence-grounded review of a large argument than the existing one-pass and four-stage alternatives?

Initial hypothesis

A hybrid architecture might preserve the simplicity of a short review pipeline while gaining some of the evidence discipline of a longer one. That was a provisional theory, not a prediction treated as fact.

Method

A development-only two-stage review was built. A deterministic source gate first bounded the available evidence. One model call produced a structured evidence map. A second call received the complete bounded source and the validated map, then produced an editorial review. Runtime checks rejected malformed structures, unknown evidence references, unsupported categories, and authority overreach.

The hybrid was then compared with the existing one-pass review and a prior four-stage review using the same 21,224-character extraction from one six-page technical paper, the same model, no tools, no provider retries, and comparable final-output limits.

Tests performed

Fifteen focused tests of the hybrid contracts and validation behavior.
The full 686-test suite, with one pre-existing optional live test skipped and no failures.
Lint, type checking, production build, and diff validation.
A controlled live comparison of the one-pass, two-call hybrid, and four-stage review lanes.
Public-file hash checks confirming that the existing public Review Lab route, interface, and prompt remained unchanged.

Observations

The two-call architecture completed after one development timeout was corrected.
The completed hybrid run validated 99 evidence references and found zero unknown references.
All three controlled lanes completed without provider refusal or in-lane retry.
The one-pass lane used one call and 5,621 total tokens; the hybrid used two calls and 15,262 total tokens; the four-stage lane used four calls and 26,671 total tokens.
The hybrid took longer and used more tokens than the one-pass review, but less than the four-stage review.
The hybrid returned a state ready for human assessment. It did not select a winner or revise the source.

Supporting evidence

The bounded source gate and both model stages executed as designed.
Evidence-reference validation accepted 99 references and rejected none as unknown in the completed run.
The same source extraction, model, tool restrictions, retry rule, and comparable output limit were used across the controlled lanes.
The completed comparison route returned a valid response.
Focused and full automated tests passed.
Lint, type checking, production build, and diff checks passed.
Recorded hashes showed that the public Review Lab files were not changed by the experiment.

Counterevidence and constraints

The first live attempt failed when the editorial stage exceeded a 60-second development timeout.
Only one source paper completed every comparison lane.
The intended three-paper benchmark was not completed.
No human scores were collected from the blinded comparison packet.
No independent replication was performed.
The hybrid was materially slower and more token-intensive than the one-pass review.

Failures and revisions

The first live run returned an error because the hybrid editorial call exceeded the development route’s timeout. The timeout for the two experimental routes was increased from 60 to 120 seconds. No provider retry, output rewrite, or hidden repair was added inside a comparison lane. The second controlled run completed.

Two candidate documents were rejected because they were derivative one-page reports rather than source papers. That preserved the evidence standard, but it also left the planned benchmark incomplete.

Final conclusion

The experiment does not support the claim that the hybrid review is better than the alternatives. It establishes a narrower result: the isolated two-call architecture can execute, validate source references, render a review, and participate in a controlled comparison.

The comparative hypothesis remains inconclusive because one paper and an unscored comparison cannot establish review quality, consistency, or general usefulness.

Confidence

High confidence in the recorded implementation and execution findings. Low confidence in any claim of comparative superiority.

Limits

The experiment tested architecture and validation behavior, not independent truth verification. A second model can repeat the assumptions of the first. The source set was too small, and no blinded human assessment was completed. The result should not be generalized to medical, legal, academic, or other high-stakes review.

Unresolved questions

Would blinded human reviewers prefer the hybrid output, and on which declared criteria?
Would the evidence-reference behavior remain stable across different kinds and qualities of papers?
Does the additional latency and token use produce a measurable editorial benefit?
Which findings remain stable across models and prompt variations?

Public project records

Private source packets, local development paths, hidden comparison mappings, and unpublished material are intentionally not included in this public record.