What must a model preserve? Synthetic tests toward evaluating artificial knowledge [Experimental Publication]

Abstract

Scientific knowledge produced through models depends not merely on obtaining an output, but on what a representation preserves and which inferences that preservation warrants. This problem becomes especially consequential when artificial systems generate predictions, transformations, or scholarly claims that may appear knowledgeable before their relation to a target has been tested. Starting from Rosen's modelling relation, this article distinguishes predictive, structural, and purpose-relative adequacy. Bounded synthetic studies provide background evidence for the distinction: a declared predictive rule accepted a same-family case, rejected an obvious misspecification, and preserved an invalid evaluation as undefined; coarse states concealed different successors and paths; and richer representations recovered different information at different descriptive costs. These studies test representational predicates, not an AI knowledge-production system. Rather than treating the local results as general validation, the article uses them to formulate three prospective experiments with explicit failure conditions. It then extends the question tentatively to scholarly production itself: whether transformations among research records, AI-produced articles, reviews, and revisions preserve claims, evidential relations, uncertainty, and declared purpose. The proposals position the synthetic results as methodological steps toward evaluating artificial knowledge and, later, iterative and potentially scaled AI-supported science. Correspondence is not presented as sufficient for knowledge, purpose-relative adequacy remains untested, and increased throughput is not equated with epistemic progress.

Abstract source