📊 Full opportunity report: Engineering Is Automated. Research Is the Residual. on ThorstenMeyerAI.com — validation score, market gap, and execution plan.
TL;DR
Recent benchmarks indicate AI systems are approaching full automation in core engineering tasks for AI research. However, the human element in research persists, raising questions about the future role of human researchers.
Recent developments confirm that AI systems are now capable of automating the core engineering tasks involved in AI research, with some benchmarks approaching full saturation. This shift significantly impacts how AI research is conducted and who conducts it, marking a potential turning point in the field.
Multiple AI benchmarks, including CORE-Bench and MLE-Bench, have shown rapid progress, with AI systems achieving near-complete automation of tasks such as reproducing research experiments and competing in Kaggle competitions. CORE-Bench, which measures research reproduction, reached 95.5% in December 2025, with the benchmark’s author stating it was ‘solved.’ Similarly, AI agents now perform at roughly two-thirds of human-level on Kaggle competitions, a level considered competitive with mid-tier human practitioners. These advances suggest that the bottleneck for research reproduction has shifted from ‘can it be reproduced’ to ‘should it be reproduced,’ as the marginal cost for AI to handle this work drops significantly.
Meanwhile, progress in kernel design—another critical aspect of AI engineering—has been documented through numerous research papers demonstrating advances in GPU kernel optimization, automated code conversion, and production-grade model deployment tools. These developments indicate that engineering tasks in AI R&D are increasingly handled by automated systems, reducing the need for human intervention in routine engineering work.
Engineering is automated.
Research is the residual.
Six skill benchmarks. Edison’s framing. The question Clark leaves open is whether research is just engineering at scale.
Jack Clark’s Import AI #455 catalogs six benchmarks measuring AI capability on AI R&D tasks and concludes “AI can today automate vast swatches, perhaps the entirety, of AI engineering.” The residual question is research. The structural read on the residual: it may not be a permanent moat.
Six skills. One trajectory.
Clark catalogs six benchmarks measuring AI capability on AI R&D-relevant tasks. Each individual benchmark could be noise. Six benchmarks moving together is a curve. The pattern is the cascade observed across the broader Clark series — visible here in the specific R&D-skill domain.
AI Tools for Finance and Accounting Professionals: Automate Tasks, Save Hours, Work Smarter
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Three data points. Mixed signal.
Clark provides three data points on the creative-spark question. Yes-evidence: Erdős-1051, centaur math discovery, sporadic Move-37-style moments. No-evidence: low yield, framing dependence, absence of acceleration. The mixed signal is the honest read.
The data supports two readings. Pessimistic: rare moments suggest creative insight is qualitatively distinct from engineering work. Optimistic: rare moments are an artifact of low-volume exploration; more shots on goal yields more discoveries. Both readings are consistent with Clark’s “vast swatches, perhaps the entirety” claim. They differ on the residual.
GPU-Accelerated Computing with Python 3 and CUDA: From low-level kernels to real-world applications in scientific computing and machine learning
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Five dimensions Clark gestures at but leaves underdeveloped.
Clark’s section is rigorous on the empirical evidence. Five strategic dimensions matter for the institutional response that the Clark series synthesis argues is structurally inadequate.
Unified AI and Machine Learning with Microsoft Fabric: From Lakehouse to Model Deployment for Scalable Enterprise Systems
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Two readings. Different equilibria.
The structural question Clark leaves open: is research a permanent moat that bounds automated AI R&D, or is it engineering at scale that dissolves with more shots on goal? Both readings are consistent with the current data. They differ by orders of magnitude in consequences.
Productivity multiplier years
Recursive loop operational
![WavePad Audio Editing Software - Professional Audio and Music Editor for Anyone [Download]](https://m.media-amazon.com/images/I/B1fcLEGCs6S._SL500_.png)
WavePad Audio Editing Software – Professional Audio and Music Editor for Anyone [Download]
Full-featured professional audio and music editor that lets you record and edit music, voice and other audio recordings
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Five audiences. Asymmetric cost of being wrong.
The institutional response should not bet on inspiration being a permanent moat. If the distinction holds, capacity built is still useful. If it closes, capacity is necessary. Asymmetric cost-of-being-wrong points toward building now.
IN INDUSTRY
IN ACADEMIA
POLICYMAKERS
INVESTORS
EVERYONE ELSE
Engineering is automated. The residual is the question. The institutional response should not bet on inspiration being a permanent moat.
Implications of Automation for AI Research Workforce
The automation of engineering tasks in AI research could reshape the research workforce, reducing the need for human engineers in routine tasks while shifting the focus toward higher-level research questions. This could accelerate innovation but also raises concerns about the future roles of human researchers and the potential for AI to handle more of the research process itself.
Recent Benchmarks Show Rapid Progress in AI Engineering Capabilities
Over the past 18 months, multiple benchmarks have tracked AI progress across different domains. CORE-Bench, measuring research reproduction, improved from 21.5% in September 2024 to 95.5% in December 2025. MLE-Bench, assessing Kaggle competition performance, advanced from 16.9% in October 2024 to 64.4% in February 2026. Concurrently, research papers have documented advances in kernel design, automation, and infrastructure optimization, signaling a broad trend toward automation in AI engineering tasks. This pattern suggests a nearing saturation point where AI can handle most engineering aspects of research, while the residual human role centers on novel, high-level research questions.
“The pattern across these benchmarks indicates that AI systems are approaching full automation in core engineering tasks for AI R&D.”
— Thorsten Meyer
Unresolved Questions About the Limits of Automation
It remains unclear how much of the entire research process—beyond engineering tasks—can be automated. While engineering is largely automated, the extent to which AI can conduct high-level research, generate novel hypotheses, and drive scientific breakthroughs without human input is still uncertain. Additionally, the structural question of whether research itself is a form of engineering at scale is open for debate.
Next Steps in AI Research Automation and Human Role
The next 32 months will likely see continued advances in automation capabilities, with benchmarks pushing toward saturation. Researchers and institutions will need to decide how to integrate these tools into their workflows, possibly shifting the human role toward high-level conceptual work. Monitoring these developments will be crucial to understanding the evolving landscape of AI research and its societal implications.
Key Questions
What does automation of AI engineering mean for human researchers?
It suggests that many routine engineering tasks may soon be handled by AI, potentially reducing the need for human labor in these areas and shifting human focus toward high-level research and innovation.
Are there limits to what AI can automate in AI research?
While engineering tasks are nearing full automation, the capacity of AI to autonomously conduct high-level research, generate hypotheses, and make scientific breakthroughs remains uncertain.
How reliable are current benchmarks in measuring AI research capabilities?
Recent benchmarks like CORE-Bench and MLE-Bench show significant progress, but they primarily measure specific tasks. Their ability to fully capture the scope of research automation is still limited, and some aspects of research are not yet fully quantifiable.
What are the potential risks of automating AI research tasks?
Potential risks include over-reliance on AI for critical research decisions, reduced human oversight, and ethical concerns about the direction and control of autonomous research processes.
Source: ThorstenMeyerAI.com
