Close Menu
techwavedispatch.com

    Subscribe to Updates

    Get the latest creative news from FooBar about art, design and business.

    What's Hot

    A Framework To Detect and Reduce Proxy Capacity

    November 20, 2026

    The most misunderstood graph in AI just got a major update

    November 20, 2026

    OpenAI Brings Its Collaborative Space Hubs To Plus Users

    November 20, 2026
    Facebook X (Twitter) Instagram
    Trending
    • A Framework To Detect and Reduce Proxy Capacity
    • The most misunderstood graph in AI just got a major update
    • OpenAI Brings Its Collaborative Space Hubs To Plus Users
    • Adaptive-View Self-Distillation by Balancing Consensus and Teacher-Specific Privileged Signals
    • Internet Archive Staffers Are Unionizing
    • Improving Imitation Policies for Multi-Phase Tasks through Residual Learning Allocation
    • Nothing’s AI Device Debuts December 8, And It Looks Like Two Vapes Stuck Together
    • Why We Should Rethink the Sequential SFT-RL Paradigm
    Facebook X (Twitter) Instagram
    techwavedispatch.com
    Subscribe
    Friday, November 20
    • Home
    • AI Tools & Software
    • Machine Learning
    • Generative AI
    • Robotics & Automation
    • Future Tech & Hardware
    • AI in Business
    • Ethics & Regulations
    • More
      • Gadgets & Smart Devices
      • Cybersecurity & AI
      • Tech Breakthroughs
    techwavedispatch.com
    Home»Tech Breakthroughs»The most misunderstood graph in AI just got a major update
    Tech Breakthroughs

    The most misunderstood graph in AI just got a major update

    adminBy adminNovember 20, 2026No Comments3 Mins Read
    Facebook Twitter Pinterest LinkedIn Tumblr Email
    The most misunderstood graph in AI just got a major update
    Share
    Facebook Twitter LinkedIn Pinterest Email

    Even if the models do lag behind human performance on open-ended tasks right now, the new report suggests that they might be getting better. Some of the newest models, such as Opus 5.5, tend to achieve what METR calls “borderline above the bar” performance on the evaluation design tasks—not yet usable for METR’s research, but promising. Just a year ago, every model was scoring in the “clearly below the bar” category.

    But because these open-ended and judgment-heavy tasks are so tough to design and score, METR includes few of them in its report, and that makes it difficult to predict whether and how models might continue to progress in these areas. “The improvement we observe, I think that does tell us something for the current moment,” says Najoung Kim, a linguist and computer scientist at Boston University. “I’m not sure just yet how to think about how it would extrapolate.”

    Matt Putz, a member of the technical staff at METR, says that predicting progress on these kinds of tasks is likely to remain difficult. “By their nature, hard-to-verify or judgment-laden tasks are much harder to study, and especially to quantify,” he said in an email. “Ultimately, I don’t think we’ll be able to establish trends as clean and robust as for the easy-to-specify [tasks].”

    In general, METR is clearly butting up against the practical difficulties of testing AI performance on difficult and underspecified tasks—precisely the tasks most relevant to AI R&D automation. METR’s time horizon plots score task difficulty by evaluating how long it would take a human to complete the task, and for the original plot they measured this quantity by actually having people try each task. 

    In the new report, however, the completion times are mostly estimates, because paying people to complete tasks that might take hundreds of hours, or that only a few experts are qualified to complete, is prohibitively expensive and time-consuming. Other tasks have more specific shortcomings—for example, for the philosophical argument task, just one person rated most of the arguments, and agents are judged by how well they agree with that particular individual. It’s possible that the scores would change if more experts were surveyed—but that, too, is time consuming.

    It’s still early days for evaluating models on tasks that are hard to validate or require judgment. And while the experts I spoke with see the inclusion of these tasks in METR’s newest time horizon plot as a clear step in the right direction, they agree that we are still far from achieving any definitive answers about whether, and how quickly, automated AI R&D might be coming. “This move towards open-ended tasks is promising,” Kapoor says. But, he adds, “there is just such a lot more to be done to actually get a fuller understanding.”

    Graph Major misunderstood Update
    Share. Facebook Twitter Pinterest LinkedIn Tumblr Email
    admin
    • Website

    Related Posts

    Valve says it might have to raise the price of supply-limited Steam Machines

    November 20, 2026

    How FAA can modernize US national airspace system without emergency ground stops

    November 20, 2026

    Scone: Symmetry-aware Neural Combinatorial Optimization for Graph Coloring

    November 20, 2026
    Leave A Reply Cancel Reply

    Top Posts

    NVIDIA Is Investigating Reports That CEO Jensen Huang Was The Target Of Chinese Spy Efforts

    November 8, 202619 Views

    ADIndex: Layer-Adaptive Sparse Attention Indexing via Group-Level Distillation

    November 11, 20265 Views

    Anthropic releases new Mythos and Fable models amid RSI concerns

    November 9, 20265 Views

    If AGI is so close, why is everyone leaving? The neolabs and their bets

    November 6, 20265 Views
    Don't Miss
    Machine Learning

    A Framework To Detect and Reduce Proxy Capacity

    By adminNovember 20, 20260

    arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly…

    The most misunderstood graph in AI just got a major update

    November 20, 2026

    OpenAI Brings Its Collaborative Space Hubs To Plus Users

    November 20, 2026

    Adaptive-View Self-Distillation by Balancing Consensus and Teacher-Specific Privileged Signals

    November 20, 2026
    About Us

    Welcome to TechWave Dispatch

    At TechWave Dispatch, we bring you the latest news, insights, and developments from the ever-evolving world of technology. Our mission is to keep readers informed about the innovations shaping our digital future.

    From artificial intelligence and machine learning to cybersecurity, robotics, smart devices, and groundbreaking technologies, we cover the topics that matter in today's fast-moving tech landscape.

    Our Picks

    A Framework To Detect and Reduce Proxy Capacity

    November 20, 2026

    The most misunderstood graph in AI just got a major update

    November 20, 2026

    OpenAI Brings Its Collaborative Space Hubs To Plus Users

    November 20, 2026
    Most Popular

    After blocking Muse, Amazon now lets users text Alexa to help them shop

    November 6, 20260 Views

    A controlled world for robot learning and evaluation

    November 6, 20260 Views

    Perplexity now has its own always-on AI agent

    November 6, 20260 Views
    Facebook X (Twitter) Instagram Pinterest
    • Terms and Conditions
    • Privacy Policy
    • Contact Us
    • About Us
    © 2026 TechWave Dispatch. All Rights Reserved.

    Type above and press Enter to search. Press Esc to cancel.