Tests of how well 19 large language models (LLMs) complete and perform complicated multi-step tasks has shown that they are both error-prone and, in many cases, unreliable.
The findings are contained a preprint paper, LLMs Corrupt Your Documents When You Delegate, written by Microsoft researchers Philippe Laban, Tobias Schnabel and Jennifer Neville based on a benchmark they created called DELEGATE-52 that allowed them to simulate workflows that might be part of a knowledge worker’s tasks. The paper is currently under review.
They said that the benchmark contains 310 work environments across 52 professional domains including coding, crystallography, genealogy and music sheet notation. Each environment consists of real documents totaling around 15K tokens in length, and five to 10 complex editing tasks that a user might ask an LLM to perform.
And, they stated in the paper’s abstract: “Our analysis shows that current LLMs are unreliable delegates: they introduce sparse but severe errors
Tests of how well 19 large language models (LLMs) complete and perform complicated multi-step tasks has shown that they are both error-prone and, in many cases, unreliable.
The findings are contained a preprint paper, LLMs Corrupt Your Documents When You Delegate, written by Microsoft researchers Philippe Laban, Tobias Schnabel and Jennifer Neville based on a benchmark they created called DELEGATE-52 that allowed them to simulate workflows that might be part of a knowledge worker’s tasks. The paper is currently under review.
They said that the benchmark contains 310 work environments across 52 professional domains including coding, crystallography, genealogy and music sheet notation. Each environment consists of real documents totaling around 15K tokens in length, and five to 10 complex editing tasks that a user might ask an LLM to perform.
And, they stated in the paper’s abstract: “Our analysis shows that current LLMs are unreliable delegates: they introduce sparse but severe errors
Some of the positions focus on AI-native development, data engineering and analytics, cloud-based engineering, and agent and model development as well as prompt engineering and new AI workflows.
AI agents are evolving into always-on autonomous systems that can remember, learn, and operate continuously across multiple platforms. OpenClaw, Hermes Agent, and Claude are leading this transformation, but each is taking a radically different approach that could define the future of AI automation.
The post NVIDIA’s Jensen Huang Says AI Will Turn Intelligence into a Commodity for Billions appeared on BitcoinEthereumNews.com.
TLDR: Jensen Huang says AI will make intelligence a commodity accessible to billions worldwide for the first time. NVIDIA chips power data centers at Amazon, Microsoft, Google, and Meta, driving the global AI buildout. Huang argues AI automates tasks but elevates human purpose, pushing back against job displacement fears. The NVIDIA CEO urges scientists, engineers, and policymakers to advance AI capabilities and safety together. NVIDIA chief executive Jensen Huang addressed graduates at Carnegie Mellon University on Sunday, May 10. He received an honorary doctorate at the commencement ceremony. Huang said artificial intelligence will make intelligence a commodity for everyone. He argued the technology will reach billions who have never accessed computing power before. His remarks touched on jobs, safety, and America’s industrial future. AI as a Tool for Closi
