Google DeepMind and Kaggle have announced a new hackathon aimed at creating benchmarks for evaluating artificial general intelligence (AGI), alongside the release of a research paper that proposes a framework for assessing AI systems against human cognitive capabilities. The initiative addresses ongoing industry debates concerning the definition and measurement of progress toward more general AI.
The hackathon, titled “Measuring Progress Toward AGI: Cognitive Abilities,” is set to run from March 17 to April 16, offering a total prize pool of $200,000 for participants who develop innovative evaluation methods. Neil Hoyne, Chief Strategist at Google, noted in a LinkedIn post that this initiative is not solely about the largest AI models; rather, it seeks to determine whether these models possess the capabilities, intuition, and focus to navigate the world similarly to humans.
Accompanying the hackathon, Google DeepMind has released a paper titled “Measuring Progress Toward AGI: A Cognitive Taxonomy,” which outlines ten cognitive abilities considered crucial for assessing general intelligence. These include perception, generation, attention, learning, memory, reasoning, metacognition, executive functions, problem-solving, and social cognition. The framework encourages testing AI systems across various tasks associated with these abilities and comparing their performance to human baselines. This approach aims to provide a more nuanced understanding of how AI systems operate across different cognitive activities, moving beyond simplistic benchmark scores.
Google DeepMind asserts that current evaluation methods often fail to differentiate between models relying on memorization and those capable of adapting to new challenges. By focusing on five key areas—learning, metacognition, attention, executive functions, and social cognition—the hackathon seeks to address these gaps. Participants are encouraged to create benchmarks utilizing Kaggle’s Community Benchmarks platform, designing tasks that test how AI systems manage new information, maintain attention, plan actions, and interpret social contexts.
The competition includes two $10,000 awards for each of the five tracks, as well as four $25,000 grand prizes for the top submissions. Judging will occur between April 17 and May 31, with results anticipated on June 1. Hoyne emphasized that participants will contribute to the development of a more truthful method for evaluating AI by creating assessments that reflect real human skills, such as adaptive learning and understanding social cues.
This initiative marks a shift in focus from assessing AI model performance to establishing robust measurement standards. As the landscape of artificial intelligence evolves, there is increasing demand for consistent evaluation criteria that can accurately reflect a system’s capabilities. Google DeepMind advocates for new evaluation methods that can highlight where AI systems demonstrate reliability and where they face limitations, particularly concerning reasoning, adaptability, and social interaction.
For developers and researchers, the hackathon positions benchmark design as an essential component of AI development, with expected outcomes aimed at enhancing future evaluation standards within the industry. The initiative holds the potential to reshape how AI systems are tested and understood, paving the way for a more comprehensive approach to measuring intelligence in artificial systems.
ETIH Innovation Awards 2026 are also now accepting entries. These awards acknowledge education technology organizations that achieve measurable impact across K–12, higher education, and lifelong learning. Open to submissions from the UK, the Americas, and international participants, the awards will assess entries based on their outcomes and real-world applications.
See also
Bala AI Launches Customizable Chatbot App, Achieves 4.5-Star Rating on iOS
Germany”s National Team Prepares for World Cup Qualifiers with Disco Atmosphere
95% of AI Projects Fail in Companies According to MIT
AI in Food & Beverages Market to Surge from $11.08B to $263.80B by 2032
Satya Nadella Supports OpenAI’s $100B Revenue Goal, Highlights AI Funding Needs
