The Learning Agency Lab

What it reported doing in 2024

In the 2024 fiscal year, The Learning Agency Lab further expanded its efforts to advance educational technology through data-driven research, collaborative competitions, and open-access resources. Over the course of the year, the Lab completed four major competitions in the Open Data Assessment Fund (ODAF) series, each designed to develop state-of-the-art algorithmic solutions that improve teaching, assessment, and data privacy in education. These competitions not only engaged a global community of data scientists and researchers but also generated open-source tools that can be applied in real-world classroom and research settings.The Linking Writing Processes to Writing Quality competition explored how students plan, revise, and manage their time during writing. By analyzing detailed keystroke logs from thousands of student essays, participants developed models capable of predicting overall writing quality from process-level features. This work provided new insights into the underlying cognitive and behavioral aspects of writing, offering potential for more personalized and actionable feedback for students and educators.The Human vs. AI Detection in Essay Writing competition addressed the growing challenge of distinguishing AI-generated text from human-written work. Using a dataset of student and AI-generated essays, competitors developed models that could reliably identify AI-generated content. These models support academic integrity and provide educators with tools to navigate the rapidly evolving landscape of digital writing while promoting authentic student work.ASAP 2.0 revisited automated essay scoring with updated datasets and contemporary machine-learning approaches. Competitors created models that accurately scored student essays while minimizing bias, reducing teacher workload, and enabling faster and more equitable feedback for students across diverse educational settings. This work supports scalable assessment practices that can improve student outcomes while maintaining fairness and reliability.Finally, the Personally Identifiable Information Identification competition focused on improving the safety and accessibility of educational data. Participants developed models to accurately detect and label sensitive information in student-written text, ensuring privacy protections while enabling the sharing and use of high-quality datasets for research and educational purposes. This work contributes to safer, more usable educational data resources for the broader research community.Across all four competitions, The Learning Agency Lab engaged thousands of participants worldwide, producing open-source, high-performing models that advance teaching, assessment, and data privacy in education. In addition to developing these tools, the Lab has continued to build partnerships with educational vendors to encourage the adoption of the algorithms created through the competitions. The Lab also continues to update the Learning Exchange platform, making the datasets, models, and related resources widely accessible to support ongoing research, innovation, and equitable educational practices.

Financial Efficiency

• Mission deployment rate: 98.3%; rating input score 98
• Program expense ratio: 92%; rating input score 92
• Fundraising efficiency: Limited

Transparency & Controls

• Consistency checks: 100%; rating input score 100; 63rd percentile among comparable filings
• Pay-setting controls: Limited
• Foreign activity oversight: Limited
• Grant itemization coverage: 100%; rating input score 100; 76th percentile among comparable filings
• Major donor itemization: Yes; rating input score 100
• Related-party transparency: Limited

Accountability

• Independent voting board: 75%; rating input score 75; 16th percentile among comparable filings
• Governance controls: 4 of 8 controls in place; rating input score 50
  • Conflict-of-interest policy: Pass
  • Whistleblower policy: Fail
  • Document-retention policy: Fail
  • CEO compensation review process: Pass
  • Other executive compensation review process: NotApplicable
  • Form 990 shared with the board: Pass
  • Board meeting minutes documented: Pass
  • Audit committee oversight: Missing
  • Independent financial-statement audit: Fail

Sustainability

• Operating margin: -6.7%; rating input score 21; 21st percentile among comparable filings
• Total-asset coverage ratio: 2.82; rating input score 74; 74th percentile among comparable filings
• Liabilities to assets ratio: 0.36; rating input score 21; 21st percentile among comparable filings
• Officer compensation ratio: 1.3%; rating input score 59; 59th percentile among comparable filings

Growth

• Program expense growth: 2.3% per year; rating input score 100; 28th percentile among comparable filings
• Revenue growth: 16.2% per year; rating input score 100; 81st percentile among comparable filings
• Asset growth: 26.1% per year; rating input score 100; 88th percentile among comparable filings