Abstract
Crowdsourced data science competitions have emerged as a powerful mechanism for advancing research in energy informatics, offering scalable pathways for developing machine learning solutions that enhance energy efficiency and smart building operations. The ADRENALIN Load Disaggregation Challenge addressed a central problem in energy analytics—non-intrusive load monitoring (NILM) of heating and cooling loads in commercial buildings—while emphasizing the importance of model generalization across different buildings. This paper presents a comprehensive reflection on the lessons learned from organizing and executing the ADRENALIN competition, including technical insights, organizational challenges, and recommendations for future energy data challenges. In addition to the ADRENALIN case, a comparative analysis is conducted with recent energy informatics competitions, including ASHRAE Great Energy Predictor III, BigDEAL 2022, CityLearn 2022, the Global AI Challenge, AIcrowd’s Brick by Brick competition, and the NYSERDA RTEM Hackathon. This analysis identifies common challenges and effective strategies, such as the importance of dataset quality and preprocessing, the impact of evaluation metric selection, and trade-offs between large-scale open platforms (e.g., Kaggle) and research-oriented platforms (e.g., CodaLab, AIcrowd). Based on these insights, the paper outlines a set of best practices for designing energy data competitions, including multi-phase evaluation structures, clearly defined scoring frameworks, enhanced participant engagement strategies, and pathways for post-competition implementation to maximize real-world impact.