# Using LLMs to Support Firefox Developers > [! note]- > The content of this page is generated by audio/video transcription and text transformation from the content and links of this source. Source: [https://fosdem.org/2025/schedule/event/fosdem-2025-6422-using-llms-to-support-firefox-developers-with-code-review/](https://fosdem.org/2025/schedule/event/fosdem-2025-6422-using-llms-to-support-firefox-developers-with-code-review/) <video src="https://video.fosdem.org/2025/k1105/fosdem-2025-6422-using-llms-to-support-firefox-developers-with-code-review.av1.webm" controls></video> ## Summary & Highlights: **Introduction to LLMs for Code Review** Mozilla is experimenting with large language models (LLMs) to support Firefox developers in code review. The aim is to improve the code review process by identifying potential issues and enhancing code quality. The project involves collaboration with Ubisoft and academic institutions, leveraging the bugbug repository for machine learning tasks. **Prompt Engineering and Experimentation** The initial approach involved simple prompts and iterative improvements, including filtering out trivial comments and focusing on meaningful feedback. The process highlighted the complexity of prompt engineering, requiring adjustments to avoid irrelevant suggestions and ensure the LLMs provide valuable insights. **Results and Developer Feedback** The experiments showed promising results, with developers appreciating the tool's ability to highlight potential code issues. Although the acceptance rate for LLM-generated comments was moderate, many developers found the suggestions useful for improving code quality. Feedback indicated that while reviews might take slightly longer, the tool contributes to more thorough evaluations. **Future Directions and Challenges** Future plans include enhancing bug detection capabilities and reducing noise in LLM outputs. Mozilla aims to integrate the tool into development environments to catch issues earlier, streamlining the review process. Challenges remain in balancing the tool's feedback with developers' needs and ensuring it adapts to different coding contexts. ## Importance for an eco-social transformation The session highlights the potential of using AI to improve software development practices, which can contribute to eco-social transformation by promoting efficient and sustainable coding practices. The collaboration between Mozilla, Ubisoft, and academic institutions demonstrates a model for open-source innovation. Eco-social designers can apply these insights to develop tools that enhance collaboration and knowledge sharing in community-driven projects. Challenges include ensuring the ethical use of AI, addressing biases in machine learning models, and fostering inclusive development environments. ## Slides: | | | | --- | --- | | ![[FOSDEM 2025/assets/Using-LLMs-to-support-Firefox-developers-with-code/preview_001.jpg\|300]] | The introductory slide presents the session's focus on using large language models (LLMs) to assist Firefox developers in code review, featuring Marco Castelluccio as the speaker. | ![[FOSDEM 2025/assets/Using-LLMs-to-support-Firefox-developers-with-code/preview_002.jpg\|300]] | This slide highlights the scale of Firefox's development, with millions of bug reports and frequent releases, underscoring the need for efficient code review processes. | ![[FOSDEM 2025/assets/Using-LLMs-to-support-Firefox-developers-with-code/preview_003.jpg\|300]] | The slide elaborates on the complexity of Firefox's codebase, with extensive contributions and a rich history, emphasizing the challenges in maintaining code quality. | ![[FOSDEM 2025/assets/Using-LLMs-to-support-Firefox-developers-with-code/preview_004.jpg\|300]] | This slide continues to discuss the scale and complexity of Firefox's development environment, reinforcing the importance of innovative review tools. | ![[FOSDEM 2025/assets/Using-LLMs-to-support-Firefox-developers-with-code/preview_005.jpg\|300]] | The slide introduces the core idea of the session: exploring automation in code reviews to enhance speed and quality, ultimately supporting developers in their work. | ![[FOSDEM 2025/assets/Using-LLMs-to-support-Firefox-developers-with-code/preview_006.jpg\|300]] | Collaboration is a key theme, with Mozilla working alongside Ubisoft and academic partners in an open-source project, aiming to publish a paper on their findings. | ![[FOSDEM 2025/assets/Using-LLMs-to-support-Firefox-developers-with-code/preview_007.jpg\|300]] | The agenda outlines the session's structure, covering the journey of prompt engineering, user feedback, and future directions for the tool. | ![[FOSDEM 2025/assets/Using-LLMs-to-support-Firefox-developers-with-code/preview_008.jpg\|300]] | The slide describes the initial approach to prompt engineering, starting with simple prompts to identify issues in code patches. | ![[FOSDEM 2025/assets/Using-LLMs-to-support-Firefox-developers-with-code/preview_009.jpg\|300]] | This slide continues the discussion on prompt engineering, emphasizing the inclusion of human examples to improve the LLM's feedback. | ![[FOSDEM 2025/assets/Using-LLMs-to-support-Firefox-developers-with-code/preview_010.jpg\|300]] | The slide addresses the challenge of filtering out non-constructive comments, focusing the LLM on providing valuable code insights. | ![[FOSDEM 2025/assets/Using-LLMs-to-support-Firefox-developers-with-code/preview_011.jpg\|300]] | Further refining the prompt engineering process, the slide discusses filtering out irrelevant comments to ensure focus on new code changes. | ![[FOSDEM 2025/assets/Using-LLMs-to-support-Firefox-developers-with-code/preview_012.jpg\|300]] | The slide emphasizes the importance of filtering in prompt engineering, removing comments unrelated to the core task of code review. | ![[FOSDEM 2025/assets/Using-LLMs-to-support-Firefox-developers-with-code/preview_013.jpg\|300]] | This slide introduces a refined approach to prompt engineering, starting with a summary before delving into code review. | ![[FOSDEM 2025/assets/Using-LLMs-to-support-Firefox-developers-with-code/preview_014.jpg\|300]] | The slide suggests providing more context around code changes to enhance the LLM's ability to generate meaningful feedback. | ![[FOSDEM 2025/assets/Using-LLMs-to-support-Firefox-developers-with-code/preview_015.jpg\|300]] | The slide explores the potential for deeper code context, including function definitions, to improve LLM-generated reviews. | ![[FOSDEM 2025/assets/Using-LLMs-to-support-Firefox-developers-with-code/preview_016.jpg\|300]] | The slide proposes adding examples of poor comments to guide the LLM in avoiding similar mistakes. | ![[FOSDEM 2025/assets/Using-LLMs-to-support-Firefox-developers-with-code/preview_017.jpg\|300]] | The agenda reiterates the session structure, highlighting key themes like prompt engineering and user feedback. | ![[FOSDEM 2025/assets/Using-LLMs-to-support-Firefox-developers-with-code/preview_018.jpg\|300]] | The slide begins the section on what the tool looks like in practice, showcasing its interface and functionality. | ![[FOSDEM 2025/assets/Using-LLMs-to-support-Firefox-developers-with-code/preview_019.jpg\|300]] | This slide continues to illustrate the tool's practical application, focusing on user interactions and feedback mechanisms. | ![[FOSDEM 2025/assets/Using-LLMs-to-support-Firefox-developers-with-code/preview_020.jpg\|300]] | The slide further demonstrates the tool's interface, detailing how developers engage with LLM-generated comments. | ![[FOSDEM 2025/assets/Using-LLMs-to-support-Firefox-developers-with-code/preview_021.jpg\|300]] | The slide showcases more examples of the tool in action, emphasizing its role in supporting code review tasks. | ![[FOSDEM 2025/assets/Using-LLMs-to-support-Firefox-developers-with-code/preview_022.jpg\|300]] | The slide illustrates the tool's interface, highlighting features that facilitate developer interaction and feedback. | ![[FOSDEM 2025/assets/Using-LLMs-to-support-Firefox-developers-with-code/preview_023.jpg\|300]] | This slide provides additional examples of the tool's application, focusing on its practical benefits in code review. | ![[FOSDEM 2025/assets/Using-LLMs-to-support-Firefox-developers-with-code/preview_024.jpg\|300]] | The agenda revisits the session's key topics, including the tool's development, user reception, and future plans. | ![[FOSDEM 2025/assets/Using-LLMs-to-support-Firefox-developers-with-code/preview_025.jpg\|300]] | The slide presents data on the acceptance rate of LLM-generated comments, comparing results from Mozilla and Ubisoft. | ![[FOSDEM 2025/assets/Using-LLMs-to-support-Firefox-developers-with-code/preview_026.jpg\|300]] | The slide discusses feedback from developers, noting that while many suggestions are useful, they often pertain more to development than review. | ![[FOSDEM 2025/assets/Using-LLMs-to-support-Firefox-developers-with-code/preview_027.jpg\|300]] | The slide provides statistics on the appreciation of LLM-generated comments, highlighting their perceived value by developers. | ![[FOSDEM 2025/assets/Using-LLMs-to-support-Firefox-developers-with-code/preview_028.jpg\|300]] | Feedback from developers indicates that while reviews may take longer, the tool enhances the overall quality of code assessments. | ![[FOSDEM 2025/assets/Using-LLMs-to-support-Firefox-developers-with-code/preview_029.jpg\|300]] | The slide examines the impact of the tool on review duration, noting a slight increase in time but improved review quality. | ![[FOSDEM 2025/assets/Using-LLMs-to-support-Firefox-developers-with-code/preview_030.jpg\|300]] | The slide discusses the editing of LLM comments, noting that most are accepted as-is or shortened for brevity. | ![[FOSDEM 2025/assets/Using-LLMs-to-support-Firefox-developers-with-code/preview_031.jpg\|300]] | The slide compares the impact of LLM and human comments on code changes, showing similar rates of influence. | ![[FOSDEM 2025/assets/Using-LLMs-to-support-Firefox-developers-with-code/preview_032.jpg\|300]] | The agenda outlines the session's focus on prompt engineering, user feedback, and future directions for the tool. | ![[FOSDEM 2025/assets/Using-LLMs-to-support-Firefox-developers-with-code/preview_033.jpg\|300]] | The slide highlights efforts to improve bug detection by leveraging Bugzilla's 'Regressed By' field for valuable examples. | ![[FOSDEM 2025/assets/Using-LLMs-to-support-Firefox-developers-with-code/preview_034.jpg\|300]] | The slide discusses strategies to reduce trivial comments, including customization and filtering based on past examples. | ![[FOSDEM 2025/assets/Using-LLMs-to-support-Firefox-developers-with-code/preview_035.jpg\|300]] | The slide introduces the concept of auto-fixing comments, allowing developers to address simple issues directly within the review system. | ![[FOSDEM 2025/assets/Using-LLMs-to-support-Firefox-developers-with-code/preview_036.jpg\|300]] | The slide explores the potential for integrating the tool into development environments to catch issues before review submission. | ![[FOSDEM 2025/assets/Using-LLMs-to-support-Firefox-developers-with-code/preview_037.jpg\|300]] | The agenda revisits the session's key themes, including prompt engineering and future plans for the tool. | ![[FOSDEM 2025/assets/Using-LLMs-to-support-Firefox-developers-with-code/preview_038.jpg\|300]] | The slide introduces the use of LLMs for QA test case generation, leveraging their creativity to identify potential software issues. | ![[FOSDEM 2025/assets/Using-LLMs-to-support-Firefox-developers-with-code/preview_039.jpg\|300]] | This slide continues to explore the role of LLMs in generating novel QA test cases, enhancing software testing processes. | ![[FOSDEM 2025/assets/Using-LLMs-to-support-Firefox-developers-with-code/preview_040.jpg\|300]] | The slide discusses smart documentation search, using LLMs to assist developers in navigating complex codebases. | ![[FOSDEM 2025/assets/Using-LLMs-to-support-Firefox-developers-with-code/preview_041.jpg\|300]] | The slide continues to explore smart documentation search, highlighting its benefits for new contributors and developers. | ![[FOSDEM 2025/assets/Using-LLMs-to-support-Firefox-developers-with-code/preview_042.jpg\|300]] | The slide addresses challenges in documentation accuracy, noting instances where LLMs identified discrepancies. | ![[FOSDEM 2025/assets/Using-LLMs-to-support-Firefox-developers-with-code/preview_043.jpg\|300]] | The slide discusses using LLMs to identify important changes in commit history, aiding in release note generation. | ![[FOSDEM 2025/assets/Using-LLMs-to-support-Firefox-developers-with-code/preview_044.jpg\|300]] | The slide highlights efforts to detect web compatibility issues, using LLMs to identify discrepancies across browsers. | ![[FOSDEM 2025/assets/Using-LLMs-to-support-Firefox-developers-with-code/preview_045.jpg\|300]] | The slide explores extending auto-fixing capabilities beyond review comments to address broader software issues. | ![[FOSDEM 2025/assets/Using-LLMs-to-support-Firefox-developers-with-code/preview_046.jpg\|300]] | The slide invites participants to engage with Mozilla's tools and explore their potential applications in other projects. | ![[FOSDEM 2025/assets/Using-LLMs-to-support-Firefox-developers-with-code/preview_047.jpg\|300]] | The final slide thanks participants for their attention and encourages further exploration of the tool. ## Links [Slides](https://fosdem.org/2025/events/attachments/fosdem-2025-6422-using-llms-to-support-firefox-developers/slides/237850/Using_LLM_bXsGC3u.pdf) [Video recording (AV1/WebM)](https://video.fosdem.org/2025/k1105/fosdem-2025-6422-using-llms-to-support-firefox-developers.av1.webm) [Video recording (MP4)](https://video.fosdem.org/2025/k1105/fosdem-2025-6422-using-llms-to-support-firefox-developers.av1.mp4)