How LLMs Can Help Make Static Source Code Analysis More Acceptable
The article in Java Magazin describes the challenges and opportunities of combining static source code analysis (SAST) with Large Language Models (LLMs) to improve software security.
Challenges with SAST:
- SAST tools identify security vulnerabilities in the source code, but often mixed with many false positives, which causes acceptance problems and additional work.
- However, common countermeasures carry the risk of false negatives (overlooked vulnerabilities).
Potential of LLMs:
- LLMs could detect false positives and also prioritize security vulnerabilities.
- However, the complexity of security problems requires detailed control and problem-specific prompts.
Combined Approach:
- SAST identifies suspicious code locations, which are then further processed/analyzed by LLMs.
- Advantages: Detection of false positives, increased acceptance of SAST tools, effective prioritization of relevant security vulnerabilities for process optimization, and the possibility of using more generous rule sets and less mature (but possibly more specialized) SAST tools.
Prototypical Implementation:
- Various proprietary and open-source LLMs were tested with a web-based application and the quality of the results was compared.
- Finding: Certain LLMs can reliably detect false positives.
Integration Challenges:
- Optimal prompt design, delivery of the correct code context, and the uncertainty inherent in LLM decisions remain (solvable!) challenges.
- Precise control and potentially the simultaneous use of multiple, diverse LLMs are necessary to achieve reliable results.
Cost-Benefit Analysis:
- The costs for LLM deployments are moderate and cost-effective compared to manual audits.
Conclusion:
The combination of LLMs and SAST can be an effective tool for improving software security by reducing false positives and making manual reviews more efficient. In the long term, LLMs could play a crucial role in automated security processes.
This article was published in the December 2024 issue of Java Magazin by entwickler.de.
