• WHAT IT IS The rise of agentic software development means code is being written, reviewed, and shipped faster than ever before across the entire industry. • It also means that testing frameworks need to evolve for this rapidly changing landscape. • Faster development demands faster testing that can catch bugs as they land in a codebase, without requiring regular updates and maintenance. • Just-in‑Time Tests (JiTTests)are a fundamentally novel approach to testing where tests are automatically generated by large language models (LLMs) on the fly to catch bugs - even ones that traditional testing might not catch - just-in-time before the code lands into production. • A Catching JiTTestfocuses specifically on finding regressions introduced by a code change.This type of testing reimaginesdecadesofsoftware testing theory and practice. • While traditional testing relies on static test suites, manual authoring, and ongoing maintenance, Catching JiTTests require no test maintenance and no test code review, meaning engineers can focus their expertise on real bugs, not false positives.
Article Summaries:
- The article argues that the rapid pace of agentic software development-where code is written, reviewed, and shipped faster than ever-has outstripped traditional testing practices. Conventional test suites require manual authoring, ongoing maintenance, and often produce false positives, making them ill‑suited for high‑velocity environments. It proposes Just‑in‑Time Tests (JiTTests) as a solution: large language models automatically generate bespoke tests for each pull request, infer the developer’s intent, create fault‑injected mutants, and run targeted checks. These LLM‑generated tests eliminate maintenance overhead, reduce false positives, and provide engineers with actionable feedback on serious regressions before code reaches production.
Sources: