I Pentested My Modernized Java App with AWS Security Agent—Here’s What It Found

In Part 1, AWS Transform modernized the Spring Boot petclinic from Java 8 to Java 21 in a single agentic run—528 lines changed, 18 files, 100% build validation. The app was live. But a modernized app isn’t the same thing as a secure app.

So I pointed AWS Security Agent at the running petclinic and asked it to do what an attacker would: find every way in. This post documents the full experience—setup, the PR code review that fired automatically, the design review, and the live penetration test that found 3 Critical, 2 High, 3 Medium, 2 Low, and 3 Informational findings.

What is AWS Security Agent?

AWS Security Agent is an AI-powered security testing platform that runs penetration tests, code reviews, and design reviews against your applications. It deploys specialized AI agents through a four-phase approach: Preflight (connectivity validation), Static Analysis (code and configuration review), Penetration Testing (runtime vulnerability exploitation), and Finalizing (validation and reporting).

Unlike traditional scanners that pattern-match known signatures, the agent reasons about your application context—reading source code, understanding what each endpoint does, and crafting targeted multi-step attack scenarios. Every finding is validated through proof-based exploitation with reproducible attack paths.

Setting Up an Agent Space

AWS Security Agent is organized around Agent Spaces—workspaces that contain your security requirements, GitHub integrations, and test runs. Setup takes about 10 minutes.

Step 1 — Create an Agent Space

Navigate to AWS Security Agent in the console and click Set up. You configure the Agent Space name and choose user access: SSO via IAM Identity Center, or direct IAM-only access.

Step 2 — Configure Security Requirements

AWS Security Agent comes with 10 managed security requirements based on industry standards, all enabled by default. These cover Authentication, Authorization, Audit Logging, Information Protection, Privileged Access, Secret Protection, Secure by Default, Tenant Isolation, and Trusted Cryptography.

Step 3 — The Agent Space Dashboard

Once set up, the Agent Space dashboard shows three capabilities: Design review (Ready), Code review (Needs setup), and Penetration testing (Needs setup). Each has its own configuration and integration path.

Step 4 — Connect GitHub

For code review and pentest remediation, you connect a GitHub repository. Click Add integration, choose GitHub, and create a new registration linking the petclinic-legacy-demo repo.

Step 5 — Configure Pentest Domain

For the penetration test, you need to verify domain ownership. Navigate to Setup Pentest configurations and enter your target domain. AWS uses a DNS TXT record to confirm you own what you’re testing.

Step 6 — Manage GitHub Capabilities

When connecting the repo, you choose what the agent can do with it: code review analysis, pentest remediation, or both. I selected Security requirements and vulnerability findings—the most comprehensive option.

PR Code Review — Automatic Security Scan on Merge

With the GitHub integration in place, AWS Security Agent automatically reviews pull requests. When the AWS Transform staging branch was merged into the main branch, the agent immediately picked it up.

The pull request contained all 7 commits from the AWS Transform run (the 6 ATX Bot steps plus the initial aws transform commit). The security agent commented within seconds:

Design Review — Architecture Analysis

Before the live pentest, I ran a design review. This is a static analysis of your architecture documentation against the 10 security requirements. You upload design files (Markdown, PDF, DOCX) and the agent evaluates compliance.

The design review completed in seconds and returned a stark result:

The 10 review findings showed every category as In Progress during analysis:

The architecture document described a Spring Boot application with no Spring Security, open actuator endpoints, and no authentication framework—exactly what the design review flagged. This was the static signal before any live exploitation.

Configuring the Penetration Test

The live penetration test is configured in the AWS Security Agent web application. I named it "demo" and pointed it at the live petclinic URL.

The agent supports 14 risk type categories. You can exclude any of them—I left all enabled:

As additional learning resources, I connected the petclinic-legacy-demo GitHub repository directly. This gives the agent access to source code context—improving coverage and precision for complex vulnerabilities.

After clicking Create pentest, the test was live immediately:

The Agent Runs — Live Attack Execution

Clicking Start run launches the agent. The four-phase pipeline begins immediately: Preflight, Static Analysis, Pentest, Finalizing.

Preflight: Infrastructure Setup

The agent first sets up logging infrastructure (CloudWatch log groups, S3), then validates endpoint accessibility. All checks passed—the petclinic was reachable and the testing environment was ready.

The Agent Thinks Like an Attacker

What makes AWS Security Agent different from a scanner is the reasoning it applies. Here’s an example from the live task logs—the agent is validating whether hardcoded database credentials in config files pose a real risk, and it reasons through it explicitly:

The agent then probes database ports (3306 for MySQL, 5432 for PostgreSQL) with Python socket connections to determine reachability. It found them firewalled from the outside—so the default credentials finding was marked a false positive. That’s reasoning, not pattern matching.

49 Actions, Multiple Attack Categories

Over the course of the run, the agent executed 49 total actions across 15+ attack categories: network scanner, code scanner, information disclosure, authentication bypass, default credentials, security misconfiguration, IDOR, business logic vulnerabilities, cross-site scripting, SQL injection, command injection, code injection, path traversal, privilege escalation, and SSTI.

Business Logic Testing In Action

The agent actively tested business logic edge cases, including race conditions in the pet name validation. Here it’s probing the duplicate validation bypass:

SQL Injection Testing

The owners endpoint was subjected to a comprehensive SQL injection battery. The agent created test records with payloads including Test' UNION SELECT NULL-- and Test'; DROP TABLE owners--, verifying they were stored as literals (not executed), confirming ORM parameterization works correctly:

Command & Code Injection Testing

The agent ran 50 command injection payloads at 1 request per 0.6 seconds, testing separators, chaining, and encoding bypasses. Similarly for code injection:

The Attack Mess Left Behind

Because the petclinic has no authentication, the agent created test records freely as part of exploitation proof. The owners list became a visible record of every attack vector tested:

13 Findings: What the Agent Actually Found

After the run completed, the Findings panel showed 18 total items (including false positives). The confirmed finding report contained 13 validated findings across the severity spectrum:

Critical Findings (CVSS 9.1–9.8)

The agent proved these findings with real exploitation evidence. Here’s the owners list showing data the agent created and modified during unauthenticated CRUD testing:

High Findings

Medium Findings

Low & Informational Findings

The Full Pentest Report

AWS Security Agent generated a downloadable PDF report with all 13 findings, CVSS v3.1 vectors, reproduction steps, and risk reasoning for each. The executive summary captures it well:

The agent also mapped every discovered endpoint during reconnaissance. Starting from a single URL, it discovered the full actuator surface and all application routes:

Key Takeaways