burpsuite-project-parser

Burp Project Parser

Search and extract data from Burp Suite project files using the burpsuite-project-file-parser extension.

When to Use

• Searching response headers or bodies with regex patterns
• Extracting security audit findings from Burp projects
• Dumping proxy history or site map data
• Analyzing HTTP traffic captured in a Burp project file

Prerequisites

This skill delegates parsing to Burp Suite Professional - it does not parse .burp files directly.

Required:

1. Burp Suite Professional - Must be installed (portswigger.net)
2. burpsuite-project-file-parser extension - Provides CLI functionality

Install the extension:

1. Download from github.com/BuffaloWill/burpsuite-project-file-parser
2. In Burp Suite: Extender → Extensions → Add
3. Select the downloaded JAR file

Quick Reference

Use the wrapper script:

{baseDir}/scripts/burp-search.sh /path/to/project.burp [FLAGS]

The script uses environment variables for platform compatibility:

• BURP_JAVA: Path to Java executable
• BURP_JAR: Path to burpsuite_pro.jar

Check the exit code. Empty output is not a clean result. Burp ignores flags it does not recognise, so without the parser extension it starts normally and drops the query — which looks exactly like a search that matched nothing.

Exit	Meaning	What to do
0	Output produced	Proceed
1	Bad usage, or a missing file, Java or JAR	Read the message; fix the path
3	No output at all	Do not report this as "nothing found". An empty result set and an unloaded extension are indistinguishable from here. Run the control query below to tell them apart
4	Output was not JSON	The extension is not loaded and Burp ignored the flags. Install it before trusting any result

Anything other than 0 means the search result is unverified, and saying "no matching traffic" on the strength of it is a false negative reported as a clean finding.

Resolving Exit 3: the control query

Exit 3 is the common case — most narrowly-scoped regexes legitimately match nothing — so it needs a resolution you can carry out yourself. You have Bash and Read; Burp runs headless here, so there is no Extensions tab to open and no GUI to inspect. Re-running the same query just returns 3 again.

Run a control query instead: a selector broad enough that it must return rows if the parser is working at all, against the same project file. Use the sub-component filter, not the bare selector — a control is still a query, and the rules above apply to it unchanged.

{baseDir}/scripts/burp-search.sh project.burp proxyHistory.request.headers | head -c 2000

proxyHistory.request.headers is the right control precisely because it is broad but bounded: it covers every record in the project, at under 1KB each. Bare proxyHistory would answer the same question and is banned above for a reason — one record with bodies can be megabytes, and head -n 1 does not stop that, it delivers exactly one of them in full.

Control result	What it means	What to do
Rows on stdout	The parser works	Your narrower query genuinely matched nothing. Report that as a result
Exit 3 again	Nothing comes back at all	Either the extension is not loaded, or this project holds no proxy history. Check you named the right project file and that it is non-empty, then ask the user to confirm burpsuite-project-file-parser under Burp Suite → Extensions
Exit 4	Burp started and dropped the flags	The extension is not loaded. Say so; do not report on traffic

Run the control before concluding anything about the project's traffic. Assuming the extension is loaded is exactly how an unverified empty result becomes a clean bill of health — and asking the user to check the GUI is a legitimate answer where the control is inconclusive. Guessing is not.

Through a pipe the exit code is not yours to read. A pipeline reports the status of its last command, and nearly every example here ends in | jq, | head or | wc -cl — so $? is head's 0, not the script's 3. Two reliable signals:

• stderr, which reaches you regardless of piping. Error: the parser produced no output. or Error: Burp produced output, but not one JSON object is the answer; no such block means the run was fine.
• set -o pipefail when you want the code itself, or read ${PIPESTATUS[0]}:

set -o pipefail
{baseDir}/scripts/burp-search.sh project.burp auditItems | jq -c 'select(.severity == "High")'
echo "exit: $?"

Non-JSON output never reaches stdout, so a downstream grep or jq cannot match a Burp startup banner and mistake it for data.

See Platform Configuration for setup instructions.

Sub-Component Filters (USE THESE)

ALWAYS use sub-component filters instead of full dumps. Full proxyHistory or siteMap can return gigabytes of data. Sub-component filters return only what you need.

Available Filters

Filter	Returns	Typical Size
proxyHistory.request.headers	Request line + headers only	Small (< 1KB/record)
proxyHistory.request.body	Request body only	Variable
proxyHistory.response.headers	Status + headers only	Small (< 1KB/record)
proxyHistory.response.body	Response body only	LARGE - avoid
siteMap.request.headers	Same as above for site map	Small
siteMap.request.body		Variable
siteMap.response.headers		Small
siteMap.response.body		LARGE - avoid

Default Approach

Start with headers, not bodies:

# GOOD - headers only, safe to retrieve
{baseDir}/scripts/burp-search.sh project.burp proxyHistory.request.headers | head -c 50000
{baseDir}/scripts/burp-search.sh project.burp proxyHistory.response.headers | head -c 50000

# BAD - full records include bodies, can be gigabytes
{baseDir}/scripts/burp-search.sh project.burp proxyHistory  # NEVER DO THIS

Only fetch bodies for specific URLs after reviewing headers, and ALWAYS truncate:

# 1. First, find interesting URLs from headers
{baseDir}/scripts/burp-search.sh project.burp proxyHistory.response.headers | \
  jq -r 'select(.headers | test("text/html")) | .url' | head -n 20

# 2. Then search bodies with targeted regex - MUST truncate body to 1000 chars
{baseDir}/scripts/burp-search.sh project.burp "responseBody='.*specific-pattern.*'" | \
  head -n 10 | jq -c '.body = (.body[:1000] + "...[TRUNCATED]")'

HARD RULE: Body content > 1000 chars must NEVER enter context. If the user needs full body content, they must view it in Burp Suite's UI.

Regex Search Operations

Search Response Headers

responseHeader='.*regex.*'

Searches all response headers. Output: {"url":"...", "header":"..."}

Example - find server signatures:

responseHeader='.*(nginx|Apache|Servlet).*' | head -c 50000

Search Response Bodies

responseBody='.*regex.*'

MANDATORY: Always truncate body content to 1000 chars max. Response bodies can be megabytes each.

# REQUIRED format - always truncate .body field
{baseDir}/scripts/burp-search.sh project.burp "responseBody='.*<form.*action.*'" | \
  head -n 10 | jq -c '.body = (.body[:1000] + "...[TRUNCATED]")'

Never retrieve full body content. If you need to see more of a specific response, ask the user to open it in Burp Suite's UI.

Other Operations

Extract Audit Items

auditItems

Returns all security findings. Output includes: name, severity, confidence, host, port, protocol, url.

Note: Audit items are small (no bodies) - safe to retrieve with head -n 100.

Dump Proxy History (AVOID)

proxyHistory

NEVER use this directly. Use sub-component filters instead:

• proxyHistory.request.headers
• proxyHistory.response.headers

Dump Site Map (AVOID)

siteMap

NEVER use this directly. Use sub-component filters instead.

Output Limits (REQUIRED)

CRITICAL: Always check result size BEFORE retrieving data. A broad search can return thousands of records, each potentially megabytes. This will overflow the context window.

Step 1: Always Check Size First

Before any search, check BOTH record count AND byte size:

# Check record count AND total bytes - never skip this step
{baseDir}/scripts/burp-search.sh project.burp proxyHistory | wc -cl
{baseDir}/scripts/burp-search.sh project.burp "responseHeader='.*Server.*'" | wc -cl
{baseDir}/scripts/burp-search.sh project.burp auditItems | wc -cl

The wc -cl output shows: <bytes> <lines> (e.g., 524288 42 means 512KB across 42 records).

Interpret the results - BOTH must pass:

Metric	Safe	Narrow search	Too broad	STOP
Lines	< 50	50-200	200+	1000+
Bytes	< 50KB	50-200KB	200KB+	1MB+

A single 10MB response on one line will show high byte count but only 1 line - the byte check catches this.

0 0 from wc -cl is not a size to act on — the script exited 3 and nothing was verified. Piping hides that, so re-run the query on its own and read the exit code before concluding the project holds no matching traffic.

Step 2: Refine Broad Searches

If count/size is too high:

1. Use sub-component filters (see table above):

   # Instead of: proxyHistory (gigabytes)
   # Use: proxyHistory.request.headers (kilobytes)
   

2. Narrow regex patterns:

   # Too broad (matches everything):
   responseHeader='.*'
   
   # Better - target specific headers:
   responseHeader='.*X-Frame-Options.*'
   responseHeader='.*Content-Security-Policy.*'
   

3. Filter with jq before retrieving:

   # Get only specific content types
   {baseDir}/scr
   

---

Content truncated.