User Guide
How to Use Prompt Tuner
Everything you need to know to go from a blank prompt to a fully analysed batch of LLM outputs in minutes.
Select a Provider
Use the Provider dropdown in the top bar to choose between Gemini (Google) and Groq.
Paste Your API Key(s)
Paste one or more API keys into the API Key(s) text area — one key per line.
AIzaSyA...
AIzaSyB...
AIzaSyC...
Multiple keys are distributed in round-robin across concurrent requests. This increases throughput and avoids hitting rate limits on a single key.
Privacy guarantee: Your keys are sent to the server only when a request is made, travel exclusively in the POST body (never in a URL), and are never written to any log or file.
Select a Model
After pasting a key, the Model dropdown auto-populates with models available on your account. Select the one you want to test with.
Variable Syntax
Use curly braces to mark where values should be injected:
{variable_name}
Variable names follow Python identifier rules: start with a letter or underscore, followed by letters, digits, or underscores.
Example Template
You are a {persona}.
Write a {length}-word summary of the following topic:
{topic}
Tone: {tone}
Prompt Tuner detects all {variable} tokens live as you type and shows them as chips below the text area.
To include a literal brace in your prompt (not a variable), double it: {{ and }} become { and } after substitution.
System Prompt (Optional)
Click the System Prompt toggle at the top of the left panel to set a persona or global instruction that applies to every run (e.g., "You are a concise technical writer").
Use this mode when you want to add individual test cases one at a time — useful for exploratory testing or when you don't have a CSV ready.
-
1
Click "Manual" in the input mode toggle
Input fields appear below the settings bar — one text field per detected variable.
-
2
Fill in each variable's value
The "Hydrated Prompt" preview in the results table updates live as you type.
-
3
Click "+ Add to Test Table"
A new row appears in the table with idle status. The row is not yet executed.
-
4
Repeat for each test case, then run them all
You can add as many rows as you like before running.
Use this mode to run tens or hundreds of variable combinations at once from a spreadsheet export.
CSV Format
Your CSV must have a header row. Each column name should match a {variable} in your prompt. Extra columns are ignored.
| persona | length | topic | tone |
|---|
| doctor | 100 | climate change | formal |
| child | 50 | climate change | friendly |
| journalist | 200 | AI safety | neutral |
-
1
Click "CSV" in the input mode toggle
A drag-and-drop upload zone appears.
-
2
Drag your CSV file onto the zone, or click to browse
The file is parsed entirely in your browser — it is never uploaded to the server.
-
3
Check the validation result
A green ✓ message confirms all template variables are present. A red ✗ lists any missing columns.
-
4
Click "+ Add to Test Table"
One row per CSV record is added, all with idle status.
You can re-upload a previously exported Prompt Tuner CSV — extra columns like LLM Output and Status are automatically ignored during validation.
Run Options
- Run All — Executes every row in the table.
- Run Selected — Check the boxes next to specific rows, then click this.
- ▶ (row button) — Runs a single specific row.
Concurrent Execution
Requests are dispatched in parallel — up to 3 at the same time by default. As each completes, the next queued row starts. You can watch the statuses update live:
idle
→
queued
→
running
→
success
/
error
While a batch is running, adding new rows and deleting rows is disabled. Wait for the batch to complete, or let it finish — individual rows update in real-time without blocking the UI.
Table Columns
- Checkbox — Select rows for selective re-runs or batch actions.
- ID — Auto-assigned sequential identifier for this session.
- Variable columns — One column per
{variable}, editable inline.
- Hydrated Prompt — The fully substituted prompt sent to the model. Click to view in full.
- LLM Output — The model's response. Click to view in full or copy.
- Key #N badge — Which of your API keys was used for this run.
- Action — ▶ re-run this row, 🗑 delete this row.
Full-Screen View
Both the Hydrated Prompt and LLM Output cells are truncated to save space. Click any cell (or the "View full" button) to open the content in a full-screen modal. You can also copy directly from the modal.
Inline Editing
Click any variable value cell in the table to edit it directly. The Hydrated Prompt column updates automatically as you type. Then click the ▶ button to re-run that row with the updated value.
Modal Editing (Long Values)
For values that are too long for the inline input (e.g., pasting paragraphs), click the ⊞ expand icon next to any variable input to open a full-size text area editor.
Re-running Rows
- Click ▶ on a row to re-run it with the current prompt template and model settings.
- Checkbox rows and click Run Selected to re-run a subset.
- Click Run All to re-run every row with fresh settings (e.g., after changing the model).
Export to CSV
Click the Export CSV button in the results bar. The downloaded file contains:
- All input variable columns
Hydrated PromptLLM OutputStatus (success / error)Key Index Used
You can re-upload an exported CSV in a later session. The tool will ignore the LLM Output and metadata columns and use only the original variable columns to rebuild the test table.
Delete All Rows
Click the "Delete All" button (previously labelled "Clear") to remove all rows from the table and reset the session. This action is instant and cannot be undone — export first if you need to keep the data.
Are my API keys stored anywhere?
No. Keys are stored only in your browser's localStorage on your own device and are sent to the server only inside the POST request body when you trigger a run. The server never writes keys to logs, files, or a database. They exist in server memory for the duration of the API call only (~1–5 seconds), then are garbage collected.
What happens if one of my API keys hits a rate limit?
The backend automatically detects rate-limit errors (HTTP 429) and rotates to the next key in your list. After a 300ms pause, it retries with the next key. If all keys fail, the row shows an error status with details.
Will I lose my data if I refresh the page?
Your prompt template, system prompt, API keys, provider, and model are auto-saved to localStorage and restored on the next page load. However, the results table is held in memory only — export to CSV before refreshing if you need to keep results.
My CSV uploaded but the variables don't match. What's wrong?
The CSV column headers must exactly match the {variable} names in your prompt template — case-sensitive, no spaces (unless the variable itself has spaces). Check the red validation message — it lists exactly which columns are missing.
Can I run tests against multiple models at the same time?
Not in the current version — the app targets one model per session. A workaround is to run and export for Model A, change the model setting, then run and export for Model B, and combine the CSVs in a spreadsheet tool.
How do I add a new LLM provider (e.g., OpenAI)?
Create a new file in the providers/ folder that extends LLMProvider, implement _call(), _parse_response(), and list_models(), then add one entry to providers/registry.py. No other files change. See the README for a full example.