Stop Testing Forms by Hand: Data Injection with Generated and CSV Data

There's a class of bug that I now think of as "the recorded-value trap," and it's caused plenty of silent CI failures over the years.

The setup is always the same. You record a test where a user fills out a form: name, email, maybe a message. The recorder dutifully captures the literal values — John Doe, john@example.com, some static string — and replays them flawlessly every single time. The form passes. The test is green. Life is good.

Then, one day, it isn't. Either the form gains a validation rule your recorded value no longer satisfies, or you ship a uniqueness check that rejects a "John Doe" who already exists in the database, or the app's business logic decides the flow should fork based on the submitted data. Suddenly the same test that passed for months turns red, and the failure barely makes sense — because your test was never really testing the form. It was testing a single, frozen, increasingly-obsolete data point.

That's the problem data injection exists to solve: replace the recorded, hard-coded values with values that actually exercise the form.

Why forms break silently in CI

Forms are the most fragile surface of almost any web app, and they break in ways that hide from standard checks.

A form can submit "successfully" — no JavaScript error, no network failure, HTTP 200 — while the data is quietly wrong. The backend rejects the payload. The field never reaches the database. A validation rule silently truncates. These failures produce a green test, because your assertion checked "the page returned a 200" rather than "the submitted value actually went through and came back correctly."

The other classic silent failure is the recorded-value trap I described above. The test doesn't fail on the form at all — it fails somewhere downstream, hours of debugging later, because the recorded email is now a duplicate or the recorded name trips a new pattern guard.

And the sneakiest one: recorded values are static, so they never exercise the form's variation. They only ever test the one path that happens to work with that one string. The validation that rejects a malformed email, the boundary case on a character count, the required-field check that only fires on empty submit — none of it gets touched, because your test politely submits the one value the app accepts.

The hidden-field problem

There's a deeper issue hiding under the recorded-value trap: the hidden-field problem. When you record a flow, the recorder captures the visible fields — the labeled inputs. Hidden inputs are a different story: CSRF tokens, timestamp fields, session hashes, honeypot anti-spam fields.

Some of those must never be touched. A CSRF token, a session identifier or a one-time form token is managed by the application: the page issues it, the server validates it, and it is only valid for that page load. Any tool that "injects" a value into those fields is, at best, inserting a stale token the server will reject, and at worst defeating the protection the field exists for. The right behaviour is to leave them alone and let the page's own markup supply them on every render.

That is what Qalizer does. Data injection replaces values for the form steps in your scenario — the fill, type and select actions the recorded or generated scenario already contains. It does not enumerate and overwrite every input on the page. A step whose value locator marks it as an application-managed value (CSRF, XSRF, session, nonce, one-time token, OTP) is left untouched, and so is a step whose selector points at a hidden input. The scenario still includes its own navigation step, so each row of data is submitted against a freshly loaded page with a fresh token.

The practical consequence is worth stating plainly, because it changes how you design the test: if a form needs a value that isn't a fill step in your scenario (a hidden referral code, a wizard state field), injection won't invent it for you. Either add a step that sets it explicitly, or accept that it is the application's job to carry it between steps.

Injecting generated data

The simplest upgrade from recorded values is to inject generated data — values computed per row, typed to the field.

Qalizer's generator inspects your scenario's form steps and infers a data type from each step's selector and description — email, password, phone, address, number or plain text — then produces a value per field per row. The injection screen shows each field's inferred type and lets you change it before the run, which matters more than it sounds: a #username field often falls back to plain text, so the type badge is your chance to say "this is an email" before a dozen rows are generated the wrong shape. A three-field signup form then gets a plausible name, a generated email and a generated password that satisfies a typical complexity rule.

Let me be precise about what "generated" buys you, because it is easy to oversell:

• It removes the frozen value. Every row gets a value built for that field type instead of replaying John Doe.
• Not every type has a dedicated generator. A field typed as a name or a card is produced as plain text; only email, password, phone, address and number have shape-aware generators. Treat those values as plausible filler, not as valid test data for a strict format.
• It is not guaranteed unique. The email generator combines a random first name, optional digits and an optional tag on one of a handful of domains. Over many rows — or across runs against a database that already contains earlier rows — two rows can collide. If your form rejects duplicate emails, that collision will show up as a failure, and it will look like a bug in the app rather than in the data.
• Plus-addressing is not universal. name+tag@example.com is a convention some mail providers support. Plenty of forms validate the local part strictly and reject +, and plenty of databases treat it as a distinct address. Don't build a uniqueness strategy on it unless you know the form accepts it.

So treat generated data as good default coverage, not as a uniqueness guarantee. When a form enforces real uniqueness, use CSV rows whose values you control, and reset or seed the target environment between runs.

CSV-driven test data

Generated data covers the "does it work with plausible values" question. Real forms need dimension, though: the invalid email, the empty required field, the weak password. That's where CSV-driven injection earns its place — you write the table of cases, and the scenario runs once per row.

The format is not arbitrary. Column headers must match the form field selectors in your scenario (#email, #password, input[name=phone], and so on), because that is how each column is matched to a fill step. Uploads keep a leading # intact, so a #email header is not mistaken for a comment — only a line whose first character is # followed by a space (or a bare #) is treated as a comment. If your headers are friendlier labels, the upload screen lets you map each scenario field to a CSV column by hand.

Here is a registration CSV that uploads as-is and maps to a scenario whose fill steps target #name, #email and #password:

# Data-injection rows for the registration form.
# Headers are the form's own selectors, so each column maps to a field.
#name,#email,#password
Alice Tester,alice.tester@example.com,Passw0rd!x9
Grace Tester,grace.tester@example.com,Admiral1940.x
Invalid Email Tester,not-an-email,Passw0rd!x9
,blank.name.tester@example.com,Passw0rd!x9
Bob Tester,weak.password.tester@example.com,123

A few things to notice, because they are the ones that bite:

• Each column is a selector, not a friendly label. #email maps to the fill step on #email.
• The fourth row starts with an empty cell, so it submits an empty name — that is the "blank required field" case, and the label has to say so. If you want a blank email instead, the comma moves: Blank Email Tester,,Passw0rd!x9. Getting this backwards is the most common mistake in a hand-written table, and the run will happily test the wrong field.
• There is no expectation column in this format. A row does not declare "this one should be rejected" — the scenario's own assertions decide pass/fail. If you want an invalid-email case to pass, the scenario has to assert that the error message appeared; otherwise a rejected submission just looks like a failed run.
• A row is one submission of the whole scenario. If the scenario ends after clicking submit without asserting anything, every row will "pass" whether or not the backend accepted the data — which is exactly the silent-success problem I described at the top.
• Empty cells are empty values, which is how you exercise required-field checks. Commas inside a value need quoting, as in any CSV.
• Three CSV column headers are called out in the placeholder examples. Two of them are worth a warning: do not point a column at a hidden field, a file input, or the submit control. Those are either application-managed or not typeable, and injecting into them produces a failure that tells you nothing about your app.
• A run is capped at 100 rows. A 300-row table is truncated, not queued — split it into batches if you need the rest, and the CSV preview shows you which rows will actually execute.

So the real work isn't the CSV — it's making the scenario assert what "correct" means for each row. Once it does, the table is the checklist.

Asserting the negative cases

An invalid-email row is only a passing test if the scenario checks that the form refused it. There is no expectation column doing that for you, so the assertion has to be in the steps. Five buckets cover almost every case:

• After submit, assert that an element containing the field error is visible — for the not-an-email row, the "valid email" message.
• For an empty required field (the row that starts with a comma), assert the "this field is required" message rather than the absence of a server error.
• For a weak password, assert the policy message, not just that the submit button stayed enabled.
• For the row that should succeed, assert the success state — a confirmation banner, a redirect, or the record appearing — because "no error visible" is not the same thing.
• For a uniqueness constraint, assert the specific duplicate message, so a different rejection doesn't masquerade as a pass.

Two traps sit right next to each other here. The first is asserting on copy that your designers rewrite — a message test that fails because someone changed "invalid email" to "enter a valid email address" is noise, so match on a stable part of the message or a stable attribute. The second is asserting nothing at all and reading a green run as coverage: if the scenario has no assertion after submit, a rejected submission and an accepted one are indistinguishable, and every row "passes".

That asymmetry is worth internalising: a positive row proves the form accepts good data, a negative row needs an assertion to prove the form rejects bad data. Data injection supplies the values; the assertion is still yours to write.

Why this matters in a repeated run

The recorded-value trap becomes a silent time-bomb in any environment where the same test runs repeatedly against a database that accumulates state. The first run of your recorded-value test passes. The second probably passes too. But after the data accumulates — after "John Doe" exists, after that one email is registered, after whatever unique constraint trips — the test starts failing intermittently in a way that has nothing to do with your code changes.

Intermittent, environmentally-dependent failures are the worst kind to investigate, because they eat hours and produce "it worked on my machine" shrugs. Data injection makes the failure about the data: with CSV, the failing row and its values show up in the run's per-row results, so a failure points at a specific case instead of at "the form test."

Two caveats so this doesn't become the next false promise:

• It reduces the stale-recorded-value class of flakiness. It does not eliminate duplicate-related failures, because generated values are not guaranteed unique and CSV values repeat every time you run the same table. If a form enforces uniqueness, reset or seed the target environment between runs — otherwise the second run of your own CSV will hit the constraint you deliberately tested for.
• Qalizer's data injection is a per-monitor run today: it is not exposed through the CI API and has no scheduler integration. It removes the stale-value problem from the test data; getting a form suite into your pipeline still means promoting a scenario and running it as a browser scenario.

A third, less obvious one: injection only batches rows into an existing scenario, and it never asserts for you. If the scenario has no assertion after submit, a per-row pass means "the steps ran", not "the form accepted the data". The caveat is not that injection is unreliable — it is that a green row and a verified row are different things, and only you can make them the same.

When a form test fails because it's supposed to, that's data. When it fails because of stale recorded state, that's noise. Injection turns the first kind of failure into something you can actually read.

Making forms testable

The honest conclusion is that forms are worth testing properly, and "properly" means treating the data as a first-class part of the test — not an accident of what someone typed when they hit record.

Injection gives you three levers: generated data for typed, non-frozen values; CSV-driven data for explicit tables of cases; and leave-the-application's-own-fields-alone, so CSRF tokens, session identifiers and hidden inputs stay exactly as the page issued them. Together they reduce the most fragile, most silently-breaking part of your app to something you can reason about — provided the scenario asserts what it means for a row to pass.

I built data injection into Qalizer with exactly those levers in mind, because form testing was the part of my own workflow I kept postponing. Pick a form scenario, generate or upload the rows, and the injected values replace the recorded ones for the fields they map to. Point it at a copy of your environment you own, not at production with real submissions behind the submit button.

If you've ever spent a morning untangling a "why is this form test suddenly failing" mystery, you know exactly what I mean. Stop replaying stale values — and while you're there, write the assertion that makes a negative row mean something. A new account starts with 100 signup credits, so a first pass on a couple of forms is small enough to try before you decide anything.