3 lessons from a software architect on AI product engineering

AI can produce code faster than most teams can decide what deserves to be built. Government engineering teams don’t need convincing that AI can write code. What they need is a way to connect that speed to user needs, measurable outcomes, and human judgment.

Over the past year, our roughly 40-person government software delivery organization has used product engineering to make that connection. Here at all. are three lessons and practices another team can try this week.

The key takeaway

Product engineering makes engineers accountable for more than just delivering code. Their responsibility also extends to understanding whether the software actually improves the user’s work. AI accelerates implementation, but people still need to frame the problem, define credible evidence, and judge the result. Hypotheses, progress notes, deterministic checks, and human review make that accountability practical.

In practice, that means teams should:

• Start with an observable user problem and expected outcome.
• Deliver a small vertical slice that can test the outcome.
• Preserve observations and decisions so another person or AI agent can resume the work.

Lesson 1: Product engineering puts outcomes first

More than a job title, product engineering means owning outcomes end to end. It combines discovery, design, implementation, iteration, and stewardship. Engineers help understand the problem and remain accountable after release.

With 30 years of experience as a software architect, my shorthand is that software is a social science delivered through engineering. 

Jesse Piascik

Software architect, Flexion

That’s a personal framing, not a statement of Flexion policy. Engineering makes the system dependable. Understanding people, their work, and the constraints around them determines whether the system is worth building.

This fits the AI operating model Flexion has described elsewhere. AI changes how work moves through a team, not just how quickly one person types. Faster construction helps only when decisions and evidence stay visible.

Lesson 2: Hypotheses make the work testable

Hypothesis-driven development (HDD) gives product engineering a concrete structure. A story states the expected change, outcome, reason, and observation that could challenge the belief. Acceptance criteria define whether the slice works. Success measures tell us whether it mattered.

Adoption is still in progress. In a 2026-09-16 snapshot of one active project’s backlog, 32 of 75 current items (43%) had an explicit Hypothesis section, compared with 10 of 29 archived items (34%). That shows a practice that’s spreading, not a completed transformation.

The backlog also works as a lab notebook. We use imdone to store markdown stories with the code. Stronger records include a hypothesis, plan, demo plan, and durable progress notes shaped as Observation, Evidence, Decision, Next. I maintain imdone. It isn’t open source, but it offers a generous free tier. The practice matters more than the tool: record enough evidence for someone else to continue responsibly.

Three examples show where judgment belongs

Stop when requirements are unresolved

Our automated implementation pipeline is designed around two human touches: greenlight, then review and merge. One run improvised around an ambiguity. Another stopped because of an unrelated file-access bug, not sound agent judgment.

The team added a deterministic preflight gate. Before invoking the model, it checks for open questions. If any remain, the workflow labels the issue, posts the question, and stops. The safeguard is now testable.

Treat perfect AI results as a warning

On a data-annotation task, two agents returned perfect completion scores. One crude signal predicted their answers on 96 of 96 rows. They’d rediscovered a shortcut our code documented as unreliable. The team quarantined both results, blocked the shortcut, and reran the experiment. Ordinary replacement packets scored 68 to 81 percent.

The lower score was more useful because it measured the intended behavior. A successful process exit isn’t a credible outcome.

Define how a metric can mislead you

Before releasing a dashboard about whether a squad was converging on shared problems, we documented how its metric could lie. Filling empty problem fields could increase the number of distinct problems even when team behavior hadn’t changed. When that happened, the team treated the movement as inconclusive.

Writing down the failure mode protected the decision. The team didn’t relax the check until the number told a better story.

Evidence and limits

The examples come from stories and progress notes in the contract backlog. Client, program, agency, and issue identifiers are omitted. The adoption figures count explicit ## Hypothesis headings, so they measure documented use, not every instance of hypothesis-driven thinking.

The annotation range covers ordinary packets. One packet reached 97.9 percent because a font limitation constrained the available evidence, so it isn’t comparable. These examples aren’t a controlled comparison or a claim that every team will see the same result.

Lesson 3: AI native delivery keeps humans in the right loop

Our working definition is simple: have AI do what AI can do, build deterministic evaluations to check it, and keep people in the loop where their judgment is needed.

AI can accelerate boilerplate, search, summarization, first drafts, and routine debugging. Choosing among its plausible options still depends on what users said, whose needs carry the most weight, acceptable risks, and what the team is trying to learn. Those aren’t syntax problems.

Better prompting isn’t the ceiling; real gains come from improving the system around the work: clearer hypotheses, smaller slices, visible evidence, reliable gates, and explicit human decisions.

What teams can try this week?

1. Write one hypothesis before coding. Name the expected outcome and the observation that could prove the belief wrong.
2. Keep one progress note using Observation, Evidence, Decision, Next. Ask a teammate to resume from it without a verbal briefing.
3. Challenge one unusually clean AI result. Identify the shortcut or failure mode that could make the result look better than it is.

Each practice moves a team from running experiments accidentally to learning deliberately. If your organization is deciding where AI belongs in its delivery model, explore Flexion’s AI enablement and development services and join the discussion on LinkedIn.