← All work

Case 02 · AI product · MedTech · Pilot · Human-in-the-loop · 4 min read

Making GenAI useful in medical device engineering

Problem statement

A local AI assistant helps engineers draft usability analyses, with human approval at every step.

Problem

Engineers spend hours drafting analyses

Medical-device usability work involves documenting hundreds of tasks. Preparing initial analyses and reports consumes time that engineers could spend evaluating the device.

Issue

Every draft still needs expert judgment

Device context matters, and plausible AI output can be wrong. Faster drafting is useful only when engineers can check, correct and approve the result.

Solution

AI drafts; engineers approve

A local Llama assistant prepares usability analyses five tasks at a time. Engineers review every output. A pilot with four to five engineers reduced analysis and drafting time by about 50%.

Progress About 50% less analysis and drafting time in a 4–5 engineer pilot.

Role
Product ideation, development and pilot ownership
Domain
Medical technology, generative AI, workflow automation
Pilot
4 to 5 engineers, then an internal project
Technology
Llama, an open-source LLM, running locally
Skills
Workflow design, AI product judgment, stakeholder buy-in, piloting

Problem

Repetitive usability drafting takes time away from engineering analysis

Usability engineers document hundreds of device tasks. Repetitive drafting takes time away from analysis.

  • Use AI to prepare initial analyses.
  • Keep engineers responsible for review and approval.

Users

Usability engineers need drafts they can verify and approve

  • User: usability engineers preparing device analyses and reports.
  • Need: accurate drafts they can check and approve.
  • Adoption: working examples, then a small engineering pilot.

Goals and metrics

Save drafting time while preserving engineer accountability

  • Goal: reduce the time engineers spend preparing initial analyses and report drafts.
  • Metric: time to produce the usability analysis and report draft.
  • Guardrail: no output is used until an engineer has reviewed, corrected and manually approved it.

The claim is deliberately narrow: faster drafting and analysis for specific activities, not a shorter medical device development or reporting timeline.

Solution

A local Llama assistant drafts five tasks for engineer review

INPUTAIAIENGINEEROUTPUTUser tasksfive at a time,not the wholereportLocal Llamaopen-source modelrunning on siteDraft analysisguided by theIEC 62366 standardReviews, corrects, approvesmanual sign-offUsed on the projectapproved contentonlyengineer corrections: errors are caught here, not downstreamLimits seen in the pilot: slow local processing, missing device context, plausible but wrong text
The assistant workflow, from tasks to approved content.
Diagram guideRead left to right. Five user tasks go to a local Llama model, which drafts an analysis guided by the IEC 62366 standard. The engineer box, highlighted, is the only way to the final box: nothing is used until it is reviewed, corrected and manually approved. The dashed line shows corrections going back to the draft. The note at the bottom lists the limits the pilot found.

A local Llama model drafted usability analyses five tasks at a time, guided by IEC 62366. Engineers corrected and approved every output.

Decisions and trade-offs

Small batches and mandatory approval made the pilot reviewable

01Scope of automation

Five tasks at a time, with a draft analysis for each batch

Why: Small batches keep each output reviewable. A smaller scope was also easier to prove.

Trade-off: End-to-end automation of the whole report

02Who signs off

Engineers review, correct and manually approve every output

Why: In medical device engineering, errors cannot be accepted without verification. Review is a product requirement, not a workaround.

Trade-off: Some of the speed

03How to introduce it

Working examples and a prototype first, then a pilot with a small group

Why: Engineers had to trust it before an internal project relied on it.

Trade-off: A faster, wider rollout

Pilot findings

The pilot exposed gaps in speed, device context and accuracy

  • Speed: local inference introduced delays.
  • Context: missing device information reduced relevance.
  • Accuracy: plausible text still required verification.

Results

Four to five engineers cut analysis and drafting time by about 50%

MeasureResult
Analysis and report drafting timeAbout 50% lower
Pilot participants4 to 5 engineers
Where the gain came fromFaster preparation of initial analyses. Engineers still reviewed, corrected and approved everything
What is not claimedThat the whole development or reporting timeline was halved

The reduction applies to analysis and drafting, not the full development timeline.

Successor

The context gaps informed a separate RAG-based successor

The pilot informed a separate successor using RAG and enterprise-protected AI.

  • Retrieve relevant device context.
  • Use appropriate enterprise safeguards.
  • Retain engineer approval.

Additional accuracy and time gains have not been measured.

What I learned

Useful AI drafts require context and a clear review workflow

  • Choose a repetitive task with a clear review step.
  • Supply the context needed for reliable output.
  • Use the pilot to learn where the model needs support.