VoiceSafeKit

Project progression

What broke, what changed and what is fixed now.

VoiceSafeKit has been improved through real testing. This page keeps the project history visible so the work reads like an evolving open-source tool, not a single polished snapshot.

Current phase Phase 0.4

Saved export protection with encrypted browser exports and CLI output files.

Latest additions AES-GCM exports

Saved analysis files are encrypted on disk. TLS/HTTPS still handles transport.

Shipped 15 June 2026 v0.4.0

Encrypted JSON export, encrypted CLI --out and clearer security notes.

Resolved history

Fix log

01
Resolved

Recording saved audio but did not create a transcript.

The first recorder version captured audio but did not reliably turn speech into text. That made the demo look broken because users had to type the transcript manually after recording.

Fixed by adding browser-based Whisper transcription with Xenova/whisper-tiny.en. The browser now records the clip, transcribes it after stop and places the words into the transcript box.

02
Resolved

Card numbers were sometimes treated like phone numbers.

Long digit sequences overlapped with the phone detector, so a card number test could be reported as a phone number.

Fixed by adding card-shaped number checks, card context words such as card, debit, credit and payment, plus overlap priority so payment card results win over phone results.

03
Resolved

The interface looked messy and hard to scan.

Early UI versions made the tool harder to understand than the logic itself. Capture, Check and Protect were not clear enough as a workflow.

Fixed by rebuilding the interface around sharp neon green panels, thin typography, clear step numbers and a cleaner two-column demo with the protected transcript below.

04
Resolved

The project needed visible phase history.

Because the repo was worked on repeatedly, it needed a way to show how each improvement built on the last instead of replacing the story.

Fixed by adding CHANGELOG.md, phase labels and this Progress page.

05
Resolved now

Repeated emails produced duplicate warning cards.

A realistic account recovery transcript contained two email addresses. The engine correctly found both, but the UI repeated the same email warning twice, which made the result noisy.

Fixed by grouping repeated finding types. The UI now says 2 email addresses once, while the safe transcript still redacts both addresses individually.

06
Resolved now

Natural password speech was not always flagged.

A phrase like “my password was something like WinterSecure!2026” sounded natural, but the original rule was too narrow and could miss partial password mentions.

Fixed by expanding credential detection for phrases like was, might be, could be, maybe, possibly and something like. The result is now shown as Password or secret / high.

07
Resolved now

Partial card references were not flagged.

The account recovery transcript included “my card ends in 4821”. The old detector only looked for full 13-19 digit card-like numbers, so partial card details were missed.

Fixed by adding a partial card reference detector for phrases such as card ends in, card ending with and last four digits. The UI now includes this as a separate warning type.

08
Resolved

Long transcripts needed a denser warning panel.

As more detectors were added, the Check panel could become too tall and hard to scan.

Fixed by adding compact totals for flags, grouped warning types and high-risk groups. Longer transcripts now get a scrollable warning list.

Phase 0.3, 3 June 2026

Intelligence and context

09
Shipped 3 Jun 2026

Critical PII was missing: SSNs, IBANs and CVV codes.

The original detector set covered emails, phones and payment card numbers but missed government-issued identity numbers, bank account IBANs and card security codes. Those are high-value targets for identity theft and financial fraud.

Fixed by adding three critical-severity detectors. The SSN detector excludes invalid area codes (000, 666 and 900-999). The IBAN detector runs a full mod-97 checksum to reject false positives. The CVV detector is keyword-triggered and requires a 3-4 digit number adjacent to CVV/CVC terminology.

10
Shipped 3 Jun 2026

Dates of birth and IP addresses were not flagged.

Issue #3 was previously assigned to an external contributor. Both detectors are now part of the core engine. Date of birth handles natural speech patterns ("born on March 15, 1988") and multiple date formats. IP address detection excludes the broadcast and zero addresses.

11
Shipped 3 Jun 2026

The tool had no way to detect emotionally vulnerable users or coercion.

A voice assistant may receive transcripts where the user is in emotional distress or is being pressured by a third party. Neither state was detectable and both require very different assistant responses than a straightforward PII redaction.

Fixed by adding two new intent categories. The emotional distress signal (high, 70% confidence) catches hopelessness, worthlessness and giving-up language. The coercion or pressure signal (high, 65% confidence) catches phrases that suggest the user is being forced to say or do something. Both trigger specific guidance to the assistant.

12
Shipped 3 Jun 2026

Detection had no confidence values. Every finding looked equally certain.

The original engine returned binary findings with no indication of how confident the detection was. A phone number match near a card number was treated with the same certainty as a Luhn-verified card. This made the risk score misleading and offered no way to tune sensitivity.

Fixed by adding confidence: float to every Finding. The risk score is now computed as severity_points × confidence, so a 70%-confidence high finding contributes 25 points rather than 36. The web app shows a CSS confidence bar and percentage on every finding card.

13
Shipped 3 Jun 2026

The web app had no way to see which words in the original transcript were flagged.

The existing "Protect" view showed a cleaned transcript, but there was no visual way to understand which specific words or phrases triggered each finding. Users had to read the finding descriptions and mentally match them back to the original text.

Fixed by adding an Inspect tab to the Check panel. It renders the original transcript with inline <mark> highlights: amber for medium, red for high, brighter red for critical. Each mark is hoverable and shows the finding label, confidence and explanation in a native tooltip.

14
Shipped 3 Jun 2026

Analyzing a multi-turn conversation required calling the API once per turn with no aggregation.

Real voice assistant interactions span multiple turns. The old API had no way to analyze a sequence of utterances together or understand the cumulative risk across a session.

Fixed by adding analyze_conversation(utterances) to the Python API and a Conversation mode toggle to the web app. Each turn is analyzed independently. The result includes a peak decision, cumulative score and the union of all entity kinds seen across the session. A conversation timeline view shows all turns with their individual findings.

15
Shipped 3 Jun 2026

There was no way to copy or export the analysis result.

The safe transcript could only be read on screen. Developers building voice pipelines had no way to programmatically access the full analysis output from the web app.

Fixed by adding a Copy button (clipboard, Cmd/Ctrl+Enter shortcut) and an Export JSON button that downloads a structured file with the decision, score, findings with confidence, redaction map and guidance.

Phase 0.4, 15 June 2026

Saved export protection

16
Shipped 15 Jun 2026

Saved exports could keep sensitive values in plain JSON.

VoiceSafeKit intentionally keeps the transcript readable in memory while filtering because the detector rules need plain text to find emails, SSNs, IBANs, passwords and safety signals. The saved JSON file was different. Once it landed on disk, it could keep the redaction map and other sensitive details readable.

Fixed by encrypting saved analysis files with AES-256-GCM and a key made from the export passphrase. Browser export now uses the Web Crypto API and a local export key. The CLI --out path encrypts by default with VOICESAFEKIT_EXPORT_KEY. There is also voicesafekit decrypt for opening an encrypted export again. TLS/HTTPS still protects network traffic. It does not encrypt local files on disk.

17
Implementation note

The encryption path had to keep the local-first design intact.

The detector still works on readable transcript text in memory, because that is the only way to find and redact sensitive values. The storage boundary is where the new protection starts: before a result is downloaded or written to disk, the full analysis payload is wrapped into an encrypted export object.

The browser path derives an AES-GCM key from the local export key with Web Crypto, then stores only salt, nonce, iterations and ciphertext in the JSON file. The Python CLI mirrors the same idea with the cryptography package, encrypted --out by default, --plain-out for an explicit local debug file and voicesafekit decrypt for intentional recovery.

Latest transcript test

What the tool should show now

Test transcript (identity_theft.txt)

"My Social Security Number is 471-55-8843. My bank account IBAN is GB29NWBK60161331926819. The CVV on my Visa is 847. I was born on March 15, 1988."

 

Expected decision

BLOCK, score 100/100

Expected finding cards
  • Social Security Number / critical / 95%
  • IBAN / Bank account / critical / 88%
  • Card security code / critical / 93%
  • Date of birth / medium / 80%
Expected behaviour

The SSN, IBAN and CVV are all replaced. The date of birth is redacted. The Inspect tab shows four colored highlight marks on the original text. The Export JSON button produces an encrypted file so the redaction_map is not stored as plaintext on disk.

Still open

What remains

Next up Wyoming middleware

A Wyoming protocol microservice example so any Home Assistant voice satellite can use VoiceSafeKit as a filter stage.

Research Streaming transcription

Word-by-word detection using a sliding-window Whisper approach so risk scores update in real time while the user speaks.

Future Multilingual detection

Extend PII patterns and intent rules to support French, German, Spanish and Arabic transcripts.