How to Accept a Meeting-Minutes System: A Six-Criterion Scorecard for Buyers
A transcription accuracy of ≥98% is only the entry ticket. Here are six acceptance criteria — accuracy, speaker separation, conclusion retention, action-item completeness, latency and offline operation — with test methods, pass lines and common pitfalls, plus a scorecard you can take straight into procurement.

During procurement, most teams put all their attention on one number: transcription accuracy. The official figure is ≥98% for standard Mandarin — which of course matters. But a common post-launch report is: "we still have to send someone to tidy it up."
The problem is that one number is being used to accept a multi-stage system. Transcription accuracy only answers "are the characters right", while whether the minutes can be used also depends on whether speakers are attributed correctly, whether conclusions survive summarization, whether action items land on an owner, whether latency is acceptable, and — in compliance-sensitive scenarios — whether the entire chain really stays on-premises.
Note: This article is organized from the buyer's acceptance perspective. Accuracy, dialect accuracy, end-to-end latency, machine-translation speed and file-transcription efficiency are official published specifications; the remaining criteria provide test methods and definitions, since vendors differ in how they count. Align these definitions before testing.
1. Fix the acceptance criteria before you shortlist vendors
The most common sequence in procurement is: watch a vendor demo, pick one, then negotiate acceptance. Two consequences follow.
First, the metrics become incomparable. For the same recording, one vendor reports "98.2% character accuracy" and another "97.6% word accuracy" — close-looking numbers measuring entirely different things. One counts punctuation, the other does not; one used a quiet room, the other used a real meeting. Without definitions agreed in advance, any number can be explained in the vendor's favour.
Second, weaknesses get averaged away. If six criteria are combined into one composite score, a system with "99% accuracy, 55% speaker separation" can still score well — the other five pull the average up. But for the user it produces minutes where nobody knows who said what, and its practical usability approaches zero.
A workable approach splits acceptance into two layers:
| Layer | Purpose | How it is judged |
|---|---|---|
| Pass lines | The minimum acceptable value for each of the six criteria | Veto: any criterion below its line fails the whole evaluation |
| Ranking score | Which candidate is better, once all criteria pass | Weighted sum, for cross-vendor comparison |
Pass lines must be written into the technical requirements before tendering. After the contract is signed, the vendor holds the stronger commercial position.
2. The six acceptance criteria
Below is a suggested six-criterion framework. The pass lines need tuning for your own scenario, but the set of criteria should not be trimmed — every criterion you drop is one more blind spot that only surfaces after go-live.
| # | Criterion | What it answers | Suggested pass line | Related official figure |
|---|---|---|---|---|
| 1 | Standard Mandarin recognition | Are the characters right | ≥98% | ≥98% (standard Mandarin or standard American English) |
| 2 | Dialect / accent recognition | Does it hold up with other speakers | ≥96% | ≥96% (major regional dialects) |
| 3 | Speaker-separation correctness | Who said this | Set per number of speakers | Speaker separation supported |
| 4 | Conclusion retention | Do the decisions survive | Item-by-item against meeting resolutions | Four summary template types |
| 5 | End-to-end latency | Is live use viable | Under 2 seconds for live meetings | End-to-end latency under 1 second (official figure) |
| 6 | Offline availability | Is it genuinely local | Whole chain works offline | Recognition and summary models both deployed locally |
What each of these maps to on VoiVision, and where to check it:
| Criterion | How VoiVision meets it | Where to verify |
|---|---|---|
| 1 Standard Mandarin recognition | Proprietary speech-front-end algorithms and a high-accuracy transcription engine; ≥98% for standard Mandarin and standard American English | VV01 Meeting Host |
| 2 Dialect / accent recognition | Compatible with 10+ Chinese dialects including Wu, Cantonese, Sichuanese and Southern Min; ≥96% for major regional dialects | VV10 Enterprise Private Deployment Host |
| 3 Speaker-separation correctness | Speaker separation outputs role attribution and is distinct from voiceprint identification; the acceptance focus is interruption segments | Technical approach · speech engine |
| 4 Conclusion retention | Summaries use four template types matched to meeting type (summary / conclusion / multi-party discussion / upward reporting), so a mismatched template cannot flatten disagreement | Meeting-minutes quality: five engineering stages |
| 5 End-to-end latency | End-to-end latency under 1 second (official figure); on-device NPU spans 46 / 166 / 326 TOPS | VV05 Edge NPU Host |
| 6 Offline availability | Both recognition and summary models deployed locally; supports government intranets and fully air-gapped environments | Private deployment for government |
On criterion 6, one more point is worth making: it is the only criterion verified at the architectural level rather than the functional level. The first five can all be tuned up to their pass lines; whether anything leaves the network can only be confirmed by re-running with the network down.
3. Test methods and pitfalls, criterion by criterion
3.1 Accuracy and dialects: the definition matters more than the number
Test method: Prepare four kinds of recordings, 20-30 minutes each (see 3.7). Run the same audio through the system, then compare against a manual or tool-assisted ground truth.
Common pitfalls:
| Pitfall | What it looks like | Consequence |
|---|---|---|
| Counting rules not aligned | One side counts punctuation, the other does not; one counts filler words, the other excludes them | Numbers incomparable; acceptance becomes an argument |
| Only quiet, single-person reading | Accuracy looks great; real meetings fall apart | Rework after go-live |
| Standard Mandarin only, no dialects | Official dialect figure is ≥96%, but only standard Mandarin was tested | Two percentage points is the difference between an incident and a non-incident |
| Scripted speech rather than speech | Accuracy is inflated | Large gap from real-world conditions |
Acceptance action: ask the vendor for the test set and the counting script, not just the final number — running it yourself on your own recordings is the only way to verify the definition.
3.2 Speaker separation: a whole misattributed segment is worse than scattered errors
Test method: Use real multi-party recordings with three or more speakers containing interruptions and overlapping speech. Pull out at least ten interruption segments and check them one by one.
Common pitfalls:
- Role confusion during interruptions — when two people speak at once, the algorithm assigns both stretches to one person, or interleaves them. This is the characteristic failure mode of separation.
- Whole segments attributed to the wrong person — more serious than scattered character errors, because readers act on the wrong role attribution.
- Undefined degradation beyond the attendee limit — when the number of attendees exceeds the supported limit, some systems merge roles (acceptable) and others mislabel (not acceptable). Test this explicitly.
- Validating separation with a single-person reading — a mono audio file cannot test separation at all.
Acceptance action: set pass lines per speaker-count band (for example, meetings of up to 5 people and of 6-10 separately), and agree in advance how overlapping segments are judged — by whole-segment accuracy or by the overlap interval. That definition must be settled before the test.
This criterion decides whether the minutes are usable at all, and it is why VoiVision treats speaker separation as a proprietary full-stack stage rather than depending on a third-party closed service. When evaluating, ask any vendor to demonstrate: hand them a real three-person recording with interruptions and check the role labels on the spot — the technical approach page sets out the engine architecture and deployment forms.
3.3 Conclusion retention: fluent summaries are not enough
Test method: Prepare four types of meeting — information sync, decision meeting, brainstorming workshop and upward reporting — apply the matching template to each, and compare the summary against a manually written list of resolutions, item by item.
Common pitfalls:
- Template mismatched to meeting type — running a decision meeting through a summary template flattens both the conclusion and the disagreement. This is the most common failure, and often not a model-capability problem.
- Checking only whether the summary reads well — fluent but missing the conclusion is more dangerous than clumsy, because it looks fine.
- Turning "there was disagreement" into "there was consensus" — in a decision meeting the disagreement is the most valuable information, and a summary that flattens it is unusable.
Acceptance action: use a manually written resolution list as the baseline and check item by item. The check table should cover at least: resolution item, direction of the conclusion, whether disagreement is preserved, and whether ownership is retained.
3.4 Action-item completeness: three elements, none optional
Test method: Pull action items from the minutes and check three things for each — the task, the owner, and the source location.
Common pitfalls:
| Missing element | What it looks like | Why it is unacceptable |
|---|---|---|
| Owner | Only "follow up on deployment" | Nobody knows who acts |
| Source location | Action item detached from the original text | Cannot be traced; cannot be checked in a dispute |
| Task description | Over-generalised | Interpretation diverges at execution time |
Acceptance action: pull three action items and push them all the way through the integration path — WeCom, DingTalk, Feishu or a government OA system — to confirm the three elements survive the hand-off. Official support for action items and office-platform integration means this path genuinely exists.
3.5 End-to-end latency: know which latency you are measuring
Test method: Measure live meeting scenarios and file transcription separately.
Common pitfalls:
- Conflating first-token latency with full-segment latency — live meetings care about when the first characters appear (can you read along); file transcription cares about when the whole segment completes.
- Test compute differing from production — the same model can behave very differently on an accelerator card versus CPU. Ask for retest data under the production configuration.
- Testing at idle — latency degrades as concurrency rises. The official end-to-end latency figure of under 1 second is achieved under a specific configuration.
Acceptance action: state the measurement start and end points (audio in to first character out, or until the full text returns) in the technical requirements.
3.6 Offline availability: the one criterion that needs architectural proof
Test method: Disconnect external network access (or enforce egress rules via firewall), re-run the full flow, and check summary, action items and mind-map generation separately.
Common pitfalls:
- Only transcription was verified — offline transcription is easy; a cloud-backed summary is the classic blind spot. Audio stays inside while the summary leaves, and the compliance claim still fails.
- It still returned results offline — that may be a local cache or a locally installed model. Go further and confirm where the model weights live and what the egress policy is.
- The offline test was run on the test environment — the test environment may have a local model while production points at a cloud API.
Acceptance action:
- Re-run the whole flow with the network down and verify all three generation steps;
- Check the model weight files on the server;
- Check egress policy and firewall rules.
On the VoiVision side this is an architectural commitment rather than a line in a spec sheet: the speech engine and the summary-generation model both run inside the customer's own servers, with model weights, caches and logs all kept in-domain. Official specifications state that private deployment meets MLPS Level 3 and classified-information requirements, with zero data egress.
Hardware is offered by concurrency level:
- One channel — VV01 Meeting Host
- Five channels — VV05 Edge NPU Host
- Ten channels and full-stack private deployment — VV10 Enterprise Host
- Offline carry — V06 card-sized recording card
3.7 Preparing test audio: four kinds of samples that cover reality
At least four kinds, 20-30 minutes each, over 100 minutes in total:
| Kind | Why it is mandatory | Typical trap |
|---|---|---|
| Routine office recordings | Baseline scenario | Sample too small for stable statistics |
| Multi-party interruptions / overlap | Where speaker separation breaks | Substituted with a single-person reading — entirely ineffective |
| Domain-terminology meetings | Tests terminology handling (finance / medical / government terms) | Everyday vocabulary used, so terminology issues never surface |
| Far-field low-SNR recordings | Tests pickup and noise suppression | Tested only next to a desktop microphone; real meeting rooms fall apart |
The acceptance focus also differs by industry — government and finance care most about audio staying in-domain and MLPS Level 3; R&D and large enterprises about multi-room concurrency and integrating with existing OA; healthcare about domain terminology. Check your own industry first: government intranet, financial sector, large enterprise, R&D teams, healthcare.
Three practical points:
- Use real audio — not TTS-generated or read from a script. Synthetic speech scores artificially high.
- Give samples to the vendor in advance and require them to run their own engine, with your manual ground truth as the baseline.
- Hold some samples back for the acceptance stage, so the engine cannot be tuned to them.
4. Working backwards from criteria to hardware
The six criteria ultimately land on a concrete configuration. VoiVision hardware is tiered by concurrency level and deployment form, so the mapping is direct:
| Requirement | Recommended model | Specification | Typical scenario |
|---|---|---|---|
| One channel, offline transcription, desktop deployment | VV01 Meeting Host | 1 channel · 46 TOPS on-device NPU | Small single room, individual minutes |
| 5 concurrent channels, edge compute | VV05 Edge NPU Host | 5 channels · 166 TOPS | Medium rooms, shared by departments |
| 10 concurrent channels, full-stack private deployment | VV10 Enterprise Host | 10 channels · 326 TOPS | Large rooms, floor-level deployment |
| Offline carry, pocket-sized recording | V06 card-sized recording card | Card-sized · on-device NPU | Travel, field visits, ad-hoc meetings |
| Bidirectional live interpretation | VT02 translation host | 1-channel bidirectional | Meetings with overseas guests |
| Enterprise interpretation, multi-channel | VT10 translation server | 5-channel dual-screen | Several simultaneous foreign-language meetings |
The logic is short: pick the model by concurrency first, then add a VT series model if interpretation is required. For the full parameter and specification comparison, see the product overview.
5. A scorecard you can use
Turning the above into a table gives you a scorecard you can take into a tender meeting.
| Criterion | Weight (pass line = veto) | Measured value | Counting rule | Passed | Remediation required | Re-verification date |
|---|---|---|---|---|---|---|
| Standard Mandarin recognition | Veto | Punctuation / filler words counted? | ||||
| Dialect / accent recognition | Veto | Which dialects covered? | ||||
| Speaker-separation correctness | Veto | Speaker count / interruption judging rule | ||||
| Conclusion retention | Veto | Item-by-item vs manual resolutions | ||||
| Action-item completeness | Veto | Are all three elements present? | ||||
| End-to-end latency | Veto | Measurement points / compute config | ||||
| Offline availability | Veto | Result of offline re-run | ||||
| Composite ranking score | Reference | Weighted sum | ||||
| Test/production compute parity | Veto | Same configuration? |
How to use it: circulate this table together with the test recording list and the counting-rule definitions as a contract attachment. Checking it line by line is far more effective than a full day of vendor demos.
6. Three final points
First, keep the test data. The test recordings, ground truth, per-criterion scores and remediation records for failed items are the only evidence available if a dispute arises later.
Second, require an offline deployment package. If the test environment depends on the vendor's cloud services, many problems only surface during the real deployment — by which time procurement is usually near its end. An offline package also lets customers run their own third-party evaluation.
Third, do not test only features.
Accelerator choice directly determines latency, so acceptance must confirm that the production compute configuration matches the test environment — otherwise the test conclusions do not extrapolate.
Official specifications note that the speech engine natively supports NVIDIA, Ascend, Cambricon and Hygon compute ecosystems, and runs real-time inference on CPU alone; on-device NPU acceleration spans three compute tiers.
Specific models and compute tiers can be checked item by item against the technical approach page.
7. Next: test first, or pilot first?
This scorecard is not tied to any vendor — it should be settled before you shortlist anyone. If you are already in selection, two routes:
Route A: put this scorecard to your existing suppliers. Send them the six criteria and pass lines, ask for written answers plus the test set and counting script. This filters out non-compliant candidates quickly and costs nothing.
Route B: go straight to a pilot and validate with real meetings. VoiVision covers the full hardware range from a desktop host to ten concurrent channels, with private deployment and both recognition and summary models running locally, so the complete acceptance flow — including the offline re-run — can be run directly against your own meeting recordings. Supported compute covers NVIDIA, Ascend, Cambricon and Hygon, and CPU-only inference runs in real time; the adaptation list and deployment forms are worth checking first on the technical approach page.
Not sure which model fits? Start from your industry: government intranet, finance, large enterprise, R&D teams, healthcare, retail and SMBs each have their own emphasis.
Bring your own meeting recordings and we can fill this scorecard in together — book a demo, or reach the sales team on WeCom.
On the figures: official published specifications — transcription accuracy ≥98% (standard Mandarin or standard American English), major regional dialects ≥96%, end-to-end latency under 1 second, machine translation ≥800 characters per second, file transcription efficiency 10:1, and on-device NPU acceleration of 46 / 166 / 326 TOPS. These vary with audio conditions, model size, concurrency strategy and compute configuration; acceptance should rely on measurement.
The translation engine behind those figures is NiuTrans, used over 100,000 times worldwide.
FAQ
Q: What pass rate should we set when accepting a meeting-minutes system?
A: Judge each of the six criteria against its own pass line rather than a single composite score: standard Mandarin recognition rate, speaker-separation correctness, conclusion retention rate, action-item completeness rate, end-to-end latency, and offline availability. Any single criterion below its line makes the system unusable — a system with 99% recognition but only 60% speaker separation still produces minutes nobody reads. A composite score is useful for ranking candidates, but it must not mask individual weaknesses.
Q: How should recognition accuracy be measured — character error rate or word error rate?
A: Look at both, and pin down the counting rules first. Character error rate tells you whether subtitles are usable as-is; word error rate is closer to how the business perceives quality — how many words in a sentence are wrong. In Chinese you also need to agree on whether punctuation and filler words are counted. Agree the definition before the test, otherwise numbers from different vendors are not comparable.
Q: How much test audio should we prepare?
A: At least four kinds of samples: routine office recordings, multi-party meetings with interruptions and overlapping speech, meetings containing domain terminology (finance, medical, government), and far-field recordings with a poor signal-to-noise ratio. Twenty to thirty minutes of each, over 100 minutes in total, is a reasonable minimum. Validating with a single quiet reading by one person is the most common and least effective approach — it cannot surface any real-world problem.
Q: How do we confirm the system genuinely runs offline?
A: Re-run the complete workflow with external network access disabled, and pay particular attention to summary generation. Verifying transcription alone is not enough — offline transcription is comparatively easy, while a summary that calls a cloud large-model API means audio stays inside while the summary leaves, and the compliance claim still fails. Disconnect the network and re-run the full flow: if the summary step errors out or degrades, it depends on the cloud. Also confirm that model weights are present locally and check the egress policy.
Q: What should the acceptance criterion for speaker separation be?
A: Look at two things: how often roles are attributed correctly, and whether entire segments get assigned to the wrong person. Validation must use real multi-party recordings with three or more speakers — never a single-person reading — and should specifically target segments where people talk over each other, which is where separation algorithms break down. Also test what happens when the number of attendees exceeds the supported limit: merging roles is acceptable, mislabeling is not.
Q: Can we keep the test data and reuse it later?
A: You should, and write it into the contract as an attachment. The test recordings, test scripts, per-criterion scores and remediation records for failed items collectively form the basis for resolving any later dispute. Agreeing in advance how underperforming items will be remediated and re-verified saves considerably more effort than an extra hour of testing.
Q: Should we require an offline deployment package for the test environment?
A: Yes. If the test environment relies on the vendor's cloud services, many compliance and performance issues only surface during the real deployment — by which time procurement is usually near its end. An offline deployment package also lets customers run their own third-party evaluation instead of depending entirely on a vendor-supplied report.
