🚧 Dembrane Docs is under construction - Things will change
ECHO
Avoiding Pitfalls
Technical

AI reliability

Transcription errors and hallucinations

ECHO uses industry-leading transcription technology to transcribe your conversations. However, there is always a chance that the transcription is not 100% accurate.

Causes of transcription errors

  • Model Limitations: ECHO may produce repeated words o`r phrases, especially in languages with less extensive training data.
  • Audio Quality: Background noise, silences, or mixed sounds can confuse the model, leading to inaccuracies.
  • Language Detection: Incorrect language settings can cause the model to transcribe in the wrong language, resulting in errors.

Mitigation

  • Language Specification: Explicitly set the correct language in the transcription settings to prevent misinterpretation.
  • Audio Quality: Try to record in a quiet environment, and make sure the recording device is not too far away from the speaker. In general, the closer the better.
  • Prompting: Add the proper nouns and slang or jargon terms you expect to see in the conversation to provide additional context to the model under Portal Editor > Advanced Settings.

In general we find that both Chat and Library are able to filter out most errors, especially single word errors, for example when transcribing "Dembrane" as "Dembrain", ECHO wont get confused.

Chat/Library Hallucinations

Both Chat and Library use industry leading LLM's to answer your questions, but they might not be able to answert all questions accurately.

Causes of hallucinations

If you ask a question and the answer is not in the dataset, ECHO might try to answer but it might not be accurate. For example, you might organise a session where the goal was to come up with SMART goals, but it turns out that the participants mostly discussed the problems they face. In this case, ECHO will try to extrapolate SMART goals from the problems they discussed, which might not be accurate.

Asking leading questions like "Make a list of the ideas they discussed" may lead to hallucinations if participants didn't discuss any ideas. Asking for quotes can help, but with strongly leading questions and bad quality audio leading to bad transcriptions, ECHO might hallucinate the quotes.

Mitigation

In general, asking more open questions like "What did participants discuss?" will help to avoid hallucinations as you give the LLM more room to give an answer that fits the data, rather than an answer that fits the question.

Social