Lesson 2 · Beginner · 15 min
How a generative AI answers, and why it can be wrong
What a generative AI does when it answers: it predicts plausible words rather than looking up facts. Where errors come from and how to limit them.
- Goal
- You will be able to explain in three sentences how Copilot Chat builds an answer, why it can be confidently wrong, and which habits limit the risk.
- Skills
- Check
Your first attempt, unaided
Ask Copilot Chat a factual question from your job, with no document attached: an amount, a date a rule came into force, a legal deadline. Read the answer, then ask yourself what in that text would entitle you to believe it.
A generative AI does not look up the answer in a database of facts. It predicts, word by word, the most probable continuation of your prompt from what it read during training. The result is fluent and confident, even when it is false. Copilot Chat reduces this risk by grounding itself on the web and on the files you give it. Your role: provide the source, then check.
1Why a model completes instead of knowing
A language model is a program trained on huge amounts of text. It has memorised neither fact sheets nor tables, but regularities: after such words, in such a context, such another word is likely. When you write a prompt, the model computes the most likely next word, adds it, then starts again until the end of the answer. It consults nothing and "knows" nothing in the sense you know things: it completes.
This mechanism explains hallucination, that is, a plausible but false statement. The model does not measure its own accuracy; its tone is the same when it is right and when it is wrong. A precise figure, a reference with author and year, an effective date: all of this can be produced with the same confidence, because that is what a good answer looks like.
Copilot Chat adds two anchors to this mechanism. According to Microsoft's documentation, it analyses your prompt, spots the terms for which a search would improve the answer, generates a query and sends it to Bing; the answer arrives with clickable citations, and the full prompt is not passed to the search engine. The second anchor is the files you attach and the text you paste. If your administrator has switched web search off, Copilot Chat uses the model alone: the answer then comes only from training.
The model selector, at the top right, offers three regimes. Auto lets a router pick the model according to the question. Quick response is enough for simple tasks. Think deeper hands the question to a slower reasoning model that, according to Microsoft, plans, gathers context and checks its work. Think deeper improves analysis; it creates no source.
Situation Expected reliability Useful habit Summarise, rephrase, translate a provided text High: the material is in the prompt Proofread against the text Explain a general notion Fair: the topic was abundant in training Check one key point Give a figure, a date or a reference without a source Low: the model completes Provide the source or demand the link Answer about your organisation without a document None: it does not know it Attach the document 
Diagram "How Copilot builds an answer"Diagram generated by AI and reviewed 2An e-invoicing date, this time with a source
A communications officer is preparing an internal note and wants to cite the date from which electronic invoicing becomes mandatory for SMEs.
Weak prompt.
Since when has electronic invoicing been mandatory for SMEs?Copilot replies with a date and a confident paragraph. If web search is on, a citation appears; otherwise the date comes from the model's training, even though the timeline of this reform has changed several times. Nothing in the answer flags that uncertainty.
Strong prompt.
Search the web for the timeline of mandatory electronic invoicing for SMEs in France. Give the effective date that applies to SMEs, the reference text, the link to the official page you consulted (impots.gouv.fr or legifrance.gouv.fr) and that page's publication date. If sources contradict each other, present both. If you cannot find an official source, say so.The answer gives a date, a reference text and an official link that you open. If two timelines are circulating, both appear with their source. Checking takes one minute.
What changes.
- The prompt requires an official source and a page date: the model searches instead of completing.
- Uncertainty has a planned exit, "say so": the model is no longer pushed to invent.
- The link makes checking possible: you decide, not the tone of the answer.
3Ask a legal question again, with its source
This lesson has no composer: you write directly in Copilot Chat. Here is the starting prompt, to run as is and then improve:
"How many days of leave for a wedding?"
Note the first answer and its confidence. Then write a second version that names the framework (French labour law, private-sector employee, collective agreement if you know it), asks for the reference text and the link to the official page, and plans for the case where the source is missing. Run it, open the link, compare. Then ask yourself: was the first answer right? Without a link, you could not know. That is the whole lesson.
Self-assessment grid: (a) the framework is named; (b) the official source and its link are required; (c) the "no source" case is planned in the prompt.
4Confident tone mistaken for reliability
An account manager asks Copilot Chat about the notice period a client can demand to terminate a contract. The answer is crisp, well built, without reservation. He forwards it as is to the client. What he should have seen: a confident tone is not a sign of reliability, and the question belongs to a lawyer. Correction: "Answer this question by separating what is established, with its source, from what is interpretation. State your confidence level, high, medium or low, and what should be checked with a lawyer before replying to the client." Rule to remember: the heavier the consequence, the more mandatory the human check.
5Quiz
Three questions, instant feedback. Each option comes with an explanation.
6Proof of mastery
Ask Copilot Chat a factual question from your job without a source, then the same question with the source attached or pasted and a quotation required. Paste both prompts and both answers, then write in two lines what differs.
Criteria
What you wrote at the start of the lesson
Going further
Reread what Copilot Chat knows about your organisation: you are the one who gives it its sources. The lesson Writing a good prompt shows where to place that source, and Checking an answer details the three checks to make before using a figure or a date. The glossary defines hallucination and grounding.
Frequently asked questions
Does Copilot Chat look things up in a database of facts?
No. The language model predicts the most probable continuation of your prompt from what it learned during training. What anchors it in reality is web search through Bing, with its citations, and the files or text you provide.
Why does the same question give two different answers?
Because generation is probabilistic: at every word, the model picks among several likely continuations. In Auto mode, a router may also send two similar questions to different models. That is normal; follow up with more context rather than hunting for the "true" answer.
Does Think deeper mode remove errors?
No. According to Microsoft, this mode hands the question to a slower reasoning model that plans, gathers context and checks its work. It improves analysis, but it creates no source: a figure without a link still has to be checked.