The rapid development and integration of AI assistance continues to be mind-blowing. Recently, my phone offered to arrange a birthday get-together for my friend Bruce (a lovely thought, but he’s in Canada and I’m in the UK so it’s unlikely to happen- sorry Bruce!). However, whilst a little geographical confusion doesn’t pose too much of an issue, given it was easy to spot, not all AI mistakes are so transparent.
Recently, my colleague Andy was preparing to trawl through research papers needed to back up his arguments for an article. Then he recalled that his workplace had installed ChatGPT. Where better to turn to for help? Afterall, that’s exactly the kind of time-saving benefit that we want to see AI used for at work. Except it didn’t quite pan out that way.
Following Andy’s prompt, ChatGPT presented him with 5 research papers, all from respected, well-known and recognisable journals. So far, so helpful. These suggested papers came with data points, helpful summaries and even key sentences and quotes for Andy’s article. The only snaffoo was…they didn’t exist. When Andy went to find the original research, he discovered that the articles were in fact invented. When he asked ChatGPT if they were genuine articles, he was reassured that ‘these were articles that could be found in respected journals such as *insert names here*’. However, Andy is a cynic. Still not convinced, Andy typed ‘did you lie and make these articles up?’ to which ChatGPT cheerfully replied ‘yes these articles do not exist’. Clearly this tech has never had to face the scrutiny of an Ethics Board or utilise Harvard referencing.
Whilst ChatGPT was doing exactly what it has been trained to do, which is to provide information and resource in response to a prompt, it is of little to no use if we cannot trust that output. What was supposed to be a time-saver, turned into a longer exercise and saw Andy return to a traditional search-engine trawl to find the accurate and relevant research papers he needed. The issue here though is not Andy’s time pressures, but rather what would have happened had Andy not been the naturally suspicious type.
A question of trust
The concept of ‘machine heuristic’ sees humans trust computers more than other humans. We are less likely to challenge computer-generated output than we are human-created output. If a colleague tells us something that doesn’t sound right then we may express doubts, but if our laptop feeds us the same information, then we are much more likely to believe it. This is a problem if the technology that is being sold to help us, is so eager to please that it lies to us.
If you are thinking that it’s early days and this is a bug that will soon be fixed, think again. In early 2023, Stephen Schwartz, a lawyer in the United States, found out the hard way that AI does not always tell the truth. Turning to ChatGPT to support his case, he was confounded when the defence lawyers told the judge that they couldn’t find any of the six judgements he had cited as evidence. Schwartz returned to ChatGPT to seek reassurance that the cases it had provided were real and to provide texts of the judgements, which it dutifully did. The only problem was, it was making them up. Less of an issue for Andy’s article, but hugely problematic when part of the legal system with potential liability or even incarceration for those involved.
AI tools are not designed to think or be nuanced. They are providing words written in an order based on statistical probability
I remain so excited for the potential of AI and the ability it has to support our work and ease the burden of tasks we find time-consuming or just plain annoying. But, as we see these tools become ubiquitous, we have to remember what we are dealing with. AI tools such as ChatGPT are not designed to think or be nuanced. They are providing words written in an order based on statistical probability. This technology saw the fictional cases Schwartz was presented with have the names of real judges and real journals attached to them; it’s just such a shame that the cases themselves didn’t exist.
In a world of fake news and misinformation leading to violence and unrest, we need to be much more cynical about how and when we use these tools. They need time to develop and get smarter before we truly trust and buy into them. If we are having to sense and fact check all the outputs we are given, then shouldn’t we just use a search engine and accept that more work needs to be done before we can trust the technology? Yes, AI is exciting. Yes, we want it to support our work and yes, we want it to make our lives easier. But we are not there yet. Whilst the accessibility and user interface has progressed massively, the reliability is stumbling behind, struggling to catch up.
Playing with this technology and experimenting with it is all well and good, although we do need to be ever mindful of the impact on the planet. But we must proceed with caution. Before using AI as our personal research assistant, we need it to prove its credentials. Much like hiring a human assistant, let’s see what experience and qualifications AI can bring to a role, before we recruit it into our companies.
Dr Stephanie Fitzgerald is an experienced Clinical Psychologist and Health and Wellbeing Consultant. Stephanie is passionate about workplace wellbeing and strongly believes everyone can and should be happy at work. Stephanie supports companies across all sectors to keep their employees happy, healthy, safe and engaged. Follow her on Instagram @workplace_wellbeing
August 27, 2024
Liar liar… the challenge AI has with the truth
by Stephanie Fitzgerald • AI, Comment, SF
Recently, my colleague Andy was preparing to trawl through research papers needed to back up his arguments for an article. Then he recalled that his workplace had installed ChatGPT. Where better to turn to for help? Afterall, that’s exactly the kind of time-saving benefit that we want to see AI used for at work. Except it didn’t quite pan out that way.
Following Andy’s prompt, ChatGPT presented him with 5 research papers, all from respected, well-known and recognisable journals. So far, so helpful. These suggested papers came with data points, helpful summaries and even key sentences and quotes for Andy’s article. The only snaffoo was…they didn’t exist. When Andy went to find the original research, he discovered that the articles were in fact invented. When he asked ChatGPT if they were genuine articles, he was reassured that ‘these were articles that could be found in respected journals such as *insert names here*’. However, Andy is a cynic. Still not convinced, Andy typed ‘did you lie and make these articles up?’ to which ChatGPT cheerfully replied ‘yes these articles do not exist’. Clearly this tech has never had to face the scrutiny of an Ethics Board or utilise Harvard referencing.
Whilst ChatGPT was doing exactly what it has been trained to do, which is to provide information and resource in response to a prompt, it is of little to no use if we cannot trust that output. What was supposed to be a time-saver, turned into a longer exercise and saw Andy return to a traditional search-engine trawl to find the accurate and relevant research papers he needed. The issue here though is not Andy’s time pressures, but rather what would have happened had Andy not been the naturally suspicious type.
A question of trust
The concept of ‘machine heuristic’ sees humans trust computers more than other humans. We are less likely to challenge computer-generated output than we are human-created output. If a colleague tells us something that doesn’t sound right then we may express doubts, but if our laptop feeds us the same information, then we are much more likely to believe it. This is a problem if the technology that is being sold to help us, is so eager to please that it lies to us.
If you are thinking that it’s early days and this is a bug that will soon be fixed, think again. In early 2023, Stephen Schwartz, a lawyer in the United States, found out the hard way that AI does not always tell the truth. Turning to ChatGPT to support his case, he was confounded when the defence lawyers told the judge that they couldn’t find any of the six judgements he had cited as evidence. Schwartz returned to ChatGPT to seek reassurance that the cases it had provided were real and to provide texts of the judgements, which it dutifully did. The only problem was, it was making them up. Less of an issue for Andy’s article, but hugely problematic when part of the legal system with potential liability or even incarceration for those involved.
I remain so excited for the potential of AI and the ability it has to support our work and ease the burden of tasks we find time-consuming or just plain annoying. But, as we see these tools become ubiquitous, we have to remember what we are dealing with. AI tools such as ChatGPT are not designed to think or be nuanced. They are providing words written in an order based on statistical probability. This technology saw the fictional cases Schwartz was presented with have the names of real judges and real journals attached to them; it’s just such a shame that the cases themselves didn’t exist.
In a world of fake news and misinformation leading to violence and unrest, we need to be much more cynical about how and when we use these tools. They need time to develop and get smarter before we truly trust and buy into them. If we are having to sense and fact check all the outputs we are given, then shouldn’t we just use a search engine and accept that more work needs to be done before we can trust the technology? Yes, AI is exciting. Yes, we want it to support our work and yes, we want it to make our lives easier. But we are not there yet. Whilst the accessibility and user interface has progressed massively, the reliability is stumbling behind, struggling to catch up.
Playing with this technology and experimenting with it is all well and good, although we do need to be ever mindful of the impact on the planet. But we must proceed with caution. Before using AI as our personal research assistant, we need it to prove its credentials. Much like hiring a human assistant, let’s see what experience and qualifications AI can bring to a role, before we recruit it into our companies.
Dr Stephanie Fitzgerald is an experienced Clinical Psychologist and Health and Wellbeing Consultant. Stephanie is passionate about workplace wellbeing and strongly believes everyone can and should be happy at work. Stephanie supports companies across all sectors to keep their employees happy, healthy, safe and engaged. Follow her on Instagram @workplace_wellbeing