EY Canada published a cybersecurity report and most citations were hallucinated

raro11

What a horrible page to navigate

chaidhat

Maybe they should stop pushing these bankers to do 48 hour shifts…

nilirl

Site is gross to scroll on mobile

ilamont

The problem we're seeing across many professions is AI output is not getting vetted by knowledgeable people, whether it's an experienced analyst, senior engineer, expert attorney, or the resident physician. At best they skim, at worst they don't even see it at all before it's published, pushed to production, distributed to clients, or submitted to the court. In many cases the skills are available in house to do the necessary vetting, but these people are already overwhelmed with their existing day to day. Anyone remember that item a few months back about Amazon now having senior engineers vet generative AI output ( https://news.ycombinator.com/item?id=47323017 )? I had to LOL when I read that. These folks are already slammed. And the idea that Amazon would allow human bottlenecks to multiply across projects and underlying infrastructure development is ridiculous.

mapontosevenths

EY has been quietly laying people off for the last year solid. It's unsurprising that trying to do more with less results in lower quality.

Our_Benefactors

Holy horrible UI

cmiles8

This sort of thing is a complete embarrassment to a firm like EY, where people are paying them a lot of money for advice. They’ve basically demonstrated that their market leading research is just someone asking questions to ChatGPT. If you ever needed evidence to not buy “advice” from such outfits, this is exhibit one. Hopefully they at least fired the partner that published this steaming pile of AI slop.

galaxyLogic

I don't quite get it why they can't take another LLM and vet the output of the first with the second one. Surely they would not have the same hallucinations and would be able to detect hallucinations of the earlier LLM. Maybe it would cost too much in terms of tokens? I don't know but I would expect it to be realtively easy for an LLM to detect "hallucinations".

galaxyLogic

I don't quite get it why they can't take another LLM and vet the output of the first with the second one. Surely they would not have the same hallucinations and would be able to detect hallucinations of the earlier LLM. Maybe it would cost too much in terms of tokens? I don't know but I would expect it to be relatively easy for an LLM to detect "hallucinations".

jonwinstanley

Did someone hallucinate how scrolling is supposed to work on a web page?

rao-v

What’s strange about how things have developed is that this report 12-18 months ago would have been a massive scandal and would have caused durable brand damage. Now nobody will remember or notice.

mentalgear

This proves (again) one think for sure: The "Big x" Consulting Firms were always BS - and now them generating all their work themselves using LLMs just profs that their 'clients' can just skip their Million Dollar fees and just ask the LLM directly.

contingencies

Basically the entire consulting industry should die due to AI. Performative executives of yesteryear that constantly need external validation and direction and operate through hive mind and groupthink are weak and will die. I believe some of the biggest problems in today's business leaders are an inability to be open to new information, to think across traditional professional boundaries, or to ask meaningful questions. AI simply exposes this unapologetically. Bad management (this includes most government): up your game or get out of the way. Sycophantic consultant firms: die. The Economist should do an article on this.

scotty79

If they can't be bothered what they are putting out, do you think that before AI, what they wrote had any merit?

meibo

Wow, your mom lets you have TWO scrollbars?

cwillu

Is there any source with just the plain text? The css styling is headache inducing and reader mode doesn't work or has been defeated.

throwrioawfo

You're not actually meant to _read_ these reports.

le-mark

The real comedy is seeing this garbage come down from senior management, clumsy prompting, hallucinated garbage that’s all fluff and zero actionable information, zero real informed analysis. “See this analysis of our support issues from jira, we must fix these top three problems!!!” And it’s all the stuff everyone has known for years but management has refused to give anyone the authority to fix anything. I’ve seen this more than twice now; needs a name. Garbagemaxxing?

zb3

Stop messing with the scroll, I thought there was something wrong with my mouse wheel. Why are you doing this?

zelphirkalt

I wish we could just stop destroying people's jobs and lives using AI. The statistics I have heard quoted say, that merely 25% of the people actually like their job. Meaning they like doing what they do for its own sake, not because it gets them money, which they desperately need to live. I get it, most people don't want to do the work. But can we stop ruining the jobs of people, who are actually dedicated to their job and would like to keep doing their job properly? But I guess since EY is a CYA hedge anyway, no one really cares about whether the reports are hallucinations or not. Someone high up spent money on EY, so that they can justify some decision and won't be held responsible that much, when it turns out the decision was shit. All that matters to them is, that it has the appearance of something genuine and then they can base the decision on what they receive from EY, which better be what they already wanted to hear/read anyway.