1. They fail when the data is sparse, or not connected well on the Internet. This is consistent with how LLMs operate, but inconsistent with why we do research. We do deep research to gain insights, not to read good, long summaries of 20 top results in Google.
2. The deep research reports hallucinate but are written in a manner that if you don't know the space already, you are going to believe it.
For example, "to address skill gaps identified in a 2024 ProductPlan survey, where 68% of startups questioned PMs’ ability to demonstrate ROI" is a snippet from my question around Product management space in India. It's a pretty convincing statement citing the survey name and exact percentage. As it turns out, there isn't such as number in the report, or anything that resembles it.
3. If you research any topic deep enough, you are going to go in areas that are sparsely available. Add it to the 2nd point around hallucination, which often increases with sparse data - and we have got a big problem at hand. We can't rely on it to do deep research. We have to cross-verify things.
Deep Research is really good to summarise a domain, and it can also surface few things you didn't know. But calling it 'Deep Research'? Still a long way to go!
This post was originally shared by on Linkedin.