This crossed my news feed today. NOTE: the accuracy of their test results is not in question. The interpretation and how they choose to do analysis of their data is the issue. Most of this is focused on the data that they used to claim that SmartGut was accurate.
The science behind uBiome’s products is flawed, according to insiders and outside experts. They say that raises new questions about the company’s future. ….. uBiome built a big set of data based on the human microbiome, but the data was flawed in ways that risk making uBiome’s tests unreliable….uBiome confirmed problems with the data and said it’s conducting an internal review.
That test, called SmartGut, would become uBiome’s biggest money-maker, according to 11 former employees. It was designed to tell people about the health of their guts and their risk of diseases like irritable bowel syndrome. ….the dataset Bik described included data from 45 minors and at least one non-human….. If a customer’s levels were on par with the results in the database, the report told them so, and gave them a “wellness score” out of 100. If their levels diverged from the database, the report noted it, and suggested a possible link to illness. In fine print at the bottom of each report, it read, “These statements have not been evaluated by the Food and Drug Administration. This product is not intended to diagnose, treat, cure, or prevent any disease.” Scientists and clinicians have questioned the utility of such a test.The new test was designed for adults, according to uBiome’s website, but relied on the same volunteer database that powered the Explorer test, according to 21 former employees. Only now, if the data in the analysis was off, it could have clinical repercussions: healthy people might be told they were sick, while sick people might be told they were healthy.
Adult patients who took the SmartGut test on a doctor’s recommendation might now use it to make medical decisions.
Consider one issue, they said: poop samples from infants and pets had likely been included in the data, which was then used to create the adults-only test.
People who’d taken the drugs would often fail to disclose they’d done so, they said, because they believed enough time had passed for their effects to disappear. But antibiotics are thought to impact the microbiome for up to a year. One ex-employee who worked on the science likened taking a uBiome test to pseudoscience. “It’s like going to a really expensive homeopathic doctor,” the person said…..
In other words, people sampled their poop at home, in private, and then answered a series of key questions that included asking whether they were healthy or sick. If the answers were wrong or incomplete, it could provide a distorted sketch of the microbiome, insiders and outside experts have said. Volunteers could have been sick at the time they submitted their samples. They could also have taken antibiotics, which can distort the microbiome. ….The three ex-employees said the babies were easily identifiable based on the high presence in their samples of Bifidobacteria, a strain of microbe that’s specially suited to breaking down the ingredients in breast milk. Another handful of the samples likely belong to people’s pets, said Bik and two of the other ex-employees, pointing to data in the paper which shows strains of bacteria at levels not found in humans.https://www.businessinsider.com/ubiome-poop-testing-startup-problems-science-microbiome-2019-8
The reports on the microbiome are accurate. Charts and values based on their samples is suspect.
As a side note, the database at http://microbiomeprescription.com/ is slightly bigger than uBiome reference set cited above, is open data (so people can slice and dice as they wish – i.e. exclude samples with high bifidobacterium (babies) ), and very likely much better data quality. Also, the preferred analysis path is using non-parametric approaches which I believe for microbiome data is superior to the parametric approaches that uBiome appear to have used. …. is Citizen Science superior to commercial Science in this scenario????