The Fitness Industry Is Trying To Lure Gym Members Back — But Experts Say It’s Using Flawed Data
Post via:- The Washington Post
The study was conducted by MXM, a company that helps health clubs understand and manage their member experience, in partnership with the International Health, Racquet, and Sportsclub Association (IHRSA). According to MXM’s methodology page, more than 2,800 gyms nationwide submitted self-reported data on the number of member visits and cases of the novel coronavirus linked to their facilities from May through early August. Only 1,155 positive cases were reported out of about 49 million visits, leading MXM and IHRSA to conclude that the infection rate in participating gyms was a “nominal” 0.002 percent.
The Sept. 2 news release from IHRSA quoted above was titled “National Study Confirms It’s Safe To Work Out At The Gym: Current Data Shows No Evidence of COVID-19 Spread in Gyms.” An accompanying graphic declared: “THE GYM IS A SAFE PLACE FOR FITNESS.”
The findings have since been cited by IHRSA in letters to governors lobbying to keep gyms open, and the research has been shared online by prominent clubs and at least one elected official. But research experts and doctors who have looked at the study say major flaws in its methodology probably skewed data in favor of gym safety.
“It certainly isn’t a blanket statement that all gyms are safe,” said Emily Landon, a chief infectious-disease epidemiologist at the University of Chicago Medicine. “Gyms may be safe. Gyms may not be safe. This study doesn’t answer that question.”
But that was the question that Blair McHaney, MXM’s chief executive, said he sought to address when he launched the “Visits to Virus” tracking project several months ago. “We just really, really wanted to help the industry with real data,” said McHaney, who is also a longtime gym owner and a member of IHRSA.
Gyms, which were flagged by experts as a “high-risk environment,” have been hit particularly hard by the pandemic. Months of mandated closures increased interest in at-home fitness routines, and public apprehension about indoor workout facilities “created a severe shock” to the industry, said Joanna Zeng O’Brien, an analyst at Moody’s covering fitness operators.
At least three large fitness companies have filed for bankruptcy during the pandemic, including 24 Hour Fitness, Gold’s Gym, and Town Sports International, which operates New York Sports Clubs and Boston Sports Clubs, among others. And although gyms have been trying to promote their improved safety measures, getting members back indoors has been challenging.
Data on the spread of the coronavirus within gym environments is scarce. Few states have detailed contact-tracing data available to the public. As of early September, Colorado had tracked more than 600 outbreaks, none of which were traced to fitness clubs, which have been open there since June. In Louisiana, gyms have been linked to six outbreaks, totaling 34 cases, according to data from the state’s health department.
“The industry needed data that would show that when clubs were following proper safety and social distancing protocols — doing everything they can to keep their members, staff, and communities safe — that these facilities are not contributing to the spread of COVID-19,” Meredith Poppler, IHRSA’s vice president of communications, wrote in an email.
So when MXM approached the association with the idea for its study, IHRSA helped get the word out to gyms. Clubs were mostly contacted through social media and by calling executives, McHaney said.
A team within MXM set up an online registry of participating gyms. These included large franchises, such as Planet Fitness, Life Time and Orangetheory, and more regional businesses, such as the D.C. Area’s VIDA Fitness, and amounted to about 10 percent of gyms in North America, McHaney said. Gyms provided identifying information about the locations, so the MXM team could verify they were legitimate companies.
Data collection started in mid-July and covered visits from May 1 to the first week of August, McHaney said. Gyms that submitted data logged cases of coronavirus flagged through calls from their state health departments or by members and employees self-reporting positive tests. The gyms also pledged that their facilities are at least following safety recommendations from the CDC.
To McHaney, the final numbers convey a clear message: “It’s pretty darn safe to go back to the gym.”
That conclusion came as a shock to Paul J. Lavrakas, a research psychologist who reviewed MXM’s data and methodology.
“Unfounded, unsupported, irresponsible,” Lavrakas said of McHaney’s interpretation of the data. “He doesn’t have credible evidence that that statement is true. As a result, he’s making a recommendation that’s based on unreliable and not valid data.”
There are several “major sources” of error in the data collection, Lavrakas said. “Possibly the biggest reason to discount the accuracy (reliability and validity) of the data that MXM has assembled is that such data are very likely to be incomplete and biased in the direction of underestimating the prevalence of COVID transmission at health clubs,” Lavrakas wrote in his analysis of the data provided to The Washington Post.
Landon agreed that relying on data from state health departments or self-reporting could fail to capture an accurate case count, noting that a majority of states are still struggling to perform contact tracing in a timely fashion.
“I’m not sure how many people would call their gym and be like, ‘Hey, I think I went out to eat at a restaurant, and I also went to the library, and I also worked out in your gym, and now I have COVID,’ ” she said. “I don’t know that that’s happening.”
Michael Traugott, a survey methodology expert, also questioned whether the club's contributing data were representative of the target population, which was all gyms in North America.
“Which ones are included and which ones are omitted?” said Traugott, a professor affiliated with the University of Michigan’s Institute for Social Research. “And is there any reason to believe that there’s some systematic difference that would produce a bias?”
Another issue with the study is a conflict of interest, Lavrakas said. IHRSA bills itself as “one of the world’s leading authorities on the commercial health club industry,” and MXM works closely with gyms. McHaney said he estimated that at least 90 percent of the gyms that reported data are members of IHRSA.
“Maybe the most damning thing here is . . . No independent, credible third party was hired to do this study,” Lavrakas said. “You have a business interest in the outcome of this study, and you do it yourself. What more can I say? It’s a total conflict of interest.”
Traugott described MXM and IHRSA’s efforts as “a case study in how a business group will try to use data to lobby legislators and policymakers to try to amend existing policies in order to extricate themselves from a very difficult financial situation.”
“I don’t believe it is a good data collection in the sense that the information that it provides is accurate and useful,” he said. “It supports their interest in getting back into business, but it does it through an unusual and inappropriate data collection activity.”
Stephanie Bagwell, a longtime Orangetheory member in Michigan, was outraged when she received an email last month from chief executive Dave Long, who linked to an IHRSA news release that included some data from the study and wrote: “There’s no evidence to support the fact that gyms are contributing to the community spread of COVID-19 above any other types of businesses.”
“Obviously Orangetheory is struggling, and they want people to come back, but by doing it in that way, it turned me off a lot, and it really made me just not want to go back at all,” said Bagwell, 30. “If I can’t trust them to use appropriate science, how can I trust them to protect my safety?”
Orangetheory and other gyms that promoted the study have emphasized that they are prioritizing safety. “A strict adherence to science has always been the bedrock of the Orangetheory experience,” Long said in an emailed statement. “That hasn’t changed.”
Lavrakas, however, said he is concerned that IHRSA’s and MXM’s promotion of the study may be “pulling the wool over a lot of people’s eyes.”
“And maybe their own eyes,” he noted. “Maybe this is not a blatant manipulation of information merely for lobbying, cheerleading purposes. Maybe they simply just don’t realize how many flaws there are in their study and that they can’t have really any confidence in what they’ve assembled.”
Although Poppler and McHaney both said that they stood by the data, McHaney added that the intent of the study was to draw attention to a need for more research into gyms.
“We never intended to say this is a perfect research study,” he said. “It was intended to say, ‘Wow, look, this is a perfect place to go do deeper research.’ ”
Popular wrote that because of the pandemic, IHRSA and MXM had “neither time nor funds to enlarge the scope and delay the process of getting the data out.” The association “put out information that we believe is important for the public and the members of our clubs to know,” she wrote.
Landon said it was “reassuring” to see that many of the gyms that responded were unaware of outbreaks in their facilities, but she warned that the study’s conclusion may be “overblown.”
“Overblowing research results really does cause harm, not just to the people who believe them, but to science in general, and we need to stop doing that,” she said.