FDA panel: Proposed changes to pulse oximeter premarket studies generally well received

Key takeaways:

• The FDA’s proposed changes to premarket pulse oximeter trial criteria were generally accepted by an advisory panel.
• One proposed change focuses on including patients of 10 different skin pigments.

At a Medical Devices Advisory Committee meeting held Feb. 2, proposed changes to premarket studies assessing pulse oximeter performance were generally well received by the panelists.

“It seems like the committee likes the subjective and objective approach to these premarket studies,” Hugh A. Cassiere, MD, FCCP, FACP, chair of the Anesthesiology and Respiratory Therapy Devices Panel of the FDA’s Medical Devices Advisory Committee, said during the meeting.

As Healio previously reported, in November 2022, panelists from this committee recommended various improvements to the FDA regarding pulse oximeters and their accuracy among patients with darker skin pigmentation.

Panelists called for subjective and objective pigmentation methods, a sample size that included a wide range of skin pigmentations, a tighter overall performance measurement and a more meaningful accuracy metric, Kumudhini Hendrix, MD, OHT1, CDRH, chief physician at the FDA, said.

Based on this feedback, the FDA developed several proposed changes to its 2013 guidance on pulse oximeters regarding what should be featured in premarket clinical trials assessing these devices.

During this meeting, the FDA posed three questions to the panel asking for their thoughts on the administration’s proposed changes to the design of premarket clinical trials, the proposed new definition of non-disparate performance and whether both these proposals should also be applied for over-the-counter pulse oximeters.

Clinical trial design

The first proposed change the FDA asked the panel to discuss focused on the design of future premarket clinical trials of pulse oximeters.

Currently, the FDA suggests including at least 10 healthy individuals ranging in age, sex and skin pigmentation, with a minimum of two individuals, or 15% of the cohort, who are darkly pigmented. Trials should also have at least 200 paired datapoints to assess SpO₂ minus SaO₂, according to the 2013 guidance document.

The FDA is now proposing that trials include at least 24 individuals with 480 paired datapoints. These individuals should come from all pigmentations on the Monk Skin Tone (MST) scale, a validated scale that ranges from 1 to 10 and captures race and ethnicity diversity.

Specifically, the FDA suggests that at least one individual/15% of the total cohort belong to each MST value group. Additionally, when assessing individuals based on MST ranges (specified as 1-4, 5-7 and 8-10), each range should include a minimum of 25% of individuals.

The FDA’s proposal also outlines that an Individual Typology Angle (ITA) — an objective pigmentation measurement — be taken at the sensor site.

“I, generally speaking, support the approach that's being proposed in terms of looking at both a visual assessment with MST and then an objective assessment with ITA,” Julian M. Goldman, MD, FASA, anesthesiologist at Massachusetts General Hospital, said.

Panelists expressed favorable views on use of the MST scale; however, use of ITA evoked some mixed feelings. One concern discussed by several panelists focused on the site of ITA use.

“The measurement of ITA on the dorsal aspect of the distal-most aspect that’s accessible on a finger seems like a good idea, but let’s keep in mind, it isn’t actually the measurement site for pulse oximetry,” Goldman said.

The use of this method in pediatric patients also brought up questions on the site of measurement.

“If we’re going to include pediatric patients in these studies moving forward, I think it’s necessary to also consider where we’re putting the probe in pediatric patients,” Cheryl K. Gooden, MD, associate professor of anesthesiology and pediatrics at Yale School of Medicine, said. “For example, many times we will use the toe, or we’ll use the foot itself for placement of the probe.”

On the same topic of ITA, Jeffrey M. Feldman, MD, MSE, professor of clinical anesthesiology and critical care at Perelman School of Medicine at the University of Pennsylvania, questioned if it was the best objective method.

“My concern, though, is that [ITA] is only related to the melanin content of the skin as determined by histologic studies,” Feldman said. “It doesn’t actually tell us how light is interfered with or interacted with in the skin of a dark finger, and I think that really is the information that we’d want an objective measure.”

Notably, some panelists, such as Thomas E. Wiswell, MD, neonatologist at Kaiser Permanente Moanalua Medical Center, thought that the MST ranges/buckets could be made smaller to allow for more patients of different pigmentations.

“While the bucket approach sounds reasonable, I’d probably narrow it down to one and two, three and four, five and six, so you have five different buckets because ... if you have it only three buckets as it were, it may not catch everybody, again especially at the darkest pigmentation end of the spectrum,” Wiswell said.

Summing up the panel’s discussion, Cassiere noted approval for the MST scale and that the ITA would be worth trying despite the fact that it has not been extensively studied.

“This is going to be I think the first objective measure that the FDA is going to require for premarket approval, so it’s a start, and I think a good start,” Cassiere said.

The second point of discussion addressed the sample size shift from 10 to 24, and several panelists expressed that they would want the sample size to be larger.

“I think doubling the sample size will be a benefit,” Rachel Brummert, MS, communications lead of the American Society of Pharmacovigilance, said. “I think 24 is just the bare minimum and I think we have a responsibility as [a] panel to get as much information as we can, and I think we can accomplish that by at least doubling it.”

When discussing sample size, statistics became a major point of conversation, and some panelists mentioned their desire to find out if differences exist between individuals based on differing skin pigmentations. Ben Saville, PhD, president and lead statistical scientist at Adaptix Trials LLC, concluded that the panel wants inferential statistics.

“Maybe that requires a bigger sample size, and maybe that perfectly justifies why people are asking for a bigger sample size, so we don’t want a bigger sample size just to be more convinced of some more ad hoc calculated criteria,” Saville said. “I feel like what people are really asking for is a more stringent criteria, really formal evaluation of whether there really is differential bias based on what skin pigmentation one has.”

Cassiere summed up this discussion stating that the new proposed sample size of 24 is better than the previous size of 10.

“In general, there seems to be a support on what was presented earlier by the FDA, that 24 — given that 80% power number — can help mitigate some of this diversity using the premarket evaluation that’s recommended by the FDA,” Cassiere said.

In terms of additional recommendations for this proposal, Goldman mentioned the potential importance of assessing perfusion.

“I think that there’s enough data to support that low perfusion, especially with darker skin pigmentation, seems to be something that is causing these inaccuracies and the bias,” Goldman said.

“I think we want to look into that and better understand whether some of the testing at a lower perfusion state is one area,” he continued.

Other members of the committee, such as Tamorah R. Lewis, MD, PhD, associate professor in the department of pediatrics at University of Toronto, thought back to two patients who shared their stories with the panel earlier in the meeting when considering additions to the FDA’s proposal.

“I bring to the panel the option of adding potentially a few more secondary outcomes, such as time from placement of device to steady accurate reading because one of the patients is worried that his skin pigmentation might be complicating his ability to quickly obtain a steady accurate reading,” Lewis said. “The second secondary outcome we might discuss is over a certain time interval ... what is the percent of time that you have an unreliable waveform, and how does that differ by MST categories?”

Non-disparate performance definition

The second main question posed by the FDA asked panelists to discuss the proposed, new non-disparate performance definition.

For SaO₂ measures greater than 85%, the estimate of the absolute difference in SpO₂ bias across ITA and MST levels is recommended to be less than 1.5%, whereas for SaO₂ measures between 70% and 85%, the estimate of absolute difference is recommended to be less than 3.5%.

When asked about the advantages and challenges of this definition, the panel demonstrated uncertainty and struggled with coming up with a clear answer for the FDA.

Saville noted that the definition could have issues if it was established using the proposed sample size.

“One thing I don’t love about these criteria is that I have the feeling that ... these criteria were decided essentially back calculated based on what’s feasible in this population with a small sample size,” Saville said.

Cassiere summarized the panel’s discussion by saying that curiosity was raised over the numbers proposed in this definition.

“One thing that kind of stands out is a couple of panel members made mention of the breaking up. Why should we have a 1.5 with greater than 85% and a higher error threshold, I guess I’d call it, for what some of us would think would be even more clinically significant D saturations?” Cassiere said.

“I’m not sure if we had a satisfactory discussion for the FDA on this,” he added.

The second part of this discussion focused on alternate acceptance criteria options.

By the end of this conversation, Cassiere stated that the panel was “pretty much okay with what has been proposed.” However, William C. Wilson, MD, MA, executive vice president of clinical operations at Masimo Corporation, did suggest simplification of the acceptance criteria if it was feasible.

“It may just be easier to look at what is the actual mean difference for those individuals in the dark-skinned category compared to those in the white skin category,” Wilson said. “If that difference is more than some number that the agency believes is clinically significant, then it would be non-disparate performance.”

As a follow-up question, the FDA asked the panel about intergroup differences, and Feldman and Cassiere both expressed that intragroup variability is not that important.

“Non-disparate to me means that for any given patient, irrespective of their skin tone, I will get a performance within limits that are clinically acceptable,” Feldman said. “So, we define what the clinically acceptable limits are, we study the populations, and then we compare the dark and the light skin, and we prove statistically that the differences are not important within the range that we’re trying to test. ... That to me is the important question, not whether the absolute difference between the groups is within a certain margin.”

The last part of the discussion focused on whether there are specific SaO₂ thresholds for which the accuracy of SpO₂ for detecting hypoxemia should be analyzed.

Notably, the FDA simplified this question and asked if it would be beneficial to have additional analysis conducted at important clinical thresholds, and the panel concluded that this analysis is not necessary.

“I think the reality is, if you set a threshold, you’re going to be wrong in some patients and right in others,” Feldman said.

“Everyone’s OK with the current criteria with not adding any kind of clinical or other thresholds for the device companies to adhere to at least at this time,” Cassiere said.

Over-the-counter pulse oximeters

The final question the FDA asked the panel to discuss focused on determining whether the above proposals should also be applied to OTC pulse oximeters for medical use.

Rather than having separate and individualized criteria, the panel agreed that the same proposals for prescription pulse oximeters should be in place for OTC devices.

“If it’s used for medical decision-making, it should meet the same criteria,” Wilson said.

“I think the standard should be the same across the board no matter in what context the monitor is being used,” Gooden said.

In addition to this stance, one panelist considered the growth of remote pulse oximeter use, emphasizing the importance of using the same proposed premarket trial design and performance definition for OTC devices.

“Being forward thinking, there’s increasing remote monitoring happening from the home streamlining information to care teams or allowing patients to self-monitor and manage their health, so I think these need to be held to the same standard,” Anne Whitney Brown, MD, transplant pulmonologist at Inova Fairfax Advanced Lung Disease and Transplant Program, said.

As the consumer representative, Brummert noted that the average consumer is unaware that pulse oximeters show disparate performance for darker skin pigmentation, so OTC pulse oximeters may be dangerous if they are not tested in the same way as prescription devices.

“I think for over the counter, it needs to be pulled off the market because we’re looking at a dangerous situation here, or we should put this behind the counter where a pharmacist can adequately explain what the issues are with this, and then the consumer can make an informed decision about it,” Brummert said.

Notably, the importance of alerting the consumer on what each pulse oximeter is used for — medical decision-making or exercise/wellbeing — was made known to the FDA.

“I would just emphasize that if there’s true medical decision-making on an individual at home, then the device should have medical grade outputs,” Wilson said. “Not every pulse oximeter manufacturer that sells over the counter needs to apply for and gain that status, but those that do, the consumer should know whether they’re using a device that works fairly well for exercise and well-being but is not as rigorous for medical decision-making.”

When asked about what should be included on the labels of OTC pulse oximeters, Lewis emphasized simple language indicating whether the device meets or does not meet the FDA’s standards.

“Having standard language and a clearly defined box almost like we have nutritional labels ... that it looks the same every time, something along the lines of, ‘Medical pulse ox devices meet certain standards. This does in bold or does not in bold meet those standards.’ Very, very simple,” Lewis said.

Another consideration for labeling brought up by the panel focused on accessibility and making sure that information about the pulse oximeter is presented in ways beyond the box of the device.

“There are good ways that exist to get very useful information at the point of care, including potentially videos, animations, instructions in the language that is needed,” Goldman said. “If that’s the focus for over the counter, it’s really helping to explain to people what to do, how to use it, signs of problems and when to go to the hospital or get medical help.”

At the end of the meeting, Wilson called on the FDA to move forward considering the panel’s input on their proposals.

“Let me just encourage the agency to go forward with the recommendations where there has been a pretty good agreement,” Wilson said. “Let’s not let what we can’t do get in the way with what we can do. It’s been quite some time since this has been brought up, and I think the public is looking for some action.”

References:

• FDA executive summary: Performance evaluation of pulse oximeters taking into consideration skin pigmentation, race and ethnicity. https://www.fda.gov/media/175828/download. Accessed Jan. 31, 2024.