Dr. Chitra Lele, Chief Scientific Officer, Sciformix

The Challenges of Capturing Reliable and Validated PRO Data

Dr. Chitra Lele Ph.D.

[This post is the follow-up to the post on the emerging importance of patient reported outcomes (PROs) in enabling better decision making]

Assessment of Quality of Life (QoL) and other Patient Reported Outcomes (PROs) has become common in clinical trials and also as part of post-approval studies to provide the ‘patient voice’ in evidence on treatment effectiveness. PROs are relevant to many primary care research questions and play a significant role in drug approval and labeling decisions. Thus, it is crucial to have a robust plan for the capture and analysis of PRO data, which sufficiently addresses all challenges of capturing reliable and validated data. Both logistical and scientific issues should be addressed to ensure that the PRO data is of a high quality so that the value of these data can be fully realized.

Collection and measurement of real-world data is not controlled and can be done anywhere, which is why the veracity of real-world data comes into question. The data is sourced through computerized databases like Electronic Health Records (EHRs) and Patient Reported Outcomes (PROs), and is difficult to quantify and calibrate. Capture of real-world data needs specific tools that lend scientific validation to the assessments of physical and mental health, and measure well-being and health from a patient’s perspective. There are very few tools that offer such a standardized approach to measure health outcomes for individuals and large populations and can be considered statistically valid patient-centered measures.

In December 2009, the US FDA released guidance for the industry on PRO measures[1]. The guidance reviews and evaluates PRO instruments used to support claims in approved medical product labelling. A PRO instrument (such as a questionnaire plus the information and documentation that support its use) is defined as a means to capture PRO data used to measure treatment benefit or risk of medical products. The guidance states that findings measured by a well-defined and reliable PRO instrument in appropriately designed investigations can generally be used to support a claim in medical product labeling if the claim is consistent with the instrument’s documented measurement capability.

The evaluation of a PRO instrument includes the following considerations:

  • The population enrolled in the clinical trial
  • The clinical trial objectives and design
  • The PRO instrument conceptual framework
  • The PRO instrument measurement properties

For a PRO instrument to be considered a sufficient measure to support medical product labeling claims, it needs to display satisfactory characteristics, conceptual framework, and other measurement properties. Thus these four factors guide the choice of a PRO instrument. The characteristics (for e.g., number of items, responder burden, scoring etc.) that are reviewed are listed in the guidance and the iterative process of development of a PRO instrument is also described. The conceptual framework is stated to be based on the concepts that are to be measured and the intended population.

PRO instruments’ measurement properties included in the review are:

  1. Reliability (intra- and inter-interviewer reliability, internal consistency)
  2. Validity (content validity and construct validity)
  3. Ability to detect change.

Content validity is the extent to which the instrument measures the concept of interest. Content validity is supported by evidence from qualitative studies that the items and domains of an instrument are appropriate and comprehensive relative to its intended measurement concept, population, and use. Content validity is specific to the population, condition, and treatment to be studied. Content validity is evaluated based on review of the process of derivation of the items, interview or focus group transcripts etc.

The FDA will determine if a labeling claim can be substantiated after reviewing the documentation of PRO instrument development and testing in conjunction with clinical trial results. The guidance document lists the type of PRO information sponsors should provide to the FDA to facilitate instrument review.

Problems may occur when the sponsor or contract research organization (not the clinical investigator) wants to maintain a direct control over source data. Sponsors have to ensure high data integrity during network attacks and software updates by establishing robust security controls and systems. Capture of PROs may also be web-based or through IVR-compatible tools.

A commonly used PRO instrument to measure functional health status is the Short Form (SF) Health Survey (SF-36v2® Health Survey of 36 questions and the shorter SF-12v2® Health Survey of 12 questions). This measures health status across eight domains and are summarized into physical and mental health scores.

References:

  1. US FDA Guidance for Industry on PRO measures, December 2009, http://www.fda.gov/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/default.htm [Accessed 12/01/2015]