Skip to Content [0]

Skip to Navigation [1]

About the Medical Group Ratings

 

"Patient Experience of Care" Scoring Methods

The Patient Experience of Care quality measures are taken from a patient survey, the 2005 California Consumer Assessment Survey (CAS). The quality measures are based on the patient-reported experiences of care and service for adults aged 18 to 64 who were commercially insured HMO members. A number of the survey questions were organized into sets of related topics. Of the topics that were evaluated the following five were chosen as the best indicators of performance for consumers:

  • Patient Rating of Care Experience
  • Communicating with Patients
  • Getting Treatment and Specialty Care
  • Timely Care and Service
  • Coordinating Patient Care

Medical groups were rated on these five summary performance topics by grouping sets of related questions ,such as four questions on doctor communications with patients - and combining the scores for these questions into a summary score. The five summary quality performance ratings are based on the mean score of the patient's responses to the survey questions. For most questions, responses are scored on a 3-2-1 basis: 3 means the patient "always" had a positive experience with a particular need like getting an appointment; 2 means the patient "usually" had a positive experience and 1 means the patient "sometimes" or "never" had a positive experience. Several questions were scored by assigning a 3 for patients who reported "not having a problem," a 2 for patients reporting "having a small problem" or a 1 for patients reporting "having a big problem." Each patient's responses are combined to create a per-patient mean score and the average of all of a medical group's patients' scores is calculated to create a medical group score. The scores represent the average or typical experience that that medical group's patients reported.

The Patient Rating of Care Experience was scored differently. This summary performance topic is based on a single question. Here the patient is asked to rate their overall care on a scale of 0-10 with zero being the worst possible and ten the best possible experience. Each medical group's score was obtained by calculating the average of all of its patients' responses to this question. Though this topic is reported as the summary CAS survey measure it is based on a single, stand-alone question in the survey - it is not a "roll-up" or combination of multiple survey questions.

The scores are adjusted for a set of patient characteristics- age, mental health status, education, overall health status, race/ethnicity and language spoken-that have been shown to influence patients' ratings of their care experience. These adjustments allow us to make apples-to-apples comparisons across groups whose patients may differ. Through the adjustments, which result in very small changes in a medical group's scores, we can represent the groups results' as if they all had a similar mix of patients.

Performance scores are not available for all medical groups for several reasons. Some medical groups are labeled "Did not report" - the medical group did not participate in the CAS project to have their performance publicly reported and compared to others. Some medical groups have "Too few patients in sample to report" labels for one or more performance topics - too few patients answered a particular question to report that result. This typically occurs with some smaller medical groups that have fewer patients; their results may not be accurate because of the low number of completed surveys. The "Too few patients in sample to report" also is used in instances in which the question is about an experience that is relevant to fewer patients in that group and we are not confident that the results represent typical patient experiences with that medical group.

Back to top

"Patient Experience of Care" Grading Methods

Medical group patient experience scores are organized into five summary topics that are described above. Each medical group's performance is reported using summary topic grades that represent the average experiences of the group's patients who completed the survey.

Grading a summary topic requires several steps. First, a score is calculated for each question by determining the average response across all of the patients from that medical group who answered the survey. Next, the scores for a small number of questions are combined into a single summary score by calculating the average across those questions. Questions about related parts of a patient's experience - like getting an appointment and getting a call back when you phone with a medical question - are organized into a summary topic. Typically the results of 4-8 questions are combined, given equal weight to each question, into a summary topic score. Once the average of these question scores is calculated the summary topic score is given one of four performance grades:

Excellent
Generally, this grade means that more than 3 of every 4 patients reported favorable experiences such as the patients' doctors listened carefully to them.

Good
Generally, this grade means that somewhat more than two of every three patients reported favorable experiences such as getting a call back from the doctor's office about the patient's test results.

Fair
Generally, this grade means that about two of every three patients reported favorable experiences such as getting care as soon as the patient needed when ill or injured.

Poor
Generally, this grade means that almost half or more of the patients did not report having a positive experience.

For more information on scoring and grading formulas see OPA's Managed Care Research & Reports page, then click on the link entitled, "2005 HMO Report Card Scoring Guidelines."

Back to top

Source of the "Patient Experience of Care" Measures

Adult patients from 174 California medical groups and Independent Practice Associations (IPAs) responded to the 2005 California Consumer Assessment Survey (CAS). Patients are asked to rate the care and service provided by their medical group and its doctors and other staff during the past year (2004). The mailed questionnaire was available in English, Spanish, Chinese and Vietnamese. Phone interviews were available in English and Spanish, and were done for some patients who did not reply to the mailed questionnaire. These participating medical groups serve more than ninety percent (90%) of commercial HMO patients statewide.

The patient surveying is done in a way intended to show the typical experience of patients in each medical group. The patients who were surveyed were randomly drawn from the medical group's full list of eligible, commercially insured HMO health plan members. Patients who had a medical visit in 2004 and who were enrolled in the medical group for the past year were included on the survey list.

Of the 900 patients who were surveyed for each medical group, roughly 320 patients per medical group responded. More than 55,000 patients responded statewide, representing thirty-five percent (35%) of those who were invited to participate. The number of patients was large enough to get a clear picture of the medical group's overall performance; however, your own experience with a medical group or individual doctor may differ.

Back to top

"Getting the Right Medical Care" Scoring Methods

The Getting the Right Medical Care quality measures are taken from records of patient services that are kept by medical groups and health plans. The measures are based on the services of commercial HMO or Point of Service (POS) members who were medical group patients during 2004. Depending upon the measure, the patient must have been a member of the medical group and health plan for a specific period of time. The medical group has the option of directly reporting the information or of having its contracting health plans report the information. The accuracy of this information is independently checked.

The measures are based on a count of all members who are eligible given that they have a particular health problem or need - like members who have diabetes or who are young children. The quality score is the proportion of these eligible members whose records indicate that they obtained a particular service. The medical group or a contracting HMO supplies the information on whether or not the member received a particular service. The patient services information is gathered from administrative records only; no information is collected from medical records. Administrative records typically include claims, encounter and scheduling systems information.

Some medical groups are labeled "Not willing to report" - the medical group refused to have their performance publicly reported and compared to others. Other medical groups are labeled "Too few patients in sample to report" - there were too few eligible patients to be confident that the results represent actual patient experiences with that medical group. A quality result is not reported if there are fewer than 30 medical group members who are eligible to be included in the measure. This label ("Too few patients in sample to report") also is used for some medical groups for the "Controlling Blood Sugar for Diabetes Patients" and “Controlling Cholesterol” measures because difficulties with collecting or transmitting the lab test results limited the available information. These two measures are available for some medical groups but are not reported for many of the groups.

Quality measures are created for the following seven topics:

  • Childhood Immunizations
  • Asthma Medication
  • Testing Blood Sugar
  • Testing Cholesterol
  • Breast Cancer Screening
  • Cervical Cancer Screening
  • Chlamydia Screening


These measures were combined to create the summary topic score Getting the Right Medical Care. The summary score is based on the medical group's average score across the seven measures. First, scores are calculated for each of the seven measures - the score is the proportion of eligible members who got the recommended service. Then, the scores for all of the measures are combined, giving equal weight to each measure, by calculating the average across the measures. The score is calculated on the measures that are available for a medical group: if a measure is missing, the score is calculated based on the remaining measures. A summary topic score is not produced for medical groups with fewer than four reportable measures.

Back to top

"Getting the Right Medical Care" Grading Methods

Medical groups' Getting the Right Medical Care performance is organized into a single summary topic that is described above. Once the average scores of these measures is calculated the summary topic score is given one of four performance grades:

Excellent
Generally, this grade means that most medical group members, more than 4 of every 5 patients, got the right care - like checking cholesterol levels for people who have heart problems or diabetes.

Good
Generally, this grade means that three of every four patients got the right care - like people with asthma getting medicines to avoid asthma attacks.

Fair
Generally, this grade means that about three of every five patients got the right care - like women getting a PAP smear test to check for cervical cancer.

Poor
Generally, this grade means that fewer than half of the patients got the right care.

For more information on scoring and grading formulas see OPA's Managed Care Research & Reports page, then click on the link entitled, "2005 HMO Report Card Scoring Guidelines."

Back to top

Comparing Medical Group and HMO Ratings

Medical group and HMO ratings should not be compared as these results often differ because the information and the performance measures for each are different:

  • There may be a very different mix of patients whose care is being measured even for similar ratings of medical groups and HMOs,
  • The standards used to measure performance differ for the medical group and HMO,
  • The patient's medical chart is used as an information source for a number of HMO quality measures but the chart is not used for measuring medical group performance,
  • The performance topics that are combined to rate medical groups overall differ from those used to rate HMOs,
  • The information that is kept by medical groups and HMOs is different; so, the steps to check the quality of each differs.

Back to top

California Cooperative Healthcare Reporting Initiative (CCHRI)

The California Cooperative Healthcare Reporting Initiative (CCHRI) is a statewide collaborative of employers, health plans, and providers. Since 1994, CCHRI has provided the public with important information on how well health plans and medical groups provide medical care and service. CCHRI also shares information on member satisfaction with health plans and medical groups and makes the results public so consumers can make informed health care choices.

CCHRI sponsored the 2005 California Consumer Assessment Survey (CAS) project. The CAS survey is part of a national research effort to create a common way to measure patient experience with care at the medical group or office-level. The survey was done by an independent survey research firm under the guidance of CCHRI. Independent experts analyzed and scored the data. The project is supported by the Pacific Business Group on Health (PBGH).

Along with the participating provider groups, the following CCHRI participant organizations provided generous financial support for the 2005 Consumer Assessment Survey project:

  • Aetna Health of California Inc.
  • CIGNA Healthcare of California
  • Blue Cross of California
  • Blue Shield of California
  • Health Net
  • Kaiser Foundation Health Plan
  • PacifiCare of California
  • Universal Care
  • Western Health Advantage

Back to top

Integrated Healthcare Association (IHA)

The Integrated Healthcare Association (IHA) is a California leadership group of health plans, physician groups, and health systems, plus at-large academic, purchaser, pharmaceutical industry, and consumer representatives, involved in policy development and special projects around integrated health care and managed care.

The IHA-sponsored Pay for Performance program generates the measures used in Getting the Right Medical Care. The Pay for Performance program is the country's largest physician incentive program based on quality of care with participation by seven California health plans (Aetna, Blue Cross, Blue Shield, CIGNA, Health Net, PacifiCare and Western Health Advantage) with 6.2 million commercial HMO enrollees and 35,000 doctors. The program promotes a common set of measures that evaluate clinical quality of care for preventive services - such as breast cancer screening and childhood immunizations - plus treatment of chronic conditions such as diabetes and asthma. Patient experience is evaluated by asking patients who saw their doctors during the year for their views on factors such as communication with their doctor and access to specialists. Finally, medical groups were rated on their investment and adoption of information technology (IT) to support patient care. This includes building patient registries for those with chronic illnesses and using physician or patient reminder systems at the point of care.

Back to top