This is the version of the paper as it was presented at the AAPOR meetings. Since the meetings, the study was completed. The final results can be obtained from me. I wanted this version to be the same as presented and sent earlier to those who requested it. John
In this paper, we present the preliminary analysis from a survey that gathers health insurance and medical cost information and job histories. The survey measures the impact of catastrophic illnesses and the availability of health insurance on "job-lock." Job-lock occurs when employees feel they cannot change jobs, or are afraid to change jobs, because they might lose their health insurance benefits. To understand the conditions associated with job-lock, we asked questions about demographic characteristics, health conditions (including the SF-36), family well-being, and current access to health care. A job history for the past ten years was collected from each respondent. In families where there was more than one employed adult, spouses' job histories were collected.
We faced three major data collection problems. First, the survey content required a lengthy interview. Interviews ranged from about one to three hours. Second, in most families we needed to collect job histories from two people, which required cooperation from the respondent's spouse. Third, we knew respondents would experience problems answering questions about health costs. To reduce burden and improve data quality, we mailed a follow-up questionnaire to all respondents.
The questionnaire contained the "fail edit" questions from the telephone interview and some additional health status questions. The preliminary results indicate that the follow-up mail survey is very effective in gathering the information missed in the telephone interview. The mail survey allows the most knowledgeable person to answer the insurance and medical cost questions and improves data quality over the telephone interview alone.
In this paper, we present the preliminary analysis of data from a project designed to gather health insurance information, medical treatment costs, and job histories. The survey measures the impact of catastrophic and chronic illnesses and the availability of health insurance on "job-lock." "Pushing the envelopes" refers to the use of a unique mail questionnaire to gather data that is troublesome to collect in a telephone interview. The procedures described in this might be used by other researchers when faced with a difficult data collection task.
Policymakers in the US are currently engaged in a thorough and far-reaching debate about the scope and availability of health care. Some proposals in the current health care reform movement are aimed at providing all citizens with full access to health care. While most arguments about the need for universal health care focus on the long-term benefits of a healthy population, there is also a significant debate on the economic value of universal health care. In particular, many argue that the lack of universal health care fosters welfare dependency. Others argue that the optimal allocation of labor is impeded when workers believe they are unable to change jobs because of "job-lock." Measuring job-lock and the conditions that create it are described in this paper.
It is suspected but not fully documented that persons who experience catastrophic illnesses find it difficult to obtain health insurance or find their health insurance has severe limitations in coverage and/or benefits. These restrictions on health insurance are further assumed to lead to job-lock. Job-lock occurs when individuals, spouses or parents perceive that they cannot change jobs, or are afraid to change jobs, because they fear that changing jobs will cause them to lose their health insurance coverage or that the insurance provided by a new employer would not provide sufficient benefits due to pre-existing medical conditions.
The empirical evidence for job-lock comes from two different sources and the empirical support for the magnitude of job-lock is mixed. About 65 percent of the under age-65 population in this country is provided health insurance through employers, so that job changes have the potential for major impacts on health insurance coverage. One source of data about job-lock are polls that ask respondents about their experiences with job changes and the impact of health insurance coverage on their job mobility decisions. Thirty percent of the respondents to a CBS/New York Times poll said they or someone in their household have stayed in a job mainly to keep health benefits. Ten percent of the respondents in a Los Angeles Times poll said they would like to leave their jobs but are afraid of losing health insurance benefits (both cited in Penrod, 1993). About 19 percent of persons with employment-related health insurance said their fear of losing health insurance benefits keeps them from changing jobs (cited in Cooper and Monheit, 1993). These numbers reflect potential experiences with job-lock but they do not adequately measure how much job-lock exists because the respondents are often merely speculating about the potential impact of employment changes.
A second form of evidence about job-lock comes from the health economics literature (Madrian, 1992; Holtz-Eakin, 1994; Penrod, 1993; Cooper and Monheit, 1993). These studies, too, provide mixed information about job-lock. For example, Madrian estimated that job-lock reduced job turnover by about 25 percent annually for married men. Using a different survey but using the same measure of job-lock, Holtz-Eakin found no evidence of job-lock. Cooper and Monheit (1993) found that employment-related health insurance created job-lock. Penrod (1993) estimated that job-lock happens to a small proportion of employed persons. These studies measured job-lock with various economic indicators of the value of health insurance and with differences in job-turnover between those with and without employment-related health insurance coverage. None of these studies asked questions directly focused on job-lock.
In our research, we are using both measures of job-lock. We ask respondents to describe their experiences with health insurance coverage. In addition, we ask questions of the respondents that will be used to estimate the amount of job-lock experienced by the respondents. We expect to provide a more comprehensive analysis of job-lock. This paper is not about job-lock, per se, but about how we attempt to measure it.
Collecting the information needed to study health insurance coverage and job-lock is difficult because it requires gathering many details related to employment and experiences with health insurance. To understand more fully the conditions associated with job-lock, we needed to ask questions about the demographic characteristics of the respondents and their families, health conditions (including the SF-36), family well-being, and current access to health care. For example, a job history for the past ten years was collected from respondents. In families where there was more than one employed adult, the spouses' job histories were collected. Questions about cost of medical care and health insurance payments were also included.
The study was designed to overcome the problems associated with the previous research. Not only did we ask questions about perceived job-lock, but we also collected information on health insurance coverages over time. To more specifically address the problem of job-lock, which really affects very few people who have a chronic or catastrophic illness or a a family member who does, we sampled only those most likely to experience job-lock. This project does not attempt to measure the amount of the job-lock in this country. Rather, it attempts to measure job-lock among those most likely to experience it. The results from this project can contribute to the discussion of the need for health care reform and can contribute significantly to the design of a new health care system.
Four groups of people of working age (18-65) were sampled for the study. These were:
1. Catastrophic illness samples:
a. breast cancer patients living in or around Marion county
Indiana - provided by Indianapolis area hospitals.
b. testicular cancer patients from throughout Indiana - provided
by Indiana University Hospital.
2. Economically disadvantaged sample: this sample was derived
from persons who participated in in-home interviews from Saint
Joseph County, Indiana. The sample was drawn from census tracts
that had high proportions of minorities and low income households.
3. Chronic illness sample: parents whose children have been
treated for a chronic illness at the Riley Hospital for Children,
a tertiary care treatment facility in Indianapolis, Indiana.
At this point in the study, only the cancer patient portion of the project is near completion, so only those results are presented here. We are using the same procedures, for the most part, with the other parts of the study.
We faced three major data collection problems. First, the content required a lengthy interview. We initially estimated the questionnaire would require up to two hours to conduct in families with two employed persons. Second, in most families we needed to collect job histories from two people, which required cooperation from the respondent's spouse. Third, we knew respondents would experience problems answering questions about medical and health insurance costs.
We designed a set of procedures aimed at giving respondents multiple opportunities to provide the needed information. The study procedures included a presurvey letter, a telephone survey, a mail questionnaire, incentives, a follow-up telephone call, and a follow-up postcard, and a follow-up mail questionnaire.
We sent each respondent a presurvey letter describing the survey. The letter informed respondents about the information needed to answer the health insurance questions. The letter also informed the respondent about the length of the interview, and it asked the spouse to participate. In the presurvey letter, we explained that we would be asking some difficult questions about the respondents' insurance and employment history. We asked them to gather any health insurance records, medical treatment costs records, and employment records that would help during the interview, and to have them available when we called.
From the outset, we determined that we could not fit all the questions into the telephone instrument. The survey length would be very tiring for both the respondent and the interviewer. We decided to put some of the less critical questions, such as the SF-36 and co-morbidity questions, on a mail questionnaire. These questions ask about the respondents' general mental and physical health, including any other serious illnesses they may have experienced. Conducting the telephone interview in shorter sections was a possibility but it would make case management more difficult and we wanted to avoid adding steps to an already complex study.
In our design, we planned to send a questionnaire that contained the "fail-edit" questions from the telephone survey. We programmed the CATI instrument to indicate quickly and clearly when the respondents did not know the answers to the critical questions. The questions involving health care coverage and costs, and the costs of the disease treatment were designated as the fail-edit questions. When the respondent and/or a spouse could not answer any of these questions, we included them in the mail questionnaire. In effect, each respondent received an individualized mail questionnaire that contained the health conditions questions and the unanswered critical telephone interview questions.
During the interview, if the respondent did not know the answers to fail-edit questions, we accepted a "don't know" response. We instructed the interviewers not to probe "don't know" responses to these questions. We also instructed the respondents not to guess when answering the fail-edit questions. We assured them that they could answer them later in a mail questionnaire when they could check their records. Using these procedures, we were able to shorten the telephone questionnaire and reduce the uncertainty experienced by the respondents. These procedures reduced respondent burden and improved the data collection process. Only seven respondents from among the 214 persons who completed the telephone survey were able to answer all the fail-edit questions in the telephone interview.
Depending on a respondent's insurance coverage, job history and marital status, we asked between 25 and 200 fail-edit questions. Some of the questions asked about pre-existing conditions that limit coverage of health insurance, insurance premiums and deductibles. We asked for the ten-year employment history of both respondents and their spouses. Respondents were asked the dollar amount of the total cost of their treatment for catastrophic illnesses. Many of these questions require the respondent to provide exact amounts for insurance policies as much as ten years old.
About two weeks after they had completed the telephone survey, we sent the mail questionnaire to every respondent. If the respondent did not return the questionnaire within a month, we sent a second one. We tested the impact of a $10 incentive early in the mail survey process. During the second mailing, we included a $10 incentive for approximately half of those who had not returned their questionnaire. Inclusion of the monetary incentive appears to have only minimally improved the return rate. Many respondents offered the incentive did not want the money, which offered additional evidence that the incentive was marginally valuable.
Following the second mailing, there were approximately 80 questionnaires that had not been returned. To determine the reasons for nonresponse, we called approximately ten respondents who had not returned them and asked why they had not returned them. Most said they were still unable to answer some of the difficult questions and they did not want to return an incomplete questionnaire. Based on these responses, and because the cancer patient portion of the study was ending, we sent a postcard to the respondents who had not returned their questionnaire. We explained that we needed the answers to the questions that they were able to answer, and asked them to return the questionnaire as soon as they could. We expect additional questionnaire returns before the study finishes.
Job-lock is measured with a complex set of variables that include the estimated costs of health insurance, wages, the estimated costs of treatments provided by health insurance, and job mobility differences over time. These variables are input into econometric models to determine the amount of job-lock. An accurate estimate of job-lock required that we obtain a significant amount of data in the most accurate manner possible. The data from all of the questions we asked are included in the model or used as control variables. Table One contains a preliminary measure of job-lock as reported by the respondents.
Table One: Self-Reports of Job-lock by Cancer Patients
and Spouses.
Respondent Spouse
Fear changing jobs
Before diagnosis 11.8 5.3
After diagnosis 46.2 25.4
Stay in job
Before diagnosis 6.4 5.3
After diagnosis 25.5 7.0
The "fear changing jobs" refers to a question that asked the respondent and/or the spouse if s/he ever was afraid to change jobs because s/he might lose health insurance. Before the cancer diagnosis, only a small proportion said that they were afraid to change jobs. After diagnosis, substantially more people said they did not want to change jobs because of the fear of the loss of health insurance. The second question referred to staying in a job they would like to leave just because it provided health insurance. Since most of the cancer patients were between ages 50 and 64, we expected stability and that they probably would not really be interested in leaving their jobs. We expect that the second question will show more variation among the other groups in the study, especially the parents of children with catastrophic illnesses. We recognize that respondents may not be able to accurately recall pre-diagnosis attitudes, which is why there are multiple measures of job-lock built into the questionnaire.
To date, the respondents have been extremely cooperative in both the cancer patient and Saint Joseph County groups. The presurvey letter advised respondents to gather appropriate health insurance and employment information to consult during the telephone interview. Many respondents made extensive efforts to provide accurate numbers such as gathering medical cost records from storage and insurance agents.
The response rate for the cancer patient group was about 98 percent (only six of 285 possible respondents refused to be interviewed) . In the St Joseph County study, which just completed the telephone portion, the response rate was 92 percent. This is surprisingly high for a low-income, minority, somewhat elderly sample.
Table Two: Cancer Patient Survey Dispositions Total Sample 311 Telephone not in service, deceased, too ill 26 Refused to participate 6 Screened out 65 No cancer 2 Over 65 44 Cancer, but not breast or testicular 19 Completed Interviews 214 Full completion 203 Partial completion (spouse refusal) 11 Mail Follow-up Mail questionnaires sent 214 Mail questionnaires returned 139 $10 incentive sent 41 $10 incentive sent, questionnaire returned 18 No incentive sent 44 No incentive, questionnaire returned 16 Follow-up postcard sent 71 Follow-up postcard, questionnaire returned 5 (postcards sent one week ago)
Most people who returned their questionnaires answered the fail-edit questions. For the easier fail-edit questions, such as the date of first diagnosis with cancer or the type of health insurance, virtually 100 percent of the respondents answered the questions. For the more difficult questions, such as the amount of the upper limit on health insurance policy and the total cost of treatment for cancer, the range of responses is about 80 - 90 percent, depending on the questions. In general, the number of additional questions answered in the mail questionnaire justifies its use.
The mail questionnaire provided additional information on many of the questions that ask about cost data. We were especially concerned that respondents would not know or not have the appropriate records to provide this information to us. We expected there would be few differences in the numbers provided in both modes. Table 3 provides some comparisons of the differences on questions related to cost of treatment for cancer which had the highest fail-edit rate. The cost of treatment questions were included in 169 mail follow-up questionnaires and, to date, we have received 77 questionnaires with the questions answered. Only 55 respondents were able to answer the three parts of the question on the telephone.
Table Three: Mean Costs of Treatment by Response Mode.
Telephone Mail
(55) (77)
Total Cost $40061 $38087
Personal Cost $5429 $3511
Insurance Paid $34910 $34806
Overall, the differences in the responses to the three cost of treatment questions between the two modes are minimal. Most of the difference is between the reported amounts paid personally by each group. The telephone respondents reported about $2000 more in out-of-pocket expenses for the cancer treatment. We cannot determine if the difference is attributable to checking records when responding to the mail survey because many respondents had their records available while answering the telephone interview.
These data indicate that we are able to provide more responses (less item-nonresponse) when two modes are available to respondents. In our telephone follow-up to the mail nonrespondents, this question was mentioned most often as the reason why they were not able to return the questionnaire. Unfortunately, this question is critical to estimating the value of health insurance in the econometric model.
The purpose of the survey is to provide a more accurate estimate of the impact of health insurance benefits on job-lock. The previous research is mixed on its impact. We are attempting to improve measurement by focusing the sample and asking questions that have not been asked previously. In addition, we are using a mixed-mode method to improve data quality.
The preliminary results indicate these procedures are effective in gathering the necessary data. To date, approximately 280 persons were interviewed. Only six persons refused to participate in the study. (A high participation rate was expected because only those who agreed to have their names released by the hospitals to the researchers were included.) Only about ten spouses refused to participate in their part of the survey. Many respondents were enthusiastic about the study, and were prepared for the cost questions. Some reported to the interviewers that they went to their safe deposit boxes to gather the necessary information. In many cases, both the respondent and spouse completed the interview at the same time, and shared information on health insurance and medical costs.
The preliminary results also indicate that the follow-up mail survey is very effective in gathering the information the respondent could not answer during the telephone interview. The mail survey allows the most knowledgeable person to answer the cost questions, even if s/he was not asked them during the telephone interview. Despite the long telephone interviews (they average over one hour), the respondents are willing to return the mail questionnaires. To this point, about 80 percent of the returned questionnaires contain information that was not answered in the telephone interview. Only a seven respondents received a mail questionnaire that did not include at least one "fail edit" question, which indicates how much we improved the quality of data with the mail questionnaire.
We learned many valuable lessons from this complicated study. Putting the fail edit questions into a mail questionnaire was very successful. Without this method of collecting the information, we would have had a great deal of missing or incorrect data. Allowing respondents to comfortably say "I don't know" made the long survey less stressful for both the respondent and the interviewer. The most important lesson learned from this project is that collecting quantitative information to study health insurance and job-lock is possible.
From this experience, we believe it is possible to gather data on topics where the respondents may not have complete information during a telephone interview. "Pushing the envelope" is a term used by test pilots when they are pushing new aircraft past their designed limits. This paper demonstrates that by careful mailings (pushing the envelopes - literally), we can collect more and better data than usually possible. We also figuratively "pushed the envelopes" to gather data that our usual procedures would not allow us to collect.