Has the Social Safety Net Been ALTARED? (continued)

Research Strategy and Methods

The client data addresses the first two research questionsÑspecifically, who receives social services from FBOs and the impact of welfare reform on the receipt of services from FBOs. In contrast, the case-matched provider data speak to the organizational capacity of FBOs, relative to non-FBOs, to help welfare recipients meet work requirements and time limits.

The client data were analyzed using bivariate and multivariate inferential statistical methods. We compared the characteristics of current and former welfare recipients who had and who had not received help from FBOs. These bivariate comparisons were further investigated using multivariate statistical models that tested the impact of welfare reform on receiving help from an FBOÑcontrolling for basic socioeconomic characteristics of each respondent. Given the randomized experimental design of the client survey, this approach provided a rigorous test of the effects of welfare reform on the likelihood that clients seek help from FBOs.

In our analyses of welfare clients, our dependent variable was receipt of one of nine specific types of help from an FBO.34 Since these are dichotomous dependent variables, we used maximum likelihood (ML) logit regressions for unordered categorical dependent variables with categorical and continuous independent variables.35 The agency data were analyzed using a case-match method.36 In general, case-matching is a more rigorous technique for examining causality in contrasting groups within a population. Case-matching techniques can be incorporated with multivariate regression techniques and are useful for demonstrating differences across groups of cases with descriptive statistics and analysis of variance (ANOVA). Utilizing this technique, we were able to match 37 FBOs with 37 similar nonprofit, non-FBOs. Three control variables were used for case-matching: organizational age (number of years since founding), total number of paid employees (full- and part-time) in 1998, and organizational budget for fiscal year 1998.37 We used these variables to generate predicted values (also known as propensity scores) to guide our matching process.

The propensity score is the predicted probability calculated by running a ML logit regression with organizational type (FBO or non-FBO) as the dependent variable. Three factors thought to predict FBO status are used as control (or independent) variables and include organizational age, total number of paid employees, and the organization's budget.

The matching process follows the randomized, nearest neighbor method. This means that the program assigns a random number to each of the treatment cases, which in this analysis are FBOs. Then, starting with the largest random number, each treatment case is matched to a control (or comparison) case based on the propensity score (i.e., the predicted values) of plus or minus one standard deviation. With each match, the comparison case is eliminated as a possible match for other treatment cases. Cases were matched only if they shared a common service type (counseling, transportation, legal and civil rights, housing, food and health, childcare and youth services, workforce development and education, or intermediary service for other social service agencies) and service area (urban, suburban, or rural).38 Profiles of the case matches are presented in the Appendix, Table A1.

Only those cases selected through the matching process are included in the analysis. The research strategy is to compare matched cases on a number of dimensions, including organizational changes since the inception of welfare reform, staffing patterns, and organizational networks. To test whether differences in these two groups exist, t-tests are performed to determine whether differences in means are real or by chance. When statistically significant differences in organizational changes since welfare reform are found, the impact of FBO status is estimated using multivariate methods, controlling for observed differences in staffing patterns, organizational characteristics and organizational networks. Qualitative responses to the interviews were transcribed and analyzed using NUD*ist software for the 37 matched cases.

Taken together, these analyses provide a systematic investigation of how welfare reform has affected FBOs by using both client and agency data to describe clients who receive help from FBOs, while assessing the impact of welfare reform on demand for services from FBOs, and the organizational capacity of these organizations to help welfare recipients meet work requirements and time limits. Thus, our results provide a context for the upcoming debate on TANF re-authorization, including whether states should continue to be encouraged to contract with FBOs to meet the needs of current and former welfare recipients.
| back | next |