Achieving Change for Texans
Evaluation

Net Impacts Through December 1997

Executive Summary


 
 
 

Deanna T. Schexnayder

Jerome A. Olson

Daniel G. Schroeder

Alicia M. Betsinger

Shao-Chee Sim
 

December 1998
 

Center for the Study of Human  Resources

Lyndon B. Johnson School of Public Affairs
The University of Texas at Austin
3001 Lake Austin Blvd., Suite 3.200    Austin, TX  78703    (512) 471-7891

This report was prepared with funds provided through Interagency Agreement 7241001 from the Texas Department of Human Services to the Center for the Study of Human Resources at The University of Texas at Austin.  The views expressed here are those of the authors and do not represent the positions of the funding agencies or of The University.



Executive Summary


In 1995, the Texas Legislature enacted H. B. 1863, which formed the basis for Texas’ waiver from existing Federal laws governing the Aid to Families with Dependent Children (AFDC) program.[1]The Texas waiver, officially known as the Achieving Change for Texans (ACT) demonstration, aims to assist participants to achieve independence from welfare through an increased emphasis on employment, training, temporary assistance and support services.  It includes four primary components:

·Time-Limited and Transitional Benefits (TL)

·Responsibilities, Employment and Resources (RER)

·Incentives to Achieve Independence (IAI)

·TANF One Time Payments

The ACT demonstration was first implemented in June 1996 and is scheduled to continue operating through March 2002.  The Texas Department of Human Services (DHS) is responsible for implementation and oversight of the ACT demonstration.

The evaluation of the ACT demonstration consists of three approaches:  a process evaluation, conducted by the DHS Office of Program Analysis and Evaluation; an impact analysis, conducted by the Center for the Study of Human Resources (CSHR) of the LBJ School of Public Affairs at the University of Texas at Austin; and follow-up interviews, conducted by the School of Social Work at the University of Texas at Austin, with persons who reached their time limits or who elected to receive TANF One Time payments instead of entering TANF.

This report includes net impacts of the ACT demonstration from its inception in June 1996 through December 1997, and describes early impacts of the time limits and RER experiments on welfare dynamics, client self-sufficiency, participation in workforce development programs, and use of subsidized child care services.[2]  Later reports will extend the period for which net impacts are computed, and also will measure the impact of the demonstration on children, including their education, immunization, and need for child protective services.

Demonstration Components Included in this Analysis

Time-Limited and Transitional Benefits

This statewide initiative limits the number of months that able-bodied adult caretakers can receive TANF benefits.  It provides 12 months of transitional Medicaid and child care for all persons reaching their time limits and 18 months to persons who voluntarily participate in the Choices program.

The time limit placed on each caretaker’s TANF benefits is based on his/her work history and educational attainment.  The most job-ready clients (Tier I) are eligible for up to 12 months of cash assistance after notification of an opening in the Choices program[3].  Less job-ready clients (Tier II) may receive up to 24 months of benefits after Choices notification, and the least job-ready clients (Tier III) may receive as many as 36 months of cash assistance.[4]

Adult clients who exhaust their time-limited benefits are disqualified from receiving TANF for five years.  Exemptions to this freeze-out period are granted if local economic conditions or severe personal hardships exist which prevent the client from remaining independent of TANF.

The time limits experiment is being evaluated in Bexar County.  Adult caretakers assigned to the experimental group are subject to both time limits and RER provisions while control group members must only meet RER requirements (see below).  Implementation of this experiment began in June 1996.

Responsibilities, Employment, and Resources in Choices Counties

The Personal Responsibility Agreement (PRA) is a central feature of RER that requires adult TANF applicants and recipients to comply with specific responsibilities as a condition of TANF eligibility.  Responsibilities include:

·compliance with Choices program participation requirements;

·cooperation with child support and paternity establishment efforts;

·completion of regularly-scheduled Texas Health Steps screenings for children;[5]

·compliance with immunization requirements for pre-school children;

·compliance with school attendance policies for children; and

·participation in parenting skills training classes if referred.

Under the PRA, parents also must:

·not voluntarily quit a paying job of at least 30 hours per week; and

·refrain from selling or abusing illegal or controlled substances or abusing alcohol.

Clients who fail to comply with PRA requirements without good cause will receive a financial penalty.  Failure to comply with Choices participation or child support cooperation requirements results in a $78 dollar penalty per month of non-compliance for single-parent families, and a $125 per month penalty for two-parent families.  Failure to comply with other PRA requirements carries a $25 per month per non-compliance penalty, with a maximum penalty of $75 per month.

Other RER provisions include the disregard of children’s earnings and resources in the calculation of family benefits, increased resource limits permitted for eligibility determination, and the elimination of the work history requirement and 100-hour work rule for TANF-Unemployed Parent (TANF-UP) families.

RER is being evaluated in four offices in Choices counties (Beaumont, Odessa, the Dillon office in Corpus Christi, and the Clint office in El Paso).  Persons in these locations who are assigned to the experimental group must comply with both RER and time limit provisions.  Neither of these welfare reform provisions are applicable to control group members, who were subject to sanctions under the pre-welfare-reform rules..[6]RER in these sites was implemented in June 1996, with time limit provisions added in January 1997.

Responsibilities, Employment, and Resources in Non-Choices Counties

Many rural counties in the state currently do not offer Choices services to TANF recipients.  The RER experiment is being evaluated in four offices in Non-Choices counties (Hondo, Huntsville, Lockhart and Luling). RER provisions are identical to those described above except for those related to Choices participation.  In these sites, experimental group members are subject to RER provisions while control group members do not have to meet RER requirements, but were subject to pre-welfare-reform rules. Because time limits are tied to an offer of participation in the Choices program, neither group is subject to time limit provisions.  RER in Non-Choices counties was implemented in January 1997.

Research Questions

Each of the treatments is expected to influence welfare recipients’ behavior in the areas of welfare dynamics, client self-sufficiency, participation in workforce development programs, education and immunization of children, the use of subsidized child care and child protective services.  A number of variables for each of these research questions has been developed to measure the demonstration’s effect on these measures.

Methodological Approaches

Data Sources

The research data set for this evaluation was created by linking a number of administrative data files on program participation, demographic characteristics, and outcomes at the individual and case levels across agencies, and programs and over time.  This report includes data collected from the Texas Department of Human Services (DHS) and the Texas Workforce Commission (TWC).  Data from most sources were collected from June 1994, two years prior to the beginning of the ACT demonstration, through December 1997.

Data needed to measure education, immunization, and child protective services outcomes were not available for this report.  Net impacts on those measures will be incorporated into future reports as the requisite data become available.

Statistical Methods

The impact analysis utilizes an experimental research design in which persons were randomly assigned to experimental and control groups when they either began receiving TANF benefits or were recertified for benefits.  Most of the analyses were performed using data covering the entire experimental and control populations.  For some research questions, additional analyses were conducted for subgroups of the population.

Tests of random assignment were conducted to verify that any differences between the experimental and control groups occurred only by chance.  After determining that each experiment passed the tests of random assignment, unadjusted net impacts were computed by comparing the differences between the means of the experimental and control groups.  These results were then adjusted to account for demographic differences between the groups to produce adjusted net impacts.

Potential Limitations of Analysis

Some of the factors that could limit the usefulness of this analysis include:

·The short time from the beginning of the demonstration to the date of this report;

·Different lengths of time available to follow evaluation participants due to their entering the evaluation in different time periods;

·The inability to obtain some of the administrative data files needed to analyze several of the research questions; and

·Possible contamination of treatments caused by caseworkers, by word of mouth or news reports, or by other agencies’ non-experimental treatments.

Most of these limitations can be overcome by allowing more time to elapse prior to completing this analysis.

Summary of Findings from Each Experiment

From June 1996 through December 1997, a total of 39,116 cases were assigned to either the control or experimental group for one of the ACT components.  Impact findings through December 1997 for each component of the demonstration are summarized below.

Time Limits

The time limit experiment was implemented in June 1996.  By the end of December 1997, 25,552 cases were assigned to participate as either experimental or control group members.  Test of random assignment revealed no statistically significant differences between the two groups.

Very few differences in outcomes between experimental and control groups in the time limits experiment were observed for the period of study included in this report.  After analyzing welfare dynamics, self-sufficiency, participation in workforce development services, and use of subsidized child care, positive net impacts were observed only for transfer from TANF cash benefits to transitional benefits and the percentage of cases using subsidized child care each month.[7]The magnitude of both of these impacts were quite small (0.9% for transfer to transitional benefits and 0.25% for cases using subsidized child care monthly).  Approximately 13 percent of all exits for both the experimental and control groups occurred in the month of or the month following the assessment of a penalty for failure to comply with PRA provisions.

By the end of December 1997, only 92 persons had exited TANF in Bexar County because of reaching their time limit.  Due to the length of time needed for time limits to take effect for Tier II and Tier III recipients, more time needs to elapse before the true impacts of this experiment can be measured for these recipients.

RER Choices Experiment

From June 1996 through December 1997, 9,566 cases were assigned to participate in the RER Choices experiment as either experimental or control group members.  Tests of random assignment revealed no statistically significant differences between the two groups.

In the first 19 months of the RER experiment in Choices counties, a statistically significant difference in the proportion of time spent on welfare was observed, with the experimental group spending less time on TANF than their counterparts in the control group.  There was also a statistically significant difference in the rate of transfer to medical assistance-only status, transitional benefits-only status and to payee-only status.  The experimental group had higher rates of transfer to medical assistance-only and transitional benefits-only status, and lower rates of transfer to payee-only status, relative to the control group.  These patterns contribute to the finding of a relatively small difference in the overall caretaker exit rates for the two groups, with 60.8 percent of caretakers in the experimental group exiting welfare compared to 58.2 percent for the control group.  Penalty-related exits[8] comprised 15 percent of all exits for experimental group members.

A statistically significant difference in the rate of welfare recidivism in the year following exit was observed between the experimental and control groups, with the experimental group members experiencing a lower rate (34.6 percent) than control group members (38.7 percent).  Although experimental group members left and remained off TANF at higher rates than control group members, there were no  statistically significant differences in employment rates or earnings between the two groups, based on earnings data from UI wage records.  A small but statistically significant difference in the rate of Choices participation was observed, with members of the experimental group having lower Choices participation rates than control group members due to lower levels of participation in education and job search activities.

Finally, although there was no statistically significant difference in the rates of child care usage between the two groups, there was a statistically significant difference in the amount of subsidy per month.  The experimental group members used a larger amount of child care subsidy ($199) per child-month than their counterparts in the control group ($190).  This finding, when coupled with the finding of higher transfers from cash assistance to transitional benefits, suggests that experimental group members may be using the larger child care subsidy to assist in their transition from welfare to work.

Taken together, these findings suggest that the combination of RER and time limits in the RER Choices experiment may be causing a number of caretakers to leave and remain off the TANF rolls in the early months of the experiment.  However, based on UI wage records, there is no evidence as yet that these persons are becoming employed at higher rates.  These measures, along with additional measures of child well-being, will continue to be examined in future reports.

RER Non-Choices Experiment

The RER experiment in Non-Choices research sites was implemented in January 1997.  By the end of December 1997, 1,358 cases were assigned to participate as experimental or control group members.  While no statistically significant differences were observed between the sizes of the two groups, there are some differences in the demographic characteristics of the groups.

Analysis of welfare dynamics for the first year of the RER experiment in Non-Choices counties showed a statistically significant difference in the proportion of time spent on TANF between the experimental and control groups, with experimental group members spending less time on TANF than their counterparts in the control group.  However, no statistically significant differences were found for any of the other welfare exit measures.  The inconsistency between results for the proportion of time spent on TANF and other welfare exit measures is probably due to the relatively short time period available in which to calculate welfare exits and the small sample size of participants in this experiment.  Approximately five percent of all exits were penalty-related.  As of December 1997, insufficient time had elapsed to calculate welfare recidivism rates for this experiment.

No statistically significant differences between the experimental and control groups were found for employment, earnings, or any of the child care measures.  A statistically significant difference in the rate of JTPA participation was observed, with experimental group members having lower JTPA participation rates than control group members. While it was statistically significant, the magnitude of the difference between the two groups was not very large—amounting to less than 0.5 percentage points.[9]

Primarily due to the limited time period included in this report and the small sample size in this experiment, it would be premature to infer much from the interim findings for this experiment.

Future Reports

Future evaluation reports on the ACT demonstration will report impacts on all of the measures included in this report, as well as impacts of the demonstration on various children’s outcomes.  The final report, due in May 2002, will include impacts through December 2001.



[1] Since the passage of H.B. 1863, the AFDC program has been replaced by a new cash assistance program called Temporary Assistance For Needy Families (TANF).  This report will use the term, TANF, to refer to Texas’ cash assistance program for welfare recipients.
[2] The TANF One-Time payment component is not included in the impact analysis because it was not implemented as a randomized experiment.  Implementation of Incentives to Achieve Independence (IAI) has not yet occurred.
[3] The Choices program has replaced the Texas JOBS program.
[4] The 36-month “clock” does not begin “ticking” until twelve months after an in-depth assessment has been completed for Tier III individuals who are offered an opportunity to participate in the Texas Choices program.
[5] Texas Health Steps (THSteps) is the Texas version of the Medicaid program known as Early and Periodic Screening, Diagnosis and Treatment (EPSDT) Program.  The mission of the THSteps is to provide preventive and primary health and dental care to Medicaid-eligible youth from birth through 20 years of age.  Through THSteps, clients receive regularly scheduled medical and dental checkups at no charge, as well as treatment for problems discovered during the checkups.
[6] While the Clint office is located in a Choices county, it does not currently offer Choices services due to remoteness. Because of this difference from other RER-Choices sites, results from the Clint office are not reported with other RER-Choices results.
[7] Transitional benefits include cases in program types 7, twelve months post MEDICAID resulting from increase in earnings or combined increase in earnings and child support, 20, four months post MEDICAID coverage resulting from increase in child support, 29, twelve or eighteen months transitional MEDICAID after expiration of TANF time-limited, and 37, twelve months post MEDICAID coverage resulting from loss of 30+1/3 disregard.
[8] For the purpose of this report, penalty-related exits are defined as exits from TANF accompanied by a penalty in the same month or prior month.
[9] Statistical procedures have been applied to determine whether observed differences between the control and experimental group are due to chance or are a consequence of the experimental treatments.  If the difference is large enough that it may not be attributed to chance, then the difference is said to be “statistically significant.”  Because of the large sample sizes used in some of the experiments, the statistical estimates of the experimental effect are unusually precise relative to most social science research.  Because of this great precision, it is often the case that an estimated difference may be “statistically significant” but still be quite trivial.