Study Details for:
Perez-Johnson, Irma, Quinn Moore, and Robert Santillano (2011). Improving the effectiveness of Individual Training Accounts: Long-term findings from an experimental evaluation of three service delivery models, Series: ETAOP 2012-06, Washington, DC: Employment and Training Administration, U.S. Department of Labor. [Maximum versus Guided]

Study URL: http://wdr.doleta.gov/research/FullText_Documents/ETAOP_2012_06.pdf (Link not working?)

Evaluation: Individual Training Account (ITA) Experiment

Program Studied: Maximum Customer Choice versus Guided Customer Choice
See Study Characteristics tab below for more information about this program.

Strength of Evidence: 1-High High (3 of 3)

Populations Targeted General low-income population
Setting Mixed (urban and rural)
Services Provided Case management, Occupational or sectoral training, Training, Work readiness activities
Outcome Domains Examined Short-term employment, Long-term employment, Short-term earnings, Long-term earnings, Favorable Impacts FoundEducation and training, Long-term benefit receipt

Findings

Toggle Short-term employmentShort-term employment
Outcome Treatment Group Comparison Group Impact Findings Strength of Evidence Study Sample Sample Size Data Source and Timing

Employed, quarter 6, %
Adjusted mean = 67 Adjusted mean = 68 -1 No Effect 1-High High (3 of 3) Maximum versus Guided Customer Choice 5,274 UI wage records, quarter 6
Toggle Long-term employmentLong-term employment
Outcome Treatment Group Comparison Group Impact Findings Strength of Evidence Study Sample Sample Size Data Source and Timing

Employed, quarter 22, %
Adjusted mean = 61 Adjusted mean = 62 -1 No Effect 1-High High (3 of 3) Maximum versus Guided Customer Choice 5,274 UI wage records, quarter 22
Toggle Short-term earningsShort-term earnings
Outcome Treatment Group Comparison Group Impact Findings Strength of Evidence Study Sample Sample Size Data Source and Timing

Earnings, quarter 6, $
Adjusted mean = 4,068 Adjusted mean = 4,270 -202 No Effect 1-High High (3 of 3) Maximum versus Guided Customer Choice 5,274 UI wage records, quarter 6
Toggle Long-term earningsLong-term earnings
Outcome Treatment Group Comparison Group Impact Findings Strength of Evidence Study Sample Sample Size Data Source and Timing

Earnings, quarter 22, $
Adjusted mean = 4,912 Adjusted mean = 4,940 -28 No Effect 1-High High (3 of 3) Maximum versus Guided Customer Choice 5,274 UI wage records, quarter 22
Toggle Education and trainingEducation and training
Outcome Treatment Group Comparison Group Impact Findings Strength of Evidence Study Sample Sample Size Data Source and Timing

Earned a certificate or degree from a training program starting within 3 years of random assignment, %
Adjusted mean = 59 Adjusted mean = 53 6 Favorable 1-High High (3 of 3) Maximum versus Guided Customer Choice 2,159 7-year survey

Earned a certificate or degree from a training program starting at least 3 years after random assignment, %
Adjusted mean = 13 Adjusted mean = 15 -2 No Effect 1-High High (3 of 3) Maximum versus Guided Customer Choice 2,159 7-year survey
Toggle Long-term benefit receiptLong-term benefit receipt
Outcome Treatment Group Comparison Group Impact Findings Strength of Evidence Study Sample Sample Size Data Source and Timing

Received Unemployment Insurance in past 12 months, %
Adjusted mean = 21 Adjusted mean = 22 -1 No Effect 1-High High (3 of 3) Maximum versus Guided Customer Choice 1,975 7-year survey

Total amount of Unemployment Insurance benefits received in past 12 months, $
Adjusted mean = 1,300 Adjusted mean = 1,205 95 No Effect 1-High High (3 of 3) Maximum versus Guided Customer Choice 1,975 7-year survey

Received Food Stamps in past 12 months, %
Adjusted mean = 18 Adjusted mean = 19 -1 No Effect 1-High High (3 of 3) Maximum versus Guided Customer Choice 1,975 7-year survey

Total amount of Food Stamps received in past 12 months, $
Adjusted mean = 321 Adjusted mean = 356 -35 No Effect 1-High High (3 of 3) Maximum versus Guided Customer Choice 1,975 7-year survey

Received other cash assistance in past 12 months, %
Adjusted mean = 16 Adjusted mean = 14 2 No Effect 1-High High (3 of 3) Maximum versus Guided Customer Choice 1,975 7-year survey

Total amount of other cash assistance received in past 12 months, $
Adjusted mean = 1,300 Adjusted mean = 1,199 100 No Effect 1-High High (3 of 3) Maximum versus Guided Customer Choice 1,975 7-year survey

Study Characteristics

Toggle Participants & Program Details Participants & Program Details
Participant Detail The study examined dislocated workers and other adults who were new customers of the participating Workforce Investment Area (WIA) agencies and who were eligible for Individual Training Accounts (ITAs). To be eligible, customers had to be determined as "in need of training," have the skills and qualifications to complete training, have received at least one core and one intensive service, and be unable to obtain funding assistance from other sources to pay for training from other sources. On average, sample members were 41 years old. Slightly more than half were female, approximately 41 percent were married, and slightly more than half had children. Most were nonwhite (38 percent were black non-Hispanic, 9 percent were Hispanic, and 11 percent belonged to another minority racial or ethnic group). Two-thirds of the sample members had been employed in the year before random assignment, and average earnings during the year before random assignment were approximately $21,000, but only about 10 percent were employed at the time of random assignment. Approximately 16 percent were receiving public assistance at the time of random assignment. Approximately one-third had a postsecondary degree, and one-fourth had a vocational or business degree or certificate.
Program Services Dislocated workers and adults (18 years and older) eligible for WIA training assigned to the Maximum Customer Choice group did not automatically receive, but could request, counseling to guide them to appropriate training selections and received a fixed ITA established for their area ($3,000 to $5,000).
Program Duration Approximately 77 percent of Maximum Customer Choice sample members and 71 percent of Guided Customer Choice sample members participated in a training program within three years after random assignment. Of those who entered a training program, members of the Structured Customer Choice group received counseling and arranged training for an average 20 weeks after random assignment and then they participated in their selected training program for an average 31 weeks. Those in the Guided Customer Choice group who participated in training received counseling and arranged training for an average of 18 weeks after random assignment and then participated in training for an average of 29 weeks.
Comparison Services Dislocated workers and adults (18 years of age and older) eligible for WIA training assigned to the Guided Customer Choice group received less intensive but still mandatory weekly counseling that guided them to appropriate training selections. Counselors could not reject client choices under this condition. Individuals in the Guided Customer Choice group also received a fixed ITA established for their area ($3,000 to $5,000).
Toggle Study Design Study Design
Strength of Evidence Description High; outcomes based on administrative data and outcomes based on long-term survey data that did not rely on data collected about job spells are rated high because the study was a randomized controlled trial and attrition was low. Employment and earnings outcomes from survey data were based on imputed cases. The authors did not establish that attrition was low before imputation; therefore, the review assumes attrition was high and required the authors to demonstrate that groups were equivalent before the study began. However, the study did not examine the earnings history of study participants more than one year before the study began and cannot therefore satisfy the baseline equivalence requirements for this review; consequently, these survey outcomes received a low rating and are not reported here.
Group Formation All customers who were eligible to receive WIA-funded training during the study intake period were randomly assigned individually by study staff on a rolling basis as their names were submitted by site staff. (To be eligible for services, clients first had to consent to participate in the study.) They were randomly assigned to one of three groups: the Structured Customer Choice group (considered in other reviews on this site), the Guided Customer Choice group (considered the comparison condition in this review), or the Maximum Customer Choice group (considered the treatment condition in this review).

For this study, researchers randomly selected a sample of cases for a follow-up survey that was conducted seven years on average after random assignment. The survey sample included 1,612 cases in the Structured Customer Choice condition (which had a 68.6 percent response rate), 1,598 cases in the Guided Customer Choice condition (which had at 67.7 percent response rate), and 1,590 cases in the Maximum Customer Choice condition (which had a 67.8 percent response rate).
Effect Calculation The effects reported by the authors and displayed on this site are the differences in weighted averages across sites of the treatment and control group means (adjusted for age, sex, race/ethnicity, marital status, presence of children, education level, vocational certification, primary language, type of worker, whether employed at baseline, and earnings in the 12 months before baseline).
Notes on Reported Outcomes The study also reports on completion of a training program, employment and earnings by quarter, and average hours worked per quarter. It also reports the number of months in the past 12 months that sample members received UI benefits, Food Stamps, and other cash assistance.
Subgroups Studied Site; dislocated worker status; age; gender; race/ethnicity; education status; vocational certificate status
Other Domains Examined None
Toggle Study Setting Study Setting
Setting Details The evaluation was conducted in eight sites (Phoenix, Arizona; Maricopa County, Arizona; Bridgeport, Connecticut; Jacksonville, Florida; Atlanta, Georgia; Northeast Georgia; North Cook County, Illinois; and Charlotte, North Carolina). Program services were provided by regional workforce boards and human services departments.
Timing of Study Enrollment into the study sample took place from December 2001 to March 2004. The longer-term follow-up survey was conducted between August 2009 and May 2010 (on average, seven years after random assignment).
Study Funding The study was funded by the U.S. Department of Labor.
Study URL http://wdr.doleta.gov/research/FullText_Documents/ETAOP_2012_06.pdf