Background

The Pennsylvania Re-employment Bonus Experiment was a social experiment conducted in the 1980s to evaluate the effectiveness of a re-employment bonus program. The program was designed to encourage unemployed workers to find new jobs by offering a cash bonus to those who found new jobs within a certain time frame. These experiments were conducted to test the incentive effects of alternative compensation schemes for unemployment insurance (UI). UI claimants are randomly assigned to a control group or one of five treatment groups. The control group is eligible for regular UI benefits, while the treatment groups are eligible for regular UI benefits plus a bonus if they find a job within a certain time frame, provided that the job was retained for a specific duration. This study will focus on the comparison of the control group and merged treatment groups 4 and 6 and analyze the treatment effects of these incentives.

Data Exploration and Cleaning

The data set has 13,913 observations across 23 variables. However, we are subsetting this with the control and merged treatment groups. We will merge treatment groups 4 and 6 into one group and subset the data including the control group 0 and treatment group 4. We will also recode the ‘tg’ variable to better distinguish the control and treatment groups.

With this subset, we have 6,384 observations with 3,354 in the control and 3,030 in the treatment group. There are also no missing data in the subset.

## # A tibble: 2 × 8
##   tg            n avg_unemp avg_log_unemp med_unemp med_log_unemp sd_unemp
##   <chr>     <int>     <dbl>         <dbl>     <dbl>         <dbl>    <dbl>
## 1 Control    3354      13.3          2.06        11          2.40     10.6
## 2 Treatment  3030      12.4          1.98        10          2.30     10.3
## # ℹ 1 more variable: sd_log_unemp <dbl>

Between the control and treatment groups, the difference in the unemployment duration, even when taking the log, is not as substantial. The control group has a slight increase in duration with roughly 0.9 more weeks of unemployment. Both the median and standard deviations are also marginally similar, indicating slight variations in the distribution of unemployment duration.

Below shows the count and proportions of the control and treatment groups based on gender, race, and age.

## # A tibble: 2 × 6
##   tg        female_count black_count othrace_count agelt35_count agegt54_count
##   <chr>            <int>       <int>         <int>         <int>         <int>
## 1 Control           1359         407            19          1795           366
## 2 Treatment         1204         348            25          1665           310
Male Female Not Black Black Not Other Race Other Race Not Younger 35 Younger 35 Not Older 54 Older 54
Control 0.522 0.53 0.524 0.539 0.526 0.432 0.533 0.519 0.523 0.541
Treat 0.478 0.47 0.476 0.461 0.474 0.568 0.467 0.481 0.477 0.459

The proportions in the control group are generally higher than the treatment group across all categories, except those who are of other race. Given there are more participants in the control group, it makes sense we would expect a higher proportion among these categories over the treatment group. This difference may not be substantial enough to indicate a significant difference in the treatment effect.

The density plot shows that the distribution of the log of employment duration is slightly lower for the treatment group compared to the control group. This suggests that the re-employment bonus program may have a positive effect on reducing the duration of unemployment.

Analyze Treatment Effects

We analyze the treatment effect of the re-employment bonus program by running a linear regression model with the log of the employment duration as the dependent variable and the treatment group as the independent variable.

##                Estimate Std. Error   t value    Pr(>|t|)   CI Lower    CI Upper
## (Intercept)  2.05682971 0.02095378 98.160332 0.000000000  2.0157533  2.09790615
## tgTreatment -0.07954211 0.03027508 -2.627313 0.008626792 -0.1388914 -0.02019279
##               DF
## (Intercept) 6382
## tgTreatment 6382

The point estimate is -0.07954, indicating that the treatment group has a lower log of employment duration compared to the control group. This suggests that the re-employment bonus program has a positive effect on reducing the duration of unemployment. The p-value associated with the treatment effect is 0.00863, making the treatment effect statistically significant.

Heterogeneity Analysis

We will also conduct a heterogeneity analysis to examine the treatment effects of the re-employment bonus program across different subgroups. We will focus on race, age, and gender. We will run separate regression models for each subgroup and compare the treatment effects. Before we do that, we will create two variables to added variability. The first being a white race group variable as well as an age group designating those who are between ages 35 and 54.

Variations for Race

##                      Estimate Std. Error    t value     Pr(>|t|)   CI Lower
## (Intercept)        2.09293822 0.02185162 95.7795246 0.000000e+00  2.0501017
## tgTreatment       -0.08604901 0.03142549 -2.7381920 6.194940e-03 -0.1476535
## black             -0.29756254 0.07127142 -4.1750610 3.018714e-05 -0.4372785
## tgTreatment:black  0.03982441 0.10649536  0.3739544 7.084507e-01 -0.1689423
##                      CI Upper   DF
## (Intercept)        2.13577475 6380
## tgTreatment       -0.02444451 6380
## black             -0.15784661 6380
## tgTreatment:black  0.24859108 6380
##                      Estimate Std. Error    t value      Pr(>|t|)   CI Lower
## (Intercept)        1.77304465 0.05779052 30.6805439 6.121828e-193  1.6597558
## tgTreatment       -0.02605964 0.08613329 -0.3025501  7.622426e-01 -0.1949098
## white              0.33884482 0.06191578  5.4726728  4.601686e-08  0.2174691
## tgTreatment:white -0.06626011 0.09186945 -0.7212420  4.707870e-01 -0.2463551
##                    CI Upper   DF
## (Intercept)       1.8863335 6380
## tgTreatment       0.1427905 6380
## white             0.4602205 6380
## tgTreatment:white 0.1138349 6380
##                        Estimate Std. Error    t value    Pr(>|t|)   CI Lower
## (Intercept)          2.05900636 0.02098431 98.1212319 0.000000000  2.0178701
## tgTreatment         -0.07943101 0.03034548 -2.6175562 0.008877099 -0.1389183
## othrace             -0.38423609 0.33732672 -1.1390621 0.254720020 -1.0455098
## tgTreatment:othrace  0.10696046 0.43151093  0.2478743 0.804239665 -0.7389459
##                        CI Upper   DF
## (Intercept)          2.10014266 6380
## tgTreatment         -0.01994367 6380
## othrace              0.27703759 6380
## tgTreatment:othrace  0.95286683 6380

When just looking at the individual race groups and their combined effects with the treatment effect there are a couple things worth noting. First, there is statistical significance when including black and white race groups regardless of treatment, but not for other race. This may be due to the smaller sample size for this group compared to the other race groups. Another thing to note is that for all interactions for each group and treatment

Now, let’s look at how the treatment effect varies across these race groups. To do this, we will set one of the groups as a reference and compare the treatment effects of the other groups to the reference group. The other race group will be the reference.

##                      Estimate Std. Error    t value     Pr(>|t|)   CI Lower
## (Intercept)        1.70718428  0.1100335 15.5151304 2.559664e-53  1.4914816
## tgTreatment        0.03362211  0.1611508  0.2086376 8.347378e-01 -0.2822876
## black              0.08819141  0.1292652  0.6822517 4.951046e-01 -0.1652118
## white              0.40470519  0.1122550  3.6052297 3.142585e-04  0.1846476
## tgTreatment:black -0.07984672  0.1905867 -0.4189522 6.752651e-01 -0.4534607
## tgTreatment:white -0.12594186  0.1642883 -0.7665907 4.433533e-01 -0.4480021
##                    CI Upper   DF
## (Intercept)       1.9228869 6378
## tgTreatment       0.3495319 6378
## black             0.3415947 6378
## white             0.6247628 6378
## tgTreatment:black 0.2937672 6378
## tgTreatment:white 0.1961184 6378

Based on the p-values of the interaction terms, we see that the treatment effect with the respective race group is not statistically significant compared to the reference group. Thus, we can conclude that the interactions between treatment and race does not have a significant impact towards the treatment effect.

Variations for Age

To check for variation across age groups, we will first look at the treatment effects for each age group.

##                        Estimate Std. Error    t value     Pr(>|t|)   CI Lower
## (Intercept)          2.17576329 0.03008895 72.3110343 0.000000e+00  2.1167788
## tgTreatment         -0.03974247 0.04387662 -0.9057777 3.650877e-01 -0.1257554
## agelt35             -0.22223020 0.04174926 -5.3229738 1.055675e-07 -0.3040728
## tgTreatment:agelt35 -0.06663565 0.06030541 -1.1049697 2.692144e-01 -0.1848545
##                        CI Upper   DF
## (Intercept)          2.23474774 6380
## tgTreatment          0.04627045 6380
## agelt35             -0.14038763 6380
## tgTreatment:agelt35  0.05158321 6380
##                       Estimate Std. Error   t value     Pr(>|t|)    CI Lower
## (Intercept)          2.0272327 0.02219463 91.338876 0.000000e+00  1.98372375
## tgTreatment         -0.0925241 0.03200685 -2.890759 3.856093e-03 -0.15526827
## agegt54              0.2712252 0.06605762  4.105888 4.077935e-05  0.14173010
## tgTreatment:agegt54  0.1449504 0.09530438  1.520920 1.283294e-01 -0.04187824
##                        CI Upper   DF
## (Intercept)          2.07074162 6380
## tgTreatment         -0.02977993 6380
## agegt54              0.40072036 6380
## tgTreatment:agegt54  0.33177894 6380
##                         Estimate Std. Error    t value    Pr(>|t|)    CI Lower
## (Intercept)           2.01195164 0.02638985 76.2396116 0.000000000  1.96021867
## tgTreatment          -0.08573031 0.03791272 -2.2612546 0.023777026 -0.16005198
## btw35_54              0.12617021 0.04329058  2.9144959 0.003575022  0.04130614
## tgTreatment:btw35_54  0.02049408 0.06281082  0.3262827 0.744221178 -0.10263621
##                         CI Upper   DF
## (Intercept)           2.06368460 6380
## tgTreatment          -0.01140865 6380
## btw35_54              0.21103429 6380
## tgTreatment:btw35_54  0.14362438 6380

Generally, we see that the effect for each age group excluding treatment is statistically significant. What is interesting is that those who are younger than 35 leads to a lower log duration of unemployment compared to those who are between 35-54 and older than 54. This may be due to the fact that younger individuals may have more flexibility in finding new jobs compared to older individuals. However, like race, their interactions with treatment are all not statistically significant.

Next, we will check if the treatment varies across age. We will use the between 35-54 age group as the reference group.

##                        Estimate Std. Error    t value     Pr(>|t|)    CI Lower
## (Intercept)          2.13812185 0.03431691 62.3052007 0.000000e+00  2.07084918
## tgTreatment         -0.06523623 0.05007818 -1.3026877 1.927284e-01 -0.16340629
## agelt35             -0.18458876 0.04489215 -4.1118268 3.974712e-05 -0.27259247
## agegt54              0.16033606 0.07105391  2.2565410 2.407048e-02  0.02104652
## tgTreatment:agelt35 -0.04114189 0.06495698 -0.6333713 5.265139e-01 -0.16847939
## tgTreatment:agegt54  0.11766248 0.10279256  1.1446595 2.523932e-01 -0.08384547
##                        CI Upper   DF
## (Intercept)          2.20539452 6378
## tgTreatment          0.03293383 6378
## agelt35             -0.09658506 6378
## agegt54              0.29962560 6378
## tgTreatment:agelt35  0.08619561 6378
## tgTreatment:agegt54  0.31917044 6378

Based on the p-values of the interaction terms, we can see that the treatment effect does not vary significantly across the age groups. This suggests that the treatment effect is not dependent to a particular age group.

Variation Across Gender and Race

Finally, we will look to see if the treatment effect varies across gender and race. We will run a regression where we include the interaction term between gender and each of the race groups. Again, we will set the other race as the reference group.

##                 Estimate Std. Error    t value     Pr(>|t|)    CI Lower
## (Intercept)   1.77440031 0.09748587 18.2016161 3.290918e-72  1.58329525
## tgTreatment  -0.08108709 0.03014564 -2.6898449 7.167137e-03 -0.14018268
## female       -0.04521616 0.17483637 -0.2586199 7.959369e-01 -0.38795420
## black         0.02631698 0.11734149  0.2242768 8.225491e-01 -0.20371177
## white         0.29130313 0.09861506  2.9539417 3.148893e-03  0.09798448
## female:black  0.07125873 0.20253712  0.3518305 7.249770e-01 -0.32578209
## female:white  0.14600568 0.17785795  0.8209117 4.117272e-01 -0.20265568
##                 CI Upper   DF
## (Intercept)   1.96550537 6377
## tgTreatment  -0.02199151 6377
## female        0.29752188 6377
## black         0.25634572 6377
## white         0.48462178 6377
## female:black  0.46829955 6377
## female:white  0.49466703 6377

Just like the results from examining race and age, we see that the interactions between gender and race do not have a statistically significant impact on the treatment effect. This suggests that the treatment effect does not significantly vary across different combinations of gender and race.

Takeaways

The re-employment bonus program overall has produced a positive on the job outlook for unemployed workers. The treatment group generally saw a lower log of employment duration compared to the control group, indicating that the program has been effective in reducing the duration of unemployment. When examining the demographic data and their impact to the treatment effect, we see that while the presence of the demographic variables have contributed to the treatment effect, their interactions with the treatment group have not been as statistically significant. Thus, the treatment effect is generally consistent across different demographic groups.

Limitations

There are a couple of limitations to this study. One of which is that the data is from the 1980s, which may not be as relevant to today’s labor market with new policies enacted since that study. Additionally, the data is from Pennsylvania, which may not be generalized to other states or regions as they may have different labor laws and regulations. Lastly, the study only focuses on the comparison of the control group and merged treatment groups 4 and 6, which may not capture the full effects of the re-employment bonus program.

Next steps

Some next steps to take with this study include: a more general study encompassing more states and regions besides Pennsylvania. Additionally, a more comprehensive analysis of all treatment groups to understand the full effects of the re-employment bonus program. Lastly, we should carry out a more updated study to reflect the current labor market and policies enacted since this study.