J-PAL Voices

Episode 7: Using Research to Understand Impact

Episode Summary

In this special episode of J-PAL voices, we present rigorous evidence on the impacts of summer youth employment programs and discuss ways for you, our listeners, to support these programs.

Episode Notes

Every year from June to August, hundreds of thousands of youth – between the ages of 14 and 24 – participate in summer jobs programs across the United States. These programs, also known as summer youth employment programs or SYEPs, have been receiving greater attention over the past few years as an important policy tool to improve youth outcomes. 

In this special episode of J-PAL Voices, we will discuss research showing that summer youth employment programs can improve the lives of young people in outcome areas as diverse as employment, criminal legal system involvement, education, and youth development. We will also revisit program participants and implementers from the first season of J-PAL Voices to put the evidence in context of the day-to-day of SYEPs. Finally, we will discuss where we can go next.

We would love to hear your comments and feedback at podcasts@povertyactionlab.org. J-PAL Voices is brought to you by J-PAL North America (https://www.povertyactionlab.org/na). Stay in touch via Twitter (https://twitter.com/JPAL_NA), LinkedIn (https://www.linkedin.com/showcase/j-pal-north-america), and Facebook (https://www.facebook.com/JPAL.NorthAmerica/).

Please take the short, one-minute survey at https://j-p.al/voicessurvey.

Links: 

The Promises of Summer Youth Employment Program: Lessons from Randomized Evaluations, a J-PAL Evidence Review synthesizing thirteen randomized evaluations across four major U.S. cities’ SYEPs. 

Mayor Adams Announces Record 100,000 Summer Youth Employment Opportunities, press conference on February 15, 2022

Biden outlines his administration's actions to curb violent crime and gun violence — 6/23/21, White House press conference, CNBC (Youtube)

YOU Boston Youth Recognition Celebration, Boston City TV (Youtube) 

Episode Transcription

Yiping Li

Every year from June to August, hundreds of thousands of youth – between the ages of 14 and 24 – participate in summer jobs programs across the United States. These youth spend their summers working in a paid part-time position, typically organized by the local government. This year, 2022, more youth stand to benefit from summer jobs programs than ever before, thanks to historic increases in investments for these programs across multiple jurisdictions.

Mayor Eric Adams of NYC

Today we are giving these opportunities with…an unprecedented investment in our summer youth employment program. 

As part of the preliminary budget we expanded the program this year to 100,00 jobs…would be available to youth 14-24. 

Yiping Li

One such example is New York City, where Mayor Eric Adams announced the expansion in February 2022.

Mayor Eric Adams of NYC

This is the largest number of jobs made available in this program’s history..

And it is a $79 million investment

-Intro music-

Yiping Li

From J-PAL North America, this is J-PAL Voices. I’m your host, Yiping Li. 

In 2020, we brought you the first season of J-PAL Voices, which explored the impacts and promises presented by summer jobs in the United States. Since then, we have witnessed the resilience of Summer Youth Employment Programs, or SYEPs, as many of these programs went virtual or hybrid for the first time on a large scale, to ensure that young people everywhere still had access to summer jobs during the pandemic. 

Now, almost two years later, we are seeing more and more jurisdictions creating or expanding their own SYEPs. At the same time, more and more rigorous evidence is being generated by researchers across the country that speak to the effectiveness of summer youth employment programs. Over the past year, my colleagues and I at J-PAL North America have been diving deeper into the evidence base on the impacts of summer jobs programs on youth. 

In this special episode of J-PAL Voices, I’m excited to show you, our listeners, what we have learned from the evidence. Along the way, we will be bringing back some of the voices of program implementers and participants that many of you came to know through our earlier episodes. A lot has happened in the past couple of years, and once you’re all caught up, we’ll share what we hope to see in the future, and how you can help us make that happen!

Now, before we jump into the evidence, I want to take a moment to briefly describe what a summer jobs program actually is.

First… Who are the participants? This is how, Julia Breitman, whose team runs the New York City SYEP, describes the population of youth taking part in it.

Julia Breitman 

There's so much ethnic diversity, multiculturalism, income. And the wonderful thing about a program like SYEP is it really brings everyone together. It's like a subway cart, everybody's in there, right? So there isn't maybe a typical young person, but I can tell you that the vast majority of our young people come from low, middle income households, about over 80 percent of our young people. Most young people are young people of color. The vast majority are public school students, and for many of them, for those who are college bound, they'll be the first in their families to attend college. 

Yiping Li

The types of jobs offered in SYEPs are also diverse. 

Julia Breitman

About 40 percent of the jobs are kind of your typical summer job, your daycare, day camp, hospital, social service. We also have another 40 percent of young people in the private sector. And that's everything from small businesses, to banks. And then we have young people who are also participating or working with government agencies.

Yiping Li

Summer jobs programs have a long history in the United States, beginning in 1963 as a federally funded program. Since then, funding for these programs has fluctuated, moving from predominantly supported by federal funds to state and local funds. This resulted in a patchwork of programs across the United States, with some cities such as Chicago and New York City having larger programs, and other jurisdictions with no summer youth employment program at all.

Today, as more governments consider increasing investments in summer jobs programs, such as tapping into available funds from the American Rescue Plan, it is all the more important that we understand how policymakers can think about these programs as a part of local budgets and where to allocate their resources.

Fortunately for us, many cities across the country allocate slots in their summer employment programs through a random lottery. This provides a unique opportunity to conduct a randomized evaluation—one of the best tools we have to measure the causal impact of a policy or program. Randomized evaluation is also J-PAL’s particular area of expertise. Here is Kim Dadisman, the Associate Director of Policy at J-PAL North America, talking about why this methodology is so effective.

Kim Dadisman

So with random assignment, youth are assigned to be offered a job and to participate, or to not be offered a job. This also ensures us that there are no systematic differences between the groups at the beginning of the study.

Again, for example, if a program was offered on a first-come, first–served basis, it could lead to youth with more connections or resources, like access to computers, taking up the majority of slots. But with random assignment, any differences between the groups observed at the end of the study can be attributed to the program itself and not to underlying factors. 

Yiping Li

Over the past fifteen years, researchers, including three from the J-PAL network, Sara Heller, Judd Kessler, and Alicia Sasser Mondestino, built randomized evaluations into SYEPs using random lotteries in Boston, Chicago, New York City, and Philadelphia. These researchers produced thirteen papers on their findings. Over the past year, we at J-PAL North America have been working to synthesize the evidence from these papers to better understand the full picture of the program’s impact.

What we found… was amazing… The full picture of the benefits of summer youth employment programs is rich and complex. It is a program that can move the needle on a variety of outcomes, from employment, to involvement in the criminal legal  system, to education, to youth development outcomes.There is also a lot more to discover in the data for other outcomes, which we will talk about later in the episode, but for now, let’s dive deeper into these findings. 

Yiping Li

First, let’s talk about employment. The first question we explored was whether summer youth employment programs were achieving their stated purpose of providing youth with a summer job. Here’s Judd Kessler of University of Pennsylvania, who researches the New York City SYEP.

Judd Kessler

During the summer, the programs provide access to jobs that youth otherwise would not have. So when youth get lotteryed into the summer youth employment programs, they are significantly more likely to have a job that summer. The likelihood they have a job if they're lotteryed in is 80 to 100 percent, whereas the probability they have a job if they don't get lottery then is closer to 20 or 30 percent, and so that's a big impact on labor market access

And that big gap in employment leads to significantly higher earnings for youth who get into the summer program. So, over the course of the summer, they can earn hundreds of dollars more, on average, than youth who are not lotteried in. 

Yiping Li

Recall, then, that the population we tend to see participating in SYEPs are low-income youth, this means that SYEPs can make a huge difference during the summer for youth from low-income households. 

Judd Kessler

Summer jobs programs are very good at transferring money to low-income youth and their families because giving someone a job in that summer, giving them the opportunity to earn money, that's a direct benefit that they get. And the youth who participate in these programs are typically lower-income and often non-white, and there might be lots of benefits to transferring funds to those groups. 

Yiping Li

But until recently, it did not look as if these employment and earning effects lasted beyond the summer. So, Judd and Sara Heller from University of Michigan proposed a new intervention: letters of recommendation, or LORs,  from the youth’s employers that they can take with them to apply for jobs after the conclusion of the summer jobs program.

Judd Kessler

And these letters provide information about what the youth’s strengths are and how they perform in the summer programs. They are pretty easy and quick to provide.

And what we are finding is that LORs provided to youth are helping those youth secure employment faster and for longer in the years following the program. 

 

Yiping Li

And Judd and Sara will continue working with Julia to study and refine this intervention to make sure there aren’t any unintended consequences. Here’s Julia talking about how this intervention will inform New York City’s SYEP in the years to come.

Julia Breitman

… and to see results of that very small intervention was really phenomenal and it's just a small lift on our part that I think it's something we're gonna end up doing for every young person participating in the program. 

 

Yiping Li

In addition to providing access to a job and earnings during the summer, SYEPs have also affected youth’s criminal legal outcomes. Here’s Sara Heller from the University of Michigan, who examined the Chicago and Philadelphia SYEPs. 

Sara Heller

Summer jobs programs consistently reduce involvement in the criminal justice system for at least a year. And we have seen that across measures of arrest, arraignments, conviction, and incarceration.

Yiping Li

Maybe even more importantly, the decrease in involvement with the criminal legal system is not only taking place during the summer. 

Sara Heller

Now, you might think that this is just a product of keeping young people busy during the summer, but in fact the crime decline lasts beyond the end of the summer, and so young people are taking something with them after the summer that changes their behavior. Now, that might be improved social, emotional, and self-regulation skills. It might be changes in peer groups or time use, or just the income itself from the summer.

Yiping Li

Further analysis also shows something exciting about who tends to be most impacted by SYEPs.

Sara Heller

Some recent work suggests that youth at a greater risk of experiencing socially costly outcome, so greater risk of involvement with the criminal justice system, for example, may in fact experience the biggest benefits from summer youth employment programs. This suggests that there is not a point where it’s too late to help youth and young adults stay safe. In fact, there might be benefits to investing the extra resources it takes to serve a population at higher risk of criminal justice involvement. 

Yiping Li

The evidence is clear: summer youth employment programs reduce contact with the criminal justice system for youth and it is more than just about keeping them off the streets by occupying their time. Habiba Khan, a former participant of New York City’s Summer Youth Employment Program in 2020, describes how summer jobs programs show youth that someone is willing to invest in them. 

Habiba Khan

 If you're investing time in the students and if you're investing time in participants of the program, then you're more likely to get something out of the students. So it does make sense that when you introduce someone to something that's good for them, that they might continue going along that pathway and continue using whatever you're giving them to their advantage.

Yiping Li

In addition to the consistent increase in employment during the program summer and reduction in contact with the criminal justice system, research indicates there is even more about the impact of summer youth employment programs that can be explored. 

As researchers look beyond traditional measures, we see how we are only beginning to understand the full potential of SYEPs.

In Boston, Alicia Sasser Modestino found increases in academic outcomes such as on-time high school graduation, which has traditionally not been observed in other locations. To help dig deeper, she and her team linked the educational outcomes to outcomes from a qualitative survey to dig deeper into what youth are taking away from SYEP beyond the summer. 

Alicia Modestino

There is emerging evidence from the Boston summer youth employment evaluations that the program can have an impact on longer term outcomes. For example, we are finding that the program improves on-time high school graduation by as much as four percentage points or seven percent. And these outcomes are largely driven by not only improvements in attendance but also improvements in college aspirations and work habits, so things like showing up on time. 

Yiping Li

We also spoke with former SYEP participants from New York City, who shared similar sentiments.

Here’s Erica Chen from New York City who has participated in NYC’s summer jobs programming for many summers as a high school student, describing how the experience developed her technical and non-technical skills:

Erica Chen

Like, I really loved my instructor because - or like my boss - because she was so friendly with everything and casual with everything, but she was also really approachable about like job things, job related things, but also like life things. 

This program was a lot more about personal development and a lot more about focusing on soft skills and how to prepare you for the small things about working in a professional environment.

Yiping Li

And as Former Boston Mayor and now Secretary of Labor Marty Walsh said to the participants in the 2018 Boston SYEP, the positive impacts of summer jobs programs are not only gained by youth but also by the broader community due to the essential work that these young people performed. 

Marty Walsh

Congratulations on being accountable, congratulations on finishing what you started, congratulations on being on time, congratulations on working together, something that’s important. You made a real difference in your city and certainly in our schools and parks and neighborhoods. 

Yiping Li

What we see from the evidence on summer youth employment programs is that they are programs with great potential, much like the youth who participate in them. And randomized evaluations have played an important role in unlocking that potential by helping to paint the full picture of SYEPs' wide-ranging impacts. 

We see consistent improvements in increasing access to jobs and earnings during the summer. We see decreased involvement with the criminal legal system for participants that last beyond the summer. 

But we now know that there is so much more these programs can offer. 

Here’s Julia from New York City again.

Julia Breitman

At the base of it is this idea that if young people are given exposure to role models, to career opportunities, to pathways for growth and progress. They will make the right decisions.

They will grow and thrive and it's incredibly heartening to know that what we do, our work, can contribute to that.

Yiping Li

And for many summer youth employment programs, research has been—and continues to be—an integral part to informing the decisions made to ensure that we are providing the best possible opportunities to our youth. 

Here is Rashad Cope, the Director of Boston’s Department of Youth Engagement and Employment, talking about the value of research in helping his department coordinate the summer jobs program with the various partners involved in the effort. 

Rashad Cope

If we can, just as a city department, better understand how to assess our programs. Then we can better support our partners, who are on the ground doing the work. So those are local, nonprofit organizations, community based organizations, city agencies. Even the private sector who works with one of our summer job intermediaries. So we can better support them in understanding and using research data to improve the experiences that they themselves are creating for young people.

Yiping Li

Thankfully, summer youth employment programs are starting to receive historic investment, and public attention. In June 2021, President Biden drew national attention to summer jobs programs as a policy tool worthy of investment. 

Joe Biden

We know summer job training and recreation for young people work…We can invest in more of these programs with the American Rescue Plan.

Yiping Li

And state and local governments across the country have answered this call. In April 2022, the US Department of Education announced that Baltimore, Lexington, Madison, and Milwaukee are investing more than 14 million dollars combined in their summer youth programs using American Rescue Plan funds.

There is so much more to learn about these programs. And here’s where you come in. As summer youth employment programs expand and scale, we at J-PAL North America are ready and excited to work with local governments who are looking to start a summer jobs program or have been running one and want to learn more about how research can help them. If you are a business owner, I encourage you to reach out to your local summer youth employment program and inquire about partnership opportunities to host youths at your business in the coming summers. 

For everyone else, if anything we have said in this episode has piqued your interest, we would love to hear from you. The more these programs are implemented, the more we can continue to improve them, and the more we can provide a better experience for our youth. Because our young people deserve the best that we can give them.

Eric Adams

We open the doors of opportunity for our sons and daughters of the city that have historically witnessed the doors being shut in their faces. Well, darn it, we are kicking those doors open and allowing them to be inside and sit down at the table of opportunities like so many have for so many years and help them get the training they need so they can move into employment. New York City is leading the way with the largest summer youth employment program in the nation. This is how we change the lives and change the course of our young people. 

Yiping Li

This special episode of J-PAL Voices was produced by Dave Lishansky and written and narrated by Yiping Li. Elizabeth Bond designed our logo. Special thanks to Erin Graeber and Kalila Jackson-Spieker for their inputs and support. Transcription assistance was provided by Caitlyn Ark. 

For this episode’s interviews, we thank Julia Breitman, Erica Chen, Kim Dadisman, Rashad Cope, Habiba Khan, Alicia Sasser Modestino, Sara Heller, and Judd Kessler.

Our email address is podcasts@povertyactionlab.org; we would love to receive your comments and feedback. Thank you so much for listening and for your continued support.