Through a partnership with three Federally Qualified Health Centers (FQHCs), the local health department in Los Angeles County implemented a produce prescription program (PPR) to increase fresh fruit and vegetable purchases among patients with diabetes or prediabetes. The PPR, which began in 2020, provided eligible patients with a $40 voucher every month for 6 months to promote the purchase of fresh produce at large chain grocery stores. This qualitative assessment describes staff experiences with the PPR, documenting the facilitators and barriers they encountered while implementing the program. Fifteen clinic staff (i.e., PPR implementers) were interviewed for this assessment. Thematic analysis and coding were conducted using the ATLAS.ti software; the coding was carried out by two separate coders. Interviewees discussed the importance of having preexisting partnerships and programs to support the PPR at their clinic site. Hidden costs related to implementing the program included a large and unexpected amount of staff time devoted to enrolling patients into the program. Collecting quality data and having limited expertise to rigorously evaluate the program were other challenges. Because patients often share their food with their family, the monthly $40 incentive was generally not enough to support their needs; interviewees suggested giving a higher inventive amount to those with a larger household. Future PPR efforts and similar food incentive programs should consider these and other facilitators and barriers to implementation and sustainability, especially when making adjustments to these programs to improve services and access to food resources.

Keywords:

Subject: Public Health and Healthcare - Public Health and Health Services

1. Introduction

Food as medicine interventions are programs and services tailored to address chronic diseases that are associated with poor diet or for situations where nutrition is a vital part of the medical treatment [1]. In recent years, the healthcare sector has explored, turned to, and implemented many of these programs—e.g., medically tailored meals, food pharmacies, produce prescription programs (PPRs)—in hopes of reducing chronic disease risk and the healthcare costs associated with these noncommunicable disease conditions, including diabetes or prediabetes [1]. In the various versions of these programs, patients are given a “prescription” from a healthcare provider or staff to increase consumption of fruits and vegetables if they meet the requirements for program enrollment [2].

Produce “prescriptions” are typically given in the form of paper vouchers or electronic benefit cards, generally with a set cash or redeemable benefit amount that patients can redeem at a local grocery store or farmer’s market. Given that consumption of nutritious food can decrease the risk of diet-related chronic conditions, such as hypertension, hyperlipidemia, and type 2 diabetes [3], a majority of PPR programs have focused on these specific conditions [4,5,6]. Several United States (U.S.) federal agencies, including the U.S. Department of Agriculture (USDA), have made substantive investments in PPRs, with the goal of not only addressing food insecurity but also demonstrating how eating more fruits and vegetables can produce salutary health impacts—i.e., lead to increased nutrition security, improved health outcomes, and reduced healthcare utilization and costs [7]. Emerging data from PPR research have shown that participation in such a program, at least in the short-term, can increase fruit and vegetable intake and reduce food insecurity, resulting in improved biometric endpoints such as better hemoglobin A1c (HBA1c) and diastolic blood pressure control [4,5,6].

Although the evidence for PPR’s health impact is growing, much remains unknown about its logistics and the ways in which the program can be implemented in a standardized, culturally relevant way [8]. For instance, recent research suggests that both traditional and nontraditional facilitators and barriers can play critical roles in determining how well a PPR fit within a given system/setting [9,10,11,12]. Findings from the literature point to multisector partnerships as a key step to running a successful PPR [11]. Clinic staff buy-in and consistent communications among all parties involved in services delivery also represent crucial steps in implementation, as are considerations of transportation barriers and the use of Electronic Health Record (EHR) data to monitor and demonstrate program outcomes [9,10,11,12].

Research findings have also provided key insights into how several multi-sector, mutually reinforcing, strategies can work together to facilitate higher quality PPR [1,9]. These promising strategies have included: enhancing clinic staff’s experiences or satisfaction with program delivery [9]; encouraging flexibility when working with state Medicaid plans (which often have rigid reimbursement requirements); troubleshooting issues with incentive distribution early and often during the start-up phase [10,11,12]; and capitalizing on technology—e.g., use of electronic benefit card—to streamline and fast-track distribution of incentive benefits [10,12].

The present qualitative assessment sought to describe and address these implementation gaps in the literature, leveraging grant funded PPRs in Los Angeles County to examine many of these issues. In Los Angeles County, 1 out of every 10 adults has type 2 diabetes [13], and nearly two thirds live at or below 300% of the Federal Poverty Level. Compared to pre-coronavirus disease 2019 (COVID-19) levels, food insecurity increased during the pandemic, affecting nearly a quarter (24.3%) of households in the county [14].

2. Materials and Methods

Context

The PPR in Los Angeles County, known locally as “Fresco y Saludable/Fresh and Healthy”, was implemented through a public health partnership with three federally qualified health centers (FQHCs) and the electronic benefit card vendor, the University of California, San Francisco Vouchers 4 Veggies (V4V). To help implement the program, the Los Angeles County Department of Public Health (DPH) provided technical support to the three health centers, while V4V managed the produce incentive process including the administration of the electronic benefit cards, their redemption, and the communications with participating large chain grocery stores. Eligible patients were recruited from nine clinic sites within the three FQHC systems: Partner 1, Partner 2, and Partner 3. The goal was to enroll ~2,300 adult patients with diabetes or prediabetes. Each enrolled patient received $40 every month for 6 months in the form of an electronic benefit card, redeemable for produce purchases at participating grocery stores. Unexpended benefits on each card could not be rolled over to the next month.

At each PPR clinic site, patients were recruited by health and diabetes educators if they met the following eligibility criteria: (i) enrolled in Medicaid (Medi-Cal in California), (ii) screened positive for household food insecurity in the past 12 months, and (iii) have a diagnosis of type 2 diabetes or prediabetes. These health and diabetes educators were responsible for handling the enrollment appointments at each site, both for the baseline and the 6-month follow-up visits. Physicians and other clinical staff within (and external to) the three FQHCs were allowed to make referrals to the PPR; enrollment did not require a physician to sign off on.

Data Collection

Fifteen interviews were conducted with clinic staff from the three FQHCs; these staff were all involved in the day-to-day operations of the PPR. They were drawn from a pool of program implementers, including program managers, program coordinators, health and diabetes educators, and family medicine care coordinators. Although seventeen staff were originally invited to participate, only 15 were able to complete the interviews; two had a time conflict and did not participate. Table 1 shows the characteristics of the interview (key informant) sample, in particular their roles and responsibilities in the PPR.

The first wave of interviews took place between April and May 2022 – they were conducted with clinic staff from Partner 1. The second wave took place in January 2023 – they were conducted with clinic staff from Partner 2 and Partner 3.

A semi-structured interview guide was developed by DPH staff with experience in PPR implementation; they were part of the original team that established the first USDA National Institute of Food and Agriculture (NIFA) Gus Schumacher Nutrition Incentive Program (GusNIP) PPR in Los Angeles County (the program began in 2020). The guide comprised 9 key interview questions (Table 2), with additional probes and follow-up questions as needed. These series of questions asked interviewees about factors that facilitated or hindered the implementation of the PPR at their clinic site. All of the interviews were conducted virtually using an online platform; they were recorded and then transcribed. Interviews were carried out in English and lasted between 20 and 30 minutes. No compensation was given for participation. All interviewees gave verbal consent to enroll and to be recorded in these sessions.

Table 2. Interview questions, excluding probes and/or questions asked during follow-ups.

Key Interview Questions
Please tell me your job title, your role in this program, and which clinic you work at. Please briefly describe the enrollment process for this program. Do you think there could be any changes made to the eligibility criteria? In terms of enrollment, what do you think has most contributed to the success so far? What have been some of the barriers to enrollment that have arisen? What are some recommendations you would suggest to help streamline the workflow of the program? What are ways that we can help patients transition off the program after it ends and ensure they have other resources to access food? Thinking longer term, what advice can you offer clinics who are new to implementing this type of program? Before we wrap up, is there anything else you’d like to share that we haven’t talked about?

Key Interview Questions

Please tell me your job title, your role in this program, and which clinic you work at.
Please briefly describe the enrollment process for this program.
Do you think there could be any changes made to the eligibility criteria?
In terms of enrollment, what do you think has most contributed to the success so far?
What have been some of the barriers to enrollment that have arisen?
What are some recommendations you would suggest to help streamline the workflow of the program?
What are ways that we can help patients transition off the program after it ends and ensure they have other resources to access food?
Thinking longer term, what advice can you offer clinics who are new to implementing this type of program?
Before we wrap up, is there anything else you’d like to share that we haven’t talked about?

Data Analysis

Prior to the qualitative analysis, each interview transcript was cleaned and de-identified. Reviews of the transcripts were then conducted to identify key interview themes; they were coded using the ATLAS.ti software (Scientific Software Development—now known as ATLAS.ti GmbH, D-10961 Berlin). To ensure accuracy and reliability of the process, themes from each interview were coded independently by two coders. A third coder was available to assist if disagreements occurred and consensus between the two coders could not be reached.

A codebook containing emergent codes was initially created by each of the two coders based on their individual reviews of the first six interview transcripts (drawn primarily from Partner 1′s data). Code generation was carried out using a deductive approach, driven in part by the PPR design and its implementation process. These initial codebooks were used to establish inter-coder agreement, and to develop a workflow for subsequent analyses of Partner 2′s and Partner 3′s data. Overall, the process produced a total of 20 codes and a 90% intercoder agreement. Coded data were analyzed for patterns and grouped/synthesized into key themes and/or sub-themes (see Table 3).

All project protocols and materials were approved by the DPH Institutional Review Board.

3. Results

Table 4 contains salient quotes corresponding to each of the key themes/sub-themes related to the implementation of the PPR in Los Angeles County.

Institutional capacity and existing partnerships and programs

Many of the interviewees (n = 13) described the synergy and value of having existing partnerships and programs with DPH prior to starting the PPR. For example, ongoing collaborations with the health department’s Supplemental Nutrition Assistance Program Education (SNAP-Ed) prepared the clinic sites to conduct food insecurity screenings, as validated tools were already integrated within their EHR systems through a patient management software. This ‘ready-to-go’ food insecurity screening process allowed the clinic sites to determine program eligibility more easily right from the start. Because their collaborations with DPH are ongoing, the sites felt confident that they can offer the culturally appropriate nutrition education, physical activity classes, and healthy food demonstrations that are needed to support the newly enrolled PPR participants (i.e., these resources were already available). Coordination of free, monthly produce distributions was also a built-in benefit of SNAP-Ed. Interviewees indicated that these complementary SNAP-Ed resources, including the food events (excellent settings for recruiting patients), promoted enrollment in the PPR and encouraged patients to return for the program’s post-visits. Mutual experiences with data sharing agreements through prior SNAP-Ed projects also made formalizing these agreements between DPH and the three FQHCs much easier, ensuring that appropriate data collection and evaluation processes were embedded within the PPR before rather than after the start of the program.

Enrollment process

For the PPR, DPH worked closely with the FQHCs to develop a standardized enrollment protocol that also contained some built-in flexibility for adjusting the workflow; this is so that clinic staff can tailor the protocol to the needs of frontline work. For example, while the general process was the same across all the clinic sites, staff titles, their roles, and some of the field activities varied. For all clinic sites, the enrollment protocol follows the following sequence. First, clinic staff extracted a list of eligible patients from their center’s EHR system through the patient management software. Second, they contacted and recruited eligible patients to the program; this step included verifying each patient’s eligibility—i.e., screened positive for food insecurity, has either diabetes or prediabetes. Third, upon obtaining the patient’s verbal consent, an appointment at a designated clinic site was scheduled—at the appointment, clinic staff provided an explanation on how the program works, gave out an activated electronic benefit card for produce purchases, measured the patient’s blood pressure, and administered a survey on diet, food insecurity, and health behaviors. Finally, before completing the enrollment, patients without a HbA1c measurement in the past three months were asked to complete the blood test as soon as possible—i.e., arrangements were made for them to go to a FQHC contracted laboratory.

Interviewees (n = 14) described both successes and challenges to recruiting and enrolling patients into the PPR. Successful efforts to enroll patients included such approaches as: having clinic wide meetings to introduce the program to the at-large clinic staff (getting frontline staff buy-in), sending patient appointment reminders, and scheduling enrollment appointments at the same time as other clinic appointments to increase patient convenience. Several interviewees (n = 12) also discussed how they needed to pivot enrollment strategies to save time. When the program first launched, each enrollment appointment took up to an hour to complete. To pare down on this time commitment, one of the three FQHCs (Partner 1) created a three-minute video describing how the program worked, texting it to the patients prior to the appointment visit. Transitioning from in-person to over the phone administration of the survey prior to an appointment was another way the FQHCs reduced the visit duration. Finally, some interviewees (n = 3) indicated that cultural differences sometimes impacted the recruitment/enrollment negatively, making it difficult for patients to accept help, typically by exacerbating shame, stigma, and denial about their food insecurity status and past experiences with hunger.

Staffing

Throughout the PPR, the need for more staff time and capacity to deliver the program became a persistent concern for the FQHCs. Interviewees (n = 7) suggested, that for future programming, more funding should be set aside or allocated specifically to hire additional staff to support the program so that regular clinic flow and patient care activities are not disrupted. They indicated that extra staff time was needed because patients often had a lot of questions about the program or had issues with the electronic benefit card after the initial appointment. Some interviewees (n = 5) talked about the importance of getting more staff buy-in to support the implementation process—i.e., obtaining support from ancillary staff beyond just the program implementers themselves. One interviewee noted that the food insecurity screener established under SNAP-Ed, which helped clinic staff identify eligible patients, was not always uniformly applied, which, at times, caused lapses in program delivery.

System-level barriers

Interviewees (n = 12) attributed many of the program delivery challenges to clinic- and enrollee-level issues. For example, patients who enrolled in the program often expressed frustration with clinic staff when their six months of benefits ran out, resulting in many electing not to return for the post-visit; this complicated and delayed the evaluation effort, which was a required part of the GusNIP funding. Lack of transportation to and from the clinic also added to this patient retention issue. When asked about program logistics, interviewees from Partner 1 sites (n = 3) raised several issues around the need for equipment, such as blood pressure machines, which were initially not available to them to measure the patients’ blood pressure (they had to borrow the machines from other clinic staff).

Eligibility

Although enrolling patients who were receiving Medicaid benefits (Medi-Cal in California) was a requirement of the funding entity, several interviewees (n = 4) felt that this stipulation was too restrictive, since many of the hard-to-reach patients for whom the program was designed for were paradoxically excluded: e.g., patients who were undocumented or who were enrolled in a local no-cost health care program because they could not participate in Medicaid but were nonetheless experiencing financial difficulties. These interviewees suggested that some of these eligibility criteria should be expanded: e.g., to include those with other chronic conditions beyond diabetes or prediabetes, such as heart disease and hypertension.

Electronic benefit card

Interviewees (n = 13) discussed at length the advantages and disadvantages of using an electronic benefit card that is automatically loaded with $40 each month for six months, and its use as the chief mechanism for distributing these produce incentives to the patients. Apparently, when the card was working, it worked very well for most of the enrolled patients. However, when it was not working, it had a rippling effects on dollar value redemptions. For instance, when one of the grocery store chain modified the way they internally processed these electronic payments, it altered the way benefits were verified and redeemed at each of the chain’s stores. Consequently, many of the cards were either declined or had their account balance zeroed out. While this glitch in the chain’s computer system was eventually resolved by V4V, at least one patient chose not to further redeem benefits using the card, out of fear they would be denied again at the store. Because most of the patients with an electronic benefit card were not told about this glitch beforehand, disappointment and distrust quickly set in. As a suggestion for addressing this problem, several interviewees recommended establishing a customer service phone line (preferably in multiple languages) that patients can call and use to get the latest information on card activation/deactivation and updates on card balances.

Program evaluation

While interviewees indicated their support for conducting program evaluation as a way to document PPR progress and impact, several of them (n = 12) felt the expectations and the procedures utilized to collect data were, at times, overly extensive and time-consuming—i.e., the program’s structured evaluation framework required patients to complete a 20-minute self-reported questionnaire, in addition to measuring their blood pressures and HbA1c levels. Three interviewees discussed the potential harm that some of the survey questions may have had on patients, in particular inquiries about food insecurity often triggered traumatic recalls of past experiences with hunger and embarrassment at not being able to feed their families.

Referrals to other community programs

At both the beginning and end of the 6-month PPR, patients were provided information about clinic and community resources. Some of these resources included SNAP-Ed nutrition education and related materials, dates for upcoming free produce distributions, a list of local food pantries that patients can go to and written/verbal guidance on how to access and use online resource generators on the World Wide Web. Some interviewees (n = 4) described how a number of patients did not know how to navigate these online resources and required the use of printed (paper) copies instead. Several interviewees (n = 3) suggested teaching patients to use these online resources, capitalizing on this opportunity as a potential “teachable moment” to increase digital literacy.

Success and future programming

When interviewees were asked to define their vision for program success, their responses varied widely. To some (n = 7), simply having patients complete the program and use their electronic benefit card every month was a success. Others (n = 7) defined it as having patients return for their post-visit or seeing that the biomarkers/clinical measures for diabetes or prediabetes had improved. A few (n = 3) felt that seeing patients take greater ownership for their own health was success enough. Others (n = 6) liked what they saw in terms of increased fruit and vegetable consumption and improved digestion and energy levels.

For future programming, all interviewees were asked to provide recommendations about how the PPR could be improved. One of the most frequently mentioned recommendations was the need to increase staff time to distribute workload and reduce disruptions of regular clinic flow. Several interviewees (n = 9) indicated that the PPR should offer a larger incentive amount for patients who share their produce benefits with a larger household. To encourage coming back for the post-visit, some suggested that the PPR provide an additional month of benefits. And some (n = 5) suggested developing a communications workflow at the start of the program so that conversations between the program, clinic administration, and the providers in the field are consistent and are in real-time; this recommendation appeared several times in the interview data.

4. Discussion

This qualitative assessment describes and examines the process of implementing a federally funded PPR at nine clinic sites across three FQHCs in Los Angeles County—home to about 10 million people. The intended program beneficiaries were: Medicaid patients with type 2 diabetes or prediabetes, who screened positive for food insecurity, and who agreed to participate in a program that provided extra cash incentives to buy fresh fruits and vegetables at local chain grocery stores. The assessment builds upon emerging research on PPR development [9,10,11,12], documenting lessons learned (facilitators and barriers) and useful practices that can be applied to help guide future directions in PPR design and implementation. Four notable takeaways were gleaned from the interviews.

First, institutional capacity and strong partnerships with local health departments or community organizations that run food assistance programs (e.g., SNAP, SNAP-Ed, food pantries) were found to be key enablers of PPR planning and implementation [11,15]. This finding is consistent with prior research, which shows that PPR requires community-based partnerships and buy-in in order to fully reach its potential/to advance [11,16]. In the qualitative assessment, all three FQHCs (nine clinics) had working relationships with DPH and were actively implementing their SNAP-Ed strategies in targeted communities; these strategies included integrating food insecurity screening questions in the clinics’ EHR systems, referring eligible patients to nutrition education (and SNAP if eligible), and arranging access to free, fresh produce distributions on-site. Because these strategies and activities were already ongoing, they served as wraparound services/resources that the PPR could integrate with. Without such wraparound services/resources, the PPR would have been less effective, only being able to offer a standalone intervention—a relatively small produce incentive amount of $40.

Second, PPR implementation had a number of ‘hidden costs’ that were not immediately recognized at the start of the program. For instance, many interviewees pointed to the unexpected need for a large amount of staff time to implement the program: i.e., extra time had to be spent on scheduling enrollment and post-visit appointments, conducting the visits, answering follow-up questions from patients, and resolving issues with the electronic benefit card which, as it turned out, were relatively common. Prior studies confirm this finding, identifying staff capacity and buy-in as key elements to implementing a successful PPR [9,10,11]. Future studies of PPR and programming should consider these ‘hidden costs’, and to the extent feasible, also account for other factors such as the local context, a health center’s fiscal outlook, and what other flexible funding streams may be available for supporting food incentive programs, beyond the USDA-NIFA GusNIP program—e.g., the Medicaid Section 1115 Demonstration Waivers that fund medically tailored meals and other food and nutrition interventions.

Third, the PPR utilized a structured evaluation framework that was relatively rigorous, garnering praise and recognition from the clinic staff for its intent to demonstrate and show health impacts of the intervention. However, this same framework also posed a significant burden on clinic staff (and for the FQHCs), requiring major efforts to collect high quality data. In addition to not having adequately trained staff for evaluation at the start [9], several of the data requirements and scientific content were, at times, problematic to handle. For example, some of the survey questions triggered traumatic recall of unpleasant/stigmatizing experiences for several patients. Trainings on evaluation methods/procedures, while available and critical to program fidelity, were often difficult to provide in its entirely/optimally, given the competing responsibilities that most of the clinic staff had; they were frequently doing regular clinic work and the PPR at the same time. Referrals to support services and tracking of HbA1c lab orders and results were also not easy to facilitate without good communications among patients, health care providers, clinic administration, and community organizations that provide these services externally.

Finally, while the use of the electronic benefit card was considered a success, some interviewees did report disruptions in how the card was accepted (or not accepted) by grocery stores, resulting in a number of patients having their card purchases declined or balance zeroed out. In the implementation literature, this type of electronic error was not uncommon, as similar incidents have occurred with other food incentive programs [16]. In the present PPR, V4V was able to resolve this issue in a relatively short period of time (~2 months). Their consistent communications with grocery stores, and facilitation of corporate level as well as local store level actions were instrumental in getting the computer redemption system back online. Conducting follow-up calls after patients enrolled in the PPR to ensure they were able to use their cards became a best practice that the sites used and relied upon to avoid a repeat of the problem.

Multiple visions of success

Although a singular vision or aligned visions of success would have been ideal for advancing the PPR, findings from the qualitative assessment suggest that the definition for program success varied widely among program implementers. These variable definitions ranged from patient satisfaction and attainment of self-efficacy in personal health management to calls for demonstrating longer-term health outcomes such as improvements in blood pressure and HbA1c. Given the complexity of PPR delivery—interventions are relatively short term, program resources are scarce/sparse, food systems are fragmented and often difficult to work with, and quality longitudinal data are not always available to help inform practice [12]—the ideal vision for ‘success’ remains somewhat elusive and likely falls somewhere between these varying opinions.

Limitations

The design and conduct of this qualitative assessment has several limitations. First, the three FQHCs, while fairly large in size and number (nine clinic sites total), represents only a fraction of the FQHC landscape in Los Angeles County; the region has over 60 such health center systems, representing 350 primary care sites [17]. This limit on generalizability lessens the assessment’s ability to project how a PPR may have worked if it had been implemented in other clinic sites in the region. Second, the assessment was focused on day-to-day operations of the PPR; and as a result, all of the interviews were conducted with clinic staff only, and not with other personnel—e.g., administrators or support staff for the organization—these staff may have had an influence on the implementation process. Lack of patient participation also contributed to the selection bias, and likely represents missed opportunities to capture a more nuanced perspective on the effects of the produce incentive program (i.e., how it may have impacted patients’ ability to truly feed their families). Finally, because the interviewees were primarily implementers of the PPR, social desirability and implicit biases probably affected how they answered the interview questions—i.e., they may have offered a more favorable view of the PPR than what patients or the clinic administrators may have offered if they were asked the same questions.

5. Conclusions

Implementing a PPR in a low-income clinic setting, such as in sites operated by FQHCs, can be challenging. The implementation process often requires proper accounting of hidden start-up costs, the extra staff time needed to run the program effectively, and a support infrastructure that is often not present even long after the program has started. The present qualitative assessment offers important insights into this process, documenting lessons learned and useful practices that can help other FQHCs and healthcare entities implement PPRs, hopefully in a more streamlined, cost-efficient way. Having strong partnerships with food assistance programs like SNAP/SNAP-Ed, or with local health departments that run these programs, has proven to be an excellent foundation for establishing a PPR. Likewise, a priori planning of staff capacity, data collection, and evaluation methods, when done strategically, can become an important preparatory step for troubleshooting real world problems and the downstream issues that are surely to arise when the PPR starts delivering services. As Food as Medicine interventions expand (e.g., through state Medicaid waivers) and become more popular, program implementers will need to exercise good judgement and remain mindful of the local context, variation in access to grocery stores, and the pros and cons of using an electronic benefit card to deliver cash incentives when they start a PPR. When these various factors are optimally managed, a PPR can be a viable and efficient solution for helping low-income households combat hunger, food insecurity, and poor nutrition that exacerbates conditions like diabetes or prediabetes.

Author Contributions

Conceptualization, V.A., J.I.C., F.D.E., D.S.P, T.K.; Methodology, V.A., J.I.C.; Use of the ATLAS.ti Software and Methods Validation, V.A., J.I.C.; Formal Analysis, V.A., J.I.C; Writing – Original Draft Preparation, V.A., J.I.C.; Writing – Review and Editing, F.D.E, D.S.P., T.K.; Produce Prescription Program (Project) Administration, F.D.E, D.S.P; Funding Acquisition, T.K., D.S.P. All authors have read and agreed to the published version of this article.

Funding

This project was funded in part by the U.S. Department of Agriculture, Gus Schumacher Nutrition Incentive Program (National Institute of Food and Agriculture GusNIP No. 2020-70030-33128 and GusCRR No. 2021-70034-34982).

Institutional Review Board Statement

All project protocols and materials were approved by the Los Angeles County Department of Public Health Institutional Review Board (Project #2021-01-918, approval granted on February 11, 2021).

Informed Consent Statement

Informed consent was obtained from all interviewees and PPR enrollees.

Data Availability Statement

The data presented are available upon request from the corresponding author. The data are not publicly available due to privacy restrictions.

Acknowledgments

The authors thank all clinic staff who participated in the qualitative assessment.

Conflicts of Interest

The authors disclose no conflicts of interest.

References

National Institutes of Health. 2020–2030 Strategic Plan for NIH Nutrition Research: A Report of the NIH Nutrition Research Task Force.; 2020. Available online: https://dpcpsi.nih.gov/sites/default/files/2020NutritionStrategicPlan_508.pdf (accessed June 27 2023).
USDA National Institute of Food and Agriculture. Gus Schumacher Nutrition Incentive Program - Produce Prescription. 2023 Available online: https://www.nifa.usda.gov/gusnip-request-applications-resources-ppr (accessed June 27 2023).
Seligman H, Laraia BA, Jushel MB. Food Insecurity Is Associated with Chronic Disease among Low-Income NHANES Participants. Journal of Nutrition. 2010;140(2):304-310. [CrossRef]
Bryce R, WolfsonBryce JA, CohenBryce A, et al. A pilot randomized controlled trial of a fruit and vegetable prescription program at a federally qualified health center in low income uncontrolled diabetics. Preventive Medicine Reports. 2021;23(101410). [CrossRef]
Cook M, Ward R, Newman T, Berney S, Slagel N, Bussey-Jones J. Food Security and Clinical Outcomes of the 2017 Georgia Fruit and Vegetable Prescription Program. Journal of nutrition education and behavior. 2021;53(9):770-778. [CrossRef]
Veldheer S, Scartozzi C, Bordner CR, Rodriguez D, Berg A, Sciamanna C. Impact of a Prescription Produce Program on Diabetes and Cardiovascular Risk Outcomes. Journal of Nutrition Education and Behavior. 2021;53(12):P1008-1017. [CrossRef]
United States Department of Agriculture. USDA Invests More Than $59M to Improve Dietary Health and Nutrition Security. 2022. Available online: https://www.usda.gov/media/press-releases/2022/11/22/usda-invests-more-59m-improve-dietary-health-and-nutrition-security (accessed on June 27 2023).
Downer S, Clippinger E, Kummer C, Hager K. Food Is Medicine Research Action Plan.; 2022. Available online: https://www.aspeninstitute.org/wp-content/uploads/2022/01/Food-is-Medicine-Action-Plan-Final_012722.pdf (accessed on June 27 2023).
Stotz SA, Nugent NB, Ridberg R, et al. Produce prescription projects: Challenges, solutions, and emerging best practices – Perspectives from health care providers. Preventive Medicine Reports. 2022;101951. [CrossRef]
Auvinen A, Simock M, Moran A. Integrating Produce Prescriptions into the Healthcare System: Perspectives from Key Stakeholders. International Journal of Environmental Research and Public Health. 2022;19(17). [CrossRef]
Dolin Stephenson L, Lucarelli J, Stewart SA, Acosta S, Yoakum B, Yoaku C. Implementing a Produce Prescription Program in Partnership With a Community Coalition. Health Promotion Practice. 2022. [CrossRef]
Newman T, Lee JS. Strategies and Challenges: Qualitative Lessons Learned From Georgia Produce Prescription Programs. Health Promotion Practice. 2021;23(4). [CrossRef]
Los Angeles County Department of Public Health. Diabetes Prevention and Management. Available online: http://publichealth.lacounty.gov/diabetes/about/facts.htm. (accessed June 27 2023).
de la Haye K, Livings M, Bruine de Bruin W, Wilson J, Fanning J, Wald R. Food Insecurity in Los Angeles County.; 2022. Available online: https://publicexchange.usc.edu/wp-content/uploads/2022/09/USC-Food-Insecurity-Research-Brief_September-2022.pdf (accessed June 26 2023).
Virudachalam S, Kim LSH, Seligman H. Produce Prescriptions and a Path Toward Food Equity for Children. JAMA Pediatrics. 2023;177(3):225-226. [CrossRef]
Parks CA, Stern KL, Fricke HE, Clausen W, Yaroch AL. Healthy Food Incentive Programs: Findings From Food Insecurity Nutrition Incentive Programs Across the United States. Health Promotion Practice. 2020;21(3):421-429. [CrossRef]
Community Clinic Association of Los Angeles County. Available online: https://ccalac.org/about/. (accessed on June 27 2023).

Table 1. Characteristics of the interview (key informant) sample, drawn from a pool of program implementers of the produce prescription program in Los Angeles County.

Position/title in the organization	Description of role(s) in the produce prescription program	Number interviewed
Program Manager	Oversees the produce prescription program for the FQHC/clinic site. Coordinates program activities, provides monthly updates, meets with program staff to ensure that the PPR is running smoothly, and addresses problems and concerns that staff may have with the enrollment process or post-visits. Makes decisions about pivoting the enrollment process as necessary. Reports updates to DPH and electronic benefit card vendor, V4V.	2
Program Coordinator	Provides weekly update emails to program staff. Provides updates to electronic benefit card vendor, V4V. Tracks benefit card inventory and patients enrolled in the program. Helps patients troubleshoot card issues such as replacing them when they are lost or are not working. Assists with booking post-visit appointments, as needed.	3
Health Educator	Recruits and enrolls patients into the program and completes post-visit appointments. This process includes completing pre and post-surveys, providing an electronic benefits card, and ensuring that each enrolled patient had recent blood pressure readings and HbA1c lab results. Also provides support to patients to help them navigate program resources and specifics, as needed.	7
Family Medicine Care Coordinator	Recruits and enrolls patients into the program and completes post-visit appointments. This process includes completing pre and post-surveys, providing an electronic benefit card, and ensuring that each enrolled patient had recent blood pressure readings and HbA1c labs results. Also provides support to patients to help them navigate program resources and specifics, as needed. Has the ability to order HbA1c labs for patients who do not have recent HbA1c results.	1
Diabetes Educator	Recruits and enrolls patients into the program and completes post-visit appointments. This process includes completing pre and post-surveys, providing an electronic benefit card, and ensuring that each enrolled patient had recent blood pressure readings and HbA1c lab results. Also provides support to patients to help them navigate program resources and specifics, as needed.	2

DPH = Department of Public Health in Los Angeles County; FQHC = federally qualified health center; HbA1c = hemoglobin A1C blood test; PPR = produce prescription program; V4V = University of California, San Francisco Vouchers 4 Veggies.

Table 3. Key themes from the interviews, produce prescription program in Los Angeles County (started in 2020).

Institutional capacity and existing partnerships and programs

Refers to the capacity of the implementing agency and existing partnerships with community programs that help facilitate implementation of the PPR program, including established nutrition education programming (e.g. SNAP-Ed).

Enrollment process

Refers to the general process by which clinic staff enroll patients into the PPR program, including ways in which healthcare staff can pivot strategies to save time or make it easier for patients to enroll. Sub-themes include program promotion, enrollment barriers, enrollment facilitators, and cultural differences.

Staffing

Refers to clinic staff, the type of staff, and day-to-day responsibilities of staff who implement the PPR program. Sub-themes include enrollment staff, administration/management staff, staff buy-in, and staff capacity.

System-level barriers

Refers to the general challenges that healthcare clinics experience when implementing the program such as broken appointments by patients, transportation needs, frequency of HbA1c labs, and equipment needs.

Eligibility

Refers to program eligibility requirements for patients that they must be enrolled in Medicaid, screen positive for food insecurity, and have diabetes or prediabetes. Sub-themes include eligibility limitations and recommended changes to eligibility criteria.

Electronic benefit card

Refers to the challenges associated with using an electronic benefit card to administer participant incentives including patient experiences using their cards at participating large-chain grocery stores, the use of a web application to check the balance on cards, and difficulties reaching a Spanish speaking customer support line.

Program evaluation

Refers to the healthcare staff’s understanding of the role of program evaluation. This include positive aspects of evaluation such as documenting program impact. And negative aspects which includes the length of the patient survey, having patients return for the post-visit to collect biometric data, and how survey questions trigger some negative feelings among patients

Referrals to other community programs

Refers to the information provided to patients about other community programs and resources that are provided with at the time of enrollment and completion of the program. A sub-theme includes transitioning off the program.

Success and future programming

Refers to how healthcare staff perceive the “success” of the program which includes having patients complete the program, seeing improvements in the biometric data of patients, and patients taking greater ownership of their health. A sub-theme includes recommendations for program implementation.

HbA1c = hemoglobin A1c blood test; PPR = produce prescription program; SNAP-Ed = Supplemental Nutrition Assistance Program Education.

Table 4. Salient quotes from the interviews by theme, produce prescription program in Los Angeles County (started in 2020).

Institutional capacity and existing partnerships and programs

“We offer certain days where it’s a drive through produce day so [patients] can come in [to the clinic], make an appointment and get a bag of fresh produce.”
Family Medicine Care Coordinator, Partner 1

“When a patient goes to their appointment to enroll in the program, they also receive a welcome packet and, in that packet, we have flyers. That includes flyers for our nutrition classes and for produce events. For example, if we get a patient from San Fernando, we’ll have the San Fernando produce flyer for them.”
Program Coordinator, Partner 1

“Our providers already conduct the food insecurity screening as part of a patient’s normal [medical] visit, likely because they are more familiar with the program now since we started with them first and are aware that that’s what we use to determine eligibility. So, a lot of times I won’t even have to screen those patients. I’ll just be able to look up their records from their last visit, so that has been helpful in making the process more efficient.”
Health Educator, Partner 2

Enrollment process

“When [patients] first hear about the program and that they’re eligible, a lot of patients are concerned that, ‘is this the same as the CalFresh program like EBT?’ And they’re unsure about the difference. And they’re scared that if they enroll in this program, ‘will it affect my other government benefits, will other benefits be decreased if they know that I’m on this program’ and things like that. So that has been a barrier where they’re initially hesitant because they’re unsure how it might affect their immigration or citizenship status. In general, there is a lot of fear from receiving these types of programs.”
Health Educator, Partner 2

“What we’re doing is conducting the survey a day before… I know some of the appointments can take a little bit longer and we want to respect the patient’s time... Someone from our team will call them a day before and conduct the survey and then we emphasize [to the patient]...that the visit the next day will only be 10 to 15 minutes. They’ll be in and out of the clinic and the patients like that. They want something that’s relatively fast.”
Program Coordinator, Partner 1

Staffing

“Patients have a lot of questions, concerns, and patients also lose their cards.... Our team provides that support and its time consuming... I think competing priorities is a huge issue and just not having enough staff to help increase enrollment… An additional, full-time staff member that can dedicate more time to the program is needed. We only have on the grant 10% budgeted for staff and even that is not enough to help with the reporting and providing quality assistance. I think that is our biggest barrier, the staff capacity.”
Program Manager, Partner 1

“...[our program] can sometimes feel isolated and people don’t really know about our work. I think clinic wide meetings or a general meeting at the start of this program to introduce [the program] and include nurses and the providers at our site would have helped… Especially because at the clinic, sometimes there is miscommunication. They don’t know what the patients are here for, or they don’t know what the program is and that requires a lot of education on our part...I think a more collaborative effort involving the entire clinic staff would have helped.”
Health Educator, Partner 2

System-level barriers

“Our biggest issue or challenge is the post visits... And getting patients to come back for their post visit without having an incentive. We’re giving out a $5 gift card, but even then, it can be challenging for a patient that doesn’t have transportation.”
Program Manager, Partner 1

“The labs are difficult because [patients] don’t like having to come [to the clinic] so often. I know sometimes the provider will go ahead and put in a HbA1c order for the next 3 months and it so happens that it doesn’t fit within the time frame of the beginning of the program, so they have to come in again and they’re like ‘why do I have to keep getting my blood drawn’.”
Family Medicine Care Coordinator, Partner 1

Eligibility

“We have many patients that are positive for food insecurity or have other chronic diseases, but don’t qualify because they aren’t diabetic.”
Program Manager, Partner 1

“…Patients that are undocumented without any other means of receiving food aid. And if we think about it, those are the patients that need it the most because they don’t qualify for CalFresh.”
Diabetes Educator, Partner 3

Electronic benefit card

“I feel bad because the [patients] already don’t have transportation. They take a bus [to the store] and then the card doesn’t work. And that’s all the money they had to buy the groceries. So, it’s not like they can pay out of pocket for their vegetables. Especially at an expensive grocery store like Ralphs and Vons.”
Diabetes Educator, Partner 3

“The patient has gone to an eligible store and has gotten the appropriate foods and the [purchase] doesn’t go through. Those incidents are upsetting to hear because [the patient] expressed being embarrassed, humiliated, and feeling ashamed.”
Health Educator, Partner 3

“[Patients] try to buy produce and then the Ralphs employees are confused about the card. Maybe just increasing communication with the stores and letting [the stores] know that we’re running this program...I don’t know how often you all talk to the people at the grocery stores to see how often the cards are not working or how often you all just visit the grocery store to see what issues they are having on their end. We only get the calls from patients that are outspoken. The [patients] that give up and don’t let us know that the cards are not working -- a whole six months goes by and they never go back because they were disappointed.”
Diabetes Educator, Partner 3

Program evaluation

“We want to see results. We want to see that the [patients’] HbA1c and blood pressure is dropping because that’s what this program is about...We want to see that receiving the benefit of fresh fruits and veggies over 6 months and having [produce] infused into your house is making a real impact and in labs.”
Program Coordinator, Partner 1

“Some of the questions [in the survey] can feel insensitive. I’ve had some patients that start feeling ashamed because they run out of food, or they didn’t eat one day. Knowing how to word some questions for patients is important. Sometimes [the questions] can be triggering and [the patients] are feeling sad during the visit, because they’re just remembering that they weren’t able to eat sometimes or that they couldn’t feed their children all the right foods that they need to be eating.”
Health Educator, Partner 3

Referrals to other community programs

“We created this booklet called Food RX Guide...it’s a booklet of nutritional resources and we include food bank locations within different areas of Los Angeles.”
Program Coordinator, Partner 1

“One of the first questions is if [the patient] qualifies or if they think they qualify for CalFresh or if they already have CalFresh. If they say they have never applied before...then we do a referral to our health insurance enrollment department… and they call the patient back to schedule an appointment to enroll them to CalFresh.”
Diabetes Educator, Partner 3

Success and future programming

“After COVID, a lot of people lost their jobs, lost countless things that are important to them. For them to move forward and I think this program has opened my eyes to see what some people count as problems. When we talk to patients, when they tell us about their stories, especially these patients that we’ve been enrolling, it’s eye opening.”
Diabetes Educator, Partner 3

“At the end, hearing from the patients...and how grateful they are for the program...[Patients will say], ‘oh yeah, I really enjoyed making this new recipe because now I actually have more freedom to purchase fruits and vegetables’... Most patients I still follow …start seeing their A1C go down or tell me they are “feeling better.” Also they tell me that their overall well-being has definitely improved.”
Health Educator, Partner 3

COVID = coronavirus disease 2019 pandemic; EBT = electronic benefit transfer; HbA1c or A1c = hemoglobin A1c blood test; CalFresh = California’s Supplemental Nutrition Assistance Program; Food RX = food prescription.

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.