As previously stated, 44 state health department employees from eight states were interviewed. On average, the interviews took 43 minutes. All but one participant was female. The average time in their agency was 11 years, with 15 years of experience working in public health. The majority of respondents were at the Program Manager or Section Director level within their organization. The following themes emerged from their interviews regarding the important leadership attributes to prevent mis-implementation: (1) engagement, use of quality improvement, and being adaptive; (2) transparent and bidirectional communication; and (3) ability to navigate political influences.
Leadership engagement, use of quality improvement, and being adaptive
Many respondents reported important qualities in leaders that deterred the mis-implementation of programs. The leadership qualities that were cited as most important in maintaining evidence-based programs included engagement, use of quality improvement, and being adaptive.
Engagement and recognition of other’s expertise
Engagement was described by respondents both in terms of engagement with internal staff on a regular basis and also engagement with external partners.
As a result of our leadership here and just on a more micro-level, within our team, it's a fairly small team but it helps that everybody really likes their job and gets along with one another and we are supported by management.
I think that's really helpful with having the consistency of, for example, staff meetings or one on ones [with upper management]. All of those opportunities to share really make it then more comfortable and then easy for addressing circumstances that arise.
I think that there's lots of leadership support to maintaining excellent relations with our communities and the programs that we're administering.
Leaders who were collaborative and set up working units that allowed for and were engaged in cross-collaboration and learning were also cited as important in deterring mis-implementation.
We're organized into teams and into structures that continue and promote effective work and building off each other
I think what's really happened because of our leadership that we have now is programs have moved away from really working in silos to trying to work more collaboratively.
Finally, respondents reported that leaders who respect and listen to experts within the community and who provide their staff autonomy to use their own expertise were most effective.
There was a lot of support from very high-level leadership to say, well the community knows best
When we decided that we were going to switch to this way, we wrote up what our intentions were, and they [management] supported it. I feel like, at least our program, has had the option to be creative and innovative and do things a little bit different in how we manage, and we've been supported in doing that.
Quality Improvement and Being Adaptive
In our review of themes related to quality improvement, respondents shared that they frequently incorporated several important continuous quality improvement and other evaluative measures to help them identify ineffective programs and prevent mis-implementation. Respondents also reported that those programs with leaders who valued quality improvement and required staff to set goals for improving processes were less likely to have continued an effective program.
The health department as a whole and each bureau in it, including us, would [set] our analytic goals for the year…including, what we're trying to achieve, some of those will surface up to the governor's office. We have a CQI process, continuous quality improvement, where we propose specific things to go through this more formal process of CQI. So if we will target something and then go through a whole process it will take say, three four months to go through and come out with a product aimed at improving processes and that kind of thing.
We have an office of public health and performance management, and one of their tasks is quality improvement, so they do participate in our program and the plan new study act, and they have a quality improvement coordinator who implements rapid improvement events in different departments within our agency.
Respondents reported that in some cases, evaluation results pointed to the need to adapt a program to ensure that mis-implementation would not occur. When respondents reported that they needed to make changes or adaptations in response to evaluation results, those respondents that had leaders who implemented shared decision making and involved all staff to ensure buy-in were more likely to be supportive of the change.
We try to have a very collaborative decision-making process. I don't know the last time that I personally came forward and said, "As your manager, you will no longer do X, Y, Z."
I think again it really comes down to "Does everybody buy into the change?" If they believe in it or they buy into it or they understand the reasons for the change, they're more likely to embrace it and do it.
In addition, respondents reported that leaders who coordinated rigorous planning efforts both with internal and external partners and who considered the diversity in capacity among partners were most effective in implementing changes to programs.
I would say [changes are managed most effectively when] it's well thought out and we really consider the process and how we're going to roll it out.
We have to flex [our changes] to that diversity [or partners]. If we don't, then we're defeated from the get go. 'Cause we're gonna ask counties to do things they just can't get to unless we really work with them.
Finally, respondents felt leaders who were most effective in adapting programs to prevent mis-implementation effectively communicated changes and actually included communication as part of planning efforts.
I like to think that we're pretty effective when we do make a change and go in a different direction that it affective in the terms of communicating it, of trying to get people on board.
I think part of it is just making sure that they have understanding about the reason for the change and then knowing how to make those changes, for example.
Leadership and transparent and bidirectional communication
Most respondents reported that their leaders promote upward communication about problems or issues that arise and are open to hearing ideas from staff. Respondents who were leaders also reported trying to be transparent with their staff and that they tried to create opportunities for dialogue about issues that program staff was encountering.
Many respondents indicated that their leaders relied on feedback from their staff to make decisions about programs. Respondents also reported that issues with programs were generally identified at lower organizational levels and communicated upward to leaders. Respondents reported that they often had back and forth conversations or brainstormed with leaders to find solutions to issues.
Our leadership would generally rely on the programmatic folks and the division directors to sort of research and understand what other alternatives would be and to come up with a recommendation.
I think they [leaders] are very supportive. They're very welcoming to new ideas or new approaches for how we're doing this work.
Other respondents indicated that their leadership had an open-door policy. Respondents expressed that these open-door policies made it easier to communicate with leadership and allowed them to more quickly address concerns they had with programs.
Our leadership is very supportive, specifically up the commissioner's level. We have an open-door policy with our commissioner so that's the way. If we see something, we can always go discuss it
Several respondents reported that using data to communicate concerns about programs with leaders was effective. They expressed that leaders were most receptive to hearing about issues when it was backed up with data collected from the program.
I also think that data speaks volumes too (when communicating with leaders), so if enough data can be shown that something needs to change, or what we're doing isn't working.
Leaders reported being in constant and transparent communication with their staff about the status of programs and their issues. They achieved this contact through regularly scheduled meetings and reports. Leaders also expressed a desire to be transparent about programs and about their expectations for their staff
There are monthly and quarterly reports and just ongoing communications. So we pretty much know on a monthly basis what kind of traffic any particular program might be having and what the issues might be, challenges, the good things, the success stories, all that. So I think we do a good job of just staying in constant contact. – CD Director
I will continue to give the authority to the division directors to run their programs as they see fit. And allowing for a review of how things go is what I'm looking at. So they know what they're doing and they know what my expectations are.
Respondents also reported issues with communications from leaders. Some respondents reported frustrations with how leaders received their feedback and with a lack of transparency in leadership communication.
Respondents reported that staff sometimes had to spend lots of time communicating with leadership without much response or attention paid to an issue. Respondents reported frustration at having to spend so much time to spend communicating with leaders.
There was an interim public health commissioner who we met with twice a week for, I don't know how long. They basically thought the program was horrible. And so we had to keep bringing data and bringing data to show him that, every objection he came to, we were able to find data to show… It was very painstaking and It was frustrating to have to do that because meanwhile again, we could have been, doing something else with those funds.
Other respondents reported unclear and non-transparent communications with leadership. Some respondents reported that leadership was unclear with the direction that they wanted to take with a program. Other participants reported that how decisions were made by leadership was not communicated clearly to them.
There could be someone who above you who can kind of make a push on a higher level and they make the decision, but when it gets to you as a program manager it may come across as coming from someone else you know. You will not really know who made the decision.
Skills to navigate political influences
The ability for leaders to navigate political influences was another critical aspect of preventing mis-implementation. Several attributes, specifically interpersonal skills, the ability to build partnerships and connect with and understand partners, and a strategic approach, were cited as important in preventing mis-implementation.
Interpersonal skills
Interpersonal skills include the ability to influence other’s thinking and behaviors, even in the absence of formal authority. Respondents perceived a leader with strong interpersonal skills to positively influence policy proposals and improvements in specific areas of programs or changes in the target population.
I think it really takes diligent and observant program directors or even the leaders of the DPPs, if they think about how ... and they're evaluating their program itself, and who they're targeting and who they're reaching. I think it would take those types of people to go to the decision maker and say, I think that this program isn't as effective as it could be, or that we could try to reach a different population in a different way.
Depending upon the makeup of your legislature determines which policies are passed and which ones are promoted. So, fortunately here in [name of the State redacted] we've had leadership that has been more in tune with health related topics.
And I think that you have some commissioners that are a strong and can advocate and are willing to advocate for programing and there's other that just want to do budget cuts
Reading people and situations
The ability to think about the dynamics among stakeholders within a social system is one of the attributes that help leaders navigate different political situations. Some respondents shared some examples that represent their capacity to assess and respond to the political environment. When doing so, they consider the impact of those dynamics on programs and how they will respond to address potential issues.
There are factors like overall we have a very conservative legislature, so we have to have a sense of receptivity to issues we work on. Probably most dramatically we've encountered those issues because we do a lot of public campaigning. A lot of social marketing. A lot of social media work and TV ads and that kind of thing so we have to make sure to write up to the governor's office before anything gets aired that they're okay with it.
And we did try to have different perspectives from the different parties that were engaged in the work to determine what that best approach would be.
Facing a situation with divisive opinions among the stakeholders, one respondent was considering the possibility of a potential push back on a program due to many factors:
So you have the school board that has to have a role, and you know city council you got to warn them about it, when a new program, the news station actually came out, there was a kid who was kind of being forced or perceived to be forced to participate in some, of the program activities you know. And the parents were upset and brought the media in, you know.
Building alignment and alliances
Building alignments and alliances seem to be an essential attribute when dealing with the challenges of mis-implementation, especially within those programs that require political support. When facing challenges to address specific health-related topics or reaching out to target-audiences of a program, the respondents reported alternative strategies relying on their partners and stakeholders to proceed with ways to continue with their work and potentially impact the program continuation.
We can work with heart, lung and cancer that can work with legislators and propose legislation but we would be limited in the fact of never being able to publicly print out our support of that. But it doesn't stop us from working with those who can advocate.
We had actually more funded partners at the time and I think as it would sound, the community-based coalitions would focus on just a broad strategy to reach any tobacco users in the community that were interested in seeking out cessation classes and then the minority-based coalitions were tasked with serving minority communities specifically.
Strategic planning and communication
A common thing mentioned by the respondents in terms of program mis-implementation was the importance of a leader having a clear vision and purpose of their work with a strategic approach to establish the needs and direction of a program and communicate that approach to policymakers
I think that particular program, we had a change in the leadership and the oversight of that, and then the person that started is very focused on scalability, sustainability, and outcomes, and the program clearly had no outcomes. She was able to really gain a support of the bureau chief to kind of identify that as an issue, and they were able to shut that down.