Recently in the Huffington Post, Christina Villegas wrote a fascinating article that provided a critique of the Violence Against Women Act (VAWA), a seminal piece of legislation that has protected vulnerable populations from abuse since 1994. Up for re-authorization in the Senate this week, VAWA failed to win approval in the House earlier in the year, primarily due to the fact that the new legislation aimed to expand protections to immigrants and LGBT communities (see stories here, here, and here).

Amid the flurry of responses, (including NFF’s Social Currency and Money & Mission blog), Villegas has raised the interesting issue of data, questioning VAWA’s community impact and arguing that it has never undergone rigorous evaluation. With the strain of tight resources overshadowing the social sector, this question of impact is on the minds of all nonprofits: how do we prove our impact and know what’s working and what’s not?

In fact, VAWA HAS gone through rigorous, ongoing evaluation. 2000 Legislation of VAWA required the Attorney General to report biennially on the ‘effectiveness’ of VAWA funding, resulting in a long-term collaboration with the Muskie School of Public Service’s Catherine E. Cutler Institute for Child and Family Policy. Muskie School went on to develop and implement a substantial reporting program that resulted in a 2010 report here. The School is also slated to release a follow up report in 2013. 

VAWA supports 12 diverse programs—from education and training to end violence against women with disabilities to promoting arrest policies and protection order programs. Therefore, indicators of effectiveness vary across program areas. Here’s the long and short of findings on the effectiveness of VAWA, according to the report, starting first with the simple ‘outputs’ – or basic measures of productivity—surfaced from the data collected:

  • Office of Violence Against Women (OVW) grantees have served a huge number of people in need: During each six-month reporting period between July 1, 2007, grantees reported serving an average of 117,436 victims/survivors. Perhaps even more impressive is that VAWA grantees reached more than 97% of all victims/survivors who requested services (p. 13-14).
  • Grantees are a key provider of safe havens for victims in need. Domestic violence is one of the leading causes of homelessness for women (United States Conference of Mayors, 2005). Rural, Tribal Governments Program, Services Training Officers Prosecutors (STOP), and Violence Against Indian Women (VAIW) grantees provided emergency housing to an average of 2,260 victims along with 2,437 of their family members during each six-month reporting period. Combined, this represents a total of 334,820 bed nights over the two-year reporting period (p. 49).

Outputs or Outcomes?

The very fact that this many people are seeking services and receiving them is incredibly important, especially in a sector where need typically outpaces available services. (According to NFF’s 2012 Nonprofit Survey, 89% of the 1,761 human services respondents reported an increase in demand for services in 2011. 58% were unable to meet that demand.) But we know that program ‘outputs,’ which quantify numbers of people served, aren’t enough.  If we’re to address and prove the goal of reducing or mitigating domestic violence, we need to move beyond counting the number of people served and look at the effects of the services received. And what makes VAWA unique is its holistic approach: the service providers and policymakers supporting VAWA have long realized that addressing domestic violence means catalyzing cultural changes to help people across the social continuum— from victims, to perpetrators, to law enforcement personnel, to policy makers—realize that domestic violence is not ok.

We think that measures of influence along this continuum are more than just outputs; they are outcomes that cumulatively lead to change in how our society thinks about and reacts to violence. Here are a few findings presented in the evaluation of VAWA that are leading indicators for a culture shift in society.

Training & education

  • More people are receiving anti-violence training. During the period under examination, 569,341 professionals from many industries and sectors received training on issues including sexual assault, domestic violence, dating violence, and stalking (p. 14).
  • Research has shown that efforts to respond to violence against women are most effective when combined and integrated as part of a coordinated community response (Pence & Shepard, 1999; Shepard, 1999). With VAWA support, diverse agencies and stakeholders came together to participate in weekly, monthly, and quarterly meetings: domestic violence programs (631), law enforcement (546), prosecutor’s offices (486), social service organizations (485), courts (465), health/mental health organizations (383), legal services organizations (361) (p. 15). 

Treating violence as a crime

One of the clearest indicators of VAWA’s effectiveness is the support it provides to the legal system and law enforcement to increase arrests and prosecutions of domestic violence cases. Over time, VAWA has been instrumental in sending the message that no forms of violence are ok, and that there are consequences.

  • Providing legal aid: The Legal Assistance for Victims (LAV) Program grantees provided services to an average of 35,577 victims/survivors during each six-month reporting period. During each six-month reporting period, grantees addressed an average of 48,230 legal issues on behalf of those victims/survivors (p. 39-40).
  • Hiring and training law enforcement personnel: During each reporting period, OVW grantees hired an average of 117 law enforcement officers and used funds to support an average of 101 specialized law enforcement units. Between July 1, 2007 and June 30, 2009, grantees trained 89,501 law enforcement officers (p. 55).
  • Improving responses to crimes: Law enforcement officers in Arrest Program-funded agencies responded to 689,578 calls for assistance from domestic violence victims, arresting more than 117,377 predominant aggressors and referring 118,973 cases to prosecutors (p. 55).
  • Better protecting victims: LAV-funded staff provided assistance to an average of 11,006 victims/survivors seeking protection orders. The number of protection orders obtained with the assistance of grant-funded staff was 247,603 (p. 45).

Measuring Impact: No Magic Bullet

Sarah Gelfand, Director of the Impact Base at the Global Impact Investing Network recently wrote, “If you ask a room of 100 impact investors—representatives from funds, foundations, institutional banks, family offices—how many measure their social and environmental performance, roughly 80 percent will raise their hands. When you ask how many think that they are doing so effectively, only a few scattered hands will remain raised.”

Villegas’s critique and the data provided in the 2010 report highlights a larger challenge facing social purpose organizations and the funders that invest in them. Nonprofits across sectors, sizes, and regions are under pressure to prove the outcomes of our interventions, particularly as resources shrink. But most social enterprises will tell you that it’s easier said than done. Measuring impact is a complex process that requires organizations to take responsibility for tracking things that aren’t necessarily in their control. For example, if an anti-violence organization wants to track if its efforts are affecting rates of violence against women, it would need a seamless system in which it could gather information from police stations, legal systems, hospitals, school campuses and other social service agencies, to name a few. And even then, if rates of violence weren’t declining, we would not know the cause: were more people standing up to report crimes or were the services actually failing?

On top of this is the vicious circle of the resource challenge. On the one hand, as resources get tighter in our post-recession era, in which every government dollar is up on the chopping block, we want to ‘fund what works’ and have tasked organizations with proving their effectiveness. On the other hand, with fewer resources, organizations are already being asked to do more with less. For many organizations, it’s becoming a choice between serving fewer people or sacrificing an organization’s staff or financial health. Nonprofits already have a very difficult time raising long-term funding for programs, let alone measuring long-term impact. Where does better evaluation even fit into this picture?

Data Should Inform Action, Not Halt It

With VAWA up for reauthorization this week, we look back on the last 30 years of work with awe for the providers, policy makers and law enforcement agencies that, against the odds, were able to work together to change our society’s understanding of violence from inside out. While it’s true that there is room for improvement in VAWA just like any complex set of services, now is the time to support VAWA and let the providers that have been improving services for decades do what they have done so well: provide safe havens and support for victims while pushing us towards lasting cultural change.