Logo

Decoding Legal Industry Surveys

Published: Apr 11, 2019

 Law       
Article image

The legal industry has myriad surveys that aim to give greater insights into law schools and law firms. But how do these surveys differ and what kinds of information and trends do they unveil?

This week at the NALP Annual Education Conference in San Diego, three industry experts discussed these questions and more at the panel, Asked and Answered: Media Surveys Decoded. The panelists included Vault’s own Claudia Grillo, Breaking Media’s Brian Dalton, and Chambers Associate’s Antony Cooke. The session was moderated by Geoff Burt, chief communications officer at Latham & Watkins.

The panel discussion included differences among surveys, survey fatigue, and recent trends uncovered by the panelists' surveys. Below are some key takeaways from the panel.

 

No One-Stop Shop 

Given the various survey approaches and goals, different pools of survey respondents, and divergent presentations of survey data, there isn’t a one-stop survey shop when it comes to researching legal employers or law schools. As Dalton noted, “Theres no one ranking that can serve all purposes no matter how comprehensive we try to be.” As a result, it’s useful to understand the purposes of each survey, who exactly is surveyed, the kinds of questions asked, from where rankings are derived if they are created, and how each company presents its data. Below is quick snapshot of the panelists approaches:

Vault: As Grillo explained in the panel, Vault’s survey focuses on law firm associates of all levels—unlike other industry surveys which focus on either junior associates or mid-level associates only. “It gives [jobseekers] a larger range and also helps laterals come back to utilize Vault because they find it significant in managing their career,” said Grillo. In the past two years, Vault has had almost 20,000 associates take our survey. The survey includes an internal component in which associates rate various “quality of life” categories within their own firm and an external portion through which associates rate the prestige of other firms overall, as well as regionally and in specific practice areas. In addition to providing ratings, associates also provide qualitative data. “Oftentimes those qualitative sections will validate those scores,” said Grillo. Vault hosts firm profiles that feature associate quotes from the survey, as well as around 60 law firm rankings varying from overall prestige to regional to practice area to summer associate program to quality of life rankings.

Chambers Associate: Cooke described Chambers Associate’s survey process as distinct in that a large portion of the data is derived through telephone interviews with firm associates. But four years ago, the company integrated a survey component as well. “I got the impression that over the phone, people were not [willing to] speak as bluntly,” Cooke said. “And it was relatively difficult for my researchers to draw comparisons [among] firms.” From Cooke’s viewpoint, this survey data helps to validate the qualitative data that his researchers collect through their interviews. In addition to its profiles, Chambers also has what Cooke describes as “performer rankings.”

Breaking Media: Breaking Media takes a different approach to surveys. The company has produced a variety of rankings, including ones that are reputational-based, editorial-based, and data-driven. According to Dalton, “We’ve not taken a particular consistent approach to any of these." Of the company’s rankings, Dalton focused on the law school rankings during the panel, which are based on employment outcomes.

 

No Need to be Tired

Another important issue that arose during the panel was survey fatigue—a concern that firms often raise regarding the multiple surveys in which they participate. So what’s a firm to do? 

First, they shouldn’t avoid surveys completely, especially with NALP’s latest change to the recruiting standards. “It’s so important to take the surveys,” said Grillo. “With the new NALP rule that there are no rules—or however you want to describe it—we’re seeing that is bringing firms on campus more … the students are not passive any more throughout the year. They’re more active because of the firms' time on campus.” And surveys provide insights into the firms that can help students as they research.

Plus, firms can embrace the value that surveys bring, especially the transparency that it lends to the industry. As Cooke shared, “I think there definitely is a place for surveys in the market. They add a level of transparency that otherwise wouldn’t exist.”

But that doesn’t mean that firms should be left with zero tools to better manage surveys. Grillo suggested a variety of ways to assuage survey fatigue. For one, she proposed that companies like Vault and Chambers collaborate on scheduling to make the survey roll-out more manageable for firms. Grillo also offered some creative ways that firms can manage the survey, such as providing a tighter deadline for associates, having a point person who can encourage participation, and being transparent with associates about when they can expect each survey throughout the year. 

And, of course, if a firm is feeling overburdened by too many surveys, Dalton offers a simple answer: Don’t take every survey. “I would identify myself as a survey fatigue skeptic. I say this because I believe I have a solution. And it’s quite simple … stop doing them or stop doing most of them.” Dalton went on to explain his view that firms would benefit from diving deeper into survey platforms that have “real market traction.” 

What’s Trending?

While some surveys provide a window into a firm, they also can offer insights into the legal industry and how it is evolving. “At Vault with the survey data we produce, it not only becomes a resource for students and laterals to research firms, but it becomes a source for Vault to understand the industry,” said Grillo. 

Below are some trends that the panelists have uncovered through survey data.

1. Culture is King

A decade ago, prestige reigned as the top factor in candidates’ decisions, according to Grillo. But times have changed, and culture has now become the top driving factor for selecting a firm.

2. It’s All About Quality

In its recent 2019 Millenial Attorney Survey of around 1,200 millennial attorneys, Breaking Media found that the number-one-identified criterion for selecting law firms was the idea that there would be a better quality of life, said Dalton.

3. Practice Over Location

And students and laterals are much more focused on quality of work and a firm’s practice area strengths. In fact, quality of work is the number two factor in selecting a firm, according to Vault’s 2019 survey. “Candidates are more aware of the substantive work,” said Grillo. “The work itself is something this generation is really focused on beyond the prestige of the firm.” Grillo shared her recent experience speaking with students at a top 10 law school: Around eight or nine out of ten students knew what practice area they wanted, but only one out of 10 students knew the location in which they wanted to work. 

4. Male vs. Female

Other interesting data shared during the panel came from Breaking Media’s 2019 Millenial Attorney Survey, which presented a stark contrast in how female and male millennial attorneys perceived certain law firm issues. Dalton shared the following data:

45 percent of female respondents agreed with the idea that law firm culture is inherently sexist, compared with 14 percent of male respondents

56 percent of female respondents believe there is a real gender pay gap in law firms, compared to 18 percent of male respondents

63 percent of female respondents agreed with the idea that a diverse and inclusive workforce should be a priority at law firms, compared to 37 percent of male respondents.

Overall, the panel provided an interesting glimpse into different approaches to surveys, as well as new strategies for managing the process and interesting trends arising from recent data.

NALP Panel Claudia

***