Vol. 43, No. 5

Panel at ABA BLI 2019: Data is a tool, so use it wisely and well

by Marilyn Cavicchia

When it comes to data, can you ever have too much? Yes, according to a recent panel discussion—if it’s an overabundance of the wrong kind of information, if it’s delivered in a way that no one wants to wade through, or if it’s used as a means to squelch innovation that sometimes requires a best guess rather than exhaustive evidence.

How can a bar’s chief staff executive help both staff and members manage and understand data so it’s a help rather than a hindrance? At the 2019 ABA Bar Leadership Institute, a panel of bar executives shared how they collect and use data effectively: Jill Snitcher McQuain, executive director, Columbus (Ohio) Bar Association; Rebecca McMahon, chief executive officer, Cleveland (Ohio) Metropolitan Bar Association; and Joe Skeel, executive director, Indiana State Bar Association.

What data do you need most?

Both Skeel and McMahon took the helm at bar associations that they said had not previously been in the habit of collecting or sharing much data. When she became CEO in 2015, McMahon said, data was gathered and studied when someone requested it, only to be discarded or forgotten—and then, six months later, someone would ask the same question again.

Skeel and McMahon both also said that the bulk of their work in developing a more data-driven bar culture has been in training the staff regarding “what to collect, what to track, and what to ignore,” as Skeel said—because collecting and analyzing every possible bit of information would overwhelm staff and members alike.

The ISBA has decided to focus on two things, Skeel said: retention, and improving the bar’s financial picture. This narrow focus, he added, has helped limit the scope of the data that the staff collects. For example, the bar has found that there’s a correlation between whether a new admittee attends events in their first (free) year of ISBA membership and whether they then renew when it’s time for their paid membership to start. So, the bar is now tracking in its database which admittees are attending events. Once renewal notices go out, the bar will be able to see whether, in fact, attendance drives renewal. For now, Skeel added, the bar is certain enough of this result that it is focusing significant communication efforts on promoting upcoming events.

Using numbers to tell stories

The staff at the ISBA hasn’t been resistant to gathering and communicating more data than they did before, Skeel said, but it has taken some time to figure out how to make it the least time consuming it can be, and how to integrate data management into the work that the staff was already doing. McMahon noted that it took about eight months of training and discussion to help the CMBA staff learn “how we can use numbers to tell stories.”

Often, McMahon explained, board members of bar associations don’t know much about how the work gets done and what the various bar departments actually do. To give board members a better picture, every CMBA department head now puts together some data that spotlights a few key numbers to inform board members of what occurred the month before and what will be important in the month ahead. These are gathered into a single document—with as many graphics and pictures, and as few words as possible—that is sent to board members prior to each meeting. The staff itself is in charge of determining what they think is most important to tell the board that month, McMahon noted, and her role is to review the information rather than directing what is to be conveyed.

Along with breaking down silos and helping the staff feel as if their work is known and respected, this approach has been helpful in terms of board training, McMahon said. Board members now know to expect to receive data often, and that they will be expected to review it. Not only does this help them track key performance indicators continuously rather than sporadically, but it also helps board members not shut down when they need to review financial data. This type of data seems less intimidating than before, McMahon explained, because the board has been trained to expect and want data in general.

Skeel noted that telling a compelling story with numbers or other data helps prevent members from focusing on other, less accurate stories that are based on their own assumptions. For example, he said, there were anecdotal accounts that attendance at the ISBA’s most recent House of Delegates meetings was low and that the bar should take steps to address it. Based on that assumption, ethics CLE was added to the meeting, as a draw. Attendance did increase—but delegates gave that meeting a very low evaluation. Then, Skeel saw some old reports on past House of Delegates meetings. It turned out that attendance had been low for years, meaning that this was not a new problem—despite those anecdotal accounts—and that perhaps there was no need to make a big, corrective effort that would divert resources from more important matters.

Can data kill innovation?

McQuain was frank and unapologetic about being the “data skeptic” on the panel. Whereas Skeel opened by talking about his efforts to create a “data-driven culture” at the ISBA, McQuain was emphatic on the point that the CBA is “data informed,” not “data driven.” She has seen many circumstances, she explained, where data is “looked to as an excuse not to do something”—that is, if the likely success of a possible innovation can’t be exhaustively proven, then an organization is unlikely to take a bold step that may be necessary.

The wrong kind of data—too narrowly focused on how the bar is doing right now—can also give members too rosy a view and make it seem as if no innovation is needed, McQuain believes. At the CBA, she explained, things look really good for right now, with 63 percent of members at the top tier of the dues structure. But what that means, she added, is that five years from now, many of those members will have jumped off the dues structure, into retirement—and she has real concerns about whether there will be enough members at mid-career who can take their places. She’s not sure how to solve that problem—but she is sure it will involve taking some chances.

“We have to take risks right now,” she said, “and there isn’t going to be data behind trying something new every time.”

Easy ways to collect and share data

Whether you love data or hold it a bit suspect, the panelists agreed that data collection should be as easy as possible. Skeel is not a big fan of surveys (when he does want member feedback, he often prefers a well-designed focus group), and has focused on data that is easily gathered from within the bar’s database. “We’ve had to work within what our database could do,” he noted. “Do not assume it’s all going to be easy.”

McMahon echoed that point and said that the CMBA would probably switch to a different association management system—one that can more easily track the kind of data the bar is seeking—in the next 12 to 18 months. This is an investment, she added, but nothing like it was five years ago, as more sophisticated systems have become more affordable.

Don’t overlook something as simple as the system you use to send emails, Skeel said. It’s easier to send them directly from the bar, but using a third-party sender such as Constant Contact or Mailchimp allows much more detailed tracking of open and click-through rates, he explained. McMahon noted that she prefers Mailchimp because the CMBA experienced a high bounce rate when it was using Constant Contact.

Panelists and audience members alike expressed difficulty in making sure board members read materials in advance of each meeting. For that reason, Skeel keeps any communication of data to one page or less. The CMBA’s monthly snapshot from each department, together, is six pages long. McMahon said that she now gathers some of the most compelling data into a one-page dashboard that also serves as the cover; that way, she said, if board members can only read one page, they’re at least seeing some important highlights.

Not just the what, but the why

The panelists all agreed that even the best data offers a glimpse that may open up further exploration, rather than giving the full picture in and of itself.

“Data tells you what,” Skeel said. “It doesn’t tell you why.” That is, it can help you identify and begin to explain something that is happening, but it may take further inquiry and conversation to get all the information you need.

McMahon noted that when board members do want to ask a question based on the department spotlights, they often feel more comfortable doing that outside of the boardroom, in a one-on-one conversation with her.

For presidents-elect in the room, McQuain advised that they make sure not to just look at the bottom line but to dig deeper into what that information may be telling you. And as with so many things in bar leadership, she said, getting a good grasp on important data depends on the relationship between the executive director and the president.

“It’s the executive director’s job to present information in a way that’s meaningful,” she said. “And presidents, tell the executive director what’s important to you.”