The Rise of Connector Roles in Data Science

News

1. Core42 and Qualcomm Launch Compass 2.0 to Boost AI Efficiency and Accessibility

2. Data Debt Emerges as Core Business Issue for Enterprises

3. Are Organizations Investing in AI Achieving Positive ROI?

4. Are Increasing Breaches Due to the Cyber Skills Gap?

5. What Are the Challenges Firms in the UAE and Saudi Arabia Face in GenAI Implementation?

6. Enterprises See the Promise of Generative AI But Lack the Guardrails to Mitigate Operational Risks

For all of the current focus on using data, analytics, and AI to improve organizational decisions and operations, too many data science projects fail. Even for those that succeed, progress is often slow and expensive. Why? Organizational gaps between teams are wreaking havoc with the ability to develop, apply, and scale data science projects. A new type of role is needed to bridge these gaps.

Let’s not minimize the many aspects of data science that can trip a company up — from defining the right business problem to developing a solution and then supporting it properly. However, our diagnosis of the situation, researched in part with our colleagues Roger Hoerl and Diego Kuonen, shows that many companies struggle less with the solution and more with the organizational issues.

We’ve identified three root causes for this. First, data science has been “bolted on” to the organizational chart. This might have been acceptable a generation ago, when data science was reserved for relatively few problems, but it needs to be built in now that data science matters to every team.

Second, data science aims to improve day-to-day operations and decision-making, and that means disruption. But most line managers strive for control, predictability, and no surprises. Given that disruption and control are polar opposites, there is a far greater-than-normal tension between line management and data science.

Third, many companies seem to expect data scientists to do it all: magically understand the business and all its nuances, deal with data quality issues, and convince recalcitrant managers and staff members to change their long-cherished processes. Many data scientists prefer to simply create better algorithms, which means those other tasks often receive short shrift.

There is no minimizing these issues — they are deep, structural, and often political and may well demand additional management strategies. Here, we will address one critical aspect of the organizational problem, namely the gaps between the various departments that inhibit the development and deployment of data science models.

Closing the Gaps With Connectors

The obvious way to close organizational gaps is to fill them with people. We generically call people in these roles connectors. For some time, companies have assigned people who have job titles such as systems analyst, business analyst, coverage officer, and systems engineer to close gaps between business and technical departments. They’ve also tried to close gaps that crop up in data science projects, labeling the people tasked with doing so data wranglers, data engineers, purple people, and translators.

While many such efforts have helped move data science projects along, it often feels like an elaborate game of whack-a-mole: Close one gap, and another pops up. Also, these efforts alone don’t resolve some of the related issues:

• Line managers and data scientists often don’t understand each other. This makes it particularly difficult for both parties to agree on the problem to be solved and for data scientists to obtain and understand the nuance in needed data.
• Implementing a model requires extensive changes within the business: redesigned processes, training on how the models work, and upskilling to run them.
• The two sides might not see eye to eye on IT infrastructure decisions, such as those related to technical debt.

For those reasons, we recommend that companies approach the issue in a more disciplined manner, following these three steps.

1. Define the data science project process and identify the people involved. While the specifics might vary, the process should start with framing the problem and understanding the business context and end with monitoring/updating the new model in production. The critical work: identifying who, outside the data science team, must be involved in each step.

For example, when framing the problem and selecting a project, companies want to involve both senior leaders and middle managers whose teams will be affected. When it’s time to obtain and prepare data, business domain experts who understand the data nuances and IT professionals who prepare data should be in the room. When developing the model, external software providers should be consulted. When it’s time to insert the model into processes, business and process analysts, managers of the affected processes, trainers, and change professionals should all be involved. Finally, as an organization deploys the model into the IT infrastructure and then monitors and updates it, IT professionals, people building the data pipes, and supply chain managers should contribute.

2. Evaluate where connectors can help. Essentially, this step aims to clarify the gaps. Ask, “Are all of the people involved fully connected — that is, do they understand their roles, and is there sufficient communication such that work gets done in a reasonably effective manner?”

We find that most often the answer is “no.” Indeed, we find that connectors are often essential in the following ways:

• Framing the problem to be solved with data science: For example, executives might have heard some of the hype surrounding AI but lack a deep enough understanding of what problems it can help solve, or the technology’s potential across the organization.
• Translating between business and technical people — an age-old gap that has yet to be closed.
• Dealing with data quality issues.
• Communicating requirements, progress, and issues within the team — in other words, tying the many pieces of a data science project together and being the connector between diverse teams.
• Keeping track of progress toward the overall goal of deployment and organizational change when nobody else sees the big picture.

3. Clarify specific roles for connectors and put them in place. This step is generally straightforward for individual projects. We recommend that companies also consider having connectors between the entire data science program and the rest of the business. There is a natural tension between line managers, who work hard to establish a measure of control in their everyday work, and data science professionals, who aim to disrupt things.

Left unchecked, this tension makes it extremely difficult for a data science team to establish itself within a company. Connectors at the program level aim to ease this tension: They help senior business leaders understand both the potential and challenges of data science, help data science leaders understand the top problems facing the business, and establish a portfolio of data science projects that aligns with business needs.

What Companies Are Trying Now

Given that building the organizational capabilities to succeed with data is so important, we are delighted to see companies experimenting and enjoying success with a wide range of connector roles. For example, Regions Bank and many other organizations make use of data product managers, who take an end-to-end perspective. Forty percent of organizations now take a data product management approach, according to a recent survey of chief data officers.

Gulf Bank in Kuwait has created a network of data ambassadors to lead quality and data science projects at the department level. The Federal Statistical Office of Switzerland employs a “data science bridge” to coordinate its many statistical functions. Kroger’s data science subsidiary, 84.51º, has a set of what it calls solution design consultants — people with business backgrounds who mediate between data scientists and business stakeholders. (84.51º refers to the longitudinal location of the subsidiary’s Cincinnati office, and like early purveyors of longitude maps, the company makes sense of complex data problems.) Kroger’s 84.51º also uses technology in the form of machine learning operations tools to monitor the status of data science projects from inception to ongoing management and keep all parties informed. Many other organizations have also adopted such connector roles and tools.

Let’s examine the work at Regions Bank and the Federal Statistical Office of Switzerland in more detail.

Data Product Partners at Regions Bank

Manav Misra, chief data and analytics officer of Regions Bank, observed that many data science projects were not being successfully implemented, so he established the concept of data products and the new connector role of data product partner to manage them.

Misra defines a data product as an end-to-end software solution to a problem that the business has. “A data product leverages data and advanced analytics heavily in order to deliver that solution,” he said. Data product partners function as both data product managers and partners with the business who bring products to fruition and manage them over time.

Each data product partner is assigned to a particular business or support unit of the bank and attends staff meetings with their business partners in order to better understand their priorities. Most partners come from the business side of the bank, though some also have analytics backgrounds.

The job of the data product partner includes understanding what the business unit and end users need and starting the development cycle with a minimum viable product. During the early stages of the process, data product partners serve as bidirectional translators, connecting the opportunities of analytics and AI with business needs. At the later stages, they are responsible for assessing how the complete solution is adopted and used, monitoring how well the user interface is working, keeping track of how many people use it, and ensuring that the product delivers or exceeds the value promised in the original business case.

The Data Science Bridge in Switzerland

The Federal Statistical Office in Switzerland recognized that it needed to improve its core data science processes if it was to meet the growing needs of the Swiss government, so it created a data science competence center, a type of center of excellence. To make sure the competence center was fully connected to the rest of the Swiss government, leaders staffed a “data science bridge” — a group of connectors specifically charged with managing six high-level processes:

• Driving collaboration.
• Developing human capital.
• Ensuring data quality.
• Managing the project portfolio.
• Ensuring a positive business impact.
• Ensuring a human-centric and trustworthy application of data science.

The bridge is paying real dividends. It has facilitated communities of practice in the customer base, helping people become more effective users of data tools and reports and encouraging them to exchange their experience with data science and AI, based on a common language and understanding. The bridge is reducing chaos in the six processes listed above.

Moreover, “most important decisions are taken on the bridge,” noted Kuonen, who serves as an adviser to the Swiss government on data science. (Kuonen is also head of Statoo Consulting, a Switzerland-based consultancy, and professor of data science at the Geneva School of Economics and Management at the University of Geneva.) Kuonen believes that the bridge will deliver more value as it matures.

Connector Management Challenges: Five Questions to Ask

Despite the apparent advantages of connectors, managing them presents some tough issues. In some respects, connectors operate in “white space” in the org chart, where management is always tough. For example, companies have not had much luck managing data stewards as they seek to align the definitions of various data elements across a company.

Companies should think through the following questions as they define their connector roles. The answers will vary by organizational circumstance, but some patterns are beginning to emerge.

1. Where should connectors sit, organizationally? Most commonly, connectors sit under the chief data and analytics officer or the data science organization, but they are sometimes found in business-side groups. If the company has been successful at matrix management for other roles, connectors could report to both areas.

2. How should companies attract, evaluate, and retain connectors? As data expert Theresa Kushner has pointed out, connector roles are challenging, given that they require knowledge of both the business and data, as well as considerable soft skills. There are now several hundred analytics and data science degree programs in business schools, which are a good place to recruit connectors. Organizations putting together a team of connectors often ask business leaders to evaluate candidates for such roles; Procter & Gamble, for example, asked leaders to identify one “breakthrough business result” that each candidate helped to achieve. Retention will depend in large part on offering these hires a viable career path.

3. What is the connector career path? Experienced technical people may very well recognize the need for connectors but fear that if they take such a role, they will lose their technical edge. Experienced business people might not want to fall off their current career path within their business unit or function. In the short run, companies should establish roles of increasing responsibility in the data science organization, such as senior and well-compensated data product managers. We believe that in the long run, many organizations will want every senior executive to have data science experience in their background.

4. Is more than one type of connector needed? Since some connecting is performed at very high levels, while other connecting is focused on the detailed technical level, the logical answer is “yes.” But a company might not be able to afford to hire more than one for a project, so some companies are asking data product managers to play a lower-level connector role as well. Other organizations ask connectors to play their roles across two or more projects at once.

5. What exactly are a connector’s responsibilities? Since connectors might have the best overall view of a project, it is common to charge them with managing it end to end (as in the role of data product manager). This is appropriate if connectors are experienced and if they understand and value the technical roles on a data science project. If they don’t, their connector responsibilities should be more circumscribed, such as translator roles that don’t involve end-to-end responsibility for project success.

More detailed answers to these questions will emerge for your organization over time.

________________

In our work with companies that have successfully employed connector roles, the software vendors and data science leaders involved have told us that the deployment percentage of data science projects has increased as connectors have proliferated. We believe that this is an important and ongoing set of roles that needs further institutionalization in almost every large company. Otherwise, data science might become a dismal science with a low deployment percentage and a poor record of improving business performance.

 

---

Keen to know how emerging technologies will impact your industry? MIT SMR Middle East will be hosting the second edition of NextTech Summit.