Data Driven Decisions: Full Guide

by Yuliya Sychikova
COO @ DataRoot Labs

Data driven decisions help businesses not only increase profitability but also avoid costly mistakes. Companies that build corporate culture based on data receive a competitive advantage. By adopting data driven strategies, companies avoid falling into a trap of gut-based decision making and ultimately increase their business profitability. That is why we hear more and more about the benefits of data analysis, data drivenness, data science, data visualization, predictive modeling, forecasting, scorings and so on. In this article, we provide a step-by-step guide on how to incorporate data-driven analysis into your organization.

Better decision making with data-driven approaches

Common decision biases and how to eliminate them with data.

  • Confirmation bias: using all the data available helps overcome a tendency of only looking at the information that confirms preexisting opinions.
  • Attention bias: grounding decisions on data driven results allows to wisely prioritize tasks and stop cherry-picking the ones that seem important.
  • Replacing optimism with realism: hard cold data changes assessments mode from optimism to realism, combating the chance of overestimating a positive outcome and underestimating a negative one.
  • Anchoring: data-driven insights based on various information sources allow for unclouded judgement, and prevent mental tendency to depend on an initial piece of information or the most obvious data while ignoring the rest.
  • Reliance on HIPPO (highly paid person opinion): with data in place, ideas are evaluated depending on numbers, regardless of who came up with an idea.

Main hurdles in adopting data driven methodology

Obstacles to a data driven organization.

If the benefits of data driven decisions are so obvious, why many businesses struggle, fail or give up on adopting them? For the most part, companies face two following problems:

  1. People. Company's management can be simply unaware or unfamiliar with the data-driven methodologies and their integration methods. Additionally, changing the way organizations work applies pressure on the management and companies resources causing resistance from within.
  2. Technology. With a range of solutions on the market, the enterprises may pick the unsuitable offering or face technical difficulties with the introduction of data storage, data analysis, and data visualization into their business processes.

Although both of these problems seem daunting, they can be solved by developing a solid transition strategy, introducing proper employee training and recruiting professionals from the market that specialize in various data domains.

Incorporating data driven methodologies into your organization

A step-by-step guide.

Learn about the first basic steps and it's components to help your business transition to a data driven enterprise.

Step 1: Collect and store the data

To make decisions you need the data, simple as that. To determine which data storage is better to use, you need to answer the following questions:

What type of data do you want to collect?

  • Structured. Simply speaking, it’s a set of interconnected tables. The typical examples of databases include PostgreSQL, MySQL, and HP Vertica. They are convenient because data can be highly accurate and provide a complete picture of any business component and process. Additional advantage of using structured databases is that data can be extracted quickly, which is a big plus for analytics.
  • Unstructured. With these databases, you don't need to worry about the architecture of databases in advance, and can start collecting data right away. The typical examples of such databases include Hadoop and MongoDB. On a flip side, the analysis of unstructured data is harder and requires specific expertise such a knowing the programming languages. Hence, businesses might have more troubles with managing it vs. the old-school structured data.

What volume of data do you plan to store?

Where are you going to keep this data?

  • Buy a dedicated server. With dedicated servers, businesses have to use internal IT capacities and expertise to manage ongoing maintenance, patches and upgrades. It’s reasonable when you have heavy I/O applications i.e. big data platforms.
  • Buy a cloud server or cloud storage. In case of the cloud servers, you can optimize IT performance without the costs associated with purchasing and managing fully dedicated infrastructure. The downsides to consider include concerns associated with vulnerability to cyber attacks, speed of data management, lifetime costs, and scalability.

Who will backup your data infrastructure?

  • Outsourcing partner outside your organization.
  • A dedicated team within the project.

Step 2: Presenting the analytics to your team

After collecting the data, you need to think of how you would deliver data-driven insights to your team. The way you present the data determines how much your organization and your team gain from it and whether the process will stay in place. Multiple business intelligence tools such as Excel, Tableau, and Matplotlib can pull together complex sets of data and present it in an simple and user-friendly way. In our article Data visualizations with Tableau, we describe in detail how data visualization impacts enterprises. Which data visualization tools are best for your company? Below is the list of data visualization tools split by the result you are going after and the skill-level needed:

SIMPLE VISUALIZATION: Excel

AnalogsGoogle SheetsLibreoffice

Required Skills: Common

Excel works well with simple charts and is great for what-if analysis. If you need sophisticated visualization, it’s possible to connect Excel with Power BI or Tableau. Additionally, Excel works with CSV files making it easy to upload the data from your tables in data storages such as MySQLPostgreSQLHP Vertica among others.

ADVANCED VISUALIZATION: Tableau

AlternativesPower BIQlikviewSpotfire

Required Skills: Medium / Advanced

Tableau is a powerful BI tool for analyzing data. Among many, it has the following useful core features:

CUSTOM VISUALIZATION: Matplotlib

Alternativesggplot2bokehplot.ly

Required Skills: Advanced

Matplotlib is often used for visualizing data when developing data science and machine learning projects. It contains a lot of advanced toolboxes and offers great visualization libraries.

Step 3: Hire the right team for developing data driven methodology

To make data driven vector stick within your organization, you need a team with a specific skill set. Their role encompass data collection, data cleaning, visuzalization, current product improvement and new product building. So, what are those roles, and how do they fit in a corporate structure?

  • Data Analysts

    They range from entry-level professionals to highly-specialized and skilled analysts. While the former mostly gather and prepare the data, the latter can be general experts or have specific industry experience that can range from understanding loyalty programs and e-mail marketing to stock market and forex.

    Required Skills: ExcelSQL

  • Data Engineers and Analytics Engineers

    They are responsible for obtaining, cleaning, and munging data and shaping it into a form that analysts can access and analyze. Data Engineers and Analytics Engineers also handle the operational tasks such as throughput, scaling, peak loads, and logging, and building business intelligence tools.

    Required Skills: HadoopHivePigSqoopFlumeSpark SQL

  • Data Scientists

    They typically hold advanced degrees in quantitative subjects and can solve complex problems such as recommendation engines or help building products with machine learning, natural language processing, and predictive modeling elements.

    Required Skills: Spark MLlibPySparkHadoopIgniteSparkHiveKafka

  • Statisticians

    They focus on statistical modeling across the organization and often have an advanced degree in statistics. Statisticians typically handle design of surveys, experiments, and collection of protocols to obtain the raw data as well as building statistical models and predictions.

    Required Skills: ExcelWordSASRLaTeX

  • Quantitative or financial analyst

    With a degree in Math, Physics, or Engineering, such analysts work in the financial services sector modeling securities, risk management, and stock movements. Their domains can range from statistical arbitrage and quantitative investment management to algorithmic trading and electronic market making. Some of them are especially strong programmers in languages that can process data and generate actions with low latency.

    Required Skills: RC++C#

The ideal team composition, of course, is very company-specific and depends on its size, activity type and goals. For small and medium size companies, we recommend starting with one general data science expert who will develop a roadmap for the data science initiatives and then help with staffing the team. Large enterprises typically have a separate data teams per a line of business comprised of a few data analysts, scientists, and engineers.

If you decide to hire an internal team, make sure you ask them some of these questions. Keep in mind that hiring a data scientist takes on average 4-6 months.

If you decide to outsource the data science department to an expert company, Clutch is a good starting point as it provides a list best vendors globally with reviews and expertise descriptions.

Step 4: Building the data-driven culture

Building a new team of data professional within the organization alone is unlikely to ensure a smooth transition to data-driven methodologies. As with other big changes, you need to gain a wide-spread adoption by developing a strategy to make the new approach stick. There are three main ways to Reinforce the Data-Driven Culture inside your enterprise:

  1. Find the allies. Find the early adopters within your business who are as enthusiastic about the new methodology as you are. They will help you to spread the word, fight the resistance and convince the skeptics.
  2. Share the information. Unless you work with very sensitive data, it is essential to provide a broad access to analytics, which ultimately involves sharing the information across business units, teams, and individuals, regardless of the employee's responsibility scope. Also, asking uncomfortable questions helps change the way employees are used to make decisions. Based on what information was this decision made? Where did you get this info? Would you show it to me?
  3. Encore the data to organization's and employee's specific goals. Introducing numbers liked to analytics to KPI and OKR frameworks encourages your employees to draw hypotheses, build experiments, and learn from the results.

Conclusions

Still not convinced about the benefits of the data driven decision making process? Here are two facts for you from companies that have successfully introduced the data driven decision making methodologies:

  1. On average, the most data-driven companies found to be 4% more productive and 6% more profitable, if comparing with the counterparts.
  2. The majority of the Fortune 1000 companies have been able to cut losses by employing data-driven business strategies.

Launching the data-driven processes in the organization is hard, and doing so in the right way manner requires time, new expertise, and additional resources. However, after the transition is complete, your business will see unparalleled benefits. Data driven management gives framework for solving the critical business problems, such as smart task prioritization, customer understanding, transparent communication between employees and departments. Together, those factors result in a higher bottom-line profitability and lower costs for organizations.

If you have any questions, contact us, and we will gladly share our knowledge to help you in the matter.


Updated Aug 11, 2019 — 00:00 UTC