Data quality issues (causes & consequences)

When you work with data (or “in data”), you inevitably encounter common data quality issues that create inefficiencies and process flaws that are hard to overlook. However, sometimes, it’s hard to put the problems into words and present a case to leadership or decision-makers. In this article, we’re trying to do exactly that.

If you are responsible for data or are trying to build a data quality management and data governance program in your organization, this guide will explain data quality issues, what causes them, and how they affect business initiatives.

Below are the top causes and consequences of poor data quality. If any of these seem familiar, it might be time to start addressing or improving your data quality management program.

If you already know the causes and consequences of low-quality data in your organization, you can download our Data Trust Playbook, a comprehensive guide to building data trust so that you can deliver trusted data that fuels business initiatives.

What we’ll discuss in this article

• What are data quality issues?
• 5 causes of data quality issues
• 8 consequences of bad data quality

What are data quality issues?

Data quality issues arise when your organization’s data is inaccurate, inconsistent, incomplete, or outdated. Essentially, a data quality issue arises when there is inaccurate data that does not reflect reality. It can happen due to missing values, duplicate records, differences between data sources, and many other reasons.

If data quality issues are not addressed,  low-quality data may be circulated and used throughout an organization, leading to poor business decisions that can have disastrous consequences.

What causes data quality issues? 5 common causes

There are various reasons that data quality issues and concerns can occur, often worsened by the impact of poor data governance. Here are 5 common causes of poor data quality:

1. Unclear ownership of data

Data ownership, which means assigning an owner to a data domain or data source, is critical, since lack of accountability is one of the most common data quality problems.

1. Without clearly defined ownership, there is no accountability for the data that is produced.
2. If a change is about to happen that might affect a particular data source or if anything goes wrong, you need to know who to alert about the change or who to contact to fix the data issue.

Not having documented data owners makes it hard to implement data quality improvement initiatives. Ownership is important because it imposes accountability for data quality.

2. Siloed operations

Teams often live in siloes. Business teams do not communicate effectively with each other, technical teams work individually, and business and technical teams are not connected. They may collaborate on data initiatives on a one-off basis, but the outcomes they produce are not documented or shared with others. This means the same issues are likely come up again on a new data project.

For example, data scientists typically spend 80% of their time collecting and cleansing data held by different teams in different locations. But if data owners, data stewards, data engineers, and data scientists all work together to create standards and expectations for critical data assets used for models and reporting, they can improve the quality of data. And with clean data more readily available, data scientists are able to spend less time preparing data and more time delivering strategic projects.

3. No data quality program in place

To conquer data quality issues, you need to look at them strategically. Data quality needs to become an enterprise-wide program, with shared tools, data quality rules, enablement, and reporting both on data quality metrics and the impact of improved data on business initiatives.

Data quality can emerge as a bottom-up push, but it absolutely needs to become a top-down motion, where the whole organization understands the importance and gets on board.

4. No visibility into the state of data and data flows

Teams don’t have an easy way to understand what data is available and how it flows through systems, which is one of the common data quality issues across enterprises. Without visibility into the data that exists in their organization, teams can’t understand its quality, leading to mistrust in the data.

This issue affects data scientists, data engineers, business analysts, business subject matter experts, and IT. However, proper tooling can solve this issue. A data catalog with data lineage and data quality capabilities provides visibility into data stored across systems and its quality, enabling teams to better understand and trust their data.

5. Manual data management processes

Many organizations manage their data manually. The following data processes are big time-wasters when done manually:

• Data collection
• Issue correction
• Data classification
• Data validation
• Data cleansing

Organizations without dedicated data quality tools, controls, and workflows for these processes end up repeating them. This can drain budgets and leave room for human errors to occur, leading to a higher chance of data quality issues and poor data quality.

What are the consequences of bad data quality?

If data quality concerns go unaddressed, they result in poor data quality that causes greater problems for your business in the present and future. Here are a few significant consequences of having unaddressed data quality issues:

1. The data engineering team is flooded with requests to fix data

Data engineers are usually responsible for fixing data quality concerns in organizations with complex data pipelines. They have to do this repeatedly, and it often takes a lot of time to find the root cause of the issue. While they hunt for the answer, more data quality issues can build up, causing poor overall data quality for the business.

Fixing all these issues leaves data engineers with less time to code or maintain checks, clearly showing the impact of poor data quality as problems grow exponentially with workload.

Data scientists spend about 60% of their time verifying, cleaning up, correcting, or even wholly scrapping and reworking data. They also spend about 19% of their time hunting and chasing the information they need.

A salesperson trying to call potential customers can waste up to 27% of their time due to contact details that are incorrect or incomplete.

2. Data-dependent teams don’t trust the data

It’s hard to be data-driven when you don’t trust your data. If your teams believe you have bad data quality, they won’t be able to do their work without double and triple-checking results. Here are signs that you have data trust issues:

• Business leaders don’t trust the reports
• Data scientists spend too much time validating and cleansing data
• Product teams are reluctant to use data to make decisions on new product development
• Teams are reluctant to use data from other business units

Since analyzing data is one of the primary ways to extract value, poor data quality leads to poor business decisions and failed projects.

Marketers waste 21 cents of every media dollar because of low-quality data. Missing or invalid customer contact information can cost an average large-sized company up to $238,000 per campaign. This is a clear example of the broader impact of poor data quality on business performance.

3. Long lead time for deriving value from data

If it takes you weeks or months to access data and create reports, something must be fixed.

Unfortunately, organizations with low data maturity operate in this fashion. When someone needs access to data, they go through a convoluted process of figuring out where that data is stored and who owns it. Then, they wait for approval to get access to that data. When they finally have access, they notice the data is inaccurate, and they either look for new data or try to fix it — on their own or by involving someone technical. By the time they’re finished, they have spent a lot of time correcting data inaccuracies and have likely used up a significant amount of other people’s time as well.

This complex issue stems from the lack of the right systems, data governance processes, and tools to manage data.

4. Your M&As didn’t go as intended

Mergers and acquisitions (M&As) are data-intensive activities, and 70-90% of them fail, with integration being one of the top reasons.

Indeed, M&As achieve little without integrating systems and data. That is why master data management best practices are so important.

If your company has been going through one or more M&As, look for the following signs of suboptimal M&As where data quality concerns were likely the issue:

• The integration timeline was extended
• Fewer systems were integrated or migrated than expected
• Organizations use inconsistent business language
• No single view of customers, employees, or other data domains exists

Learn more about data management for M&As here.

5. AI models have questionable ROI and performance

It’s a cliché at this point, but “garbage in, garbage out” holds true for AI and machine learning (ML) models built on bad data.

Data quality is one of the top factors influencing model performance, deployment speed, and long-term reliability. Top performers in AI generate up to 20% of their EBIT from AI models, but getting to that level requires solid investment in data management foundations, such as:

• Data & AI governance
• Automated data quality management
• Monitoring models for data drift

If you don’t have these basics in place, you’re likely dealing with one or several of the following issues:

• Frequent reports of data drift and long investigation times (days to weeks)
• Fewer models are deployed than expected on a consistent basis
• AI projects do not deliver the expected results (model accuracy)
• Head of AI or Chief Data Scientists regularly bring up data quality issues

Here are more resources on the importance of data quality for AI:

• How AI models and data products benefit from data quality and lineage tools
• AI-powered enterprise: A leader’s guide to strategic AI implementation in finance
• AI-powered enterprise: A leader’s guide to strategic AI implementation in insurance

6. System modernization projects go over time and budget

Data and system modernization are on the agenda of every data-driven and innovative organization. Modernization projects simplify the IT system landscape and data flows, consolidate billing and operations, and accelerate data-related activities. Some examples are ERP and core systems consolidation, CRM migrations, Customer 360 projects, and data consumption modernization, like moving from an on-premises data warehouse to a data lakehouse architecture in the cloud.

All of these projects depend on the state of your data. When data inaccuracies accumulate and information is not consistent or valid, projects eventually grind to a halt. You definitely have a larger problem with data if:

• Modernization projects go over time and budget
• Projects are often scrapped or put on hold

88% of all data integration projects fail entirely or significantly overrun their budgets because of poor data quality. 33% of organizations have delayed or canceled new IT systems for the same reason.

Poor data quality will impact about 10% of the savings you get from IT initiatives, meaning if you save $21.7 million in labor productivity, $2.2 million will be lost.

7. Reporting is manual, ad hoc, and unreliable

Accurate reporting is the bedrock of any data-driven organization. Companies in regulated industries, such as banking, insurance, and life science companies, must submit regulatory reports to authorities, which sets an even higher standard.

Here are some common pains that these organizations experience when dealing with data quality concerns:

• Reporting periods end, and the responsible teams must work overtime to manually compile data for reporting.
• Teams aggregate spreadsheets in a data mart manually.
• Authorities often turn down reports, and teams have to fix data quality concerns manually and prepare reports again.

Companies whose data does not meet GDPR standards risk fines of up to €20 million or 4% of annual turnover.

Noncompliant or inaccurate data can also result in costly business impacts, including:

• Business disruption: $5.1M
• Productivity loss: $3.8M
• Revenue loss: $4M
• Fines and penalties: $2M

8. Customer acquisition and retention metrics are degrading

Customer-centricity is the single most important factor for successful businesses. Companies built around and for their customers are 60% more profitable than others. They are also more likely to receive more information from their customers. It’s a virtuous cycle.

However, what happens when bad data quality causes customer data to degrade over time? Here is a non-exhaustive list of signs that your customer data needs attention:

• Marketing ROI is declining.
• Lack of agility and long lead times to prepare data for marketing campaigns.
• Marketing leadership questions, reporting & analytics.
• Frequent customer complaints around preferred methods of communication.
• Delayed billing & reconciliation.

Customer data typically degenerates at 2% monthly and 25% annually.

Individuals will receive offers they aren’t interested in, sometimes two or three times if a duplicate record exists.

Low-quality geolocation or inventory data can lead to delayed deliveries, scanning errors, and phantom stock-outs.

Here are more resources on the importance of data quality for customer data:

• The urgency of consolidating customer data with MDM: Are you facing these critical challenges and missing out?

Overcome data quality concerns today with our data trust playbook

Data quality concerns and bad data quality aren’t just a nuisance. They can significantly hinder your organization’s ability to thrive.

From operational inefficiencies and missed opportunities to failed projects and damaged customer relationships, the cost of neglecting data quality can be substantial, making it vital to explore data quality solutions for your organization. To ensure your organization’s data is a valuable asset rather than a liability, it’s crucial to proactively address data quality issues.

Download our free ebook, “The Data Trust Playbook“, and discover a comprehensive guide that explores data quality issues and solutions to help you build a robust program that improves data trust.