DAMA Webinar - Big and Little Data Quality

LITTLE DATA IN A BIG
DATA WORLD
The case of health care
Dataversity / DAMA
February 2016
Laura Sebastian-Coleman,
Ph.D., IQCP
Offered by: Connecticut General Life Insurance Company or Cigna Health and Life Insurance Company.

About me
• Doing data quality in health care since 2003
• Have worked in banking, manufacturing, distribution,
commercial insurance, and academia.
• All have influenced my understanding of data, quality,
and measurement
• Developed the Data Quality Assessment Framework
(DQAF); published in Measuring Data Quality for
Ongoing Improvement (2013).
• IAIDQ Distinguished Member Award 2015.
• DAMA Publications Director, beginning summer 2015
• Influences on my thinking about data:
• The challenge of how to measure data quality
• The concept of measurement itself: A problems of
measurement is a microcosm of the general
challenge of data definition and collection.
• The demands of data warehousing, especially of data
integration.
2

• Abstract:
– While technological innovation brings constant change to the data landscape, many
organizations still struggle with the basics: ensuring they have reliable, high quality data.
– In health care, the promise of insight to be gained through analytics is dependent on ensuring
the interactions between providers and patients are recorded accurately and completely.
– A lot of health care data is dependent on person-to-person contact. This fact influences its
quality, as well as how it is captured, stored, accessed and used.
– This presentation will ask the audience to think about data in old and new ways in order to gain
additional insight about how to improve the quality of data, regardless of size.
• Order of information
– The promise of big data in health care vs. the reality of the health care system
– The data quality in health care
– How we think about data and data quality
– Making good on the promise of health care analytics
Abstract and Agenda
3
Confidential, unpublished property of Cigna. Do not duplicate or distribute. Use and distribution limited solely to authorized personnel. © 2015 Cigna

• According to the blog Big Data Landscape, Big Data Health Care apps enable people
and organizations to
– Make well-informed decisions
– Enhance patient engagement
– Encourage prescription medication compliance
– Adjust lifestyle choices
– Maximize levels of well-being
– Predict and prepare for individual illnesses
– Manage demand within the health care system
• In Health Care Delivery of the Future, Price Waterhouse Coopers describes a New
Health Economy in which:
– Digitally-enabled care is a fundamental business imperative.
– There are major shifts in how care is being delivered.
– Digital technology bridges time, distance and the expectation gap between
consumers and clinicians.
The Promise of Big Data in Health Care
4

• At a recent insurance summit in
Hartford, Connecticut, health care
executives grappled with a
quandary:
“Only in health care does
innovation lead to higher costs.”
• Not only that, but within the health
care system, there are
inefficiencies. One executive
estimated
“Thirty cents on the dollar, at least,
is not used effectively.”
• Getting people to act in the best
way for their health is
“Very easy to say. Very hard to do.”
The Reality of Health Care
5

• The promise of health care analytics and big data depends on having high quality data.
• Technology limitations
– Data output from older, legacy claim adjudication systems differs greatly within and
between organizations
– Medical provider transition from paper to Electronic Health Records (EHR) is
relatively recent and still underway
– In 2008, only 4% of Providers had full EHR, 13% had a basic system
• Lack of standards /
standards implemented
inconsistently
• No standards for data quality
Challenges with the Quality of Health Care Data
6

• Goal: Improve knowledge of the quality of data gathered in non-clinical trial settings.
– As research data warehouses and large scale data networks begin to become
established sources of observational data, it becomes more critical that consistent
methods for assessing and reporting data quality are developed and adopted so that
users of data and consumers of results understand the potential impact of data
quality on study results.
• Factors that can cause misrepresentation of clinical events:
– Inflexible systems
– Coding practices
– Gaps in standards
• Example: Screening for high blood pressure in children
– Clinicians were directed to screen children for high blood pressure.
– After the initial screening, they continued to monitor some of the children.
– When researchers looked at the data, it appeared children were increasingly at risk
of high blood pressure.
– Examination of records showed that the ‘increase’ was caused by incorrect use of
hypertension diagnosis code
– But there was not a diagnosis code for “considering hypertension”
Academy Health Example
7

• Person-to-person interactions
between patients and health care
providers
• Interactions influence factors that
may result in different interpretation
of symptoms
• Interactions are represented by data
• Different choices in how to
represent these interactions create
different data stories
Health Care data is about people
8

Definition: Data
• Data’s Latin root is dare, past participle of to give. Data means “something given.” In
math and engineering, the terms data and givens are used interchangeably.
• The New Oxford American Dictionary (NOAD) defines data as “facts and statistics
collected together for reference or analysis.”
• ISO defines data as “re-interpretable representation of information in a formalized
manner suitable for communication, interpretation, or processing” (ISO 11179).
• Observations about the concept of data
– Data tries to tell the truth about the world (“facts”)
– Data is formal – it has a shape
– Data’s function is representational
– Data is often about quantities, measurements, and other numeric
representations “facts”
– Things are done with data: reference, analysis, interpretation, processing
• What the definitions leave out:
– Data is made by people. We choose what characteristics to represent. The creation
of data implies a set of expectations about data’s condition.
– People also use data. The uses of data imply a set of expectations about data’s
condition.
9

Science
– Focuses on measurement as a
means to create knowledge
– Plans for data that is accurate
and complete
– Sets standards so that results
are collected correctly and can
be understood and reproduced
– Tests and re-tests
• Data is a PRODUCT of science.
• High quality data is a necessity.
Commerce
– Creates measurements
pragmatically to meet goals
– Generates data by accident
while executing its core
processes
• Data is a BY-PRODUCT of
commerce
• Historically, data quality has not
an end in itself for commerce.
Science, Commerce and Data Quality
10
• The History of Measurement = history of
our ideas about data
• Science & Commerce have different
goals for and approaches to
measurement and thus to data.
• Our ideas about data come largely from
science, but we create data based on
commerce.

The Scientific Method vs. Commerce

Data Quality
• Our ideas about data quality
come largely from science,
even though we create data
based on commerce.
• Today, we are using
organizational data in
scientific ways – to learn
about our business.
• We expect the data to be fit
for this purpose, but we
have not focused on
ensuring representational
effectiveness.
12
Fitness for
purpose
Representational
effectiveness

• Even scientists are struggling with data quality in the age of Big Data.
• A February 2015 article in Science News reported researchers are struggling to find
insights as they sort through “mounds of data”:
“Just keeping track of big data is a monumental undertaking. Sharing the data
with other researchers, a critical piece of transparency and efficiency in science,
has its own set of problems. And the tools used to analyze complex datasets are
just as important as the data themselves. Each time a scientist chooses one
computer program over another or decides to investigate one variable rather than
a different one, the decision can lead to very different conclusions.”
Data Quality and Big Data
13
• The inability to reproduce results has
brought some studies into question.
• The answer? Study the factors that
influence reproducibility.
• They have discovered a kind of “butterfly
effect” in studies using big data:
– Small changes in the initial conditions
of an experiment can have significant
effects on the outcome of replication
attempts.

Butterfly effect in the clinical space
A) Decision features
a. Framing (e.g., gain vs. losses) (2 factors)
b. Order of choices (e.g. A à B vs. BàA in a simple two
choice decision) (2 factors)
c. Choice justification (e.g., effect of regret, guilt etc. on
dissonance reduction; yes vs. no) (2 factors)
B) Situational factors
a. Time pressure (e.g., yes vs no) (2 factors)
b. Cognitive load (e.g., high vs. low) (2 factors)
c. Social context (e.g., important vs. not important) (2
factors)
C) Characteristics of decision-maker
a. Individual [e.g., age (old vs. young), gender (female vs.
male) (4 factors)
b. Group (e.g, small vs. large group) (2 factors)
c. Cultural factors (e.g., present vs. not preset/important) (2
factors)
D) Individual differences
a. Decision styles (e.g. intuitive vs. analytic) (2 factors)
b. Cognitive ability (e.g., high vs. low) (2 factors)
c. Personality (e.g., openness, conscientious, extraversion,
agreeableness, neuroticism) (“Big 5” factors)
Table 1. Minimum number of the factors affecting decision-making
From Effect of Initial Conditions on Reproducibility of Scientific
Research, by Benjamin Djulbegovic and Iztok Hozo
• Small changes in the initial
conditions of an experiment can
have significant effects on the
outcome of replication attempts.
• Researchers used Doctor/Patient
interactions to study the butterfly
effect and identified 12 factors
that influence clinical decision
making.
• Those initial factors make up
20,480 combinations that could
represent the initial conditions of
the experiment.
• Yes, 20,480! Initial conditions can
influence clinical decision making
and the data that is recorded as
part of it.

The butterfly effect takes on a whole new meaning
15

• Recognize that there is variation within the health care system and use this variation to
improve the system.
• Reduce variation to simplify the system.
• Recognize that data is a critical product of the health care system and plan for quality
data.
Making good on the promises of health care analytics
16

• Variation may itself be meaningful.
• For example, variation in how doctors make decisions
– If we understand how doctors make decisions, and we understand he outcome of
those decisions, then we can assess the results of those outcomes and provide that
feedback to the health care system.
• This is the basic promise of evidence-based medicine and health care analytics, but it
requires a lot of data.
Making good on the promises of health care analytics:
Recognize variation
17

• Establish and enforce data standards within health care
– Including data quality standards
• Collect more data through instruments, devices to reduce ambiguity of meaning
– In 2015, PWC reported that 28% of consumers have at least one health related app
on their tablet or smart phone.
– Up from 13% just three years earlier.
– About two-thirds of physicians reported that they would be willing to prescribe an
app that would help patients manage a chronic disease.
– Being able to integrate this kind of data into an electronic medical record represents
an additional opportunity.
• Enable feedback about the data from health care providers, patients, and other
stakeholders
– More health care data is being exposed directly to physicians and to consumers.
– Physician compensation and Consumer incentives depend on this data.
– To reduce costs and improve quality of care, health care companies will need means
to respond to feedback and ensure data is accurate.
Reduce variation
18

• Recognize that improvement in health care and in quality of life depend on having
trustworthy, reliable data.
– Data is not secondary to successful outcomes, it is central to successful outcomes.
• Focus from the start on ensuring the quality of that data, whether that data is little
(such as a record of the facts of an office visit) or big (petabytes of biometric data
collected from all the Fitbits in the USA).
• High quality data is:
– Clearly defined
– Consistently collected through reliable processes
– Presented through comprehensible standards
• The processes and systems that we build to create this new health care data should
be designed with the data itself in mind.
– The industry as a whole must cease to create data as a by-product and recognize
data as critical input to its primary goal: helping improve people’s health, well-being
and sense of security.
Plan for data as a product
19

References
• http://www.bigdatalandscape.com/blog/big-data-health-find-apps-can-improve-wellbeing
• http://www.pwc.com/us/en/health-industries/top-health-industry-issues/assets/pwc-healthcare-delivery-of-
the-future.pdf
• http://touch.courant.com/#section/-1/article/p2p-84932278/
• https://en.wikipedia.org/wiki/Electronic_Medical_Record#United_States
• http://www.academyhealth.org/
• http://www.pcori.org/about-us/why-pcori-was-created
• http://courses.washington.edu/geog482/resource/14_Beyond_Accuracy.pdf
• https://www.sciencenews.org/article/redoing-scientific-research-best-way-find-truth
• https://www.sciencenews.org/article/big-data-studies-come-replication-challenges
• http://www.scopemed.org/fulltextpdf.php?mno=162185
• http://consumerhealthchoices.org/wp-content/uploads/2015/05/HealthDataGuide-June2015.pdf
• http://www.pwc.com/us/en/health-industries/top-health-industry-issues/assets/pwc-healthcare-delivery-of-
the-future.pdf
• http://kff.org/health-reform/fact-sheet/summary-of-the-affordable-care-act/
• http://www.forbes.com/sites#/sites/emc/2014/01/22/can-big-data-and-mobile-make-health-care-more-
effective/
• http://www.nytimes.com/2016/01/17/opinion/sunday/how-measurement-fails-doctors-and-
teachers.html?_r=0
•

All Cigna products and services are provided exclusively by or through operating subsidiaries of Cigna Corporation, including Cigna Health and Life Insurance Company,
Connecticut General Life Insurance Company, Cigna Behavioral Health, Inc., and HMO or service company subsidiaries of Cigna Health Corporation. The Cigna name, logo,
and other Cigna marks are owned by Cigna Intellectual Property, Inc.
000000 00/15 © 2015 Cigna. Some content provided under license.

DAMA Webinar - Big and Little Data Quality

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (14)

Similar to DAMA Webinar - Big and Little Data Quality

Similar to DAMA Webinar - Big and Little Data Quality (20)

More from DATAVERSITY

More from DATAVERSITY (20)

Recently uploaded

Recently uploaded (20)

DAMA Webinar - Big and Little Data Quality