Impact of providing patients access to electronic health records on

NevesAL, etal. BMJ Qual Saf 2020;0:1–14. doi:10.1136/bmjqs-2019-010581

SYSTEMATIC REVIEW

► Additional material is

published online only. To view

please visit the journal online

(http:// dx. doi. org/ 10. 1136/

bmjqs- 2019- 010581).

For numbered afﬁliations see

end of article.

Correspondence to

Dr Ana Luisa Neves, Patient

Safety Translational Research

Centre, Institute of Global

Health Innovation, Imperial

College London, London, UK;

ana. luisa. neves14@ imperial.

ac. uk

Received 4 November 2019

Revised 12 May 2020

Accepted 20 May 2020

To cite: NevesAL, FreiseL,

LaranjoL, etal. BMJ Qual Saf

Epub ahead of print: [please

include Day Month Year].

doi:10.1136/

bmjqs-2019-010581

Impact of providing patients access

to electronic health records on

quality and safety of care: a

systematic review andmeta- analysis

Ana Luisa Neves ,

1,2

Lisa Freise ,

Liliana Laranjo,

3,4

Alexander W Carter,

Ara Darzi,

Erik Mayer

employer(s)) 2020. Re- use

permitted under CC BY.

Published by BMJ.

ABSTRACT

Objective To evaluate the impact of sharing electronic

health records (EHRs) with patients and map it across

six domains of quality of care (ie, patient- centredness,

effectiveness, efﬁciency, timeliness, equity and safety).

Design Systematic review and meta- analysis.

Data sources CINAHL, Cochrane, Embase, HMIC,

Medline/PubMed and PsycINFO, from 1997 to 2017.

Eligibility criteria Randomised trials focusing on adult

subjects, testing an intervention consisting of sharing

EHRs with patients, and with an outcome in one of the

six domains of quality of care.

Data analysis The Preferred Reporting Items for

Systematic Reviews and Meta- Analyses guidelines were

followed. Title and abstract screening were performed

by two pairs of investigators and assessed using the

Cochrane Risk of Bias Tool. For each domain, a narrative

synthesis of the results was performed, and signiﬁcant

differences in results between low risk and high/

unclear risk of bias studies were tested (t- test, p<0.05).

Continuous outcomes evaluated in four studies or more

(glycated haemoglobin (HbA1c), systolic blood pressure

(SBP) and diastolic blood pressure (DBP)) were pooled as

weighted mean difference (WMD) using random effects

meta- analysis. Sensitivity analyses were performed for

low risk of bias studies, and long- term interventions only

(lasting more than 12 months).

Results Twenty studies were included (17 387

participants). The domain most frequently assessed was

effectiveness (n=14), and the least were timeliness

and equity (n=0). Inconsistent results were found for

patient- centredness outcomes (ie, satisfaction, activation,

self- efﬁcacy, empowerment or health literacy), with

54.5% of the studies (n=6) demonstrating a beneﬁcial

effect. Meta- analyses showed a beneﬁcial effect in

effectiveness by reducing absolute values of HbA1c (unit:

%; WMD=−0.316; 95% CI −0.540 to −0.093, p=0.005,

=0%), which remained signiﬁcant in the sensitivity

analyses for low risk of bias studies (WMD= −0.405;

95% CI −0.711 to −0.099), and long- term interventions

only (WMD=−0.272; 95% CI −0.482 to −0.062). A

signiﬁcant reduction of absolute values of SBP (unit: mm

Hg) was found but lost in sensitivity analysis for studies

with low risk of bias (WMD= −1.375; 95% CI −2.791 to

0.041). No signiﬁcant effect was found for DBP (unit: mm

Hg; WMD=−0.918; 95% CI −2.078 to 0.242, p=0.121,

=0%). Concerning efﬁciency, most studies (80%,

n=4) found either a reduction of healthcare usage or

no change. A beneﬁcial effect was observed in a range

of safety outcomes (ie, general adherence, medication

safety), but not in medication adherence. The proportion

of studies reporting a beneﬁcial effect did not differ

between low risk and high/unclear risk studies, for the

domains evaluated.

Discussion Our analysis supports that sharing EHRs

with patients is effective in reducing HbA1c levels, a

major predictor of mortality in type 2 diabetes (mean

decrease of −0.405, unit: %) and could improve patient

safety. More studies are necessary to enhance meta-

analytical power and assess the impact in other domains

of care.

Protocol registration http://www. crd. york. ac. uk/

PROSPERO (CRD42017070092).

INTRODUCTION

Providing patients with access to elec-

tronic health records (EHRs) may improve

quality of care by providing patients with

their personal health information, and

involving them as key stakeholders in

the self- management of their health and

disease.

With the widespread use of these

digital solutions, there is a growing need

to evaluate their impact, in order to better

understand their risks and benefits, and

to inform health policies that are both

patient- centred and evidence- based.

According to the Institute of Medicine

(IOM), there are six domains of health-

care quality: patient- centredness, effec-

tiveness, efficiency, safety, timeliness and

equity.

Patient- centred care is based on

the provision of services that respect

and respond to individual patients’ pref-

erences and needs, and incorporates

these aspects in clinical decisions and

processes.

2 3

Effective healthcare services

result ultimately in measurable improve-

ments in health outcomes,

while ensuring

the prevention of errors and adverse

effects, ie, ensuring patient safety.

Other

on August 3, 2024 by guest. Protected by copyright.http://qualitysafety.bmj.com/BMJ Qual Saf: first published as 10.1136/bmjqs-2019-010581 on 12 June 2020. Downloaded from

NevesAL, etal. BMJ Qual Saf 2020;0:1–14. doi:10.1136/bmjqs-2019-010581

Systematic review

dimensions of quality care delivery include minimising

waste of resources (ie, efficiency), minimising delays

in the provision of care (ie, timeliness) and avoiding

differences in the provision of services to all groups of

healthcare users (ie, equity).

Despite the claims on the theorised benefits of

providing patients with access to EHRs, there is still

a considerable lack of evidence of their demonstrated

impact. Though evidence suggests that these interven-

tions improve patient satisfaction and communica-

tion

5 6

no clear benefits were found on effectiveness.

Previous studies

5 6

were also unable to find a beneficial

effect on efficiency measures, such as number of face-

to- face visits and telephone appointments.

Five landmark reviews provided a comprehen-

sive characterisation of the literature published until

2013.

5–9

One of them

included studies evaluating the

impact of both paper- based and electronic records, a

heterogeneity that challenges the identification of indi-

vidual benefits of the digital approach. The authors of

previous systematic reviews highlight the paucity of

published papers, and a tendency to include small and

methodologically less robust studies,

with a high risk

of bias.

In fact, only one systematic review specifically

including randomised trials was published in 2012,

having found only two studies investigating the impact

on effectiveness.

Recent discussions around patients’

rights and data ownership have acted as strong drivers

to allocate resources to interventions capitalising on

EHRs with patient access.

Therefore, it is plausible

that the more recent literature has provided new

evidence to shed light on this subject.

This work builds on the previous landmark reviews,

and aims to capture recent, highest quality evidence

(ie, randomised trials) in order to clarify the impact of

providing patients access to EHRs. The main objective

of this systematic review was to assess the impact of

these interventions on the six dimensions of quality

of care.

METHODS

The Preferred Reporting Items for Systematic Reviews

and Meta- Analyses guidelines

were followed in

conducting this systematic review (online supplemen-

tary file 1). The study protocol was registered with

the International Prospective Register of Systematic

Reviews (PROSPERO) (CRD42017070092) and is

available as an open access paper.

Any differences

between the protocol and review are described in

online supplementary file 2.

Search strategy

A systematic search of the literature published between

1997 and 2017 was performed on Current Index to

Nursing and Allied Health Literature (CINAHL),

Cochrane, Embase, Health Management and Policy

Database (HMIC), Medline/PubMed and PsycINFO,

using free terms and controlled vocabulary, whenever

supported.

The reference lists of relevant articles

(including systematic reviews), and grey literature

(including PROSPERO, reports of relevant stakeholder

organisations (NHS Digital, AMIA, eHealth at WHO,

International Society for Telemedicine and eHealth),

and conference proceedings (last 5 years) of related

conferences (American Medical Informatics Associa-

tion, MedInfo, Medicine 2.0, Medicine X)) were also

screened.

Study selection criteria

We included randomised trials only (see online supple-

mentary file 2) that met the following criteria: (1)

Focused on adults subjects (eg, patients, carers). (2)

Included an intervention consisting of sharing EHRs

with patients (either isolated or as part of a multicom-

ponent intervention, that could include the identifi-

cation of discrepancies in records, messaging systems,

access to educational material, or other). (3) Had an

outcome evaluating at least one of the six domains

of quality of care. Studies were excluded if they (1)

Included participants aged 16 years and under. (2) Had

an intervention consisting of health reminders only. (3)

Only reported cognitive outcomes (eg, intent) or other

subjective measures only (eg, subjective perception

of health and/or well- being). The detailed screening

strategy is described in the study protocol.

Data extraction

One investigator extracted information from the

included studies into a standardised computer- based

spreadsheet, which was reviewed by a second investi-

gator for consistency. The data collected for each study

included: name of the first author, year of publication,

number of participants, participants’ characteristics

and setting, date of the intervention, study duration,

study design, intervention characteristics, domain of

healthcare quality assessed, main outcomes (specifying

if primary or secondary), effect size (means (SD) or

% for every group, whenever possible; or difference

between groups, if the only information available),

statistical significance, overall quality score.

Risk of bias assessment

Risk of bias was evaluated using the Cochrane Risk

of Bias Tool.

Two investigators reviewed all eligible

studies in order to appraise their risk of bias (ALN, LF;

ALN, LL). A third investigator resolved disagreements

(LL, LF). A study was considered as ‘overall low risk’

if scoring low risk for at least 50% of the criteria eval-

uated; otherwise, the study was considered having an

‘overall high/unclear risk’.

Data synthesis and meta-analysis

A narrative synthesis of results was performed by

domain of quality of care (IOM framework).

For the

meta- analysis, continuous outcomes representing the

same variable and reported in at least four studies were

on August 3, 2024 by guest. Protected by copyright.http://qualitysafety.bmj.com/BMJ Qual Saf: first published as 10.1136/bmjqs-2019-010581 on 12 June 2020. Downloaded from

NevesAL, etal. BMJ Qual Saf 2020;0:1–14. doi:10.1136/bmjqs-2019-010581

Systematic review

Figure 1 Flow diagram of included studies. CRT, cluster randomised trial; RCT, randomised controlled trial.

pooled using random effects. This was the case for

HbA1c (reported as the percentage of glycated haemo-

globin over the total, %), and for systolic and dias-

tolic blood pressure (SBP and DBP, respectively; both

reported in mm Hg). All effect sizes are shown as abso-

lute difference in means (DM) (weighted mean differ-

ence (WMD)) and classified as negative when in favour

of the intervention, and positive when in favour of the

control. Heterogeneity was assessed using I

(<30%:

low; 30%–60%: moderate; 60%–90%: substantial;

>90%: considerable).

. The presence of publication

bias was evaluated by a funnel plot. Comprehensive

meta- analysis V.2.3. was used for statistical analysis.

Sensitivity analysis and subgroup analysis

For each domain of quality, we described the propor-

tion of studies showing beneficial effects in both ‘low

risk’ and ‘unclear/high risk of bias’ groups. Sensitivity

analyses were conducted, excluding high/unclear risk

of bias studies (for HbA1c and SBP), and short- term

interventions (lasting less than 12 months) for HbA1c.

Further information is provided in online supplemen-

tary file 2.

Patient and public involvement

Our research question emerged from the implemen-

tation evaluation of the Care Information Exchange

(https://www. care info rmat ione xchange- nwl. nhs.

uk/), a portal/EHR with patient access available to

2.4 million people in North- West London. Lay part-

ners will be involved in summarising the research find-

ings into lay summaries and reports.

RESULTS

The database search retrieved 6594 citations (figure 1).

Titles and abstracts were screened, and 1698 duplicates

were excluded, as well as 4801 articles that did not

meet the inclusion criteria. After the full- text screening

of the remaining articles (n=95), 72 additional papers

did not meet inclusion criteria and were therefore

excluded. The kappa statistic measuring intercoder

agreement in title and abstract screening was 0.40 (fair

agreement). Screening of reference lists of systematic

reviews revealed 13 additional studies that met our

predefined criteria. A total of 36 papers was obtained,

which included 20 randomised trials (17 randomised

controlled trials (RCTs) and 3 cluster randomised trial

(CRTs)).

Description of included studies

The 20 included studies involved a total of 17 387

participants (table 1). Publication year ranged from

1999 to 2013 and study duration varied between 3

months and 32 months. Participants included had a

range of health conditions, including type 2 diabetes

(n=7),

14–20

heart failure (n=2),

21 22

arterial hyper-

tension (n=2),

23 24

cancer (n=1),

type 1 diabetes

on August 3, 2024 by guest. Protected by copyright.http://qualitysafety.bmj.com/BMJ Qual Saf: first published as 10.1136/bmjqs-2019-010581 on 12 June 2020. Downloaded from

NevesAL, etal. BMJ Qual Saf 2020;0:1–14. doi:10.1136/bmjqs-2019-010581

Systematic review

Table 1 Characteristics of the included studies

Author, year Study type N* (I;C)

Date of

intervention

Participants

(setting) Duration

Study design and

comparison

Retention rates

†

(%) total (I:C)

Intervention

EHR- sharing

component

Other components of

the intervention

Chrischilles,

2013

RCT 1075

(I:802;

C:273)

2010–2011 General population

(>65 years old)

(general public—

survey to voters)

6 M 2- arm study

Standard care control

group

100.0 (I:100.0;

C:100.0)

► Web- based health record

► Access to current and

past medicines, allergies,

health conditions, and

health event tracking

over time

► Medication safety

messages

► Display of general

medication- use patient

safety indicators

Earnest, 2004

RCT 107

(I:54;

C:53)

2002 Patients with chronic

heart failure

(secondary care)

12 M 2- arm study

Standard care control

group

75.7

(I:70.4; C:81.1)

► Web- based health record

► Access to medical record

with clinical notes,

laboratory reports, and

test results, as well as

information regarding

heart failure

► Secure messaging system

Fonda, 2009

RCT 104

(I:52; C:52)

NA Patients with poorly

controlled T2DM

(primary and

secondary care)

12 M 2- arm study

Standard care control

group

► Web- based health record

► Access to website

which accepts electronic

transmissions from blood

pressure and glucose

monitoring devices and

displays these data in

graphic and tabular form

for the participant and

care manager to review

► Secure messaging system

► Web- enabled diabetes

educational modules

► Links to other web- based

diabetes resources

Grant, 2008

RCT 244

(I:126; C:118)

2005–

2007

General population

(primary care)

12 M 2- arm study

Active care control group

(ie, access to a PHR

to update and submit

family history and health

maintenance information)

64.0

(I:65.0; C:34.7)

► Web- based health record

► Access to medications

lists, glucose, blood

pressure, LDL- cholesterol,

preventive care and

recent results and current

treatment information

► Secure messaging system

(platform to reply to

questions regarding

adherence barriers

and adverse effects of

medication; check boxes

and free text boxes within

the PHR encouraged

patients to enter therapy

concerns and requests

to address speciﬁc care

limitations)

Continued

on August 3, 2024 by guest. Protected by copyright.http://qualitysafety.bmj.com/BMJ Qual Saf: first published as 10.1136/bmjqs-2019-010581 on 12 June 2020. Downloaded from

NevesAL, etal. BMJ Qual Saf 2020;0:1–14. doi:10.1136/bmjqs-2019-010581

Systematic review

Author, year Study type N* (I;C)

Date of

intervention

Participants

(setting) Duration

Study design and

comparison

Retention rates

†

(%) total (I:C)

Intervention

EHR- sharing

component

Other components of

the intervention

Green,

2008

RCT 778

(I1:259;

I2:261; C:258)

NA Hypertensive patients

(primary and

secondary care)

12 M 3- arm study

Standard care control

group

Intervention 1(I1): home

BP monitoring and secure

patient Web services

training only

Intervention 2(I2): home

BP monitoring and Web

training plus pharmacist

care management

delivered through web

communications

In this work, only control

and intervention one

were considered

93.8

(I1:94.9; I2:90.8;

C:95.7)

► Web- based health record

► Ability to view current

health conditions,

laboratory test results,

clinic visit summaries,

and lists of allergies,

immunisations, and

medications

► Secure messaging system

► Ability to reﬁll

medications and make

appointments

Holbrook, 2009

RCT 511

(I:253;

C:258)

2002–

2003

Patients with type 2

diabetes

(primary care)

6 M 2- arm study

Standard care control

group

68.7 (I:68.4;

C:69.0)

► Patient and primary

care provider access to

diabetes tracker of 13

risk factors

► Targets of risk factors

► Personalised

recommendation

messages

► Appointment and

medication reminders

Jones,

1999

RCT 525

(I1:167;

I2:178;

C:180)

1997 Radiotherapy patients

(secondary care)

3 M 3- arm study

Standard care control

group

Intervention 1(I1): Access

to general information on

a computer)

Intervention 2(I2): Access

to personal and general

information in varying

order via a computer

In this work only

comparisons between

control and I2 will be

considered

83.4

(I1:76.6; I2:87.6;

C:85.6)

► Touch screen health

record kiosk

► Summary of medical

record, or choice between

personal or general

information

► Printout of information

viewed sent to patients

► Explanation about terms

used were linked to in the

medical records

► General information

about cancer

Table 1 Continued

Continued

on August 3, 2024 by guest. Protected by copyright.http://qualitysafety.bmj.com/BMJ Qual Saf: first published as 10.1136/bmjqs-2019-010581 on 12 June 2020. Downloaded from

NevesAL, etal. BMJ Qual Saf 2020;0:1–14. doi:10.1136/bmjqs-2019-010581

Systematic review

Author, year Study type N* (I;C)

Date of

intervention

Participants

(setting) Duration

Study design and

comparison

Retention rates

†

(%) total (I:C)

Intervention

EHR- sharing

component

Other components of

the intervention

Khan,

2010

RCT 7368

(I:3856; C:3512)

NA Patients with type 2

diabetes (primary and

secondary care)

32 M 2- arm study

Standard care control

group

100.0

(I:100.0; C:100.0)

► Centralised laboratory

results from independent

laboratories

(haemoglobin A1c,

cholesterol, serum

creatinine, and urine

protein results) accessible

to patients

► Overdue reminders and

alerts to patients with

elevated test results

► Generation of ﬂow sheets

with laboratory results,

reminders of overdue

laboratory tests, and

summary population

reports for providers

Krist,

2012

RCT 4500 (I:2250:

C:2250)

2008–

2009

General population

(primary care)

16 M 2- arm study

Standard care control

group

► Access to relevant details

in the patient’s history

(prior laboratory test

values and dates)

► Preventive services

recommendations based

on EHR data

► Links to relevant

informational material

and decision aids

McCarrier, 2009

RCT 78

(I:42;

C:36)

2005–

2006

Patients with type

1 diabetes (primary

care)

12 M 2- arm study

Standard care control

group

83.3 (I:85.7;

C:80.6)

► Web- based health record

► Access to entire EHR

with clinical encounters,

physician notes, and test

results

► Blood glucose readings

uploaded by patients

► Medication, nutrition,

and exercise data can

be registered by both

patients and case

managers

► Generations of action

plans for self- efﬁcacy and

self- management support

► Educational information

on diabetes

McMahon, 2005

RCT 104

(I:52; C:52)

2004 Patients with type

2 diabetes patients

(both primary and

secondary care)

12 M 2- arm study

Standard care control

group

75.9 (I:75.0;

C:76.9)

► Web- based care-

management site with

value upload for blood

pressure and glucose

monitoring devices

► Graphical and tabular

view of measurements

provided for patients

and HCPs

► Half- day self-

management training on

diabetes

► Computer training and

support available to

intervention participants

► Messaging with care

manager via site

► Educational material

Table 1 Continued

Continued

on August 3, 2024 by guest. Protected by copyright.http://qualitysafety.bmj.com/BMJ Qual Saf: first published as 10.1136/bmjqs-2019-010581 on 12 June 2020. Downloaded from

NevesAL, etal. BMJ Qual Saf 2020;0:1–14. doi:10.1136/bmjqs-2019-010581

Systematic review

Author, year Study type N* (I;C)

Date of

intervention

Participants

(setting) Duration

Study design and

comparison

Retention rates

†

(%) total (I:C)

Intervention

EHR- sharing

component

Other components of

the intervention

Nagykaldi, 2012

CRT 384

(I:NA;

C:NA)

NA General population

(primary care)

12 M 2- arm study

Standard care control

group

68.5

(I:NA;

C:NA)

► Web- based patient portal

with option to manage

health information and

download a personal

health record

► Personalised wellness

plan, prevention and

longitudinal health

information available

Quinn, 2008

RCT 26

(I:13; C:13)

2006 Patients with type

2 diabetes patients

(primary care)

3 M 2- arm study

Standard care control

group

► Blood glucose meter

value sent directly to the

patient’s mobile phone

► Real- time feedback on

blood glucose levels

► Display of medications

► Educational information

Ralston, 2009

RCT 83

(I:42;

C:41)

2002–

2004

Patients with type 2

diabetes

(secondary care)

12 M 2- arm study

Standard care control

group

90.3 (I:92.95;

C:87.8)

► Web- based EHR access

► Feedback on blood

glucose measurements

► Messaging system for

patients and staff

► Educational information

(exercise, diet and

medication)

Ross,

2004

RCT 107

(I:54;

C:53)

2001 Patients with heart

failure (secondary

care)

12 M 2- arm study

Standard care control

group

75.7

(I:81.1;

C:70.3)

► Web- based EHR access

practice

► Messaging system for

patients and staff

► Educational information

Schnipper, 2012

CRT 541

(I:267;

C:274)

2005–

2007

General population

(primary care)

NA 2- arm study

Active care control group

(ie, patients received

a different EHR- linked

intervention)

74.3%

(I:100.0%

C:49.3%)

► Web- based medication

module linked to EHR

► Ability to request

appointments and

referrals

► Communication with their

physician via secure email

► Prescription renewals

and access a health

information library

Shaw,

2008

RCT 193

(I:97;

C:96)

2004–

2006

Maternity centre

(primary care)

NA 2- arm study

Active care control group

(ie, patients received

access to the same

website but with links to

general pregnancy health

information alone)

54.9

(I:64.9; C:44.8)

► Web- based access to

antenatal health record

► Access to general

pregnancy health

information website for

control and intervention

groups

Table 1 Continued

Continued

on August 3, 2024 by guest. Protected by copyright.http://qualitysafety.bmj.com/BMJ Qual Saf: first published as 10.1136/bmjqs-2019-010581 on 12 June 2020. Downloaded from

NevesAL, etal. BMJ Qual Saf 2020;0:1–14. doi:10.1136/bmjqs-2019-010581

Systematic review

Author, year Study type N* (I;C)

Date of

intervention

Participants

(setting) Duration

Study design and

comparison

Retention rates

†

(%) total (I:C)

Intervention

EHR- sharing

component

Other components of

the intervention

Tang,

2013

RCT 415

(I:202; C:213)

2008–

2009

Patients with type

2 diabetes patients

(both primary and

secondary care)

12 M 2- arm study

Standard care control

group

91.3

I:92.0; C:90.6

► Web- based patient portal

access to EHR

► Automatic upload of

blood glucose values

with visual feedback;

Personalised diabetes

summary; nutrition,

exercise, and insulin

records

► Online messaging with

HCPs

► Advice and medication

management from HCPs

► Personalised e-

educational materials

Tuil,

2007

RCT 244

(I:122;

C:122)

2004 Patients undergoing

IVF or ICSI (secondary

care)

NA 2- arm study

Standard care control

group

73.7

(I:83.6; C:63.9)

► Web- based EHR

► Personal and general

information regarding

treatment

► Communication with

other patients and HCPs

Wagner, 2012

CRT 443

(I:194;

C:252)

NA Patients with

hypertension

(both primary and

secondary care)

12 M 2- arm study

Standard care control

group

71.9

(I:61.8;

C:75.8)

► Web- based EHR

► Patients could view

problem and medication

lists, information

on allergies and

immunisation

► Messaging function,

educational materials,

medication interaction

checking, health

measurement tracking,

and health diaries

*Total number of participants randomised for each study.

†Retention rates were calculated as the proportion of patients randomised in each study that completed follow- up.

BP, blood pressure; C, Control group; CRT, cluster randomised trial; EHR, electronic health records; HCP, healthcare professionals; I, Intervention group; ICSI, intracytoplasmic sperm injection; IVF, in vitro fertilisation; LDL, low-

density lipoprotein; M, months; NA, information not available; PHR, personal health record; RCT, randomised controlled trial; T2DM, type 2 diabetes mellitus.

Table 1 Continued

on August 3, 2024 by guest. Protected by copyright.http://qualitysafety.bmj.com/BMJ Qual Saf: first published as 10.1136/bmjqs-2019-010581 on 12 June 2020. Downloaded from

NevesAL, etal. BMJ Qual Saf 2020;0:1–14. doi:10.1136/bmjqs-2019-010581

Systematic review

Figure 2 Risk of bias assessment cells were colour- coded in orange for

high risk of bias, in green for low risk of bias and in grey if risk of bias was

unclear.

(n=1),

fertility issues (n=1)

and pregnancy.

Five studies included service users in general, without

focusing on a specific health condition.

29–33

Summary of risk of bias assessment

Overall, 50% of the studies included (n=10) were

considered good quality, scoring low risk in at least

half of the domains evaluated in the risk of bias assess-

ment (figure 2).

15 18 20 22 23 26–28 30 32

. Four studies stood

out with an overall low risk of bias for most of the

domains evaluated.

15 20 23 27

Due to the nature of the

intervention, most studies scored a high risk of bias

regarding blinding of participants and personnel; one

study showed unclear risk.

Blinding of the outcome

assessment also showed a high risk of bias in several

studies.

14 17 21 22 24–26 28 29 31 32

Only three studies

15 20 23

provided information on trial protocol registration.

Interventions and retention rates

Although all interventions provided participants with

web- based access to EHRs, the content made available

varied greatly (table 1). Content available to partici-

pants included access to previous medical history and

risk factors,

15 20 24 25 28 29 31

test results,

14 16–19 21 26 31–33

medication lists,

23 24 30 31 33

list of allergies,

23 24

current

health conditions,

23 31

and clinical encounters and

physician notes.

26 31

One study specifically mentioned

the existence of a functionality to download EHR

data.

In all studies, the patient access to EHRs was

part of a complex intervention with other compo-

nents. Intervention components included educa-

tional materials,

14 18–20 22 24–26 28 30–32

generation of

personalised action plans/messages,

15 26 31 32

self-

management training,

and medication and appoint-

ment reminders.

15 16

Twelve studies included secure

messaging systems.

14 17 20–24 27 29–31 33

Two studies

provided incentives (either financial,

or use of the

portal after the study),

and one explicitly mentioned

that no incentives were provided.

Retention rates

were calculated as the proportion of randomised

patients in each study that completed follow- up.

Three studies did not provide enough information

to adequately calculate retention rates (total and per

arm).

14 19 32

Among the other studies, only one

had a

retention rate below 60%.

Comparisons

In most studies, the comparator was usual care (ie,

no patient access to EHRs).

14–22 24 26 27 29 31 32

In three

studies, the comparisons were active controls.

28 30 33

Two

studies comprised three arms,

23 25

which are described

in further detail in table 1.

Outcomes

Most papers assessed outcomes covering more than

one domain (median=2). The domain most frequently

assessed was effectiveness (n=14), and the least

frequently evaluated were timeliness and equity (n=0).

Patient- centredness, safety and efficiency were evalu-

ated, respectively, in 11, 4 and 5 studies. A detailed

overview of the outcomes evaluated is provided in

online supplementary file 3.

Patient-centredness

Eleven studies evaluated the impact of sharing EHRs

with patients on patient- centredness, including

C RTs

24 30 31

and eight RCTs.

19–22 25–28

While six studies

found a beneficial impact in at least one patient-

centredness outcome,

20 24–26 30 31

it is important to

note that the exact measure of patient- centredness

varied considerably across studies. Although patient

satisfaction improved in two studies

20 25

(46% vs 40%,

p=0.04% and 27.7% vs 24.5%, p<0.0001, respec-

tively), two other failed to show a significant effect.

22 27

One study

showed an increase in patient activation,

as measured by the Patient Activation Measure

(47 vs

45, p=0.0014), but these results were not replicated

in a similar study.

Self- efficacy scores improved in

one study

using the Diabetes Empowerment Scale

(+0.14 vs −0.16, p=0.04), but no differences were

found in two other studies

22 27

using the Kansas City

Cardiomyopathy Questionnaire (KCCQ) and the

General Self- Efficacy Scale.

Patient empowerment

was accessed by the Patient Empowerment Scale

on August 3, 2024 by guest. Protected by copyright.http://qualitysafety.bmj.com/BMJ Qual Saf: first published as 10.1136/bmjqs-2019-010581 on 12 June 2020. Downloaded from

NevesAL, etal. BMJ Qual Saf 2020;0:1–14. doi:10.1136/bmjqs-2019-010581

Systematic review

Figure 3 Forest plots of effect sizes and 95% CIs representing the effect

of interventions providing patients access to EHRs in HbA1c, SBP and DBP,

using a random- effects model. The area of each square is proportional to

the study's size, and therefore to its weight in the meta- analysis. For each

study, CIs are represented by horizontal lines; a vertical line representing

no effect is also plotted. The meta- analysed measure of effect is plotted as

a diamond, the lateral points of which indicate CIs for this estimate. DBP,

diastolic blood pressure; EHRs, electronic health records; HbA1c, glycated

haemoglobin, SBP, systolic blood pressure.

two studies

21 24

but a significant improvement in mean

scores was found only in one (41.2 vs 40.1, p=0.019).

Three studies evaluated health literacy (ie, patients

acknowledging to have learnt something new),

19 25 28

but only one found the intervention to be beneficial

(96% vs 74%, p=0.02). Six out of 11 studies (54.5%)

scored an overall low risk of bias. The proportion of

studies showing a significant positive effect for at least

one of the outcomes evaluated was 50% in low risk of

bias studies, and 80% in the remaining studies.

Effectiveness

A total of 14 studies appraised the impact of providing

patients with access to EHRs on effectiveness, including

2 CRTs

24 31

and 12 RCTs.

14 15 17–20 22 23 25 26 32 33

Ten out

of 14 studies (71.4%) demonstrated a positive impact

on effectiveness- related outcomes.

15 17–20 22 23 25 31 32

These studies evaluated the impact on a wide range of

health conditions, including depression and anxiety,

heart failure,

cardiovascular risk (Framingham

Score),

obesity,

15 23

smoking status,

adherence

to preventive services

31 32

dyslipidaemia,

17 18 20 24 33

diabetes

14 15 17–20 26 33

and hypertension.

15 17 18 20 23 24 33

In one study using the Hospital Anxiety and Depres-

sion Scale,

patient access to EHRs did not change

patients’ depression scores, and patients in the general

computer information group were more anxious than

the ones accessing personal records (DM=+18%,

95% CI 3.7 to 26.5, p=0.001).

One study found a

dramatic improvement in symptom stability scores,

assessed by the KCCQ (DM:+17, 95% CI 9 to 29,

p<0.001).

Two studies found an improvement in

LDL- cholesterol levels.

17 20

No significant changes

were observed on triglycerides,

high- density lipo-

protein (HDL)- cholesterol,

total cholesterol,

body

weight,

15 23

smoking status

or total cardiovascular

risk.

Adherence to preventive services improved

in the two studies evaluating this aspect

31 32

(ie, use

of low- dose aspirin (84.4% vs 67.6%, p<0.0001),

complete immunisation (95.5% vs 87.2%, p=0.044),

and uptake of cancer screening (increases ranging from

10.3% to +14.3%, all p<0.05)).While two studies

specifically evaluated adherence to pneumococcal

immunisation,

31 32

only one found a beneficial effect.

Seven out of 14 studies scored an overall low risk

of bias (50.0%). The proportion of studies showing a

positive effect was 85.7% in the low risk of bias group,

and 57.1% in the remaining studies.

Meta-analysis

Data from RCTs evaluating HbA1c and SBP were

pooled together, and the respective meta- analyses

performed. The six studies evaluating HbA1c

17–20 26 33

comprised 950 participants, from which 894 completed

follow- ups. Meta- analyses showed a beneficial effect

in effectiveness by reducing HbA1c (unit, %; WMD=

−0.316; 95% CI −0.540 to −0.093, p=0.005, I

=0%)

(figure 3), which remained significant in sensitivity

analyses for low risk of bias studies (WMD= −0.405;

95% CI −0.711 to −0.099) (online supplementary

figure 1), and long- term interventions only (WMD=

−0.272; 95% CI −0.482 to −0.062) (online supple-

mentary figure 2). It is important to note that the study

showing a high risk of bias,

was also the one showing

the smallest study sample. The funnel plot indicates

asymmetry (online supplementary figure 3), suggesting

potential publication bias.

The four studies evaluating the impact on blood

pressure

17 18 20 23

(comprising 1308 participants, of

which 1021 completed follow- ups) were pooled in

a meta- analysis, and showed a significant beneficial

effect in SBP (unit: mm Hg; WMD=−1.416; 95%

CI −2.814 to −0.018, p=0.047, I

=0%) (figure 3).

However, significance was lost after removing the

high/unclear risk of bias study

(WMD=−1.375;

95% CI −2.791 to 0.041) (online supplementary

figure 1). No significant effect was found in DBP in the

meta- analysis (unit: mm Hg; WMD=−0.918; 95% CI

−2.078 to 0.242, p=0.121, I

=0%) (figure 3), nor in

the sensitivity analysis for low risk of bias studies only

(WMD=−0.916; 95% CI −2.089 to 0.257) (online

on August 3, 2024 by guest. Protected by copyright.http://qualitysafety.bmj.com/BMJ Qual Saf: first published as 10.1136/bmjqs-2019-010581 on 12 June 2020. Downloaded from

NevesAL, etal. BMJ Qual Saf 2020;0:1–14. doi:10.1136/bmjqs-2019-010581

Systematic review

supplementary figure 1). The funnel plots appear

symmetrical for SBP and DBP (online supplementary

figures 4 and 5), indicating a similar proportion of

studies in each direction of the effect size.

Safety

All studies

19 22 29 30

showed a beneficial effect for at least

one of the outcomes evaluated (online supplementary

file 3). Two studies evaluated adherence, including

general adherence to medical regimens

(using the

General Adherence Scale from the Medical Outcomes

Study (MOS)

and medication adherence.

22 29

General adherence (MOS Scores) improved with the

intervention

(+2.3, 95% CI −3.7 to 8.3, p=0.01),

but no significant changes were found in adherence to

medication.

22 29

A beneficial effect was observed in all

studies evaluating medication safety,

19 29 30

including a

higher likelihood of reporting discrepancies (53% vs

24%, p<0.01),

to change medications

19 29

(88.3% vs

67.2%, p<0.01; and 84% vs 23%, p=0.002, respec-

tively), and a resulting slightly lower proportion of

patients with medication discrepancies (29% vs 30%,

p=0.01).

Two out of four studies scored an overall

low risk of bias, and the proportion of studies showing

a positive effect was the same in both risk groups

(100.0%).

Efciency

The impact of providing patients with access to EHRs

was assessed in five studies.

16 18 22 24 31

As less than four

studies assessed the same construct, meta- analysis was

not performed, and a descriptive analysis is provided.

Number of hospitalisations per subject was lower in

one study (0.17 vs 0.20, p=0.01),

while total number

remained unchanged in another (22 vs 21, p=1.00).

Length of stay (in days) did not change in two studies

(+0.2 vs –0.3, and 0.42 vs 0.34, respectively),

18 24

but

was shorter in another (0.99 vs 1.1, p<0.01).

the three studies evaluating the number of emergency

visits, total numbers were either reduced,

increased

or remained unchanged.

Number of primary care

visits was lower in one study (2.9 vs 4.3, p<0.0001),

but no changes were observed in another (0.0 vs

–0.2).

Two out of five studies scored an overall low

risk of bias, and the proportion of studies showing a

positive effect was 50.0% and 66.7% in low- risk and

high/unclear- risk groups, respectively.

Timeliness and equity

While none of the studies assessed either timeliness or

equity as primary outcome, three studies

21 24 32

eval-

uated the predictors of usage of EHRs by patients.

Earnest et al

did not find any associations between

usage and race, symptom scores or number of visits;

two studies found significant associations between

usage and higher education,

number of illnesses,

younger age,

clinic attended by the patient

self- reported computer skills,

and higher number of

internet- use items.

DISCUSSION

Key findings in context of published literature

This work systematically appraised the impact of EHRs

with patient access across the six domains of quality

of care as defined by the IOM:

patient- centredness,

effectiveness, efficiency, safety, timeliness and equity.

Regarding patient- centredness, results were incon-

sistent. More than half of the studies included in this

domain showed a significant positive effect for at least

one outcome, but no clear effect was found for specific

outcomes, such as patient satisfaction, patient activa-

tion, self- efficacy, patient empowerment or health

literacy. These results are line with previous studies

5 6 8

that found mixed evidence about the impact in patient-

centred outcomes. While providing patients access to

EHRs is envisaged as a key strategy to deliver patient-

centred care, the diversity of outcomes evaluated, and

scales and tools used, hinders pooling of results and

the use of meta- analytical approaches. It is critical,

therefore, to identify and standardise measures and

constructs to evaluate patient- centredness, to allow the

application of meta- analytical methods in this domain.

A few studies included showed a positive impact

in effectiveness in a range of outcomes (ie, anxiety,

cardiac symptoms, LDL- cholesterol), but no signif-

icant improvements were found for triglycerides,

HDL- cholesterol, total cholesterol, body weight,

smoking status or total cardiovascular risk. Two addi-

tional studies not captured by our search also suggest

that providing patients access to EHRs may improve

glaucoma control

and quality of life in patients with

asthma.

A positive effect was also found in adher-

ence to several preventive services (ie, use of low-

dose aspirin, cancer screening), an approach that

can be particularly relevant in the context of cancer

screening, where higher expected adherence rates

have the potential to reduce cancer incidence and

mortality.

However, the number of studies published

per outcome is limited, and further research is needed

to increase meta- analytical power and explore the size

and impact of the potential effect in specific health

conditions.

Our meta- analysis showed a beneficial effect on

HbA1c reduction, which remained significant after

removing low/unclear- risk studies, or studies in which

the intervention lasted less than 12 months. In 2013,

Goldzweig et al identified several examples of improved

outcomes for patients with chronic diseases (including

hypertension and diabetes).

In 2012, Ammenw-

erth et al

performed a systematic review of studies

published between 1990 and 2011 and concluded that

there was insufficient evidence to document a bene-

ficial effect in effectiveness in patients with access

to EHRs. However, by then only two studies (out

of the four included in the review) investigated the

on August 3, 2024 by guest. Protected by copyright.http://qualitysafety.bmj.com/BMJ Qual Saf: first published as 10.1136/bmjqs-2019-010581 on 12 June 2020. Downloaded from

NevesAL, etal. BMJ Qual Saf 2020;0:1–14. doi:10.1136/bmjqs-2019-010581

Systematic review

effect on health outcomes. Our meta- analysis demon-

strates a mean reduction in absolute values of HbA1c

of 0.316% (95% CI −0.540% to −0.093%), with a

low heterogeneity (I

=0.0) reflecting the specificity

of our inclusion criteria. These results have important

clinical implications, since an absolute reduction of

1 point on HbA1c levels (expressed in the same unit

considered in our meta- analysis) is associated with

a significant reduction of deaths related to diabetes

(−21%), myocardial infarction (−14%) and microvas-

cular complications (−37%).

Visual inspection of the

funnel plot suggests a potential publication bias, with

studies with a lower precision (higher SE) reporting a

greater beneficial effect. However, the meta- analysed

effect remained significant after removing the study

that stood out with a smallest sample size.

Although our meta- analysis found a beneficial effect

in SBP, statistical significance was lost in sensitivity

analysis for low risk of bias studies only; no significant

effect was found in DBP. It must be noted, however,

that the number of studies included is low, and further

evidence is needed to establish robust conclusions.

For the efficiency domain, most studies included

found either no change, or a reduction of healthcare

usage (in primary care visits,

or inpatient or emer-

gency contacts).

Ammenwerth et al,

have also previ-

ously suggested a significant reduction in office visit

rates. Further studies are required to clarify the impact

on this dimension and pave the way to meta- analytical

approaches that can provide further insights on the

effect size in the various dimensions of healthcare

usage.

Our work suggests that the intervention improves

general adherence, but not medication adherence—

however, a strong body of evidence showed a posi-

tive effect in medication safety. A previous study has

suggested that patients find this approach valuable,

and reported either unchanged or improved rela-

tionships with their clinician when using it.

Further

studies should further explore patients’ willingness

and ability to report errors in their records, and also

which specific groups are most likely to benefit. These

results are in line with the findings of Mould et al,

de Lusignan et al and Ammenwerth et al, who previ-

ously suggested that these digital solutions positively

impacted patient safety.

6 7 9

Finally, we found no studies specifically focusing on

the impact on timeliness or equity. Uptake of portals

may differ by patient- specific factors, with lower use by

racial and ethnic minorities, patients with lower educa-

tion level or literacy, thus leading to digital- led health

inequities.

Davis Giardina et al

reported that, up to

2012, no studies had assessed any of these domains.

Eight years later, these aspects remain unexplored.

Strengths and limitations

Five landmark reviews have been published to date

evaluating the impact of EHRs with patient access

on different aspects of quality of care.

5–9

Only one

systematic review had focused on randomised trials,

having found two studies investigating the impact on

effectiveness.

Our systematic review included studies published

between 1997 and 2017 and retrieved a total of 20

randomised trials. This study has several strengths: a

predefined, openly available protocol was followed

(with any changes described in online supplemen-

tary file 2); only randomised trials were included;

focused exclusively on EHRs; and impact was assessed

in all domains of quality of care, with meta- analysis

performed whenever possible.

Only half of the studies included had an overall low

risk of bias score. A possible approach to improve

blinding in web- based interventions, or to test the

impact of specific components, could be using A/B

testing, a technique used for website optimisation that

compares variation against a standard experience, and

determines which variant is more effective.

CONCLUSION

Our results suggest that providing patients with access

to EHRs can improve patient safety and effectiveness.

More methodologically robust studies are necessary

to increase the strength of these conclusions, and to

enhance meta- analytical power. For EHRs with patient

access to be broadly used, it is important to focus on

interventions that enhance adoption and measure

usage, and issues of equity in both aspects need to be

addressed by policy makers when implementing such

programmes.

Author affiliations

Patient Safety Translational Research Centre, Institute of Global Health

Innovation, Imperial College London, London, UK

Center for Health Technology and Services Research / Department of

Community Medicine, Health Information and Decision (CINTESIS/MEDCIDS),

Faculty of Medicine, University of Porto, Porto, Portugal

Westmead Applied Research Centre, Faculty of Medicine and Health, University

of Sydney, Sydney, New South Wales, Australia

Centre for Health Informatics, Australian Institute of Health Innovation, Sydney,

New South Wales, Australia

Department of Health Policy, London School of Economics & Political Science,

London, UK

Twitter Ana Luisa Neves @ana_luisa_neves

Contributors Conception and design of the work: ALN, AWC.

Database searching: ALN, LF. Full text screening: ALN, LF.

Outcome data extraction: ALN, LF. Risk of bias: ALN, LL,

LF. Data analysis and interpretation: ALN, LL, LF, AWC, EM.

Critical revision of drafts for important intellectual content.

ALN, AWC, LF, LL, EM, AD. Final approval of the version to

be published: ALN, AWC, LF, LL, EM, AD.

Funding This work is supported by the National Institute for

Health Research (NIHR) Imperial Patient Safety Translation

Research Centre. Infrastructure support was provided by

the NIHR Imperial Biomedical Research Centre. The study

funder(s) did not play a role in study design; in the collection,

analysis, and interpretation of data; in the writing of the

report; and in the decision to submit the article for publication.

In addition, researchers were independent from funders, and

all authors had full access to all of the data included in this

on August 3, 2024 by guest. Protected by copyright.http://qualitysafety.bmj.com/BMJ Qual Saf: first published as 10.1136/bmjqs-2019-010581 on 12 June 2020. Downloaded from

NevesAL, etal. BMJ Qual Saf 2020;0:1–14. doi:10.1136/bmjqs-2019-010581

Systematic review

study and can take responsibility for the integrity of the data

and the accuracy of the data analysis.

Competing interests None declared.

Patient consent for publication Not required.

Provenance and peer review Not commissioned; externally

peer reviewed.

Data availability statement All data relevant to the study

are included in the article or uploaded as supplementary

information. Data supporting the findings of this study are

available within the article and its supplementary materials.

Open access This is an open access article distributed in

accordance with the Creative Commons Attribution 4.0

Unported (CC BY 4.0) license, which permits others to copy,

redistribute, remix, transform and build upon this work for any

purpose, provided the original work is properly cited, a link

to the licence is given, and indication of whether changes were

made. See: https:// creativecommons. org/ licenses/ by/ 4. 0/.

ORCID iDs

Ana Luisa Neves http:// orcid. org/ 0000- 0002- 7107- 7211

Lisa Freise http:// orcid. org/ 0000- 0002- 3558- 0063

Erik Mayer http:// orcid. org/ 0000- 0002- 5509- 4580

REFERENCES

1 Dickey LL. Promoting preventive care with patient- held

minirecords: a review. Patient Educ Couns 1993;20:37–47.

2 Institute of Medicine (US) Committee on Quality of Health

Care in America. Crossing the quality chasm: a new health

system for the 21st century. Washington DC: National

Academy Press (US), 2001.

3 Scholl I, Zill JM, Härter M, et al. An integrative model of

patient- centeredness - a systematic review and concept analysis.

PLoS One 2014;9:e107828.

4 Arah OA, Klazinga NS, Delnoij DMJ, et al. Conceptual

frameworks for health systems performance: a quest for

effectiveness, quality, and improvement. Int J Qual Health

Care 2003;15:377–98.

5 Davis Giardina T, Menon S, Parrish DE, et al. Patient access to

medical records and healthcare outcomes: a systematic review.

J Am Med Inform Assoc 2014;21:737–41.

6 de Lusignan S, Mold F, Sheikh A, et al. Patients' online

access to their electronic health records and linked online

services: a systematic interpretative review. BMJ Open

2014;4:e006021.

7 Ammenwerth E, Schnell- Inderst P, Hoerbst A. The impact of

electronic patient portals on patient care: a systematic review

of controlled trials. J Med Internet Res 2012;14:e162.

8 Goldzweig CL, Orshansky G, Paige NM, et al. Electronic

patient portals: evidence on health outcomes, satisfaction,

efficiency, and attitudes: a systematic review. Ann Intern Med

2013;159:677–87.

9 Mold F, de Lusignan S, Sheikh A, et al. Patients' online

access to their electronic health records and linked online

services: a systematic review in primary care. Br J Gen Pract

2015;65:e141–51.

10 Blumenthal D, Squires D. Giving patients control of their EHR

data. J Gen Intern Med 2015;30:42–3.

11 Moher D, Shamseer L, Clarke M, et al. Preferred reporting

items for systematic review and meta- analysis protocols

(PRISMA- P) 2015 statement. Syst Rev 2015;4:1.

12 Neves AL, Carter AW, Freise L, et al. Impact of sharing

electronic health records with patients on the quality and

safety of care: a systematic review and narrative synthesis

protocol. BMJ Open 2018;8:e020387.

13 Higgins JPT, Thomas J, Chandler J, et al. Cochrane Handbok

for systematic reviews of interventions. 2nd edn. Chichester,

UK, 2019.

14 Fonda SJ, McMahon GT, Gomes HE, et al. Changes in

diabetes distress related to participation in an Internet- based

diabetes care management program and glycemic control. J

Diabetes Sci Technol 2009;3:117–24.

15 Holbrook A, Thabane L, Keshavjee K, et al. Individualized

electronic decision support and reminders to improve diabetes

care in the community: compete II randomized trial. CMAJ

2009;181:37–44.

16 Khan S, Maclean CD, Littenberg B. The effect of the Vermont

diabetes information system on inpatient and emergency room

use: results from a randomized trial. Health Outcomes Res Med

2010;1:e61–6.

17 McMahon GT, Gomes HE, Hickson Hohne S, et al. Web-

Based care management in patients with poorly controlled

diabetes. Diabetes Care 2005;28:1624–9.

18 Ralston JD, Hirsch IB, Hoath J, et al. Web- Based collaborative

care for type 2 diabetes: a pilot randomized trial. Diabetes

Care 2009;32:234–9.

19 Quinn CC, Clough SS, Minor JM, et al. WellDoc mobile

diabetes management randomized controlled trial: change in

clinical and behavioral outcomes and patient and physician

satisfaction. Diabetes Technol Ther 2008;10:160–8.

20 Tang PC, Overhage JM, Chan AS, et al. Online disease

management of diabetes: engaging and motivating patients

online with enhanced resources- diabetes (EMPOWER- D),

a randomized controlled trial. J Am Med Inform Assoc

2013;20:526–34.

21 Earnest MA, Ross SE, Wittevrongel L, et al. Use of a patient-

accessible electronic medical record in a practice for congestive

heart failure: patient and physician experiences. J Am Med

Inform Assoc 2004;11:410–7.

22 Ross SE, Moore LA, Earnest MA, et al. Providing a web-

based online medical record with electronic communication

capabilities to patients with congestive heart failure:

randomized trial. J Med Internet Res 2004;6:e12.

23 Green BB, Cook AJ, Ralston JD, et al. Effectiveness of

home blood pressure monitoring, web communication, and

pharmacist care on hypertension control: a randomized

controlled trial. JAMA 2008;299:2857–67.

24 Wagner PJ, Dias J, Howard S, et al. Personal health records

and hypertension control: a randomized trial. J Am Med

Inform Assoc 2012;19:626–34.

25 Jones R, Pearson J, McGregor S, et al. Randomised trial of

personalised computer based information for cancer patients.

BMJ 1999;319:1241–7.

26 McCarrier KP, Ralston JD, Hirsch IB, et al. Web- Based

collaborative care for type 1 diabetes: a pilot randomized trial.

Diabetes Technol Ther 2009;11:211–7.

27 Tuil WS, Verhaak CM, Braat DDM, et al. Empowering patients

undergoing in vitro fertilization by providing Internet access to

medical data. Fertil Steril 2007;88:361–8.

28 Shaw E, Howard M, Chan D, et al. Access to web- based

personalized antenatal health records for pregnant women:

a randomized controlled trial. J Obstet Gynaecol Can

2008;30:38–43.

29 Chrischilles EA, Hourcade JP, Doucette W, et al. Personal

health records: a randomized trial of effects on elder

medication safety. J Am Med Inform Assoc 2014;21:679–86.

30 Schnipper JL, Gandhi TK, Wald JS, et al. Effects of an

online personal health record on medication accuracy and

on August 3, 2024 by guest. Protected by copyright.http://qualitysafety.bmj.com/BMJ Qual Saf: first published as 10.1136/bmjqs-2019-010581 on 12 June 2020. Downloaded from

NevesAL, etal. BMJ Qual Saf 2020;0:1–14. doi:10.1136/bmjqs-2019-010581

Systematic review

safety: a cluster- randomized trial. J Am Med Inform Assoc

2012;19:728–34.

31 Nagykaldi Z, Aspy CB, Chou A, et al. Impact of a wellness

portal on the delivery of patient- centered preventive care. J Am

Board Fam Med 2012;25:158–67.

32 Krist AH, Woolf SH, Rothemich SF, et al. Interactive

preventive health record to enhance delivery of

recommended care: a randomized trial. Ann Fam Med

2012;10:312–9.

33 Grant RW, Wald JS, Schnipper JL, et al. Practice- linked

online personal health records for type 2 diabetes

mellitus: a randomized controlled trial. Arch Intern Med

2008;168:1776–82.

34 Hibbard JH, Mahoney ER, Stockard J, et al. Development

and testing of a short form of the patient activation measure.

Health Serv Res 2005;40:1918–30.

35 Anderson RM, Funnell MM, Fitzgerald JT, et al. The diabetes

Empowerment scale: a measure of psychosocial self- efficacy.

Diabetes Care 2000;23:739–43.

36 Schwarzer R, Jerusalem M. Generalized self- efficacy scale. In:

Weinman J, Wright S, Johnston M, eds. Measures in health

psychology: a user’s portfolio. Causal and control beliefs.

Windsor: NEFR- Nelson (United Kingdom), 1995.

37 Bulsara C, Styles I, Ward AM, et al. The psychometrics of

developing the patient empowerment scale. J Psychosoc Oncol

2006;24:1–16.

38 Zigmond AS, Snaith RP. The hospital anxiety and depression

scale. Acta Psychiatr Scand 1983;67:361–70.

39 DiMatteo MR, Sherbourne CD, Hays RD, et al. Physicians'

characteristics influence patients' adherence to medical

treatment: results from the medical outcomes study. Health

Psychol 1993;12:93–102.

40 Kashiwagi K, Tsukahara S. Impact of patient access to

Internet health records on glaucoma medication: randomized

controlled trial. J Med Internet Res 2014;16:e15.

41 Ahmed S, Ernst P, Bartlett SJ, et al. The effectiveness of

web- based asthma self- management system, my asthma portal

(MAP): a pilot randomized controlled trial. J Med Internet Res

2016;18:e313.

42 D'Andrea E, Ahnen DJ, Sussman DA, et al. Quantifying the

impact of adherence to screening strategies on colorectal

cancer incidence and mortality. Cancer Med 2020;9:824–36.

43 Stratton IM, Adler AI, Neil HA, et al. Association of glycaemia

with macrovascular and microvascular complications of type

2 diabetes (UKPDS 35): prospective observational study. BMJ

2000;321:405–12.

44 Bell SK, Gerard M, Fossa A, et al. A patient feedback reporting

tool for OpenNotes: implications for patient- clinician safety

and quality partnerships. BMJ Qual Saf 2017;26:312–22.

45 Kohavi R, Longbotham R. Online Controlled Experiments

and A/B Tests. In: Sammut C, Webb G, eds. Encyclopedia of

machine learning and data mining. Boston: Springer (US),

2017.

46 Yamin CK, Emani S, Williams DH, et al. The digital divide in

adoption and use of a personal health record. Arch Intern Med

2011;171:568–74.

on August 3, 2024 by guest. Protected by copyright.http://qualitysafety.bmj.com/BMJ Qual Saf: first published as 10.1136/bmjqs-2019-010581 on 12 June 2020. Downloaded from