Development of an audit instrument for nursing care plans in the

Papers

Development of an audit instrument for nursing

care plans in the patient record

C Björvell, I Thorell-Ekstrand, R Wredling

Abstract

Objectives—To develop, validate, and test

the reliability of an audit instrument that

measures the extent to which patient

records describe important aspects of

nursing care.

Material—Twenty records from each of

three hospital wards were collected and

audited. The auditors were registered

nurses with a knowledge of nursing docu-

mentation in accordance with the VIPS

model—a model designed to structure

nursing documentation. (VIPS is an acro-

nym formed from the Swedish words for

wellbeing, integrity, prevention, and secu-

rity.)

Methods—An audit instrument was devel-

oped by determining speciﬁc criteria to be

met. The audit questions were aimed at

revealing the content of the patient for

nursing assessment, nursing diagnosis,

planned interventions, and outcome. Each

of the 60 records was reviewed by the three

auditors independently and the reliability

of the instrument was tested by calculat-

ing the inter-rater reliability coeYcient.

Content validity was tested by using an

expert panel and calculating the content

validity ratio. The criterion related valid-

ity was estimated by the correlation

between the score of the Cat-ch-Ing

instrument and the score of an earlier

developed and used audit instrument. The

results were then tested by using Pearson’s

correlation coeYcient.

Results—The new audit instrument,

named Cat-ch-Ing, consists of 17 ques-

tions designed to judge the nursing docu-

mentation. Both quantity and quality

variables are judged on a rating scale from

zero to three, with a maximum score of 80.

The inter-rater reliability coeYcients

were 0.98, 0.98, and 0.92, respectively for

each group of 20 records, the content

validity ratio ranged between 0.20 and 1.0

and the criterion related validity showed a

signiﬁcant correlation of r = 0.68 (p<

0.0001, 95% CI 0.57 to 0.76) between the

two audit instruments.

Conclusion—The Cat-ch-Ing instrument

has proved to be a valid and reliable audit

instrument for nursing records when the

VIPS model is used as the basis of the

documentation.

(Quality in Health Care 2000;9:6–13)

Keywords: audit instrument; nursing care plans; quality

assurance

The patient record is a principal source of

information in which the nursing documenta-

tion of patient care is an essential part.

Traditionally, nurses have written down their

performance of the medical interventions or

observations ordered by the physician. The

rationale behind this was to show that the

instructions had been adhered to and to inform

other nurses or physicians.

Recently, the nursing profession has moved

towards a more independent practice with a

clear recognition of nursing care. With increas-

ing recognition of these nursing components

the documentation of nursing care must

include not only timely and accurate recording

of the performed interventions—medical and

nursing—but also the decision process, ex-

plaining and evaluating why a speciﬁc nursing

action was chosen.

In 1967, Yura and Walsh ﬁrst described the

nursing process model (ﬁg 1) as a structured,

problem solving approach to nursing practice

and its evaluation.

This process originally

contained four steps: assessment, planning,

implementation, and evaluation. In a later ver-

sion, the nursing diagnosis was included. The

nursing process model is a central and widely

accepted concept,

both for nursing practice

and documentation. The nursing process

model is based on the scientiﬁc approach of

investigation and goal oriented action. The pri-

mary purpose of the model is to relate

individualised nursing care to the individual

patient rather than generalised care based on

routines.

The nursing care plan is an essential tool in

the delivery of modern nursing care. The care

plan is a document containing the three

nuclear parts of the nursing process model: a

nursing diagnosis, describing the problem or

need; the aim of the nursing care; and the

interventions that have been planned to achieve

the aim. The purpose of the nursing care plan

is to have a reference easily accessible in the

clinical setting that describes the patient’s

needs and wishes and the nursing interventions

Quality in Health Care 2000;9:6–136

Division of Nur sing

Research at Karolinska

Hospital, Department

of Nursing, Karolinska

Institutet, Stockholm,

Sweden

C Björvell, PhD student,

registered nurse

I Thorell-Ekstrand, senior

lecturer

R Wredling, associate

professor

Correspondence to:

C Björvell

Email:

catrin.bjorvell@medks.ki.se

Accepted 13 December 1999

that have been planned for the patient. It is

used to ascertain the continuity of care among

caregivers. The nursing care plan is part of the

permanent patient record.

Since 1980, major health related organisa-

tions and some western countries have begun

to develop standards, laws, and regulations

stating that the nursing process should be

included with nursing documentation. The

World Health Organisation,

the International

Council of Nursing,

the American Joint Com-

mission on Accreditation of Hospital Nursing

Service Standards,

and the United Kingdom

Central Council

all promoted the use of the

nursing process in nursing care. The Swedish

law on this subject was passed in 1986,

and

was further clariﬁed speciﬁcally for nursing by

the National Board of Health and Welfare in

1993 (box 1).

The development of written care plans has

been slow, however, and in Sweden nurses have

only recently started to produce a more struc-

tured documentation of nursing. Nurses con-

tinue to document care retrospectively rather

than document prospective care. Ehnfors

showed in 1993 that 90% of the audited patient

records lacked identiﬁed nursing problems,

goals, and nursing discharge notes.

In two

thirds of the records, planned interventions

were not stated. In 1996, Nordström and Gar-

dulf stated that the nursing assessment was

insuYciently described in 60% of records

;

only 10% contained identiﬁed nursing prob-

lems and goals; and less than 45% of the

records contained planned nursing interven-

tions. Even in 1999 Ehnfors and Ehrenberg

showed that only one of 120 patient records

contained a comprehensive description of a

patient problem, as prescribed by Swedish

law.

Beneﬁts of documentation

The main beneﬁt of the documentation is

improvement of the structured communication

between healthcare professionals to ensure the

continuity of individually planned patient care.

Without an individualised care plan, nursing

care tends to become fragmentary and based

predominantly on institutional routine and

schedules. The care plan deﬁnes the focus of

nursing care not only to the nursing staV but

also to the patient and his relatives.

By docu-

menting the agreement between patient and

nurse, an opportunity is provided for the

patient to participate in the decision making

about his own care.

13 14

Moreover, the docu-

mentation of expert nursing provides an

important source of knowledge to the novice

nurse and a potential instigation of the further

development of nursing theory.

The care plan

yields criteria for reviewing and evaluating

care, ﬁnancial reimbursement,

and staYng.

Furthermore, a correlation between care plans

and positive patient outcomes, such as a

reduced stay in hospital, has been described.

Documentation model

In 1992, a new documentation model was

developed and tested by Ehnfors, Thorell-

Ekstrand, and Ehrenberg.

16 17

The model is

called VIPS, an acronym formed from the

Swedish words for wellbeing, integrity, preven-

tion, and security, which are seen as the major

goals of nursing care (ﬁg 2). This model is

Assessment

Diagnosis and goal

Planned interventions

Implementation

Evaluation

Figure 1 The nursing

process model.

Nursing

history

• Reason for contact

• Health history

•

Care in progress

• Hypersensitivity

• Social history

•

Service

• Lifestyle

General

information

• Information

source

• Significant other

• Temporary

information

• Confidentiality

• Primary nurse

• Incidental/

progress notes

Nursing

status

Nursing

diagnoses

Nursing

goals

• Communication

• Knowledge/developement

• Breathing/circulation

• Nutrition

• Elimination

• Skin/tissues

•

Wound

• Activity

• Sleep

• Pain/perceptions

• Sexuality/reproduction

• Psychosocial

•

Emotions

•

Relations

• Spiritual/cultural

• Wellbeing

• Composite assessment

•

Medications

Nursing

interventions

Nursing

outcome

Planned - implemented

• Participation

• Information/education

• Support

• Environment

• General care

•

Advanced care

• Training

• Observation/

monitoring

• Special care

•

Wound care

• Drug handling

• Coordination

•

Coordinated care

planning

•

Discharge planning

Medical information

• Medical assessment

Discharge

notes

Figure 2 Flow diagram of the VIPS model for nursing documentation. Reproduced from Ehrenberg et at

with permission.

Regulations about nursing

documentation

“The patient record shall include a distinct

and clear nursing documentation. The

nursing documentation shall, from the

patient’s individual needs, describe the

planning, implementation and eVects of the

nursing care. The documentation shall be

designed in such a way that it contributes to

secure patient safety and provides a basis for

continuous evaluation and revision of nurs-

ing interventions. The nursing care shall be

summarised in a patient discharge note at

the time of discharge.”

(Authors’ transla-

tion.)

Box 1 Regulations about nursing documentation as

stipulated by the Swedish National Board of Health and

Welfare

Development of an audit instrument for nursing care 7

designed to be used in the documentation of

the nursing process and therefore includes a

nursing care plan. The model also includes a

nursing discharge note. The purpose of the

model is to guide the nurse in the sequences of

assessment, problem identiﬁcation, aim, plan-

ning of interventions, implementation, and

evaluation of results and thereby to make nurs-

ing documentation structured, adequate, and

easy to use in clinical care.

In the VIPS model, 13 keywords are used for

classifying the information collected by the

nurse about the patient’s situation and status

into categories, for example communication,

nutrition, and psychosocial status. Ten key-

words classify the nursing interventions into

categories such as information, support, and

environment. The use of keywords simpliﬁes

information retrieval, although, to retrieve the

information asked for, a consensus about deﬁ-

nitions of categories must be reached.

The

VIPS model provides such a lexicon, in which

each category, labelled by a keyword, has a

deﬁnition, a description, and prototypical

examples given in a manual. Keywords may be

seen as a ﬁrst step towards a uniﬁed nursing

language for patient care.

The VIPS model has been received with

interest and appreciation by nurses in Sweden

and is now the most commonly taught and

used model for nursing documentation in hos-

pitals and primary health care.

Auditing patient records

Audit has to be distinguished from traditional

review (box 2). The audit of patient records

may be done for several reasons. The most

common reason is because the audit is part of

an ongoing process of quality improvement. A

clinic may have agreed to document in a

certain way, possibly with a minimum data set

as a standard, or wants to evaluate speciﬁc cri-

teria for quality health care.

19–21

Evidence exists that a continuously per-

formed audit of patient records, combined with

discussions about improvements, is one way to

improve the quality of the records and to

change certain behaviours of healthcare

professionals.

22–24

Another beneﬁt of auditing

documentation is that it makes comparisons

possible over time and among wards or hospi-

tals, provided that a reliable audit instrument is

used to put a numerical value on the written

content.

25 26

Audit is also used to evaluate the

eVects of quality management

by identifying

the necessary professional strengths and the

weaknesses that need to be addressed and cor-

rected. Craig reports that nurses using her

audit tool acquired a better understanding of

what was expected of them for recording of

care and patient care itself, and also focused on

the areas that they speciﬁcally needed to

concentrate for improvement.

It is important to diVerentiate between

auditing records for the sake of measuring the

quality of record keeping and auditing records

for the sake of measuring the quality of given

care. There is important criticism in the litera-

ture about the auditing of patient records for

the purpose of checking patient care, the argu-

ment being that patient records do not

necessarily reﬂect the reality of the given

care.

29 30

This raises the question whether it is

possible to claim that audit measures the qual-

ity of care. Whether better documentation can

also inﬂuence and improve patient care is

another question, not dealt with in this article.

Well written records, however, may be seen as a

step towards a process of quality assurance, as

a structured element in the nursing care. Don-

abedian is careful to stress that good structure

only increases the likelihood of a good process

in the actual care given and that the correlation

between process and outcome has yet to be

shown.

Two Swedish audit instruments have been

developed earlier, one by Ehnfors

and the

other one by Gardulf and Nordström.

Both

instruments are based on the nursing process

and evaluate the record in its quantitative

aspect; is there documentation for each func-

tion or is there not? In addition, Ehnfors evalu-

ates, for each patient problem, the ﬂow of

information in accordance with the nursing

process. Consequently, neither of the instru-

ments includes a qualitative evaluation of the

written content in the sense of the amount of

information, wording, pertinence, etc. Several

other audit instruments described in inter-

national journals

29 32–36

were also inadequate

with regard to the quality and quantity aspects

of auditing.

Improving nursing documentation is an

urgent issue. Poor documentation is an indica-

tion that further investigation is needed to

judge whether or not the given care is less than

optimal. Audit instruments for nursing records

are therefore needed to identify poor assess-

ment, poor structure, and the lack of a plan for

the patient’s care. They are also needed for

evaluating the eVects of interventions aimed to

improve the documentation.

The aims of this study were to develop an

audit tool to measure both the quantitative and

the qualitative aspects of nursing documenta-

tion, based on the VIPS model, and to evaluate

the validity and reliability of that tool.

Methods

DEVELOPMENT OF THE INSTRUMENT

Before the instrument was constructed by two

of the authors (CB, IT-E), a set of criteria was

identiﬁed to determine what questions needed

to be answered about nursing documentation

“DiVerences between audit and traditional

review:

x Use of explicit criteria for measurement

rather than implicit judgments

x Numerical comparison of current prac-

tice patterns against these criteria

x Formal identiﬁcation of action required

to resolve any discrepancies disclosed

x Recording the process to retain infor-

mation and increase impact of audit on

future management.”

Box 2 The diVerence between audit and review

8 Björvell, Thorell-Ekstrand, Wredling

in the patient record. These criteria were

derived from the following sources:

x The Swedish law that stipulates that nursing

documentation should include the steps of

the nursing process as described above, the

signing and dating of each entry, a minimum

degree of legibility, and a nursing discharge

note

x The VIPS model which includes the nursing

process, the use of speciﬁed keywords, the

correct classiﬁcation of the keywords in

accordance with the user manual, and a

nursing discharge note

x Common hospital policies that prescribe

that each patient should have a named nurse

with the primary responsibility for the

patient’s nursing care and care plan docu-

mentation.

At this stage, 19 questions were formulated

to determine whether this information was

documented in the patient record. Each

question was constructed to reveal both the

quantity and the quality of the written content

on a rating scale. A manual was designed to

explain how to score each question.

The quality and quantity values were scored

on a rating scale from zero to three, zero indi-

cating “poor” and three indicating “very

good”. The quantity value is expected to meas-

ure whether or not there is a written note and,

if so, how much is written. For example, for the

patient’s nursing status, a certain minimum

number of nursing areas, represented by

keywords in the VIPS model and relevant to

surgical care, should be described for a patient

in a surgical ward. The quality value is used to

measure to what degree the written notes are

clear and concise, without superﬂuous text,

and include all relevant nursing information

with a correct use of language. If all notes fulﬁl

these criteria, a full score of three is given; if

more than 50% of the notes, but not all of

them, fulﬁl the criteria, a score of two is given;

if less than 50% fulﬁl the criteria, though some

notes still fulﬁl the criteria, a score of one is

given, etc. Furthermore, the instrument is

expected to measure the extent to which it is

possible to follow a patient problem through

the nursing process. That is, whether the prob-

lem is properly assessed and described in a

diagnosis, with the expected outcome, planned

and implemented interventions, and an evalua-

tion. The instrument was named Cat-ch-Ing.

To test usability for understanding questions

and phrasing of the instrument, ﬁve patient

records collected from one hospital ward were

independently reviewed by three nurses using

the new instrument. The instrument was

revised after each of the three audits. The revi-

sions concerned the clariﬁcation of deﬁnitions

in the manual and the deletion or rephrasing of

questions. Two questions were omitted, one

about the evaluation of nursing care, which was

already covered by other questions, and the

other about the use of keywords other than

those stipulated by the VIPS model. One ques-

tion about the discharge note was rephrased.

TESTING OF RELIABILITY AND VALIDITY

Inter-rater reliability was tested by comparing

diVerent reviewers’ total Cat-ch-Ing scores

given to the same record. Twenty patient

records from each of three hospital wards at a

university hospital in Stockholm, Sweden were

used for this part of the development. The

records were selected from the registers of the

wards and were coded to protect patient iden-

tity. The specialty wards were surgery, neurol-

ogy, and rehabilitation. The criteria for the col-

lection of the records were that they should

concern the ﬁrst 20 patients from each ward

who were admitted for ﬁve days or more during

a speciﬁc time period. The collected records

were audited three times, each time by a diVer-

ent reviewer. The auditors were nurses know-

ledgeable and experienced in nursing docu-

mentation and in the use of the VIPS model.

Before the audit, a calibrating process was

undertaken, which means that the use of the

instrument was taught and discussed with the

reviewers.

The inter-rater reliability was statistically

investigated by calculating the inter-rater reli-

ability coeYcient

between raters’ total scores

of each record. Additionally, score diVerences

between reviewers, on each question in the

same patient record, were compared and

calculated as percentages of agreement.

The content-validity ratio was calculated as a

means of quantifying the degree of consensus

in a panel of 10 experts, who made judgments

about the instrument’s content validity. Each

expert was asked to judge whether or not the

10 questions in the instrument, meant to

measure the nursing process, were indeed

essential in measuring the parts of the nursing

process documented in a patient record. The

method, developed by Lawshe,

is described

by the formula:

where CVR is the content-validity ratio, ne is

the number of panellists indicating “essential”

about a speciﬁc question and N is the total

number of panellists.

The criterion-related validity was estimated

by the degree of correlation between the score

of the Cat-ch-Ing instrument and the score of

the audit instrument developed by Ehnfors

and used in previous research. The Ehnfors

instrument was constructed to measure

whether each part of the nursing process (and

thereby also the VIPS model) was documented

for each nursing problem identiﬁed in the

patient record. The nursing process was the

chosen criterion in both the Ehnfors and the

Cat-ch-Ing instrument. The Ehnfors instru-

ment has a score from zero to ﬁve, giving one

point for each documented part of the nursing

process: assessment, goal and diagnosis,

planned intervention, implemented interven-

tion, and a discharge note, concerning each

speciﬁed nursing problem. The Ehnfors instru-

ment scores mainly the quantity; the quality

aspect is only present for evaluating the ﬂow of

Development of an audit instrument for nursing care 9

information in the nursing process for each

patient problem.

A mean Ehnfors score of all identiﬁed nurs-

ing problems in a record was calculated and

compared with the total score given by the

Cat-ch-Ing instrument for the same record.

The results were then tested by the use of

Pearson’s correlation coeYcient.

The research protocol was approved by the

regional ethical committee of the Karolinska

Institutet.

Results

The ﬁnal version of the Cat-ch-Ing instrument

(appendix), which was completed in December

1996, consists of 17 questions: 10 reﬂecting the

presence of each step of the nursing process;

four about dating, signatures, and legibility;

one about keywords; and one asking about the

existence of the individual patient’s named

nurse.

SCORING

The total score ranges from zero to 80 points.

Sixty eight per cent of the total score may be

achieved by questions that are posed to

measure the content of the nursing process;

15% relate to questions that judge legibility,

signing, and dating; 7% correspond to key-

words; 5% to the nursing discharge note; and

5% to the identiﬁcation of a primary nurse.

Nine of the questions may be rated for quan-

tity and quality. Five questions can be rated

only for quantity, for example “are all entries

signed?”, and one question about legibility is

rated only for quality. The two remaining

questions have “yes” or “no” answers (ﬁg 3).

The inter-rater reliability coeYcients were

calculated to be 0.98, 0.98, and 0.92 for each

group of patient records from the three wards.

The content validity ratio between the expert

panellists (table 1) ranged between 0.20 and

1.0. Of the 12 items measuring the nursing

process in the instrument, all but three were

judged to be essential by the expert panellists.

The criterion related validity for the Cat-ch-

Ing instrument was illustrated by the signiﬁ-

cant correlation (r = 0.68, p = <0.0001, 95%

CI 0.57 to 0.76) between the scores of the

Ehnfors and the Cat-ch-Ing instruments (ﬁg

4).

On examining the score diVerences between

the three reviewers on each question (n=4680

comparisons) we found no diVerences in scores

in 64% of the comparisons. Thirty two per cent

of the comparisons diVered by one point and

4% diVered by two or three points. The largest

score diVerences (two or three points) occurred

on the items of qualitative judgment of nursing

assessment and nursing interventions. Larger

records (for example, 56 pages of text) had a

greater discrepancy among raters scoring the

same record than less extensive records.

Discussion

This study has resulted in a new instrument for

auditing nursing documentation of the patient

record. The instrument has proved to be valid

for measuring information pertinent to the

nursing process, and to possess a high degree of

reliability when used by diVerent auditors.

In the few records in which an increased dis-

crepancy among auditors was noted, the

patient records proved to be comprehensive.

The reasons for the discrepancy may be that it

Is there a nursing history?

Is there a patient status:

On arrival?

Updated?

At discharge?

Is there a nursing care plan:

Nursing diagnosis?

Expected outcome?

Interventions:

Planned?

Implemented?

Is the underlying information

for nursing diagnosis described

in the nursing status?

Is the nursing outcome described?

Explanation of the interpretation of given scores

The box describes the questions in the instrument that reflect the nursing

process, with the shaded area encompassing the parts adherent to the

nursing care plan. The scores for quantity and quality about the care plan

show that, for this patient, there were few, but still some, planned nursing

interventions documented (quantity = 1). Those that were documented

had excellent quality (quality = 3). However, there is no description at all

of analysis of the patient problem (nursing diagnosis) or the aim of the

care that supposedly led the nurse to her choice of intervention.

Quantity: 3

Quantity: 2

Quantity: 3

Quantity: 0

Quantity: 1

Quantity: 2

Quantity: 0

Quantity: 3

Quality: 1

Quality: 2

Quality: 1

Quality: 2

Quality: 0

Quality: 3

Quality: 2

Figure 3 Example of scoring in an audit of one patient record.

Table 1 Content validity ratio between expert panellists judging items of the instrument as

essential or not in measuring the nursing process in the patient record

Item

Judged as “essential”

by experts (n=10) Ratio

Is there a nursing history? 9/10 0.80

Is there a nursing status:

On arrival? 10/10 1.0

Updated? 9/10 0.80

On discharge? 8/10 0.60

Is there a nursing care plan:

Nursing diagnosis? 10/10 1.0

Expected outcome? 7/10 0.40

Interventions:

Planned? 9/10 0.80

Implemented? 9/10 0.80

Is the underlying information for the nursing diagnosis described

in the nursing status?

6/10 0.20

Is the nursing outcome described? 9/10 0.80

4.5

2.5

1.5

0.5

3.5

Ehnfors score

10 20 30 40 50 60 70 80

Cat-ch-Ing score

Figure 4 Correlation between the Ehnfors audit instrument

and the Cat-ch-Ing audit instrument for nursing

documentation, r = 0.68, p <0.0001, 95% CI 0.57 to

0.76.

10 Björvell, Thorell-Ekstrand, Wredling

is harder to keep track of pertinent information

in a large mass of nurses’ notes, or simply that

the auditor loses concentration after reading

the same record over a long period.

The Cat-ch-Ing instrument has been thor-

oughly investigated for validity and reliability.

Two types of validity have been conﬁrmed, and

using three reviewers in the reliability testing

strengthens the results. The Cat-ch-Ing instru-

ment showed a strong validity for measuring

whether the nursing process existed in the

patient record, which is one of the main

strengths of the instrument. It clearly gives a

measure not only of the amount of written text

but also, most importantly, of the quality of the

information that has been documented. Fur-

thermore, the Cat-ch-Ing is only a one page

instrument with a two page manual, whereas

other instruments usually have more extensive

instructions, and this may be a facilitating

factor.

Various limitations to the study should be

highlighted, however. Firstly, the auditors were

selected because of their knowledge and

experience in documentation, as well as in

nursing. This was thought to be a necessary

prerequisite when developing a new instru-

ment. We have not tested the instrument

among nurses in general. Secondly, the study

dealt with records from the wards of a major

university hospital that provides somatic acute

care and short term rehabilitation. The testing

we did could be considered valid for this type of

record only. Thirdly, the weighting of the scores

between the various questions may have to be

adjusted; as much as 32% of the score can be

achieved by dating and signing correctly, by

recording the named nurse, by using a

typewriter, and by using the keywords of the

VIPS model correctly. None of this indicates

the nursing process.

The development and testing of the Nord-

ström and Gardulf audit instrument for

nursing documentation has not been scientiﬁ-

cally described in the literature. The Phaneuf

Nursing Audit tool has been described in

numerous papers. Neither of these produces a

clear result of validation and reliability testing.

Also, the Phaneuf instrument claims to meas-

ure the quality of care by auditing the patient

record and has received criticism for this.

29 32 36

The Ehnfors instrument,

used as a compari-

son in this study, has an obvious, high, face

validity and inter-rater reliability when tested

by Kohen’s ê (ê = 0.93).

Group level comparisons with inter-rater

reliability coeYcients in the vicinity of 0.70

show suYcient reliability.

Thereby the reli-

ability of the Cat-ch-Ing instrument, with a

coeYcient of 0.98, proved to be very satisfac-

tory. It is a known problem that auditing

patient records involves subjective judgments.

Less inference is required of the auditor when

reviewing the documentation of demographic

information compared with that required when

assessing the adequacy of documentation

related to the patient speciﬁc needs and speciﬁc

nursing skills, such as educational strategies

and information giving.

According to Lawshe,

the minimum value

of the content validity ratio to ensure that

agreement is unlikely to be due to chance, with

10 panellists, is 0.62 per identiﬁed item. This

indicates that the Cat-ch-Ing instrument to a

large degree measures the documented nursing

process in the patient record, as intended.

Seven out of 10 items in the instrument

received satisfactory values. The three items

that received a lower value will be considered

for exclusion. The reasons for the lower value

for the questions, “is the underlying infor-

mation for the nursing diagnosis described in

the nursing status?” and “is there a new nursing

status at the time of discharge?”, may be

because the questions were thought to be

already covered by other questions in the

instrument. Why a lower value was given to the

question, “is the expected outcome (goal)

documented in the care plan?” is more diYcult

to explain. One reason may be that the expert

panellists all work with the nursing process

model in a theoretical setting where the patient

outcome is not an explicit part of the model,

whereas in the practical setting it is an explicit

part of the model.

The fact that yet another nursing audit

instrument has been developed and tested

implies to nurses that the auditing of nursing

performance is an important subject, possibly

making more nurses familiar with auditing and

quality improvement. One approach to increas-

ing the awareness and knowledge of the audit-

ing of nursing documentation and care plan-

ning is to encourage the use of a peer review

system. By using an instrument like the

Cat-ch-Ing, peer review of patient records may

be a means not only of improving patient

records but also of instigating a discussion and

thereby possibly reaching a consensus on best

nursing care in speciﬁc situations, which may

improve direct care.

The criterion based audit is a concept used

in medicine

which may be applicable to nurs-

ing also. In this study, the nursing process was

used as the evaluated criterion because this is

what Swedish law prescribes, and may be seen

as the short term goal—to improve nursing

documentation and record keeping. However,

the Cat-ch-Ing instrument is constructed so

that it could be modiﬁed to measure speciﬁc

criteria of nursing care quality, as documented

in the patient record. Modiﬁcation would then

be described in the user manual, for example

what interventions are expected in the nursing

care plan for a patient with a speciﬁc problem

to get a full score, or what speciﬁc information

will be expected under the keyword of

nutrition for a patient with newly discovered

diabetes in order to get a full score. This may

be seen as a long term goal of auditing within

nursing care.

Conclusion

It can be concluded that the Cat-ch-Ing

instrument proved to be a valid and reliable

audit instrument for nursing documentation

in patient records when the VIPS model was

used as the basis of the documentation. Nurs-

ing, as a growing scientiﬁc discipline, is

Development of an audit instrument for nursing care 11

constantly adding new knowledge to clinical

care and thereby increasing the need to be able

to detect whether patient care was docu-

mented in accordance with scientiﬁc ﬁndings.

The next step, once record keeping is

improved, will be to evaluate the eVect that it

has on patient care.

We are grateful to the Stockholm County Council, whose gen-

erous grant made this study possible. We also thank Anders

Sjöberg for statistical advice.

1 Yura H, Walsh M. The nursing process. Assessing, planning,

implementing, evaluating. 5th edition. Norwalk, CT: Apple-

ton & Lange, 1988.

2 Meleis A. Theoretical thinking: development and progress. 2nd

edition. Philadelphia: Lippincott Company, 1991.

3 World Health Organisation. Nursing process workbook.

Copenhagen: WHO Regional OYce for Europe, 1982.

4 Clark J. An international classiﬁcation for nursing practice.

In: Bakken S, Holzemer W, Tallberg M, et al, editors.

Informatics: the infrastructure for quality assessment improve-

ment in nursing. Proceedings of the 5th international nursing

informatics symposium post-conference; 1994 June 24–25;

Austin, Texas. San Francisco: UC Nursing Press, 1994.

5 Joint Commission on Accreditation of Healthcare Organiza-

tions. Accreditation manuals for hospitals. Nursing Care Stand-

ards. Outbrook Terrace: The Commission, 1991.

6 The United Kingdom Central Council for Nursing,

Midwifery and Healthvisiting. Standards for records and

record keeping. London: The United Kingdom Central

Council for Nursing, Midwifery and Healthvisiting, 1993.

7 Svensk författningssamling. 1985:562 Patientjournallagen.

Stockholm: Liber Allmänna förlaget; 1985. (Swedish law on

patient record keeping.)

8 Socialstyrelsen. Föreskrifter och allmänna råd i omvårdnad

inom hälso- och sjukvården. SOSFS 1993:17. Stockholm:

Socialstyrelsen; 1993. (Swedish National Board of Health

and Welfare.)

9 Ehnfors M. Nursing care as documented in patient records.

Scand J Caring Sci 1993;7:209–20.

10 Nordström G, Gardulf A. Nursing documentation in

patient records. Scand J Caring Sci 1996;10:27–33.

11 Ehrenberg A, Ehnfors M. Patient problems, needs and

nursing diagnoses in Swedish nursing home records. Nurs

Diag 1999;10:65–76.

12 Carpenito LJ. Nursing diagnosis. Application to clinical

practice. 7th edition. Philadelphia: Lippincott Company,

1997.

13 Jairath N. Strategies for motivating CCU patients. Dimens

Cr it Care Nurs 1994;13:324–33.

14 Kramer M. Nursing care plans. Power to the patient. J Nurs

Adm 1972;Sept-Oct:29–34.

Record #:

Date:

Read the User Manual carefully

* = see the User Manual.

Scores within brackets.

Is there a primary nurse indicated?

Is there a nursing history?

Is there a nursing status:

On arrival?

Updated?*

At discharge?

Is there a nursing care plan:

Nursing diagnosis?*

Expected outcome?*

Interventions:

Planned?

Implemented?

Is the underlying information for the

nursing disgnosis described in nursing status?

Is nursing outcome described?*

Are the VIPS keywords used?*

(regarding history, status, interventions)

Is there a nursing discharge note?

Are all entries dated (year, month, day)?

Are all entries signed?

Is there a clarification of signature?

Is the record legible?*

only by surname

by surname and christian name

(0)

(2)

(4)

Ward:

Reviewer:

Hospital:

Quantity*

Complete = (3)

Partly = (2)

Occasional = (1)

None = (0)

Quality*

Very good = (3)

Good = (2)

Less good = (1)

Poor = (0)

Quantity: Quality:

Quantity:

Quantity: Quality:

Yes (4) No (0)

Quantity:

Quality:

Total score: (max 80)

Appendix

12 Björvell, Thorell-Ekstrand, Wredling

15 Black S, Taunton R, Thomas J, et al. Evaluation of a scale to

assess nurses’ attitudes towards written care plans. Appl

Nurs Res 1989;2:92–5.

16 Ehnfors M, Thorell-Ekstrand I, Ehrenberg A. Towards

basic nursing information in patient records. Vard i Norden

1991;21:12–31.

17 Ehrenberg A, Ehnfors M, Thorell-Ekstrand I. Nursing

documentation in patient records: experience of the use of

the VIPS-model. J Adv Nurs 1996;24:853–67.

18 Grobe J, Hughs C. The conceptual validity of a taxonomy of

nursing interventions. J Adv Nurs 1993;18:1942–61.

19 Shaw C. Criterion based audit. BMJ 1990;300:649–51.

20 GriYths J, Hutchings W. The wider implication of an audit

of care plan documentation. J Clin Nurs 1999;8:57–65.

21 Honnas R, Zlotnick C. Quality improvement in action:

development of a tool. J Nurs Care Qual 1995;9:72–7.

22 Mashru M, Lant A. Inter-practice audit of diagnosis and

management of hypertension in primary care: educational

intervention and review of medical records. BMJ 1997;314:

942.

23 Gabbay J, McNicol M, Spiby J, et al. What did audit

achieve? Lessons from preliminary evaluation of a year’s

medical audit. BMJ 1990;301:526–9.

24 Heath D. Random review of hospital patient records BMJ

1990;300:651–2.

25 Hansebo G, Kihlgren M, Ljunggren G. Review of nursing

documentation in nursing home wards - changes after inter-

vention for individualised care. J Adv Nurs 1999;29:1462–

73.

26 Fagrell B, Funcke L, Nyberg K. Nursing documentation

according to the VIPS-model at nursing home. Vard i

Norden 1998;18:40–5.

27 Webb C, Pontin D. Evaluating the introduction of primary

nursing: the use of a care plan audit. J Clin Nurs

1997;6:395–401.

28 Craig D. Audit design. Recent Advances in Nursing 1987;17:

65–93.

29 Sparrow S, Robinson J. The use and limitations of

Phaneuf’s nursing audit. J Adv Nurs 1992;17:1479–88.

30 McElroy A, Corben V, McLeish K. Developing care plan

documentation: an action research project. J Nurs Manage

1995;3:193–9.

31 Donabedian A. The quality of care. how can it be assessed?

JAMA 1988;260:1743–8.

32 Manfredi C. Reliability and validity of the Phaneuf Nursing

Audit. WJNR 1986;8:168–80.

33 Goldstone L, Ball J, Collier M. Monitor: an index of the qual-

ity of nursing care for acute medical and surgical wards.

Newcastle upon Tyne: Newcastle upon Tyne Polytechnic

Products, 1983.

34 Harvey G. An evaluation of approaches to assessing the

quality of nursing care using (predetermined) quality assur-

ance tools. J Adv Nurs 1991;16:277–86.

35 Vandelt M, Ager J. Quality patient care scale. New York:

Appleton-Century-Crofts, 1974.

36 Ventura M. Correlation between the quality patient care

scale and the Phaneuf Audit. IntJNursStud1980;17:155–

62.

37 Winer, B. Statistical principles in experimental design. 2nd edi-

tion. London: McGraw-Hill, 1971.

38 Lawshe C.H. A quantitative approach to content validity.

Personnel Psychology 1975;28:563–75.

39 Polit D, Hungler B. Nursing research. principles and methods.

5th edition. Philadelphia: Lippincott, 1995.

40 Edwards N, Pickard L, van Berkel C. Community health

nursing audit: issues encountered during the selection and

application of an audit instrument. Public Health Nursing

1991;8:3–9.

Development of an audit instrument for nursing care 13