Report on the Food Safety and Inspection Service‘s
Microbiological and Residue Sampling Programs
United States Department of Agriculture
Food Safety and Inspection Service
December 2011
1
Executive Summary .............................................................................................................................5
1.0 Introduction ....................................................................................................................................5
Background......................................................................................................................................6
Current Intent and Purpose of FSIS Sampling Programs ................................................................6
Sampling Along the Farm-to-table Continuum ...............................................................................8
Relationship between FSIS Sampling and USDA/FSIS Strategic Plan Goals to Utilize a Data-
Driven Approach and Reduce Foodborne Illness ............................................................................8
Organization of Sampling Program Report .....................................................................................9
FSIS Public Health Information System (PHIS)............................................................................ 10
Definition of Terms........................................................................................................................ 10
2.0 Domestic, Regulated Establishments .......................................................................................... 11
Section 2.1: Microbiological Sampling Programs ......................................................................... 11
2.1.1: Salmonella ........................................................................................................................... 11
Overview of Sampling Programs......................................................................................11
Salmonella Verification Program: HC01..........................................................................13
MT43S Sampling Program ............................................................................................... 25
Processed Egg Products—EM 31-37................................................................................ 26
Salmonella Programs Measure of Success ....................................................................... 28
2.1.2 Campylobacter.................................................................................................................... 32
Campylobacter Verification Program: HC11 ................................................................... 32
Campylobacter Programs Measure of Success.................................................................32
2.1.3: E. coli O157:H7 Raw Sampling Project ............................................................................. 34
Routine testing of domestic components to raw ground beef – other than trim – at
Follow-up testing to a positive in trim or components at a domestic establishment
Overview of Sampling Projects ........................................................................................ 34
Historical Basis for Sampling Raw Ground Beef ............................................................. 34
Historical Basis for Sampling of Products Other than Raw Ground Beef........................36
Description of the FSIS Domestic Sampling Projects for E. coli O157:H7 ..................... 38
Risk-based testing of raw ground beef at domestic establishments (MT43) .................... 38
Follow-up testing to a raw ground beef positive at a domestic establishment (MT44)....41
Routine testing of manufacturing trimmings at domestic establishments (MT50)...........42
domestic establishments (MT54)......................................................................................43
Routine testing of bench trim at domestic establishments (MT55) .................................. 45
(MT53).............................................................................................................................. 46
2
Follow-up testing at supplier establishments following a positive in raw ground beef and
bench trim (MT52)............................................................................................................ 47
Limitations of Current Sampling Programs......................................................................49
E. coli O157:H7 Measures of Success..............................................................................49
2.1.4: RTE Meat and Poultry Products (Listeria monocytogenes (Lm) and Salmonella) .............. 54
Overview of Sampling Projects ........................................................................................ 54
Historical Basis of Sampling Programs ............................................................................ 55
Description of the FSIS RTE Domestic Sampling Projects for Lm and/or Salmonella.... 56
ALLRTE ........................................................................................................................... 56
RTE001............................................................................................................................. 58
RLm .................................................................................................................................. 59
Intensified Verification Testing (IVT)..............................................................................60
Limitations of Current Sampling Programs......................................................................61
Lm Sampling Program Measures of Success .................................................................... 63
Section 2.2: FSIS Microbiological Baseline Data Collection........................................................66
Section 2.3 Chemical Residues..................................................................................................... 67
Overview of Sampling Projects ........................................................................................ 67
Background Information...................................................................................................67
Fast Antimicrobial Screen Test (FAST) ........................................................................... 69
Kidney Inhibition Screen (KIS™) Test ............................................................................ 70
3.0 Imports.................................................................................................................................... 72
Section 3.1: Microbiological Sampling Programs ......................................................................... 72
Overview of Sampling Programs......................................................................................72
3.1.1: Salmonella .......................................................................................................................... 73
EGGIMP—Processed Egg Products.................................................................................74
3.1.2: E. coli O157:H7................................................................................................................... 75
Ground Beef (MT08) ........................................................................................................ 76
Non-Intact Beef (MT51)...................................................................................................76
3.1.3 RTE Meat and Poultry Products ........................................................................................... 78
IMVRTE—Imported Intact RTE Product ........................................................................ 79
Imports Measures of Success............................................................................................79
Section 3.2: Chemical Residues..................................................................................................... 81
Fresh and Processed..........................................................................................................81
3
4.0 In-Commerce Activities......................................................................................................... 82
Overview........................................................................................................................... 82
MT05 and MT06—Raw Ground Beef at Retail ............................................................... 84
In-Commerce Measures of Success .................................................................................. 85
5.0 Foodborne Illness Investigation and Consumer Complaint Sampling Programs .................. 86
6.0 Appendices .................................................................................................................................... 89
Appendix A: Definition of Terms..................................................................................................90
Appendix B: E. coli O157:H7 Sampling Program Description and Features................................92
4
Executive Summary
Background
The mission of the Food Safety and Inspection Service (FSIS) is to ensure that meat, poultry and
processed egg products are safe, wholesome and properly labeled and packaged. Further, as FSIS is a
public health agency, one of the primary goals of the Agency is to reduce and ultimately prevent
foodborne illness in the population.
The overall purpose of FSIS inspection and sampling is to ensure that establishments maintain control of
their production processes and adhere to FSIS regulations, policies and performance standards, which
the Agency believes helps protect the public from foodborne illnesses. Ensuring that products are free
of pathogen contamination and chemical residues is a responsibility of industry, but on-going sampling
in FSIS-regulated domestic and import establishments allows the Agency to assess the effectiveness of
industry process controls, compliance with performance standards and efforts to control the presence of
pathogens on products being produced for American consumers. Additionally, sampling provides a
strong incentive for the meat, poultry and processed egg product industries to reduce the presence of
pathogens on products they produce. It also provides the regulated industries with critical information
to improve current processes and focus its resources as efficiently and effectively as possible.
FSIS Sampling Programs
Currently, FSIS maintains sampling programs in three major venues: 1) Domestic, federally inspected
establishments, 2) In-Commerce and 3) Import. FSIS also conducts an outbreak investigation and
consumer complaint sampling program. Further, FSIS conducts two distinct types of sampling: 1)
Microbiological and 2) Chemical Residues.
Purpose of Report
In September 2011, FSIS published the FSIS Strategic Plan for 2011-2016. The Plan, as a part of FSIS‘
goal to ensure that food safety inspection aligns with existing and emerging risks, identifies the
Agency‘s commitment to develop an annual sampling report that comprehensively identifies and
describes the Agency‘s sampling programs. This report was developed to address that commitment and
is being released now, subsequent to the release of the Strategic Plan, to reflect FSIS‘ commitment to
transparency and provide information on the Agency‘s sampling programs in a timely manner. This
report was developed with input from all FSIS program areas and includes information on how the
Agency‘s sampling programs were carried out in fiscal year (FY) 2011. Specifically, with the
publication of this report, FSIS is documenting its current approach to microbiological and residue
sampling. This report includes information on the historical basis, design, statistical/policy basis and
limitations of FSIS‘ current sampling programs.
However, the development and maintenance of robust, responsive and meaningful sampling programs
requires an iterative process, including review of sampling results, incorporating new technological and
methodological advancements, new and modified FSIS policy and feedback from all stakeholders to
promote continuous improvement. As such, this report is the first of two publications developed by
FSIS. The second part of this effort, which is currently in development, is the FSIS annual Sampling
Program Plan, which will identify programmatic sampling changes that will be implemented by the
Agency in FY2012. FSIS anticipates that it will share this plan publicly by early calendar year (CY)
2012.
5
1.0 Introduction
Background
The mission of the Food Safety and Inspection Service (FSIS) is to ensure that meat, poultry and
processed egg products are safe, wholesome and properly labeled and packaged. Further, as FSIS is a
public health agency, one of the primary goals of the Agency is to reduce and ultimately prevent
foodborne illness in the population.
Product testing, whether performed by industry or FSIS, is particularly important in gauging the safety
of regulated product. Ensuring that products meet pathogen contamination standards and chemical
residue levels is the responsibility of industry. The routine sampling in FSIS-regulated domestic and
import establishments allows the Agency to assess the effectiveness of industry process controls,
compliance with performance standards and the monitoring the proportion of finished product where
microbiological or chemical contaminants are detected on products being produced for American
consumers. Additionally, sampling serves as a strong incentive for the meat, poultry and processed egg
product industries to reduce the presence of pathogens on products they produce. Further, product
sampling provides the regulated industries with critical information to improve current processes and
focus their resources as efficiently and effectively as possible.
The Hazard Analysis and Critical Control Points (HACCP) system is an established food safety system,
whereby meat and poultry establishments identify and evaluate hazards that can affect the safety of their
products, institute controls necessary to prevent those hazards from occurring or keeping them within
acceptable limits, monitor the performance of controls and maintain records of these practices.
Microbiological and chemical residue sampling are critical components of HACCP and, as a part of
FSIS‘ verification responsibilities, are used to help ensure that foods regulated by the Agency are safe to
eat, verify that prevention efforts undertaken by a domestic establishment are successfully controlling
pathogens and chemical residues and ensuring products imported from foreign countries are safe and
wholesome. FSIS published the ‗‗Pathogen Reduction; Hazard Analysis and Critical Control Point
(PR/HACCP) Systems‘‘ Final Rule (61 FR 38806) in July 1996. The overarching goal of FSIS‘ food
safety strategy and the PR/HACCP regulations is to reduce, to the maximum extent possible, the risk of
foodborne illness associated with the consumption of meat and poultry products.
Current Intent and Purpose of FSIS Sampling Programs
FSIS inspects regulated establishments in a comprehensive fashion. The overall purpose of FSIS
inspection and sampling is to verify that establishments maintain control of their production processes
and adhere to Agency regulations, policies and performance standards, which FSIS believes helps
protect the public from foodborne illnesses. Because sampling is part of FSIS‘ verification activities,
samples are collected at regular intervals, (e.g., once a week or month) though the frequency of
microbiological testing is sometimes stratified based on an establishment‘s production volume. FSIS‘
microbiological testing programs were developed to be pathogen-specific. Consequently, the outcome
of a positive sample can vary with each program. For example, some FSIS sampling programs enforce
the Agency‘s zero tolerance policy for the presence of pathogens, such as Escherichia coli (E. coli
O157:H7) and Listeria monocytogenes (Lm). Others, such as verification programs for Salmonella and
Campylobacter in raw products, evaluate how well regulated establishments demonstrate process control
for these pathogens. In the section below, the intent of each sampling program is briefly described. A
6
more complete description of the intent and purpose of each microbiological testing program, as well as
chemical residue testing, are included in the following chapters.
Domestic, Federally Inspected Establishments
Microbiological
Currently, FSIS conducts microbiological sampling for four major pathogens of human health concern:
Salmonella, Campylobacter, E. coli O157:H7
1
and Lm. Briefly, for Salmonella sp., FSIS conducts a
verification testing program per the 1996 PR/HACCP Rule to ensure that the Salmonella reduction
performance standards are being met for the eight raw product classes, as well as a sampling program
for processed egg products. For Campylobacter, FSIS initiated a sampling program in young chicken
and turkeys in July 2011 based on new performance standards. For E. coli O157:H7, the objective of
the verification testing program is to verify the effectiveness of the food safety system on a national
level and encourage continuous industry improvement in the reduction of the pathogen in raw ground
beef and other raw beef products. Finally, for ready-to-eat (RTE) meat and poultry products, the
objective of the multiple regulatory testing programs is to verify the effectiveness of food safety systems
in preventing the presence of Lm and Salmonella in RTE products and in the establishment‘s
environment. In addition to these routine sampling programs, most microbiology programs have a
consequential (for cause) sampling component when the production process within a regulated
establishment is determined to be out of control.
Chemical Residue
FSIS sampling programs for chemical residues both in domestic and import establishments are designed
to achieve the following: (1) a structured process for identifying and evaluating compounds of concern
by production class; (2) the capability to analyze for compounds of concern; (3) appropriate regulatory
follow-up to reports of violative tissue chemical residues; and (4) collection, statistical analysis and
reporting of the results of these activities.
In-Commerce Sampling
The in-commerce sampling program is one facet of a comprehensive set of activities conducted by FSIS
Compliance and Investigations Division (CID) Investigators. Microbiological sampling of FSIS-
regulated products in-commerce is intended to verify that persons and firms—whose business activities
involve FSIS-regulated products— prepare, store, transport, sell or offer for sale or transportation these
products in compliance with the Agency‘s statutory and regulatory requirements.
Import Sampling Programs
In order to focus FSIS resources on imported products that may pose the greatest threat to public health,
the Agency utilizes a performance-based approach to define the scope of equivalence verification audits
of foreign countries and to determine the frequency of Point-Of-Entry (POE) reinspections. Consistent
with domestic programs, this approach relies on previous audit findings and inspection results, as well as
information regarding the product types and product volumes presented for importation into the U.S. In
1
FSIS recently published a proposed rule in the Federal Register stating the Agency‘s intention to carry out verification
procedures, including sampling and testing manufacturing trim and other raw ground beef components, to ensure control of
both E. coli O157:H7 and six other serogroups of Shiga toxin-producing E. coli (STEC) (O26, O45, O103, O111, O121, and
O145). FSIS has determined that they, as well as O157:H7, are adulterants of non-intact raw beef products and product
components within the meaning of the Federal Meat Inspection Act (FMIA).
http://www.fsis.usda.gov/OPPDE/rdad/FRPubs/2010-0023.pdf.
7
import sampling, FSIS utilizes foreign country/establishment past performance, sampling history,
product type and product volume produced to assist in the design of sampling plans for Salmonella, E.
coli O157:H7, and Lm. The overall purpose of FSIS import sampling is to ensure that meat, poultry and
processed egg products imported into the U.S. are produced under an equivalent inspection system and
are as safe as products produced domestically.
Sampling Along the Farm-to-Table Continuum
FSIS‘ sampling programs in establishments, in-commerce and for imported products are key
components of a farm-to-table approach to food safety. This approach includes all facets of the food
production process; from animal slaughter within FSIS regulated establishments, to food processing
within regulated establishments, to retail establishments and home environments. FSIS seeks to have an
active presence in all these venues and includes pre-harvest activities, establishment inspection, industry
education at retail and consumer education in the home. For example, in domestic E. coli O157:H7
sampling, FSIS collects samples of beef, such as bench trim and components, as it is processed. FSIS
also conducts a risk-based sample collection program in raw ground beef. If any positive samples are
identified, FSIS conducts additional sampling to ensure that the establishment regains control of its
processes and produces safe product. Finally, FSIS maintains an independent sampling program in-
commerce for raw ground beef. While these three sampling projects were designed and are maintained
separately, by sampling in these three venues—slaughter, processing, and in-commerce—FSIS seeks to
verify the effectiveness of industry process controls across multiple sectors and target areas for
intervention along this continuum when more attention is needed. The same principles apply to
sampling for other pathogens and product classes. For example, FSIS samples in establishments on food
contact and non-food contact environmental surfaces, as well as sampling for Lm in both non-post-
lethality exposed and post-lethality exposed RTE products. As with E. coli O157:H7 sampling, any
positive test results are followed-up with additional, intensified verification sampling to ensure that
establishments identify the source of contamination and bring processes back under control. Finally,
FSIS seeks to harmonize sampling programs for both domestic and imported meat, poultry and
processed egg products. As such, FSIS samples for Salmonella, E. coli O157:H7 and Lm in imported
products such as beef, processed egg products and RTE. In summary, by adopting a farm-to-table
approach not only in FSIS‘ overall approach to food safety, but also within the Agency‘s sampling
programs, FSIS seeks to address food safety risks along the food chain and reduce the overall number of
foodborne illnesses associated with FSIS regulated products.
Relationship between FSIS Sampling and USDA/FSIS Strategic Plan Goals to Utilize a Data-
Driven Approach and Reduce Foodborne Illness
In September 2010, FSIS released two reports; the FSIS Strategic Data Analysis Plan for Domestic
Inspection
2
and Data-Driven Inspection for Processing and Slaughter Establishments: Public Health
Decision Criteria.
3
These reports were developed to communicate FSIS‘ strategy for a data-driven
approach to domestic inspection and describe the Agency‘s public health-based, data-driven decision
criteria and a decision tree to select meat and poultry establishments for additional inspection activities.
Further, these reports were designed to directly support FSIS‘ strategic goals by providing the data and
analyses necessary to effectively allocate resources and measure performance. The release of this 2011
2
Please see the following website for more information:
http://www.fsis.usda.gov/OPPDE/NACMPI/Sep2010/2010_Strategic_Data_Analysis_Plan.pdf.
3
Please see the following website for more information:
http://www.fsis.usda.gov/OPPDE/NACMPI/Sep2010/2010_Public_Health_Decsion_Criteria_Report.pdf.
8
Sampling Program report continues FSIS‘ efforts to comprehensively identify Agency activities and
consider them in light of data-driven strategic planning efforts.
Similarly, the U.S. food safety system involves multiple stakeholders along the farm-to-table continuum
and includes multiple federal and state partners, regulated industries, as well as active participation from
the American consumer. While FSIS recognizes that inspection and sampling at regulated slaughter and
processing establishments are just two pieces of FSIS‘ food safety activities, the Agency believes that
reducing the overall presence of harmful foodborne pathogens on products regulated by FSIS can bring
about reductions in foodborne illnesses from Salmonella, Campylobacter, E. coli O157:H7 and Lm in
the U.S. population. FSIS is working internally and externally with federal food safety partners to
further refine our understanding of the relationship between Agency activities and foodborne illness.
Organization of Sampling Program Report
This report serves as the first of a two-part effort by FSIS to increase transparency and share information
regarding the Agency‘s microbiological and residue sampling programs. The purpose of this report is to
provide a historical grounding and a detailed description of FSIS‘ current sampling programs. This
report contains information on how FSIS carries out the Agency‘s sampling programs through fiscal
year 2011. Given the timeframe in which this report is being released, however, tables containing
sample analysis numbers are current as of fiscal year 2010. Fiscal year 2011 sample collection and
analysis numbers will be provided in the Agency‘s annual Sampling Program Plan, scheduled for release
in the beginning of CY2012.
This report comprehensively describes the current design, statistical/policy basis and current limitations
of FSIS‘ sampling programs. For ease of understanding and to correspond with FSIS‘ current
organizational structure, this report is organized by the major venues in which FSIS conducts sampling;
1) Domestic, federally-inspected establishments, 2) Imports and 3) In-Commerce. FSIS also conducts
outbreak investigation and sampling in response to illnesses and consumer complaints. Further, FSIS
conducts two distinct types of sampling: 1) Microbiological and 2) Chemical Residues.
In general, sampling for a specific pathogen, such as Salmonella, is referred to in this report as
―sampling program‖, whereas individual sampling for specific pathogens, such as HC01 for Salmonella,
are considered ―sampling projects‖. As such, this report will be organized first around the venue in
which the sampling occurs and second, around the type of sampling program. Each major section of this
report also contains information about the type of analysis FSIS conducts to determine whether or not,
and to what extent, a pathogen exists on regulated product and a discussion of the volume of product
produced by regulated industries, as it relates to FSIS sampling programs.
Future FSIS Sampling Program Activities
FSIS anticipates that the Agency‘s sampling activities will evolve over time for myriad reasons,
including new and emerging public health hazards, as well as technological and methodological
advancements and updates to FSIS policy. Consequently, the second part of this effort involves the
development, as per the recently published FSIS Strategic Plan for 2011-2016, of a sampling program
Plan that identifies fiscal year (FY) 2011 sample collection and analysis numbers, changes to the
Agency‘s sampling activities to be implemented in FY2012 and provides summary measures for
existing and emerging food safety hazards. FSIS anticipates that the plan will also be shared publicly
and should be finalized by the beginning of calendar year (CY) 2012.
9
FSIS Public Health Information System (PHIS)
On April 11, 2011, FSIS launched its dynamic, comprehensive data analytics system called the Public
Health Information System (PHIS). PHIS is a web-based application that integrates and automates
FSIS‘ paper-based business processes into one comprehensive and fully automated data-driven
inspection system. It will help facilitate the sharing of data among inspection personnel, their managers
and headquarters on a daily basis. PHIS is also a powerful decision-making tool that will enable FSIS to
protect public health more efficiently, effectively and rapidly than under existing systems.
As a result of implementing PHIS, many of FSIS' existing systems, such as the Performance Based
Inspection System (PBIS) and, eventually, the Automated Import Information System (AIIS), will be
phased out and replaced by PHIS. As such, the way in which sampling information is scheduled, shared
and stored will change under PHIS. However, it is important to note that none of the fundamental
elements of FSIS‘ sampling activities, such as the sampling frame, methodology or collection
methodology will change and the transition to PHIS will not affect the day-to-day operations of
pathogen verification and other FSIS sampling programs.
Definition of Terms
As this report focuses primarily on FSIS sampling programs, the reader may encounter several terms
that might not be familiar. Consequently, a catalogue of technical terms utilized in this report is
included in Appendix A.
10
2.0 Domestic, Regulated Establishments
Section 2.1: Microbiological Sampling Programs
2.1.1: Salmonella
Overview of Sampling Programs
FSIS collects Salmonella samples in eight raw meat and poultry product classes (Salmonella Pathogen
Reduction Performance Standards-- project code HC01 and HC11)
4
, two RTE meat and poultry products
(project codes ALLRTE and RTE001) and nine processed egg products categories (project codes EM
and EGGDOM). Broiler and turkey samples collected under HC11 are co-analyzed for Campylobacter
(see Section 2.1.2). In addition, Salmonella testing is conducted on some raw ground beef samples
collected for E. coli O157:H7 testing, with the Salmonella results recorded under project code MT43S.
In the section below, the HC01, HC11, MT43S and Egg Sampling programs are reviewed. The
Salmonella sampling program for RTE meat and poultry products (ALLRTE and RTE001) will be
reviewed in the Lm RTE section of this report. Information on different domestic Salmonella sampling
projects is summarized in Table 2.1.1.1.
5
4
Sample sets for Market Hogs, Cows/Bulls, or Steers/Heifers were not scheduled in the latter half of FY 2011.
5
As of July 1, 2011 all broiler and turkey sets are being co-analyzed for Campylobacter and scheduled as HC11, not HC01.
11
Table 2.1.1.1 FSIS Salmonella Domestic Sampling Projects
Product class
Number of
Salmonella
Samples
Analyzed
FY2010
Regulatory Purpose of
Sampling Project
Type of
Sampling
Project
Steers/heifers
6
6,550
Verify consistent process
control
Risk Based
Cows/bulls
7
1,688
Verify consistent process control
Risk Based
Raw Ground beef
8,982
Verify consistent process control
Risk Based
Market hogs
8
305
Verify consistent process control
Risk Based
Broilers
9
762
Verify consistent process control
Risk Based
Ground chicken
3,913
Verify consistent process control
Risk Based
Ground turkey
3,811
Verify consistent process control
Risk Based
Turkeys
10
1,303
Verify consistent process control
Risk Based
Raw Ground beef
2,957
Verify consistent process control
Random
Ready-to-eat (RTE)
meat and poultry
products
2990
Verify adequacy of an
establishment's ability to prevent
microbiological contamination
Random
Ready-to-eat (RTE)
meat and poultry
products
8700
Verify adequacy of an
establishment's ability to prevent
contamination from Salmonella
and Lm
Risk Based
Egg whites with or
without added
ingredients
292
Verify adequacy of a
establishment‘s ability to prevent
contamination from Salmonella
Random
Whole eggs/yolks
with <2% added
ingredients other
than salt or sugar
389
Verify adequacy of a
establishment‘s ability to prevent
contamination from Salmonella
Random
Whole eggs/yolks
with ≥2% added
ingredients other
than salt or sugar
141
Verify adequacy of a
establishment‘s ability to prevent
contamination from Salmonella
Random
6
Supra footnote 4.
7
Supra footnote 4.
8
Supra footnote 4.
9
No longer being scheduled for HC01 with the implementation of the new Salmonella and Campylobacter performance
standards in July 2011.
10
Supra footnote 9.
12
Product class
Number of
Salmonella
Samples
Analyzed
FY2010
Regulatory Purpose of
Sampling Project
Type of
Sampling
Project
Whole eggs/yolks
with ≥2% salt or
sugar added
287
Verify adequacy of a
establishment‘s ability to prevent
contamination from Salmonella
Random
Dried yellow egg
products
114
Verify adequacy of a
establishment‘s ability to prevent
contamination from Salmonella
Random
Spray dried egg
whites (with or
without added
ingredients)
104
Verify adequacy of a
establishment‘s ability to prevent
contamination from Salmonella
Random
Pan dried egg
whites
10
Verify adequacy of a
establishment‘s ability to prevent
contamination from Salmonella
Random
Domestic liquid,
frozen or dried egg
products
61
Verify adequacy of a
establishment‘s ability to prevent
contamination from Salmonella
Random
Salmonella Verification Project: HC01
Historical Basis
In 1996, FSIS adopted the PR/HACCP final rule that instituted pathogen reduction performance
standards in the major species and raw product classes.
11
At that time, there were eight classes of raw
products for which FSIS had conducted nationwide baseline studies. These classes were: steers/heifers,
cows/bulls, raw ground beef, broilers, market hogs, ground turkey, ground chicken and turkeys. From
these baseline studies, FSIS estimated the mean prevalence of Salmonella in each class and then
developed establishment-level performance standards to encourage all establishments to produce
product whose prevalence of contamination was less than or equal to the mean prevalence for each of
these classes of raw product. ―Minor species,‖ such as sheep, goats, equines, ducks, geese and guineas,
were not addressed because FSIS chose to first address the most commonly consumed foods under its
jurisdiction. At the time, FSIS intended to address how best to gather data and develop pathogen
reduction performance standards for these other food animals at a future date.
Salmonella, regardless of serogroup/serotype, was selected as the target organism for a number of
reasons: (1) it is the most common bacterial cause of foodborne illness; (2) FSIS baseline data show that
Salmonella colonizes a variety of mammals and birds and occurs at frequencies that permit changes to
be detected and monitored; (3) current methodologies can recover Salmonella from a variety of meat
The most complete record of the original design and implementation strategy for this verification testing program is contained in the
Pathogen Reduction/Hazard Analysis and Critical Control Point System final rule (61 FR 38806, July 25, 1996; see:
http://www.gpo.gov/fdsys/pkg/FR-1996-07-25/pdf/96-17837.pdf. In addition, on February 17, 2005, FSIS published updated pathogen
reduction performance standards for the raw classes of product that were identified as under development but not yet ready to publish in the
July 25, 1996, Federal Register (see: http://www.fsis.usda.gov/OPPDE/rdad/FRPubs/02-046N.pdf). FSIS‘ use of baseline studies to
determine performance standards was also described in a May 6
th
, 2002 FSIS Symposium entitled ―Symposium on Pathogen Reduction: A
Scientific Dialogue. Transcripts from this symposium can be found at: http://www.docstoc.com/docs/23091585/Pathogen-Reduction-A-
Scientific-Dialogue---May-5-2002.
13
11
and poultry products; and (4) intervention strategies aimed at reducing fecal contamination and other
sources of Salmonella on raw product should also be effective against other pathogens.
FSIS continues to conduct baseline studies to adjust the pathogen reduction performance standards. It is
important to note that the pathogen reduction performance standards are not lot-release standards and the
detection of Salmonella in a specific lot of raw product does not by itself result in the condemnation of
that lot. The pathogen reduction performance standard policy was based on the public health judgment
that reducing the percentage of carcasses with Salmonella would reduce the risk of foodborne illness to
the public. The policy was also based on the regulatory policy judgment that establishing a clear
standard for Salmonella, in conjunction with the implementation of PR/HACCP, would lead to
significant reductions in contamination rates. At the time that the pathogen reduction performance
standards were developed, there had not been a quantitative assessment of the public health impact
associated with implementation.
FSIS created a verification testing project—HC01—to ensure that the Salmonella pathogen reduction
performance standards are being met. Testing is conducted in sample sets, as described in the sampling
methodology section below. The design of the pathogen reduction sample sets for raw classes of
product was explicitly predicated on daily testing. When FSIS published the PR/HACCP rule, the
Agency proposed requiring that establishments, not FSIS, conduct daily testing and to complete at least
one sample set within a 12-month period. These design features were discussed at length at multiple
public meetings, including a technical meeting in Philadelphia in which FSIS invited subject matter
experts to discuss approaches to verification testing.
12
Numerous comments were received on the
proposed rule specific to these design features. In response to comments, FSIS elected to conduct the
daily testing for Salmonella to allow the Agency to have a direct measure of accountability for
individual establishment control in reducing harmful bacteria in raw meat and poultry. In the
implementation of the FSIS verification testing program, the Agency maintained the consecutive day
sampling and 12-month completion features. FSIS designed the verification testing program in a
manner to force establishments to test process control variables during a 90-day initial validation period
and to maintain process control that resulted in consistently wholesome and safe product, even if
changes occur in source materials or processing conditions. Consecutive day testing, with one sample
collected per day, was deemed to be an appropriate timeframe to cause establishments to assess potential
variability in their pathogen control program and to provide sufficient time to adjust their control
program to attain compliance when the production process trended out of control early in the sample set.
Daily testing for more than 50 consecutive days of production was recognized as a means for
establishments to cycle through numerous source material suppliers and better reflect operational control
capability over time, rather than collecting multiple samples a day, possibly all from the same source
material supplier, and not demonstrate sustained control of the production process.
Under the Salmonella verification program as conducted from 1996 to 2006, the Agency verified that
establishments were meeting a Salmonella standard or guideline by having FSIS inspection program
personnel (IPP) collect randomly-selected product samples from randomly-selected individual
establishments over the course of a defined number of sequential days of production to complete a
sample set. Generally, these tests were conducted once each year for each establishment. Procedures
for testing are described in Appendix E of the PR/HACCP Final Rule (61 FR 38917–38928). Although
the original enforcement strategy for the Salmonella reduction performance standards was designed to
12
Please see the following website for more information: http://www.gpo.gov/fdsys/pkg/FR-1998-10-07/pdf/98-26543.pdf
14
cause inspection to be withdrawn from an establishment after three consecutive sample set failures, a
court challenge (known as the Supreme Beef case) now prevents FSIS from enforcing this provision in
raw beef grinding operations, particularly when the beef is derived from carcasses that passed the
Agency‘s pathogen reduction performance standard. As a result, FSIS modified its enforcement strategy
for all classes of raw products and now uses the Salmonella set result as one piece of information in
determining the adequacy of an establishment‘s food safety system.
13
In order to address issues of resource allocation and time management, as well as provide U.S. States
with ―equal to‖ meat or poultry inspection programs, in 1997 FSIS crafted shorter Salmonella sample
sets than were expected to be used by the Agency for State inspection programs. FSIS recognized that
―equal to‖ State establishments typically would be classified as HACCP establishment size ―very small‖
and that these establishments typically produce more than one class of product subject to Salmonella
verification testing on the same day. In addition, these establishments typically processed a given class
of product intermittently (e.g., weekly or seasonally). Because the minimum number of days for any
sample set size was 51 days, with most sets of less than 60 days and only one set greater than 80 days, a
sample set likely would not be completed within a 12-month period. Consequently, FSIS crafted shorter
sample sets that were designed to achieve similar statistical confidence regarding the establishment‘s
control of Salmonella.
14
Sample collection for States was updated in FY2011 to reflect the Salmonella
and Campylobacter performance standards.
FSIS routinely reviews the Salmonella performance standards to identify gaps in the current policy and
to tailor the standards to better protect the public‘s health. As a result, FSIS has made several changes
over the years to refine the standards. One such change occurred in February 2006, when FSIS
introduced a categorization system for Salmonella set results to address adverse trends, whereby
establishments with consecutive sets with less than or equal to half of the current acceptable number of
Salmonella positive test results in the sample set would be identified as being in Category 1.
15
Establishments with half or more, but not exceeding the acceptable number of Salmonella positive test
results in a set, are placed in Category 2. Establishments exceeding the acceptable number of
Salmonella positive test results in a set are placed in Category 3 and consequently fail the Salmonella
pathogen reduction performance standard. FSIS also posted quarterly Salmonella sample set results on
the Agency‘s website and provided results back to establishments immediately upon completion of each
test, rather than waiting until the end of the sample set.
By January 2008, FSIS was using the new sample scheduling algorithm adopted in 2006 and scheduling
approximately 75 new verification sample sets for Salmonella in raw classes of product each month.
FSIS allocates its sampling resources within classes of raw product based on consideration of specified
16,17
criteria, as provided in the Salmonella HC01 description below. As a result of allocating sampling
resources in a targeted manner, FSIS is able to fulfill many of the higher priority criteria, such as
sampling establishments of greater concern. Now, Category 1 establishments are sampled in a period of
time that may extend up to two years, whereas Category 2 establishments are scheduled at least once
13
Please see the following website for more information: http://www.fsis.usda.gov/Fact_Sheets/FSIS_Sets_New_Procedures/index.asp.
14
Please see the Salmonella (Connie Bacon) Sampling Letter for more details and sample collection requirements.
15
Please see the following website for more information: http://www.fsis.usda.gov/OPPDE/rdad/FRPubs/04-026N.pdf.
16
Please see the following website for more information: http://www.fsis.usda.gov/OPPDE/rdad/FRPubs/2006-0034.pdf.
17
Please see the following website for more information:
http://www.fsis.usda.gov/Science/Scheduling_Criteria_Salmonella_Sets/index.asp.
15
each year until the establishment‘s category changes. Category 3 establishments are scheduled as close
to continuously as possible, until the establishment produces better results.
In May 2006, FSIS implemented a program to obtain serotype data from Salmonella PR/HACCP project
isolates and share of serotype data with establishments in a timely manner. After FSIS laboratories
report the analysis results, isolates of Salmonella-positive HACCP samples are serotyped at the USDA
Animal and Plant Health Inspection Service‘s (APHIS) National Veterinary Services Laboratories
(NVSL) in Ames, Iowa. In recent years, virtually all samples positive for Salmonella have been
serotyped. Identified serotypes are communicated to establishments as soon as they are reported by
APHIS to FSIS, usually within two weeks after a HACCP sample has been reported as positive. A
report listing aggregate identified serotypes by year was posted in August 2007 on the FSIS website at
http:// www.fsis.usda.gov/Science/ Serotypes_Profile_Salmonella_Isolates/ index.asp.
Additionally, in August 2007, when FSIS and the Agricultural Research Service (ARS) signed a
cooperative Memorandum of Agreement for subtype data sharing, FSIS implemented a program to
obtain timely access to pulsed field gel electrophoresis (PFGE) subtype data, identifying specific strains
of Salmonella serotypes obtained from HACCP testing. Under this agreement, PFGE subtype
information on Salmonella isolates collected by FSIS from raw meat and poultry products is matched
with subtype information from isolates associated with human illness in PulseNet, a database maintained
by the Centers for Disease Control and Prevention (CDC). FSIS has routine access to subtype data for
all isolates maintained by ARS, in a time frame short enough to be relevant to in-establishment and
public health investigations.
18
In January 2008, FSIS chose to exclude from the Salmonella verification testing project schedule any
slaughter establishment that processes all slaughtered carcass into RTE (e.g., cooked) product or diverts
all of its raw products to another federally-inspected establishment for further processing into a RTE
product. This decision is justified because FSIS conducts separate verification testing for Salmonella in
RTE meat and poultry products via the ALLRTE and RTE001 sampling projects. If the establishment is
undergoing sampling for Salmonella, but then elects to send all affected product to RTE, FSIS will
continue to sample until the set is completed. At the end of the set, FSIS will verify that all products
undergo further processing into RTE product within the establishment or in another Federal
establishment. If and when such establishments again produce raw product for sale, they will be re-
scheduled for Salmonella verification sets.
Also at that time, FSIS announced in a Federal Register Notice additional activities for low-volume
ground beef operations, minor species and import samples, which have since been implemented by the
Agency.
19
FSIS recognized that low-volume raw ground beef producers, which produce less than 1000
pounds of product per day, constitute a large part of the sampling frame for establishments eligible for
verification sample set scheduling, though they account for a very small proportion of the raw ground
beef supply. Since production of ground beef at these establishments may not occur throughout a week
or month, sampling them for Salmonella may be extended for a year or more before a set is completed,
as opposed to no more than a couple of months of sampling for higher volume establishments. To
18
FSIS currently uses PFGE and Multiple Locus VNTR Analysis Method (MLVA) for subtyping pathogens. PFGE is considered the ‗gold
standard‘, and MLVA may help further differentiate subtypes to assist in making connections between isolates from case patients and
FSIS-regulated products.
19
Please see the following website for more information: http://www.fsis.usda.gov/OPPDE/rdad/FRPubs/2006-0034.pdf.
16
increase efficiency, FSIS announced that samples collected at these establishments for E. coli O157:H7
testing would be tested for Salmonella as well. As a result, these establishments were removed from the
Salmonella verification testing sample set scheduling frame. The FSIS laboratories now perform
qualitative testing for the presence or absence of Salmonella using the same methodology, discard
criteria and reporting as those in place for Salmonella ground beef HACCP samples, as described below.
A description of this sampling project, known as MT43S, is provided below.
Most recently, on May 14, 2010, FSIS published a Federal Register Notice (75 FR 27288) in which it
announced the implementation on July 1, 2011 of new performance standards for Salmonella and
Campylobacter for chilled carcasses in young chicken (broiler) and turkey slaughter establishments.
The new performance standards were developed in response to a charge from President Obama‘s Food
Safety Working Group (FSWG) and based on recent FSIS Nationwide Microbiological Baseline Data
Collection Programs. The standards are applied to sample sets collected and analyzed by FSIS to
evaluate establishment performance with respect to requirements of the PR/HACCP Rule. These
performance standards are the basis for assessing an establishment‘s process control for Salmonella and
Campylobacter and for determining whether an establishment passes or fails a Salmonella or
Campylobacter verification set. All sample sets scheduled for young chicken and turkey establishments
are analyzed for both Salmonella and Campylobacter and follow-up sample sets responding to sample
set failures for either organism are analyzed for both organisms.
In addition to process control, FSIS identifies the serotype, PFGE and antimicrobial susceptibility
pattern of Salmonella isolates from each positive verification sample. FSIS uses the subtyping results to
identify historical trends within the sampling data to determine whether an isolate has a historical
association with human illness, and to identify clusters of patterns. Since FSIS has not established a
regulatory performance standard for Salmonella subtypes, this information is not used to determine the
status of a Salmonella verification set, including whether the establishment has passed or failed the set.
Effective with samples sets starting in or after July 2011, Salmonella performance Categories 1 and 2,
based on the new performance standards, are applied exclusively for FSIS internal analysis and not for
web-posting purposes. FSIS posts quarterly aggregate reports showing the Category 1/2/3 distribution
for each relevant product class subject to FSIS Salmonella testing, but does identify individual
establishments.
Intent of Program
FSIS considers Salmonella verification testing to be a direct indicator of the effectiveness of process
control. The raw pathogen performance standards program also serves a variety of other functions,
including assessing establishment compliance with the performance standard, comparing industry-wide
and peer-to-peer trends regarding percent positives over time, identifying serogroups of public health
concern and their origins, as well as capturing pathogen isolates for PulseNet comparison and analyzing
chilled product at specific points in the production process.
Type of Analysis
For each pathogen, FSIS performs different types of analysis on the sample collected. There are two
possible types of analysis that the FSIS laboratories can conduct. First, FSIS laboratories can determine
if the pathogen is present or absent in the sample. This is considered to be qualitative analysis. Second,
FSIS laboratories can determine not only whether the pathogen is present, but also what the level of
contamination is or the number of microorganisms present in the sample. This is considered to be
17
quantitative analysis. FSIS uses several types of quantitative analyses, including direct plating
enumeration methods and the Most Probable Number (MPN) enumeration procedure, which is used to
estimate the population density of viable microorganisms in a test sample.
For Salmonella, analysis of samples for the HC01 sampling project are qualitative, in that samples are
tested for the presence or absence of Salmonella, rather than a count of the number of organisms in the
sample. For the MT43S testing project for Salmonella in raw ground beef, FSIS also uses the MPN
enumeration procedure to estimate the level of contamination in samples that first test positive on a
qualitative screening test.
Volume Data
Yearly production volumes for raw intact beef, chicken, turkey and pork are acquired from slaughter
volume data in the Electronic Animal Disposition Reporting System (eADRS) database. Volumes for
ground beef production are obtained by FSIS inspectors during sampling. The inspector classifies the
establishment‘s volume into one of four volume groups and notes monthly days of production, from
which an average annual volume can be estimated. Yearly production volumes for ground chicken and
turkey are obtained from the PBIS, which contains each establishment‘s most recent production volume
for various product types over the past 30 days. Each product class has a minimum slaughter volume,
under which an establishment is excluded from sampling eligibility. For example, young chicken and
turkey sampling eligibility requires a minimum of 20,000 birds slaughtered per year. For livestock, a
minimum of 500 animals slaughtered per year is required.
Sampling Methodology
FSIS schedules eight product classes (young chickens, young turkeys, cows/bulls, steers/heifers, market
hogs, ground chicken, ground turkey and ground beef) for Salmonella sampling on monthly basis—see
Table 2.1.1.2.
20
An establishment's compliance with the applicable pathogen reduction performance
standard is evaluated by taking the indicated number of samples—generally at the rate of one or more
per day— and testing each sample for Salmonella to determine whether the number of positive results is
above the maximum acceptable for that product class. Daily testing is considered the minimal sampling
necessary to detect process deviations within a realistic time frame. FSIS currently collects multiple
samples from each establishment in the form of sets. FSIS verifies that establishments are meeting the
Salmonella standards by having IPP collect product samples from individual establishments over the
course of a defined number of sequential days of production to complete a sample set, as described
previously. Once a sample set begins, sampling is conducted daily until the set is completed.
Depending on the frequency of production, product type and availability of resources, the time for FSIS
to complete a sample set ranges from two months to over a year. Salmonella sets are scheduled using a
―risk-based‖ approach, where establishments with a higher rate of Salmonella are scheduled more
frequently than establishments with lower Salmonella rates. In establishments that produce more than
one product subject to Salmonella verification testing, only one product is tested at a time. Annual
reports summarizing results for calendar years are available on the FSIS web site.
21
Raw products with
established performance standards are carcasses of cows/bulls, steers/heifers, market hogs, broilers and
turkeys.
22
Processed products with performance standards are raw ground beef, ground chicken and
20
Supra note 6
21
Please see the following website for more information: http://www.fsis.usda.gov/Science/Microbiology/index.asp.
22
Supra note 6.
18
ground turkey. The performance standards are currently based on the prevalence of Salmonella as
determined from FSIS‘ nationwide microbiological baseline studies.
Table 2.1.1.2: Salmonella Performance Standard Set Definitions by Product Class
Product class
Performance
standard
Number
of
samples
tested
Sampling Method
Testing
Method
Maximum
number of
positives to
achieve
standard
Steers/heifers
1.0%
82
Flank, brisket, rump
surface sampling- 100
cm
2
for each using one
cellulose sponge
hydrated with BPW
23
MLG 4.x-
enrich sponge
and BPW
diluent
1
Cows/bulls
2.7%
58
Flank, brisket, and rump
surface sampling- 100
cm
2
for each using one
cellulose sponge
hydrated with BPW
MLG 4.x-
enrich sponge
and BPW
diluent
2
Ground beef
7.5%
53
One sample per event
MLG 4.x-25
gram test
portion
5
Market hogs
8.7%
55
Ham, belly, and jowl
surface sampling- 100
cm
2
for each using one
cellulose sponge
hydrated with BPW
MLG 4.x-
enrich sponge
and BPW
diluent
6
Broilers
7.5%
51
400 ml BPW rinsate
MLG 4.x- 30
ml test portion
5
Ground
chicken
44.6%
53
One sample per event
MLG 4.x-25
gram test
portion
26
Ground turkey
49.9%
53
One sample per event
MLG 4.x-25
gram test
portion
29
Turkeys
1.7%
56
Back and thigh surface
sampling- 50 cm
2
for
each using one cellulose
sponge hydrated with
BPW
MLG 4.x-
enrich sponge
and BPW
diluent
4
Constructing the Sampling Frame
The Salmonella HC01/HC11 project includes eight different product classes for sampling: cow/bull,
steer/heifer, market hog, broilers, turkeys, ground beef, ground chicken and ground turkey.
24
Eligibility
requirements for the intact raw products differ from the intact ground products. This multiple step
process is described below.
23
BPW is Buffered Peptone Water
24
Supra note 6.
19
I. Establishments Producing Eligible Product
The first step in this process is to create a list of all establishments that produce sufficient volumes of
eligible products.
1) Intact Raw Products
Using the eADRS, the total number of each class of eligible intact product that has been
slaughtered at FSIS regulated establishments in the last 12 months is collected. Establishments
that meet the minimum production volume requirements are kept in the sampling frame. The
minimum requirement is as follows for each product class:
A. Cow/Bull: minimum of 500 heads/year slaughtered
B. Steer/Heifer: minimum of 500 heads/year slaughtered
C. Market Hog: minimum of 500 heads/year slaughtered
D. Broilers: minimum of 20,000 heads/year slaughtered
E. Turkeys: minimum of 20,000 heads/year slaughtered
2) Raw Ground Beef
Establishments that had samples successfully collected and analyzed under the MT43 sampling
project (E. coli O157:H7 in raw ground beef) in the last 12 months are eligible for Salmonella
Verification Sampling. Inspectors report the daily raw ground beef production volume every
time a MT43 sample is collected, so that the mode (most frequent) response over the last 12
months can be used to determine whether an establishment meets minimum production volume
requirements.
25
A. Raw Ground Beef: minimum of 1000 pounds/day produced
26
3) Raw Ground Poultry (Chicken and Turkey)
Salmonella Verification Sampling Program eligibility for raw ground poultry is currently limited
to establishments included in the ground chicken and ground turkey baseline studies conducted
in the 1990‘s.
27
This is because currently FSIS neither collects detailed production volume data
for ground poultry, nor does the sampling program include establishments that produce ―raw
comminuted poultry‖ products.
28
II. Active Establishments
The second step is to include only establishments that are currently listed as active in their establishment
profiles. Establishments that are shut down or withdrawn from inspection are removed from eligibility,
as well as establishments that are currently inactive for any reason, such as seasonal producers and
temporary closure.
III. Exclusions
Next, establishments that meet certain exclusionary criteria are removed from the sampling frame.
These criteria include the following:
25
With the implementation of PHIS, production volume data will be available in establishment profiles, which may affect how FSIS
determines establishments that meet production volume requirements for raw ground product (i.e. MT43 response no longer necessary).
26
Establishments producing less than 1,000 pounds per day are tested for Salmonella under the MT43S (low volume) sampling program.
27
Please see the following website for more information: http://www.fsis.usda.gov/Science/Baseline_Data/index.asp.
28
FSIS Notice to expand Salmonella Verification testing to include other raw comminuted poultry products is currently in development.
This will help in increasing the number of establishments eligible for ground poultry sampling.
20
1) At Establishment Level
A. Establishments that are currently in an ongoing set for any product are removed from the
sampling frame.
29
B. Establishments that completed and passed a set within 30 days of the date the sets are
scheduled to start collection are removed from the sampling frame. This is also known as
the Category 1 exemption, where establishments are excluded from sampling for up to
two years.
2) At Establishment Product Level
A. Occasional producers of products are identified by responses to prior sampling requests.
Establishments for which IPP returned a Salmonella verification sampling request form
with a code 72 response (product not produced in last 30 days) within 60 days of the date
the sets are being scheduled to start collection are removed from the sampling frame for
that product.
B. Establishments that perform only exempt slaughter (custom or religious slaughter) are
identified by responses to prior sampling requests.
30
Establishments for which IPP
returned a Salmonella verification sampling request form with a code 60 response
(product no longer produced) within 12 months of the date the sets are being scheduled to
start collection are removed from the sampling frame for that product.
Exclusion Criteria
FSIS maintains a number of additional exclusion criteria in its HC01 sampling methodology not
mentioned in the exclusion section list provided above. FSIS maintains exclusion criteria for low-
volume establishments. For raw ground beef, establishments that produce less than 1,000 pounds per
day are excluded from Salmonella verification sampling, although these establishments are sampled
using the MT43S program. Please see MT43S sampling section for more information on this program.
Establishments which slaughter less than 20,000 birds and less than 500 animal head per year are also
excluded from Salmonella set-based sampling. Finally, FSIS maintains product class exclusions. All
production classes other than young chickens, young turkeys, ground chicken, ground turkey,
cows/bulls, steer/heifers and market hog are not eligible for Salmonella verification sampling. For
example, sheep and lambs fall under this exclusion.
FSIS Scheduling Criteria and Algorithm for the Salmonella HC01/HC11 for Raw Meat and
Poultry
31
Objective:
FSIS schedules up to 75 new sample sets each month for raw meat and poultry.
32
The establishments
and products selected for sample sets are chosen according to a risk-based algorithm that involves
sorting the list of eligible establishments and their respective products by certain criteria and selecting
33 34
the top 75 from this list. These priority groups are sorted in descending order of importance.
29
If an establishment products multiple products, these products are scheduled independently; however, it will not be scheduled for more
than one set at a time.
30
Establishments that perform both exempt and non-exempt slaughter are eligible for Verification Sampling for the non-exempt products.
31
Includes Campylobacter testing for broiler and turkey carcass sets begun after July 1, 2011.
32
See Federal Register Notice of July 25, 1996 regarding the HACCP Systems Final Rule;
http://www.fsis.usda.gov/OPPDE/rdad/FRPubs/93-016F.pdf
33
See ―SOP to Identify Establishments Eligible for Inclusion in Salmonella and Campylobacter Verification Sampling Program‖ on FSIS
website at http://www.fsis.usda.gov/Science/Microbiology/index.asp
21
Criteria:
I. Establishment Category or Establishment Status
35
Establishments are initially sorted by category or status:
1) Category 3 Establishments (includes establishments with one completed set that was
failing)
2) New Establishments and existing establishments with newly eligible product
36
3) Establishments with one completed set that was passing
37
4) Category 2 Establishments
5) Category 2T Establishments
6) Category 1 Establishments (≥660 days since last set)
7) Category 1 Establishments (365 to 659 days since last set)
38
II. Product
Within each status category, establishments are then sorted based on product priority:
1) Broilers
39
2) Young Turkeys
40
3) Market Hogs
41
4) Ground Chicken
42
5) Ground Turkey
43
6) Ground Beef
44
7) Cows/Bulls
45
8) Steers/Heifers
46
III. Most Recent Set Result
Within each product class, priority is assigned based on the result of the most recent Salmonella set.
1) Failed; Exceeded Performance Standard
2) Passed; > 50% Performance Standard
3) Passed; ≤ 50% Performance Standard
34
This algorithm is subject to periodic intra-program review and adjustment; during natural disasters (e.g., hurricane), Category 2
establishments not currently scheduled may be scheduled.
35
See FSIS method to categorize Salmonella establishments published in Federal Register Notice of January 28, 2008 (73 FR 4767-4774;
http://www.fsis.usda.gov/OPPDE/rdad/FRPubs/2006-0034.htm).
36
Includes eligible turkey slaughter establishments (carcasses) as of May 2006, and any new establishment operating for at least 90 days
(to accommodate 9 CFR 304.3).
37
These establishments, technically category 2 or 2T, are placed ahead of other category 2 and category 2T establishments.
38
Category 1 establishments are not routinely scheduled for sampling until at least 12 months after their last set.
39
Broiler and young turkey establishments are considered new establishments until 2 sets have been completed under the
new Salmonella/Campylobacter performance standards, and will be targeted for expedited sampling. If an establishment fails
EITHER Salmonella OR Campylobacter it will be treated as a failed set (for the purposes of scheduling only) and be
scheduled for an immediate follow-up set the next month.
40
Supra note 39.
41
Supra note 6.
42
If establishment is combination carcass/grinding operation, these products are scheduled independently (these
establishments will have a category for each product); carcass and ground product sets will not be scheduled concurrently.
43
Supra note 42.
44
Supra note 42.
45
Supra note 6.
46
Supra note 6.
22
IV. Human Health Serotypes Linked to Product Class
Within each group of recent set results, the next priority is given to the number of human health serotype
isolates identified in samples from an establishment‘s last set.
47
V. Days Since Last Completed Set
Final priority is given to the number of days since each establishment‘s last completed set.
Additional Requirements:
As monthly availability permits, additional establishments may also be scheduled for Salmonella sample
sets by request from FSIS.
As of July 1, 2011, the new Salmonella performance standards for broilers and young turkeys became
effective. For the purposes of scheduling only, Salmonella categorization of establishments for these
two market classes will be determined as if the standards had been in place at the time an
establishment‘s most recent two Salmonella verification sets were performed, including sets performed
prior to July 1, 2011.
48
Also, broiler and turkey Salmonella sets scheduled after July 1, 2011 are being
co-analyzed for Campylobacter. All broiler and turkey establishments will maintain their Salmonella
category status, but will be marked as having ―newly eligible product‖ or ―one completed set‖ until two
sets are completed under the new testing program.
49
Establishment categories for these poultry establishments are based on the Salmonella set results.
However, if a establishment fails a set for either Salmonella or Campylobacter, it will be treated as a
failed set (for the purposes of scheduling only) and be scheduled for immediate follow-up set the next
month.
In the initial phase of implementation with regard to FSIS laboratory sampling capacity, a maximum of
16 broiler and turkey sets can be scheduled each month. Currently, the goal is to schedule the top 12
broiler establishments and the top four turkey establishments from the sorted list of eligible
establishments. These target numbers are subject to quarterly review and adjustment by FSIS. The
remainder of the 75 sample sets will come from other products. A discussion of FSIS‘ new sampling
project for Campylobacter is included in Section 2.1.2.
Statistical or Policy Basis for Current Sampling Plan
The procedure for determining a compliance criterion to evaluate an individual establishment‘s
performance with respect to the standard is based on an approximate 80% probability of passing the
criterion, when the establishment‘s prevalence is equal to the performance standard percentage. Stated
differently, the type 1 error rate of asserting (incorrectly) that the establishment‘s prevalence is greater
than the performance standard is about 0.2 (20% probability of failing) when the establishment is
performing at the performance standard. For a given performance standard equal to a proportion; p; the
compliance criterion should be no more than m positive results in n analyzed samples comprising a
sample set. Thus, m is selected such that P(m) – the probability of a random, binomially distributed
variable with number parameter, n, and proportion parameter, p is less than or equal to m – is closest to
47
Number calculated using top 20 Human Health Serotypes for most recent calendar year as reported by the CDC.
48
See FSIS Notice 31-11 (dated 6/30/11) for information on how actual individual and aggregate set results and establishment
categorizations will be reported.
49
Once 90% of broiler and turkey establishments have 2 completed sets under the new standard, FSIS will begin putting them into
categories for Campylobacter as well as Salmonella.
23
0.8 compared to any other value of P(m′) for m′ not equal to m. The other constraint is that n is a
minimum of 50 samples.
Output from Sampling Results
End of Set Letters
At the completion of each Salmonella set, FSIS sends an ―end-of-set (EOS) letter‖ to the sampled
establishment explaining the establishment‘s status based on the overall set results. Each letter lists
specific set factors: the number of Salmonella serotypes associated with human illness (high, average or
low for the product class tested) and the timeframe for when the next sample set will begin at that
establishment. With the new FSIS performance standards, the EOS Letter will report the
establishment‘s Salmonella Category status based on the standard in effect when each reported set was
started. FSIS is working with the CDC and the ARS to establish mechanisms to routinely share and
compare subtyping information. As a result, when reporting sampling results, FSIS will include
information on subtypes found in the sampling that are associated with human illness when that
information becomes available. In the interim, the EOS letter will now include information regarding
not only positive or negative test results, but also detailed serotype information for all positive
Salmonella results. In addition, for young chicken and young turkey sets, the EOS letter will include
Campylobacter results.
Volume-Weighted Percent Positive
From these sampling results, the volume weighted percent positive is calculated. This method gives
weight to the establishment-level percent positive estimates using the volume of each product type that
is produced at the sampled establishments, which is necessary to estimate the amount of contaminated
product. Thus, samples testing positive for Salmonella from establishments that generate higher
volumes of product have greater influence on the final statistic because the public health risk increases
in proportion to the production volume.
Limitations of Current Sampling
1. Risk-based Sampling
The current scheduling algorithm is risk-based, which is critical in positively affecting public health,
but disproportionately focuses sample collection. This means that there is a large difference between
well-performing establishments (Category 1) and poor, or potentially poor, ones (Category 3), in that
the former might not be scheduled for sampling for a year or more, whereas the latter could be
scheduled quite often. For this reason, not all establishments in the collection frame have a non-zero
probability of selection each month.
2. Product Priority
Establishments producing certain products are scheduled ahead of others. This prevents those
establishments/products with lower priority from being sampled regularly because only a given
number of sample sets can be scheduled each month. This results in data that are not representative
of certain product classes. Furthermore, some product classes that have been completely excluded
during certain months would not have a probability of selection for sampling for that period.
3. Announced Sampling
Once a sample set begins, an establishment is aware that it will be sampled every day the product is
being produced over the next few months (or longer for smaller establishments that produce less
frequently), until the set is completed. This knowledge might create a bias because establishments
may, intentionally or not, be more conscientious in adhering to proper sanitary procedures during
24
this time. This could result in an abnormally low number of positive Salmonella results than would
occur otherwise.
4. Sample Sets
Salmonella samples are scheduled in sets, which results in a high degree of clustering. That is,
establishments are sampled intensively, but then not at all for a period of time. Moreover, this is
problematic from a process control perspective because historical data from well-performing
(Category 1) establishments does not exist. Thus, it is unknown whether these establishments are
consistently maintaining low levels of Salmonella, or if their good performance was a temporary
result of announced sampling.
5. Production Volume
The major difference between the sampling data for intact and ground products is that volume
information is not available at the establishment level for ground chicken or ground turkey and only
a rough estimate can be determined for ground beef.
6. Exclusion Criteria
As discussed above, FSIS maintains a number of different exclusion periods. For example,
establishments that reach Category 1 status are not scheduled for another Salmonella set for up to
two years. Category 2 status establishments are not scheduled for up to 100 days. These exclusion
periods mean that establishments do not have a consistent probability of selection across all time
periods. FSIS also maintains exclusion criteria for low-volume establishments, though these criteria
also apply to FSIS baseline studies. Excluding these establishments prevents the sampling project
from representing all production under FSIS jurisdiction, but allocates resources for logistical
reasons.
7. Regulatory Restrictions
FSIS has published the Salmonella Performance Standards in the Federal Register. Therefore, changes
to the current project require policy changes.
8. Seasonal Fluctuations
Many types of pathogenic microorganisms exhibit seasonal patterns, but FSIS verification sampling
programs currently make no allowances for season fluctuations.
MT43S Sampling Project
Historical Basis
In 2008, FSIS established the MT43S sampling project so that sample collection at very low volume
establishments (producing less than 1,000 pounds of product per day) would not be overly burdensome
to the establishments. These establishments were already receiving regular raw ground beef sample
requests under the E. coli O157:H7 sampling program. FSIS therefore decided to perform an additional
Salmonella analysis under MT43S on the same sample. In this way, FSIS can effectively test for two
sampling projects, without overly burdening IPP and the establishment with additional sample
collection.
Type of Analysis
Samples that confirm positive are quantitatively analyzed. For example, the Salmonella organisms
present are enumerated using the MPN procedure.
50
50
MPNs are only done if there is enough product left over for that analysis. Also, E. coli O157 MPN analysis needs trump those of
Salmonella if there is limited product available and the sample confirms for both E. coli O157:H7 and Salmonella.
25
Volume Data
Ground beef production volumes under the MT43S sampling program are obtained by FSIS inspectors
51, 52
during the sampling process. The IPP classifies the establishment‘s volume into one of four volume
groups, from which average annual ground beef volumes are estimated. The volume classes are as
follows;
1) > 250,000 pounds per day,
2) 50,001 – 250,000 pounds per day,
3) 1,000 – 50,000 pounds per day, and
4) < 1,000 pounds per day.
Sampling Methodology
FSIS does not schedule establishments sampled under MT43S in the regular Salmonella verification
testing project (9 CFR 310.25(b)). Rather, these establishments are selected under the MT43 E. coli
O157:H7 sampling project.
53
The MT43 monthly sample is randomly selected with replacement and
weighted by production volume and historical test results from the frame of eligible establishments. In
addition, the MT43 project limits very low volume establishments to no more than one sample per
month (sampling ceiling) and each establishment must be selected at least three times per year (sampling
floor). The very low volume establishments selected every month under MT43 are also selected in the
same month for MT43S. These establishments produce less than 1,000 pounds of raw ground beef on
an average production day. The Salmonella results are recorded as part of the MT43 project.
Limitations of Current Sampling
The MT43 sampling algorithm has both ceilings and floors, which impedes the ability of the weights to
perform optimally. Additionally, prior notification is required, so establishments can prepare to hold
product, as recommended by FSIS, pending Agency test results.
Processed Egg Products—EM 31-37
Historical Basis
FSIS carries out its food safety responsibilities with respect to processed egg products under the
provisions of the Egg Products Inspection Act (EPIA) (21 U.S.C. 1031-1056). Section 1036(a) of the
Act requires egg products inspected at any official egg products establishment and found not to be
adulterated and must be pasteurized before they leave the official establishment. Section 1043 delegates
to the USDA Secretary of Agriculture the authority to promulgate such rules and regulations deemed
necessary to carry out the purposes or provisions of the EPIA. This authority was delegated to the
USDA‘s Under Secretary of Agriculture for Food Safety in October 1994. Under the provisions of 9
CFR 590.580, ―Laboratory tests and analyses‖, FSIS operates a directed microbiological sampling
program to verify official establishment control of Salmonella; the pathogen of concern when the EPIA
was originally enacted in 1970. In addition, the processed egg products industry has to conduct lot
acceptability testing. Establishment or private contract laboratories who analyze egg product official
54 55
surveillance samples are approved and audited under the PEPRLab program administered by FSIS.
51
Volume groups were developed by a multi-disciplinary team of scientists and technical staff within the FSIS prior to 2003.
52
With the implementation of PHIS, volume information will be obtained from the establishment profile data. For this reason, questions
referring to product volume were taken off of all forms collected through PHIS.
53
Further information on the MT43 sampling project can be found in this document under Section 2.1.4 E. coli O157:H7.
54
Pasteurized Egg Product Recognized Laboratory Program (PEPRLab) at
http://www.fsis.usda.gov/Science/PEPRLab_Program/index.asp.
55
Program is administered by the Laboratory Quality Assurance Division (LQAD), Office of Public Health Science, FSIS
26
Processed egg product establishments are not currently operating under HACCP and only limited
information about establishments is collected in PBIS. Further, compliance/non-compliance data is
documented manually. In addition, processed egg product establishments are not included in the current,
phased approach to PHIS implementation for meat and poultry establishments. However, FSIS processed
egg product inspection is expected to be incorporated in the future into PHIS and implementation of
HACCP will be considered after full implementation of PHIS for meat and poultry establishments is
complete.
Volume Data
Production volume data for processed egg products is collected from establishments producing these
products and submitted to FSIS on a monthly basis. Volume data on each of the seven processed egg
products categories are not currently collected. Instead, summary data on the volume of whole eggs
broken or received for further processing is provided by establishments. These data are further
subdivided into categories of whole egg, yolks and whites. Egg processing establishments also provide
volume information on the total number of ingredients added to processed egg products, the volume of
liquid, frozen and dried egg products distributed in commerce and the volume of product sent to other
establishments for further processing. Using this information, the total volume of processed egg
products produced by each establishment can be determined. This can be further broken down into the
volume of liquid, frozen and dried processed egg products produced. Using this data, establishments are
placed into four volume groups, based on their annual production.
56
Current Design of Sampling Project
There are four liquid and three dried processed egg product process categories in the domestic processed
egg products sampling program—see Table 2.1.1.3. Each month, one processed egg products sample
per process is collected from each establishment that produces processed eggs products.
Table 2.1.1.3: FSIS Processed Egg Products Classes
EM-31
egg whites with or without added ingredients
EM-32
whole eggs/yolks with <2% added ingredients other than salt or sugar
EM-33
whole eggs/yolks with ≥2% added ingredients other than salt or sugar
EM-34
whole eggs/yolks with ≥2% salt or sugar added
EM-35
dried yellow egg products
EM-36
spray dried egg whites (with or without added ingredients)
EM-37
pan dried egg whites
EGGDOM
Pasteurized domestic liquid, frozen or dried egg products
Frame Definition
The number of processed egg products establishments has stayed fairly consistent over the past 15 years,
generally ranging from 75 to 85 active establishments. When changes are identified in the types of
processed egg products that an official establishment is producing, an establishment withdraws or a new
establishment comes on board, FSIS District Office and FSIS field personnel notify FSIS Headquarters
staff of the changes so that the database that generates the FSIS sampling forms for egg products can be
updated. Once processed egg products establishments are included in PHIS, this will be done
automatically.
56
Group 1 (≤ 5,000,000 lbs.), Group 2 (5,000,001 - 45,000,000 lbs.), Group 3 (45,000,000 - 75,000,000 lbs.), Group 4 (> 75,000,000 lbs.)
27
Sampling Methodology
Each month, FSIS conducts a census by sending one processed egg product sample request per process
from each establishment that produces eggs products. Thus, an egg products processing establishment
could be selected for collection up to seven times per month, depending on the number of production
processes occurring during the month.
Collection Methodology
FSIS inspection personnel are directed to follow the instructions set forth in FSIS Directive 10,210.1,
Unified Sampling Form, dated December 18, 2003. The directive lists each sampling project by number
(EM31, EM32, EM33, EM34, EM35, EM36 and EM37). The directive also provides instruction to the
FSIS inspector on how to collect the sample, complete the form and ship the sample. Finally, the
directive provides instruction to IPP on notifying establishment management. FSIS has also developed
guidance for IPP on how the establishments should be sampled under FSIS Directive 10,230.4,
Microbial Sampling of Ready-to-Eat (RTE) Products for the FSIS Verification Testing Program for the
Salmonella sampling project.
Mean Response Rate
FSIS collected 60% of all processed egg products forms that were assigned and 58% of forms scheduled
by FSIS were analyzed.
Statistical or Policy Basis
Neither the sample size nor the sampling frequency for processed egg products is statistically based on a
national prevalence estimate, as calculated from a baseline study. Further, there is currently no policy
basis for the current sampling program. Rather, the processed egg products sampling program was
historically designed to make certain that enough samples were collected and analyzed to ensure a broad
understanding of contamination rates among the different types of processed egg products. Future
efforts to introduce a HACCP-based processed egg products program will likely mark the development
of a statistical analysis of the processed egg product data and therefore the development of a
statistically-based sampling program.
Limitations of Current Sampling
Processed egg products sampling frames are not part of PBIS and no instructions have been provided to
FSIS field personnel on how to update establishment profile data to incorporate processed egg products.
Additionally, no baseline studies have been conducted to inform the sampling methodology. Therefore,
statistics produced from the testing data are likely to be biased and have high levels of error. In
September 2011, FSIS did begin the shakedown period for the Agency‘s Nationwide Raw Liquid Egg
Microbiological Baseline Survey (RLEBS), though this baseline study will focus on pre-pasteurized egg
products.
57
Salmonella Programs Measure of Success
Each Salmonella sampling project has slightly different goals. However, the overall purpose of the
Salmonella verification testing program to ensure that the pathogen reduction performance standards are
being met, which ensures that the industry is maintaining process control.
57
http://www.fsis.usda.gov/OPPDE/rdad/FSISNotices/52-11.pdf
28
Consequently, FSIS believes it is appropriate to measure the success of the different sampling projects in
broad terms. Further, as FSIS recently implemented new performance standards for Salmonella,
58
adjustments to these measures may be appropriate moving forward.
As such, to measure the success of the Salmonella sampling projects, FSIS evaluates its efforts in terms
of five key metrics:
1. Volume-weighted percent positives for Salmonella in eight raw product classes;
2. Estimated number of Salmonella foodborne illnesses associated with FSIS-regulated products;
3. Establishment categorization based on new Salmonella performance standards;
4. FSIS End of Set Letters; and
5. Reductions in case rates (as reported by the CDC) for the top 10 serotypes of human health
concern.
Volume-Weighted Percent Positives
FSIS conducts pathogen verification testing for many of the Agency‘s sampling projects. FSIS uses this
sampling to calculate a percent positive for many of the Salmonella sampling projects. FSIS believes
that percent positives are a good measure of the effectiveness and success of the sampling project
maintained by the Agency, with declines in percent positives potentially indicating greater control of
Salmonella in raw product. Table 2.1.1.4 provides the production category volume-weighted percent
positive rate for Salmonella in all raw products and on broiler chickens alone.
Table 2.1.1.4: Quarterly Salmonella Volume-Weighted Percent Positive Rates for All Raw
Products and Broiler Carcasses Alone
Year/Quarter
Volume Weighted
Percent Positive Rate
(All Raw Products)
Volume Weighted
Percent Positive
Rate (Broilers)
FY 2009
2.74%
5.31%
FY10Q3
2.88%
5.29%
FY10Q4
2.62%
5.14%
FY11Q1
2.70%
5.22%
FY11Q2
2.58%
5.05%
FY11Q3
2.32%
5.11%
Foodborne Illness Estimates:
As FSIS‘ ultimate goal is to prevent foodborne illnesses from regulated products, it is important to
measure reductions in foodborne illness as a result of FSIS inspections, sampling, policies and other
activities. FSIS calculates a performance measure, known as the All-Illness Measure, which represents
all foodborne Salmonella, Lm and E. coli O157:H7 illnesses from FSIS-regulated meat, poultry and
processed egg product. FSIS updated the All Illness Measure in Q3, FY2011 to reflect the release of
59 60
new illness burden estimates from the CDC and the Healthy People 2020 goals , as well as to coincide
with the release of the FSIS Strategic Plan for 2011-2016. Objectives for the All-Illness measure were
58
Please see the following website for more information: http://www.fsis.usda.gov/OPPDE/rdad/FRPubs/2009-0029.pdf
59
Scallan E, Hoekstra RM, Angulo FJ, Tauxe RV, Widdowson M-A, Roy SL, et al. Foodborne illness acquired in the United States—
major pathogens. Emerg Infect Dis [serial on the Internet]. 2011 Jan [November 2011]. http://www.cdc.gov/EID/content/17/1/7.htm
60
Please see the following website for more information:
http://www.healthypeople.gov/2020/topicsobjectives2020/objectiveslist.aspx?topicId=14.
29
set using a combination of data from published CDC FoodNet case rates and outbreak data and are
aligned with Healthy People 2020 goals. For Salmonella, FSIS uses a rolling 12 month window of case
rate data from the CDC, in addition to an attribution estimate, to estimate the total number of Salmonella
illnesses from FSIS regulated products. Using this methodology, the illness measure is the estimate of
the total annual illnesses for the fiscal year, rather than independent measures of illness for each quarter.
Performance Measure
Using the newly updated All Illness Measure data sources and methodology, FSIS set a target of
399,852 Salmonella illnesses associated with FSIS regulated products for Q3, FY 2011; FSIS missed
that target with 472,859 estimated illnesses. However, there was an approximate 4,000 illness decline
from the second quarter to the third quarter of FY2011. Figure 2.1.1.1 illustrates the quarterly targets
for Salmonella illnesses and the estimated illnesses for the most recent four quarters.
Figure 2.1.1.1: Estimated Salmonella illnesses associated with FSIS regulated products
Establishment Categorization
As described in this section, FSIS employs a ―category‖ system to measure the performance of
establishments producing raw products. This ―Category 1‖ measure is publicly reported in the USDA
Strategic Plan for FY2010-FY2015 and the annual USDA Performance and Accountability Report
(PAR), and tracked internally at FSIS in the Quarterly Performance Report. As described above,
starting in July 2011, the new Salmonella performance standard accepted five positive samples in a 51-
sample set for young chickens and four positive samples in a 56-sample set for turkeys. Salmonella
performance Categories 1 and 2 for young chicken and turkey establishments, based on the new
performance standards, will be applied exclusively for Agency internal analysis and quarterly aggregate
reporting. FSIS will track industry progress in achieving Category 1 status and achieving the new
standard, with greater numbers of establishments maintaining Category 1 status conveying a public
health benefit in reduced Salmonella illnesses in the population. Finally, FSIS will web-post the names
of young chicken and turkey establishments that fail the new Salmonella standards (―Category 3‖) for
30
their last set. This web posting will begin as sample sets scheduled for July 2011 are completed. This
new standard is also a key metric in the recently published FSIS Strategic Plan for 2011-2016.
End of Set Letters
FSIS developed a set of performance measures that allow the Agency to evaluate its progress in meeting
a variety of human health and other goals. These measures focus on both short-term, process-oriented
goals and long-term, outcome-based FSIS goals, including reducing the burden of foodborne illness
from FSIS regulated products. One of measures developed by FSIS was the ―percent of slaughter
establishments with an End of Set (EOS) Letter indicating a Salmonella serotype of human health
concern was detected‖—see Table 2.1.1.5. The CDC produces a list each year of the top 20 Salmonella
serotypes (PFGE patterns) of human health concern through its PulseNet program. FSIS also uses
information from the ARS VetNet program. FSIS seeks to achieve a one percent decrease annually in
the percent of establishments with an EOS letter with a serotype of human health concern.
Table 2.1.1.5: FSIS Performance Measure for EOS Letters
Objectives
Goal
Measures
Baseline
FY 2009
FY 2010
FY 2011
FY 2012
FY 2013
FY 2014
FY 2015
Percent of slaughter establishments
with an End of Set Letter indicating
a Salmonella serotype of human
health concern was detected.
37.5%
37.1%
36.8%
36.4%
36.0%
35.7%
35.3%
31.1%
Top Ten Salmonella Serotypes
In the CDC‘s annual FoodNet report, the Agency identifies the ten most common Salmonella serotypes
as reported by states and localities participating in the FoodNet program.
61
FSIS also collects individual
Salmonella subtype (serotype and PFGE pattern) results from positive samples as part of the FSIS
Salmonella sampling program. FSIS has established the technical mechanisms to automatically
populate Salmonella EOS Letters with this enhanced serotype information, in addition to what is
currently included in EOS letters to establishments from the appropriate data tables in the FSIS data
warehouse. FSIS and CDC are currently working to finalize both the data sharing agreement and the
data itself. Once available, FSIS will provide establishments with this information. Additionally, once
the data pathways are finalized, FSIS will provide the information to establishments and verify that they
appropriately consider it when evaluating their food safety systems through the performance of a Food
Safety Assessment (FSA). An FSIS Notice announcing the inclusion of this information in FSIS
Salmonella EOS Letters has been developed and will be issued once the data pathways between FSIS
and CDC have been finalized. Changes in the types of serotypes identified over time can be tracked via
the annual FoodNet report.
61
Please see the following website for more information: http://cdc.gov/foodnet/factsandfigures/2009/Table6_top10ss_09.pdf.
31
2.1.2 Campylobacter
Campylobacter Verification Project: HC11
Historical Basis
As discussed in Section 2.1.1, on May 14, 2010, FSIS published a Federal Register Notice (75
FR 27288) in which it announced the implementation on July 1, 2011 of new performance
standards for Salmonella and Campylobacter for chilled carcasses in young chicken (broiler) and
turkey slaughter establishments. The new performance standards were developed in response to
a charge from the FSWG and based on results from the FSIS Nationwide Microbiological
Baseline Data Collection Program. The standards are being applied to sample sets collected and
analyzed by FSIS to evaluate establishment performance with respect to requirements of the
PR/HACCP Rule. All sample sets scheduled for young chicken and turkey establishments will
be analyzed for both Campylobacter and Salmonella, and follow-up sample sets responding to
sample set failure for either organism will be analyzed for both organisms.
Intent of Project
FSIS considers Campylobacter verification testing to be a direct indicator of the effectiveness of
process control. The raw pathogen reduction performance standards program for Campylobacter
also serve, once fully established and once sampling has been conducted for an extended period
of time, a variety of other functions, including assessing establishment compliance with the
performance standard and comparing industry-wide and peer-to-peer trends regarding percent
positive over time.
Type of Analysis
Although the Campylobacter performance standards are based on the positive/negative results
from the quantitative test portion, the Campylobacter laboratory method includes both a
qualitative and quantitative method that will further inform FSIS of the presence of this pathogen
in regulated product.
Volume Data
Yearly production volumes for chicken and turkey are acquired from slaughter volume data in
the eADRS database.
Constructing the Sampling Frame
Campylobacter verification sampling is applicable only to broiler and young turkey carcass sets
scheduled to begin after July 1, 2011. Establishment categories for these poultry establishments
are based on the Salmonella set results. However, if a establishment fails a set for either
Salmonella or Campylobacter it will be treated as a failed set (for the purposes of scheduling
only) and be scheduled for immediate follow-up set the next month. Additional details about the
sampling criteria, exclusion criteria and sampling algorithm and frame are provided above in the
Salmonella Section 2.1.1.
Campylobacter Programs Measure of Success
The overall purpose of the Campylobacter verification testing program is to ensure that the
pathogen reduction performance standards are being met, which ensures that the industry is
maintaining process control. Consequently, FSIS believes it is appropriate to measure the
success of the different sampling project in broad terms. As FSIS recently implemented the new
32
performance standards for Campylobacter,
62
the measures of success listed below are under
currently being developed and therefore adjustments to these measures may be made moving
forward.
To measure the success of the Campylobacter sampling project, FSIS is considering evaluating
its efforts in terms of three key metrics:
1. Volume-weighted percent positives for Campylobacter
2. Reductions in case rates (as reported by the CDC FoodNet) for Campylobacter
3. Estimated number of Campylobacter foodborne illnesses associated with FSIS-regulated
products
62
Please see the following website for more information: http://www.fsis.usda.gov/OPPDE/rdad/FRPubs/2009-0029.pdf.
33
2.1.3: E. coli O157:H7 Raw Sampling Projects
Overview of Sampling Projects
FSIS maintains many E. coli O157:H7 sampling projects for domestic establishments. The
different E. coli O157:H7 sampling projects are summarized in Table 2.1.3.1.
Table 2.1.3.1 FSIS E. coli O157:H7 Sampling Projects
Product Type
E. coli
Sampling
Projects
Number of
E. coli
Analyzed
samples
CY2010
63
Regulatory
Purpose of
Sampling
Program
Type of Sampling
Program
Raw ground beef
MT43
11,291
Verify zero-
tolerance
Weighted Random
Follow-up testing to a raw
ground beef positive
MT44
309
Verify corrective
measures
Targeted Consecutive
Beef Manufacturing trim
MT50
1,274
Verify HACCP
compliance
Random
Raw ground beef
components other than
trim
MT54
169
Verify HACCP
compliance
Random
Bench trim
MT55
547
Verify HACCP
compliance
Random
Follow-up testing at
supplier establishments
following MT43, MT44,
or MT55 positive
MT52
636
Verify corrective
measures
Targeted Consecutive
Follow-up testing to a
MT50, MT54, MT55, or
MT52 positive
MT53
125
Verify corrective
measures
Targeted Consecutive
Domestic Sampling Programs (Federally inspected establishments)
Historical Basis for Sampling Raw Ground Beef
The following section provides a general overview of the historical progression of FSIS
sampling for E. coli O157:H7. In addition to the material provided below, a timeline of FSIS
activities related to E. coli O157:H7 can be found on the FSIS website.
64
In 1994, FSIS announced that raw ground beef products contaminated with E. coli O157:H7
would be considered adulterated, unless the ground beef was processed further to destroy the
pathogen. A sampling project for E. coli O157:H7 in raw ground beef was initiated in October
1994. The objectives of the verification testing project were to verify the effectiveness of control
measures at individual grinding establishments and to stimulate industry action to reduce the
presence of the pathogen in ground beef. FSIS initiated the verification testing program on raw
ground beef, rather than on source materials (e.g., carcasses or beef manufacturing trimmings
used to make ground beef) for a number of reasons, including that ground product was well-
63
Please see the following website for more information:
http://www.fsis.usda.gov/Science/Ecoli_Raw_Beef_Testing_Data_YTD/index.asp.
64
Please see the following website for more information: http://www.fsis.usda.gov/PDF/Ecoli_O157_Timeline.pdf.
34
blended, was in the form closest to the consumer and was the product most frequently associated
with foodborne illness.
The verification testing program was not statistically designed at initiation; some samples were
randomly selected and others were targeted. The frame also contained retail stores and federally
inspected establishments that produced raw ground beef. Random samples were scheduled at
retail and federally inspected establishments. Targeted samples were scheduled at
establishments identified as performing below average using Review and Assessment data,
65
PBIS performance data for sanitation, receiving, or processing deficiencies or consent orders.
Additional targeted follow-up samples were collected when a confirmed positive was detected.
These samples were collected from the establishment that tested positive for E. coli O157:H7
and from all other establishments associated with the same corporate structure. Targeted
samples were also collected from State-inspected establishments and imported ground beef
products.
In February 1998, FSIS issued updated instructions for raw ground beef sampling to verify the
effectiveness of controls at individual grinding establishments.
66
It was and is FSIS policy that
the establishment is responsible for having a high degree of confidence that E. coli O157:H7
does not contaminate the production lot. Each month, FSIS randomly selected establishments,
while FSIS Compliance Investigators (CI) targeted retail stores for sample collection.
At that time, FSIS based its sampling plan on information from the CDC FoodNet sentinel sites‘
historical data of outbreaks (e.g., geographical locations in the U. S. where public health
laboratories actively collect human illness data).
67
If an establishment or a retail outlet initiated
its own routine sampling program, had certification from suppliers that the product was tested,
or, in the case of an inspected establishment, used in-establishment validated pathogen reduction
interventions on beef carcasses, FSIS did not collect samples. When a sample tested positive for
E. coli O157:H7, subsequent samples from new lots were collected daily until negative results
were obtained in 15 consecutive samples. Additionally, if ground beef at an FSIS inspected
establishment or retail outlet was associated with an outbreak of foodborne illness linked to E.
coli O157:H7, FSIS sampled daily until 15 consecutive samples tested negative. All raw,
comminuted beef products produced on the shift represented by the positive sample were subject
to voluntary recall.
In 1999, FSIS further clarified in a Federal Register Notice an expanded adulteration policy,
where raw beef source materials used to manufacture ground beef, as well as beef that had been
handled in a way that could transfer the external surface contamination to the interior of the
65
From 1995-1996, FSIS had a program office called Review and Assessment. The office would visit establishments and
develop review reports that could lead to suspensions in establishments where in-establishment inspection personnel had never
written a PDR (a predecessor of the NR). A major focus area was establishment review triggered by high levels of fecal
contamination.
66
FSIS Directive 10,010.1 "Microbiological Testing Program for Escherichia coli 0157:H7 in Raw Ground Beef," February 1,
1998.
67
The CDC FoodNet sites were used for retail sample guidance as, prior to HACCP implementation, retail stores were thought to
be closest potential source of contamination for consumers. Retail samples were requested from the Compliance Offices located
in or near FoodNet sites. The FoodNet site priority did not apply to Federally Inspected Establishments. Directive 10,010.1
(February 1, 1998) included the policy for inspected establishments, but the exemptions were canceled in 2002.
35
product, would also be considered adulterated.
68
In Apri1 2003, FSIS announced with FSIS
Notice 11-03 that it would begin testing all products that met the standard of identity in 9 C.F.R.
319.15 (a-c), which could include coarse, ground beef. Thus, if an establishment producing
coarse ground beef shipped that product to another establishment that re-ground the coarse
ground product into finely ground beef, FSIS would now sample the product at both
establishments.
In March 2004, FSIS implemented a major revision to FSIS Directive 10,010.1, Microbiological
Testing Program for Escherichia coli 0157:H7 in Raw Ground Beef. All establishments
producing raw ground beef products, raw ground beef components or raw beef patty components
were eligible for FSIS sampling for E. coli O157:H7. In addition, the directive provided the
following instructions to FSIS field personnel:
1. Traceback procedures were to be performed to collect source supplier information after
an FSIS positive test result.
69
This information was entered into a database, and suppliers
identified repeatedly in the database were subject to a comprehensive FSA of the
supplier;
2. Verification of process control for lots testing presumptive positive or positive for E. coli
O157:H7 was required; and
3. Follow-up actions to an FSIS positive test result were to be implemented both at the
original positive establishment and at all supplier establishments.
In 2008, an FSIS analysis found that production volume is a better determinant of risk for E. coli
O157:H7 than HACCP size. This analysis determined that E. coli O157:H7 in cattle and the
incidence of foodborne illness from E. coli O157:H7 positive products displayed positive
seasonal effects during warmer months. As a result, in January 2008, FSIS implemented risk-
based sampling of raw ground beef, weighted by production volume and historical test results.
70
Under this new verification testing program, larger volume operations are tested more frequently
than in the past. FSIS also implemented a change in the laboratory testing method at this time
that included a single 325 gram test portion, enriched at a 4:1 ratio of enrichment broth in
product as an alternative sample preparation procedure.
FSIS analytical capacity has allowed for increased sampling over time. FSIS analyzed
approximately 3,000 to 7,000 samples per year from 2001 – 2003. Starting in 2005, FSIS
laboratories started analyzing around 11,000 samples per year.
Historical Basis for Sampling of Products Other than Raw Ground Beef
In a January 1999 policy statement, FSIS noted that when the E. coli O157:H7 sampling
methodology became sufficiently refined to enhance the likelihood of finding the pathogen on
source materials used to make ground beef, the Agency expected to begin supplementing the
verification testing project for ground beef with FSIS testing of source materials.
68
See: http://www.federalregister.gov/articles/1999/01/19/99-1123/beef-products-contaminated-with-escherichia-
coli-o157h7.
69
With FSIS Notice 58-10, this information is collected at the time the sample is collected.
70
Withee J., Schlosser, W. (February 2008). Risk-based sampling for Escherichia coli in O157:H7 in ground beef
and beef trim. USDA/FSIS/OPHS/Risk Assessment Division:
http://www.fsis.usda.gov/PDF/Ecoli_Sampling_RA_Report_Feb08.pdf, accessed on November 18.
36
In April 2003, FSIS Directive 10,010.1 stated that head and cheek meat used for production of
ground beef not treated with antimicrobial interventions could present an elevated risk for
presence of E. coli O157:H7 and should be addressed in an establishment‘s HACCP plan. FSIS
also stated that the Agency would soon begin testing manufacturing trimmings and carcasses to
supplement the ground beef testing project, although implementation was delayed as FSIS had
not developed a laboratory procedure at that time to effectively test manufacturing trimmings.
In August 2003, FSIS requested that the National Advisory Committee on Microbiological
Criteria for Foods (NACMCF) provide feedback on the Agency‘s baseline study design for raw
ground beef components to ensure that appropriate priority was given to the different categories
of product. NACMCF reordered the priority ranking of the categories provided by USDA/FSIS
based on volume, the perceived contribution to the risk of illness, expert opinion on the use of
the components in ground beef and processing variables, such as chilling rates during
production. NACMCF recommended that FSIS reprioritize the rank order for engaging in
baseline studies as follows:
1. Domestic trim and subprimals,
2. Advanced Meat Recovery (AMR),
3. Low-Temperature-Rendered Products (LTRP ),
4. Imported frozen and fresh beef, and
5. Weasand, cheek, and head meat.
Domestic trim and subprimals were considered the number one priority since these components
comprise the largest volume of raw materials used in ground beef and are known to contain E.
coli O157:H7.
71
In March 2007, FSIS began routine verification sampling of beef manufacturing trimmings
intended for use in raw ground beef, hamburger or beef patty products at the slaughter
establishments that produced those trimmings. Trim sampling was initiated with FSIS Notice
18-07 in March 2007 at a rate of 50 scheduled samples per week (MT50 project). This decision
was partly informed by grinding establishments‘ claim that meat was already contaminated upon
receipt by the establishment. FSIS conducted a baseline study in 2007 that showed a higher level
of E. coli O157:H7 in tested beef manufacturing trimmings than in tested raw ground beef.
Incorporating the results of the baseline study, FSIS performed a statistical analysis to determine
the minimal number of samples needed to be analyzed in a year to be able to detect a 50%
change in positive rates, as compared to the prevalence estimate calculated from the baseline
study (under standard statistical assumptions). The evaluation concluded that an increase to
approximately 3,500 analyses per year was necessary and, at the time, FSIS was sending out
2,600 sample request forms per year. However, due to budgetary constraints, this proposed
change was never implemented in the MT50 project.
Starting in late 2007, FSIS made several changes to the sampling program to expand the scope of
products tested for E. coli O157:H7. FSIS began collecting samples of raw ground beef
components other than trim in December 2007 (MT54 project). FSIS also began testing trim and
other raw ground beef components to enforce the policy that E. coli O157:H7 adulterates intact
71
http://www.fsis.usda.gov/OPHS/NACMCF/2003/gb_base.pdf
37
product intended for use in raw non-intact product. Consequently, FSIS began verifying that
both grinders and slaughter supplier establishments had effective controls for E. coli O157:H7 to
incentivize suppliers and grinders to maintain effective controls for the pathogen. The frame
definition for manufacturing trimmings (MT50 project) was used for this new sampling project
and the sample size was calculated based on the capacity of the FSIS laboratories, which allotted
780 analyses per year. No oversampling to account for response rate was incorporated into the
sample size.
Based upon research resulting from the 2007 FSIS E. coli Checklist,
72
the Agency decided to
begin testing bench trim intended for use in raw ground beef, hamburger or beef patty products
(MT55 project). Bench trim is a component of raw ground beef that is not produced at slaughter
establishments. Rather, it is produced in the process of cutting down purchased carcasses, primal
or subprimals into steaks or roasts. The trim produced in this process is then sold to a grinding
establishment as a component of raw ground beef. Bench trim sampling began in 2009 and FSIS
allocated 1,800 analyzed samples annually for this new project. No oversampling to account for
response rate was incorporated into the sample size.
Type of Analysis
In general, samples for E. coli O157:H7 are qualitatively and quantitatively assessed for the
presence of the organism. FSIS collects information on samples that confirm positive using the
MPN procedure on sample reserves and this information is captured by the FSIS laboratories,
though not routinely reported in public venues.
Description of the FSIS Domestic Sampling Projects for E. coli O157:H7
A description of each of the domestic sampling projects for E. coli O157:H7 follows below.
Please see Appendix B for additional information on the E. coli O157:H7 sampling projects.
Risk-based testing of raw ground beef at domestic establishments (MT43)
Purpose
The purpose of the MT43 risk-based sampling project is to provide verification of HACCP
policy and to assess, and minimize, the risk to the public‘s health from contaminated raw ground
beef. FSIS initially implemented risk-based sampling of raw ground beef, weighted by
production volume and historical test results, in January 2008.
73
The MT43 project replaced the
MT03 project, which was simple random sampling of raw ground beef. The current
methodology was implemented in late 2009.
Sampling Frame Definition
The frame includes establishments that meet one of the following requirements: 1) Federally
inspected establishments that have the 9 CFR flag identified in their PBIS profile (per FSIS
Notice 105-08, previously FSIS Notice 86-07) or report producing raw ground beef finished
products in their PHIS profile and 2) Federally inspected establishments that have had an MT43
72
Alvares, C., Lim, C., & Green, K. (August 2008). Results of checklist and reassessment of control for Escherichia coli in
O157:H7 in beef operations. USDA/FSIS: http://www.fsis.usda.gov/PDF/Ecoli_Reassement_&_Checklist.pdf, accessed on
November 18, 2010.
73
Withee J., Schlosser, W. (February 2008). Risk-based sampling for Escherichia coli in O157:H7 in ground beef and beef trim.
http://www.fsis.usda.gov/PDF/Ecoli_Sampling_RA_Report_Feb08.pdf.
38
sample collected in the last 12 months. An exclusion list is maintained to exclude establishments
for special circumstances when documentation is provided by FSIS field personnel.
Average Frame Size
Generally, the frame is around 1,300 establishments. This value fluctuates monthly as new
establishments become eligible and others become ineligible. Ineligibility may result from
seasonal processing, closure, withdrawal of inspection, change in business practices and other
reasons.
Sample Size
FSIS selects 1,300 establishments from the frame every month. The decision was not
statistically based, but based upon lab capacity constraints at the time the MT03 project was
initiated.
Sampling methodology
MT43 is weighted random sampling with replacement under the constraints of sampling ceiling
by volume strata and annual sampling floors. See below for discussion of weights, ceilings and
floors. A sampling algorithm is used, which selects the sample from the frame. The
methodology was based upon the results of an FSIS analysis that identified production volume
and historical test results as a significant risk factor for public health exposure.
Sampling weights
Sample selection is weighted by scaling factors to produce a probability of selection for each
establishment, p
i
. The scaling factor formula for n establishments has a number of inputs.
1. Historical testing data - those establishments that have had a positive test result within the
last six months are five times more likely to have a positive in the near future, so the weight
for these establishments has a factor of five applied. This is referred to as the hazard score,
hs
i
. The hazard score is five for establishments that have had a recent positive and one for
establishments without a positive test result.
2. Production volume data - those establishments that have higher estimated annual production
volume are weighted more heavily. The volume weights are calculated from the four daily
production volume groups, as described below, and are reported on every MT43 form or in
the PBIS (and, moving forward, PHIS) establishment profile.
74
An establishment‘s volume
group is assigned by taking the mode of all the reported groups from every form collected in
the last 12 months. See Table 2.1.3.2 for details on assigning estimated annual volume by
volume group. A volume score, vs
i
, is then calculated that transforms the data relative to the
smallest production volume.
74
Volume groups were developed by a multi-disciplinary team of scientists and technical staff within the FSIS prior to 2003.
There are currently more volume groupings for MT43 sampling within PHIS, but they map exactly to the Agency‘s existing PBIS
categories.
39
Table 2.1.3.2: Estimated Annual Volume by Volume Group for E. coli O157:H7 Sampling
Projects
Assigning Estimated Annual Volume by Volume Group
Volume
Group (i)
Estimated daily volume
in lbs/day
Midpoint of volume
in lbs/day (V
i
)
Volume Score
(
1
> 250,000*
375,000
32
2
50,001 – 250,000
150,000
13.375
3
1,000 – 50,000
25,500
3.069
4
< 1,000
500
1
* 500,000 lbs/day is assumed to be the maximum.
3. Scaling factor constants - FSIS decided that the volume factor of the weight being allowed to
function freely did not meet needs of the Agency. That is, very large volume producers were
selected at too burdensome a rate and very small volume producers were selected at a rate too
low to support HACCP verification. FSIS tested various scaling factors until the Agency
was satisfied with the general rate of selection by production volume group. The scaling
factors, = 32 and , cause the volume scores to range from 1 to 32 rather than from
1 to 750.
Sampling ceilings
FSIS established sampling ceilings to ensure that the Agency does not over-burden very small
establishments. The sampling ceilings were raised for large volume establishments in 2009
because sampling at a higher frequency in establishments that produce more volume of product
results in a higher level of confidence in an estimate that is weighted by production volume.
75
Please see Table 2.1.3.3 for sampling ceilings.
Table 2.1.3.3: Sampling Ceilings for E. coli O157:H7 Sampling Projects
Sampling Ceilings
Volume Group
Maximum Samples Allowed per Month
1
4 per month for large volume producers
2
3 per month for medium volume producers
3
2 per month for small volume producers
4
1 per month for very small volume producers
Sampling floors
FSIS established sampling floors to ensure that each establishment in the frame is sampled every
year. The current sampling floor is three analyzed samples in 12 months.
Collection methodology
Field inspectors are to collect one pound of raw ground beef per FSIS Directive 10,010.1,
Revision 3 Ch. II, III.
76
75
FSIS temporarily implemented sampling ceiling and sample size increases in August and September 2011 through
FSIS Notice 36-11.
76
http://www.fsis.usda.gov/OPPDE/rdad/FSISDirectives/10010.1Rev3.pdf
40
Mean response rate
Since 2006, the mean annual response rate of collected samples, as compared to printed forms, is
72%. Sample requests have a 30 day window, which means that it is likely that some establishments
will be scheduled, but may not have product available within that 30 day window due to a
combination of intermittent production and changing FSIS inspector patrol assignments.
Mean analyzed samples
Since 2006, FSIS laboratories have analyzed a mean of 11,482 samples per year.
Follow-up testing to a raw ground beef positive at a domestic establishment (MT44).
Purpose
An MT44 sample follows an MT43 positive. The purpose of the MT44 sampling project is to
follow-up in establishments that recently had an MT43 positive, providing more frequent,
targeted sampling at these establishments as a means of verifying that HACCP systems are back
in control. MT44 replaced the MT04 project at the same time that MT43 replaced MT03.
Sampling Frame definition
This methodology does not define a frame because it is not a statistical sampling project. Rather,
establishments that receive MT44 sample request forms are those that have recently had an
MT43 positive sample.
Average frame size
This does not apply because MT44 is targeted sampling based upon MT43 results.
Sample size
For each MT43 positive at an establishment, 16 follow-up sample request forms are
automatically scheduled at that establishment. In the case where the establishment produces less
than a thousand pounds per day, then only eight follow-up sample request forms are collected at
that establishment. The decision to collect 16 follow-up samples was made in 2008, when an
FSIS analysis showed that establishments that test positive were five times more likely to receive
another positive in the next 160 days than those that do not. At that time, the national average
for grinders was approximately 0.17% positive in sampled lots. Under binomial distribution, 16
negative samples from 16 lots gives a 95% confidence that the establishment is less than 100
times above the national average. While FSIS performed these calculations, they were not
seriously considered out of practicality as 1,750 follow-up samples over four months would need
to be scheduled to verify that the establishment was at or below 0.17% positive.
Sampling methodology
Samples for the MT44 project are automatically scheduled for the establishment following an
MT43 positive.
Collection methodology
Inspectors are to collect one pound of raw ground beef per FSIS Directive 10,010.1, Revision 3,
Ch. II, III.
41
Mean response rate
Between 2006 and 2009, the mean annual response rate of collected samples, as compared to
printed forms, was 56%. However, there was marked improvement in samples received for
analysis starting in 2009, with a mean response rate of 83%. Also, 16 forms are sent to all
establishments, but inspectors are instructed to collect only eight samples in very small volume
establishments. The non-response rate may therefore be artificially inflated. Additionally, some
establishments choose to stop producing raw ground beef after an MT43 positive, eliminating the
possibility of follow-up sampling.
Mean analyzed samples
The annual mean of analyzed samples is 254 per year, but this value ranges from 24 samples
analyzed in 2006 to 484 samples analyzed in 2009. This variation can partly be attributed to
changes in the number of follow-up samples scheduled in response to a single positive.
Additionally, the annual tally is based on the number of positive MT43 samples analyzed during
the year.
Routine testing of manufacturing trimmings at domestic establishments (MT50)
Purpose
The intended purpose of the MT50 sampling project is to verify HACCP. The program was
initiated to randomly sample establishments that produce beef manufacturing trimmings. The
intention was to enhance the program at some future date to make it more risk-based and support
measuring prevalence. In January 2011, FSIS published a report entitled the National Prevalence
Estimate of Pathogens in Domestic Beef Manufacturing Trimmings (Trim). This report was
based on data collected from a 2007 baseline study on trim. The report indicates that, from
2005-2007, the estimated national prevalence of E. coli O157:H7 in beef trim was 0.39%, with a
95% confidence interval between 0.05% and 0.73%.
77
Currently, the beef manufacturing
trimmings verification sampling program is conducted as originally designed. Enhancements to
the program are still being considered.
Sampling Frame definition
The frame includes all active, federally inspected beef and veal slaughter establishments that
produce trim for use in raw ground beef and identified sister establishments. If an MT50
sampling form is returned to the laboratories with code 60 (product not produced) selected, then
the establishment is excluded from the frame for 12 months. An exclusion list is maintained to
exclude establishments for special circumstances when documentation is provided by FSIS field
personnel.
Average frame size
Generally, the frame is around 480 establishments. This value fluctuates monthly as new
establishments become eligible and others become ineligible. Ineligibility may result from
seasonal processing, closure, withdrawal of inspection, change in business practices and for other
reasons.
77
Please see the following website for more information:
http://www.fsis.usda.gov/PDF/Baseline_Data_Domestic_Beef_Trimmings_Rev.pdf.
42
Sample size
FSIS selects between 200 and 250 establishments from the frame every month, depending upon
the number of weeks in a month. Annually, this amounts to mailing 2,600 sample forms per
year. This sample size is not adequate to detect practical changes from the national prevalence
level as estimated by the 2007 baseline study. The choice of implementing 50 forms per week
was the result of FSIS laboratory capacity at the time of implementation.
Sampling methodology
MT50 is a simple random sample without replacement.
Collection methodology
Inspectors were directed to collect an N60 sample weighing 325 grams where each piece
matches a template (1‖ wide x 3‖ long x 1/8‖ deep) and the goal is collection of 60 pieces, per
FSIS Directive 10,010.1, Revision 3, Ch. II, IV. The N60 sample is to be placed in one container
and a second container is filled with small pieces. The purpose of an N60 sample is to gather a
more representative sample consisting mostly of surface area, where E. coli O157:H7 is most
likely to exist. In practical terms, however, it is nearly impossible for an inspector to collect an
N60 sample that meets the number of pieces, piece dimension and total sample weight
specifications. As of last year, the FSIS laboratories have redefined the analytical portion to
consist of the analysis of up to 60 pieces (N60) weighing up to 715 grams (up to two 325 gram ±
32.5 gram subsamples, rather than limit the sample to 325 grams. This decision was reached
after the method was validated to process a single 325 gram sample (previously it was 5-65 gram
subsamples).
Mean response rate
Since 2007, the mean annual response rate for collected samples, as compared to printed forms,
is 40%. The low response rate may be due to a poorly defined frame; that is, not all slaughter
establishments may produce this product or they produce them infrequently. Additionally, the
collection methodology is N60, which is an onerous and resource-intensive method. This may
also contribute to low collection rates.
Mean analyzed samples
Since 2007, FSIS laboratories have analyzed a mean of 1,092 samples per year. The necessary
number of analyzed samples to be able to estimate a change in prevalence is between 3,000 and
4,000 per year. As a result, FSIS cannot say with certainty that the results of the MT50 sampling
project are statistically different from the baseline prevalence estimate.
Routine testing of domestic components to raw ground beef – other than trim – at domestic
establishments (MT54)
Purpose
The intended purpose of the MT54 sampling project is to verify HACCP.
43
Sampling Frame definition
The frame includes establishments that meet one of the following requirements: 1) Active,
federally inspected beef and veal slaughter establishments and 2) Active, federally inspected
ammoniated lean finely textured beef (LFTB) producing establishments (there are six of these
known to be operating in the U.S.). If an MT54 sampling form is returned to the laboratories
with code 60 (product not produced) selected, then the establishment is excluded from the frame
for 12 months. An exclusion list is maintained to exclude establishments for special
circumstances when documentation is provided by FSIS field personnel.
Average sample frame size
Generally, the frame is around 450 establishments. This value fluctuates monthly as new
establishments become eligible and others become ineligible. Ineligibility may result from
seasonal processing, closure, withdrawal of inspection, change in business practices or for other
reasons.
Sample size
FSIS selects between 60 and 75 establishments from the frame every month, depending upon the
number of weeks in a month. Annually, this amounts to mailing 780 sample forms per year.
The choice of implementing 15 forms per week was due to laboratory contracting limitations of
780 analyses per year at the time of implementation.
Sampling methodology
MT54 is a simple random sample without replacement, except for ammoniated beef
establishments. The ammoniated LFTB establishments are selected with certainty every month.
Collection methodology
Inspectors are directed to collect two pounds of component product other than trim, per FSIS
Directive 10,010.1, Rev. 3. How the inspector collects the sample depends upon the type of
product being collected. For Advanced Meat Recovery (AMR) product and Low Temperature
Rendered (LTR) product, IPP are to select randomly a sample consisting of no less than one
pound, but not more than two pounds of product from a specific production lot. For other raw
beef components (e.g., heart meat), IPP are to collect randomly one piece, or enough pieces, of
the beef components to equal no less than one pound, but not more than two pounds of product
from a specific production lot. If the component is very large, IPP are to collect an N60 sample
where the goal is collection of 60 pieces, in the manner as described in Ch. II, IV. In practical
terms, however, it is nearly impossible for an inspector to collect an N60 sample that meets the
number of pieces, piece dimension and total sample weight specifications. As of last year, the
FSIS laboratories have redefined the analytical portion to consist of the analysis of up to 60
pieces (N60) weighing up to 715 grams (up to two 325 gram ± 32.5 gram subsamples) rather
than limit the sample to 325 grams. This decision was reached after the method was validated to
process a single 325 gram sample (previously it was 5-65 gram subsamples).
Mean response rate
Since 2008, the mean annual response rate of collected samples, as compared to printed forms, is
27%. The low response rate may be due to a poorly defined frame; that is, not all slaughter
establishments may produce this product.
44
Mean analyzed samples
Since 2008, FSIS laboratories have analyzed a mean of 224 samples per year.
Routine testing of bench trim at domestic establishments (MT55)
Purpose
The intended purpose of the MT55 sampling project is to verify HACCP. This sampling project
was implemented in September 2009.
Sample frame definition
The frame includes all active, federally inspected beef processing establishments that produce
trim from purchased product. These establishments were initially identified from a combination
of PBIS extension profile data and the 2007 FSIS E. coli checklist responses. If an MT55
sampling form is returned to the laboratories with code 60 (product not produced) selected, then
the establishment is excluded from the frame for 12 months. An exclusion list is maintained to
exclude establishments for special circumstances when documentation is provided by FSIS field
personnel.
Average sample frame size
Generally, the frame is around 1,100 establishments. This value fluctuates monthly as new
establishments become eligible and others become ineligible. Ineligibility may result from
seasonal processing, closure, withdrawal of inspection, change in business practices or for other
reasons.
Sample size
FSIS selects 150 establishments from the frame every month. Annually, this amounts to mailing
1,800 sample forms per year. The decision to select 150 forms per month was due to a desire to
send one form to each establishment in the original frame within the first 12 months of project
implementation.
Sampling methodology
MT55 is a simple random sample without replacement.
Collection methodology
Inspectors are directed to collect an N60 sample weighting 325 grams where each piece matches
the template (1‖ wide x 3‖ long x 1/8‖ deep) and the goal is collection of 60 pieces, per FSIS
Directive 10,010.1, Revision 3, Ch. II, IV. The N60 sample is to collect 60 pieces that weigh
325 grams in one bag and to collect a second bag of small pieces. If the establishment produces
trim derived from primals and subprimals resulting in large pieces, IPP are to sample the product
using the N60 sampling procedures in Chapter II, IV. If the establishment produces trim derived
from primals and subprimals resulting in trim too small to be sampled using the N60 sampling
procedure or produces trim derived from steaks, roasts or other cuts designated for non-intact
use, IPP are to collect enough pieces to equal two pounds of product for sampling. If the
establishment produces both types of trim as described above, IPP are to sample only the product
that can be sampled using the N60 sampling procedure. However, in practical terms, it is nearly
impossible for an inspector to collect an N60 sample that meets the number of pieces, piece
dimension, and total sample weight specifications. As of last year, the FSIS laboratories have
45
redefined the analytical portion to consist of the analysis of up to 60 pieces (N60) weighing up to
715 grams (up to two 325 gram ± 32.5 gram subsamples) rather than limit the sample to 325
gram. This decision was reached after the method was validated to process a single 325 gram
sample (previously it was 5-65 gram subsamples).
Mean response rate
Since September 2009, the mean annual response rate of collected samples, as compared to
printed forms, is 29%. The low response rate may be due to a poorly defined frame; that is, not
all establishments in the frame may produce this product or produce them infrequently.
Mean analyzed samples
Since September 2009, FSIS laboratories have analyzed a mean of 538 samples per year.
Follow-up testing to a positive in trim or components at a domestic establishment (MT53)
Purpose
An MT53 sample follows an MT50, MT54, MT55, or MT52 positive. The purpose of the MT53
sampling project is to follow-up in establishments that recently had a trim or components
positive, providing more frequent, targeted sampling at these establishments as a means to verify
that HACCP systems are back in control.
Sample frame definition
This project does not define a frame because it is not a statistical sampling project. Rather,
establishments that receive MT53 sample request forms are those that have recently had a trim or
components positive sample.
Average sample frame size
This project does not have an average frame size because MT53 is targeted sampling based upon
positive trim or component results, which can include coarse raw ground beef.
Sample size
For each positive at an establishment, 16 follow-up sample request forms are automatically
scheduled at that establishment. In the case of a low-volume producing establishment, then only
eight samples are collected. The decision to collect 16 follow-up samples was made in 2008,
when an FSIS analysis showed that establishments that test positive were five times more likely
to receive another positive in the next 160 days than those that do not. At that time, the national
average for grinders was approximately 0.17% positive in sampled lots. Under binomial
distribution, 16 negative samples from 16 lots gives a 95% confidence that the establishment is
less than 100 times above the national average. While FSIS performed these calculations, they
were not seriously considered out of practicality as 1,750 follow-up samples over four months
would be needed to verify that the establishment was at or below 0.17% positive.
Sampling methodology
Samples for the MT53 project are automatically scheduled following a trim or components
positive. The follow-up samples are scheduled at the same establishment with the initial positive
sample.
46
Collection methodology
Inspectors are to collect an N60 sample weighing 325 grams, where each piece matches the
template (1‖ wide x 3‖ long x 1/8‖ deep) and the goal is collection of 60 pieces, per FSIS
Directive 10,010.1, Revision 3, Ch. II, IV. The N60 sample is to collect 60 pieces that weigh
325 grams in one bag and to collect a second bag of small pieces. If available, the product
collected should be the same as the original positive. As discussed above, FSIS‘ laboratories
have redefined the analytical portion to consist of the analysis of up to 60 pieces (N60) weighing
up to 715 grams (up to two 325 gram ± 32.5 grams subsamples) rather than limit the sample to
325g. This decision was reached after the method was validated to process a single 325 gram
sample (previously it was a 5-65 gram subsamples).
Mean response rate
Since 2008, the mean annual response rate of collected samples, as compared to printed forms, is
53%. Although 16 forms are sent to all establishments, inspectors are instructed only to collect
eight samples in very small volume establishments. These forms are difficult to identify in the
data structures, so they may artificially deflate the response rate. Additionally, some
establishments choose to stop producing after a positive sample, so it would be impossible to
collect a sample from them.
Mean analyzed samples
The annual mean of analyzed samples is 161 per year. This value is dependent upon the number
of positive trim and component samples analyzed during the year.
Follow-up testing at supplier establishments following a positive in raw ground beef and
bench trim (MT52)
Purpose
An MT52 sample follows a MT43, MT44, or MT55 positive. MT52 samples are also taken for
the Agricultural Marketing Service (AMS) School Lunch Program and of suppliers, when raw
ground beef or bench trim are recalled. The purpose of the MT52 sampling project is to follow-
up at originating slaughter establishments and ammoniated LFTB following an E. coli O157:H7
positive, providing more frequent, targeted sampling at the implicated supplier establishments, as
a means of verifying that HACCP systems are back in control.
Sample frame definition
This project does not define a frame because it is not a statistical sampling project. Rather,
establishments that receive MT52 sample request forms are those that have supplied trim or
components to an establishment that had a recent positive sample. Supplier establishments are
identified by FSIS traceback to the originating slaughter establishments. These supplier
establishments are documented and tracked in the FSIS Supplier Traceback to E. coli Positive
System (STEPS).
Average sample frame size
This does not apply because MT52 is targeted sampling based upon positive E. coli O157:H7
results.
47
Sample size
If an originating slaughter establishment was the only supplier, or if any of the originating
slaughter establishments were suppliers identified in STEPS within approximately four months
(or 120 days) of the current raw ground product positive result, then 16 follow-up sample request
forms are automatically scheduled at that establishment. If the establishment is a low volume
producer (less than 1,000 lbs. per day), only eight follow-up samples are collected. The decision
to collect 16 follow-up samples was made in 2008, when an FSIS analysis showed that
establishments that test positive were five times more likely to receive another positive in the
next 160 days than those that do not. At that time, the national average for grinders was
approximately 0.17% positive in sampled lots. Under binomial distribution, 16 negative samples
from 16 lots gives a 95% confidence that the establishment is less than 100 times above the
national average. While FSIS performed these calculations, they were seriously considered out
of practicality as 1,750 follow-up samples over four months would be needed to verify that the
establishment was at or below 0.17% positive. If a supplier is not a sole supplier or a repeat
supplier in STEPS, FSIS will request a single follow-up sample from the supplier for each
component used in the positive raw ground beef product.
Sampling methodology
Samples for the MT52 project are requested following traceback investigation. The follow-up
samples are scheduled at the originating slaughter suppliers or ammoniated LFTB, not at other
intermediate suppliers.
Collection methodology
Inspectors are to collect an N60 sample weighing 325 grams, where each piece matches the
template (1‖ wide x 3‖ long x 1/8‖ deep) and the goal is collection of 60 pieces, per FSIS
Directive 10,010.1, Revision 3, Ch. II, IV. The N60 sample is to collect 60 pieces that weigh
325 grams in one bag and to collect a second bag of small pieces. Collection depends on the
type of product being collected—see FSIS Directive 10,010.1, Revision 3. Inspectors should
collect the same component identified in traceback. As discussed above, FSIS‘ laboratories have
redefined the analytical portion to consist of the analysis of up to 60 pieces (N60) weighing up to
715 grams (up to two 325 gram ± 32.5 gram subsamples) rather than limit the sample to 325
grams. This decision was reached after the method was validated to process a single 325 gram
sample (previously it was 5-65 gram subsamples).
Mean response rate
Since 2007, the mean annual response rate of collected samples, as compared to printed forms, is
77%. IPP are only to submit eight samples and mail the remaining eight follow-up forms with
the last sample collected. Response rates may also be influenced by production volume of
scheduled establishments.
Mean analyzed samples
The annual mean of analyzed samples is 610 per year. This value is dependent upon the number
of positive E. coli O157:H7 samples analyzed during the year.
48
Limitations of Current Sampling Projects
FSIS has identified several limitations to the Agency‘s current sampling projects, most of which
influence FSIS‘ ability to compute estimates.
1. Prior Notification
It is possible that prior notification affects the ability of FSIS to collect representative
samples. However, policy constraints require that notification be given to establishments so
that they can plan for holding product until FSIS laboratory test results are to help prevent
recalling product and posing a risk to the public‘s health. FSIS recently requested comments
on a new Federal Register Notice that would change the Agency‘s procedures and withhold a
determination as to whether meat and poultry products are not adulterated, and thus eligible
to enter commerce, until all test results that bear on the determination have been received.
78
2. Sample Sizes for MT50, MT54, MT55 Programs
Precision usually improves as sample sizes increase. In addition, the precision of the
estimate may provide an indicator of its reliability. That is, larger sample sizes typically lead
to smaller variances. In particular, rare event sampling requires large sample sizes to obtain
reasonable precision. The MT50 program detected four positive trim samples out of 1,274
analyzed samples in 2010, which gives an indication that this is rare event testing. Likewise,
there were no positives out of the 169 analyzed MT54 samples in 2010 and no positives out
of the 574 analyzed in MT55 samples in 2010.
3. Representativeness of the Samples
For MT50, MT54 and MT55, the sample scheduling is representative of establishments.
Because the sample designs do not incorporate stratification or weighting by production
volume, the samples may not be adequately representative of product from each production
class.
4. Industry Testing Affecting FSIS Estimates
Industry test and divert practices may result in a lower percent positive estimate obtained by
FSIS verification testing than would be obtained through baseline testing, because a portion
of positive product would already be removed.
Recent improvements to the domestic sampling
FSIS recently stopped E. coli O157:H7 RTE sampling in dried/semi-dried, fermented sausages
and cooked meat patties. This testing was discontinued after an analysis showed that testing over
10,000 such products for E. coli O157:H7 over a nine-year period yielded no positive samples.
Additionally, FSIS recently shifted from five 65 gram sub sample analyses per collected sample
to one 325 gram analysis for raw ground beef samples and two 325 gram sub samples for N60
samples or component samples, which should reduce the total number of analyses conducted by
FSIS laboratories and release resources to conduct other analyses.
E. coli O157:H7 Measures of Success
As described above, there are several different E. coli O157:H7 sampling projects, each with
slightly different goals. Yet, the overall purpose of the E. coli O157:H7 sampling projects is to
provide verification of HACCP policy implementation and to assess, and minimize, the risk to
public health from contaminated product.
78
http://www.fsis.usda.gov/OPPDE/rdad/FRPubs/2005-0044.pdf
49
Consequently, FSIS believes it is appropriate to measure the success of the different sampling
projects in broad terms, rather than focusing solely on volume-weighted percent positives in raw
ground beef. As such, to measure the success of the E. coli O157:H7 sampling program, FSIS
evaluates its efforts in terms of three key metrics:
1. Volume-weighted percent positives from FSIS‘ raw ground beef E. coli O157:H7
sampling project and unweighted percent positives from other FSIS E. coli O157:H7
testing projects.
2. Estimated number of E. coli O157:H7 foodborne illnesses associated with FSIS-regulated
products.
3. Number of E. coli O157:H7 recalls.
Volume-Weighted Percent Positives
As described in this report, FSIS samples product regulated by the Agency to verify HACCP
policy implementation and to assess, and minimize, the risk to public health from contaminated
product. FSIS uses this sampling to calculate percent positives for many of the E. coli O157:H7
sampling projects. FSIS believes that percent positives are a good measure of the effectiveness
of the individual E. coli O157:H7 sampling projects maintained by the Agency, with declines in
percent positives indicating greater control and prevention of E. coli O157:H7 in finished
product. Table 2.1.3.4 displays the volume-weighted percent positives from FSIS sampling of
raw ground beef over time. Figure 2.1.3.1 provides unweighted percent positives for four of
FSIS‘ major E. coli O157:H7 sampling projects. The reason this metric is presented is because
historically, FSIS has only calculated volume weighted percent positive for the raw ground beef
verification sampling project (MT43).
Table 2.1.3.4: Volume-Weighted Percent Positives from FSIS E. coli O157:H7 Sampling for
Raw Ground Beef
Year/Quarter
Volume-Weighted Percent
Positive (All FSIS MT43
Samples)
FY 2009
0.32%
FY10Q3
0.23%
FY10Q4
0.25%
FY11Q1
0.16%
FY11Q2
0.14%
FY11Q3
0.08%
50
Figure 2.1.3.1: Unweighted Percent Positives for Select Group of FSIS Sampling Projects
for E. coli O157:H7
Foodborne Illness Estimates:
As FSIS‘ ultimate goal is to prevent foodborne illnesses from regulated products, it is important
to measure reductions in foodborne illness as a result of FSIS inspections, sampling, policies and
other activities. FSIS calculates a performance measure, known as the All-Illness Measure,
which represents all foodborne Salmonella, Lm and E. coli O157:H7 illnesses from FSIS-
regulated meat, poultry and processed egg product. FSIS updated the All Illness Measure in Q3,
FY2011 to reflect the release of new illness burden estimates from the CDC
79
and the Healthy
People 2020 goals
80
, as well as to coincide with the release of the FSIS Strategic Plan for 2011-
2016. Objectives for the All-Illness measure were set using a combination of data from
published CDC FoodNet case rates and outbreak data and are aligned with Healthy People 2020
goals. For E. coli O157:H7, FSIS uses a rolling 12 month window of case rate data from the
CDC, in addition to an attribution estimate, to estimate the total number of E. coli O157:H7
illnesses from FSIS regulated products. Using this methodology, the illness measure is the
estimate of the total annual illnesses for the fiscal year, rather than independent measures of
illness for each quarter.
79
Scallan E, Hoekstra RM, Angulo FJ, Tauxe RV, Widdowson M-A, Roy SL, et al. Foodborne illness acquired in the United
States—major pathogens. Emerg Infect Dis [serial on the Internet]. 2011 Jan [November 2011].
http://www.cdc.gov/EID/content/17/1/7.htm.
80
For more information please see the following website:
http://www.healthypeople.gov/2020/topicsobjectives2020/objectiveslist.aspx?topicId=14.
51
Performance Measure
Using the newly updated All Illness Measure data sources and methodology, FSIS set a target of
reducing the estimated E. coli O157:H7 illnesses associated with FSIS regulated products to
20,071 in Q3 FY 2011; FSIS achieved that target with 18,798 estimated illnesses. Figure 2.1.3.2
illustrates the quarterly targets for E. coli O157:H7 illnesses and the estimated illnesses for the
most recent four quarters. While it is difficult to interpret these trends with a great deal of
statistical certainty, this data suggests that the E. coli O157:H7 testing projects are effective in
reducing human illness.
Figure 2.1.3.2: Estimated E. coli O157:H7 illnesses associated with FSIS regulated products
Recalls
The number of recalls FSIS supports is a good indication of the effectiveness of the E. coli
O157:H7 sampling projects maintained by the Agency. Recalls occur when adulterated product
is found in commerce. A recall is a firm‘s action to remove product from commerce (e.g., by
manufacturers, distributors, or importers) to protect the public from adulterated or misbranded
products. Although it is a firm‘s decision to recall product, FSIS coordinates with the firm to
ensure the firm has properly identified and removed recalled product from commerce by
verifying the effectiveness of the firm‘s recall activities. FSIS also notifies the public about
product recalls.
81
In general, the discovery of a positive sample in FSIS testing should prevent
contaminated product from reaching the consumer marketplace. While FSIS does not currently
mandate that establishments hold product until negative test results are received, the Agency
recently requested comments on a new Federal Register Notice that would change the Agency‘s
procedures and withhold a determination as to whether meat and poultry products are not
adulterated, and thus eligible to enter commerce, until all test results that bear on the
determination have been received.
82
Therefore, when firms hold product pending FSIS test results, FSIS sampling projects can
prevent recalls from occurring. The possibility of recalls has prompted industry to increase its
capacity and willingness to hold product while it is being tested, to institute their own test and
divert programs and to ultimately contribute to the lower estimate of E. coli O157:H7 in the food
81
FSIS Directive 8080.1 http://www.fsis.usda.gov/OPPDE/rdad/FSISDirectives/8080.1.pdf
82
http://www.fsis.usda.gov/OPPDE/rdad/FRPubs/2005-0044.pdf
52
supply. Therefore, evaluating the number of recalls due to positive E. coli O157:H7 results over
time allow FSIS to, in part, evaluate the overall effectiveness of its policies, as Figure 2.1.3.3
demonstrates.
Figure 2.1.3.3: E. coli O157:H7 reported recalls, 1994-2010.
Key:
1996: HACCP rule published
1. 1999-2000—HACCP implementation complete
2: 2002—Industry to reassess HACCP with O157:H7 a hazard likely to occur, also industry started widespread test
and divert programs
3: 2007—Started MT50 and MT54
4: 2008—Switched to risk (production volume) based sampling
5: 2009—Started MT55
53
2.1.4: RTE Meat and Poultry Products (Listeria monocytogenes (Lm) and Salmonella)
Overview of Sampling Projects
FSIS conducts microbiological testing of RTE meat and poultry products for Listeria monocytogenes (Lm) and Salmonella.
83
Lm
domestic sampling projects are summarized in Table 2.1.4.1. As the projects are different, they will be described separately.
Table 2.1.4.1 FSIS RTE Domestic Sampling Projects
Product class
RTE Sampling
Projects
Pathogens
Tested
Number
of
FY2010
samples
Regulatory Purpose of
Sampling Program
Type of
Sampling
Program
Both post-lethality exposed
and non-post-lethality exposed
RTE products
ALLRTE
Lm,
Salmonella
2,990
Monitor industry performance
Random
Post-lethality exposed RTE
products
RTE001
Lm,
Salmonella
8,700
Verify non-detectable standard
Risk Based
RLm product samples
RLMPROD
Lm
1,960
Monitor industry performance
Risk Based
RLm food contact surface
samples
RLMCONT
Lm
6,600
Monitor industry performance
Risk Based
RLm non-food contact
environmental samples
(Composited 5-sample Units;
Lm)
RLMENVC
Lm
690
Monitor industry performance
Risk Based
Intensified Verification
Testing (IVT) product samples
INTPROD
Lm or
Salmonella
225
Response to positive ALLRTE, RTE001,
RLMPROD and/or RLMCONT sample
Targeted
IVT food contact surface
samples
INTCONT
Lm or
Salmonella
550
Response to positive ALLRTE, RTE001,
RLMPROD and/or RLMCONT sample
Targeted
IVT non-food contact
environmental samples
INTENV
Lm or
Salmonella
275
Response to positive ALLRTE, RTE001,
RLMPROD and/or RLMCONT sample
Targeted
In addition to Lm and Salmonella, testing for E. coli O157:H7 was performed for dry and semi-dry fermented sausages and fully cooked meat patties until April, 2011. FSIS
officially announced the discontinuation of the program in a May 13, 2011 Constituent Update which can be found at:
http://www.fsis.usda.gov/News_&_Events/Const_Update_051311/index.asp. This testing was discontinued after an analysis showed that testing over 10,000 such products for E.
coli O157:H7 over a sixteen year period (1994-2011) yielded no positive samples.
54
83
Historical Basis of Sampling Programs
Lm has been implicated in illness outbreaks since the early 1980s. FSIS has conducted a
regulatory microbiological testing program in RTE meat and poultry products since 1983. From
1983 until 2004, establishments were randomly selected for regulatory samples from different
sub-populations or from the total population of establishments producing RTE products.
84
In
1987, FSIS increased testing for Lm in regulated products, including domestic cooked meat and
poultry and imported cooked products.
85
In 1989, after a confirmed human listeriosis case was
linked to cooked poultry, FSIS identified Lm as an adulterant subject to recall if found in
86 87
commerce. After the implementation of PR/HACCP regulations in 1996, FSIS organized Lm
testing around the four HACCP processes of; 1) fully cooked, not shelf stable products, 2) heat-
treated, shelf stable products, 3) not heat-treated, shelf stable products and 4) products with
secondary inhibitors that are not shelf stable. In addition to the material provided below, a
timeline of FSIS activities related to Lm can be found on the Agency‘s website.
88
FSIS began random testing of RTE product samples in the 1990s, while risk-based testing of
RTE products for Lm began in 2005. Since the inception of the Lm verification testing program
for RTE meat and poultry products, FSIS has also sampled packaged RTE products for the
presence of Salmonella.
Type of Analysis
All samples collected for analysis in the RTE product testing projects are evaluated using
qualitative and quantitative methods. In 2006, FSIS initiated MPN analysis of positive RTE food
products identified during follow-up testing.
Volume Data
Production volume data for the ALLRTE and RTE0001 sampling projects are obtained in two
ways:
1. For all establishments with post-lethality exposure (RTE001 and RLm sampling projects),
volume information was provided on an annual basis using FSIS Form 10,240-1, as
required under Federal Regulation 9 CFR 430. This form contained the establishment's
annual production volume of post-lethality exposed RTE meat and poultry products for
89 90
each control Alternative in each of nine product categories.
2. For RTE establishments that produce RTE products with no post-lethality exposure
(which are a part of the ALLRTE sampling projects), volume information is provided on
a voluntary basis.
91
84
Please see the following website for more information: http://www.fsis.usda.gov/Science/Micro_Testing_RTE/index.asp.
85
Federal Register, Volume 52, No. 47, March 11, 1987.
86
Federal Register Volume 54, No. 98, Tuesday May 23, 1989.
87
Pathogen Reduction/Hazard Analysis and Critical Control Point System final rule (61 FR 38806, July 25, 1996).
88
Please see the following website for more information: http://www.fsis.usda.gov/PDF/Lm_Timeline.pdf
89
For Alternative 1, the establishment uses a post-lethality treatment for its product and an antimicrobial agent or process that
suppresses or limits of growth of Lm. For Alternative 2, the establishment uses either a post-lethality treatment for product
(choice 1) or an antimicrobial agent or process that suppresses or limits the growth of Lm (choice 2). For Alternative 3 the
establishment uses a sanitation program that controls Lm contamination in the processing environment and on the product.
90 th
This form was discontinued as of September 30 , 2011. Moving forward, this information will be collected through PHIS.
91
This volume information will be collected in PHIS once it has been fully implemented.
55
Thus, approximately 90% of establishments in the ALLRTE sampling project have volume
information available from the RTE001 sampling project, the exceptions (prior to PHIS) being
those establishments with no post-lethality exposure.
Current Design of Sampling Plans
FSIS conducts regulatory microbiological testing of RTE meat and poultry products for two
microorganisms: Lm and Salmonella. Currently, there are three verification testing projects for
the detection of Lm contamination: ALLRTE, RTE001 and RLm. ALLRTE and RTE001
product samples also are concurrently tested for Salmonella. Intensified Verification Testing
(IVT) is conducted in establishments with positive ALLRTE, RTE001, and RLMPROD and/or
RLMCONT (product and food contact surface) samples.
In general, all sampling projects for Lm in post-lethality exposed RTE products rely on 9 CFR
430.4 ―Control of Listeria monocytogenes in post-lethality exposed ready-to-eat products‖,
published on June 6, 2003 (68 FR 34207). Other relevant regulations/directives/notices for each
sampling program are listed after each program description.
Description of the FSIS RTE Domestic Sampling Projects for Lm and/or Salmonella
A description of each domestic sampling project for RTE meat and poultry products, food
contact surfaces and non-food contact environmental surfaces follows below.
ALLRTE
Purpose
The ALLRTE sampling project began in January 2004 and was designed to obtain random
samples across all RTE products and across all establishments producing a RTE product,
regardless of risk or product type. The ALLRTE sampling program is structured with the intent
of verifying compliance with zero tolerance for Lm in RTE products. Products are sampled for
Lm and Salmonella. Both post-lethality exposed and non-post-lethality exposed products are
tested. Currently, samples are randomly selected by FSIS.
The following FSIS policy relates to ALLRTE: FSIS Directive 10,240.4 Revision 2 ―Verification
Procedures for Consumer Safety Inspectors for the Listeria monocytogenes (Lm) Regulation and
Lm Sampling Program‖, dated February 3, 2009.
Any RTE products testing positive for Lm and Salmonella are considered to be adulterated and
subject to regulatory control. FSIS recommends that establishments hold product pending FSIS
confirmed test results so that adulterated products are not sold into commerce. If an
establishment releases a product into commerce that later confirms positive Lm or Salmonella,
FSIS recommends a recall to remove the product from commerce and requires the establishments
to rework, re-cook or condemn the product in a manner validated to destroy the adulterant. As
discussed above, FSIS recently requested comments on a new Federal Register Notice that would
change the Agency‘s procedures and withhold a determination as to whether meat and poultry
products are not adulterated, and thus eligible to enter commerce, until all test results that bear on
the determination have been received.
92
92
Please see the following website for more information: http://www.fsis.usda.gov/OPPDE/rdad/FRPubs/2005-0044.pdf.
56
Frame Definition, Frame Size, and Sample Size
The ALLRTE frame contains approximately 2,400 active RTE establishments under Federal or
Talmadge/Aiken (T/A) inspection authority,
93
based on information available in PBIS. This
value can fluctuate as new establishments become eligible and others become ineligible.
Ineligibility may result from seasonal processing, closure, withdrawal of inspection or change in
business practices. A total of 85 samples per week (4,420 per year) are selected from the frame,
with the goal of each establishment being selected at least once per year, with a number of
establishments being selected twice or more annually.
Several establishments were excluded from the ALLRTE sampling project prior to 2009 because
they produced excepted products, which include oils, shortening, lard, margarine, lard margarine,
pork skins, pork rinds, dried soup bases and mixtures of rendered animal fats and products
labeled ―for further processing,‖ among other products.
94
FSIS Directive 10,240.4, Revision 2
removed the exceptions for these products (excluding products for further processing) from the
ALLRTE sampling project. However, FSIS Notice 10-10 stated that ―oils, shortening, lard,
margarine, lard margarine, and mixtures of rendered animal fats are not to be collected for
sampling under the ALLRTE or RTE001 sampling projects because there is no validated method
for testing these products for Lm.‖
95
FSIS will continue to sample popped pork skins, pork rinds,
dried soup bases, concentrated (high salt content) soup mixes, and pickled pig‘s feet under both
RTE sampling projects. Because there is not enough information on which specific products are
produced at a given establishment at any given time, it is not possible to automatically exclude
establishments that produce such products from sample scheduling algorithms. However, FSIS
anticipates that this issue will be addressed through full PHIS implementation.
Sampling Methodology
On a monthly basis, FSIS randomly selects the samples from the frame using simple random
selection. The sample size is 85 per week. Establishments that have already been sampled once
within a fiscal year are removed from the frame and then reinstated on a cyclic basis. Samples
are selected so that all RTE establishments, regardless of HACCP size, production volume or
process design have an equal chance of being sampled each fiscal year (e.g., no weighting is
applied during sample selection).
Collection Methodology
Two pounds of product in its final packaged form are collected at the establishment and sent to
FSIS laboratories for testing.
Response Rate
In fiscal year 2010, an average of 66% of scheduled ALLRTE samples were collected and
analyzed for both Lm and Salmonella.
93
Talmadge-Aiken establishments, formally known as Federal-State Cooperative Inspection Plants, are the approximately 250
meat and poultry establishments in 10 states where USDA has contracted with state agency inspectors to conduct federal
inspection activities. Even though state employees conduct the inspections in these establishments, they are considered to be
under federal rather than state inspection Talmadge-Aiken (7 U.S.C. 450)
94
FSIS Directive 10,210.1, Amendment 6
95
Please see the following website for more information: http://www.fsis.usda.gov/OPPDE/rdad/FSISNotices/10-10.pdf.
57
RTE001
Purpose
The RTE001 sampling project is a risk-based verification testing project, implemented in
January 2005. This sampling project is used primarily to verify that establishments producing
post-lethality exposed meat and poultry products are controlling Lm and are in compliance with
the zero tolerance requirements of 9 CFR 430. In this project, products are sampled for both Lm
and Salmonella. This project is also used to calculate volume-adjusted percent positives for Lm.
The following FSIS policy relates to the RTE001 sampling project: FSIS Directive 10,240.4
Revision 2.
Any RTE products testing positive for Lm or Salmonella are considered to be adulterated and
subject to regulatory control. Currently, FSIS recommends that establishments hold product
pending FSIS confirmed test results so that adulterated products are not sold into commerce. If
an establishment releases product into commerce that later confirms positive for Lm or
Salmonella, then FSIS recommends a recall to remove the product from commerce and requires
the establishment to otherwise rework, re-cook or condemn the product in a manner validated to
destroy the adulterant. As discussed above, FSIS recently requested comments on a new Federal
Register Notice that would change the Agency‘s procedures and withhold a determination as to
whether meat and poultry products are not adulterated, and thus eligible to enter commerce, until
all test results that bear on the determination have been received.
96
Frame Definition, Frame Size, and Sample Size
The RTE001 frame contains about 2,170 active RTE establishments with post-lethality exposure
under Federal or T/A inspection authority, based on information available in PBIS. This value
can fluctuate as new establishments become eligible and others become ineligible. Ineligibility
may result from seasonal processing, closure, withdrawal of inspection and change in business
practices, among other possibilities. A total of 200 samples per week (10,400 per year) are
selected from the frame.
Sampling Methodology
Establishments are identified for sampling based on a risk-ranking algorithm, which takes into
account the Lm control Alternative,
97
the production volume, the type of product produced and
the sampling history. An establishment is selected from the frame as little as once per year to, at
most, once per month, depending on its position in the risk-ranking algorithm.
Collection Methodology
Two pounds of product in its final, packaged form are collected at the establishment, with higher
risk products given priority for collection. The order of risk for the various types of products is
provided in FSIS Directive 10,240.4, Revision 2. The current list of product types used for risk-
ranking purposes is as follows:
96
Please see the following website for more information: http://www.fsis.usda.gov/OPPDE/rdad/FRPubs/2005-
0044.pdf.
97
Control Alternative refers to any one of four procedures used to prevent (control) the growth of Lm in post-
lethality exposed RTE products. See footnote 89 for a complete description of the Alternatives.
58
1) Deli-meats that are sliced in the federal establishment,
2) Deli-meats shipped whole from the federal establishment (this does not include cook-
in-bag products; only those exposed post-lethality),
3) Hotdog Products,
4) Deli salads, pâtés, and meat spreads,
5) Fully cooked type products (other than cooked products in 1-4 above),
6) Fermented products,
7) Dried products,
8) Salt-cured products, and
9) Products labeled as "Keep Frozen".
Response Rate
In fiscal year 2010, the percent of RTE001 samples that were collected and analyzed for both Lm
and Salmonella averaged 83%.
RLm
Purpose
The RLm sampling project, implemented in April 2006, is a routine, risk-based sampling project,
which consists of food contact samples, non-food contact environmental samples, and product
samples taken during the production of RTE meat and poultry products exposed to the post-
lethality environment. Samples are analyzed only for Lm and are taken during the same day of
production. In conducting the RLm project, FSIS assesses the compliance of establishments
with regulation 9 CFR 430 regarding the control of Lm in post-lethality exposed RTE production
areas and ensures that RTE products are safe for consumption at the end of the production
process.
RLm sampling, done in conjunction with an FSA, provides an in-depth evaluation of the
effectiveness of the food safety practices employed by an establishment. The ability to use the
product, contact and environmental data collected from the establishments can help identify
possible risk factors that could be associated with positive results.
The following FSIS policy relates to the RLm sampling project: FSIS Directive 10,240.5
Revision 2.
Frame Definition, Frame Size, and Sample Size
The RLm frame is identical to that of RTE001. It contains about 2,170 active RTE
establishments with post-lethality exposure under Federal or T/A inspection authority, based on
information available in PBIS. This value can fluctuate as new establishments become eligible
and others become ineligible. Ineligibility may result from seasonal processing, closure,
withdrawal of inspection and change in business practices, among other possibilities. In 2009,
FSIS policy was to require an RLm/FSA in every establishment with post-lethality exposure at
least once every four years. Pursuant to an Office of the Inspector General
(OIG) mandate,
starting in August 2009, a sample size of 45 establishments per month (540 per year) is selected
from the frame.
59
Sampling Methodology
RLm establishments are selected from the frame using the FSA prioritization model, which takes
into account FSIS‘ public health decision criteria,
98
control alternative and type of product
produced. The RLm project consists of the following three concurrent sampling projects:
1. RLMPROD—routine, risk-based testing of intact RTE food product samples throughout
the selected production shift; three samples are collected per sampling unit.
2. RLMCONT— routine, risk-based testing of surfaces that have direct contact with RTE
products in the RTE production area (e.g., conveyor belts, storage racks, slicer blades,
loaders and table tops).
3. RLMENVC—routine, risk-based testing of environmental (non-food contact) surfaces in
the RTE production areas (e.g., floors, drains, walls and floor mats). Starting in August
2009, environmental samples were composited at the testing labs.
Collection Methodology
Microbiological samples are collected as units. A unit consists of 10 food contact surface swabs
(RLMCONT), five environmental swabs, which are later composited at the testing laboratories
(RLMENVC), and three intact product samples (RLMPROD). In establishments that use brine
chillers, the FSIS Enforcement Investigations and Analysis Officers (EAIO) collect a sample of
brine from each line using a brine chiller. Brine samples could be either RLMENVR
(environmental/not composited) if the product package is impermeable or RLMCONT if the
product package is permeable. The number of units per establishment is based on HACCP size.
FSIS collects three sample units from large HACCP establishments, two sample units from small
establishments and one sample unit from very small establishments.
99
Response Rate
In fiscal year 2010, the average response rate for RLm samples (combined RLMPROD,
RLMCONT and RLMENVR/RLMENVC units) was 93%.
Intensified Verification Testing (IVT)
Purpose
IVT is a follow-up, targeted sampling project, where FSIS tests product, food contact surfaces,
and environmental surfaces for either Salmonella or Lm. An IVT is initiated after an
establishment has either a positive Salmonella or Lm result in either finished product (ALLRTE,
RTE001 and RLMPROD) or on a food contact surface (RLMCONT). An IVT can also be
initiated at the discretion of an FSIS District Manager, in response to continuing sanitation non-
compliances at the establishment. The IVT is performed after the establishment has taken
corrective and preventative measures in response to FSIS‘ findings. As described above for the
RLm program, in an IVT, FSIS collects samples in units. As with RLm, IVTs are performed
with a FSA to provide an in-depth evaluation of food safety systems at the establishment.
However, the FSA is conducted for-cause, rather than being routine in nature. A maximum of
five units in a given establishment are considered per IVT.
98
FSIS‘ Public Health Decision Criteria can be found at:
http://www.fsis.usda.gov/OPPDE/NACMPI/Sep2010/2010_Public_Health_Decsion_Criteria_Report.pdf.
99
HACCP Establishment Sizes are as follows: Large; 500 or more employees, Small: 10-499 employees, and Very
Small: < 10 employees and annual sales >2.5 million.
60
The following FSIS policy relates to the IVT sampling project: FSIS Directive 10,300.1
―Intensified Verification Testing (IVT) Protocol for Sampling of Product, Food Contact Surfaces
and Environmental Surfaces for Listeria monocytogenes,‖ dated February 3, 2009, which
provides instructions for performing an IVT.
Frame Definition, Frame Size, and Sample Size
There is no sampling frame for IVT, as it is dependent on a positive Lm or Salmonella ALLRTE,
RTE001, RLMPROD and/or RLMCONT sample from a given establishment.
Sampling Methodology
IVTs are scheduled in accordance with FSA prioritization criteria from FSIS Directive 10,200.1,
with all establishments with Lm or Salmonella-positive ALLRTE, RTE001, RLMPROD and
RLMCONT samples requiring an IVT. The FSIS districts have 30 days in which to schedule the
IVT.
Collection Methodology
A
s described above for the RLm project, IVT microbiological samples are collected in units. A
sampling unit for Lm consists of ten food contact surface samples, five environmental samples
and three product samples per RTE processing line in operation on the day of sampling, whereas
a unit for Salmonella consists of eight food contact surface samples, five environmental samples
and five product samples per processing line. If the establishment uses a brine chiller, FSIS will
also collect one brine sample per line from the brine chiller.
Response Rate
In fiscal year 2010, the IVT response rates in response to Lm- and/or Salmonella-positive
ALLRTE and RTE001 samples were 100% and 71%, respectively. IVT for positive RLMPROD
and/or RLMCONT samples was not implemented until October 2009 with the issuance of FSIS
Notice 62-09. The RLm response rate was 56% for the fiscal year.
Limitations of Current Sampling Programs
FSIS has identified several limitations to the current Lm sampling projects; namely sampling
rates not being met for specific establishments, sampling frequency, project overlaps, sampling
biases, volume-weighted percent positive results and regulatory considerations. These issues
will be discussed in detail below.
1. Sampling Rates for Establishments
One objective of the ALLRTE and RTE001 sampling projects is to sample every RTE
establishment (ALLRTE) and every RTE establishment with post-lethality exposure (RTE001) at
least once each year. In FY 2011, virtually all of the approximately 2,400 active RTE producing
establishments were scheduled for collection in one of the three RTE verification testing projects
(ALLRTE, RTE001 and/or RLm). However, not every establishment is sampled annually and a
small number of establishments were not sampled at all between 2005 and the present. The
reasons for this are varied. In the case of ALLRTE, some establishments produce products such
as popped pork skins, which were previously exempt from testing. In other instances, an
establishment may be producing seasonally and could not be scheduled during a period of
61
production. As of October 2011, approximately 98% of all RTE establishments were sampled at
least once over the calendar year.
2. Sampling Frequency
For ALLRTE, the random monthly sampling of scheduled RTE product producing
establishments, plus the inclusion of all establishments in the sampling frame, may permit a
comparative annual Lm positive rate in FSIS inspected establishments for an aggregate of the
RTE products collected. Because the specific product to be collected is determined by IPP,
specific products may be either over or under-sampled in relation to national production. For
RTE001, higher risk establishments are scheduled more frequently. Accordingly, positive rates
may vary as a function of the samples collected and tested from the scheduled establishments,
resulting in a different measure of aggregate positive rates.
3. Project Overlaps
FSIS acknowledges that the RTE001 and ALLRTE projects exhibit a high degree of overlap due
to independent scheduling. As the ALLRTE is independent of RTE001, sampling results are not
currently combined, even though a single establishment may be sampled in both projects in a
given month. However, sampling of the same establishment in the ALLRTE and RTE001
projects in the same month often results in only the risk-based (RTE001) sample being collected,
which can cause a non-response bias.
100
4. Sampling Biases
RTE001 data are biased towards high-risk products based on the program structure. There may
be a similar, though less pronounced bias in the ALLRTE data, as historical instructions to the
field allowed for sampling of higher-risk products in ALLRTE. Regardless, for both ALLRTE
and RTE001, no mechanism exists for truly randomizing what products are collected at a given
establishment over time. These issues may arise from such factors as samples not being
collected, lack of random product selection at the establishment level and lack of randomness in
sample selection from the frame (see also project overlaps above).
5. Volume-Weighted Percent Positive Results
The ALLRTE and RTE001 percent positive numbers were standard FSIS performance measures
for the Agency‘s annual Performance and Accountability Report (PAR) for Lm until 2008; these
have subsequently been replaced by volume-weighted positive rates. Such rates attempt to take
into account the proportion of national volume represented by an individual sample, but do not
adjust for missing establishments, missing, over or under-represented products or sampling bias
not related to the design of the project.
6. Regulatory Considerations
RTE projects were developed as a result of regulatory activities. Consequently, changes to the
current projects may require reissuance of existing policies or issuance of new policies.
100
FSIS has attempted to reduce this bias by issuing instructions for the ALLRTE sampling program in Directive 10240.4 stating
that ―Consumer Safety Inspectors (CSIs) should make every effort to sample all the RTE products produced at an establishment
by rotating through the products when CSIs receive sample request forms.‖
62
Lm Sampling Program Measures of Success
There are several different Lm sampling projects, each with slightly different goals. Yet, the
overall purpose of Lm sampling program is structured with the intent of verifying compliance
with zero tolerance for Lm and Salmonella in RTE products.
Consequently, FSIS believes it is appropriate to measure the success of the different sampling
projects in broad terms, rather than focusing solely on volume-adjusted percent positive rates.
As such, to measure the success of the Lm sampling projects, FSIS evaluates its efforts in terms
of three key metrics;
1. Volume-weighted percent positives from Lm sampling projects,
2. Estimated number of Lm foodborne illnesses associated with FSIS-regulated products,
and
3. Number of Lm recalls.
Percent Positive Rates
FSIS conducts pathogen verification testing for the Agency‘s sampling programs. As described
in this report, FSIS samples product regulated by the Agency to verify HACCP policy and to
assess, and minimize, the risk to public health from contaminated product. FSIS uses this
sampling to calculate a positive rate for many of the Lm sampling projects. FSIS believes that
positive rates are a good measure of the effectiveness or success of the Lm sampling projects
maintained by the Agency, with declines in percent positives potentially indicating greater
control and prevention of Lm in RTE and meat and poultry products.
Table 2.1.4.2 provides the production category volume-weighted percent positive rate for the
RTE001 project and the ALLRTE project.
Table 2.1.4.2: Quarterly Volume-Weighted Percent Positive Rates for Lm Sampling
Projects
Year/Quarter
Volume-Weighted
Percent Positive
(ALLRTE)
Volume-Weighted
Percent Positive
(RTE001)
FY 2009
0.10%
0.24%
FY10Q3
0.01%
0.18%
FY10Q4
0.00%
0.10%
FY11Q1
0.00%
0.10%
FY11Q2
0.03%
0.14%
FY11Q3
0.04%
0.14%
Foodborne Illness Estimates:
As FSIS‘ ultimate goal is to prevent foodborne illnesses from regulated products, it is important
to measure reductions in foodborne illness as a result of FSIS inspections, sampling, policies and
other activities. FSIS calculates a performance measure, known as the All-Illness Measure,
which represents all foodborne Salmonella, Lm and E. coli O157:H7 illnesses from FSIS-
regulated meat, poultry and processed egg product. FSIS updated the All Illness Measure in Q3,
63
FY2011 to reflect the release of new illness burden estimates from the CDC
101
and the Healthy
People 2020 goals
102
, as well as to coincide with the release of the FSIS Strategic Plan for 2011-
2016. Objectives for the All-Illness measure were set using a combination of data from
published CDC FoodNet case rates and outbreak data and are aligned with Healthy People 2020
goals. For Lm, FSIS uses a rolling 12 month window of case rate data from the CDC, in addition
to an attribution estimate, to estimate the total number of Lm illnesses from FSIS regulated
products. Using this methodology, the illness measure is the estimate of the total annual illnesses
for the fiscal year, rather than independent measures of illness for each quarter.
Performance Measure
Using the newly updated All Illness Measure data sources and methodology, FSIS set a target of
reducing the estimated Lm illnesses associated with FSIS regulated products to 866 in Q3
FY2011; FSIS achieved that target with 718 estimated illnesses. Figure 2.1.4.1 illustrates the
quarterly targets for Lm illnesses and the estimated illnesses for FY2010 and FY2011.
Figure 2.1.4.1: Estimated Listeria monocytogenes illnesses from FSIS regulated products
Recalls
The number of recalls FSIS supports is a good indication of the effectiveness or success of the
Lm sampling projects maintained by the Agency. Recalls occur when a FSIS regulated product
is found in commerce. A recall is a firm‘s action to remove product from commerce (e.g., by
manufacturers, distributors or importers) to protect the public from consuming adulterated or
misbranded products. Although it is a firm‘s decision to recall product, FSIS coordinates with the
firm to ensure it has properly identified and removed recalled product from commerce by
101
Scallan E, Hoekstra RM, Angulo FJ, Tauxe RV, Widdowson M-A, Roy SL, et al. Foodborne illness acquired in the United
States—major pathogens. Emerg Infect Dis [serial on the Internet]. 2011 Jan [November 2011].
http://www.cdc.gov/EID/content/17/1/7.htm
102
Please see the following website for more information:
http://www.healthypeople.gov/2020/topicsobjectives2020/objectiveslist.aspx?topicId=14.
64
verifying the effectiveness of the firm‘s recall activities. FSIS also notifies the public about
product recalls.
103
In general, the discovery of a positive sample in FSIS testing prevents
contaminated product from reaching the consumer marketplace. Consequently, FSIS sampling
programs can prevent recalls from occurring. Additionally, FSIS recently announced a new
Federal Register Notice to encourage establishments to hold product while testing is underway to
prevent contaminated product from reaching the marketplace. Further, evaluating the number of
recalls over time allows FSIS to evaluate, in part, the effectiveness of its policies, as Figure
2.1.4.2 demonstrates.
Figure 2.1.4.2: Reported FSIS recalls for Products Contaminated with Lm, 1994-2010.
103
FSIS Directive 8080.1, Revision 6. http://www.fsis.usda.gov/OPPDE/rdad/FSISDirectives/8080.1.pdf
65
Section 2.2: FSIS Microbiological Baseline Data Collection
In 1992, FSIS began a concerted effort to identify the microbiological profile of various classes
of inspected raw meat and poultry carcasses and ground product. The purpose was to use the
data to establish both pathogen reduction performance standards (for carcasses and ground
product) and process control performance criteria (for carcasses). The pathogen reduction
performance standards were intended to spur industry to control the presence of pathogens of
public health concern, particularly Salmonella. The process control performance criteria were
designed to reflect the prevalence and levels of contamination of E. coli (Biotype 1) on
carcasses, as an indicator of process control for fecal contamination. In designing the baselines,
FSIS intended to capture microbiological profile data for microorganisms of varying degrees of
public health concern, and organisms or groups of organisms of value as indicators of general
hygiene or process control. In addition, FSIS stated its expectation to repeat the baseline studies
over time to document changes. Although microbiological data were collected by FSIS prior to
1992, these earlier efforts were not conducted as part of anticipated rulemaking. A more
complete description of the various microbiological baseline data collection efforts and their
design considerations can be found in the preambles to the proposed and final rules associated
with the PR/HACCP system rulemaking effort.
104
The carcass baselines, conducted periodically, were specifically designed to identify nationwide
changes in the prevalence of Salmonella, expressed as a percentage of contaminated carcasses.
Enough samples were taken to describe the annual distribution of test results and to account for
possible seasonal variation, as well as provide for missing samples and incomplete data. These
baselines were targeted at the major classes of livestock and poultry slaughtered, comprising
approximately 95 percent of all livestock slaughtered, and approximately 99 percent of domestic
birds slaughtered. The carcass baselines were originally designated as data collection
―programs‖ because of the scope and length of time for the data collection effort, generally
comprising at least one year. By contrast, the ground product baselines were originally
designated as data collection ―surveys.‖ The surveys reflected the prevalence of Salmonella
(expressed as a percentage of positive samples) as a snapshot over a short period of time,
generally limited to a six month data collection effort. In the preamble to the PR/HACCP final
rule, FSIS referred to both data collection efforts, collectively, as FSIS baseline surveys.
From the FSIS baseline surveys, the prevalence for Salmonella was used for setting the
qualitative pathogen reduction performance standards. However, for the process control
performance criteria, a statistical procedure known as a ―3-class attributes sampling plan‖
applied in a moving window was used. Consequently, the ―m‖ and ―M‖ criteria were set at the
th th
closest power of 10 to the actual numbers estimated for the 80 and 98 percentiles from the
FSIS baseline surveys. These criteria are quantitative (colonies per square centimeter of carcass
surface area).
The data generated from FSIS baseline surveys continue to be used to set pathogen reduction
performance standards, and to inform design of FSIS regulatory testing programs, as well as in
guidance to industry related to effective process control. Since the time that the PR/HACCP
final rule was implemented, FSIS has also used FSIS baseline survey data in risk assessments to
predict the public health impact of risk mitigation strategies. During the most recent poultry
104
60 FR Federal Register 6774, February 3, 1995, and 61 Federal Register 38806, July 25, 1996.
66
carcass baseline surveys, a method for isolating Campylobacter was developed and validated and
has since been adopted by the agency. Some of the commodities for which baseline surveys have
been conducted since the original FSIS baseline surveys supporting the PR/HACCP final rule
include: beef trim, young chicken carcasses and young turkey carcasses. The most recent FSIS
baseline surveys underway include market hogs and chicken parts. A second FSIS baseline
survey of unpasteurized liquid egg product is underway with the intent of using the data to
establish lethality performance standards for pasteurized egg product.
Section 2.3 Chemical Residues
Overview of Sampling Projects
FSIS conducts testing for chemical residues in regulated meat, poultry and processed egg
products. Domestic sampling projects are summarized in Table 2.3.1.
Table 2.3.1: Residue Sampling Projects
Residue
Sampling
Projects
Number of
Residue
Analyzed
samples
FY2010
Regulatory
Purpose of
Sampling
Program
Routine-NRP
1
14,929
Chemical
Residue
Exposure
Assessment-
Random
KIS™ Test-Field
157,524
Targeted
KIS™ Test –Lab
2
8,041
Targeted
FAST-Field
47,676
Targeted
FAST-Lab
291
Targeted
1. National Residue Program samples for meat, poultry and processed egg products, as well as residue monitoring
and inspector generated samples.
2. Verification/confirmation sampling conducted by the FSIS Laboratories. Includes confirmatory KIS™ tests on
field positives.
Background Information
Since 1967, FSIS has administered the United States National Residue Program (NRP). FSIS
collects samples of raw meat, poultry and processed egg products, as well as imported product
and analyzes the samples at one of the three FSIS laboratories. The NRP is designed to detect
contamination of meat, poultry and processed egg products with residual veterinary drugs,
pesticides and heavy metals. Under this program, FSIS inspectors sample meat, poultry and
processed egg products in slaughter and processing establishments for chemical residues and
compare, when applicable, the results with tolerances established by the Food and Drug
Administration (FDA) and the Environmental Protection Agency (EPA) to prevent adulterated
meat, poultry and processed egg products from entering into commerce. The NRP is an
interagency program operating under a 1984 Memorandum of Understanding. There are three
interagency workgroups that coordinate residue sampling operations: 1) the Interagency Residue
67
Control Group (IRCG), 2) the Surveillance Advisory Team (SAT), and most recently, 3) the
Senior Executive Council (SEC).
The IRCG meets about once a month to discuss all pertinent residue testing issues and is
attended by chemical residue subject matter experts from the USDA (FSIS, AMS and ARS),
FDA and EPA. The SAT meets once a year and identifies the priority public health residues of
concern. FSIS then develops specific sampling plans, which guide the allocation of FSIS‘
laboratory and inspection resources. In response to a 2010 OIG recommendation to have a
process for elevating issues and resolving broader policy issues, the SEC was formed to ensure
that senior level management meet regularly to resolve long-standing chemical residue issues.
Each year, FSIS publishes the ―National Residue Program Scheduled Sampling Plans‖ (the Blue
Book) as a means of informing stakeholders about the sampling program and ―National Residue
Program Data‖ (the Red Book) as a means of reporting the results of the Agency‘s NRP testing.
The Blue Book provides a description of the principles and methods used to design the sampling
plans for the NRP and summarizes the planned domestic and import sampling plans on a
calendar year basis. The Red Book presents details on the testing results of the various NRP
sampling projects conducted throughout the calendar year. The Blue Book also includes a
summary of adjustments to the previous year‘s NRP.
105
Taken together, these books provide a
comprehensive view of the program and the analyses of the data.
An important component of the NRP is to provide verification of chemical residue control in
HACCP systems. As part of the HACCP regulation, slaughter and production establishments are
required to identify all chemical residue hazards that are reasonably likely to occur and develop
systems that prevent, eliminate or minimize these hazards. A vigilant chemical residue
prevention program is essential to ensure the prudent use of veterinary drugs and pesticides in
food animals and is a complement to the NRP.
A violation occurs when a FSIS laboratory detects the presence of a chemical compound or a
given compound level in excess of an established tolerance or action level in a sample. FSIS
shares violation findings with FDA through the Residue Violation Information System (RVIS).
To assist FDA in investigating producers at an on-farm level, FSIS obtains the names of
producers and other parties involved in offering the animals for sale. FSIS informs producers
through certified letters that a product from their animal tested positive for violative chemical
residues and that they will be considered repeat violators if associated with more than one
violation. FSIS also maintains Residue Repeat Violator Lists for FSIS field personnel,
establishments and livestock markets to help identify producers with more than one residue
violation in the last 12 months either in the same establishment or different establishments.
106
Intent of Sampling Program
The NRP is designed to: 1) provide a structured process for identifying and evaluating chemical
compounds of concern in food animals; 2) analyze chemical compounds of concern; 3) collect
105
Information on the National Residue Program can be obtained from the FSIS website at:
http://www.fsis.usda.gov/Science/Chemistry/index.asp#nrp.
106
Please see the following website for more information: http://www.fsis.usda.gov/science/chemistry/index.asp.
68
and report results and 4) provide appropriate regulatory follow-up when violative levels of
chemical residues are found.
Current Design of Sampling Program
Domestic chemical residue sampling consists of scheduled and inspector-generated sampling.
Scheduled sampling plans consist of random sampling of tissue from food animals that have
passed ante-mortem inspection. The development of scheduled sampling plans is a process that
proceeds in the following manner: 1) determine which compounds are of food safety concern; 2)
use algorithms to rank the selected compounds; 3) pair these compounds with appropriate
production classes and 4) establish number of samples to be collected. The annual SAT provides
an interagency forum to discuss prioritization of chemical hazards, as well as to determine the
compound/production class pairs to be sampled. Inspectors receive the scheduled requests for
sampling from FSIS headquarters.
Inspector-generated sampling is conducted by in-establishment Public Health Veterinarians
(PHVs) or a designated and trained IPP. If the PHV/IPP believes an animal may contain
violative levels of chemical tissue in any of its edible tissues, the inspector may use his/her own
judgment in collecting a sample, including guidance from FSIS Directives 10,800.1 and
10,220.3. When an inspector-generated sample is collected, a residue quick test (Kidney
Inhibition Screen (KIS)
TM
test or Fast Antimicrobial Screen Test (FAST)) is performed on tissue
collected from the suspect animal. If the KIS
TM
or FAST is positive, muscle, liver and kidney
tissue from the positive animal is sent to the FSIS laboratory and the carcass is held pending
confirmation from the laboratory. If a carcass or parts of the carcass is found to contain violative
levels of chemical residues, the carcass, parts or both are condemned.
Objectives of Sampling
The NRP consists of two different types of projects. First, the FSIS chemical residue control
projects seek to: 1) Monitor the occurrence of meat, poultry and processed egg products
contaminated with chemical residues; 2) Document the use, non-use or misuse of certain
compounds and 3) Maintain equivalency status with international trading partners. Second, the
NRP conducts exploratory assessments, which seek to determine the identity and the
concentrations of a particular chemical residue that may be in meat, poultry and processed egg
products, such as melamine in baby food or dioxin in chicken products.
Statistical or Policy Basis for Current Sampling Programs
The FSIS domestic scheduled sampling program consists of random sampling of tissue from
food animals that have passed ante-mortem inspection. Since 2006, FSIS has selected 300
samples for each compound/production class pair to provide a 95 percent assurance that with
zero violations in the samples, the violation rate in the entire population for a particular chemical
or chemical compound is less than one percent. If one or more violations are found in the 300
samples for each compound/production class pair, then the violation rate is one percent or more.
Description of the FSIS Residue Sampling Projects
Fast Antimicrobial Screen Test (FAST)
Historical Basis
When FSIS suspects, based on herd history or ante-mortem or post-mortem examination, that
animals may have illegal levels of antimicrobial drug residues, the Agency conducts an in-
69
establishment screening test to determine whether IPP will need to submit a sample to an FSIS
laboratory for further testing. The FAST is one of the biological screening tests used for the
detection of antimicrobial residues in animal tissues. FAST has been validated for use in testing
swine and cattle for antimicrobial residue levels and is performed by a veterinarian or a
designated food inspector in a slaughtering establishment. FAST is an adaptation of the
antimicrobial screening test that was used in FSIS laboratories for many years. FAST replaced
the Swab Test on Premises (STOP) in-establishment screen for testing in livestock, including
sheep, goat and horses.
Purpose/Intent of the Project
FAST is an in-establishment screen performed by in-establishment personnel as part of a
targeted testing project. FAST testing is necessary in problematic slaughter classes or
subpopulations of these classes (those with a high prevalence of antimicrobial residue violations)
and helps to detect carcasses with violative antimicrobial residues so they cannot enter the food
supply. It is also used to more closely monitor producers and others who are known historically
to have marketed animals with violative concentrations of antimicrobial residues. Further, the
FAST is used to determine whether establishment noncompliances have been corrected and to
verify the performance of an establishment‘s HACCP system in preventing or eliminating
chemical (residue) hazards.
Statistical or Policy Basis
Targeted testing in establishments allows FSIS to verify that establishments have adequate
residue control projects. FSIS IPP are instructed to perform in-establishment screening when
they suspect animals (ante-mortem inspection) or carcasses (post-mortem inspection) have
violative levels of chemical residues in their tissues. FSIS Directive 10,220.3 provides a list of
pathologies and conditions warranting sampling and retention for in-establishment testing. As
this screen is intended to target animals suspected of having violative residue levels and testing is
at the discretion of field personnel, the FAST project is not statistically based.
Limitations of Sampling
FAST materials are no longer being produced, and FSIS laboratories are maintaining supplies
until the Agency has completely phased in the KIS™ Test.
Functionally, FAST will screen for approximately 20 antibiotics and is not as sensitive for many
of these drugs as the KIS™ Test, which is described below. For example, FAST does not detect
Non-Steroidal Anti-Inflammatory Drugs (NSAIDS), a common class of pharmaceuticals used in
cattle.
Kidney Inhibition Screen (KIS™) Test
Historical Basis
In mid-2007, the only company supplying the FAST kits discontinued its contract with FSIS.
FSIS thus had to seek other options to continue in-establishment screening for residues and
selected the KIS
TM
test produced by Charm Sciences, Inc. A pilot trial was successfully
completed in six bovine establishments simulating real-life situations and the FSIS Midwest
laboratory confirmation validated this new test. The Charm KIS™ Test is an antibiotic detection
test for kidney tissue and the principle of detection is microbial inhibition. Bacteria, cultured in
70
agar with purple pH indicator media and kidney extract, generate acid that produces a yellow
color. In the presence of antibiotic, the bacterial growth is inhibited, and the test remains
blue/purple. KIS™ Test screening was implemented in the highest slaughter volume cattle
establishments in July 2009 and expanded to cover all bovine slaughter establishments in 2010.
Purpose/Intent of the Project
The KIS™ Test is an in-establishment screen performed by IPP as part of a targeted testing
program. KIS™ testing is necessary in problematic slaughter classes or subpopulations of these
classes (those with a high prevalence of antimicrobial residue violations) and helps to detect
carcasses with violative antimicrobial residues so they cannot enter the food supply. It is also
used to more closely monitor producers and others who are known historically to have marketed
animals with violative concentrations of antimicrobial residues. Further, KIS™ testing can be
used to determine whether establishment noncompliances have been corrected and to verify the
performance of an establishment‘s HACCP system in preventing, eliminating or minimizing
chemical (residue) hazards.
Statistical or Policy Basis
Targeted testing in establishments is a means for FSIS to verify that establishments have
adequate residue control projects. FSIS field personnel are instructed to perform in-
establishment screening when they suspect animals (ante-mortem inspection) or carcasses (post-
mortem inspection) to have violative levels of chemical residues in tissues. Directive 10,220.3
provides a list of pathologies and conditions warranting sampling and retention for in-
establishment testing.
107
As in-establishment field screen tests are intended to target animals
suspected of having violative residue levels and testing is at the discretion of field personnel, the
KIS™ testing program is not statistically based.
Limitations of Sampling
Similar to the FAST sampling program, the KIS™ program does not screen for NSAIDS.
Overall Limitation of Current Residue Sampling
The current algorithm for the annual sampling plan has been unchanged for approximately ten
years and contains variables (measured qualitatively) that may no longer be appropriate measures
for prioritizing hazards. In addition, the scheduling algorithm is a ―one size fits all‖ strategy that
determines the number of samples collected, regardless of product class/compound pairing,
geographical area or seasonal trends. In an attempt to reduce oversampling issues, several ad
hoc rules have been created to manage the scheduling algorithm, which reduces the random
nature of the program. Finally, there are continual complaints that the NRP is a resource
intensive sampling program that provides FSIS with minimal information on the true chemical
residue burden in Agency regulated products and is structured in such a manner that the program
is slow to respond to emerging residue issues.
107
FSIS is in the process of revising Directive 10,220.3 to incorporate the implementation of KIS™ testing.
71
3.0 Imports
Section 3.1: Microbiological Sampling Programs
Overview of Sampling Programs
The U.S. imports over three billion pounds of meat, poultry and egg products annually.
108
All
shipments of meat and poultry and processed egg products that enter the U.S. must be presented
to an FSIS inspector at one of the approximately 130 official FSIS import facilities located at
major ocean ports and land border crossings. One hundred percent of imported product entering
the U.S. is reinspected by FSIS at the point-of-entry (POE) for that product and every shipment
is examined for proper documentation and box count, general condition, labeling and
transportation damage.
In addition to these POE verifications, FSIS performs random reinspections on shipments of
meat, poultry and processed egg products. The POE random reinspection activities include
physical product examinations, condition-of-container reinspections and laboratory testing (e.g.,
microbiological sampling, food chemistry analysis, species verification and chemical residue
testing). This process is assisted by FSIS‘ Automated Import Inspection System (AIIS), a
centralized computer database that generates and stores reinspection results. Acceptable products
are marked as ―Inspected and Passed‖ and released into U.S. commerce. Non-compliant
products are marked as ―Refused Entry‖ and prohibited from entering U.S. commerce. More
intensive reinspection is automatically applied to subsequent product shipments from a foreign
establishment that produces products failing reinspection.
FSIS‘ POE verifications and reinspections involve evaluation of products that have first been
inspected under an equivalent food safety system established by the exporting country. Thus,
FSIS‘ POE activities are intended to monitor the effectiveness of exporting countries‘ inspection
systems and overall food safety programs. Reinspections are one component of FSIS'
comprehensive quality assurance/quality control process designed to ensure the equivalence of
exporting country‘ food safety systems.
Sections 3.1, 3.2, and 3.3 describe FSIS‘ POE pathogen testing programs. Section 3.4 describes
FSIS‘ chemical residue testing in imported regulated products.
Please see the following website for more inforamation:
http://www.fsis.usda.gov/factsheets/importing_meat_poultry_egg_products/index.ASP
.
72
108
3.1.1: Salmonella
FSIS maintains a formal sampling project for Salmonella in imported processed egg products. This program is described in Table
3.1.1.1 below. FSIS also tests for Salmonella in RTE products as part of the IMVRTE program—this project is described in Section
3.3.
Table 3.1.1.1: FSIS Sampling for Salmonella in Imported Product
Product
Type/Class
Sampling
Project
Total
Number
of
Samples
Analyzed
(Failures)
Average
Number of
Samples
Collected Per
Establishment
Number of
Establishments
Included in
Sampling
Population
Regulatory
Purpose of
Sampling
Program
Type of
Sampling
Program
Pasteurized
imported
liquid,
frozen or
dried
products
(POE
Sampling)
EGGIMP
74 (0)
Not
Applicable
Not Applicable
Component
of FSIS' on-
going
equivalence
verification
program
Performance
-Based
73
EGGIMP—Processed Egg Products
Historical Basis and Overview:
The imported processed egg products sampling program is a carry-over from when the AMS
administered the program. Up until October 1, 2006, whenever a lot of pasteurized dried, liquid
or frozen egg product was presented for import reinspection, a product examination and a
Salmonella sample were assigned. After that date, a policy decision was made that processed
egg products examinations would be consistent with the meat and poultry testing requirements.
Volume Data
Import volumes (number of lots presented for reinspection and presented net weights) are
reported and tracked by exporting country, foreign establishment, species, process category and
process sub-category within AIIS. When PHIS is implemented, process subcategories will be
expanded and replaced with product category and product group. FSIS compiles this
information and utilizes it in determining sample sizes for the various import sampling programs.
Statistical or Policy Basis
The current processed egg product sampling project utilizes the same approach that FSIS uses to
determine the required number of product examinations to be performed in imported meat and
poultry products. Under this program, the number of product examinations performed annually
is based on the number of lots of product imported annually. When the meat and poultry
procedures were implemented for processed egg products, a product examination and a
Salmonella sample were randomly assigned in advance of the processed egg products shipment
arrival at a FSIS regulated import facility. Thus, this was the beginning of ―Skipped‖ lot
sampling in imported processed egg products.
Sample Sizes
Annual sample sizes for the various import reinspection programs are based on the average
number of lots presented in the previous two years and the country history by each eligible
country for the applicable species/process category combination.
Limitations of Current Sampling
Currently, imported processed egg product sampling is not a part of the AIIS and is not based on
a statistical methodology specifically related to processed egg products food safety.
74
3.1.2: E. coli O157:H7
There are two primary E. coli O157:H7 sampling projects for imported products;
1) Raw ground beef (MT08)
2) Raw, non-intact beef (MT51)
Please see Table 3.1.2.1 for more details.
Table 3.1.2.1: FSIS E. coli O157:H7 Sampling Projects for Imported Products
Product
Type/Class
Sampling
Project
Total
Number
of
Samples
Analyzed
(Failures)
Average
Number of
Samples
Collected Per
Establishment
Number of
Establishments
Included in
Sampling
Population
Regulatory
Purpose of
Sampling
Program
Type of
Sampling
Program
Component
Imported raw
of FSIS' on-
ground beef
(POE
MT08
23 (1)
Not Applicable
Not Applicable
going
equivalence
Performance-
Based
Sampling)
verification
program
Trim and other
Component
raw ground
of FSIS' on-
beef
components
MT51
695 (2)
Not Applicable
Not Applicable
going
equivalence
Performance-
Based
(POE
verification
Sampling)
program
75
Ground Beef (MT08)
Current Design
The current sampling project is based on a desired number of Normal level samples to be
assigned and analyzed over a given calendar year for each country. The samples are divided and
allocated based on the amount of raw ground beef/veal product the country has exported to the
U.S. over the past 24 months. This approach results in approximately 25 samples scheduled
annually for MT08, a number that was calculated by FSIS in 2008. Because of the small number
of lots of raw ground beef imported annually, this number has remained approximately static.
When the FSIS identifies a foreign establishment as a ―Multi-source‖ positive E. coli O157:H7
supplier in STEPS, FSIS management places the foreign establishment on Increased inspection.
Foreign establishments may also be placed on Increased inspection as a result of a management
decision triggered by other concerns, such as failure to present.
109
Under Increased inspection,
the AIIS is programmed to assign samples for E. coli O157:H7 to a minimum of the next 15
consecutive lots of applicable beef/veal product. If all samples are negative, the Increased level
is removed from the AIIS and sampling returns to Normal.
Similarly, when a foreign establishment or country is identified during a U.S. audit as having
issues, FSIS may place the country or establishment on Increased inspection. The sampling rate
is determined by FSIS. The Increased level is removed when the objective has been met, or by
management decision and sampling returns to Normal.
When a positive E. coli O157:H7 sample is reported, the AIIS is programmed to place the
foreign establishment that produced the product on an Intensified level of inspection. This
means that, at a minimum, the next 15 consecutive lots of raw ground beef/veal and 15 times the
weight of the failed lot are assigned E. coli O157:H7 sampling. If all samples are negative, the
Intensified level is removed from the AIIS and sampling returns to Normal.
Limitations of Current Sampling
The statistical power of the MT08 project is limited by the small number of lots of raw ground
beef imported into the U.S. annually.
Non-Intact Beef (MT51)
Current Design
The project is based on a desired number of Normal level samples to be assigned and analyzed
over a given calendar year for each country. The samples are divided up and allocated based on
the amount of beef/veal trimmings a country has exported to the U.S. over the past 24 months.
This resulted in 356 samples annually, a number calculated by FSIS in 2008. This sample size
remained approximately static in 2009 and 2010. In 2011, the number of normal samples
scheduled to be collected was increased to reflect the large number of lots of non-intact beef
presented for reinspection at U.S. POE.
Beginning CY2010, because the AIIS was incapable of assigning only to beef/veal, a decision
was made to pro-rate the samples by import region and have them assigned. Based on the
109
Failure-to-Present (FTP) is when product has not been presented to the FSIS inspector for an AIIS assignment
and enters commerce. Failure to present for FSIS inspection may result in penalties.
76
number of lots presented in the past, a set number of samples were provided to each import
region to assign throughout the year for each country. Import regions were given time periods in
which they should sample based on number of positive samples and where the producing country
is located geographically. When FSIS identifies a foreign grinding establishment as a ―multi-
source‖ supplier of E. coli O157:H7 positive beef/veal, FSIS management will place the foreign
establishment on Increased inspection.
The designation of a ―multi-source‖ or ―sole-source‖ positive E. coli O157:H7 supplier
notification result is reported in STEPS. The AIIS is programmed to assign samples for E. coli
O157:H7 to a minimum of the next 15 consecutive lots for a ―multi-source‖ supplier notification
of applicable beef/veal product. In the case of a ―sole-source‖ supplier notification, the next 15
consecutive lots of applicable beef/veal product and 15 times the weight of the lot (if known) are
sampled. If all samples are negative, the Increased level is removed from the AIIS and sampling
returns to Normal.
FSIS conducts periodic audits of those countries certified to export meat, poultry and processed
egg products to the U.S. The audits focus on ensuring that the country maintains a food safety
system equivalent to that of the U.S.. Audit findings that result in a food safety concern, such as
inadequate government oversight, will be brought to FSIS headquarters attention. Based on the
health risks associated with the food products and the nature of the failure, FSIS management
may decide to place the country or establishment on Increased inspection for the product
exported by that country.
When a positive sample is reported, the AIIS is programmed to place the foreign establishment
that produced the product on an Intensified level of inspection. This means that, at a minimum,
the next 15 consecutive lots of applicable beef/veal product and 15 times the weight of the failed
lot are assigned E. coli O157:H7 sampling. If all samples are negative, the Intensified level is
removed from the AIIS and sampling returns to Normal.
Statistical or Policy Basis
The current E. coli O157:H7 import sampling project is based on the number of positive
samples.
Limitations of Current Sampling
The current MT51 sampling project relies on implementation by the FSIS field supervisors to
ensure that amenable product subcategories (e.g., boneless cuts) are sampled at the correct
intervals. As a result, the MT51 project is more time-consuming to administer and monitor than
the import sampling projects, such as MT08, that are fully implemented through the AIIS.
77
3.1.3 RTE Meat and Poultry Products
Overview of Sampling Programs
FSIS maintains one sampling project for RTE products from importing countries. This project is listed below in Table 3.1.3.1 and
described in more detail below.
Table 3.1.3.1: FSIS RTE Sampling for Imported Products, FY 2010
Pathogen
Product
Type/Class
Sampling
Project
Total
Number
of
Samples
Analyzed
(Failures)
Average
Number of
Samples
Collected Per
Establishment
Number of
Establishments
Included in
Sampling
Population
Regulatory
Purpose of
Sampling
Program
Type of
Sampling
Program
Salmonella
and Lm
Imported
Intact RTE
Product
IMVRTE
(POE
Sampling)
4,512 (3)
Not
Applicable
Not Applicable
Component
of FSIS' on-
going
equivalence
verification
program
Performance-
Based
78
IMVRTE—Imported Intact RTE Product
Current Design
The current project is based on a desired number of Normal level samples to be assigned and
analyzed over a given calendar year. The samples are divided up and allocated based on the
amount of RTE product a country has exported to the U.S. over the past 24 months, and then
further subdivided and allocated by HACCP process category to each country. Consequently,
approximately 3,000 total samples should be analyzed, as calculated by FSIS in 2008. Based on
the way the AIIS is programmed for the import RTE sampling (IMVRTE) project, the AIIS
assigns analyses for both Lm and Salmonella for a given sample unit. This results in more than
6,000 sample analyses for Lm and Salmonella. However, this number is beyond FSIS‘ current
capacity. Therefore, the sampling target was reduced to approximately 1,500 to 2,000 samples
annually and each sample submitted to the lab is analyzed for the applicable pathogens.
When a foreign establishment or country is identified as having issues during a U.S. audit or by
some other means, FSIS management may place the country or establishment on Increased
inspection. The sampling rate is determined by FSIS management, in accordance with
guidelines developed by the Agency, and monitored to ensure that the specific management
objective is met. The Increased level is removed when the defined management objective has
been met, or by management decision, and sampling returns to Normal.
When a positive sample is reported, the AIIS is programmed to place the foreign establishment
that produced the product on an Intensified level of inspection. This means that, at a minimum,
the next 15 consecutive lots or 15 times the weight of the failed lot are assigned sampling in the
same HACCP process category for the pathogen that tested positive. If all samples are negative,
the Intensified level is removed from the AIIS and sampling returns to Normal.
Limitations of Current Sampling
Increased sampling is manual, and comes with the same difficulties Intensified sampling does,
namely that while targeting is a good practice, it is not always one hundred percent accurate.
Additionally, product sampling following a positive is limited to the same process category.
Consequently, other products may be produced in the same area/line, but under a different
HACCP process, which means that Intensified sampling does not take it into account.
Imports Measures of Success
POE reinspections, including pathogen and residue testing, are one component of a
comprehensive, ongoing verification process designed to ensure equivalence of exporting
countries‘ food safety systems. POE reinspections help to ensure that imported FSIS-regulated
products are safe and wholesome by supporting FSIS‘ overall equivalence program through:
Identifying shipments that do not meet FSIS requirements and refusing entry of these
products into the U.S.
Providing detailed information to support FSIS‘ equivalence verification audit programs.
Providing detailed information to support FSIS‘ performance-based sampling programs.
Providing detailed information to support the NRP.
79
In addition, POE reinspection findings are used to identify foreign establishments warranting
increased/intensified reinspection, such as more frequent reinspection of subsequent shipments
following presentation of a shipment that failed reinspection.
To measure the success of the import sampling programs, FSIS evaluates its efforts in terms of
four key operational metrics, as described in Table 3.1. Operational measures are included here
as it is not currently possible to estimate the number of foodborne illnesses that come from
imported products, as the number of samples collected do not warrant a measure of prevalence
and the CDC does not differentiate illnesses acquired from eating contaminated imported food,
as opposed to domestically produced food.
Table 3.1: Measures of Success for FSIS Import Sampling Program
Measure
Goal
Operational Performance
Measure
FY 2009
FY 2010
110
FY 2015
Percent of AIIS assigned E. coli
samples that are collected.
> 99 %
>98%
95 %
Percent of AIIS assigned E. coli
samples that are not analyzed due to
inspector error.
< 1%
<1%
5 %
Percent of AIIS assigned E. coli
foreign establishment-follow-up
samples (e.g., establishment under
intensified inspection status) that are
collected.
100 %
100%
95 %
Percent of AIIS assigned E. coli
foreign establishment-follow-up
samples (e.g., establishment under
intensified inspection status) that are
not analyzed due to inspector error.
[none]
<5%
5 %
110
The performance measure is calculated by using a 12 month rolling window, so the measure reflects the most recent 12
months of data up to and including the current year.
80
Section 3.2: Chemical Residues
Overview:
Imported meat, poultry and processed egg products are sampled at U.S. POE to detect chemical
residues as part of a POE reinspection. POE reinspection is a monitoring program conducted to
verify the equivalence of inspection systems in exporting countries. The chemical residue
sampling program is one of several Types of Inspection (TOI) conducted during FSIS
reinspection of imported products. The following are the three levels of chemical residue
reinspection:
Normal sampling, defined as random sampling from a lot;
Increased sampling, defined as above the normal sampling as the result of an FSIS
management decision; and
Intensified sampling, defined as occurring when a previous sample for a TOI failed to
meet U.S. requirements.
For both normal and increased sampling, the importer may choose to retain the lot pending the
laboratory results, but it is not required. However, the lot is subject to recall if it is not retained
and is found to contain violative levels of chemical residue. For Intensified sampling, the lot
must be retained pending laboratory results. The data obtained from laboratory analyses are
entered into the AIIS.
Fresh and Processed
Current Design
The current program is based on a desired number of Normal level samples to be assigned and
analyzed by compound over a given calendar year for each country, product (fresh or processed)
and species. When the import volume of a particular product class is less than one percent of the
total amount of meat, poultry and processed egg products imported, then eight samples are
assigned to each country. The sample numbers come from the NRP SAT and are programmed
into the AIIS. Please see Table 3.2.1 for fresh and processed samples analyzed in FY2010.
Residue
Sampling
Total Number of
Samples Analyzed
Regulatory Purpose of
Sampling Program
Fresh and
Processed
3,408
Component of FSIS' on-going equivalence verification
program
Table 3.2.1: FSIS Residue Sampling for Imported Products, FY 2010
Statistical or Policy Basis
The allocation of samples under the import program is based on several factors including:
country of origin, product type, volume imported and chemical tested. The FSIS NRP Blue
Book provides the algorithms used to prioritize reinspection sampling.
Limitations of Current Sampling
The current sampling program is based primarily on compounds known to be used domestically,
and may exclude compounds of concern in the foreign country or compounds which exporting
countries may use, but are prohibited for use in the United States. FSIS is currently evaluating
its protocol for prioritizing hazards from chemical exposure, which may impact both
domestically produced products, as well as those imported.
81
4.0 In-Commerce Activities
Overview
FSIS Compliance Investigators (CI) conduct surveillance to protect the health and welfare of
consumers by ensuring that meat, poultry and processed egg products in-commerce are safe,
wholesome, correctly labeled and packaged and secure from intentional acts of contamination.
These activities are carried out at in-commerce locations such as warehouses, distribution centers
and retail establishments, as well as POE and U.S. borders, to verify that persons and firms,
whose business activities involve FSIS-regulated products, prepare, store, transport, sell or offer
for sale or transportation such products in compliance with FSIS statutory and regulatory
requirements. These activities require data collection and analysis that differs from that required
for the domestic inspection activities covered by this report.
Current Design of Sampling Plan
FSIS has the following sampling projects currently in place at retail:
1) E. coli O157:H7 testing in raw ground beef at businesses operating under a retail
exemption (MT05).
2) Follow-up testing for E. coli O157:H7 in raw ground beef products (MT06) scheduled
only when an MT05 sample tests positive.
Table 4.1: FSIS E. coli O157:H7 Sampling Projects for In-Commerce Surveillance
Products
Sampling
Projects
Number of
E. coli
O157:H7
samples
analyzed in
CY2010
Regulatory
Purpose of
Sampling
Program
Type of Sampling
Program
Raw ground
beef at retail
stores
MT05
905
Verify
compliance
with
regulatory
standard
Targeted
Follow-up
testing to a
MT05
sample
MT06
0
Verify
corrective
measure
Targeted, Consecutive
Objectives of Retail Sampling:
Retail sampling is an important part of FSIS‘ overall E. coli O157:H7 sampling activities. The
retail sampling program addresses several objectives for FSIS:
1. Helps ensure hazard controls at retailers are adequate to prevent product from becoming
adulterated.
82
Statutory provisions requiring inspection do not apply to the types of operations
traditionally and usually conducted at retail stores. However, FSIS‘ adulteration and
misbranding provisions do apply to exempt retail businesses. Retailers have the potential
to adulterate product in the absence of adequate hazard controls. FSIS testing of retail
samples for E. coli O157:H7 gives the Agency additional assurances that products are not
being adulterated at retail facilities operating under a retail exemption.
2. Encourages industry to adopt complete and accurate product tracing systems for food.
Retail facilities should consistently maintain adequate records concerning suppliers of
source material for raw ground beef products, as required by regulations and the Federal
Meat Inspection Act (FMIA). With regard to investigations associated with raw ground
beef consumption, product lot coding, production date and beef manufacturing
establishment information are required to successfully conduct product traceback.
In many circumstances, however, investigators are provided with only purchase
information, such as date and location of purchase or type of ground beef. FSIS CI then
must rely heavily on grinding records kept in retail stores, meat markets and other
operations to gather the information needed to undertake traceback actions.
Unfortunately, CI frequently find these grinding records to be incomplete or inaccurate,
thereby delaying or preventing the traceback of potentially adulterated products, which
could result in additional illnesses.
When FSIS collects samples of raw ground beef from retail businesses, the Agency
collects the relevant information using the FSIS Form 8010-1, Retail Ground Beef
Sampling Worksheet, which is used if an E. coli O157:H7 positive finding is later
identified. FSIS has stated it expects retail facilities to consistently maintain complete
and adequate records, as required by the regulations (9 CFR part 320) and the Section
202 of the FMIA (21 U.S.C. 601 et seq.).
3. Encourage industry to conduct testing programs for E. coli O157:H7.
FSIS considers product sampling to be one of several activities conducted to verify
supplier claims and the effectiveness of hazard controls at retail facilities. Since the
adoption of the FSIS E. coli O157:H7 testing program, many grinders and suppliers of
raw ground beef components have instituted programs to routinely test their raw ground
beef products or raw materials used in raw ground beef products for E. coli O157:H7.
4. Serves as an indicator of the overall trend of the presence of E. coli O157:H7 in raw ground
beef.
Although FSIS views results from verification samples from Federal establishments as
the best indicator of the overall trend of the presence of E. coli O157:H7 in raw ground
beef, the test results from retail sampling gives the Agency another indicator of trends in
the presence of E. coli O157:H7 in raw ground beef products.
83
MT05 and MT06—Raw Ground Beef at Retail
Historical Basis and Background Information
Although most microbiological samples are collected at federally inspected establishments, FSIS
collects samples from retail stores in accordance with criteria listed in FSIS Directive 8010.1,
Rev. 2, Methodology for Conducting In-Commerce Surveillance Activities, dated June 25, 2008.
―Retailers‖ are one of more than a dozen business types in the FSIS In-Commerce Surveillance
System (ICS) and constitute about 60% of businesses in the system. FSIS samples at retail
when: 1) the retail store produces raw ground beef using whole muscle or trimmings from a
cutting/boning operation conducted at the store, 2) the retail store does not maintain records of
raw beef suppliers or records documenting clear and accurate grinding logs or 3) the retail store
is not cleaning and sanitizing the grinder between the use of different source materials.
Retail samples are not scheduled from an existing list of businesses producing raw ground beef.
Rather, CI are instructed to collect a one-pound sample of raw ground beef for E. coli O157:H7
testing at every retail business they visit for a surveillance review, if the business has raw ground
beef it prepared under the retail exemption and the business meets one of the criteria listed
above. Whether a raw ground beef sample will be collected depends on what the CI observes
during a surveillance review at a retail business.
Of the approximately 29,000 surveillance activities that FSIS has conducted since October 1,
2008, over one-third (about 11,000) were performed at retailers and 2,627 ground beef samples
were collected by the Agency. In FY 2011, CI collected 1,280 verification samples at retail
businesses. Of the retail samples collected since October 1, 2008, 0.11% tested positive for E.
coli O157:H7.
In September 2011, as a result of a formal review of FSIS in-commerce activities by the National
Academies of Science (NAS), the Agency shifted retailers from Tier 2 to Tier 3 facilities,
111,112
making them lower priority for Agency surveillance. Tier 3 businesses are surveilled only
―for cause.‖
113
Therefore, while the number of retail surveillances FSIS will likely perform ―for
cause‖ next year and in future years is unknown, it is unlikely to be high enough to generate
comparable numbers of retail ground beef samples. Indeed, it is possible that the number of
samples collected will be fewer than one hundred each year.
Statistical or Policy Basis
Approximately 75,000 retail businesses grind beef in the U.S. The MT05 sampling project for E.
coli O157:H7 in raw ground beef at retail venues was started in 1994 to monitor compliance with
regulatory standards and continues with that intent today. Neither the MT05 nor the MT06
projects are conducted using a random method. Rather, they are targeted sampling projects to
verify corrective actions that have been conducted as a result of a previous positive test and are
therefore not statistically based. Present targeting criteria for sample selection are described in
111
Please see the following website for more information: http://www.nap.edu/openbook.php?record_id=12786.
112
Please see the following website for more information: http://www.fsis.usda.gov/OPPDE/rdad/FSISNotices/53-11.pdf.
113
Tier 1 and Tier 2 businesses, generally, have significant inherent hazards, handle large volumes of meat, poultry, and egg
products, and receive minimal scrutiny by other regulatory authorities and accordingly are considered higher priority by FSIS.
Tier 3 businesses, on the other hand, generally receive significant scrutiny from other regulatory authorities and therefore are
considered lower priority by FSIS.
84
FSIS Directive 8010.1. There is no set, required sample collection frequency or sample size,
though the sample size is generally one pound. Samples are collected by a CI when a retail
business meets at least one of the criteria described in FSIS Directive 8010.1. The number of
samples collected and the percent positive have varied widely over time. Collection criteria also
have varied over time. These issues, and other concerns regarding the representativeness of
samples, make it inappropriate to extrapolate findings to all products sold at retail. Starting in
October 2011, the sampling program was modified to collect a purposive sample of
approximately 460 samples per year to provide a 99% probability of detecting one or more
positive samples, if the actual percent positive rate reaches as high as 1%, with 90% probability
of detecting one or more positives if the true percent positive is 0.5%. If every FSIS CI collects
approximately one sample quarterly, this will produce about 460 samples per year, distributed
evenly through the year and around the country.
Not all retailers produce ground beef and not all of those produce it under the conditions
specified in Appendix 1 of Directive 8010.1. Historically, FSIS CI has collected ground beef for
E. coli O157:H7 testing during one out of four retail surveillances. However, CI will have to
surveil- on average- only three retailers to identify a retailer eligible for sampling. Nationally,
this would mean about 1,380 surveillances to generate about 460 samples. While this constitutes
about 14% of FSIS‘ surveillances, these surveillances would be ―for cause.‖
Limitation of Current Sampling
Past sampling objectives and strategies cannot be confirmed, but appear to vary. However, the
present sampling strategy as outlined in Directive 8010.1 would result in samples that would not
accurately represent the prevalence of E. coli O157:H7 in raw ground beef processed at retail.
FSIS collects ground beef for testing from a very small proportion of retail businesses processing
ground beef and virtually none are collected on a non-risk basis. Without further understanding
of the situation, it would be difficult to compare yearly results from the retail program, with
results from Federal establishments, or to all ground beef in-commerce.
In-Commerce Measures of Success
Though the in-commerce E. coli O157:H7 sampling program differs in purpose and intent from
the Federal establishment sampling programs, the overall purpose of the program is to assess,
and minimize, the risk to public health from contaminated product. As such, to measure the
success of the E. coli O157:H7 sampling projects, FSIS can evaluate its efforts in terms of the
percent positives from the in-commerce E. coli O157:H7 sampling program.
Percent Positives
FSIS believes that percent positives are a good measure of the effectiveness of the E. coli
O157:H7 individual sampling projects maintained by the Agency, with declines in percent
positives indicating greater control and prevention of E. coli O157:H7 in finished product. In FY
2011, CI collected 1,280 samples. Of these samples, only 0.08% tested positive for E. coli
O157:H7.
85
5.0 Foodborne Illness Investigation and Consumer Complaint
Sampling Programs
Domestic Programs
Current Design of Sampling Program
FSIS regulated products are collected during foodborne illness and consumer complaint
investigations.
Statistical or Policy Basis for Current Sampling Plan
A foodborne illness investigation is defined as an investigation of the possible association
between human illnesses and FSIS-regulated product that includes epidemiologic, laboratory and
environmental assessments. Foodborne illness investigations are conducted as described in FSIS
Directive 8080.3 and internal Standard Operating Procedures (SOPs). A consumer complaint is
any complaint reported to FSIS that is initiated by or on behalf of a consumer and that is directly
related to a meat, poultry or processed egg product. Consumer complaint investigations are
conducted as described in FSIS Directive 5610.1 and internal SOPs.
Purpose of Program
During the course of foodborne illness and consumer complaint investigations, previously
opened (non-intact) products consumed by case-patients (e.g., individuals identified by an illness
investigation) or complainants may become available for investigative sampling and analysis by
FSIS. Similarly coded intact products may be collected directly from the identified individuals
at a point of purchase or at the producing establishment. These samples can provide the best
opportunity for detecting foodborne hazards, including microbial pathogens, extraneous
materials and chemicals. The data obtained from outbreak and consumer complaint sampling
supplements available epidemiologic and environmental findings and assists with determining
the type and extent of product contamination. Collectively, the information provides rapid
results for timely and informed risk management decisions.
Sampling Frame Definition
To determine whether to sample and test potentially implicated products, FSIS investigators
consider the following questions:
1. Do the epidemiologic investigation data, including the reported food history, support a link
between the illness or other described hazard and FSIS-regulated product?
2. Do the laboratory findings support a link between the illness or other described hazard and
FSIS-regulated product?
3. Does the environmental assessment support a link between the illness or other described
hazard and FSIS-regulated product?
4. Is there product available to test that meets FSIS criteria for product identity, chain of
custody and product handling? If not, are there reasons for testing product that may not meet
all of these criteria?
5. Has product already been tested by a non-FSIS laboratory with reliable methodology?
114
6. Can testing be carried out by or in association with FSIS?
114
See FSIS Directive 10,000.1 Policy On Use Of Results From Non-FSIS Laboratories.
86
To determine whether to sample and test a non-intact product, FSIS investigators consider the
following questions:
1. Was the non-intact product directly handled by the case-patient or complainant?
2. Was the non-intact product stored properly to avoid cross-contamination and temperature
abuse?
3. Are packaging materials and product labels that identify the non-intact product available?
If not, was traceback successful in determining the product identity?
Sampling size and methodology
After determining whether to collect an investigative sample, FSIS drafts a sampling plan that
takes into account the product available for sampling and the available laboratory resources and
capacity. Potential sampling plans are described below:
1. When a limited amount of product is available (e.g., if a few packages and lots are
available for sampling):
a. FSIS may request that all intact packages be submitted to the FSIS laboratory.
2. When an un-manageable number of product packages are available:
a. FSIS will provide guidance on sampling procedures consistent with available
resources. FSIS has identified the following procedures when a unmanageable
number of packages are available:
i. Sampling by perceived relative risk. If evidence indicates that product of
one lot/code is more suspect than another potentially implicated lot/code,
FSIS may propose to stratify and allocate random sampling either
proportionally or in tiers. For example, the FSIS laboratory determines it
can analyze 50 samples per day. Lot A product appears to be of greater
risk or concern compared to Lot B, but there is some reason to believe that
the latter lot may be at risk as well. Both production lots are identified and
under regulatory control. If the priority for the investigation is to
determine status for both lots as quickly as possible, FSIS may propose
weighted stratified sampling for each lot. For example, 30 samples from
Lot A and 20 from Lot B.
ii. Sequential sampling over time. If time is available for multiple analyses,
FSIS may propose sequential sampling over time. For example, 50
samples of Lot A may be analyzed initially and an additional 50 samples
may be analyzed at a later date, as resources allow.
iii. Sampling by relative volume. For sampling product from one or more lots
or consignments of different size, FSIS may propose proportional random
sampling by total volume. For example, Consignment A is 500 pounds, B
is 1500 pounds and C is 3000 pounds. For 50 total sample analyses, 5, 15
and 30 samples for Consignments A, B and C, respectively, would be
selected.
Sampling Weights, Ceilings, and Floors
Not applicable to foodborne illness and consumer complaint investigation related sampling.
87
Collection methodology
If product sampling and testing is warranted, FSIS will follow procedures in FSIS Directive
8080.3, Section IX or FSIS Directive 5610.1. FSIS staff would be directed to collect product
samples using Domestic Laboratory Report Form 10,000-2 to document chain of custody, as
described in FSIS Directive 8010.3 and to use sample seals as described in FSIS Directive
7355.1. Samples are delivered to an FSIS Field Service Laboratory or another laboratory with
available capacity and expertise. Results from foodborne illness and consumer complaint
investigation related sampling is reported by email to FSIS management and other designated
recipients, but are not reported through
Biological Information Transfer and E-mail System
(
BITES) and Laboratory Electronic Application for Results Notification (LEARN). Results are
also available in the FSIS Laboratory Information Management Systems (LIMS). Non-microbial
test results are reported through the Consumer Compliant Monitoring System II (CCMS II).
Mean Response Rate, Mean Analyzed Sample, and Percent Positive Rate for Samples
Analyzed
Not applicable to foodborne illness and consumer complaint investigation related sampling.
Limitations of Current Sampling
Testing should be performed within the context of available resources
Testing cannot guarantee that a sampled lot is free from targeted hazard(s).
Foodborne illness and consumer complaint investigation related testing is often
performed outside the scope of the ISO 17025
115
accreditation maintained by the FSIS
laboratory system.
FSIS laboratories may not have the expertise or capacity to test for certain analytes.
116
In some situations, FSIS may arrange for outbreak samples to be tested by other
laboratories, such as FDA-Center for Food Safety and Applied Nutrition (CFSAN), ARS,
the Food Emergency Response Network (FERN) or state laboratories of agriculture.
.
115
ISO/IEC 17025 is the main standard used by testing and calibration laboratories.
116
A summary of routine FSIS laboratory analyses is available at http://dchqintra/learn/docfile/analyses.htm.
88
6.0 Appendices
89
Appendix A: Definition of Terms
Data Warehouse: FSIS collects numerous types of data from a variety of different sources. This
data is stored in an electronic ―warehouse,‖ known as the FSIS Data Warehouse (DW).
Exclusion Criteria: Exclusion criteria are the standards FSIS uses to determine whether an
establishment should be included in the sampling frame. For example, establishments that
produce a very low volume of product may be excluded from the sampling frame. Therefore,
producing low volume is the exclusion criterion.
Percent Positive: The percentage of positive samples is expressed as a percentage, determined as
the number of positive samples for the pathogen per the total number of samples tested,
multiplied by 100. The expected value of this percentage in this document is called ―the percent
positive.‖
117
Performance Based Sampling: A sampling plan in which establishments are sampled at a greater
or lesser frequency based on their performance. For example, establishments that have fewer
positive pathogen test results might be considered to be high performers and are therefore
sampled less frequently than establishments that have more positive pathogen test results.
Random Sampling: A random sample is one chosen by a method involving an unpredictable
component. Random sampling can also refer to taking a number of independent observations
from the same probability distribution, without involving any real population.
Replacement: When a sampling unit is drawn from a finite population and is returned to that
population, after its characteristic(s) have been recorded, but before the next unit is drawn, the
sampling is said to be ―with replacement.‖ In the contrary case, the sampling is ―without
replacement.‖ A different usage occurs in sample surveys when samples are taken on successive
occasions. If the same members are used for successive samples there is said to be no
replacement; but if some members are retained and others are replaced by new individuals there
is said to be ―partial replacement‖.
118
Risk Based Sampling: A sampling plan in which establishments are sampled at a greater or lesser
frequency based on the risk the establishment poses. For example, establishments that have
fewer positive pathogen test results might be considered to be low risk and are therefore sampled
less frequently than establishments that have more positive pathogen test results.
Sampling Frame
119
: Sampling frame is the actual set of units from which a sample has been
drawn. In the case of a simple random sample, all units from the sampling frame have an equal
chance to be drawn and to occur in the sample. In the ideal case, the sampling frame should
coincide with the population of interest.
117
Please see the following website for more information:
http://www.fsis.usda.gov/PDF/Draft_Guidelines_Sampling_Beef_Trimmings_Ecoli.pdf.
118
Please see the following website for more information: http://stats.oecd.org/glossary/detail.asp?ID=3835.
119
Please see the following website for more information: www.statistics.com.
90
Sample Size: The sample size of a statistical sample is the number of observations that constitute
it. It is typically denoted n, a positive integer. The sample size is an important feature of any
empirical study in which the goal is to make inferences about a population from a sample. In
practice, the sample size used in a study is determined based on the cost of data collection, and
the need to have sufficient statistical power. In a census, data are collected on the entire
population; hence the sample size is equal to the population size. Larger sample sizes lead to
increased precision when estimating unknown parameters. For example, to know the proportion
of cattle that is infected with a pathogen, a more accurate estimate of this proportion will result
from a sample of 200, rather than 100 cattle.
Sample Ceiling: The maximum number of samples in a sampling frame.
Sample Floor: The minimum number of samples in a sampling frame.
Time Series: A time series is a set of regular, time-ordered observations of a quantitative
characteristic of an individual or collective phenomenon taken at successive, in most cases
equidistant, periods/points of time. Breaks in statistical time series occur when there is a change
in the standards for defining and observing a variable over time. Such changes may the result of
a single change or the combination of multiple changes at any one point in time of observation of
the variable.
120
For example, changes to the way in which the E. coli O157:H7 sampling frame
is constructed over time disrupts the time series and makes it difficult to compare results from
year to year.
Please see the following website for more information: http://stats.oecd.org/glossary/search.asp.
91
120
Appendix B: E. coli O157:H7 Sampling Program Description and Features
Average
Sampling
Mean
Mean
Project
Frame
Sampling
Collection
Sampling
Sampling
Purpose
Frame
Sample Size
Ceilings/Fl
Response
Samples
Code
Definition
Method
Method
Weights
Frequency
Size
oors
Rate
Analyzed
Ceilings: 4/
month for
large
volume
producers,
3/month for
medium,
Verification
Federally
Weighted
Production
2/month for
1 lb sample
of HACCP
inspected
random
11,482
1300 per month
1 lb of raw
volume and
small
of raw
72%
MT43
and assess
beef
1300
sampling
samples
(15600 per year)
ground beef
historical
volume,
ground
collected
risk to
grinding
with
per year
test results
and
beef
public health
estab.
replacement
1/month for
very small
Floor: 3
analyzed
samples per
year per
estab.
Federally
Follow-up to
inspected
NA
16 (or 8) follow-
Targeted
1 lb of raw
56%
MT44
MT43
beef
up samples per
samples
sampling
ground beef
collected
positives
grinding
MT43 positive
per year
estab.
92
254
MT50
To track rate
of O157:H7
in manufact.
trimmings
over time as
compared to
the 2007
prevalence
estimate.
Federally
inspected
beef and
veal
slaughter
estab.
480
50 per week
(2600 per year)
Simple
random
sampling
without
replacement
N60 sample
weighting 325
g
40%
collected
1092
samples
per year
MT53
Follow-up to
positive trim
or
component
sample
Federally
inspected
slaughter
estab.
NA
16 (or 8) follow-
up samples/
initial positive
Targeted
sampling
N60 sample
weighing 325 g
or 2 lbs of
component
52%
collected
161
samples
per year
Federally
MT54
Verification
of HACCP
inspected
beef
slaughter,
veal
slaughter,
and
ammoniate
d beef
estab.
450
15 per week (780
per year)
Simple
random
sampling
without
replacement
2 lb collection
27%
collected
224
samples
per year
Federally
Simple
random
N60 sample
538
Verification
inspected
150 per month
29%
MT55
1100
sampling
weighting 325
samples
of HACCP
estab. (non-
slaughter)
(1800 per year)
without
replacement
g
collected
per year
1 sample at each
MT52
Follow-up at
suppliers to
positive
sample
Federally
slaughter
inspected
estab.
N/A
supplier if there
are multiple
suppliers OR 16
(or 8) samples if
there is only one
Targeted
sampling
N60 sample
weighing 325 g
or 2 lbs of
component
77%
collected
610
samples
per year
supplier
93