1. The basics of risk assessment

1.1 Risk and risk analysis in plain language

Risks from microbiological and chemical hazards are of serious concern to human health. As the discipline of risk analysis matures, it is developing its own tools and language, and this paper explains what those tools can do, in simple language. To begin, the definitions and terms used in risk analysis are set out in the CAC Principles and guidelines for the conduct of microbiological risk assessment (CAC/GL-30, 1999). The Codex words are in italics and some explanatory words are in normal type.

Risk

A function of the probability of an adverse health effect and the severity of that effect, consequential to a hazard(s) in food.

Hazard

A biological, chemical or physical agent in, or condition of, food with the potential to cause an adverse health effect.

There are two very useful books that give information on seafood hazards:

• Assessment and management of seafood safety and other quality aspects (FAO, 2004).

• Fish and fisheries products hazards and controls guide (FDA, 2001).

Risk analysis

A process consisting of three components:

• risk assessment

• risk management

• risk communication

A common question is "Which of the three elements do I do first?" In most cases, the risk managers identify the need for a risk assessment and select an assessment team. Ideally, they should also begin the risk communication process as early as possible so that all interested and affected groups know what is happening from the first day. Tactically, it is a mistake to keep people uninformed - even if they agree with the assessment they will be displeased to have been excluded from the process.

Risk assessment

A scientifically based process consisting of the following steps:

• hazard identification

• hazard characterization

• exposure assessment

• risk characterization

The aim of risk assessment is to estimate the level of illness that may be expected in our target population from a product or group of products.

The information flow for the four components in a risk assessment is shown below:

Hazard identification

The identification of biological, chemical and physical agents capable of causing adverse health effects and that may be present in a particular food or group of foods.

This is the first stage in risk assessment and is a screening process to make certain that the hazard really does exist in this particular product. For example, Clostridium botulinum is readily identified as a hazard in canned, smoked and vacuum-packed seafoods, but is unlikely to be a hazard for any other seafood product. So hazard identification is a primary screen that allows risk managers to eliminate product: pathogen pairs that are of no concern.

You will find material on hazard identification for all of the hazards associated with seafoods in the Resources Bank.

Hazard characterization

The qualitative and/or quantitative evaluation of the nature of the adverse health effects associated with biological, chemical and physical agents that may be present in food. For the purpose of microbiological risk assessment the concerns relate to micro-organisms and/or their toxins.

There are two parts to hazard characterization:

• a description of the effects of the hazard (micro-organism or toxin);

• the dose-response relationship (if it exists).

Dose-response assessment

The determination of the relationship between the magnitude of exposure (dose) to a chemical, biological or physical agent and the severity and/or frequency of associated adverse health effects (response).

For any particular individual, dose-response links the amount of the hazard you ingest (dose) with the chance of your becoming infected and the scale of the illness if you do. For example, most healthy individuals can consume large numbers of Listeria monocytogenes (maybe as many as 100 million cells) without becoming seriously ill. By contrast, in susceptible people (foetuses, the aged or individuals with impaired immune systems) a much smaller dose (maybe as few as 10 000 cells) can cause serious illness and, in around 30 percent of cases, death. In the Resources Bank you will find a list of dose-responses for several micro-organisms and their toxins.

Exposure assessment

The qualitative and/or quantitative evaluation of the likely intake of biological, chemical and physical agents via food as well as exposures from other sources if relevant.

To carry out an exposure assessment you need data in two areas:

• number of servings of potentially dangerous food eaten;

• level of contamination with the micro-organism or toxin at the time of consumption.

To arrive at these types of data you will probably follow the micro-organism or toxin through the processing-food preparation chain and estimate changes that occur to the hazard throughout the chain.

Risk characterization

The process of determining the qualitative and/or quantitative estimation, including attendant uncertainties, of the probability of occurrence and severity of known or potential adverse health effects in a given population based on hazard identification, hazard characterization and exposure assessment.

When you do the risk characterization, you integrate hazard identification, exposure assessment and hazard characterization to provide an estimate of the risk.

Risk estimate

Output of risk characterization

This may vary from a qualitative estimate (high, low, medium) to a quantitative estimate where you predict the number of people you expect will become ill from the particular product:hazard pairing. Alternately, your risk characterization may be semi-quantitative and you make a risk ranking that is a number in a specific range, 0-100, for example.

Risk management

The process, distinct from risk assessment, of weighing policy alternatives, in consultation with all interested parties, considering risk assessment and other factors relevant to the health protection of consumers and for the promotion of fair trade practices, and, if needed, selecting appropriate prevention and control options.

Risk managers have a difficult responsibility because they must take into account the views of various groups. Trying to find compromises between the views of scientists, industry, consumer groups, politicians and lawyers is almost impossible, but it is what risk managers are required to do.

Risk communication

The interactive exchange of information and opinions throughout the risk analysis process concerning hazards and risk, risk-related factors and risk perceptions among risk assessors, risk managers, consumers, industry, the academic community and other interested groups, including the explanation of risk assessment findings and the basis of risk management decisions.

Communicating risk is a very difficult task because it involves the full range of stakeholders. A major problem is informing consumers that no food product is risk-free and, as a consequence, they must be prepared for X deaths and Y illnesses each year from this particular product. Risk communication includes changing perceptions of stakeholders so they all move towards some central positions that are not far removed from each other.

Quantitative risk assessment

A risk assessment that provides numerical expressions of risk and indication of the attendant uncertainties (WHO, 1995).

A typical quantitative risk assessment (QRA) was carried out by Lindqvist and Westöö (2000) for smoked fish in Sweden, where the predicted annual number of illnesses varied between 47 and 2 800 (mean 168) for consumers at most risk.

Qualitative risk assessment

A risk assessment based on data which, while forming an inadequate basis for numerical risk estimations, nonetheless, when conditioned by prior expert knowledge and identification of attendant uncertainties, permits risk ranking or separation into descriptive categories of risk.

A typical qualitative risk assessment was done by Huss, Reilly and Ben Embarek (2000), who estimated the risk as high for consumption of molluscan shellfish, fish eaten raw, lightly preserved fish and mildly heat-treated fish. Low-risk products were chilled/frozen fish and crustaceans, semi-preserved fish and heat-processed (canned) fish. Dried and heavily salted fish were considered to have no risk.

Risk profile

A description of a food safety problem and its context developed for the purpose of identifying those elements of a hazard or risk that are relevant to risk management decisions. This approach has been used in Australia to profile entire food industries.

Risk profiling can be a way of quickly identifying those products within a particular sector that are of most concern. This is exactly what Huss, Reilly and Ben Embarek (2000) did in the previous example for the seafood industry, as a whole. If you did a risk profile of your industry you might find some difference in risk rating. For example, dried and heavily salted fish usually have no risk. But what if the rainy season led to mould formation and the moulds were able to produce aflatoxin? The risk rating will no longer be zero.

A recent report of a joint FAO/WHO (2002) consultation defines that the purpose of a risk profile is to enable a decision on what will be done next and whether resources should be allocated to a more detailed scientific assessment. A risk profile comprises a systematic collection of information needed to make a decision, and is the responsibility of the risk manager (although it may be commissioned out to appropriate parties).

Transparency

Characteristics of a process where the rationale, the logic of development, constraints, assumptions, value judgements, decisions, limitations and uncertainties of the expressed determination are fully and systematically stated, documented and accessible for review.

Whenever risk assessments are submitted for peer review or public comment, the reviewers often comment that there is a lack of transparency. This means that they were not able to find important data, or they could not understand a calculation, or the risk assessors did not fully explain their logic.

Uncertainty analysis

A method used to estimate the uncertainty associated with model inputs, assumptions and structure/form.

Risk assessments almost always contain a statement specifying that insufficient data were available in one or more areas and, as a result, a certain amount of caution should be attached to the estimate. Caution, as a result of lack of precise information, leads to uncertainty and you should always record the data gaps that lead to uncertainty. Later, if that knowledge becomes available, the level of uncertainty will be reduced so that the risk estimate becomes more accurate.

Principles and guidelines for risk assessment

In 1999 the CAC set out general principles and guidelines for the conduct of microbiological risk assessment (FAO/WHO, 2001). As we also consider non-microbiological hazards, these principles have been amended from the Codex Principles for Microbial Risk Assessment by omitting "microbiological" where appropriate. The principles state that:

1.2 Types of risk assessment

There are several types of risk assessment that fall under three broad categories:

• qualitative risk assessment;

• semi-quantitative risk assessment;

• quantitative risk assessment.

All three categories provide useful information and your choice of assessment will depend on the speed and complexity you require from your assessment.

1.1.1 Qualitative risk assessments

These are the simplest and quickest to do, but they can be rather subjective, which reduces their value. Every HACCP plan contains simple qualitative risk assessments in the HACCP worksheet.

For every hazard, an estimate of risk is made by inserting high, medium or low in answer to questions on the severity of the hazard and the likelihood of it occurring. A basic problem is that the three descriptors (high, medium, low) are often inadequate. For example, suppose the process step is retorting in fish canning and the hazard is Clostridium botulinum. Almost everyone will describe the severity of the hazard as high. But how likely is the hazard to occur? Most people will put low because billions of cans of fish are manufactured each year with no sign of the hazard. High severity and low likelihood - how would you link these to estimate risk?

Type 1: Hazard control worksheet

Another type of qualitative risk assessment is shown below, in which the risk estimate is a risk ranking - high, low and medium.

Type 2: Qualitative risk ranking

This assessment is based on factors which are linked with exposure assessment (likelihood of occurrence and exposure in the diet) plus one which is linked with hazard characterization (severity of hazard). If the hazard: product pairing has some linkage with epidemiology (it has caused food poisonings), this serves to remind you that there is some probability that it will happen again.

So, in Type 2 (above) we can make some assessment of exposure from our responses to likelihood of occurrence and exposure in the diet. Suppose we are considering ciguatera in two different populations, e.g. people in a Pacific island atoll community and the population of the United Kingdom. For the Pacific you would probably say the likelihood of occurrence of ciguatera is high. For the United Kingdom, you would probably say likelihood of occurrence is very low. There are strong links with epidemiology in atoll communities where the hazard is more or less accepted as an unavoidable fact of life; in contrast, ciguatera only rarely occurs in the United Kingdom from imported reef fish.

When all the information is brought together into a risk ranking you probably have a high or very high ranking for the Pacific and a low or very low ranking for the United Kingdom. The ranking will have value if you need a clear-cut answer in a relatively short time. To get the answer you will need to research the hazard and discover that it may have a cumulative effect but that it is rarely fatal. You will also look into epidemiology of the two target consumer groups - a few thousand atoll residents and 60 million United Kingdom residents. If you can find a recent review of ciguatera, especially one that is written in a risk assessment context, you could complete your research in a short time.

Another qualitative scheme for categorizing risk from seafoods has been developed by Huss, Reilly and Ben Embarek (2000) who ascribe pluses to hazard, then rank risks as "high" (four or more pluses) or "low" (less than four pluses). The scheme takes into account epidemiology (bad safety record) and then focuses on the process, searching for a critical control point (CCP) for each hazard and assessing possibilities for growth and death of microbial hazards.

Type 3: Qualitative risk assessment based on the process

Source: after Huss, Reilly and Ben Embarek (2000).

So, as shown in Type 3, molluscan shellfish, fish eaten raw, lightly-preserved fish and mildly heat-treated fish are considered "high" risk, while chilled/frozen fish and crustaceans, semi-preserved fish and heat-processed (canned) fish are considered "low" risk; dried and heavily salted fish are considered to have no risk.

1.1.2 Semi-quantitative risk assessment

In qualitative risk qssessment, we estimated risk according to subjective terms such as high, low or medium. In semi-quantitative risk assessment we obtain a numerical risk estimate based on a mixture of qualitative and quantitative data. To do this type of assessment you need much of the data that will be used in a full quantitative risk assessment. There is a great deal of work involved, but not as much as for a full quantitative risk assessment.

Ross and Sumner (2002) developed a simple spreadsheet tool to describe the risk that emerges from pathogens in products manufactured by typical processes (canning, chilling, cooking, etc). Table 1 lists risk criteria needed for a semi-quantitative risk assessment. These are simple questions and they can be answered qualitatively in terms such as "high" and "low". But the researchers found it possible to insert a quantitative basis to the answers. The tool is in Microsoft® Excel spreadsheet software and uses standard mathematical and logical functions. You can mouse-click your qualitative inputs, and the software will automatically convert them into quantities for calculations.

You must generate some data in order to answer the eleven questions in Table 1. To help you make your inputs as objective as possible, and to maintain transparency of the model, descriptions of the subjective descriptors are provided and many of the weighting factors are specified in the lists of descriptors. Alternatively, where the options provided do not accurately reflect the situation being modelled, you can enter a numerical value that is appropriate.

TABLE 1 Typical risk criteria in a semi-quantitative risk assessment

The details behind the model can be read from the publication of Ross and Sumner (2002). Section 4 gives details about the tool, called Risk Ranger, and you can use it to work through some examples. The most robust risk estimates from Risk Ranger are a risk ranking (score from 0 to 100) and the number of illnesses per annum. This tool was used to provide a risk profile for the Australian seafood industry; later we will show you how its estimates were used to focus on those products and pathogens which required most attention from the industry.

1.1.3 Quantitative risk assessment

Quantitative risk assessments (QRAs) are done for specific purposes and provide numerical risk estimates to answer questions that were posed by the risk managers who originally commissioned the assessment. In the seafood area there have been three QRAs:

• Listeria monocytogenes in smoked fish in Sweden (Lindqvist and Westöö, 2000);

• Vibrio parahaemolyticus in oysters in the United States (FDA, 2000);

• Listeria monocytogenes in a range of seafoods in the United States (FDA, 2001).

The United States risk assessments were very large, taking more than one year to prepare and then moving to a 1-2 year review period of public comment. The L. monocytogenes risk assessment involved more than 30 people arranged in six teams, each of which was assigned specific tasks; more than 50 additional participants were acknowledged for their assistance. It must be stressed that this QRA involved a range of foods, not just seafoods, but the QRA of V. parahaemolyticus in oysters also involved more than 20 people who received information from scientists at more than 20 institutions in the United States and internationally. The Swedish QRA had two authors and acknowledged the help of two collaborators.

The resources invested in the two United States risk assessments were undoubtedly in response to large outbreaks of food poisoning in that country. In 1997 and 1998 there were two incidents involving V. parahaemolyticus in oysters involving more than 700 cases of illness, which led to the commissioning of the QRA. Also in the late 1990s there were two listeriosis incidents in the United States involving hot dogs and delicatessen meats in which more than 130 were seriously ill and 28 died.

Setting objectives - statement of purpose

In a QRA, it is vital to define what you want the work to achieve, and to do this right at the beginning. This is called a Statement of Purpose. In the United States, the risk managers stipulated that, for V. parahaemolyticus in oysters, the risk assessors:

1. produce a mathematical model of the risk of illness incurred by consumers of raw oysters containing pathogenic V. parahaemolyticus;

2. provide the regulators with information to assist with reviewing current regulations to ensure that they protect public health by evaluating:

• current criteria for closing and reopening shellfish waters to harvesting;

• preventive and intervention measures for controlling V. parahaemolyticus in oysters;

• current guidance on allowing up to 10 000 cfu/g of V. parahaemolyticus in oyster meat.

For L. monocytogenes, the Statement of Purpose was to examine available scientific data systematically in order to estimate the relative risks of serious illness and death that might be associated with consumption of different types of ready-to-eat foods that might be contaminated with L. monocytogenes. The work produced mathematical models to predict contamination at the retail level and in the home, and different consumer groups were included in the assessment. The result was predicted rates of listeriosis from various foods for various at-risk groups.

In Sweden, Lindqvist and Westöö (2000) set the objective to develop a QRA for estimating the exposure and risk of acquiring listeriosis from consumption of packaged smoked or gravad salmon and rainbow trout.

Modelling the process

In the seafood industry, the process is usually stretched out from harvesting, storing prior to processing, processing in the seafood plant, storing/distributing, retailing and consumption. Whatever the seafood product you are considering, the hazard may change throughout the process, either in prevalence or in concentration. We need to chart these changes often by making a process flow diagram and then mathematically measure or estimate changes in the hazard at each stage. In risk assessment this is called "modelling". Usually modellers try to make a "farm-to-fork" model that takes in changes to the hazard all along the harvest-process-consumption route. This part of the risk assessment is best done by people who understand the industrial process and combined with microbiologists who understand the hazard and how it reacts to changes, particularly to changes in temperature and time.

When the model of the system has been set, data must be gathered (exposure assessment). Ideally, there would be time to carry out experiments that give you exactly the data you need but, almost always, there are not sufficient resources or time to do this. So you need to investigate all sources of existing data and try to incorporate them into the model. This is where the modeller on your team takes the data and constructs mathematical relationships that describe changes in the hazard throughout the process. The modeller will encounter a number of problems, the most common being variability and uncertainty.

Variability

This occurs because of the diversity in any population, and it cannot be reduced, no matter how much the property is studied. To illustrate, let us use height as an example. In any population there is variability in height. We could do a survey by measuring how tall people are, and we would find most adults are 160-175 cm tall but that some are 220 cm while others are 120 cm. This is an example of variability within a population.

Uncertainty

This is due to our (the risk assessor's) lack of knowledge about a parameter and our inability to measure it. Uncertainty can be reduced if we study the characteristic. Using the same example of peoples' height, we could do a national survey and measure everyone. Then there would be no uncertainty.

Distributions

The risk is never fixed - it varies according to a range of parameters. For example, take the risk of dying in an air crash. For the vast majority of people on this earth the risk is zero because they never fly but, among those many millions who do fly, the risk varies according to how often they fly (likelihood), the airline (some have more crashes than others), the weather conditions (many crashes occur in bad weather) and the country (some have better systems than others). So estimating the risk is difficult because there is a distribution of risk from very low, through average to very high. Often the best estimate of distribution is minimum, most likely (average) and maximum value. For example, we might say the bacterial levels of shrimp landed aboard a trawler ranged from 10/g to 10 000/g, with the most likely count being 100/g.

Type of model

Modellers generally use simulation or stochastic modelling in which data are inserted into a spreadsheet. Computer software is then used to analyse the data. Each analysis is called an iteration where a value is selected from the distribution describing each variable range, more or less at random, but according to the probability distribution of that variable (more likely values are run more frequently than minimum or maximum values). A large number of iterations is run (10 000 is a popular number) and collated; the technique is called Monte Carlo simulation. The result is a distribution frequency of possible outcomes, which forms the basis of the risk estimate.

Risk estimate

The way you estimate the risk in a QRA is usually set by the statement of purpose. For example, Lindqvist and Westöö (2000) estimated the risk of acquiring listeriosis, and so risk estimates included the number of cases per annum and risk of becoming ill on a per serving basis. The researchers used two models and so had two estimates for each output. In the United States, the relative risk of acquiring listeriosis from a range of foods was the estimate, with pâtés, smoked seafoods, soft cheeses and delicatessen meats being the four most likely to cause the illness. For V. parahaemolyticus in oysters the single most important factor related to risk of illness was temperature - of air and water (seasonality). The model predicted nationwide illnesses of 4 750 per annum with a range of 1 000 to 16 000 cases. The model also indicated that risk of illness was reduced if product temperature could be lowered soon after harvest.

Reality check

When you have the risk estimates it is a good idea to do a reality check to see that the model is not predicting something that will seem absurd. For example, suppose you are estimating the number of cases of listeriosis caused by consumption of smoked fish and the model predicts the most likely scenario of 1 million cases each year. If your country statistics on illness and death state that there are 1 000 such cases each year, you know there is something wrong either with the model or with the inputs. You have more work to do!

Sensitivity (importance) analysis

As the software grinds through the iterations it also keeps a record of which factors have the biggest effect on risk estimate. This allows you to do sensitivity or importance analysis to identify those factors most influencing risk - either reducing or increasing it. This analysis then points risk managers to those areas where process control can be increased.

Summary

Risk assessments range in complexity from qualitative, through semi-quantitative to quantitative. As assessments become more complex, they also become more expensive and take longer to complete. So before you begin a risk assessment be sure you know exactly what you want or you may end up using resources unnecessarily.