This article's lead section may be too short to adequately summarize the key points. (February 2021) |
Risk assessment determines possible mishaps, their likelihood and consequences, and the tolerances for such events.[1] The results of this process may be expressed in a quantitative or qualitative fashion. Risk assessment is an inherent part of a broader risk management strategy to help reduce any potential risk-related consequences.[1][2]
![](http://upload.wikimedia.org/wikipedia/commons/thumb/a/a1/Risk_Assessment_and_Risk_Management.png/220px-Risk_Assessment_and_Risk_Management.png)
More precisely, risk assessment identifies and analyses potential (future) events that may negatively impact individuals, assets, and/or the environment (i.e. hazard analysis). It also makes judgments "on the tolerability of the risk on the basis of a risk analysis" while considering influencing factors (i.e. risk evaluation).[1][2]
Categories
editIndividual risk assessment
editRisk assessments can be done in individual cases, including in patient and physician interactions.[3] In the narrow sense chemical risk assessment is the assessment of a health risk in response to environmental exposures.[4] The ways statistics are expressed and communicated to an individual, both through words and numbers impact his or her interpretation of benefit and harm. For example, a fatality rate may be interpreted as less benign than the corresponding survival rate.[3] A systematic review of patients and doctors from 2017 found that overstatement of benefits and understatement of risks occurred more often than the alternative.[3][5] A 2017 systematic review from the Cochrane collaboration suggested "well-documented decision aids" are helpful in reducing effects of such tendencies or biases.[3][6][needs update] An individual´s own risk perception may be affected by psychological, ideological, religious or otherwise subjective factors, which impact rationality of the process.[3] Individuals tend to be less rational when risks and exposures concern themselves as opposed to others.[3] There is also a tendency to underestimate risks that are voluntary or where the individual sees themselves as being in control, such as smoking.[3]
Systems risk assessment
editRisk assessment can also be made on a much larger systems theory scale, for example assessing the risks of an ecosystem or an interactively complex mechanical, electronic, nuclear, and biological system or a hurricane (a complex meteorological and geographical system). Systems may be defined as linear and nonlinear (or complex), where linear systems are predictable and relatively easy to understand given a change in input, and non-linear systems unpredictable when inputs are changed.[7] As such, risk assessments of non-linear/complex systems tend to be more challenging.
In the engineering of complex systems, sophisticated risk assessments are often made within safety engineering and reliability engineering when it concerns threats to life, natural environment, or machine functioning. The agriculture, nuclear, aerospace, oil, chemical, railroad, and military industries have a long history of dealing with risk assessment.[8] Also, medical, hospital, social service,[9] and food industries control risks and perform risk assessments on a continual basis. Methods for assessment of risk may differ between industries and whether it pertains to general financial decisions or environmental, ecological, or public health risk assessment.[8]
Concept
editRapid technological change, increasing scale of industrial complexes, increased system integration, market competition, and other factors have been shown to increase societal risk in the past few decades.[1] As such, risk assessments become increasingly critical in mitigating accidents, improving safety, and improving outcomes. Risk assessment consists of an objective evaluation of risk in which assumptions and uncertainties are clearly considered and presented. This involves identification of risk (what can happen and why), the potential consequences, the probability of occurrence, the tolerability or acceptability of the risk, and ways to mitigate or reduce the probability of the risk.[2] Optimally, it also involves documentation of the risk assessment and its findings, implementation of mitigation methods, and review of the assessment (or risk management plan), coupled with updates when necessary.[1] Sometimes risks can be deemed acceptable, meaning the risk "is understood and tolerated ... usually because the cost or difficulty of implementing an effective countermeasure for the associated vulnerability exceeds the expectation of loss."[10]
Mild versus wild risk
editBenoit Mandelbrot distinguished between "mild" and "wild" risk and argued that risk assessment and risk management must be fundamentally different for the two types of risk.[11] Mild risk follows normal or near-normal probability distributions, is subject to regression to the mean and the law of large numbers, and is therefore relatively predictable. Wild risk follows fat-tailed distributions, e.g., Pareto or power-law distributions, is subject to regression to the tail (infinite mean or variance, rendering the law of large numbers invalid or ineffective), and is therefore difficult or impossible to predict. A common error in risk assessment and management is to underestimate the wildness of risk, assuming risk to be mild when in fact it is wild, which must be avoided if risk assessment and management are to be valid and reliable, according to Mandelbrot.
Mathematical conceptualization
editTo see the risk management process expressed mathematically, one can define expected risk as the sum over individual risks, , which can be computed as the product of potential losses, , and their probabilities, :
Even though for some risks , we might have , if the probability is small compared to , its estimation might be based only on a smaller number of prior events, and hence, more uncertain. On the other hand, since , must be larger than , so decisions based on this uncertainty would be more consequential, and hence, warrant a different approach.
This becomes important when we consider the variance of risk
as a large changes the value.
Financial decisions, such as insurance, express loss in terms of dollar amounts. When risk assessment is used for public health or environmental decisions, the loss can be quantified in a common metric such as a country's currency or some numerical measure of a location's quality of life. For public health and environmental decisions, the loss is simply a verbal description of the outcome, such as increased cancer incidence or incidence of birth defects. In that case, the "risk" is expressed as
If the risk estimate takes into account information on the number of individuals exposed, it is termed a "population risk" and is in units of expected increased cases per time period. If the risk estimate does not take into account the number of individuals exposed, it is termed an "individual risk" and is in units of incidence rate per time period. Population risks are of more use for cost/benefit analysis; individual risks are of more use for evaluating whether risks to individuals are "acceptable".
Quantitative risk assessment
editIn quantitative risk assessment, an annualized loss expectancy (ALE) may be used to justify the cost of implementing countermeasures to protect an asset. This may be calculated by multiplying the single loss expectancy (SLE), which is the loss of value based on a single security incident, with the annualized rate of occurrence (ARO), which is an estimate of how often a threat would be successful in exploiting a vulnerability.
The usefulness of quantitative risk assessment has been questioned, however. Barry Commoner, Brian Wynne and other critics have expressed concerns that risk assessment tends to be overly quantitative and reductive. For example, they argue that risk assessments ignore qualitative differences among risks. Some charge that assessments may drop out important non-quantifiable or inaccessible information, such as variations among the classes of people exposed to hazards, or social amplification.[12] Furthermore, Commoner[13] and O'Brien[14] claim that quantitative approaches divert attention from precautionary or preventative measures.[15] Others, like Nassim Nicholas Taleb consider risk managers little more than "blind users" of statistical tools and methods.[16]
Process
editOlder textbooks distinguish between the term risk analysis and risk evaluation; a risk analysis includes the following 4 steps:[1]
- establish the context, which restricts the range of hazards to be considered. It is also necessary to identify the potential parties or assets which may be affected by the threat, and the potential consequences to them if the hazard is activated.
- Hazard identification, an identification of visible and implied hazards and determining the qualitative nature of the potential adverse consequences of each hazard. Without a potential adverse consequence, there is no hazard.
- frequency analysis If a consequence is dependent on dose, i.e. the amount of exposure, the relationship between dose and severity of consequence must be established, and the risk depends on the probable dose, which may depend on concentration or amplitude and duration or frequency of exposure. This is the general case for many health hazards where the mechanism of injury is toxicity or repetitive injury, particularly where the effect is cumulative.
- consequence analysis. For other hazards, the consequences may either occur or not, and the severity may be extremely variable even when the triggering conditions are the same. This is typical of many biological hazards as well as a large range of safety hazards. Exposure to a pathogen may or may not result in actual infection, and the consequences of infection may also be variable. Similarly, a fall from the same place may result in minor injury or death, depending on unpredictable details. In these cases, estimates must be made of reasonably likely consequences and associated probability of occurrence.
A risk evaluation means that judgements are made on the tolerability of the identified risks, leading to risk acceptance. When risk analysis and risk evaluation are made at the same time, it is called risk assessment.[1]
As of 2023, chemical risk assessment follows these 4 steps:[4]
- hazard characterization
- exposure assessment
- dose-response modeling
- risk characterization.
There is tremendous variability in the dose-response relationship between a chemical and human health outcome in particularly susceptible subgroups, such as pregnant women, developing fetuses, children up to adolescence, people with low socioeconomic status, those with preexisting diseases, disabilities, genetic susceptibility, and those with other environmental exposures.[4]
The process of risk assessment may be somewhat informal at the individual social level, assessing economic and household risks,[17][18] or a sophisticated process at the strategic corporate level. However, in both cases, ability to anticipate future events and create effective strategies for mitigating them when deemed unacceptable is vital.
At the individual level, identifying objectives and risks, weighing their importance, and creating plans, may be all that is necessary. At the strategic organisational level, more elaborate policies are necessary, specifying acceptable levels of risk, procedures to be followed within the organisation, priorities, and allocation of resources.[19]: 10
At the strategic corporate level, management involved with the project produce project level risk assessments with the assistance of the available expertise as part of the planning process and set up systems to ensure that required actions to manage the assessed risk are in place. At the dynamic level, the personnel directly involved may be required to deal with unforeseen problems in real time. The tactical decisions made at this level should be reviewed after the operation to provide feedback on the effectiveness of both the planned procedures and decisions made in response to the contingency.
Dose dependent risk
edit- Dose-Response Analysis, is determining the relationship between dose and the type of adverse response and/or probability or the incidence of effect (dose-response assessment). The complexity of this step in many contexts derives mainly from the need to extrapolate results from experimental animals (e.g. mouse, rat) to humans, and/or from high to lower doses, including from high acute occupational levels to low chronic environmental levels. In addition, the differences between individuals due to genetics or other factors mean that the hazard may be higher for particular groups, called susceptible populations. An alternative to dose-response estimation is to determine a concentration unlikely to yield observable effects, that is, a no effect concentration. In developing such a dose, to account for the largely unknown effects of animal to human extrapolations, increased variability in humans, or missing data, a prudent approach is often adopted by including safety or uncertainty factors in the estimate of the "safe" dose, typically a factor of 10 for each unknown step.
- Exposure Quantification, aims to determine the amount of a contaminant (dose) that individuals and populations will receive, either as a contact level (e.g., concentration in ambient air) or as intake (e.g., daily dose ingested from drinking water). This is done by examining the results of the discipline of exposure assessment. As a different location, lifestyle, and other factors likely influence the amount of contaminant that is received, a range or distribution of possible values is generated in this step. Particular care is taken to determine the exposure of the susceptible population(s).
The results of these steps are combined to produce an estimate of risk. Because of the different susceptibilities and exposures, this risk will vary within a population. An uncertainty analysis is usually included in a health risk assessment.
Dynamic risk assessment
editDuring an emergency response, the situation and hazards are often inherently less predictable than for planned activities (non-linear). In general, if the situation and hazards are predictable (linear), standard operating procedures should deal with them adequately. In some emergencies, this may also hold true, with the preparation and trained responses being adequate to manage the situation. In these situations, the operator can manage risk without outside assistance, or with the assistance of a backup team who are prepared and available to step in at short notice.
Other emergencies occur where there is no previously planned protocol, or when an outsider group is brought in to handle the situation, and they are not specifically prepared for the scenario that exists but must deal with it without undue delay. Examples include police, fire department, disaster response, and other public service rescue teams. In these cases, ongoing risk assessment by the involved personnel can advise appropriate action to reduce risk.[19] HM Fire Services Inspectorate has defined dynamic risk assessment (DRA) as:
The continuous assessment of risk in the rapidly changing circumstances of an operational incident, in order to implement the control measures necessary to ensure an acceptable level of safety.[19]
Dynamic risk assessment is the final stage of an integrated safety management system that can provide an appropriate response during changing circumstances. It relies on experience, training and continuing education, including effective debriefing to analyse not only what went wrong, but also what went right, and why, and to share this with other members of the team and the personnel responsible for the planning level risk assessment.[19]
Fields of application
editThe application of risk assessment procedures is common in a wide range of fields, and these may have specific legal obligations, codes of practice, and standardised procedures. Some of these are listed here.
General human health
editThere are many resources that provide human health risk information:
The National Library of Medicine provides risk assessment and regulation information tools for a varied audience.[20] These include:
- TOXNET (databases on hazardous chemicals, environmental health, and toxic releases),[21]
- the Household Products Database (potential health effects of chemicals in over 10,000 common household products),[22]
- TOXMAP (maps of the U.S. Environmental Protection Agency Superfund and Toxics Release Inventory data).
The United States Environmental Protection Agency provides basic information about environmental health risk assessments for the public for a wide variety of possible environmental exposures.[23]
The Environmental Protection Agency began actively using risk assessment methods to protect drinking water in the United States after the passage of the Safe Drinking Water Act of 1974. The law required the National Academy of Sciences to conduct a study on drinking water issues, and in its report, the NAS described some methodologies for doing risk assessments for chemicals that were suspected carcinogens, recommendations that top EPA officials have described as perhaps the study's most important part.[24]
Considering the increase in junk food and its toxicity, FDA required in 1973 that cancer-causing compounds must not be present in meat at concentrations that would cause a cancer risk greater than 1 in a million over a lifetime. The US Environmental Protection Agency provides extensive information about ecological and environmental risk assessments for the public via its risk assessment portal.[25] The Stockholm Convention on persistent organic pollutants (POPs) supports a qualitative risk framework for public health protection from chemicals that display environmental and biological persistence, bioaccumulation, toxicity (PBT) and long range transport; most global chemicals that meet this criterion have been previously assessed quantitatively by national and international health agencies.[26]
For non-cancer health effects, the terms reference dose (RfD) or reference concentration (RfC) are used to describe the safe level of exposure in a dichotomous fashion. Newer ways of communicating the risk is the probabilistic risk assessment.[27]
Small sub-populations
editWhen risks apply mainly to small sub-populations, it can be difficult to determine when intervention is necessary. For example, there may be a risk that is very low for everyone, other than 0.1% of the population. It is necessary to determine whether this 0.1% is represented by:
- all infants younger than X days or
- recreational users of a particular product.
If the risk is higher for a particular sub-population because of abnormal exposure rather than susceptibility, strategies to further reduce the exposure of that subgroup are considered. If an identifiable sub-population is more susceptible due to inherent genetic or other factors, public policy choices must be made. The choices are:
- to set policies for protecting the general population that are protective of such groups, e.g. for children when data exists, the Clean Air Act for populations such as asthmatics or
- not to set policies, because the group is too small, or the costs too high.
Acceptable risk criteria
editAcceptable risk is a risk that is understood and tolerated usually because the cost or difficulty of implementing an effective countermeasure for the associated vulnerability exceeds the expectation of loss.[28]
The idea of not increasing lifetime risk by more than one in a million has become commonplace in public health discourse and policy.[29] It is a heuristic measure. It provides a numerical basis for establishing a negligible increase in risk.
Environmental decision making allows some discretion for deeming individual risks potentially "acceptable" if less than one in ten thousand chance of increased lifetime risk. Low risk criteria such as these provide some protection for a case where individuals may be exposed to multiple chemicals e.g. pollutants, food additives, or other chemicals.[citation needed]
In practice, a true zero-risk is possible only with the suppression of the risk-causing activity.[citation needed]
Stringent requirements of 1 in a million may not be technologically feasible or may be so prohibitively expensive as to render the risk-causing activity unsustainable, resulting in the optimal degree of intervention being a balance between risks vs. benefit.[citation needed] For example, emissions from hospital incinerators result in a certain number of deaths per year. However, this risk must be balanced against the alternatives. There are public health risks, as well as economic costs, associated with all options. The risk associated with no incineration is the potential spread of infectious diseases or even no hospitals. Further investigation identifies options such as separating noninfectious from infectious wastes, or air pollution controls on a medical incinerator.
Intelligent thought about a reasonably full set of options is essential. Thus, it is not unusual for there to be an iterative process between analysis, consideration of options, and follow up analysis.[citation needed]
Public health
editIn the context of public health, risk assessment is the process of characterizing the nature and likelihood of a harmful effect to individuals or populations from certain human activities. Health risk assessment can be mostly qualitative or can include statistical estimates of probabilities for specific populations. In most countries, the use of specific chemicals or the operations of specific facilities (e.g. power plants, manufacturing plants) is not allowed unless it can be shown that they do not increase the risk of death or illness above a specific threshold. For example, the American Food and Drug Administration (FDA) regulates food safety through risk assessment, while the EFSA does the same in EU.[30]
An occupational risk assessment is an evaluation of how much potential danger a hazard can have to a person in a workplace environment. The assessment takes into account possible scenarios in addition to the probability of their occurrence and the results.[31] The five types of hazards to be aware of are safety (those that can cause injury), chemicals, biological, physical, and ergonomic (those that can cause musculoskeletal disorders).[32] To appropriately access hazards there are two parts that must occur. Firstly, there must be an "exposure assessment" which measures the likelihood of worker contact and the level of contact. Secondly, a "risk characterization" must be made which measures the probability and severity of the possible health risks.[33]
Human settlements
editThe importance of risk assessments to manage the consequences of climate change and variability is recalled in the global frameworks for disaster risk reduction, adopted by the member countries of the United Nations at the end of the World Conferences held in Kobe (2005) and Sendai (2015). The Sendai Framework for Disaster Risk Reduction brings attention to the local scale and encourages a holistic risk approach, which should consider all the hazards to which a community is exposed, the integration of technical-scientific knowledge with local knowledge, and the inclusion of the concept of risk in local plans to achieve a significant disaster reduction by 2030. Taking these principles into daily practice poses a challenge for many countries. The Sendai framework monitoring system highlights how little is known about the progress made from 2015 to 2019 in local disaster risk reduction.[34]
Sub-Saharan Africa
editAs of 2019, in the South of the Sahara, risk assessment is not yet an institutionalized practice. The exposure of human settlements to multiple hazards (hydrological and agricultural drought, pluvial, fluvial and coastal floods) is frequent and requires risk assessments on a regional, municipal, and sometimes individual human settlement scale. The multidisciplinary approach and the integration of local and technical-scientific knowledge are necessary from the first steps of the assessment. Local knowledge remains unavoidable to understand the hazards that threaten individual communities, the critical thresholds in which they turn into disasters, for the validation of hydraulic models, and in the decision-making process on risk reduction. On the other hand, local knowledge alone is not enough to understand the impacts of future changes and climatic variability and to know the areas exposed to infrequent hazards. The availability of new technologies and open access information (high resolution satellite images, daily rainfall data) allow assessment today with an accuracy that only 10 years ago was unimaginable. The images taken by unmanned vehicle technologies allow to produce very high resolution digital elevation models and to accurately identify the receptors.[35] Based on this information, the hydraulic models allow the identification of flood areas with precision even at the scale of small settlements.[36] The information on loss and damages and on cereal crop at individual settlement scale allow to determine the level of multi-hazard risk on a regional scale.The multi-temporal high-resolution satellite images allow to assess the hydrological drought and the dynamics of human settlements in the flood zone.[37] Risk assessment is more than an aid to informed decision making about risk reduction or acceptance.[38] It integrates early warning systems by highlighting the hot spots where disaster prevention and preparedness are most urgent.[39] When risk assessment considers the dynamics of exposure over time, it helps to identify risk reduction policies that are more appropriate to the local context. Despite these potentials, the risk assessment is not yet integrated into the local planning in the South of the Sahara which, in the best of cases, uses only the analysis of vulnerability to climate change and variability.[39]
Auditing
editFor audits performed by an outside audit firm, risk assessment is a crucial stage before accepting an audit engagement. According to ISA315 Understanding the Entity and its Environment and Assessing the Risks of Material Misstatement, "the auditor should perform risk assessment procedures to obtain an understanding of the entity and its environment, including its internal control". Evidence relating to the auditor's risk assessment of a material misstatement in the client's financial statements. Then, the auditor obtains initial evidence regarding the classes of transactions at the client and the operating effectiveness of the client's internal controls. Audit risk is defined as the risk that the auditor will issue a clean unmodified opinion regarding the financial statements, when in fact the financial statements are materially misstated, and therefore do not qualify for a clean unmodified opinion. As a formula, audit risk is the product of two other risks: Risk of Material Misstatement and Detection Risk. This formula can be further broken down as follows: inherent risk × control risk × detection risk.
Project management
editIn project management, risk assessment is an integral part of the risk management plan, studying the probability, the impact, and the effect of every known risk on the project, as well as the corrective action to take should an incident be implied by a risk occur.[40] Of special consideration in this area are the relevant codes of practice that are enforced in the specific jurisdiction. Understanding the regime of regulations that risk management must abide by is integral to formulating safe and compliant risk assessment practices.
Information security
editInformation technology risk assessment can be performed by a qualitative or quantitative approach, following different methodologies. One important difference[clarification needed] in risk assessments in information security is modifying the threat model to account for the fact that any adversarial system connected to the Internet has access to threaten any other connected system.[41] Risk assessments may therefore need to be modified to account for the threats from all adversaries, instead of just those with reasonable access as is done in other fields.
NIST Definition: The process of identifying risks to organizational operations (including mission, functions, image, reputation), organizational assets, individuals, other organizations, and the Nation, resulting from the operation of an information system. Part of risk management incorporates threat and vulnerability analyses and considers mitigations provided by security controls planned or in place.[42]
There are various risk assessment methodologies and frameworks available which include NIST Risk Management Framework (RMF),[43] Control Objectives for Information and Related Technologies (COBIT),[44] Factor Analysis of Information Risk (FAIR),[45] Operationally Critical Threat, Asset, and Vulnerability Evaluation (OCTAVE),[46] The Center for Internet Security Risk Assessment Method (CIS RAM),[47] and The Duty of Care Risk Analysis (DoCRA) Standard,[48] which helps define 'reasonable' security.
Cybersecurity
editThe Threat and Risk Assessment (TRA) process is part of risk management referring to risks related to cyber threats. The TRA process will identify cyber risks, assess risks' severities, and may recommend activities to reduce risks to an acceptable level.
There are different methodologies for performing TRA (e.g., Harmonized TRA Methodology[49]), all utilize the following elements:[50][51][52] identifying of assets (what should be protected), identifying and assessing of the threats and vulnerabilities for the identified assets, determining the exploitability of the vulnerabilities, determining the levels of risk associated with the vulnerabilities (what are the implications if the assets were damaged or lost), and recommending a risk mitigation program.
Megainvestment projects
editMegaprojects (sometimes also called "major programs") are extremely large-scale investment projects, typically costing more than US$1 billion per project. They include bridges, tunnels, highways, railways, airports, seaports, power plants, dams, wastewater projects, coastal flood protection, oil and natural gas extraction projects, public buildings, information technology systems, aerospace projects, and defence systems. Megaprojects have been shown to be particularly risky in terms of finance, safety, and social and environmental impacts.
Software evolution
editStudies have shown that early parts of the system development cycle such as requirements and design specifications are especially prone to error. This effect is particularly notorious in projects involving multiple stakeholders with different points of view. Evolutionary software processes offer an iterative approach to requirement engineering to alleviate the problems of uncertainty, ambiguity, and inconsistency inherent in software developments, including uncertainty, ambiguity, and inconsistency inherent in software developments.[clarification needed]
Shipping industry
editIn July 2010, shipping companies agreed to use standardized procedures in order to assess risk in key shipboard operations. These procedures were implemented as part of the amended ISM Code.[53]
Underwater diving
editFormal risk assessment is a required component of most professional dive planning, but the format and methodology may vary. Consequences of an incident due to an identified hazard are generally chosen from a small number of standardised categories, and probability is estimated based on statistical data on the rare occasions when it is available, and on a best guess estimate based on personal experience and company policy in most cases. A simple matrix is often used to transform these inputs into a level of risk, generally expressed as unacceptable, marginal or acceptable. If unacceptable, measures must be taken to reduce the risk to an acceptable level, and the outcome of the risk assessment must be accepted by the affected parties before a dive commences. Higher levels of risk may be acceptable in special circumstances, such as military or search and rescue operations when there is a chance of recovering a survivor. Diving supervisors are trained in the procedures of hazard identification and risk assessment, and it is part of their planning and operational responsibility. Both health and safety hazards must be considered. Several stages may be identified. There is risk assessment done as part of the diving project planning, on site risk assessment which takes into account the specific conditions of the day, and dynamic risk assessment which is ongoing during the operation by the members of the dive team, particularly the supervisor and the working diver.[54][55]
In recreational scuba diving, the extent of risk assessment expected of the diver is relatively basic and is included in the pre-dive checks. Several mnemonics have been developed by diver certification agencies to remind the diver to pay some attention to risk, but the training is rudimentary. Diving service providers are expected to provide a higher level of care for their customers, and diving instructors and divemasters are expected to assess risk on behalf of their customers and warn them of site-specific hazards and the competence considered appropriate for the planned dive. Technical divers are expected to make a more thorough assessment of risk, but as they will be making an informed choice for a recreational activity, the level of acceptable risk may be considerably higher than that permitted for occupational divers under the direction of an employer.[56][57]
Outdoor and wilderness adventure
editIn outdoor activities including commercial outdoor education, wilderness expeditions, and outdoor recreation, risk assessment refers to the analysis of the probability and magnitude of unfavorable outcomes such as injury, illness, or property damage due to environmental and related causes, compared to the human development or other benefits of outdoor activity. This is of particular importance as school programs and others weigh the benefits of youth and adult participation in various outdoor learning activities against the inherent and other hazards present in those activities. Schools, corporate entities seeking team-building experiences, parents/guardians, and others considering outdoor experiences expect or require[58] organizations to assess the hazards and risks of different outdoor activities—such as sailing, target shooting, hunting, mountaineering, or camping—and select activities with acceptable risk profiles.
Outdoor education, wilderness adventure, and other outdoor-related organizations should, and are in some jurisdictions required, to conduct risk assessments prior to offering programs for commercial purposes.[59][60][61]
Such organizations are given guidance on how to provide their risk assessments.[62]
Risk assessments for led outdoor activities form only one component of a comprehensive risk management plan, as many risk assessments use a basic linear-style thinking that does not employ more modern risk management practice employing complex socio-technical systems theory.[63][64]
Environment
editEnvironmental Risk Assessment (ERA) aims to assess the effects of stressors, usually chemicals, on the local environment. A risk is an integrated assessment of the likelihood and severity of an undesired event. In ERA, the undesired event often depends on the chemical of interest and on the risk assessment scenario.[65] This undesired event is usually a detrimental effect on organisms, populations or ecosystems. Current ERAs usually compare an exposure to a no-effect level, such as the Predicted Environmental Concentration/Predicted No-Effect Concentration (PEC/PNEC) ratio in Europe. Although this type of ratio is useful and often used in regulation purposes, it is only an indication of an exceeded apparent threshold.[66] New approaches start to be developed in ERA in order to quantify this risk and to communicate effectively on it with both the managers and the general public.[65]
Ecological risk assessment is complicated by the fact that there are many nonchemical stressors that substantially influence ecosystems, communities, and individual plants and animals, as well as across landscapes and regions.[67][68] Defining the undesired (adverse) event is a political or policy judgment, further complicating applying traditional risk analysis tools to ecological systems. Much of the policy debate surrounding ecological risk assessment is over defining precisely what is an adverse event.[69]
Biodiversity
editBiodiversity Risk Assessments evaluate risks to biological diversity, specially the risk of species extinction or the risk of ecosystem collapse. The units of assessments are the biological (species, subspecies or populations) or ecological entities (habitats, ecosystems, etc.), and the risk are often related to human actions and interventions (threats and pressures). Regional and national protocols have been proposed by multiple academic or governmental institutions and working groups,[70] but global standards such as the Red List of Threatened Species and the IUCN Red List of Ecosystems have been widely adopted, and are recognized or proposed as official indicators of progress toward international policy targets and goals, such as the Aichi targets and the Sustainable Development Goals.[71][72]
Law
editRisk assessments are used in numerous stages during the legal process and are developed to measure a wide variety of items, such as recidivism rates, potential pretrial issues, probation/parole, and to identify potential interventions for defendants.[73] Clinical psychologists, forensic psychologists, and other practitioners are responsible for conducting risk assessments.[73][74][75] Depending on the risk assessment tool, practitioners are required to gather a variety of background information on the defendant or individual being assessed. This information includes their previous criminal history (if applicable) and other records (i.e. Demographics, Education, Job Status, Medical History), which can be accessed through direct interview with the defendant or on-file records.[73]
In the pre-trial stage, a widely used risk assessment tool is the Public Safety Assessment,[76] which predicts failure to appear in court, likelihood of a new criminal arrest while on pretrial release, and likelihood of a new violent criminal arrest while on pretrial release. Multiple items are observed and taken into account based on which aspect of the PSA is being focused, and like all other actuarial risk assessments, each item is assigned a weighted amount to produce a final score.[73] Detailed information such as transparency on the items the PSA factors and how scores are distributed are accessible online.[77]
For defendants who have been incarcerated, risk assessments are used to determine their likelihood of recidivism and inform sentence length decisions. Risk assessments also aid parole/probation officers in determining the level of supervision a probationer should be subjected to and what interventions could be implemented to improve offender risk status.[74] The Correctional Offender Management Profiling for Alternative Sanctions (COMPAS) is a risk assessment too designed to measure pretrial release risk, general recidivism risk, and violent recidivism risk. Detailed information on scoring and algorithms for COMPAS are not accessible to the general public.
See also
edit- Acceptable loss – Military euphemism
- Benefit shortfall – When the actual benefits of a venture are less than the projected or estimated benefits
- Control self-assessment – Technique to assess process effectiveness
- Cost overrun – Unexpected incurred costs in excess of budgeted amounts
- Digital continuity
- Duty of care – Type of legal obligation
- Edwards v National Coal Board – 1949 English Court of Appeal ruling on the meaning of "reasonably practicable"
- Extreme risk – risk of very bad outcomes or "high consequence", but of low probability.
- Environmental impact assessment – Assessment of the environmental consequences of a decision before action
- Flood risk assessment – assessment of the risk of flooding from all flooding mechanisms, identification of flood mitigation measures and recommendation on actions to be taken before and during a flood
- Form 696 – Metropolitan Police risk assessment
- Global catastrophic risk – Hypothetical global-scale disaster risk
- Hazard – Situation or object that can cause damage
- Hazard analysis – Method for assessing risk
- Hazard analysis and critical control points (HACCP) – Systematic preventive approach to food safety risk assessment in food
- Health impact assessment – method to assess impacts of an action or risk factor on health and to produce a set of evidence-based recommendations to inform decision-making
- Horizon scanning – Methodology in futures studies
- Information assurance – Multi-disciplinary methods for decision support systems security
- Index of auditing-related articles
- ISO 28000 – Management system standard
- ISO 31000 – Family of standards relating to risk management
- ISSOW – standard in the Petroleum industry
- Megaprojects and Risk – 2003 book by Bent Flyvbjerg, Nils Bruzelius and Werner Rothengatter
- Network theory in risk assessment
- Occupational exposure banding – Process to assign chemicals into categories corresponding to permissible exposure concentrations
- Optimism bias – Type of cognitive bias
- PIMEX a video exposure monitoring method
- Planning fallacy – Cognitive bias of underestimating time needed
- Probabilistic risk assessment – Methodology for evaluating risks
- Probit model – Statistical regression where the dependent variable can take only two values
- Project risk management – activities to minimize project risks and ensure a project completes on time and on budget, while meeting its objectives
- Reference class forecasting – Method of predicting the future
- Reliability engineering – Sub-discipline of systems engineering that emphasizes dependability
- Risk – The possibility of something bad happening
- Risk assessment using qualifiers – estimate of risk associated with a particular hazard using qualifiers like high likelihood, low likelihood, etc
- Risk-based auditing – type of auditing which focuses upon the analysis and management of risks with the greatest potential impact
- Risk management tools – tool addressing uncertainty by identifying and generating metrics, parameterizing, prioritizing, and developing responses, and tracking risk
- Risk matrix – Risk assessment comparing the likelihood of a risk to its severity
- Safety engineering – Engineering discipline which assures that engineered systems provide acceptable levels of safety
- Security risk – The possibility of something bad happening
- Statistical risk
- Strategic misrepresentation – Cognitive bias of underestimating time needed
- Gordon–Loeb model for cyber security investments
References
editReferences
edit- ^ a b c d e f g Rausand M (2013). "Chapter 1: Introduction". Risk Assessment: Theory, Methods, and Applications. John Wiley & Sons. pp. 1–28. ISBN 9780470637647.
- ^ a b c Manuele FA (2016). "Chapter 1: Risk Assessments: Their Significance and the Role of the Safety Professional". In Popov G, Lyon BK, Hollcraft B (eds.). Risk Assessment: A Practical Guide to Assessing Operational Risks. John Wiley & Sons. pp. 1–22. ISBN 9781118911044.
- ^ a b c d e f g Levi R (1 June 2018). "Getting Real About Both Benefits and Risks". Science & Practice, English Special 2018. Swedish Agency of Health Technology Assessment and Assessment of Social Services. ISSN 1104-1250. Retrieved 2018-06-14.
- ^ a b c Varshavsky JR, Rayasam SD, Sass JB, Axelrad DA, Cranor CF, Hattis D, et al. (January 2023). "Current practice and recommendations for advancing how human variability and susceptibility are considered in chemical risk assessment". Environmental Health. 21 (Suppl 1): 133. Bibcode:2023EnvHe..21S.133V. doi:10.1186/s12940-022-00940-1. PMC 9835253. PMID 36635753.
- ^ Hoffmann TC, Del Mar C (February 2015). "Patients' expectations of the benefits and harms of treatments, screening, and tests: a systematic review" (PDF). JAMA Internal Medicine. 175 (2): 274–86. doi:10.1001/jamainternmed.2014.6016. PMID 25531451.
- ^ Stacey D, Légaré F, Lewis K, Barry MJ, Bennett CL, Eden KB, et al. (April 2017). "Decision aids for people facing health treatment or screening decisions". The Cochrane Database of Systematic Reviews. 2017 (4): CD001431. doi:10.1002/14651858.CD001431.pub5. PMC 6478132. PMID 28402085.
- ^ Rausand M (2013). "Chapter 6: Accident Models". Risk Assessment: Theory, Methods, and Applications. John Wiley & Sons. pp. 137–76. ISBN 9780470637647.
- ^ a b Vamanu BI, Gheorghe AV, Kaina PF (2016). Critical Infrastructures: Risk and Vulnerability Assessment in Transportation of Dangerous Goods: Transportation by Road and Rail. Springer. p. 11. ISBN 9783319309316.
- ^ Lacey P (2011). "An Application of Fault Tree Analysis to the Identification and Management of Risks in Government Funded Human Service Delivery". Proceedings of the 2nd International Conference on Public Policy and Social Sciences. SSRN 2171117.
- ^ Shirey R (August 2007). "Internet Security Glossary, Version 2". Network Working Group. The IETF Trust: 9. Retrieved 19 July 2018.
- ^ Mandelbrot B, Hudson RL (2008). The (mis)Behaviour of Markets: A Fractal View of Risk, Ruin and Reward. London: Profile Books. ISBN 9781846682629.
- ^ Kasperson RE, Renn O, Slovic P, Brown HS, Emel J, Goble R, Kasperson JX, Ratick S (1988). "The social amplification of risk: A conceptual framework" (PDF). Risk Analysis. 8 (2): 177–187. Bibcode:1988RiskA...8..177K. doi:10.1111/j.1539-6924.1988.tb01168.x.
- ^ Commoner B. "Comparing apples to oranges: Risk of cost/benefit analysis". In Iannone AP (ed.). Contemporary moral controversies in technology. pp. 64–65.
- ^ O'Brien M (2002). Making better environmental decisions: an alternative to risk assessment. Cambridge, Massachusetts: MIT Press. ISBN 0-262-65053-3. Retrieved 27 September 2010.
- ^ Shrader-Frechette K, Westra L (October 1997). Technology and Values. Lanham, Md.: Rowman & Littlefield Publishers. ISBN 978-1-4616-4399-9.
- ^ Taleb NN (September 2008). The fourth quadrant: a map of the limits of statistics (PDF). An Edge original essay (Report).
- ^ Holzmann R, Jørgensen S (2001). "Social Risk Management: A New Conceptual Framework for Social Protection, and Beyond". International Tax and Public Finance. 8 (4): 529–56. doi:10.1023/A:1011247814590. S2CID 14180040.
- ^ Nakaš N (21 November 2017). "Three Lessons About Risk Management from Everyday Life". Knowledge Hub. Center of Excellence in Finance. Retrieved 19 July 2018.
- ^ a b c d Lock G (June 2017). Phillips M (ed.). "Public Safety Diving-Dynamic Risk Assessment" (PDF). PS Diver Magazine (116): 9. Retrieved 20 June 2017.
- ^ "Risk Assessment and Regulation Information from the NLM". National Library of Medicine. Retrieved 9 June 2013.
- ^ "Databases on toxicology, hazardous chemicals, environmental health, and toxic releases". TOXNET. NLM. May 2012. Retrieved 9 June 2013.
- ^ "Household Products Database". U.S. Dept. of Health & Human Services. January 2013. Retrieved 9 June 2013.
- ^ "Risk Assessment Portal". EPA. 13 May 2013. Retrieved 9 June 2013.
- ^ EPA Alumni Association: Senior EPA officials discuss early implementation of the Safe Drinking Water Act of 1974, Video, Transcript (see pages 11,14).
- ^ "Risk Assessment". www.epa.gov. US Environmental Protection Agency. 2013-09-26. Retrieved 2016-04-07.
- ^ Szabo DT, Loccisano AE (March 30, 2012). "POPs and Human Health Risk Assessment". Dioxins and Persistent Organic Pollutants (3rd ed.). pp. 579–618. doi:10.1002/9781118184141.ch19. ISBN 9781118184141.
- ^ Nielsen GH, Heiger-Bernays WJ, Levy JI, White RF, Axelrad DA, Lam J, et al. (January 2023). "Application of probabilistic methods to address variability and uncertainty in estimating risks for non-cancer health effects". Environmental Health. 21 (Suppl 1): 129. Bibcode:2023EnvHe..21S.129N. doi:10.1186/s12940-022-00918-z. PMC 9835218. PMID 36635712.
- ^ R. Shirey (August 2007). Internet Security Glossary, Version 2. Network Working Group. doi:10.17487/RFC4949. RFC 4949. Informational.
- ^ Hunter PR, Fewtrell L (2001). "Acceptable Risk" (PDF). World Health Organization.
- ^ Merrill RA (1997). "Food safety regulation: reforming the Delaney Clause". Annual Review of Public Health. 18: 313–40. doi:10.1146/annurev.publhealth.18.1.313. PMID 9143722. This source includes a useful historical survey of prior food safety regulation.
- ^ Current intelligence bulletin 69: NIOSH practices in occupational risk assessment (Report). 2020-02-01. doi:10.26616/nioshpub2020106.
- ^ "OSHA's 5 Workplace Hazards". Grainger Industrial Supply.
- ^ Waters M, McKernan L, Maier A, Jayjock M, Schaeffer V, Brosseau L (2015-11-25). "Exposure Estimation and Interpretation of Occupational Risk: Enhanced Information for the Occupational Risk Manager". Journal of Occupational and Environmental Hygiene. 12 (Suppl 1): S99-111. doi:10.1080/15459624.2015.1084421. PMC 4685553. PMID 26302336.
- ^ UNDRR (2019). Global Assessment Report on Disaster Risk Reduction. Geneva: UNDRR. p. 472. ISBN 978-92-1-004180-5. Retrieved 22 June 2020.
- ^ Tiepolo M (2019). "Flood Assessment for Risk-Informed Planning along the Sirba River, Niger". Sustainability. 11 (4003). doi:10.3390/w11051018.
- ^ Massazza G (2019). "Flood Hazard Scenarios of the Sirba River (Niger): Evaluation of the Hazard Thresholds and Flooding Areas". Water. 11 (5): 1018. doi:10.3390/w11051018.
- ^ Tiepolo M (2018). "Multihazard Risk Assessment for Planning with Climate in the Dosso Region, Niger". Climate. 6 (67): 67. Bibcode:2018Clim....6...67T. doi:10.3390/cli6030067.
- ^ International Organization for Standardization (8 November 2017). "ISO Guide 73: 2009. Risk management – Vocabulary". ISO. Retrieved 22 June 2020.
- ^ a b Tarchiani V (2020). "Community and Impact Based Early Warning System for Flood Risk Preparedness: The Experience of the Sirba River in Niger". Sustainability. 12 (2196). doi:10.3390/su12062196.
- ^ Managing Project Risks - Retrieved May 20th, 2010
- ^ Spring J, Kern S, Summers A (2015-05-01). "Global adversarial capability modeling". 2015 APWG Symposium on Electronic Crime Research (eCrime). pp. 1–21. doi:10.1109/ECRIME.2015.7120797. ISBN 978-1-4799-8909-6. S2CID 24580989.
- ^ "Risk assessment". NIST Computer Security Resource Center Glossary. National Institute of Standards and Technology (NIST).
- ^ "NIST". NIST. 30 November 2016.
- ^ "ISACA COBIT". ISACA.
- ^ "FAIR". FAIR.
- ^ "Carnegie Mellon University". Software Engineering Institute, Carnegie Mellon University. 31 August 1999.
- ^ "Center for Internet Security". Center for Internet Security (CIS).
- ^ "DoCRA". Duty of Care Risk Analysis (DoCRA).
- ^ Canadian Centre for Cyber Security (2018-08-15). "Canadian Centre for Cyber Security". Canadian Centre for Cyber Security. Retrieved 2021-08-09.
- ^ Baingo D (2021). "Threat Risk Assessment (TRA) for Physical Security". In Masys AJ (ed.). Sensemaking for Security. Advanced Sciences and Technologies for Security Applications. Cham: Springer International Publishing. pp. 243–270. doi:10.1007/978-3-030-71998-2_14. ISBN 978-3-030-71998-2. S2CID 236706551.
- ^ "An Overview of Threat and Risk Assessment | SANS Institute". www.sans.org. Retrieved 2021-08-09.
- ^ Treasury Board of Canada Secretariat (2006-03-06). "Rescinded [2019-06-28] - Security Organization and Administration Standard". www.tbs-sct.gc.ca. Retrieved 2021-08-09.
- ^ "ISM CODE – Amendments from 1st July 2010 Risk Assessment". Archived from the original on 27 April 2014.
- ^ "Diving Regulations 2009". Occupational Health and Safety Act 85 of 1993 – Regulations and Notices – Government Notice R41. Pretoria: Government Printer. Archived from the original on 4 November 2016. Retrieved 3 November 2016 – via Southern African Legal Information Institute.
- ^ Staff (August 2016). "15 - General safety requirements". Guidance for diving supervisors IMCA D 022 (Revision 1 ed.). London, UK: International Marine Contractors Association. pp. 15–5.
- ^ Staff (1977). "The Diving at Work Regulations 1997". Statutory Instruments 1997 No. 2776 Health and Safety. Kew, Richmond, Surrey: Her Majesty's Stationery Office (HMSO). Retrieved 6 November 2016.
- ^ Gurr K (August 2008). "13: Operational Safety". In Mount T, Dituri J (eds.). Exploration and Mixed Gas Diving Encyclopedia (1st ed.). Miami Shores, Florida: International Association of Nitrox Divers. pp. 165–180. ISBN 978-0-915539-10-9.
- ^ "2018 Accreditation Rubric" (PDF). Seattle, Washington: Northwest Association of Independent Schools.
- ^ "Adventure Activities Regulations". supportadventure.co.nz.
- ^ "Health and Safety at Work (Adventure Activities) Regulations 2016 (LI 2016/19)". New Zealand Legislation.
- ^ "Adventure Activities Licensing". The Health and Safety Executive (HSE). gov.uk.
- ^ "Adventure activities". Work Safe. New Zealand.
- ^ Dallat C, Salmon PM, Goode N (2015). "All about the Teacher, the Rain and the Backpack: The Lack of a Systems Approach to Risk Assessment in School Outdoor Education Programs". Procedia Manufacturing. 3: 1157–1164. doi:10.1016/j.promfg.2015.07.193.
- ^ Baierlein J (2019). Risk Management for Outdoor Programs: a Guide to Safety in Outdoor Education, Recreation and Adventure. Seattle, WA: Viristar LLC.
- ^ a b Goussen B, Price OR, Rendal C, Ashauer R (October 2016). "Integrated presentation of ecological risk from multiple stressors". Scientific Reports. 6: 36004. Bibcode:2016NatSR...636004G. doi:10.1038/srep36004. PMC 5080554. PMID 27782171.
- ^ Jager T, Heugens EH, Kooijman SA (April 2006). "Making sense of ecotoxicological test results: towards application of process-based models". Ecotoxicology. 15 (3): 305–14. Bibcode:2006Ecotx..15..305J. CiteSeerX 10.1.1.453.1811. doi:10.1007/s10646-006-0060-x. PMID 16739032. S2CID 18825042.
- ^ Goussen B, Rendal C, Sheffield D, Butler E, Price OR, Ashauer R (December 2020). "Bioenergetics modelling to analyse and predict the joint effects of multiple stressors: Meta-analysis and model corroboration". The Science of the Total Environment. 749: 141509. arXiv:2102.13107. Bibcode:2020ScTEn.74941509G. doi:10.1016/j.scitotenv.2020.141509. PMID 32827825.
- ^ Landis WG (2005). Regional scale ecological risk assessment : using the relative risk model. Boca Raton, FL: CRC Press. ISBN 1-56670-655-6. OCLC 74274833.
- ^ Lackey R (1997). "If ecological risk assessment is the answer, what is the question". Human and Ecological Risk Assessment. 3 (6): 921–928. Bibcode:1997HERA....3..921L. doi:10.1080/10807039709383735.
- ^ Nicholson E, Regan TJ, Auld TD, Burns EL, Chisholm LA, English V, et al. (2015). "Towards consistency, rigour and compatibility of risk assessments for ecosystems and ecological communities". Austral Ecology. 40 (4): 347–363. Bibcode:2015AusEc..40..347N. doi:10.1111/aec.12148. hdl:1885/66771. ISSN 1442-9985. S2CID 82412136.
- ^ Keith DA, Rodríguez JP, Brooks TM, Burgman MA, Barrow EG, Bland L, et al. (2015). "The IUCN Red List of Ecosystems: Motivations, Challenges, and Applications". Conservation Letters. 8 (3): 214–226. Bibcode:2015ConL....8..214K. doi:10.1111/conl.12167. hdl:10536/DRO/DU:30073631. ISSN 1755-263X.
- ^ Brooks TM, Butchart SH, Cox NA, Heath M, Hilton-Taylor C, Hoffmann M, et al. (2015). "Harnessing biodiversity and conservation knowledge products to track the Aichi Targets and Sustainable Development Goals". Biodiversity. 16 (2–3): 157–174. Bibcode:2015Biodi..16..157B. doi:10.1080/14888386.2015.1075903. ISSN 1488-8386.
- ^ a b c d "What is Risk Assessment". Bureau of Justice Assistance. U.S. Department of Justice.
- ^ a b Monahan J, Skeem JL (2016). "Risk Assessment in Criminal Sentencing". Annual Review of Clinical Psychology. 12: 489–513. doi:10.1146/annurev-clinpsy-021815-092945. PMID 26666966.
- ^ Heilbrun K (2009). "Risk Assessment in Evidence-Based Sentencing: Context and Promising Sues". Chapman Journal of Criminal Justice. 1: 127–142.
- ^ "Advancing Pretrial Policy & Research: What is the PSA?". Advancing Pretrial Policy and Research (APPR).
- ^ "How the PSA Works". Advancing Pretrial Policy and Research (APPR).
Further reading
edit- Dorne JC, Kass GE, Bordajandi LR, Amzal B, Bertelsen U, Castoldi AF, et al. (2011). "Chapter 2. Human Risk Assessment of Heavy Metals: Principles and Applications". In Sigel A, Sigel H, Sigel RK (eds.). Metal Ions in Toxicology. Metal Ions in Life Sciences. RSC Publishing. pp. 27–60. doi:10.1039/9781849732116-00027. ISBN 978-1-84973-091-4. S2CID 24530234.
- Mumtaz MM, Hansen H, Pohl HR (2011). "Chapter 3. Mixtures and Their Risk Assessment in Toxicology". In Sigel A, Sigel H, Sigel RK (eds.). Metal Ions in Toxicology. Metal Ions in Life Sciences. RSC Publishing. pp. 61–80. doi:10.1039/9781849732116-00061. ISBN 978-1-84973-091-4.
- Committee on Risk Assessment of Hazardous Air Pollutants (1994), Science and judgment in risk assessment, Washington, D.C: National Academy Press, ISBN 978-0-309-04894-1, retrieved 27 September 2010
- Hallenbeck WH (1986). Quantitative risk assessment for environmental and occupational health. Chelsea, Mich.: Lewis Publishers.
- Harremoës P (ed.). Late lessons from early warnings: the precautionary principle 1896–2000.
- Lachin JM. Biostatistical methods: the assessment of relative risks.
- Lerche I, Glaesser W (2006). Environmental risk assessment : quantitative measures, anthropogenic influences, human impact. Berlin: Springer. ISBN 978-3-540-26249-7. Retrieved 27 September 2010.
- Kluger J (November 26, 2006). "How Americans Are Living Dangerously". Time. Archived from the original on November 27, 2006. Retrieved 27 September 2010. Also published as December 4 cover title: "Why We Worry About the Wrong Things: The Psychology of Risk" |work=Time
- A Review of risk assessment methodologies (Report). Washington: U.S: Congressional Research Service, Library of Congress, for the Subcommittee on Science, Research, and Technology. 1983.
- Mayo DG (1997). "Sociological versus metascientific views of technological risk assessment". In Shrader-Frechette K, Westra L (eds.). Technology and values. Lanham, Maryland: Rowman & Littlefield. ISBN 978-0-8476-8631-5. Retrieved 27 September 2010.
- Rozell DJ (2020). Dangerous Science: Science Policy and Risk Analysis for Scientists and Engineers. London: Ubiquity Press. doi:10.5334/bci. ISBN 978-1-911529-90-3. S2CID 213952232.