A method of identifying and analyzing irrational system behavior in a system of systems

ing away from physical components and subsystems to the functional level can help practitioners to consider potential new initiating event sources that otherwise may bemissed. • Step 2, Part 2: Remove all flows from the list of potential irrationality initiators that are alreadymodeled as initiating events throughother failure analysis methods, such as FFIP and PRA. • Step 2, Part 3: Identify any potentially impossible candidate irrationality initiators that cannot be emitted by the generic black box system. Before eliminating a candidate irrationality initiator, the practitioner must attempt to identify ways that the irrationality initiator may be able to be generated even if it is highly implausible or unlikely. For instance, almost any material can produce spectral emissions that would normally be unexpectedwith sufficient energy applied to thematerial. • Step 2, Part 4: Assign probabilities of occurrence to each of the irrationality initiators remaining on the list. We advocate that practitioners follow initiating event probability guidance from PRA, such as Refs. 10 and 49. Now that potential irrationality initiators within an SoS that may impact the SoI have been identified and probabilities assigned, the flow paths by which the irrationality initiators enter the system must be defined. Irrationality initiators may be introduced to a system along nominal flow paths or along non-nominal flow paths, such as the uncoupled failure flow paths advanced in the UFFSR method.24 Additions to or modifications of the failure model for a systemmay be necessary to sufficiently capture irrationality initiator entry points. 3.3 Analysis of potential irrationality initiators The next step in the method is to conduct failure analysis on the SoI using the identified potential irrationality initiators. We advocate for and use in the case study the FFIP family of failure analysis tools to conduct failure analysis on the SoI. In order to produce a more accurate analysis of potential irrationality initiators using FFIP and F IGURE 2 Steps to developing irrationality initiators VANBOSSUYT ET AL. 525 related tools, we recommend that the analysis be performed using data collected from the proposed physical architecture that solves the functional architecture of the SoI. The number of potential failure scenarios, often called “cut-sets” in PRA and sometimes in FFIP, resulting from the analysis of irrationality initiators, is directly related to thenumberof irrationality initiators and the functionalmodel of the SoI. Each irrationality initiatormayproceed along many different flow paths in an SoI, causing functional failure along the way, which in turn may lead to system failure. The number of potential failure scenariosmay further be expanded by havingmultiple potential component solutions available for specific functions before down-selection of component solutions has been conducted. While probabilities for specific irrationality initiators were calculated in a prior step in the method, there are several options for how irrationality initiators are analyzed based on what type of analysis results a practitioner is interested in reviewing. These include an uninformative prior and an informative prior. Further, irrationality initiators that are either independent or dependent can be considered to provide additional insights into potential irrational failure scenarios, such as when multiple irrationality initiators often occur together. Informative and uninformative priors, and independent and dependent irrationality initiators may be combined together. Further explanation immediately follows: 3.3.1 Uninformative and informative priors In order to understand the sensitivity of an SoI to irrationality initiators, the uninformative prior sets all irrationality initiators to the same probability of occurrence. It should be noted that using the uninformative prior approachdoes not allow for direct comparisonof resultswith other FFIP results. The results are specifically useful to understand what high severity failure outcomes are present that otherwise may be truncated during computation. The uninformative prior method can also be used to perform a sensitivity analysis on the irrationality initiators by changing their probabilities and comparing results. This may help to identify irrationality initiators that are not particularly sensitive to changes in their probabilities of occurrence and may also identify specific irrationality initiators that warrant extended scrutiny to ensure a higher degree of accuracy and realism in the probability statistics. In contrast to the uninformative prior that uses arbitrarily assigned probabilities to determine potential low probability but very severe outcomes and to examine irrationality initiator probability sensitivity, the informative prior uses probabilities of occurrence that were already developed in a previous step of the methodology and that are based in reality. This allows for direct comparison of irrationality initiator-derived failure scenario probabilities with failure scenario probabilities produced from FFIP. In the event that a probability was unable to be developed previously because of a lack of information, we suggest using a probability value that is 3x the highest probability of the highest known irrationality initiator probability. Using a 3x higher probability may help to ensure that any potential high consequence failure scenarios are identified and will help to motivate the development of a better estimation of the probability. If a failure scenario of a particular irrationality initiator that used the 3x higher probability is sufficiently probable, then this indicates the irrationality initiator probability needs to be better understood. However, if no failure scenarios are within a few orders of magnitude of the most likely failure scenario, then this is an indication that there is likely no further refinement of that irrationality initiator’s probability. It is worth noting that we do not advocate for setting the multiplier higher than 3x for irrationality initiators without well-founded probabilities. While such an approach would almost certainly highlight every single potential failure scenario caused by the irrationality initiator in question, setting the irrationality initiator probability needlessly high without a rigorous analysis to back up the choice is likely to overwhelm a user of this method with many failure scenarios that masquerade as being of high likelihood while in reality being of vanishingly small probability. This in turn may lead to much wasted time and effort to disprove all of the failure scenarios. The suggestion of a 3x multiplier is based on our prior professional experience as risk analysts and reliability engineers and from examining the sensitivity of several failure models to which we have access to changing initiating event probabilities.Whilewe believe the 3x multiplier is a good starting point, we recommend that systems engineering practitioners carefully examine the sensitivity of their own systems to initiating event probabilities and make adjustments as warranted and using their professional engineering judgment. We recommend that both the informative and uninformative prior methods are used to analyze irrationality initiators in the SoI. The uninformative prior can shed light on potential high consequence failure scenarios that otherwise may be missed and can also be used to perform sensitivity studies on the irrationality initiators. The informative prior quantifies failure scenarios in a way that can be directly compared with standard FFIP results. This may help practitioners to prioritize wheremoney and time is spent tomitigate potential issues. 3.3.2 Independent and dependent irrationality initiators In almost every implementation of FFIP that we have encountered, initiating events are exclusively considered to be independent from each other. The same is true in many PRA analyses. However, we suspect based on our professional practice and observations that irrationality initiators may have a higher likelihood of being dependent upon one another to some extent. In other words, if one irrationality initiator occurs, then it is more likely that another will occur at the same time. We propose that irrationality initiators should be modeled both as independent and dependent events. By analyzingmultiple irrationality initiators as single events, a practitioner can gain insight into scenarios where a system in an SoS begins emitting many irrationality initiators. This may help to identify “worst case scenarios” where completely unanticipated emergent system behaviors occur due to the SoI receiving several irrationality initiators at once. Recent research on external initiating events for autonomous robotic systems has indicated that unique emergent system behaviors not predicted by other research 526 VANBOSSUYT ET AL. methods can be caused by several external initiating events simultaneously occurring and interacting with one another inside of an SoI.59,89 We suggest that all possible irrationality initiator-dependent combinations be investigated. For example, in the case of three irrationality initiators [A, B, C], the following initiator-dependent combinations should be investigated: [A & B], [A & C], [B & C], and [A, B, & C]. A generalized formula to determine the number of dependent combinations is shown by Equation (1). Note that the formula intentionally subtracts 1 to acknowledge that the baseline case of no irrationality initiators being present in the SoI is assumed to have been previously assessed.