Overall:
Once installed the new version ran well for me (actually seems faster than v 1.0). I had no crashes or other glitches while I was using the tool.
Installation:
This will be the biggest hurdle for getting new users on-board with the tool as the process for installation will be onerous for many. More detailed step-by-step instructions would help. A couple of suggestions based on my experience:
-
I had to uninstall the “ExpostatsExcelAddIn” file used for the earlier version of IHSTAT_Bayes in order to proceed with “Install” of version 2.0. This should be mentioned in the install instructions along with steps on how to do the uninstall.
-
More information on where to go to find the starting “options” for setting up the install folder as a “trusted Location” would be useful for those of us less familiar with the Microsoft infrastructure. (Is Microsoft trust center a separate app? Part of windows? Part of Excel? Part of the folder? How does one get there?)
Combining Results From The Measurements with User-Selected Probabilities:
As you know, this is a wonderful feature if used carefully and appropriately. If not, it can lead to significant exposure decision errors. Unfortunately, I continue to see examples of gross misuse of similar features in place in the IHDA Bayesian tool because people generally 1) don’t understand the biases and accuracy limitations of “professional judgment” (may be true for exposure models as well but I have seen few real-world examples where model results were used to define the user probabilities); and 2) don’t understand the influence of extreme user probabilities on the final combined probabilities – particularly when the measurement probabilities are based on very small sample sizes. Here are a couple of suggestions for helping people avoid these errors.
-
Set the software up so that it prohibits user input of extreme probability values Suggest an upper limit of 50% in any category for both the 5 AIHA categories and the 11 SDM2.0 categories. A lower limit of 5% seems to work well for the 5 AIHA categories (Johns Hopkins has a graduate student doing some simulation to see if another percentage might be better – stay tuned!). I don’t have experience with the 11 SDM2.0 categories. Maybe 2% or 3% since there are roughly twice as many as the AIHA scheme?
-
Display Rule-of-Thumb guidance for using user-defined probabilities on both the AIHA and SDM2.0 entry screens.
Other questions and suggestions regarding the AIHA and SDM2.0 pages for user-entered probabilities:
-
Why doesn’t the “combined” match the “measurements” when the “user” probabilities are uniform (uninformed)? (This is evident on the AIHA page. Because I could not enter uniform probabilities in the SDM2.0 page I don’t know whether it occurs there as well.)
-
Set the default user probabilities in the SDM2.0 frame to be uniform / uninformed (like the AIHA screen) rather than the current informed. This should not be an issue for a programed default that can incorporate fractional percentages (e.g. 100%/11 = 9.0909% in each).
-
The many categories in the SDM2.0 user probability frame can make it difficult to see where the percentiles lie relative to the OEL. Suggest the following: As a minimum, add a line indicating the OEL. Also consider summarizing the total percent above the OEL so that people can easily see the total likelihood that the percentile of interest is above the OEL.
-
Ensure that it is easy for people who want to focus on the five AIHA categories to use the 11-category SDM2.0 output to “feed” an 11-category user probability chart but display the combined probabilities output as the 5-category AIHA chart. Note that I am not familiar with the SDM2.0 output. If there is an option in SDM2.0 to deliver the tool’s output in terms of the five AIHA categories than this is not needed.
Expert Mode:
As noted in the software, these features really are for “Advanced users ONLY” in recognition of the extreme error that can be introduced by their misuse, here are a couple of suggestions:
-
Add some kind of label or other indication to the three calculation output screens that clearly indicate that the calculations were performed while in “Expert Mode”.
-
Consider a separate “expert version” of the tool that is limited in distribution. In the future we may have many people ready for these tools, but right now there are very few people with the deep understanding needed and few resources currently available for them to learn how to use them properly.