So, you’ve got a design that you want to prove is better than the existing design from your own or another company. What’s the most efficient, fastest way to get to that answer with a very small sample size? Whle there may be several options you have, including accelerated testing, they each can have their benefits and drawbacks. Here, I will advocate for zero-failure testing as a realistic and useful option, especially since it is something we often do anyways, but without the mathematical justification.
Zero-failure reliability testing is also often called substantiation testing. It is a test conducted to substantiate that a given design is better than a requirement or a previous design. If you know the likely Weibull shape factor parameter (beta), then you can easily calculate the size and length of a test to substantiate the design in question for a given confidence level. Most often, either the number of test units available or the test time available is restricted for you.
First, before going any further, determine your required confidence interval. This is usually expressed as a percentage. For engineers, 90% or 95% are generally utilized. The number may be higher or lower depending on the criticality of your design’s function. You don’t want to be in a position after the fact of selecting a low enough confidence interval to make your test count as a success.
Second, you need to determine your design’s beta, or Weibull shape factor. If you are comparing against a similar previous design, you can determine beta from the failure records of that design. If you are evaluating a new design, you can usually gain some kind of insight from reliability handbooks and other published information on similar systems or systems that contain similar piece part components. A beta of less than 1.0 reflects an infant mortality failure mode, while 1.0 represents a random failure mode, and higher than 1.0 represents a wearout failure mode.
Third, you need to have your required characteristic life, eta. This can either be derived from a required characteristic life, or from a MTBF (Mean Time Between Failures) value, or it can be calculated based on the failure data of a previous design.
Fourth, if either your number of test units or your available test time is limited, you need to understand those limits.
Now, that you have those data points, you can calculate ‘k’, the characteristic life multiplier for your particular test from the following equation. Beta is the Weibull shape factor for your design to be tested, N is the number of test units you plan to use, and Confidence is your desired confidence expressed as a number between 0 and 1.
Then, your required test time to complete without any failures and substantiate that this design is better than the last one or better than required is simply the required characteristic life multiplied by the value ‘k’ as determined above.
Example: Consider that we have 3 units of a mechanical transmission that we want to test to demonstrate a better than 1000 hour characteristic life with 90% confidence. From a previous similar design we expect the value of beta (the Weibull shape factor) to be 2.2. Using the equation above, we calculate k to be 0.8867. And, then our required test time is 887 hours. Therefore, to demonstrate compliance with our requirement with 90% confidence, 3 units must each complete 887 hours of testing without failure.