Friday 7 December 2007

100% functional coverage is not enough

Let me address this age old question once again. As we all know that 100% functional coverage does not mean bug free design, but it surely adds to the confidence of the designer. It is one of the criteria for verification completion. So, what are the other criteria people use for their verification completion. Most of us will answer that 100% code coverage achievement and few of us will also add that closing all the pending review items derived from verification review. But is it enough? We still feel that it is not enough. Deep inside our hearth, we still feel the need of some more testing/verification.

Well, this feeling will be there because of the nature of the verification task. But, let me come to the technical aspect of the 100% functional coverage. If we ask our self, what is the functional coverage, we will be scratching our head to find exact wordings to explain the functionality. Let me try to explain it in my words. For me, Functional Coverage Points are nothing but the measure of all possible combinations of inputs which can be applied to the design. So, in our functional coverage grid, we always cover all the input stimulus parameters and we assume that the output behaviour will be checked somewhere else in the verification environment. So, 100% functional coverage means we have stimulated the design with all possible stimulus. But this does not guarantee that all outputs and behaviors of the design are checked. So, 100% functional coverage does not mean bug free design. But, if we also have one more coverage grid which could be checker coverage grid, then, we can definitely say that we have applied all possible combinations of inputs and we have also checked all possible combinations of outputs. Even we need co-relation between these two grids. I mean, for each functional coverage point (one possible combination input), there should be at lease one checker coverage point (one possible output checker). And for each checker coverage point, there should be at least one functional coverage point. Thus co-relating these two grids will definitely help to improve the confidence in the design.

So, in my opinion, verification completion declaration should be based on following criteria.

1) All test cases passed.
2) 100% Code Coverage (of course with some waivers)
3) 100% Functional Coverage (and again, with some waivers)
4) 100% Checker Coverage
5) Correlation between Functional Coverage Grid and Checker Coverage Grid.

4 comments:

Suresh said...

Sandeep, I was asked this question from various people. But I didn't give the relative answer. I told'em that 100% statement, condition, branch and toggle coverage is enough. But to make sure 100% functional coverage, you have to read the spec. line by line and write test plan which is not that much easy.

Anonymous said...

Independent coverage points (both code and functional) do not measure complete paths and give a false sense of completeness. Verification must exercising paths from inputs to checkers. Functional qualification measures the ability of a verification environment to find design bugs by measuring the ability to exercise, propagate and check.

Anonymous said...

Cool article you got here. I'd like to read a bit more about this theme. Thanks for sharing that information.
Sexy Lady
A level escort

mahesh said...

hi,
this is mahesh i am having some confusion about coverage, where we have to write this coverage, i have extend the base struct or i have to write in any unit like monitor,please help me.