Software Metrics
1.
2.
a0a2a1a4a3a4a5a7a6a4a8a4a9a11a10a13a12a4a5a11a14a11a3a4a12a16a15a13a9a4a17a18a3a20a19a21a1a4a12a23a22a24a6a4a8a4a9a11a12a4a17a18a3a11a15a24a25a26a3a4a12a16a27a13a28a29a5a31a30a32a12a16a33a4a8a4a9a26a22a35a34a4a12a4a5a4a36a11a10a13a12a16a5
a3a31a37a13a25a26a17a18a3a4a15a13a15a7a28a38a22a31a28a29a5a11a5a4a9a4a14a11a33a16a3a4a17a18a15a13a34a13a6a4a8a4a9a11a27a13a5a4a8a23a19a39a15a13a8a4a14a11a3a23a22a29a1a4a28a35a5a13a30a20a12a4a33a4a8a16a9a26a22a13a28a38a22a35a40a4a33a4a9a23a22a4a19a41a1a4a3a4a5
a6a16a8a4a9a11a10a13a12a4a5a4a5a4a8a23a22a13a14a11a3a4a12a4a15a13a9a4a17a18a3a11a28a42a22a29a34a26a19a41a1a4a3a16a5a43a6a16a8a4a9a11a10a13a12a4a5a4a5a4a8a26a22a31a3a13a37a13a25a26a17a18a3a4a15a13a15a7a28a42a22a13a28a29a5a11a5a16a9a4a14a11a33a4a3a4a17a18a15a13a34
a6a16a8a4a9a4a17a26a27a31a5a4a8a26a19a41a44a29a3a4a36a31a30a26a3a11a28a29a15a7a8a4a45a4a12a11a14a11a3a4a12a31a30a26a17a18a3a46a12a16a5a4a36a11a9a4a5a4a15a13a12a26a22a35a28a29a15a31a45a38a12a4a10a16a22a29a8a4a17a42a6a11a27a13a28a35a5a4a36a4a47a11a48a49a22a31a14a11a12a13a6
a33a16a3a32a22a35a1a4a3a11a33a4a3a13a30a23a28a29a5a4a5a16a28a29a5a13a30a20a8a4a45a16a27a13a5a4a8a26a19a41a44a29a3a16a36a13a30a26a3a4a34a4a33a16a9a26a22a24a6a4a8a4a9a11a1a4a12a23a50a24a3a11a15a13a10a13a12a4a17a18a10a13a3a4a44 a6a11a28a29a5a7a6a4a8a4a9a4a17
a22a35a1a4a8a4a9a31a30a26a1a26a22a35a15a43a12a16a36a26a50a24a12a4a5a4a10a13a3a16a36a20a22a29a8a20a22a35a1a4a3a11a15a4a22a29a12a13a30a23a3a46a8a16a45a4a15a13a10a13a28a35a3a4a5a4a10a13a3a16a51
Lord Kelvin, a physicist
a48a42a5a20a22a29a17a18a9a26a22a35a1a4a34a4a12a7a30a26a8a16a8a4a36a11a10a13a12a4a15a13a3a11a10a13a8a4a9a16a44a29a36a11a33a4a3a11a14a11a12a4a36a4a3a20a22a35a1a4a12a26a22a13a28a35a45a13a6a4a8a4a9a16a17a26a27a13a5a4a8a23a19a21a44a35a3a4a36a13a30a26a3a11a28a35a15
a14a11a3a16a12a13a30a26a17a18a3a11a12a4a5a4a36a11a9a4a5a4a15a13a12a23a22a29a28a35a15a13a45a38a12a16a10a4a22a29a8a16a17a38a6a16a34a26a22a29a1a16a3a46a44a35a12a4a15a4a22a16a22a29a1a4a28a35a5a13a30a20a28a29a5a20a22a35a1a4a3a20a19a41a8a4a17a18a44a29a36a7a6a4a8a4a9a11a15a13a1a4a8a16a9a4a44a29a36
a36a16a8a46a28a35a15a7a14a11a12a4a27a13a3a11a14a11a3a4a12a4a15a13a9a16a17a49a3a16a14a11a3a4a5a26a22a35a15a13a40a20a22a29a1a4a3a11a10a13a1a16a12a4a5a4a10a13a3a11a28a29a15a7a5a4a3a31a30a26a44a29a28 a30a26a28a35a33a4a44a29a3a20a22a35a1a4a12a26a22a24a6a16a8a4a9a20a19a21a28a35a44a29a44
a14a11a3a16a12a4a15a13a9a4a17a18a3a20a22a29a1a16a3a46a17a18a28 a30a26a1a26a22a4a22a35a1a4a28a35a5a13a30a26a15a7a12a4a10a13a10a13a28a35a36a4a3a4a5a23a22a29a12a4a44a35a44a52a6a16a51
George Miller, a psychologist
a53
Software Metrics
Product vs. process
Most metrics are indirect:
No way to measure property directly or
Final product does not yet exist
For predicting, need a model of relationship of predicted variable
with other measurable variables.
Three assumptions (Kitchenham)
1. We can accurately measure some property of software or process.
2. A relationship exists between what we can measure and
what we want to know.
3. This relationship is understood, has been validated, and can be
expressed in terms of a formula or model.
Few metrics have been demonstrated to be predictable or related
to product or process attributes.
a54
Software Metrics (2)
Code
Static
Dynamic
Programmer productivity
Design
Testing
Maintainability
Management
Cost
Duration, time
Staffing
.
a55
Code Metrics
Estimate number of bugs left in code.
From static analysis of code
From dynamic execution
Estimate future failure times: operational reliability
a56
Static Analysis of Code
Halstead’s Software Physics or Software Science
n1 = no. of distinct operators in program
n2 = no. of distinct operands in program
N1 = total number of operator occurrences
N2 = total number of operand occurrences
Program Length: N = N1 + N2
Program volume: V = N log
2
(n1 + n2)
(represents the volume of information (in bits) necessary
to specify a program.)
Specification abstraction level: L = (2 * n2) / (n1 * N2)
Program Effort: E = (n1 + N2 * (N1 + N2) * log
2
(n1 + n2)) / (2 * n2)
(interpreted as number of mental discrimination required
to implement the program.)
a57
McCabe’s Cyclomatic Complexity
Hypothesis: Difficulty of understanding a program is
largely determined by complexity of control flow graph.
Cyclomatic number V of a connected graph G is the
number of linearly independent paths in the graph or
number of regions in a planar graph.
R1 R2
R4
R5
R3
Claimed to be a measure of testing diffiiculty and
reliability of modules.
McCabe recommends maximum V(G) of 10.
a58
Static Analysis of Code (Problems)
Doesn’t change as program changes.
High correlation with program size.
No real intuitive reason for many of metrics.
Ignores many factors: e.g., computing environment,
application area, particular algorithms implemented,
characteristics of users, ability of programmers,.
Very easy to get around. Programmers may introduce
more obscure complexity in order to minimize
properties measured by particular complexity metric.
a59
Static Analysis of Code (Problems con’t)
Size is best predictor of inherent faults remaining
at start of program test.
One study has shown that besides size, 3 significant
additional factors:
1. Specification change activity, measured in pages of
specification changes per k lines of code.
2. Average programmer skill, measured in years.
3. Thoroughness of design documentation, measured
in pages of developed (new plus modified) design
documents per k lines of code.
a60
Bug Counting using Dynamic Measurement
Estimate number remaining from number found.
Failure count models
Error seeding models
Assumptions:
Seeded faults equivalent to inherent faults in
difficulty of detection.
A direct relationship between characteristics and
number of exposed and undiscovered faults.
Unreliability of system will be directly proportional
to number of faults that remain.
A constant rate of fault detection.
a61
Bug Counting using Dynamic Measurement (2)
What does an estimate of remaining errors mean?
Interested in performance of program, not in how
many bugs it contains.
Most requirements written in terms of operational
reliability, not number of bugs.
Alternative is to estimate failure rates or future
interfailure times.
a53a63a62
Estimating Failure Rates
Input-Domain Models: Estimate program reliability
using test cases sampled from input domain.
Partition input domain into equivalence classes,
each of which usually associated with a program path.
Estimate conditional probability that program correct
for all possible inputs given it is correct for a specified
set of inputs.
Assumes outcome of test case given information about
behavior for other points close to test point.
Reliability Growth Models: Try to determine future
time between failures.
a53a64a53
Reliability Growth Models
Software Reliability: The probability that a program
will perform its specified function for a stated time
under specified conditions.
Execute program until "failure" occurs, the underlying
error found and removed (in zero time), and resume
execution.
Use a probability distribution function for the interfailure
time (assumed to be a random variable) to predict future
times to failure.
Examining the nature of the sequence of elapsed times
from one failure to the next.
Assumes occurrence of software failures is a stochastic
process.
a53a65a54
Software Uncertainty
Assumption: The mechanism that selects successive
inputs during execution is unpredictable (random).
F
I
Input space I
Output space O
O
Program p
F
O
F
is the image set of I
F
under the mapping p
a53a63a55
Sample Interfailure Times Data
3 30 113
77
242
0
452
233
330
810
1755
2930
109
447
5509
5485
81 115
24
9
88
180
65
193
193
2
670
91
120
112
26
600
457
816
369
529
828
33
865
875
1082
6150
15
114
15
300
1351
748
379
1011
868
1435
245
22
3321
138 50
55
108
422
227
325
36
97
148
0
44
445
724
30
729
75
1045
68
8
255
10
176
6
236
10
281
983
261
943
990
371
1146
58
79
31
16
160
707
1800
700
948
790
4
263
21
232
129
296
2323
143
1897
482
648
197
134
365
290
1064
1461
0
386
100
1160
357
1222
300
1783
843
3110
446
10
1864
543
529
860
12
1247
122
1071
4116
Execution time in seconds between successive failures.
(Read left to right in rows).
a53 a56
Using the Models
LV
JM
2400
2200
2000
1800
1600
1400
1200
1000
800
600
400
200
40 50 60 70 80 90 100 110 120 130
Different models can give varying results for the same
data; there is no way to know a priori which model
will provide the best results in a given situation.
‘‘The nature of the software engineering process is too
poorly understood to provide a basis for selecting a
particular model."
a53a65a57
Problems with Software Reliability Modeling
There is no physical reality on which to base our assumptions.
Assumptions are not always valid for all, or any, programs:
Software fault (and failures they cause) are independent.
Inputs for software selected randonly from an input space.
Test space is representative of the operational input space.
Each software failure is observed.
Faults are corrected without introducing new ones.
Each fault contributes equally to the failure rate.
Data collection requirements may be impractical.
a53a63a58
