ABSTRACT: System security metrics have evolved side by side with the advent of cyber security tools and techniques. They have been derived from the techniques rather than specified as system requirements. This dissertation surveys the evolution and state of the practice of system security metrics from both a technical and historical perspective. The survey leads to the conclusion that currently accepted methodology for measuring system security has no empirical basis. This research provides new criterion with which to evaluate security metrics, and proposes a new methodology for security theory attribute construction (“STAC”). The STAC framework has been applied to case studies in Cloud Computing and Mobile Communications. Specific research in a variety of system security topics is recommended to reinforce these results, and provide theoretical foundation for more effective tools and techniques for systems security engineering.

Full Thesis including appendices (239-page pdf)
Slides on Thesis used in Defense Presentation (25-page pdf)
Thesis without Appendices (124-page pdf)

This research included a Security Subject Matter Expert Survey on Security Metrics. My sincere appreciation to all who participated in the survey. It is my pleasure to share the results.

For those who are only interested in the top 10 security metrics as identified by survey participants, here are the top 10 things to measure:

1. User identification and authentication
2. Withstand targeted penetration attacks by skilled attack teams
3. Incident detection and response
4. System interfaces accept only valid input
5. Articulate, maintain, and monitor system mission
6. Security awareness
7. Evaluate the extent to which systems are protected from known threats
8. Physical and environmental protection
9. Personnel screening and supervision
10. System recovery planning