Application Security Risk Assessment Guidelines

This document is intended to provide guidelines for assessing the overall risk profile of a new project or application.


This document is intended to provide guidelines for assessing the overall risk profile of a new project or application. The guidelines are not rulesthey are intended to provide a framework for understanding the threats posed by and to an application, and as such may sometimes need to be flexible.


Before the guidelines are listed, it is beneficial to define several terms that will be used throughout the discussion.

  • Application. An application can be anything as complex as a multi-tier web application down to an individual script. For the purposes of the analysis, it should be considered as any piece of executable code that can "stand on its own." This would include the aforementioned web application, a desktop application, a shared code library intended for reuse, or a script in some cases.
  • Project. A project is an application (or set of applications) that belongs to one or more stakeholders. In some cases, a project may have a number of subcomponents that are individual applications, but all those pieces are assembled into a larger functioning unit. For the purposes of risk assessment, a project is something that belongs to a particular project sponsor, that has its own inherent risks, and that influences the risks on the infrastructure and other applications.
  • Service. A service is some piece of functionality provided to one or more users or systems. Applications (and, by extension, projects) are responsible for providing services.
  • Stakeholder/Project Sponsor. The person, people, or department(s) who define the requirements for the application. They will typically be responsible for its feature list, funding, etc.
  • Data Custodian(s). Individuals who are responsible for the content, management, and security of a set of data elements. They are also responsible for the consequences of a data breach.
  • Developer. A person responsible for coding an application, script, component, etc. In some cases, this definition may be extended to database administrators and other individuals who write code directly or indirectly used by an application.
  • Restricted Data. Data classified according to Wis. Stats. 895.507^ ^, as well as other personal health information. When breaches of restricted data occur, Wisconsin law specifies that certain actions must be taken in the event of a breach. The data constituting this statute is last name and the first name or first initial of the party, plus any of the following:
  • Social security number
  • Driver's license or state identification number
  • Financial account numbers, including credit or debit cards, security and access codes, or passwords that allow access to financial information
  • DNA profile
  • Unique biometric data including fingerprints, voiceprints, and retina or iris images
  • Personal health information (not covered by the statute, but considered restricted by the University)
  • Sensitive Data. Data that is not classified as restricted, but whose loss would be considered detrimental to the University. Such information might include student demographic data, enrollment information and any other FERPA-protected information.
  • Public Data. Data whose loss is considered acceptable or low-risk.
  • Risk. The likelihood of an agent (a person, natural event, system, etc.) taking advantage of a vulnerability in an application, system, etc. and causing a negative business impact. Risk can be thought of by the general formula:
  • *

Risk Assessment Criteria

When a new project is begun, it would be useful to gauge how much risk the application will be exposed to, and how much of a threat the application itself may pose to the infrastructure. While the form of this assessment may take different shapes, and will likely evolve over time, some baseline criteria may assist in making that determination. The remainder of this document suggests some criteria that can be used.
The criteria are broken into several high-level categories that map onto a typical software project lifecycle. These categories are:

  • Project Kickoff
  • Architecture
  • Implementation
  • Testing
  • Post-release

Project Kickoff Criteria

What are the uptime requirements of the application? Determining business criticality More business-critical applications (as measured by uptime requirements) need to have security controls that will help reduce downtime. Applications that have stringent uptime requirements (for example, must not be down for more than an hour) should have their potential risk increased.
  • Application monitoring tools (high-risk applications)
  • Change management processes
  • NIST 800-64
What is the impact of downtime? Determining business criticality More business-critical applications (as measured by uptime requirements) need to have security controls that will help reduce downtime. If the application's downtime will affect safety or significantly impact one or more financial areas, the application should be considered high-risk.
  • Application monitoring tools (high-risk applications)
  • Change management processes
Where will the application reside in production? Infrastructure impact Location of a given application or service can have a significant impact on its exposure, and may also have an impact on other applications and services. If an application resides in a shared environment with other applications, risk to that application will be increased. Additionally, if a sensitive application must be hosted in an area without proper security controls over the infrastructure, risk is also increased.
  • Application inventory system
  • Change management processes
Where will development be performed (development server, individual developer desktops, etc.)? Infrastructure impact Development activities have a natural tendency to decrease security posture. Any hosts that will be involved in the development process will be exposed to increased risk in most cases.
  • Minimal privilege to development devices
  • Dedicated development servers (high-risk applications)
  • Server isolation such as in a separate DMZ (high-risk applications)
Are there separate test and/or staging environments? Infrastructure impact/rollout management It is highly advantageous to have a three-stage release system (Development-Staging-Production), which can serve as a focal point not only for technical controls, but policies and procedures as well. If a test environment exists, there is less likelihood of test code causing a production issue. When a three-stage release system is used, risk is substantially reduced, especially when strict policies governing release to production are in place.
  • Minimal privilege to all devices
  • Developers not allowed access to promote to production (high-risk applications)
  • Change management processes
What is the migration plan for moving code into production? Infrastructure impact/rollout management Placing code into production is inherently risky, both to the new application as well as any existing services. If a defined process exists for placing code into production, it is less likely that such a release will have an impact on production systems. For very risky deployments, a full change management strategy is in order, as is a review process prior to allowing production rollout.
  • Minimal privilege to all devices
  • Developers not allowed access to promote to production (high-risk applications)
  • Change management processes
What is the back-out strategy should a production release have a negative impact? Infrastructure impact/Rollout MANAGEMENT All production releases should be accompanied by a plan to revert changes should failure occur. This necessitates monitoring not only the application after release, but also the associated infrastructure. If the backup plan can revert systems to the exact state prior to deployment, risk is reduced.
  • Change management processes
  • Server monitoring
What individuals will be involved in the project? List the names and access levels required for any external system access. Infrastructure impact By enumerating specific permissions and duties, it will be easier to provide restrictive access lists and any necessary monitoring. Smaller lists of individuals involved generally means less risk, though not always. Additionally, the higher the access level required, the higher the risk. Access management processes
Is there a project manager for the project? Project management Dedicated project managers may make it easier to ensure compliance with functional and security requirements. This is largely unnecessary for smaller projects. A project manager, particularly one with security awareness, will help to reduce risk by providing risk analysis input as a project progresses, and by communicating specific risks to project stakeholders and data custodians. As such, the project manager should be up to speed on any compliance or regulatory aspects of the project.
  • Project management templates
  • Project manager training for standards and compliance issues
What is the expected lifetime of the application? Rollout management Applications that expect to have longer lifetimes will, in general, need to have better security controls due to the length of time the application will be exposed to a potentially hostile user base. Shorter-life applications (12-18 months) may not need such controls, and it may be cost prohibitive to provide comprehensive security controls. Vulnerability scanning on a regular basis is also essential (quarterly is recommended). Longer-lived applications should have increased risk assigned to them, and as such should have additional controls put in place. Periodic vulnerability scanning
What is the strategy for providing test data? Data security Data sanitization in a test environment is often not considered. Testing environments tend to be less controlled than production environments. Production data must be used carefully in a test environment. Any restricted data that is used in testing increases risk to that data. Sanitization of such data elements helps reduce this risk.
  • Test data management products such as Applimation (Informatica), IBM Optim, or Solix EDMS (high-risk applications)
  • Test data migration processes
How will the systems on which the application runs be maintained? Data security/infrastructure impact Security is an ongoing process, and as such, activities such as patch and configuration management must be taken into account. All applications should have a strategy developed for system maintenance after release. High-risk applications should additionally account for periodic vulnerability testing.
  • Patch management tools such as Windows Software Update Services, or more comprehensive tools like Altiris Server Management Suite
  • Accurate inventory of systems

Architecture Criteria

Identify the types of data that will be handled by the application, and the classification of that data. Data classification determines if the data is restricted, sensitive or public. For example:
  • Student registration information: name, address, year (sensitive)
  • Course offering schedule (public)
  • Credit card number and CVV code (restricted)
Data security Data classification is a primary determining factor for the risk to an application. If the application will handle any data that falls into the restricted category, the application should be considered high-risk. Depending on the quantity of other types of information, purpose of the application and its exposure, other applications may fall into a higher risk category. Data classification strategy and documentation
Identify all external systems, data stores, etc. that the application will access, along with the rights required. For example, "read access to the student demographic information stored on server X in database Y." Infrastructure impact/data security External systems can both pose a threat to an application, as well as also be threatened by an application. If the application interacts with more secure systems, the risk should be assigned accordingly. In particular, if connected systems contain restricted data, the risk posed by the application becomes much higher, and appropriate controls to mitigate the risks to the external systems and their data should be selected.
  • Project documentation
  • Data flow diagrams
  • Access management processes
Does the application require authentication? If yes, what data store will be used to validate credentials? Identity management/access control Proper management of credentials is key to protecting the "front door" of a given application. Credential stores that are reusable and have a known security history (such as LDAP or Active Directory) have less risk than a home-grown credential management solution. If credentials are stored in clear-text, the risk is significantly increased. High-risk applications should use a well-known credential management system.
  • LDAP
  • Active Directory
  • Enterprise access management solutions, such as Oracle Adaptive Access Manager (high-risk applications)
Does the application have varying levels of access (i.e., do certain functions need to differentiate among users based on their privilege levels)? If so, how will these authorization levels be checked and maintained? Identity management/access control If an application has different levels of user interaction, it is more difficult to secure. Consideration must be given as to how authorization information will be expressed and utilized, as well as controls put in place to ensure that roles cannot be subverted. An application with only one level of access poses significantly less risk. If an application requires differing levels of access, centralized authorization checks based on a known credential management system will also reduce risk.
  • LDAP
  • Active Directory
  • Enterprise access management solutions, such as Oracle Adaptive Access Manager (high-risk applications)
How will the application be organized in the revision control system? Rollout management Properly organized applications are easier to maintain going forward, and can improve code reuse. Projects should be broken down into functional components and stored in a hierarchical structure. All components should be stored in the revision control system in order to reduce the risk of loss, as well as having the ability to control the risk inherent to the release process. Revision (version) control system. Subversion is highly recommended due to the large number of supporting tools available, but other solutions like CVS or QVCS are also acceptable.
Will the application use two-factor authentication? Identity management/access control If a user will be accessing restricted data belonging to another person, two-factor authentication should strongly be considered. Stronger authentication should always be used for applications in which it is possible to access another user's restricted data.
  • Enterprise access management solutions, such as Oracle Adaptive Access Manager (high-risk applications)
  • RSA or other token-based authentication scheme

Implementation Checklist

Has each application component been coded according to security best practices? Adherence to standards Standards are built on industry-accepted best practices such as the OWASP Top Ten list,^ ^ and have been shown to greatly increase the overall security posture of an application. Additionally, some standards like PCI are founded on such best practices, and so adherence is required for compliance. Failure to follow secure coding guidelines should be, in general, prohibited. The risk posed by an application that does not adhere to such standards is very high.
  • OWASP Top Ten
  • PCI Data Security Standards (high-risk applications)
  • NIST standards 800-122, 800-107, 800-95, 800-63, 800-53, 800-14
  • Training for developers, whether through in-house knowledge sharing, written materials or remote training opportunities is also essential. Such training should be refreshed periodically in order to keep current.
Have all changes been checked into the revision control system? Project documentation There is a tendency to leave "minor" changes out of a revision control system in the name of speed. Failing to check in all changes increases the risk of code alterations being overwritten. It also makes it more difficult to see when a project was last modified.
  • Project code management standards
  • Revision control system
Have all third-party components been verified as having no significant known security vulnerabilities? Detection of security flaws Third-party components are often considered outside the scope of security review, but can often contribute significantly to the security posture of a project. Third-party components should all be reviewed for security, and any components with a history of vulnerabilities should be considered as posing an increased risk to the application.
  • CVE lists and documentation for the affected components
Are security controls abstracted to a central location? Adherence to standards By ensuring that security controls are centralized within a common function, base class, include file, etc., an application can more easily ensure that controls are applied consistently. Maintenance of security functions is also easier. Using such centralized controls decreases risk, including potential future risks.
  • For object-oriented languages, derive security controls from common classes that are then reused as necessary
  • Procedural languages should store security functions in one or more common files which can be included by reference
How will keys used to encrypt data be secured? Data security High risk applications must not simply store encryption keys in the code. Doing so constitutes an unacceptably high level of risk. High risk applications should be using a key management system or other means of obtaining encryption keys when they are required.
  • Windows protected storage
  • Windows registry
  • Files stored outside the web root (accessible only to the account running the app)
Are any reusable libraries being used? Data security Reusable libraries provide a standardized method of performing common tasks, and are an ideal solution for a number of complex security issues like input filtering. Applications which are higher risk should attempt to leverage known libraries as much as possible. Applications which do not take advantage of such libraries may be at a higher risk of compromise.
  • Third-party input filtering, logging, and other reusable libraries
  • Internally-developed and documented libraries

Testing Checklist

Will a code review for this project be performed? How will it be carried out? Adherence to standards / detection of security flaws Code reviews can detect security issues before they are placed into a running environment. Such reviews can also ensure that coding standards are properly followed. Failure to do any sort of code review increases the risk that the application will have one or more security flaws when it is released.
  • Automated code review tools such as AppScan (high-risk applications)
  • Peer code reviews for larger projects (high-risk applications)
  • Empirix (high-risk applications)
  • Independent security reviews by a third party (high-risk applications)
Will the application be subjected to penetration/load testing? Detection of security flaws Proper penetration and load testing can identify security and uptime weaknesses before they can be exploited. All publically-accessible websites that have been deemed moderate to high risk should have a penetration test performed. Failing to account for such tests should be considered an unacceptably high risk for such applications.
  • Penetration testing frameworks such as w3af, or commercial tools like Core Impact, that can perform application-level penetration testing (high- risk applications)
  • Network penetration testing tools like Nessus are also useful for verifying server configurations
  • Load-testing tools like Empirix may also be used when the application has high uptime requirements (high-risk applications)
Is there a test plan in place? Adherence to standards/ detection of security flaws Having a test plan helps to ensure compliance with functional and security requirements, and provides a roadmap to testers. Test plans should test both normal use cases as well as exception paths. Having a test plan reduces the risk that testing will miss a flaw. A robust test plan is also useful for future regression testing.
  • Test plan templates and testing scripts if appropriate
  • NIST 800-115
  • Empirix (high-risk applications)
Who will perform testing and validation? Adherence to standards/detection of security flaws Testing is one of the best ways to ensure that security flaws are caught prior to release. Having an external tester (a person not involved in development in any way) reduces risk by making it more likely that such a person will locate conditions and provide input that may give rise to a software flaw.
  • Fuzz testers (such as Peach) can be of limited help by providing a wide variety of input to various subsystems (high-risk applications)
  • External testing by a qualified tester may also be helpful (high-risk applications)
  • NIST 800-115
How often will testing be performed? Detection of security flaws Security post-rollout must be considered, and routine penetration (and, optionally, code review) are essential. High-risk applications should be tested at least quarterly with an automated scanner. Lower-risk applications can be tested yearly.
  • Nessus (high-risk applications)
  • Empirix (high-risk applications)
  • AppScan (high-risk applications)

Post-release Checklist

Was any documentation generated for the project? If so, where is it stored and who has access to it? Project documentation Larger projects should be documented in order to identify components and how they related to one another. User manuals and the like may also be helpful. Maintenance time will be reduced, and it will be easier to identify any security impact to the application after it is rolled out.
  • Central project document stores, i.e. wikis, are useful for this sort of task. SharePoint and other similar portal options also work.
  • Data can also be stored in the revision control system.
Are any of the code modules reusable by other developers? If so, please provide documentation and the location in the revision control system. Project documentation Code modules may not be reused to their full extent without documentation, or may possibly be misused, resulting in a potential security impact. Well-documented, reusable modules significantly reduce risk to the applications that use them. Caution must be exercised to update any dependent applications when security flaws are uncovered in these modules.
  • Central project document stores, i.e. wikis, are useful for this sort of task. SharePoint and other similar portal options also work.
  • Data can also be stored in the revision control system.
Are there any post-rollout meetings scheduled? Rollout management For a project of any size, it makes sense to touch base after the project has been released. This allows project sponsors and development staff to ensure all issues have been resolved and identify any future plans. Having such a meeting will reduce post-rollout risks by ensuring that security requirements have been met, and that any shortcomings will be addressed.
  • Project lifecycle processes
How are problems with the application reported? Rollout management One or more individuals should be identified to handle any issues with the application after rollout. Risk is increased if no such notification structure exists, as problems may continue to occur without any remediation.
  • Project lifecycle processes
  • Application problem reporting websites, etc.
What is the plan should a security incident or breach occur? Rollout management As with any general issue, one or more individuals should be tasked with responding to security problems. It is important to note that these can be reported directly, or may come about as a result of the discovery of something like a new attack vector. Any project should have a plan for addressing security issues, since not having a plan for these types of issues constitutes a substantial risk.
  • Project lifecycle processes
  • Notification system for application-related security issues on campus

Keywords:appsec cdw guidelines risk   Doc ID:20262
Owner:Matt A.Group:Office of Campus Information Security
Created:2011-09-14 14:36 CDTUpdated:2014-05-23 14:48 CDT
Sites:DoIT Help Desk, Office of Campus Information Security
Feedback:  35   5