The software development industry has widely adopted the mantra of “shifting left” with security or in other words, moving security activities earlier in the development lifecycle. While developers, architects, and other team members are generally supportive of this theory, they sometimes lack the skills needed to effectively implement it. Training can fill much of the gap and is usually necessary, but organizations find that both live and computer-based training produces a spike in security activities, but that the activity quickly drops off as participants get pulled back to their old routines by the legitimate demands of the business..
Meristem believes that one powerful tool to shift the security culture within a development organization is to provide a regular cadence of security activities that stimulate developer’s interests. Naturally, these activities must be small and constructed so that they can work into small gaps in a developer’s day to avoid significantly disrupting their regular schedule.
The Program
The Vulnerability-a-Month program is a series of quick lessons coupled with hands-on activities designed to be completed in 30 minutes or less. The topics center around the OWASP Top 10 application security risks with current high visibility vulnerabilities mixed in. For example, one lesson explains the mechanics of Cross-site Scripting (JavaScript injection) and then gives students access to a website where they can execute . Another module covers the Log4Shell vulnerability, and again provides a vulnerable application so that students can see the exploit in action.
Target Audience
For convenience, this document frequently uses the term “developers” when referring to students. While developers are the most common audience, other participants in the software development lifecycle often have the technical skills required to complete the modules and receive the full benefit of the program. No programming skill is required to understand the modules, but a strong familiarity with web technologies will probably be needed. If the participant could look at the text of a raw POST request and understand what is being submitted or look at an HTML response and find an error message among the tags, they likely have enough of a background to benefit from the program.
Changing the Psychology
The activities associated with each module deliberately puts students in the role of the attacker rather than the defender. When developing an application is very easy to focus on meeting business requirements and never stepped back think about what could be done by a user instead of just what users should do. Putting on the attacker’s hat, even briefly, shifts their mental model from crafting a functional application to considering all of those parts as potential tools to gain access to private data or features. Also, exploiting a vulnerability themselves makes believers out of developers when it comes to discussing the risks associated with security requirements. Even if the developer does not have the skill to exploit all vulnerabilities, when I developer knows they could exploit some vulnerabilities, it is a shorter leap of faith to believe that someone has that extra bit of skill required.
Sample Curriculum
the following is a set of topics that might be put together into a year-long program. Meristem can customize these topics for your company’s needs if there are particular vulnerabilities that pose a greater risk to your application based on its functionality or technology stack. In some cases, a particular type of vulnerability has been reported frequently during security scans or assessments and a customized modular to can be part of a plan to address this vulnerability across an organization.
- Cross-site Scripting - Session Tokens
- Cross-site Scripting - Internal Function Calls
- JWT Manipulation
- SQL Injection
- Hidden API Endpoints
- Hidden JavaScript Endpoints
- Bypassing Client-side Input Validation
- File Downloads - Directory Traversal
- File Downloads - CSV Injection
- File Uploads - MIME-type Validation
- Famous Vulnerability - Shell Shock
- Famous Vulnerability - Log4Shell
Logistics
For each module, Meristem will provide web-based learning materials and host a vulnerable application where each student can perform the exploit in a legally-allowed environment, without interfering with other students. Typically, you will provide a list of student email addresses and Meristem will send an email to each student that contains links to the learning materials and practice environments. If desired, you may handle the distribution of these links yourself, recognizing that the link for each student may be slightly different depending on the modules.
Meristem recommends that the completion of each module be tied to small reward. Something as trivial as a $10 gift card goes a long way towards encouraging participation and putting students in a positive mindset with respect to the effort required. Meristem can handle the distribution of these rewards which may avoid the need to treat the reward as benefits for tax purposes. You are free to manage the distribution of rewards yourself as well. In either case, Meristem will provide a list of students who have completed the activity, and their time of completion, so that you can monitor participation.
No security assessment can hope to find every vulnerability. The time scoped by Meristem for each assessment is intended to find the majority of vulnerabilities, including those likely to be found by a skilled attacker. Still, the more efficient the testers can be, the more attack surface they can cover, and ultimately, the more secure your application will be. Meristem is always looking for ways to optimize our process, but there are steps that you can take to ensure that the majority of your assessment time is spent testing which will maximize your return on investment.
Choosing The Environment
The environment that the testing will be done in has a huge impact on the coverage of the test. Ideally, a clone of a pre-production environment can be made, with the desired code version deployed and populated with a complete set of simulated data. Alternatively, a pre-production environment could be dedicated to pentesting for the duration of the engagement, with a database backup taken just before testing begins so that the environment can be reset after the test. If there are no other options, testing can be performed in a production environment, but no matter how careful the tester is, there is a risk that site performance could be degraded, or that side-effects from testing could be encountered by a legitimate user. Also, if the testers do find gaps in access control restrictions, they could gain access to sensitive information of real users which may be considered a data breach in some contexts.
An Ideal Test Environment Is:
- Dedicated - Many tests involve storing attack payloads that may end up being triggered in other parts of the application. Having other users working in the same environment puts them at risk of falling victim to one of these attack payloads.
- Disposable - Many automated tools submit a broad array of attack payloads and then test to see if any of them were successfully triggered. This often results in the creation of many business objects that may or may not be removable. The best environment is one where the database can be reset at the conclusion of the test.
- Realistic - An environment that is populated with realistic values for all application workflows helps the tester understand how the application is actually used, and lets them focus on testing rather than figuring out what realistic data values might be. Some features are triggered only by specific data values which might not be discovered if the tester is populating the application themselves.
- Configured like Production - The findings included in a report must describe the environment that is actually tested. Even if a mitigating control exists in production, when a vulnerability is found in the test environment, the vulnerability must be included in the report. In some cases, it may be appropriate to test the mitigating control in production, and update the report accordingly, but this would need to be coordinated ahead of time.
- Static - If the environment changes while testing is being performed, such as by code being deployed or a firewall being reconfigured, the testers are usually forced to go back and repeat tests, taking time away from other areas of the application. Of course, if a critical vulnerability is discovered, it may make sense to deploy and test a fix immediately, since eliminating that risk would be more important than identifying additional, typically lower severity findings.
- Stable - An application that isn’t functioning correctly can’t be adequately tested. Freezing changes to the environment is one way to accomplish this. Also, Meristem will report any odd behavior, and appreciates being notified promptly of any outages that occur that impact the environment being tested.
Providing Test Users
In general, Meristem asks for two user accounts for each role that will be tested in order to properly assess both horizontal and vertical access controls. Horizontal access controls protect one user’s data from being accessed by another. If the application supports multiple tenants, it is important that the two users be configured in different tenants, since cross-tenant access is typically a worst case scenario. If business rules dictate that data should be protected both from other users in the same tenant, and across tenants, three users from the same role may be required.
Vertical access controls prevent a user in one role from using features restricted to another role. The “higher” privileged role will be used to understand how the features work, and then the tester will try to use that feature as the other lower privileged roles. Even highly privileged roles that are not given to external-users should be included in the assessment, since these would carry the highest risk if compromised.
Applications that support dynamic roles, where custom roles can be created with specific permissions, are more complicated to test. Here, it often makes sense to create a role with all allowed permissions and another role with a minimal amount of permissions. Again, this allows the tester to understand the feature using the high-access role and test the limiting controls using the low-access user. Of course, if there is a “standard” set of roles, it may also make sense to simply use these.
Access to Source Code
A source code assisted application security assessment is different from a full source code review. In an application security assessment the tester focuses on the user interface and network requests observed during use of the application. A source code review examines the code-level implementation of functions and security controls and uses automated tooling to scan the entirety of the code base if possible. While a source code review is likely to identify more vulnerabilities than application security assessment, it requires more time and effort to complete.
Having access to source code during an application security assessment is still very beneficial even if a full source code review is not being performed. Access to source code allows the tester to quickly look at the implementation of a specific control rather than deducing the implementation based on the results of multiple tests. For example, the tester can see exactly what rules are being applied during input validation, predict and test specific payloads that would bypass these rules, and provide recommendations on improving the specific implementation. Also if the tester identifies an injection vulnerability in one field they can examine the implementation of that feature and use code searching features to see if the same vulnerable construction is used elsewhere in the application. Access to source code may also enable the tester to identify specific line numbers where fixes should be made, reducing mitigation time.
Providing access to source code is not required and goes beyond most common definitions of an application security assessment, but will significantly increase the value received from the testing engagement without increasing the cost.
Conclusion
During an application security assessment the testers attempt to identify the vulnerabilities an opportunistic attacker or malicious user might exploit, with the exception that Meristem will not perform attacks that are likely to cause an outage or degrade the performance of the environment without specific permission. Attackers start with the advantage of unlimited time to gather information about the site and attempt attacks. Since a finite time is allotted to an application assessment, limiting testers to the identical conditions an attacker would work under is counter productive. Providing additional information in the form of access and even source code, tilts the scale towards the testers. An application pentest is not a contest to see how good the testers are, the goal is to find as many of the vulnerabilities that already exist as possible, so they can be eliminated, ensuring no attacker can find them...ever.
Mark Hoopes (of Meristem InfoSec) and Heidi Hoopes presented a workshop at RailsConf: 2022 titled "Gaps in the Magic - Exploiting Security Edge Cases in Rails." This class introduced the fundamental concepts of two key vulnerability classes from the OWASP Top 10, SQL injection and Unsafe Deserialization (or unmarshalling in Ruby parlance). It included practice environments where the students could exploit real-world applications (or a replica) to solidify their understanding and enable them to more effectively communicate the risks associated with these vulnerabilities.
SQL Injection
The workshop began with a breakdown of injection vulnerabilities in general and then looked at specific Active Record methods and what constructions would leave them vulnerable to attack. Students were then given a workbook application (building on the work of Justin Collins, published at https://rails-sqli.org/) where they could explore vulnerable Active Record methods and construct their own SQL injection payloads to accomplish typical hacker goals of bypassing login checks and retrieving sensitive information from the database.
This portion concluded by pointing students at an installation of a vulnerable application found on GitHub (link to be posted once a remediating pull request is accepted) and allowing them to apply their skills to recover the password hash of a simulated user.
Unmarshalling
The workshop then shifted gears to the risks of processing serialized objects that could have been created or modified by a user. Again, the discussion began with the basics of serialization and the specifics of the Marshal protocol in Ruby. Note that the talk "Caching without Marshal" by Chris Salzberg also at RailsConf 2022 gave an excellent overview of the protocol and a possible alternative and is well worth watching when available. Once the concepts behind marshalling had been presented, the process of unmarshalling was examined, focusing on the fact that the once the application has called the marshal.load method, no further defensive checks by the application are possible until a potential payload has been triggered.
Next the concept of a gadget was presented, where a gadget is a set of classes that can be arranged in such a way that when they are unmarshalled, executes code that has been embedded as an attribute of one of the classes. One gadget was discussed in detail as an example of how this is accomplished, and references to other well known gadgets were shared.
Finally, a walk-through of a vulnerability identified during a commercial engagement was performed using a replica of the real site's functionality. This included identifying the serialized object, preparing a payload, and sending the marshalled object to execute code on the demonstration server.
Resources
The SQLi Workbook application, a replica of the vulnerable commercial application, and the slides from the presentation are posted in the Resources Section.