No security assessment can hope to find every vulnerability.  The time scoped by Meristem for each assessment is intended to find the majority of vulnerabilities, including those likely to be found by a skilled attacker.  Still, the more efficient the testers can be, the more attack surface they can cover, and ultimately, the more secure your application will be.  Meristem is always looking for ways to optimize our process, but there are steps that you can take to ensure that the majority of your assessment time is spent testing which will maximize your return on investment.

 

Choosing The Environment

The environment that the testing will be done in has a huge impact on the coverage of the test.  Ideally, a clone of a pre-production environment can be made, with the desired code version deployed and populated with a complete set of simulated data.  Alternatively, a pre-production environment could be dedicated to pentesting for the duration of the engagement, with a database backup taken just before testing begins so that the environment can be reset after the test.  If there are no other options, testing can be performed in a production environment, but no matter how careful the tester is, there is a risk that site performance could be degraded, or that side-effects from testing could be encountered by a legitimate user. Also, if the testers do find gaps in access control restrictions, they could gain access to sensitive information of real users which may be considered a data breach in some contexts.

An Ideal Test Environment Is:

• Dedicated - Many tests involve storing attack payloads that may end up being triggered in other parts of the application. Having other users working in the same environment puts them at risk of falling victim to one of these attack payloads.
• Disposable - Many automated tools submit a broad array of attack payloads and then test to see if any of them were successfully triggered. This often results in the creation of many business objects that may or may not be removable. The best environment is one where the database can be reset at the conclusion of the test.
• Realistic - An environment that is populated with realistic values for all application workflows helps the tester understand how the application is actually used, and lets them focus on testing rather than figuring out what realistic data values might be.  Some features are triggered only by specific data values which might not be discovered if the tester is populating the application themselves.
  
• Configured like Production - The findings included in a report must describe the environment that is actually tested. Even if a mitigating control exists in production, when a vulnerability is found in the test environment, the vulnerability must be included in the report. In some cases, it may be appropriate to test the mitigating control in production, and update the report accordingly, but this would need to be coordinated ahead of time.
• Static - If the environment changes while testing is being performed, such as by code being deployed or a firewall being reconfigured, the testers are usually forced to go back and repeat tests, taking time away from other areas of the application. Of course, if a critical vulnerability is discovered, it may make sense to deploy and test a fix immediately, since eliminating that risk would be more important than identifying additional, typically lower severity findings.
  
• Stable - An application that isn’t functioning correctly can’t be adequately tested. Freezing changes to the environment is one way to accomplish this. Also, Meristem will report any odd behavior, and appreciates being notified promptly of any outages that occur that impact the environment being tested.
  

 

Providing Test Users

In general, Meristem asks for two user accounts for each role that will be tested in order to properly assess both horizontal and vertical access controls. Horizontal access controls protect one user’s data from being accessed by another. If the application supports multiple tenants, it is important that the two users be configured in different tenants, since cross-tenant access is typically a worst case scenario.  If business rules dictate that data should be protected both from other users in the same tenant, and across tenants, three users from the same role may be required.

Vertical access controls prevent a user in one role from using features restricted to another role.  The “higher” privileged role will be used to understand how the features work, and then the tester will try to use that feature as the other lower privileged roles.  Even highly privileged roles that are not given to external-users should be included in the assessment, since these would carry the highest risk if compromised.

Applications that support dynamic roles, where custom roles can be created with specific permissions, are more complicated to test.  Here, it often makes sense to create a role with all allowed permissions and another role with a minimal amount of permissions.  Again, this allows the tester to understand the feature using the high-access role and test the limiting controls using the low-access user.  Of course, if there is a “standard” set of roles, it may also make sense to simply use these.

 

Access to Source Code

A source code assisted application security assessment is different from a full source code review. In an application security assessment the tester focuses on the user interface and network requests observed during use of the application. A source code review examines the code-level implementation of functions and security controls and uses automated tooling to scan the entirety of the code base if possible. While a source code review is likely to identify more vulnerabilities than application security assessment, it requires more time and effort to complete.

Having access to source code during an application security assessment is still very beneficial even if a full source code review is not being performed. Access to source code allows the tester to quickly look at the implementation of a specific control rather than deducing the implementation based on the results of multiple tests. For example, the tester can see exactly what rules are being applied during input validation, predict and test specific payloads that would bypass these rules, and provide recommendations on improving the specific implementation. Also if the tester identifies an injection vulnerability in one field they can examine the implementation of that feature and use code searching features to see if the same vulnerable construction is used elsewhere in the application. Access to source code may also enable the tester to identify specific line numbers where fixes should be made, reducing mitigation time.

Providing access to source code is not required and goes beyond most common definitions of an application security assessment, but will significantly increase the value received from the testing engagement without increasing the cost.

 

Conclusion

During an application security assessment the testers attempt to identify the vulnerabilities an opportunistic attacker or malicious user might exploit, with the exception that Meristem will not perform attacks that are likely to cause an outage or degrade the performance of the environment without specific permission. Attackers start with the advantage of unlimited time to gather information about the site and attempt attacks.  Since a finite time is allotted to an application assessment, limiting testers to the identical conditions an attacker would work under is counter productive. Providing additional information in the form of access and even source code, tilts the scale towards the testers.  An application pentest is not a contest to see how good the testers are, the goal is to find as many of the vulnerabilities that already exist as possible, so they can be eliminated, ensuring no attacker can find them...ever.

Mark Hoopes (of Meristem InfoSec) and Heidi Hoopes presented a workshop at RailsConf: 2022 titled "Gaps in the Magic - Exploiting Security Edge Cases in Rails." This class introduced the fundamental concepts of two key vulnerability classes from the OWASP Top 10, SQL injection and Unsafe Deserialization (or unmarshalling in Ruby parlance). It included practice environments where the students could exploit real-world applications (or a replica) to solidify their understanding and enable them to more effectively communicate the risks associated with these vulnerabilities.

SQL Injection

The workshop began with a breakdown of injection vulnerabilities in general and then looked at specific Active Record methods and what constructions would leave them vulnerable to attack. Students were then given a workbook application (building on the work of Justin Collins, published at https://rails-sqli.org/) where they could explore vulnerable Active Record methods and construct their own SQL injection payloads to accomplish typical hacker goals of bypassing login checks and retrieving sensitive information from the database.

This portion concluded by pointing students at an installation of a vulnerable application found on GitHub (link to be posted once a remediating pull request is accepted) and allowing them to apply their skills to recover the password hash of a simulated user.

Unmarshalling

The workshop then shifted gears to the risks of processing serialized objects that could have been created or modified by a user.  Again, the discussion began with the basics of serialization and the specifics of the Marshal protocol in Ruby.  Note that the talk "Caching without Marshal" by Chris Salzberg also at RailsConf 2022 gave an excellent overview of the protocol and a possible alternative and is well worth watching when available.  Once the concepts behind marshalling had been presented, the process of unmarshalling was examined, focusing on the fact that the once the application has called the marshal.load method, no further defensive checks by the application are possible until a potential payload has been triggered.

Next the concept of a gadget was presented, where a gadget is a set of classes that can be arranged in such a way that when they are unmarshalled, executes code that has been embedded as an attribute of one of the classes.  One gadget was discussed in detail as an example of how this is accomplished, and references to other well known gadgets were shared.

Finally, a walk-through of a vulnerability identified during a commercial engagement was performed using a replica of the real site's functionality.  This included identifying the serialized object, preparing a payload, and sending the marshalled object to execute code on the demonstration server.

Resources

The SQLi Workbook application, a replica of the vulnerable commercial application, and the slides from the presentation are posted in the Resources Section.