Upgrading to Rails 3: Printing Escaped Strings
As the release of Rails 4 is coming closer, many people are still considering to upgrade from Rails 2 to 3. One the most challenging issues in this upgrade is handling the issues arising from the XSS protection in Rails 3. According to OWASP, XSS was the second-largest security risk of web applications in 2010. Therefore Rails 3 defaulting to escape all custom strings is a really useful pattern. However, if your application is large and you have plenty of places you are printing HTML, you need to figure out a good process for ensuring all the strings get printed as expected after the upgrade.
Installing rails_xss in Rails 2 is a very good method to see how the XSS protection is going to change things after the upgrade to Rails 3. Depending what kind of content management system solutions, asset packaging tools or helper methods you are using, you can have problems in a plethora of different places. As a developer solving these issues in UserVoice, I would like to share my process for the XSS protected strings handling.
Typically problems that arise look like this. To solve them all, first step is to identify the different types of problems. Reserve some time for this, as there might be tens of different problem patterns, which all might not be easy to find.
The Process
Here I explain how I approached the problem and present some lessons learned during the process. This is not a definitive guide on the subject, but a rather a set of principles which I recommend to be applied during and after the rails_xss installation.
1. Install rails_xss
Install rails_xss and its dependency (erubis). After this, your Rails 2 application escapes all your “unsafe” strings by default. Explore your application and run tests to see on a general level how serious problems the default HTML escaping causes.
2. Identify the Problems
You should identify all the different types of problems and address each problem with a solution. While testing and reviewing the codebase of UserVoice, I identified many different problem patterns. I have described some of the most common ones:
2.1 Custom Helpers
For example, the return value of these kind of helpers can be marked html_safe by calling .html_safe. In this case it’s even better to use the Rails helper link_to, whose return value is html_safe by default. Additionally, you don’t have to worry about escaping the name of the user separately using the h method.
def user_link(user)
"<a href=#{user.url}>#{h(user.name)}</a>".html_safe
end
# better:
def user_link(user)
link_to(user.name, user.url)
end
2.2 Printing Conditional Strings in Views
Often you want to print a certain string depending on some condition. In these cases the resulting string must be marked html_safe. Possibly you also want to extract a commonly occurring pattern in your codebase to its own helper.
<span <%= 'style="display: none"'.html_safe unless content.visible? %> >
Content shown if content.visible?
</span>
2.3 Concatenating Strings in Views
Your view files might include concatenation of unsafe and safe strings like this.
<%= '<hr />'.html_safe + link_to('User Information', user_path(user)) %>
By default, unsafe + safe results in an unsafe string, which means that the link above gets unexpectedly escaped. To prevent this, ‘<hr />’ can be marked html_safe or even better, just extract it out side the ERB interpolation tag.
2.4 Printing Custom User-Defined HTML
<%= user.custom_html.html_safe %>
In these cases you need to mark the user-defined HTML html_safe. At the same time, you should also ensure that the user input is sanitized, because you do not want to allow printing arbitrary HTML in your views. A good sanitizing includes defining the set of allowed tags and denying any others. Using a good and well-tested library for sanitizing HTML is also a good idea, unless you really know what you are doing, which you most probably are not.
3. Design a Strategy
When identifying all the various cases of printing HTML, JSON and HTML entities, you might start to wonder how to cover all the cases in all your application’s views. The short is answer is that 100% coverage requires probably too much effort. However, there are many ways how you can come close to that. OWASP states about XSS vulnerability finding: “complete coverage requires a combination of manual code review and manual penetration testing, in addition to any automated approaches in use”.
3.1 Custom Automated Tool
Now that rails_xss had escaped all the output strings by default, I needed to find the places where HTML has been doubly-escaped. So I thought it wouldn’t hurt to write a utility method for matching the doubly-escaped HTML strings:
class HtmlDoubleEscapeReporter
def self.assert_sane(str)
if (str.match(/<[a-z]/) || str.match(/&(quot|rarr|larr|amp|#)/)) &&
!str.match(/looks something you do not want to print/
send_problem_report('#{str}' looks something you do not want to print")
end
return str
end
end
This basically recognizes some commonly occurring strings when HTML has been accidentally escaped twice (as usually you don’t want to print this). Depending on the application, the regular expressions might be much different but the basic idea stays the same. The idea is not to have a fully-functional problem analysis tool, but to just report about any suspicious strings which the team can check and fix if necessary. The method send_problem_report represents a method which reports the problem using e.g. Airbrake or ExceptionNotifier if it happens on a deployed application instance. It could also open a debugger using ruby-debug by calling debugger. When the debugger line is reached when running tests, you are immediately able to see the backtrace to determine if there is a problem or not. Another way is of course throwing an exception, which would print nice error in your CI system and send an exception report in testing environments. This is useful during manual testing, as testers might not notice all the problems.
Notice also the !str.match(/looks something you do not want to print/ part in the if condition. This prevents situations where the error message is used in another sanity check causing an infine loop. Whether this is possible or not, depends on your codebase, however, this occurred to me a couple of times. This check prevents that from happening.
The tricky part is, when to call this method? One solution is to hook it in all HTML printing code by alias_method_chaining simple_format, content_tag_string and link_to, which are the ones rails_xss also chained.
module ActionView
module Helpers
module TextHelper
def simple_format_with_double_escape_reporting(*args)
HtmlDoubleEscapeReporter.assert_sane(simple_format_without_double_escape_reporting(*args))
end
alias_method_chain :simple_format, :double_escape_reporting
end
module TagHelper
private
def content_tag_string_with_double_escape_reporting(*args)
HtmlDoubleEscapeReporter.assert_sane(content_tag_string_without_double_escape_reporting(*args))
end
alias_method_chain :content_tag_string, :double_escape_reporting
end
module UrlHelper
def link_to_with_double_escape_reporting(*args, &block)
HtmlDoubleEscapeReporter.assert_sane(link_to_without_double_escape_reporting(*args, &block))
end
alias_method_chain :link_to, :double_escape_reporting
end
end
end
Remember, that this kind of hooking might not work for your application, so plan and test carefully.
3.2 Manual Testing
In addition to this automated check, you most definitely want to run adequate manual regression tests on your application and involve several people in testing. Additionally, every time you find a problem, you want to grep the whole application for similar cases, as usually these problems exist in many places.
3.3 Code Review
Grepping all the code for suspicious patterns might not always be enough. Therefore you should review the code as much as possible. After seeing various problem situations, you have the best knowledge and “eye” for the problems that might still be hidden in the code. Therefore reading as much code through as possible is a highly recommendable practice in addition to automated and manual testing. Just because testing everything is often impossible.
4. Execute your Strategy
Follow your strategy and make notes on the progress. After completing all the planned steps, instruct your whole team on how to keep the application XSS safe using the new tools which you installed. This helps to keep the application both functional and secure in the future.
~ Raimo
