As the release of Rails 4 is coming closer, many people are still considering to upgrade from Rails 2 to 3. One the most challenging issues in this upgrade is handling the issues arising from the XSS protection in Rails 3. According to OWASP, XSS was the second-largest security risk of web applications in 2010. Therefore Rails 3 defaulting to escape all custom strings is a really useful pattern. However, if your application is large and you have plenty of places you are printing HTML, you need to figure out a good process for ensuring all the strings get printed as expected after the upgrade.
Installing rails_xss in Rails 2 is a very good method to see how the XSS protection is going to change things after the upgrade to Rails 3. Depending what kind of content management system solutions, asset packaging tools or helper methods you are using, you can have problems in a plethora of different places. As a developer solving these issues in UserVoice, I would like to share my process for the XSS protected strings handling.
Typically problems that arise look like this. To solve them all, first step is to identify the different types of problems. Reserve some time for this, as there might be tens of different problem patterns, which all might not be easy to find.
The Process
Here I explain how I approached the problem and present some lessons learned during the process. This is not a definitive guide on the subject, but a rather a set of principles which I recommend to be applied during and after the rails_xss installation.
1. Install rails_xss
Install rails_xss and its dependency (erubis). After this, your Rails 2 application escapes all your “unsafe” strings by default. Explore your application and run tests to see on a general level how serious problems the default HTML escaping causes.
2. Identify the Problems
You should identify all the different types of problems and address each problem with a solution. While testing and reviewing the codebase of UserVoice, I identified many different problem patterns. I have described some of the most common ones:
2.1 Custom Helpers
For example, the return value of these kind of helpers can be marked html_safe by calling .html_safe. In this case it’s even better to use the Rails helper link_to, whose return value is html_safe by default. Additionally, you don’t have to worry about escaping the name of the user separately using the h method.
def user_link(user)
"<a href=#{user.url}>#{h(user.name)}</a>".html_safe
end
# better:
def user_link(user)
link_to(user.name, user.url)
end
2.2 Printing Conditional Strings in Views
Often you want to print a certain string depending on some condition. In these cases the resulting string must be marked html_safe. Possibly you also want to extract a commonly occurring pattern in your codebase to its own helper.
By default, unsafe + safe results in an unsafe string, which means that the link above gets unexpectedly escaped. To prevent this, ‘<hr />’ can be marked html_safe or even better, just extract it out side the ERB interpolation tag.
2.4 Printing Custom User-Defined HTML
<%= user.custom_html.html_safe %>
In these cases you need to mark the user-defined HTML html_safe. At the same time, you should also ensure that the user input is sanitized, because you do not want to allow printing arbitrary HTML in your views. A good sanitizing includes defining the set of allowed tags and denying any others. Using a good and well-tested library for sanitizing HTML is also a good idea, unless you really know what you are doing, which you most probably are not.
3. Design a Strategy
When identifying all the various cases of printing HTML, JSON and HTML entities, you might start to wonder how to cover all the cases in all your application’s views. The short is answer is that 100% coverage requires probably too much effort. However, there are many ways how you can come close to that. OWASP states about XSS vulnerability finding: “complete coverage requires a combination of manual code review and manual penetration testing, in addition to any automated approaches in use”.
3.1 Custom Automated Tool
Now that rails_xss had escaped all the output strings by default, I needed to find the places where HTML has been doubly-escaped. So I thought it wouldn’t hurt to write a utility method for matching the doubly-escaped HTML strings:
class HtmlDoubleEscapeReporter
def self.assert_sane(str)
if (str.match(/<[a-z]/) || str.match(/&(quot|rarr|larr|amp|#)/)) &&
!str.match(/looks something you do not want to print/
send_problem_report('#{str}' looks something you do not want to print")
end
return str
end
end
This basically recognizes some commonly occurring strings when HTML has been accidentally escaped twice (as usually you don’t want to print this). Depending on the application, the regular expressions might be much different but the basic idea stays the same. The idea is not to have a fully-functional problem analysis tool, but to just report about any suspicious strings which the team can check and fix if necessary. The method send_problem_report represents a method which reports the problem using e.g. Airbrake or ExceptionNotifier if it happens on a deployed application instance. It could also open a debugger using ruby-debug by calling debugger. When the debugger line is reached when running tests, you are immediately able to see the backtrace to determine if there is a problem or not. Another way is of course throwing an exception, which would print nice error in your CI system and send an exception report in testing environments. This is useful during manual testing, as testers might not notice all the problems.
Notice also the !str.match(/looks something you do not want to print/ part in the if condition. This prevents situations where the error message is used in another sanity check causing an infine loop. Whether this is possible or not, depends on your codebase, however, this occurred to me a couple of times. This check prevents that from happening.
The tricky part is, when to call this method? One solution is to hook it in all HTML printing code by alias_method_chaining simple_format, content_tag_string and link_to, which are the ones rails_xss also chained.
module ActionView
module Helpers
module TextHelper
def simple_format_with_double_escape_reporting(*args)
HtmlDoubleEscapeReporter.assert_sane(simple_format_without_double_escape_reporting(*args))
end
alias_method_chain :simple_format, :double_escape_reporting
end
module TagHelper
private
def content_tag_string_with_double_escape_reporting(*args)
HtmlDoubleEscapeReporter.assert_sane(content_tag_string_without_double_escape_reporting(*args))
end
alias_method_chain :content_tag_string, :double_escape_reporting
end
module UrlHelper
def link_to_with_double_escape_reporting(*args, &block)
HtmlDoubleEscapeReporter.assert_sane(link_to_without_double_escape_reporting(*args, &block))
end
alias_method_chain :link_to, :double_escape_reporting
end
end
end
Remember, that this kind of hooking might not work for your application, so plan and test carefully.
3.2 Manual Testing
In addition to this automated check, you most definitely want to run adequate manual regression tests on your application and involve several people in testing. Additionally, every time you find a problem, you want to grep the whole application for similar cases, as usually these problems exist in many places.
3.3 Code Review
Grepping all the code for suspicious patterns might not always be enough. Therefore you should review the code as much as possible. After seeing various problem situations, you have the best knowledge and “eye” for the problems that might still be hidden in the code. Therefore reading as much code through as possible is a highly recommendable practice in addition to automated and manual testing. Just because testing everything is often impossible.
4. Execute your Strategy
Follow your strategy and make notes on the progress. After completing all the planned steps, instruct your whole team on how to keep the application XSS safe using the new tools which you installed. This helps to keep the application both functional and secure in the future.
Today I wanted to talk about another feature we've added to the iOS
SDK that I didn't cover
before: custom stylesheets. Since you are integrating UserVoice into your app,
we thought it would be good for you to have an easy way to tweak the look and
feel to match your existing UI a bit better. To do this, we extracted most of
the colors that our UI works with into a UVStyleSheet class, and added a simple
API that allows you to substitute your own stylesheet.
To create a custom stylesheet, add a new Objective-C class to your project and
call it something like CustomUVStyleSheet. Make it a subclass of UVStyleSheet.
Then you can override any of the instance methods of UVStyleSheet to provide
your own colors. Here's a simple example:
And with that, you should be able to run your app and see custom colors
throughout the UserVoice UI! You can find a complete list of the colors you can
override in UVStyleSheet.h, with explanations of how they are used.
Our iOS SDK has been
available as a public beta for quite a while. We've gotten some great feedback
from you guys, and I've been working hard over the last few weeks to fix all of
the known bugs and layout issues. We're finally ready to make it 1.0!
At the same time we are announcing that the SDK now supports the iPad!
Landscape mode on the iPhone is now ready to use as well.
In addition to polishing the SDK itself, we have released an example iOS
app on GitHub showing
how to use it. We think this will make it a lot easier for you to get
everything set up correctly in XCode.
To get started, download the example project and pull down the SDK. Replace the
demo URL with your own UserVoice site. Log into your UserVoice account and add
an iOS app under Settings -> Channels -> iOS Apps. Copy the API key and secret
into the example codebase. That should be all you need to do to see your own
feedback forum in the example app.
To set up the SDK in your existing app, you will need to do the following things.
Run XCode 4.2 (there is an XCode 3 branch, but it is not up to date)
Have your project inside an XCode workspace
Download the SDK and drag it into your workspace
Add libUserVoice.a under the Link Binary With Libraries build phase
Set User Heading Search Paths to $(BUILD_PRODUCTS_DIR)
Set Always Search User Paths to Yes
Add -ObjC and -all_load to the Other Linker Flags setting
Drag the Include group from the SDK into your project
Create groups for any added subfolders should be selected
Uncheck any targets
You may want to rename the group something like UVHeaders
Drag the Resources group from the SDK into your project
Create groups for any added subfolders should be selected
Add to your target
You may want to rename the group something like UVResources
Once you have completed these steps, you are ready to launch the UserVoice UI
from your code. Import UserVoice.h and call one of the three methods on the
UserVoice class.
1. Standard Login: This is the most basic option, which will allow users to
either sign in, or create a UserVoice account, from inside the UserVoice UI.
This is ideal if your app does not have any information about the user.
2. SSO for local users: This will find or create a new user by passing a name,
email, and unique id. However, it will only find users that were previously
created using this method. It will not allow you to log the user in as an
existing UserVoice account. This is ideal if you only want to use UserVoice
with your iOS app.
3. UserVoice SSO: This is the most flexible option. It allows you to log the
user in using a UserVoice SSO
token. This is
ideal if you are planning to use single signon with UserVoice across multiple
platforms. We recommend you encrypt the token on your servers and pass it to
the iOS app.
Thanks to the lovely people over at AppFusions we will soon have an awesome integration into Jira 5.0. To wet your appetite for that they have put together this video
I've recently had a few people having issues with the RestSharp library for DotNet. Digging into the code for RestSharp if found that during the encoding process the values of parameters are correctly encoded but the names where being left. You can solve this by change line 161 in OAuth1Authenticator.cs from:
var encodedName = OAuthTools.UrlEncodeStrict(p.Name.ToString());
parameters.Add(new WebPair(encodedName, p.Value.ToString()));
and recompiling RestSharp. The value is encoding later in the OAuth workflow in OAuthTools.cs. I suspect there is a neater solution, which I'll have a think about before submitting a pull request. In the meantime this will get you going. The part of the spec that refers to this states:
3.4.1.3.2. Parameters Normalization
The parameters collected in Section 3.4.1.3 are normalized into a
single string as follows:
1. First, the name and value of each parameter are encoded
(Section 3.6).
Our friends over at BlogPig have created a great integration between UserVoice and Pivotal Tracker. You can see the code and some docs over on their GitHub page here. They also created this short video walk through showing the main features and how easy it is to setup:
Here is another example using our API. This one shows how to create a user from an SSO token, then use that user to create a ticket. Finally we will then list out the tickets.
# helper method
def symbolize_hash(hash)
symbolized_hash = {}
# puts hash.inspect
hash.each do |key, value|
symbolized_hash[key.to_sym] = value
end
return symbolized_hash
end
consumer = OAuth::Consumer.new(KEY, SECRET)
puts "Get request token"
response = consumer.request(:get, "#{SITE}/api/v1/oauth/request_token.json")
request_token_hash = JSON.parse(response.body)
puts request_token_hash
request_token = OAuth::RequestToken.from_hash(consumer,
symbolize_hash(request_token_hash["token"]))
# this is just a library we use to create SSO tokens, similar code
# snippets can be found under the SSO docs here
encoder = Sso::Schemes::AesCbc128::Base64Encoder.new(SUBDOMAIN, API_KEY)
# create the SSO token setting the user to be an admin so
# we can list the tickets back later
sso_token = encoder.process({
:guid => "test@test.com",
:expires => "2012-12-31",
:email => "test@test.com",
:avatar_url => "http://external.com/users/1.png",
:display_name => "Testie McTest",
:admin => "accept"
})
# get the access token for this new user
response = consumer.request(:post, "#{SITE}/api/v1/oauth/authorize.json", nil, {}, {
:sso => sso_token, :request_token => request_token_hash["token"]["oauth_token"]
})
user_hash = JSON.parse(response.body)
access_token = OAuth::AccessToken.from_hash(consumer, symbolize_hash(user_hash["token"]))
# create the params for the ticket
ticket = {
"ticket[message]" => "Everything is broken......",
"ticket[subject]" => "Help!!!"
}
# create a ticket
response = access_token.request(:post, "#{SITE}/api/v1/tickets.json", ticket)
body = JSON.parse(response.body)
# get our ticket
response = access_token.request(:get, "#{SITE}/api/v1/tickets.json?per_page=1")
body = JSON.parse(response.body)
puts body["tickets"][0].inspect
A recent question to our support was how to use 2-legged OAuth to create a new forum, so as the first in a series of api examples here is some code to do exactly that:
