Thursday, 31 July 2014

Why you should never mix HTTP and HTTPS

Hopefully if you are reading this, you are reasonably familiar with HTTPS otherwise known as HTTP-secure and based on encryption, which is usually TLS, although known by its older name of SSL (apparently the name was changed when it moved from the private to the public domain in order to avoid accusations of bias to Netscape who developed it)

Without HTTPS, if your page contains any kind of data, let's say it displays a user name and password to you, then even if you have to login to see that information, at some point that data is sent over the internet in plain text including all the HTML and any scripts etc. that make up the page.

You have pretty much no control or knowledge of where that data goes between the server and the browser. It could go through a country where things are spied on (like the UK!) or it could pass through a corporate network that has spyware installed, hacked network routers or any one of various things that could send that private data to someone else.

It should be obvious that if that data is encrypted in some way, like SSL/TLS, between the server and the client, then even if someone can read the network data, they will not know what the data contains and your data will remain private. The clever thing about SSL/TLS is the encryption key is generated and sent in a way that cannot be intercepted by a spy using the SSL certificate of the server, which can be matched to the URL you typed in and to the authority that issued it.

On some sites, however, you might see this:


or this




And if you hover over it you get a warning that says, "this page contains both secure and insecure data..." and is often caused by a page that uses SSL but includes adverts or plugins/analytics that are only available on plain HTTP. I occasionally see it on Facebook, of all places (naughty Facebook)



but why does this matter? Surely as long as the important stuff is sent over SSL, the rest of the page content can come however it wants can't it?

The problem that this presents is that not only is HTTP readable, it is also modifiable to someone who has a certain amount of network access and who can intercept your request/response. How does that work? Well, if I can act as a proxy and see you (or more specifically, a part of your page) request, let's say http://advertiser.com/someadvert.php, then I can retrieve the result myself and add a few extra pieces of data to it, let's say that I just add an additional script to the result. This script will be invisible and there is no way that the browser will know that the response is not from the server but has been modified in transit. Now imagine that my script runs after the page loads and takes the contents of the username and password form fields and then sends them to my site to collect and user later!

Is that really possible? Absolutely and it is very easy to anyone who can act as a proxy between your browser and the server you think you are talking to. In some countries that is everyone who uses the internet. All of the protection that your page got from SSL has gone in the trash by a simple single use of an HTTP connection.

Sometimes a company might provide an SSL option for extra money and most of us want the free version which is HTTP only but really you should not compromise your site security for the sake of a few dollars. You might not even know your site is hacked because it is not exactly hacked, you simply start getting reports of unexpected logins or unauthorised transactions and have to assume that your site is compromised somehow.

If any part of your site needs to use SSL, then your WHOLE site needs to use it and we should never see the mixed-content warning every again. You can also use a content security policy and/or strict HTTP security http header to force the browser to prevent loading of non-SSL content.

Another cool article here: https://community.qualys.com/blogs/securitylabs/2014/03/19/https-mixed-content-still-the-easiest-way-to-break-ssl

Reporting Services, shared datasets and Parameters

I had a couple of reports in SSRS that I realised shared basically the same dataset with one parameter different. I created a shared dataset, pointed one of the reports at it and starting getting a load of errors. Clearly, I had done something wrong but I wasn't quite sure what. I got errors like "object reference not set to an instance" as well as something about forward references and another about un-passed parameters. Here is what I should have done:


  1. Create your shared dataset. You probably don't want default values specified here but you can if you want to share these defaults across the reports. 
  2. Use the syntax @name to put a parameter into your SQL as you would do in the normal dataset.
  3. If your dataset allows null parameters, these need to be declared in the shared dataset, not just in the report parameters. You can do this in the parameters tab for the shared dataset.
  4. Modify your report dataset and click the radio button, "use a shared dataset" and select your shared dataset. The fields for the shared dataset will appear in the Report Data window in just the same way as if the dataset was local to the report. If you cannot see the Report Data window, it is under the View menu (right at the bottom of my menu) and is also hot key Ctrl-Alt-D by default.
  5. For your parameters, double-click the dataset in the Report Data window and clicking the Parameters tab, assign your report parameters to the dataset parameters. Mine tend to have the same name so @EventId is mapped to [@EventId]
  6. If you need to, change the report parameters (under the Parameters section of the Report Data window) to set defaults, selection sets, nulls etc. It is these that will define what the user can see/set when they run the report.
I think that's it.

Tuesday, 22 July 2014

Setting up report designer on Windows

Reporting Services, part of SQL Server, is very useful for producing management reports based on SQL data. These can be formatted to suit, can be produced on a schedule and can be exported to various formats.

Setting up the report designer requires a few things:

User Pre-requisites

The person who will be designing reports needs to have a basic (or more advanced) understand of how to build SQL database queries. Unless someone is going to create the datasets and leave the report design to the user, they cannot produce a report without either creating a stored procedure or otherwise producing some kind of SELECT.... Once you add report parameters and grouping/sorting, there is a lot of scope for flexibility but also additional learning requirements for the user. If the user is from a database or general programming background (including databases) there should not be too much problem but if the user is not, this stage might take a long time and include many, "Can you help me..." emails!

Ideally, the user should be familiar with Visual Studio since the report designer is mostly based around standard Visual Studio commands and setup. People who are not used to it might feel very intimidated by the number of menus and commands in Visual Studio even though not many of these are needed.

Since there are a number of technical steps, it is helpful if you know the user can find, download and install the software they need, otherwise this would potentially be a 2 hour or more assisted piece of work! As an admin, you will still need to setup the user and permissions information before this person can access the report server.

Software Pre-requisites

There are actually a number of ways to produce reports, including some third-party tools but I am recommending SQL Server Data Tools which is a module for Visual Studio and which comes from Microsoft for free (you need a licence for your report server however). This is partly because I know it works and is supported by MS but also because I am more familiar with it. It can be downloaded from here

Permission Setup

You should setup user permissions on the report server for each user, or possibly at group level depending on how many people you have. You should NOT just share credentials across multiple users. If you need to lock someone out or change the credentials, you do not want to have to manage this across multiple people. It also allows you to track who has changed something when it is done incorrectly.

To do this, you need to setup a top-level site permission (admin or user) and then add item-level permissions, although the item-level permissions are inherited down into folders so you can set this once at the root level if desired. You might also have to consider who can access what with regard to data sources etc. to avoid someone accidentally overwriting one.

To set the site-level permissions, choose "Site Settings" in the top-right of the reports URL:


Then choose the Security tab on the left and add your new user. If this is not a domain account that already exists, you will need to create a local account on the report server itself with a relevant password that the user will need to type in when they visit this site. Choose whether they will be an admin or just a normal user.

Once this is done, go back to the home (top level) directory and assuming you want to apply permissions from here, click the Folder Settings button in the middle:


And again add the user to the list in security with a relevant role to what they are doing.

Setting up the Software

You might be sharing a project, using source control, across multiple users, in which case you should just be able to open this, make your change and deploy directly. If not, however, you will need to create a new Report Project and setup the relevant endpoints in your project properties to point to your server. The defaults should all be OK but you want your TargetReportFolder to point to /Reports/ (by default) and your TargetServerURL to point to http(s)://yourserver/ReportServer (by default).

Once this is done, you can create a new test report and attempt to deploy it to check that it is all setup correctly. Before deploying, you can also visit the site in Internet Explorer (http(s)://yourserver/Reports) in order to check that your login credentials work and you can access the existing reports.




SQL Server Reporting Services (SSRS) with wildcard SSL and Server 2008 R2

Reporting Services is a really useful way of designing reports for data extracted from SQL Server databases, which can be exported to various formats and run on a schedule. It is, however, slightly rough around the edges and takes a bit more effort than it used to, even when installed from SQL Server 2012 SP2!

This post describes what I had to do to get it working over HTTPS with a wildcard certificate but because I ended up doing so much, I don't know which bits related to the cert and which related to other errors and compatibility issues!

Installation

Installation is directly from the SQL Server CD. I had some problems with SP1 of SQL Server 2012 where some kind of broken installation could never be recovered but SQL Server 2012 with SP2 seemed to be fine once I ran an MS fix tool to uninstall all the old stuff. If this happens to you, the setup files install and then nothing more happens!

In my case, I was installing the databases and report server on the same machine (see my previous post for why!) so I chose to install the Database Engine, Reporting Services and the full set of Management Tools. I then just chose the defaults for the instance name and also told it to configure the report server as well.

Broken

So then I ran up the Reporting Services Configuration Manager and went through and set the URLs for the report server and report manager to he https and use the wildcard sll cert that was already installed on the box. I then visited both URLs from a client browser (you MUST use Internet Explorer to work properly) and the ReportServer endpoint looked OK but the Reports (report manager) was blank and said there was an Internal Server Error.

Fixing It

The first thing to look for are the error logs, which are NOT in the Windows Event Log but are written to file in a folder under the SQL Server installation, in my case: C:\Program Files\Microsoft SQL Server\MSRS11.MSSQLSERVER\Reporting Services\LogFiles and they begin with ReportServerService and then have a date and time added onto the end - just look for the one that is modified most recently.

Problem 1 - FIPS compliance

FIPS is a scheme which verifies that encryption-type libraries have been verified secure and correct. More recent versions of Windows contain these new libraries and more importantly, a registry switch which will block an attempt to use a non-FIPS compliant encryption library. Guess what? Reporting Services does and for some reason, this hasn't been updated sadly. The error you get at the bottom of the log is something like: "This implementation is not part of the Windows Platform FIPS validated cryptographic algorithms".

There are two claimed fixes, although the first didn't seem to work. The one that didn't work suggested telling the report server to use triple-des instead of AES for machine key encryption by adding the following line into C:\Program Files\Microsoft SQL Server\MSRS11.MSSQLSERVER\Reporting Services\ReportServer\web.config


I restarted the service (as you should do after anything you change to be sure) and it didn't seem to fix it. The second workaround involves disabling the FIPs compliant requirement in the registry, which seems a bit severe but shouldn't cause any real problems in most cases. You do this by running regedit and modifying the key HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Lsa\FipsAlgorithmPolicy and set Enabled = 0. Again, restart the report server service (easiest to do this from the SQL Server Configuration Manager)

Problem 2 - Wildcard Certificates and SSL

I don't know if this problem is specifically about wildcard certificates or SSL in general but one of the errors I found in the log was about not being able to establish a trust relationship. What is happening is that when you visit the report manager page, under the covers, other web services are being called and by default, they are accessed using "localhost", which is fine unless you are using SSL which requires a domain name to match to the SSL certificate.

Fixing this is reasonable easy. Open your report server configuration file: C:\Program Files\Microsoft SQL Server\MSRS11.MSSQLSERVER\Reporting Services\ReportServer\rsreportserver.config and set SecureConnectionLevel to 3 instead of 1 (not sure why!). Also, set the value of ReportServerUrl and UrlRoot elements to the same value which should be a fully-qualified domain name such as https://myfqdn.compmany.com/ReportServer, which will cause the web application to use this FQDN instead of localhost to access the web service.

Double-check your setup in the Reporting Services Configuration Manager for the endpoints. I used the advanced button on both to just set an SSL endpoint. Even though I run IIS on the same box, it seems to work OK. It ends up looking something like https://+:443/ReportServer and https://+:443/Reports

Problem 3 - Permissions

At this point, I could reach the report manager endpoint and it was displayed correctly but then had an error displayed: The request failed with HTTP status 401: Unauthorized

Obviously related to permissions but which ones? Well first, without really knowing what the report server was trying to access, I decided to set my Windows service credentials to use the Network Service instead of a special account. This way, I could know that it would be able to access most stuff locally without a problem.

This didn't work but then I found another unusual error report that suggested there was a problem with loopback addresses and the credentials used to log in to the report manager page. I don't really understand what was happening but another visit to regedit and creating a DWORD key underneath HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Lsa called DisableLoopbackCheck and set to 1 made it all come to life! The MS support article said this would only happen when run locally but this was happening to me from a browser on another machine so who knows!

Scale-out SQL Server Reporting Services (SSRS) aka Separating Databases and Web Servers

Problem

I have a small pair of test servers, one is not publicly accessible and contains some SQL databases, the other is publicly accessible and has various web applications running on it. I wanted to start using Reporting Services and naturally wanted to put the web server bits on the web server and the databases on the database server.

It turns out this is either not possible or very hard unless you on a domain, something which I am 1) not and 2) unable to do due to other network conflicts with an existing domain.

Explanation

The mechanism to separate databases and the web report servers works using something called a "Scale Out Deployment". The actual installation is easy enough, you first install reporting services databases and configuration onto the database machine (but tell the installation NOT to configure it, in other words, don't create all the web servers etc.). You then run up the configuration tool and check that your settings are as desired such as the correct user names to run the reporting services Windows service as (this is affected by all manner of decisions like whether you need to hop onto other servers etc.).

Once this is done, you install just the reporting services module onto your web server and even though it looks like it is also installing the database engine, it doesn't. You also tell this not to configure during installation and you also run up the reporting services configuration tool although this time, you want to connect to the remote server that is hosting the databases.

This is where it gets difficult because MS uses WMI to call across from one server to another and this requires credentials to pass between the machines. From what I could tell, it uses the credentials of the report server Windows service to gain access to the other machine. If you are not on a domain, no credentials will work across both machines - there is no trust relationship - so the initial connection dialog won't work and you get the error message, "A connection to the computer cannot be established" when you click the Find button.

If you were on a domain, you could use a domain account for the local Windows service and then add the same account into the WMI DCOM permissions on the origin server to allow access as described here: http://msdn.microsoft.com/en-us/library/ms365170.aspx

I couldn't so I ended up installing the ssrs web servers and databases onto the web server all on the same box. It still didn't all work but that is for another post...

Friday, 11 July 2014

Azure In-role cache and multiple instances

Problem

Some random reboots on our cache role took > 15 minutes and being a single point of failure, it made our site go down with a server error!

Cause 

My assumption that a cache role could only have one instance since it was, by nature, shared memory for (in my case) session information.

Solution

Increase the number of instances: MS recommend 4 but I went with 3. In the cache settings for the cache role, tick the "high availability" check box and this will tell the Azure fabric to duplicate cache data between the multiple instances so that if one of them falls over, the fabric will automatically switch over to another one. This would still leave 2 instances but I think MS are thinking that it would be feasible for all of them to be updated and rebooted before the first one has come back to life again. This is low risk for our application, I think.

Alternative

Azure now provide a cache service as part of the top level cloud products. The problem is that it seems expensive for what it is. I can have 3 instances, each with 1.7GB of RAM for a total of about £30 per month, whereas this would only buy me 256MB of Azure cache and it is not high availability and doesn't support notifications. For those features, you need to go up a level to their middle and top tier versions which have a larger minimum size and are much more expensive (the top one starts at £260/month for 5 GB).

Monday, 7 July 2014

XCode linking problems

XCode, the more I know, the more I dislike.

It seems that even easy to understand problems take ages to fix!

I wanted to create Universal static libraries, for some reason, this is not automatic and it means you would otherwise need to link different libraries to run in the simulator as you would to run on a phone. I don't know why the simulator doesn't support all architectures but that is another story...Anyway, using universal libraries (or fat libraries) it means you can effectively bundle both into the same library and have the compiler of the application choose the correct part of it to link with. That's great and the article about how to do it is here:

http://www.raywenderlich.com/41377/creating-a-static-library-in-ios-tutorial

So, that was all done. I then deleted the libraries that my app was linking to (the arm versions) and then chose to link to the universal binaries. I expected it to work. Imagine my anger and confusion when the linker then said:


ignoring file /Users/luke/Library/Developer/Xcode/DerivedData/XrLib-awcaejkcuywzxtdcadbvidrfnbgt/Build/Products/Debug-iphoneos/libXrLib.a, missing required architecture i386 in file.....

So the universal library that I linked is not being used but the iphoneos version (the original), which quite rightly doesn't include the i386 architecture. I went into the "Link binary with libraries" section and sure enough, it WAS linked to the universal one so why on earth was the linker using the wrong one? I tried cleaning, restarting and everything but nope.

Well it turns out that when you add a library to the "Link binary with libraries", it adds the path to the "Library search paths" in Build Settings. More importantly, it does NOT remove this path if you remove the link. The linker then looks for the library you linked to but doesn't use its path, it uses the library search paths and finds the wrong one first!

What I had to do was remove the old paths from the "library search paths" and it all worked as expected.

XCode is so old-fashioned and it seems that ever error is caused by at 10 different things and has 100 different solutions, many of which are nasty and manual and some even (shock horror) suggest that you manually edit the XML project file. I mean really Apple? Can you not actually rewrite this shocking piece of software and make it work as expected?

More Eclipse/Java and XCode/iOS woes

I am at the Zapp Attack hackathon in London's Hoxton. Although I came in with a good idea of what I was going to hack, one of the pre-requisites was having the mobile code to change. I write Android and iOS but am more comfortable with Android so asked for the code, which was given to me.

The thing was, it used some called gradle to build (because the world needs another tool for development) and it uses maven to pull in dependencies. Naturally, the people who own the code had not really designed it to be copied onto other computers so when I did so, I then realised that some of the dependencies were not available online and were pulled in locally. I didn't have a local repository so I thought I'd create one. I downloaded maven (another tool) and although it installed easily enough, it wasn't obvious how to actually use it. I then found what looked like a promising github project that promised to get all the Android dependencies and install into my local repo, another tool, another waste of time. It required everything in the Android SDK to be downloaded, otherwise it would error (rather than ignore the stuff it couldn't install) so I did that but it was still really confusing about how to get the support and appcompat libraries, not helped by the lack of documentation and the way in which Google have changed it over the years. Also, you see promises of "you can get this direct from Google" but how? All I have is a dependency in some text file.

I eventually got it to run but after all that time, it didn't even seem to do anything. No APKs no nothing really. The guy suggested I tried the gradle plugin for Eclipse which I did and although that didn't take too long to install, that didn't do anything either.

I decided to opt for the iOS code. Even though I am less skilled with iOS, the support guy said he could help and it would be easier since it was just an XCOde project. Well, it was just another 3 hours of pain trying to get it all to work. It's fine when it's all set up on a working machine somewhere but copy it across and then try and get all the dependencies working! Which projects to open and build? What to re-link?

I ended up with another whole world of pain with their infamous linker errors but that is for another post....

Moral? Become a green grocer or something!