karlcow

Opening The Web one bug at a time

CAFBank and Server Side User Agent Sniffing

,

I always hesitate to share the issues we have when we try to contact companies with Web sites not working in Opera. The Open The Web Team is doing that every day. The intent of this post is to illustrate some of the issues we are facing. Hopefully it will help people:

  1. understand the context of our work
  2. take actions when they are users of these Web sites


Let's talk about OTW-2621. The bug was created on September 9, 2006 (more than 5 years ago). The bug report was very clear. At this time Opera 9 was released. Opera Users of this bank were unable to use the Bank Web site. In this case, sometimes a more advanced user will report the issue to Opera. It's normal. Users have no idea if the site is broken or if Opera has a bug. In Cinemascope…

What is happening for users



I start a clean version of Opera Next (beta 12) no cookies, no cache, nothing and I enter the address of the Web site. Type enter.



OK two things have happened. The address bar has changed. Opera has been redirected to a new address, and there is a request asking for certificate confirmation. Let's accept the certificate request. The browser is redirected to a Browser Unsupported page.

https://www.cafbank.org.uk/unsupported.htm



OK. Let's try something else. ctrl+click or right+click on the page and choose "Edit Site Preferences" then select the Network Tab and finally identifying Opera as Firefox. It means the user agent will send this User Agent String instead of the normal one.

User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7.3; fr; rv:2.0) Gecko/20100101 Firefox/4.0 Opera 12.00


We try to access the Web site and drumroll…



https://www.cafbank.org.uk/online/ASPScripts/Logon.asp


Opera is redirected this time to the right page. Note at this moment that in the Web site stats, Opera will be identified as Firefox. Maybe we could change the motto of Opera to "We increase the market share of Firefox" wink

Our friend - User Agent Sniffing



So as usual in these circumstances, I go to the command line to see what is happening.

→ curl -sI -A "Opera/9.80 (Macintosh; Intel Mac OS X 10.7.3; U; fr) Presto/2.10.229 Version/11.62" http://www.cafbank.org.uk/

HTTP/1.1 302 Redirect
Content-Length: 156
Content-Type: text/html
Location: https://secure.cafbank.org/online
X-Powered-By: ASP.NET
Date: Thu, 26 Apr 2012 20:55:19 GMT


An initial redirection, then a second one.

→ curl -sI -A "Opera/9.80 (Macintosh; Intel Mac OS X 10.7.3; U; fr) Presto/2.10.229 Version/11.62" https://secure.cafbank.org/online

HTTP/1.1 301 Moved Permanently
Content-Length: 157
Content-Type: text/html
Location: https://secure.cafbank.org/online/
X-Powered-By: ASP.NET
Date: Thu, 26 Apr 2012 20:57:56 GMT


And finally the last one.

→ curl -sI -A "Opera/9.80 (Macintosh; Intel Mac OS X 10.7.3; U; fr) Presto/2.10.229 Version/11.62" https://secure.cafbank.org/online/

HTTP/1.1 302 Object moved
Cache-Control: private
Content-Length: 163
Content-Type: text/html
Location: https://www.cafbank.org.uk/unsupported.htm
X-Powered-By: ASP.NET
Set-Cookie: ASPSESSIONIDCSTCAABS=GLINBDNAGGOAPPLAHFHABCFL; path=/
Date: Thu, 26 Apr 2012 20:58:45 GMT


OK Nothing much we can do. The site is working when identify as Firefox. There is no implementation bug on Opera side. The redirection is happening on the server side. Let's contact CAFBank.

Contacting companies



The first issue usually contacting Web sites is that it is almost impossible to reach the right persons. Options are terse. The bigger the company is, the harder it is. Some companies have zillions of Web sites, working with local Web agencies. When there is a contact form, the person receiving the message doesn't often have the right knowledge to be able to communicate the information.

But we try. It goes usually something like this.

Madam, Sir,

I'm working for Opera Software's Developer Relations team. 
We have received multiple reports from your customers that 
www.cafbank.org.uk website does not work properly in our 
browser  product in some circumstances.

Could you put me in contact with the appropriate person in 
the Communications/Marketing team and/or Technical team 
in charge of your Web site.

Issue

When accessing https://www.cafbank.org.uk/ with  Opera 
browsers, users are being redirected to 
https://www.cafbank.org.uk/unsupported.htm

They also can't access to their accounts.
https://secure.cafbank.org/online/ASPScripts/Logon.asp
Solution

It seems that the Web site is doing server-side sniffing
for https://secure.cafbank.org/online/ASPScripts/Logon.asp

  HTTP/1.1 302 Object moved
  Cache-Control: private
  Content-Length: 163
  Content-Type: text/html
  Location: https://www.cafbank.org.uk/unsupported.htm
  X-Powered-By: ASP.NET
  Set-Cookie: ASPSESSIONIDQAASSRBS=HFKPDKOAHHKNDBHEFPOHOOBL; path=/
  Date: Wed, 11 May 2011 14:32:31 GMT

We would like to find a solution for fixing these. Could you 
tell us what were your difficulties into creating this site which 
led to this user agent sniffing?

It would be very kind of you, if you could tell us when 
you have fixed this issue. If you have any additional 
issues with Opera browser, we would like to work together 
on solving them. 

Best regards


I tried to contact them on May 11, 2011 from customer services telling me that they would contact their IT services. And then asked for status

  • July 11, 2011. No answer.
  • October 28, 2011. No answer.
  • March 29, 2012. No answer.


Finally today: April 26, 2012! Hurrah! I received an answer. Imagine how happy I was… before reading this email.

Dear Mr Dubost
 
Thank you for your email, my apologies for our 
delay in replying.  CAF Bank customers are able 
to contact us in a variety of methods and not 
just through CAF Bank online.  We do not have 
any plans at present to extend the browsers 
which are supported for CAF Bank online however 
we have noted your comments for when we next 
review our online facility.
 
Thank you,
Regards,
**** *****


Bummer!

What is next?



In fact, I should have done something a lot earlier when we didn't receive replies to the request for the status. I should have asked that CAFbank was added to the sitepatch list. So that each time an Opera user tries to access this bank web site, he/she will be identified as Firefox and have a peaceful experience on the Web site.

What does it achieve?


  • Opera looses market share
  • Opera Users can use the Web site
  • Project Managers continue to ignore Opera because it is not in their stats


I want to make something very clear here. I'm not complaining, I'm just expressing a sad reality. I can share stories like this very often. Maybe I should. I don't know.

curl and User-AgentOracle Support difficult to reach

Comments

ouzowtfouzoWTF Thursday, April 26, 2012 9:35:09 PM

worried

Please share those stories so people out there can see the problems and also which companies are doing such a crappy job.
Maybe it helps if users of such pages are complaining about that too and can point the contact persons of that company to the useful information of your articles.

alharawi Friday, April 27, 2012 7:36:44 AM

Yep, I second ouzoWTF's comment.

Galileo Friday, April 27, 2012 11:22:30 AM

down I agree, its best to share these stories, maybe users of the sites that Opera is ignored will also start to complain to convince the sites to do something.

Karl Dubostkarlcow Friday, April 27, 2012 12:25:17 PM

OK. I will share more in the coming months. Thanks for the feedback.

ouzowtfouzoWTF Friday, April 27, 2012 5:16:49 PM

Thanks smile

Another feedback "slightly" off-topic: The MyOpera username and the "real name" are shown one after another here in the comments. I dont think this is intended wink

Karl Dubostkarlcow Friday, April 27, 2012 7:54:16 PM

hehe smile I have to tweak the CSS.
Fixed! thanks.

nkame Monday, April 30, 2012 3:12:19 PM

Perso je ne mettrais pas Firefox en User-Agent d'impersonification, ça ne fait qu'encourager les gens dans leur approche "WebKit" du Web.

Karl Dubostkarlcow Monday, April 30, 2012 4:20:12 PM

@nkame: tu mettrais quoi ?

Çağlar Yeşilyurtgreench Monday, April 30, 2012 8:06:56 PM

Karl, you should share more and more stories. I learn lots of debugging techniques and use them in here. We have stories but happy ending. http://www.operaturkiye.net/etiket/acik-web/

Stephane Deschampsnotabeneorg Tuesday, June 12, 2012 1:10:58 PM

Interesting story, but only part of it.

Seen from the other side (big company speaking wink), sometimes it's a matter of "some things, a while ago, didn't work with Opera, so we decided to ask people to use another browser".

Then you write to an anonymous "Madam/Sir" who says "thanks, we'll review the question later."

Actually, behind-the-scenes, sometimes things did not work a few versions back and no one has taken the time to check if things were better.

I'm sure that most of the time your emails help, as you know because you've sent some to me that were forwarded to the right person and Opera is going to be taken into account in further iterations of product release.

So sending emails helps. Naming names publicly, I'm not sure (people being people, they don't like being pointed out as the mean little duck). Your mileage may vary, though.

Karl Dubostkarlcow Tuesday, June 12, 2012 2:11:36 PM

Originally posted by notabeneorg:

So sending emails helps. Naming names publicly, I'm not sure (people being people, they don't like being pointed out as the mean little duck). Your mileage may vary, though.



do you mean talking about the company or the person. Above for example, I removed the name of the person. Note also that in the past for Starbucks, it actually helped, because the right people at Starbucks noticed it and fixed the issue.

Stephane Deschampsnotabeneorg Tuesday, June 12, 2012 5:31:05 PM

I was thinking about the company. You may end up having developer teams being cross for having been pointed out by outsiders as what they consider 'being bad at their job'.

But this is why I said I'm not sure, your example is valid in every way. I was thinking out loud, maybe you're right.

Maybe I just like it when things go smoothly. smile

Write a comment

New comments have been disabled for this post.