Subscribe to RSS feed

Posts tagged with "http"

Introducing Device-Stock-UA: a new request header and proposal

, ,

Our latest releases of Opera Mobile and Opera Mini will include a new header: Device-Stock-UA. The value of this header matches that of the stock user agent bundled with the operating system on which Opera Mobile or Mini is running. Below is an example of what this header might look like on an Android device (a T-Mobile myTouch4G).

Device-Stock-UA: Mozilla/5.0 (Linux; U; Android 2.3.4; en-us; myTouch4G Build/GRJ22) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1

We've set a precedent for such a header with X-OperaMini-Phone-UA. Opera Mini includes this header with every request. Other headers, such as X-Original-User-Agent and X-Device-User-Agent exist in the wild and function similarly. None of these headers are standard. Nor do they comply with RFC 6648.

To that end, we're proposing Device-Stock-UA as an industry standard. A draft version of this RFC is available on GitHub. Please contribute and comment. We've worked closely with dotMobi on this proposal; it is also implemented in their goMobi service.

Why introduce a new header?

The goal of Device-Stock-UA is to help mobile site and application developers determine the device on which an HTTP client is running and adapt content accordingly. In mobile-centric web development, there's a tension between the “One Web” philosophy and the realities of networks, protocols, device hardware, and user agent capabilities. We think that this header will lead to a better experience for Opera Mobile and Opera Mini users. And we think this can be useful for other user agents as well.

By embracing the “One Web” philosophy, developers can choose to build a single, responsive experience for a range of devices. Feature-detection and progressive enhancement ensure that any capable browser — not a single, dominant browser — has access to a given resource. Innovations such as meta viewport / @viewport and media queries allow us to streamline development and maintenance with a single code base. It's the approach we advocate here at Opera.

But from a pragmatic point-of-view: data plans cost money and time. Serving fewer bytes helps companies contain bandwidth costs. While we're looking forward to how responsive images will solve a number of these problems through declarative markup, we see a lot of developers choosing server-driven content negotiation: serving markup, CSS, JavaScript, and images based on the value of the User-Agent header.

Most widely-used APIs for server-driven content negotiation use the User-Agent header to infer browser and device capabilities. They assume a one-to-one relationship between a User-Agent string and a device. Opera Mobile and Mini use platform-specific user agent strings — as with desktop browsers — that do not include device information. For Opera Mobile in particular, this created site compatibility and content optimization issues. With no way to determine the device, Opera Mobile would be served the same small images and basic markup as feature phones. The X-OperaMini-Phone-UA header prevented Opera Mini from facing similar issues.

Phasing out X-OperaMini-Phone-UA

In the short-term, Opera Mini will send both the Device-Stock-UA and X-OperaMini-Phone-UA headers. Opera does not have a definitive cut-off date for removing the X-OperaMini-Phone-UA header, but it will happen eventually. We'll also work with popular mobile content negotiation APIs to add support for Device-Stock-UA.

ASP.NET sites - Served With The Wrong Mime-Type

, , , ...

You remember the article I had written about Starbucks solving an error on their server. They were serving HTML with the wrong mime-type to Opera only because of user agent sniffing. They fixed it. They are definitely not the only ones. Other Web sites have exactly the same bad behavior. You may see that there is a pattern. The servers and libraries used are almost always the same.

All these sites and the ones previously solved are into the package OTW-6223. I'm trying to contact each of these sites one by one. It is time consuming with not always a success at the end. Not everyone is as diligent as Rohan Singh (still big respect for him).

  1. if you are a user of this Web site or if you know someone working in the IT department of these companies, please point them to this blog post and ask them to contact me.

  2. If someone from Microsoft is listening, please please please, send a fix to Web developers using ASP.net on their Web sites. I suspect it is a rogue library doing very bad user agent sniffing in ASP.Net. These developers will love you. They will not have to suffer me anymore wink

Web sites serving HTML as application/xhtml+xml to Opera
Website HTTP Headers
New York-New York Hotel
Server: Microsoft-IIS/6.0
        X-Powered-By: ASP.NET
        X-AspNet-Version: 2.0.50727
ASPCA
Server: Microsoft-IIS/7.5
        X-Powered-By: ASP.NET
        X-AspNet-Version: 2.0.50727
MGM Grand
Server: Microsoft-IIS/6.0
        X-Powered-By: ASP.NET
        X-AspNet-Version: 2.0.50727
Share Builder
Server: Microsoft-IIS/6.0
Logitech
Server: Microsoft-IIS/7.0
        X-AspNetMvc-Version: 2.0
        X-AspNet-Version: 4.0.30319
        X-Powered-By: ASP.NET
Sam Shortline Railway
Server: Microsoft-IIS/7.0
        X-AspNet-Version: 4.0.30319
        X-Powered-By: ASP.NET
AllHipHop.com
Server: Microsoft-IIS/6.0
        X-Powered-By: PleskWin
        X-Powered-By: ASP.NET
        X-AspNet-Version: 2.0.50727
        CommunityServer: 2.0.60217.2664
Car Buyers Guide
Server: Microsoft-IIS/7.5
        X-AspNet-Version: 2.0.50727
        X-Powered-By: ASP.NET
Responsible Sports
Server: Microsoft-IIS/7.0
        X-AspNet-Version: 2.0.50727
        X-Powered-By: ASP.NET
Go Army
Server: Microsoft-IIS/6.0
        X-Powered-By: ASP.NET
        X-AspNet-Version: 2.0.50727
Phenomblue
Server: Microsoft-IIS/6.0
        X-Powered-By: ASP.NET
        X-AspNet-Version: 2.0.50727
P.F. Chang's China Bistro
Server: Microsoft-IIS/6.0
        X-Powered-By: ASP.NET
        X-AspNet-Version: 2.0.50727
McAfee
Server: Microsoft-IIS/7.0
        X-Powered-By: ASP.NET
Excalibur - Las Vegas
Server: Microsoft-IIS/6.0
        X-Powered-By: ASP.NET
        X-AspNet-Version: 2.0.50727
Merrill Edge
Server: Microsoft-IIS/6.0
        X-Powered-By: ASP.NET
Walmart Money Card
X-Powered-By: ASP.NET
        X-AspNet-Version: 2.0.50727
        X-AspNetMvc-Version: 2.0
Nelnet
Server: Microsoft-IIS/6.0
        X-Powered-By: ASP.NET
        X-AspNet-Version: 2.0.50727
Soundview Executive Book Summaries
Server: Microsoft-IIS/6.0
        X-Powered-By: ASP.NET
        X-AspNet-Version: 2.0.50727
Ella Moss Official Store
X-OS-Node: 22.5.4

Being Strict About HTTP Link Header

, , , ...

Opera users are reporting Web sites which seem to not work properly. That is good. There are two cases:

  • The browser is wrong and we have to fix it
  • the Web site is wrong is wrong and we try to contact them

But sometimes it is trickier. The browser can be perceived as being wrong when it is right on the technical side.

Someone reported us that he could not connect with Opera to Restlet. When looking at the HTTP headers, we can discover something a bit funny:

% curl -sI http://wiki.restlet.org/
HTTP/1.1 200 OK
Date: Wed, 06 Jul 2011 14:30:33 GMT
Server: Jetty(6.1.9)
X-Cocoon-Version: 2.1.11-dev
X-Daisy-Version: 2.4.1 (build: banana/20101216 16:32:48+0100; run: Linux/i386/2.6.34.6-xxxx-grs-ipv6-32 java/1.5.0_22-b03)
Content-Type: text/html; charset=utf-8
Content-Length: 3825
Link: "http://wiki.restlet.org:80/amcd.json>; rel='acct-mgmt';
X-Account-Management-Status: none
Via: 1.1 wiki.restlet.org (Apache/2.2.9)
Vary: Accept-Encoding

Indeed, the HTTP Link header is broken. The correct syntax is:

Link: <http://wiki.restlet.org:80/amcd.json>; rel='acct-mgmt';

We contacted the owners of the website and they were indeed surprised. Not looking at their websites with Opera, they didn't notice the issue. Other browsers do voodoo magic and silently ignore the faulty HTTP header. The Restlet site is using DaisyCMS. And indeed, all sites using DaisyCMS are exhibiting the same issue. Fortunately, DaisyCMS is an open source project, so I opened a ticket about this bug.

What about Opera?

This is tricky. The perception by common users is that Opera is buggy, when Opera is just saying: "Hey these headers are bogus, I stop here because it might lead to a wrong interpretation of the document." Same usual story. The fact is that most people are not tech savvy enough to understand that, specifically when other browsers have a different behavior and silently recover.

Note that it might be a dangerous path. When the browser does not send an error message to users, they don't see the website's mistakes. It is then less likely that these errors will be reported. It also might lead in some cases in some serious security issues. So there is a right balance to be found for each case. We decided to open a bug, CORE-38210, so that Opera will ignore this type of bogus HTTP Link header in future releases.

Finding the right balance in between a strict implementation of the technology and a smooth user interaction is a challenging art.