Rules for Responsible Reformatting: A Developer Manifesto

Editor: Luca Passani, luca.passani at Gmail dot com
Created on March 21, 2008, Last Edit, April 23, Latest Signature, March 5, 2009

This document is about a recent development in mobile web technologies, often referred to as content reformatting.

The mobile web has never been a simple platform to develop for, given the fragmentation of the underlying technology. Different devices, different browsers and different networks have made mobile web development a challenge for programmers and content authors from the beginning.
Content reformatting, which seeks to convert pages designed for PC users to be usable from mobile phones may add another layer of complexity when not properly configured.

Some Mobile Network Operators (MNOs) are implementing reformatting proxies into their networks that intercept and reformat all HTTP traffic to any HTTP server. This is being done without safeguarding those sites which strive to deliver a mobile-optimised user-experience, or, at the very least, not safeguarding them sufficiently.

The introduction of reformatting systems can lead to the disruption of mobile web sites for all of the users in their respective networks.

This situation is a concern for mobile web neutrality and has the potential to jeopardize the mobile web as a platform in the years to come.

To this end, mobile developers of various nationalities and backgrounds, assembled on the WMLProgramming list at Yahoo Groups, have created this manifesto to make their position known.

Consequences of the
Introduction of a Reformatting Proxy

Symptoms of a network that has deployed a reformatting proxy are:

What can be done?

While this document represents the viewpoint of mobile developers, it is not meant to be hostile to proxy vendors and Mobile Network Operators who recognize the importance of the ecosystem and the disruption that the introduction of aggressively configured reformatters can cause to existing mobile sites and services.

To this end, developers have established guidelines that a reformatting proxy should respect, in order to avoid reformatting mobile-optimized sites.

Vendors of reformatting proxies can subscribe to the manifesto when the following two situations are verified:

The first rule is meant to make sure that a reformatting proxy can effectively be configured to respect mobile websites. The second rule is meant to pre-empt carriers' attempts to blame proxy vendors for failure to respect mobile-optimized sites.

The recommended way to endorse the Manifesto is to add the following sentence prominently in the documentation and in the configuration files of the proxy:

"[Vendor] supports the rules established by the Mobile Developer Manifesto for Responsible Reformatting (available at: http://wurfl.sourceforge.net/manifesto/) for its default configuration."

This sentence is meant to remind MNOs of the existence of a mobile ecosystem and the necessity to respect it, a fact which may be unknown to MNO employees at different levels in the respective organizations.

Mobile Network Operators are also invited to subscribe to the Manifesto in order to express their acknowledgment of the rules contained in this manifesto.

Vendors and MNOs who sign the Manifesto will be entitled to a short statement to better qualify their position with regards to the Manifesto.

How to Avoid Reformatting Mobile-Optimised Sites

As a general rule, transcoders should err on the side of non-reformatting, meaning that a reformatting proxy should refrain from reformatting in all of those cases when a site might already be suitable for mobile.
In particular, developers request that the following rules are observed:

No USER AGENT Spoofing: under no circumstances should the original User-Agent string be removed, modified or moved to a different header.

Note: It is considered acceptable for a transcoder to perform a second HTTP fetch of site content with a spoofed HTTP request, as long as the first request has been performed with the original headers and the proxy has positively identified that the website is NOT mobile optmised

Preserve headers: under no circumstances should a transcoder modify or delete existing HTTP headers. The addition of extra x-* headers is admissible.

Recognize Mobile-Specific MIME-types and Document Type Declarations (DTDs): documents served with the following MIME types:

application/xhtml+xml,

text/vnd.wap.wml,

application/vnd.wap.xhtml+xml

and documents served with the XHTML-Basic 1.0 or XHTML-MP 1.1 DTD:

<!DOCTYPE html PUBLIC "-//OMA//DTD XHTML Mobile 1.2//EN"
       "http://www.openmobilealliance.org/tech/DTD/xhtml-mobile12.dtd">

<!DOCTYPE html PUBLIC "-//WAPFORUM//DTD XHTML Mobile 1.1//EN"
       "http://www.wapforum.org/DTD/xhtml-mobile11.dtd">

<!DOCTYPE html PUBLIC "-//WAPFORUM//DTD XHTML Mobile 1.0//EN"
       "http://www.wapforum.org/DTD/xhtml-mobile10.dtd">

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML Basic 1.1//EN"
       "http://www.w3.org/TR/xhtml-basic/xhtml-basic11.dtd">

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML Basic 1.0//EN"
       "http://www.w3.org/TR/xhtml-basic/xhtml-basic10.dtd">


MUST be considered mobile-optimised and no transcoding should be applied

Do Not Adapt pictures of mobile-optimized sites: mobile-optimized sites should not have their graphics adapted.

30Kb Limit: pages that have a total weight of 30 kilobytes or less MUST not be adapted.

Note: An exception can be made in those cases when none of the other rules apply AND the reformatting proxy is able to positively identify devices which cannot handle 30kb of content through a Device Description Repository (DDR).
In this case a lower threshold for reformatting may be adopted. The limit cannot be lower than 15 kilobytes under any circumstances.
A further exception can be made in those cases when the reformatting proxy can positively identify the web-only nature of the site by using more sophisticated techniques, such as: detecting the use of IFRAMES and other "web-only" tags and scripting. The limit cannot be lower than 15 kilobytes under any circumstances.

Do not reformat pages that return the following HTTP header:

Cache-Control: no-transform

This also applies to the case when the header is specified through the <meta> HTTP-Equiv tag

Do not reformat pages which contain the following meta tag:

<link rel="alternate" type="text/html" media="handheld" href="[url]" title="[title]" />

The proxy should redirect the user to the [url] contained in the href attribute.

Transcoders MUST not reformat sites with URLs matching one of the following patterns:

m.*
mobile.*
wap.*
*.mobi
pda.*
avantgo.*
iphone.*
wml.*
wireless.*
xhtml.*

Note: In the spirit of erring on the side of non-reformatting, vendors are also welcome to support the following exclusion patterns, where "/" represents the first slash in the url PATH:

*/mobile/
*/iphone/
*/wireless/

Support for these extra exclusion patterns is only recommended

Whitelists are acceptable only as inclusion list to inform the reformatter of sites which would normally escape reformatting, but which are marked as reformattable through human intervention. Use of exclusion whitelists is not acceptable.

Transcoders must identify themselves using the Via header as enforced by the HTTP protocol, RFC 2616
( See http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.45 )

Transcoders must send along the IP address of the device/gateway and other proxies using the X-Forwarded-For header, a de-facto standard, http://en.wikipedia.org/wiki/X-Forwarded-For

Reformatter vendors who abide by the rules above for their products' standard configuration and who agree to notify their customers of the importance of behaving sympathetically in the ecosystem are invited to sign the Manifesto.

Operators who need to deploy content reformatting solutions on their network are welcome to read this document and to derive the requirements for their systems from it.

Mobile developers are welcome to sign the Manifesto by sending an email to the editor and requesting that their signature is added.

Signers:
Vendors of Reformatting Proxies
Ed Moore
OpenWeb Product Manager
Openwave Systems.
Statement from Openwave
Eran Wyler
CEO & Founder, InfoGin Ltd.
Statement from InfoGin
Martin Jones
Senior Product Manager
Volantis Systems Ltd.
Statement from Volantis
Duane Wilson
CEO - Massive Mobile Media
Statement from Massivo Mobile Media
Operators
 
Developers
Luca Passani,
WURFL Inventor and WMLProgramming List Owner.

Miha Valencic, technical consultant, Mobitel d.d.

David Harper, Founder & CEO, Winksite

Nigel Choi, Software Engineer, AdMob Inc.

Chris Abbott, CTO, DetectRight.com

C. Enrique Ortiz, Mobility Blogger and CTO at eZee inc.

Christophe Lassus, FlirtyMob, Founder and Director

James R. McLachlan, Managing Director, Objective Software Services Ltd.

Ron Mandel, Adobe Systems, Mobile and Device Development.

Alex Kerr, Managing Director, PhoneThing Ltd,

Deepak Jaiswal, co-founder, Pinngit Mobility Solutions, Bangalore.

Richard Spence, Director, Bluetrail.

Tom Thurston, Vice President, Mobile Animation FC.

Gareth Reakes, CTO, We7 Ltd.

Steven Green, Technical Director, Oxford Softworks Ltd.

Jason Sabella, Senior Software Engineer, Winksite

Karel De Beule, Co-Founder, Kimia Solutions

Raj Singh, Mobile Developer

Herval Freire, CTO, ShotCode.com

Braxton Woodham, CTO, Zannel

Jason Navarro, Mobile Web developer, Mobiquio

Paul Manze, CEO & Founder, EmbedAi Ltd.

Jose Geraldo Magalhaes, CTO, wapja.net

Simon Woodside, CEO, Semacode Corporation

Erik Saastad, Mobile Channel Manager, Dagbladet Medialab

Håvard Solheim, Mobile project manager, Edda Digital

Sylvain Wallez, CTO, Goojet

Chris Thompson, Managing Director, Speak Languages!

Bengt Welin, Head of Technology and Production, Mobiento, Sweden

Ray Anderson, CEO, Bango plc.

Nick Gyles, Chief Technology Officer, WDSGlobal

Barbara Ballard, President, Little Springs Design

Pedro Barros José, Managing Director, Mdatalogic Ltd.

Julio Rabadán González, Founder, Somms Multimedia Solutions.

Paul Poutanen, CEO, Mob4Hire Inc.

Andrea Trento, Founder & CEO, Interpreting.it

Jan Michael Hess, CEO, Mobile Economy GmbH

Brian Dally, Principal, Phone People Software & Strategy Partners

Giles Payne, Co-founder, Telepathix.net

Salvador Gonzalez, Founder & Business development, entumovil.net

Gabriel Palomino, Mobile Industry Consultant, Mobile Media Consult

August Z. Flatby, Founder, Apparat AS

Ed Pimentel, Founder, AgileCO

Francisco Velázquez, IT Director, Froggie

Raul Jimenez, Mobile Developer, Spanish Adobe User Group

Juan Nin, General Manager of Software Development, 3Cinteractive

Richard Yates, Mobile Web Strategist, ARYates Consulting, LLC

Sayuti Chow, Mobile VAS Developer, PT Metadome Mobitech

David Walton, Mobile Developer

Geoff Ballinger, CTO, Mixipix Ltd.

Pål Bråtelund, Head of Mobile, VG Mobil

Tomislav Car, CEO, Infinum Ltd.

Anton Bar, Chief Gheek, http://G.ho.st

Richard Sinclair, Founder and CEO, www.sno.mobi, UK

Fabrizio Capobianco, CEO, Funambol

Dev Gandhi, CEO, NEXAGE / MyCorner.com

Jon Arne Sæterås, Head of Product Development, Mobiletech AS

Morten Hjerde, Senior Interaction Designer,mBricks AS.

Jose Maria Martinez Burgos, 'hafo', CEO & Founder, Animatu

Kirk Bateman, Manging Director, Synaptic Technologies Limited

Mikael Krogius, Founder & CEO, Winwap Technologies

Jason Delport, Developer and co-founder, Paxmodept

Peter J. Cranstone, CEO, 5o9, Inc.

Dennis Bournique, mobile blogger, wapreview.com

Manuel Groeneweg, Owner, Bor3d

Mat Williams, CTO, Reporo Ltd.

Alessandro Pace, FlashLite Development, Biskero.org

Ryan Unger, Partner and Creative Director, Punchkick Interactive

Shaun Zelber, Co-Founder TexoMobile Inc.

Jakub Danilewicz, Alembik Project Architect, Kimia Solutions.

Elias Giannopoulos, Founder & IT Manager, Sports.comm ltd

Benno Bartels, Co-Founder / Developer, insertEFFECT - Web Mobile Specialists

Julien Menard, CTO, SBW Paris

Farez Rahman, Developer and Founder, Redkey Digital Ltd.

Mark Hillsdon, CEO, Barefoot Software Inc.

Roberto Pinia, Business Development Manager, Aspiro AS

Muntasir Mamun Joarder, Data Service Specialist, AKTEL

James Finn, Head of Sales & Business Development, RingRing Media Ltd

Marco Argenti, Managing Director, Dada.net

Mike Rowehl, Founder, Mowser Inc.

Neil Osman, CEO, WW3 (web and mobile web development)

Alex Tur, Co-founder and CEO, Netsoul, LLC

Letizia Jaccheri,
Professor in Software Engineering,
Department of Computer and Information Science,
Norwegian University of Science and Technology

Robert Oberhofer, Phd, Mobile Consultant and Co-Founder JotInfo Inc.

Kanwar Dhaliwal, Mobile Consultant

Terren Suydam, Head of Mobile Development, SingleClick Systems

Adriano Fonseca, Partner and Co-Founder, TopSolutions.pt

Nihan Sin, Mobile Web Developer

Michael Dykman, Distributed System Architect and Developer.

Alexandru Stanciu, Co-Founder, StarTech.ro

Michael Maguire, Founder & CTO, Blue Whale Systems Ltd

Gabriel Balogh, Mobile Developer, MOBIcomp

Scott Beaumont, Co-Founder, Mippin.com

Steve Kamerman, Director of IT / Co-Founder, Tera Technologies Inc

Alfredo Pinkus, Applications Manager, CycleLogic

Ian Homer, Co-Founder, Bemoko

Tim Dedischew, Founder, MobileLeap Inc.

Alex Matthews, Technical Director, Marvellous.

Michael Stone, CEO, Amethon Solutions Pty Ltd

B. Vervoort, Co-Founder & Mobile Developer, InMovil Media Europa

Manuel Dominguez Sarmiento, General Director, Renxo

Idel Fuschini, Mobile Evangelist, Italy

Note:Questions about the Reformatting Manifesto can be directed to the Manifesto editor or publicly on the WMLProgramming list at Yahoo Groups.
Requests to sign the Manifesto can be directed to the editor via email at passani (at) eunet dot no