Wil Tan

Don’t ask why, I just somehow fell into the mood of blogging again. It’s strange, for a long period I have had absolutely no intention to write. Even when I tried, words failed to flow.

Then, all of a sudden, now it’s all coming back. Every other minute, I’m getting hit by ideas and everything I do, I just feel like blogging it. I could’ve easily spurted in under 140 characters on Twitter or something, but it doesn’t feel right. I’m being possessed by the need to elaborate.

Is this the reverse of the Writer’s block — Writer’s drainage?

Let’s see if I can define it:

Writer’s Drainage

a phenomenon whereby a person who seldom writes suddenly gets streams of inspiration in during a period of time.

If that’s the case, I sure hope this is a chronic case because I certainly need the practice.

Moving on to the next post…

The Unicode Technical Committee is working on encoding emoji (絵文字) in the Unicode Standard and ISO10646. It has spurred loads of discussions on the Unicode mailing list with more than a handful of forked threads, leading to fundamental questions like whether we should even encode them, and what constitutes a character.

Not to worry, the Unicode consortium is a veteran when it comes to dealing with the hairy issues of creating standards that work across languages, cultures and geographic regions. They simply can’t please everyone.

To me, the motivation for this is clear — interoperability. The current state of affairs in the Japanese mobile industry leaves a lot to be desired: across the carriers, there exist different sets of supported emoji’s, different private-use characters, substitution mappings, and code pages (user-defined characters in Shift_JIS, really). As one can imagine, the results is chaos, and as I software engineer, I really don’t want to imagine what those poor software engineers have to do to make it “just work” when a message cross the carrier boundaries.

To illustrate my point, let’s look at what Google does when you send a message with some emoji characters from GMail to each of DoCoMo, Softbank, and au.

(more…)

Normally, I’m a Django-head, but today I wanted a quick way to write a small web app and have heard good things about web.py so I decided to give it a try. I also wanted minimum fuss to get something up-and-running quickly, and decided to have it served by my existing Apache installation with mod_python.

In the process of setting it up, I ran into a few bumps, and would like to share them:

• My setup is: Python 2.5.2, web.py 0.31, Apache 2.2.8 with mod_python 3.3.1
• The section on configuring mod_python on the web.py installation page doesn’t work for this version. In particular, it tells you to do this:

  main = web.wsgifunc(web.webpyfunc(urls, globals()))
  

  which seems to be the API in web.py 0.2. Using it on 0.3 will land you a nice AttributeError: 'module' object has no attribute 'wsgifunc'. Instead, do this:

  app = web.application(urls, globals(), autoreload=False)
  main = app.wsgifunc()
  

  It’s also worth noting that the version of modpython_gateway.py I’m using is revision 106. With this version, there is no need to place it in the wsgiref package directory since it no longer depend on wsgiref.

• My Apache config is:

    Alias /path /vhosts/dready.org/apps/myapp/
    <Directory /vhosts/dready.org/apps/myapp/>
      Allow from all
  
      <IfModule python_module>
        SetHandler python-program
        PythonHandler modpython_gateway::handler
        PythonOption wsgi.application codep::main
        PythonOption SCRIPT_NAME /path
      </IfModule>
    </Directory>
  

• Notice that I specified the autoreload argument, because the other problem I ran into was that web.py complained the following:

  Traceback (most recent call last):
    File "/usr/local/lib/python2.5/site-packages/web.py-0.31-py2.5.egg/web/application.py", line 209, in process
      return p(lambda: process(processors))
    File "/usr/local/lib/python2.5/site-packages/web.py-0.31-py2.5.egg/web/application.py", line 536, in processor
      h()
    File "/usr/local/lib/python2.5/site-packages/web.py-0.31-py2.5.egg/web/application.py", line 73, in reload_mapping
      mod = __import__(module_name)
  ImportError: No module named _mp_4ba3724007eff4c08a72054c4da2c222
  

  I’m not sure the reason for this, but definitely has something to do with web.py—s auto-reload feature, and the fact that it is running under mod_python so the module name seems to be mangled.

• The other thing is that if you ever run into the problem of getting a file save dialog when requesting the root URL of your app, with the content type of httpd/unix-directory, but not other URLs, this could be due to either Apache or mod_python passing the hint on the request, so if you don’t respond with a Content-Type header (using web.header('Content-Type', 'text/html'), that hint gets copied over to the response.

This presentation by =zigorou (Toru Yamaguchi) titled “Considering OpenID for Mobile” (Thanks =peterd and =nat) is particularly interesting for me due in part to MojiPage as well as my dabblings in various OpenID implementations.

Knowing that mobile usability is an important topic for OpenID implementors and users, and the rest of world probably has a lot to learn from the Japanese mobile ecosystem, I decided to exercise my Japanese skills and extract some points out of the slides (no, I’m not going to, nor qualified enough to do a full translation. Try Google for that). Please note that this is merely a best-effort translation and any error you find is likely to come from my ignorance.

Translated version follows the actual presentation:

• Considering Mobile OpenID
  • Taking into account Japan’s unique mobile scenario, explore the feasibility and methods of implementing mobile OpenID
  • Investigate if the protocol needs to be modified
  • Look at ways to make use of mobile characteristics to enhance usability
• Limitations and Characteristics of Mobile (Devices)
  • Technical Limitations
    • maximum URL length
    • Cookies support
    • IP address blocks and individual identification number (are we talking about carrier gateway IPs and identification headers?)
  • Device Limitations
    • manual URL entry – no way
    • manual ID/password input seems too difficult
• Maximum URL Length — Carrier Restrictions
  • DoCoMo
    • According to “NTT DoCoMo’s standard / guide for common device capabilities”…
    • Encoded URL maximum length = 512 bytes
  • au
    • From Open Appli (Java)
      • Maximum 256 bytes and ASCII only
      • Since this is only for “Open Appli (Java)” stuff, this source may not be reliable
  • SoftBank
    • From Web & Network Technical Information / Development Tool -> HTML Chapter – 2.5.8 URI Restrictions
      • Roughly 1024 bytes
      • Since this is only for “Open Appli (Java)” stuff, this source may not be reliable
  • Summary: Assume 256 bytes
    • Surprisingly small – is that ok?
    • As for au, since source code is in Java, should assume at most 512 bytes
• URL Length — Practical Observation
  • Logging in to pibb using myopenid (with SREG)
    • checkid_setup > 567 bytes
    • id_res > 921 bytes
    • checkid_immediate > 430 bytes
• URL Length — Think Harder
  • UA is used for Indirect Communication, so we only need to consider checkid_setup/checkid_immediate, id_res/setup_needed/cancel
  • checkid_* authentication requests can be done using GET or POST. With POST requests, the URL parameters go into the request body.
  • id_res/setup_needed/cancel comes in the form of HTTP redirect, so GET requests only
  • Empirically: id_res parameters requires a combined size of 728 bytes + the URL endpoint itself.
• Content Body Limitation for POST
  • According to ziguzagu: minimum 5KB (for DoCoMo PDC) => should be sufficient
  • we still need to solve the problem of indirect communication: id_res requires a lot of parameters
• Tackling id_res
  • Basics: OP attaches the authentication result as query parameters to the return_to URL and redirects the UA to it. Do we need anything other than the following for signature verification?
    • openid.mode
    • openid.ns
    • openid.response_nonce
    • openid.assoc_handle
    • openid.signed
    • openid.sig
  • According to OpenID Authentication 2.0, Section 11 “Verifying Assertion”:
    • The following steps are taken
      1. Verifying return_to URL
      2. Verifying discovered information
      3. Checking the nonce
      4. Verifying Signatures
    • Are the first two steps absolutely necessary?
    • The basic goal is to verify the data from the OP via indirect communication, it might be good enough to verify signature + nonce in this step?
    • If we had the ability to verify signature in check_authentication, that might work?
  • Therefore, adding mode, signed, sig, response_nonce to the URL, we get 468 bytes!
  • Moreover, if we delegate everything to the OP, we only need response_nonce and op_endpoint => 289 bytes.
  • Conclusion
    • Minimize id_res as much as possible
    • If we have a method to ask the OP via direct communication (keying off response_nonce), that would work.
    • We could also obtain parameters like identity and claimed_id from the OP in the same vein
• Cookie support in Mobile
  • DoCoMo doesn’t support cookies in the first place
  • au has separate secure/non-secure characteristics (?), and there’s a pitfall in using cookies in non-secure mode. (Translator’s note: Not quite sure what this means)
  • For Softbank, one cannot access cookies in SSL realm in non-SSL pages (much like the secure-characteristic operation)
  • Therefore, cookies are not quite useful for us
• The way that mobile web security works (Translator’s note: heavily interpreted)
  • There are published IP blocks that each mobile carrier uses
  • Unique identification number for the device/subscriber can be provided to the sites (via HTTP header). Used with published IP block, this is an effective way to identify a returning user.
  • Generally, it’s better to obtain the subscriber ID rather than device ID
    • imode ID
    • EZ-number
    • x-jphone-uid
  • Instead of cookies, device/subscriber IDs are used for session management
  • The availability of device/subscriber ID also eases the login process considerably at the OP’s end (during checkid_setup/checkid_immediate)
• What’s possible with Mobile OpenID
  • Unified identity for PC and mobile world
  • Possible ways to link the PC and mobile
    • enabling mobile to as your private information store
    • enabling mobile for settling payment
• Summary
  • There are challenges:
    • Maximum URL length problem
      • First, URL length needs to be shortened
      • Still, without fixing the protocol, we could exceed the URL length limit => counting on =nat for OpenID Authentication 2.1
    • Unique identification + IP — increases the convenience of using OpenID on mobile, but one must always remember to update the IP blocks lists.
  • Looking forward to more active discussions on Mobile OpenID next year