I wanted to get acquainted with using socat for UDP, so here’s something whipped up with the help of DNSPython:

python -c "import dns.message, dns.rdataclass, dns.rdatatype; "\
"print dns.message.make_query(dns.name.Name(['dready', 'org', '']), dns.rdatatype.A).to_wire()" \
| socat - UDP:8.8.4.4:53 \
| python -c "import sys, dns.message; "\
"print '\n'.join(str(m) for m in dns.message.from_wire(sys.stdin.read()).answer)"

No related posts.

        ControlMaster auto
        ControlPersist yes
        ControlPath "/tmp/ssh-c-%r@%h:%p"

This works by having all SSH sessions sharing a single “control” connection.

The ControlMaster auto directive tells OpenSSH client to start a control master if one is not already running, otherwise it will reuse a previous one.

The ControlPersist yes directive tells the control master process to continue running in the background even though you’ve exited from the session, allowing subsequent sessions to reuse its connection.

The ControlPath is the control socket rendezvous point for masters and slaves. If you’re on a shared host, you might want to set it to a private location such as in your ~/.ssh/ directory instead of /tmp.

In my case, I stuck these into the wildcard Host section:

Host *
        Compression yes
        ServerAliveInterval 60
        ServerAliveCountMax 5
        ControlMaster auto
        ControlPersist yes
        ControlPath "/tmp/ssh-c-%r@%h:%p"

I’ve always known that OpenSSH had this functionality, but hadn’t bothered to try it out until now and it’s sooo very handy!

No related posts.

GovCampNSW turned out to be one of the most inspiring unconferences I’ve attended, to the credit of the (un)organisers, volunteers, and supporting partners. For the uninitiated:

GovCampNSW is an invitation to be part of an emerging conversation, that may inspire and shape new opportunities for innovation in government in Sydney, in NSW and beyond.

It is an opportunity to talk with a mix of people – from inside and outside government, from the worlds of technology and policy, of community and universities – to talk about shaping an agenda for innovation in NSW and to make a start on that agenda.

My agenda for attending was simple: to gauge the level of awareness of new gTLDs among fellow attendees and take home some pertinent lessons.

It was shaping up to be a great day. I arrived bright and early and bumped into Paul Wallbank, with whom I had an interesting conversation about new gTLDs over coffee! I also managed to speak to the Information Commissioner Deirdre O’Donnell briefly about Whois privacy and the .sydney project.

When the time came to nominate sessions on the white board, I scribbled in one of the slots:
.sydney manifesto ~ public-citizen collaboration

The question I had in mind was “how should the government of NSW involve the stakeholders in its design of the city’s digital identity, including the name space planning and governance aspects of the dotSydney gTLD?”

After the crowd had settled in our little cozy break out spot, I presented the story:

I was hopeful that the discussions would help craft some important points to inform the government about what matters to the stakeholders; hence, the “collaborative manifesto” in the slide title.

Following are my notes from the session. While they are far from being a “manifesto”, it highlights several important areas that may not be apparent to those of us who have been working in the industry for far too long.

• On the Environment and Scoping Study:
  • Dominic Campbell: almost as if they were trying to obscure the whole thing with a name like that
  • Emily Mitchell and others wondered about the results of the study
• Paul Wallbank: the $185,000 ICANN application fee smells like a scam
• Paul Wallbank: How do you define Sydney, and the granularity of it? Who owns it? The NSW government? Or the people?
• John Wells advised us to look at auDA’s Community Geographic Domain Name which has an interesting governance model
• Several participants expressed the concern that: do businesses need to do defensive registration their product or name in every city that the business has a presence in, that also happen to have its own TLD?
• There were also concerns about pricing of domains
• Judy: .sydney could foster a sense of community
• Some people agreed that there’s immense peer pressure — if major metropolitan cities like New York City, Paris and London have their own top-level domains, Sydney must have one too.

It it obvious that there is a lot of work to be done on outreach and public engagement in order to ensure that the .sydney, .melbourne and .victoria TLDs are well-integrated into the respective communities that they represent. We are keen to extend the efforts to the greater public sphere.

Please leave a comment if you have any ideas, suggestions or criticisms.

No related posts.

I’d love that J

WTF is that “J”? Does it stand for “joke”? “Jesus”?

After a while it became apparent that it’s somewhat equivalent to a smiley face, but I was still puzzled by it until I peeked under the hood today and found an email sent from Outlook with the following bit in the HTML part:

I'd love that <span style="font-family:Wingdings">J</span>

A-ha!

When rendered using the Windings font, indeed you get a smiley face:

I'd love that J

And the text/plain part of the email actually does contain the regular :), so you’d only see the “J” showing up if your device is trying to display the HTML version but it doesn’t have the Windings font available.

No related posts.

It got me paranoid and I wanted to quickly scan through my code to make sure that I haven’t got a lurking bug somewhere. This is the script that I came up with, which takes a .py file and look in its AST to identify such constructs and report them:

This is the first time I’m using the ast module, and it took a grand total of 30 minutes to find the module, read the documentation, experiment on the shell and come up with the script. I’d say that’s a pretty good testament to the power and intuitiveness of Python!

Hope you find it useful!

Related posts:

1. Querying the DNS by hand Well, almost. I wanted to get acquainted with using socat for UDP, so here’s something whipped up with the help...

In order to keep my deployment recipe sane, I’ve decided to fork the project on github.

Now, all you need is the following in requirements.txt:

-e git://github.com/wil/hash_ring.git#egg=hash-ring

No related posts.

There are two recurring issues that I keep running into costing hours of fruitless Googling. Hopefully, this will help others:

• SCRIPT_NAME is evil! At least it’s useless in Django, and if present as a FastCGI parameter, Django will trip that away from your request URI and you’re almost certainly going to get / no matter what URI you navigate to! So, save SCRIPT_NAME for your PHP stuff, and leave it out of the parts where you expect to be talking to Django.
  If you have include fastcgi_params in your nginx.conf, make sure it’s not hiding behind the fastcgi_params file.
• If you want SSL to work, i.e. for Django to know that it is serving in https mode, and to build proper Location header for redirects, you need to include the following in the appropriate section of your nginx.conf:
  
fastcgi_param HTTPS on;


  Obviously, if you’re running both plain and SSL version, you shouldn’t be sharing that section or Django will just thinking that it is in SSL mode even when it’s not.

No related posts.

It’s probably mod_php, and try recompiling PHP without zend_multibyte. If you have WITH_MULTIBYTE=true in /var/db/ports/php5/options, change it to WITHOUT_MULTIBYTE=true and rebuild the port.

Long Story

The Apache 2.2 processes on my FreeBSD box keep dying intermittently, with messages such as:


Jul 3 08:08:37 yama kernel: pid 13526 (httpd), uid 80: exited on signal 10
Jul 3 08:25:58 yama kernel: pid 33050 (httpd), uid 80: exited on signal 11


I’ve googled around and tried various extensions.ini reordering tricks without luck. Eventually, I figured out how to get Apache to dump core. First, you need to tell Apache where to dump core by adding the CoreDumpDirectory directive to your httpd.conf. Restart apache.

Then, check to make sure that the kern.coredump and kern.sugid_coredump sysctl variables are set to 1. Because Apache calls setuid and setgid to the configured user after listening on port 80 (privileged port), FreeBSD by default will refuse to dump core on segfaults. Setting kern.sugid_coredump will make it do so.


$ sysctl kern.coredump
kern.coredump: 1
$ sysctl kern.sugid_coredump
kern.sugid_coredump: 0
$ sudo sysctl kern.sugid_coredump=1
kern.sugid_coredump: 0 -> 1


Of course, you’d also need to make sure that your ulimit doesn’t prevent it from dumping core:


$ ulimit -c
unlimited


Great! Now wait for Apache to segfault (or, if you have a reproducible test case, hit it!)

You should now find httpd.core in the directory that you configured in httpd.conf (in my case, it is /tmp.)

Do the usual gdb stuff:


$ gdb /tmp/httpd.core /usr/local/bin/httpd

In my case, I found zend_multibyte_read_script near the top of the stack, so it’s pretty clear that PHP is to blame.
Did I mention that I abhor PHP? Someday I will get rid of it entirely!

This led me to more googling and found that I may have accidentally enabled Zend Multibyte support (well, its purpose is not very well documented and I just assumed that since I do lots of i18n work, it’ll come in handy. Okay so it’s my fault after all but it still shouldn’t crash!)

Changing WITH_MULTIBYTE=true to WITHOUT_MULTIBYTE=true in /var/db/ports/php5/options and rebuilding the port made the problem go away.

Hopefully, the above will help someone in a similar situation.

No related posts.

So, I lurk in that mailing list because Tornado is one of those awesome technologies in the Python world, and there are usually gems flying around.

As with most discussion groups I join, English is the norm. In fact, some have gone as far as to argue that English should be the lingua franca of hackers. To a great extent, I agree.

So, all is well until one day some Chinese messages started appearing on the list. Now, I read and write Chinese but I can’t making a living with Chinese geek-speak. Those messages didn’t bother me though I was thinking it must be annoying to non-Chinese speaking subscribers. And true enough, we got a message from one of the subscribers today:

我们看不懂你的中文信息，写英语，请。

Obviously the output of machine translation with an input to the effect of:

We cannot understand your Chinese messages, write in English, please.

I find it pretty amusing, and hope our fellow Chinese hackers don’t take offense.

No related posts.

When a conflict occurs during a pull operation from remote repository, we get this:

$ git pull kumo1 develop
From ssh://kumo0/home/wil/...
 * branch            develop    -> FETCH_HEAD
Auto-merged src/.../file.py
CONFLICT (content): Merge conflict in src/.../file.py
Automatic merge failed; fix conflicts and then commit the result.

What git did was to fetch objects from the remote repository, and tried to merge it in the branch that you specified. Sometimes the merge operation fails due to a conflict, and the conflicting edits are left in the file. It is then up to you to eyeball the file, straighten it and then “commit the result” (as the message said.)

However, if you tried to commit that file (after fixing the conflict), you’d get this:

$ git commit -m "my fixes" src/.../file.py
fatal: cannot do a partial commit during a merge.

What you’d want is to add the -i argument to the git commit command, which tells it to stage the additional file before committing.

No related posts.