I ended up writing more than I had planned, so instead of leaving it to rot in a private message, I thought I’d publish it to get some feedback from others. Comments and criticisms are welcome.

What do I think about migration: I think it can offer many new opportunities, as well as many risks for the migrant. The people who migrate (to live) are the ones who have least to lose, or the most to gain.

The less they have in their own country, the more likely they are to migrate. At the bottom of the luck pyramid, people are most likely to migrate to escape war, and then in roughly decreasing order, famine, poverty, physical aggression (violence) or oppression (e.g. sexual, sexuality, race, nationality or freedom).

In the middle, diplomats and aid workers migrate because it’s required by their work; volunteers migrate to meet their calling and improve their skills and employability.

At the top of the pyramid, consultants migrate because they get paid fantastic sums, far more than most of us could hope to earn, to offer their unique expertise.

In many countries, immigrants are unpopular or even persecuted, as people feel they are taking advantage of better healthcare without paying taxes, or stealing their jobs. Denmark has a large immigrant community who don’t mix with the Danes, and they are very unpopular in a country that extremely tolerant and liberal in every other way imaginable.

Why did I live in other countries: so far because of work, but I also chose a job which offers me this opportunity regularly and I take it at every opportunity, simply because I love travel, new experiences, meeting new people and learning about new cultures.

Did I have any difficulties? A few:

I was mugged twice in Ghana and robbed once in Italy;

I’m not allowed to walk around at night in Kenya, so I have to travel everywhere by taxi;

it’s expensive living abroad (I got a supplement from my work which helped a bit);

food poisoning is a risk (I just ignore it and be prepared to be sick occasionally);

malaria is a risk in Africa (I always have to take antimalarial drugs, which are expensive);

it doesn’t affect me, but Africans have an acquired semi-immunity to malaria, which they lose if they spend too long living abroad, and is impossible to get back, so if they move back to Africa tey tend to get very sick;

hot humid weather is uncomfortable for me (air conditioning and fans help);

I miss my family, especially my brother’s kids, and my friends and colleagues when I’m abroad, especially for a long time (I return to the UK every so often);

it can be difficult to build a new social life, especially if you’re not staying for long (learning the language helps);

people treat you like an outsider and they can be aggressive towards you if you have more money than them, or if they think you’re taking their jobs (learning the language helps, and learning to ignore beggars and brush off insults);

moving money is expensive (travellers’ cheques, bank charges) (not much you can do about this one, except change as much as you can in one go);

cash machines won’t take your card or they swallow it, cutting off your supply of ready cash (you might have to courier in a new card or return to your own country to collect a replacement);

What’s it like to live in Africa:

Services are very basic to nonexistant. There is usually no hot water for showers unless you stay in a hotel. If there is some, it’s not reliable. You can’t buy soap or beer or wine or any vegetables except local produce in the towns and villages. Electricity is unreliable, sometimes only a few hours per day, so fridges don’t work well. There’s no TV or radio in rural areas.

Fixed line phones don’t really exist, nor do mobile contracts. Everyone is on pay as you go, and you can buy credit everywhere (almost every shop sells it, from butchers to bars), even if there’s no signal. Sometimes you have to go to one spot in the village where there’s signal to make or receive a call.

Everyone wants to talk to you. Kids point at you because you’re white. Some of them have never seen a white person before. People don’t like being photographed. Education tends to be rough, violent, rote learning, in schools with no glass in the windows and wooden benches that are falling apart. Kids have creativity and imagination beaten out of them. I think this is the biggest tragedy of the continent.

In the capitals you see rich people living and working in high-rise flats and office buildings, and just outside, on the street, will be people walking through the traffic (usually gridlocked), selling apples, maps, toys, phone credit, fried plantain chips, meat pies, football shirts, newspapers, anything you can imagine, or just begging from car to car. Taxis have no seatbelts and most are over 10 years old, second-hand from Europe or Japan, and have chipped or smashed windscreens.

Public transport is virtually nonexistent apart from minibuses that usually carry up to 30 people, cheaply and dangerously, from 6am to 7pm. All have taxis and some countries have motorcycle taxis. Taxis are absurdly cheap, usually a few dollars for a 10 minute ride. US dollars are universal currency (good for emergencies) although the exchange rate is terrible. Bicycles are fairly common, as are carts pulled by donkeys.

Government is usually corrupt and useless. You can’t usually get anything from a government office without paying a bribe. You also can’t expect your local representative to do anything for you or your village, town or state, unless it builds their prestige. Government officials pay themselves very highly, often as much as in Europe, and drive the most expensive cars, although most of their electorate survive on a few dollars a day.

Police earn virtually nothing, and you can only expect trouble from them. They’ll stop your car as an excuse for a bribe. They won’t lift a finger over a crime unless there’s a bribe or someone important was robbed or hurt. Hospitals are few, insanitary (never let them inject you unless you bring your own needles), paid for (although not expensive), overcrowded and disorganised. There is no fire service.

Sewage often flows through open gutters on the street, into the nearest river or the sea, when they have running sewage at all. Most villages have no running water, only hand pumps on a well of uncertain quality. Some people walk for hours every day to fetch water.

Generally do you think that an african coming here would find a great difference in attitudes, civilization level etc?

Capital cities tend to have a lot of wealth and infrastructure compared to the rest of the country. I guess that someone coming from a village would be shocked and awed at the level of civilisation that we have, and someone coming from a city would probably take it in their stride.

People think that the UK had very good, efficient government, but Italy for example doesn’t, so maybe an immigrant would not be surprised. Perhaps an immigrant from Asia might even find that some European countries function less efficiently than their own.

Many African countries have restricted freedom of speech. In most it’s illegal to criticise the monarchy, if there is one; in Uganda it’s illegal to be homosexual; in Zimbabwe and Rwanda people are very careful what they say. Two journalists in Rwanda were recently sent to prison for saying that “some Rwandans were unhappy with the country’s rulers.” Many immigrants would be shocked at the level of honesty and criticism of officials in our press.

Regarding attitudes, I’m not so sure. People in Africa seem to be very religious, and shocked at how secular we are. I’m regularly asked which church I go to. People dress very brightly in Africa, particularly women, and they might find our clothing dull. Men in Africa tend to dress smarly, wearing three-piece suits in 40′C heat, and they might find our slack office dress insulting.

Most Africans are probably used to people being very friendly, open and having plenty of time for everyone, and they might find a highly efficient, ordered and controlled society like Denmark or Germany to be oppressive, boring or just unfriendly. People also tend to get married young, and have more children than we do, and several people have been surprised that I’m not married with kids by age 30.

I hope that’s an interesting and not too biased or untruthful report.

2009-06-27 21:14:58 host name alias list truncated for 69.10.169.230

I guessed that this meant that the host had a long list of reverse name mappings (IP to name). Curious as to why, I did a DNS lookup on that IP:

chris@top ~ $ host 69.10.169.230 | wc -l
86

chris@top ~ $ host 69.10.169.230 | head -5
;; Truncated, retrying in TCP mode.
230.169.10.69.in-addr.arpa domain name pointer heavenlydonut.com.
230.169.10.69.in-addr.arpa domain name pointer pitrivertribe.org.
230.169.10.69.in-addr.arpa domain name pointer shastawebmail.com.
230.169.10.69.in-addr.arpa domain name pointer vidalvineyard.com.

So, the host has 86 names, right? And they all look like spam domains to me.

This looks like someone is trying hard to get around SMTP HELO verification, by providing a valid domain with forward and reverse lookups that map to their own IP. But they tried a bit too hard, because that’s a LONG list of domains. Nobody does that in the real world, I think.

So I decided to block mail from anyone with more than four reverse DNS entries. I have no idea what the collateral damage will be. I’m going to keep an eye on it.

Luckily, Exim makes this very easy:

defer
        set acl_c_ptr_count = ${reduce {${lookup dnsdb{>: \
                ptr=$sender_host_address}}} {0} {${eval:$value+1}}}
        condition = ${if >{$acl_c_ptr_count}{4}}
        message = Too many PTR records ($acl_c_ptr_count)

This counts the number of entries in the PTR list, assigns it to a local variable, and tests whether that number is greater than four. If so, it defers the message (tells the sender to come back later). This gives me a chance to fix it if I discover that it’s rejecting valid email, and still get the message.

The code to count the number of entries in a list is pretty ugly. I don’t suppose anyone wants to implement a “count” operation to count the number of items in a list in Exim?

Adobe uses a proprietary encrypted communications system between their Flash player and their Media Server product. This is intended to ensure that only people who pay for Flash Media Server can stream Flash movies, and only official clients can access them.

In other words, it’s a copy protection (DRM) scam. It’s completely antithetical to the goals of running a free software desktop or serving content to users using free software. However, despite Adobe’s claims, it doesn’t actually provide any security except through the obscurity of the protocol and some short secret keys.

lkcl claims to have created an open source, clean-room implementation of this protocol, called RTMPE, and published it on Sourceforge. Despite promising in January to open RTMP, Adobe wants to protect their revenue stream, so they sent a DMCA takedown notice to Sourceforge, who complied by censoring the project.

If you value your freedom to publish and receive Flash videos using free software, help us fight Adobe and embarrass SourceForge by nominating rtmpdump for “Best Project for Multimedia” in the SourceForge Community Choice awards.

If you just want to download it, here are some handy links now that it’s been censored by SourceForge: LKCL sehe.nl megashare.com mininova.org sumotorrent.com fulldls.com btjunkie.org mybittorrent.com demonoid.com mininova/TOR.

Sometimes you want to use a live CD environment without a CD. CDs are slow, get lost and scratched, and require a CD drive. If you’re going to use live environments a lot, you’d probably prefer to boot them over the network from a machine with a hard disk and a cache.

Luckily, Ubuntu’s live CD includes all the necessary support to do this easily, if you know how to use it. Unfortunately, it’s not really documented as far as I can tell. Please correct me if I’m wrong about this.

I managed to make the live CD boot over the network on a PXE client using the following steps.

• set your DHCP server up to hand off to a TFTP server. For example, add the following lines to your subnet definition in /etc/dhcp3/dhcpd.conf:

next-server 10.0.156.34;
filename "pxelinux.0";

• get a copy of pxelinux.0 from the pxelinux package and put it in the tftproot of your TFTP server.
• copy the casper directory off the CD and put it into your tftproot as well.
• get an NFS server on your network to loopback-mount the Desktop ISO (e.g. ubuntu-8.04.2-desktop-i386.iso) and export the mount directory through NFS. Let’s say your NFS server is 1.2.3.4 and the ISO is mounted at /var/nfs/ubuntu/live. Edit /etc/exports on the server and export the mount directory to the world by adding the following line:

/var/nfs/ubuntu/live *(ro,all_squash,no_subtree_check)

• put the following section into your tftproot/pxelinux.cfg/default file:

DEFAULT live-804
LABEL live-804
  kernel casper/vmlinuz
  append file=/cdrom/preseed/ubuntu.seed boot=casper initrd=ubuntu/ubuntu-8-04/casper/initrd.gz netboot=nfs nfsroot=1.2.3.4:/var/nfs/ubuntu/live quiet splash --

• test that the PXE client boots into the live CD environment
• if it doesn’t, remove the “quiet splash” from the end of the “append” line and boot it again, to see where it gets stuck.

I hope this helps someone, and that NFS-booting a live environment will be properly documented (better than this!) one day.

(Also filed on Ubuntu bug 296089.)

To all members of the IT Industry & Technical Community,

Everyone is well aware that global financial recession has hit even the Tech Giants where companies like Microsoft and Intel have being saying goodbye to thousands of their employees. The situation doesn’t seem to be getting better but interestingly our Pakistani National ICT R&D Fund is thinking about helping Microsoft in Pakistan and we from the industry feel that it is sad that instead of supporting local Hi-Tech Start-ups and struggling IT Entrepreneurs [they are]  funding the usual “Non-Useful” activities like conferences [and] so-called accelerator programs for Pakistan…

To be fair, they have funded a number of open source projects, and funding for conferences and other networking activities is always in short supply for those without a significant marketing budget.

I have come to know through my friends in the IT Industry that the National ICT R&D Fund has signed an MoU with Microsoft to fund the Microsoft Developers Conference and something called an “Innovators Accelerator Program”. The funds haven’t been disbursed yet but it definitely annoys me and many of my friends in the IT industry that our government should fund Microsoft initiatives which is already a global giant. I have heard that around 5 million rupees [about USD 60,000] or thereabouts for the innovation accelerator program which will involve Microsoft training, entrepreneurship training and connecting with Microsoft partners and similar amounts related.

I also find it strange that Pakistan would choose it invest money in Microsoft at this time, despite their obvious experience and competence with open source. Others come to the Fund’s defence, saying:

ICT R&D Fund is one of the few institutions in the country that are doing an excellent job… [it] is the role of a funding agency to encourage collaborations for promoting research cultures and provide help in bringing the best minds closer.

But nobody has denied that the Fund has signed an MoU with Microsoft, or argued for its benefit to Pakistan. Fouad also writes:

When will our national institutions support its people, the vulnerable, not the already empowered? Why doesn’t it support the local entrepreneurs, the ones that don’t have large companies or university backings? Why does it have liabilities to include universities whereas it knows what the state of R&D in universities has been except for a few handful? Why doesn’t it include this money for Social Enterprise and created a NATIONAL INCUBATION AND ACCELERATION CENTRE where people like me or you or anyone can walk in and build their ideas and companies?

Ashiq Anjum replies that “No funding agency can build incubators for industry, probably this is outside of their scope.” But the Fund’s stated goal is “To transform Pakistan’s economy into a knowledge based economy by promoting efficient, sustainable and effective ICT initiatives through synergic development of industrial and academic resources.”

It sounds entirely reasonable on this basis for them to assist university graduates in gaining skills that are useful in the knowledge industry, and in setting up their own companies in the knowledge industry. Indeed, another stated goal is to “make Pakistan an attractive destination for service oriented and research and development related outsourced jobs.”

We can establish centres like http://www.socialinnovation.ca/
and help local entrepreneurs in business development and social innovation with the same amount of money[.] That helps and benefits our people and companies directly as well as innovate for local and international markets.

I agree that all countries should support local development, training and entrepreneurship as much as possible.

(read more from the horse’s business end at http://blogs.msdn.com/ie/archive/2009/02/16/just-the-facts-recap-of-compatibility-view.aspx)

I think this is interesting. On the one hand Microsoft has finally (finally!) decided to bite the bullet and fix some of the bugs in IE that cause web developers so much pain. In my experience, supporting IE’s buggy CSS takes about as much effort as developing the CSS for Firefox in the first place.

Microsoft has always used the excuse before that users would view sites that rendered badly in a new standards-compliant IE and blame IE for the problems. This is an understandable, if self-serving excuse. Perhaps with IE’s market share below 70%, they feel that they can no longer get away with it on the basis of user base alone.

On the other hand, the list has some very interesting entries, apart from nearly every chinese website in existence:

• amazon.com
• blogger.com
• ebay.com
• facebook.com
• google.com
• live.com
• microsoft.com
• msn.com
• myspace.com
• wikipedia.org
• yahoo.com
• youtube.com

I can’t think of a high-profile site that’s not on the list. I think Microsoft has asked a million monkeys to beta-test IE8 and they’re hitting the error report button randomly.

Otherwise, I can only assume that IE8 doesn’t support any websites at all. Perhaps this is the EU-competition-commission version of IE8 that they were testing?

(thanks to PC The Great at lugm.org for the heads-up)

I found it well-written and convincing right up to the last
paragraph but one:

If open source is going to make any real headway in the government, there needs to be an incentive to choose it, not a rule. Time and again, this is where the open source community falls short: Quality code isn’t enough of an incentive. You can put the best engineering in the world
into your product, but if you don’t know how to market, your project will rot in the source repository.

Uhh, non sequitur? Needs to be an incentive to choose it => needs better marketing? Where’s the incentive in marketing? Surely the incentive should be that it’s a better product or that it saves money or time, not that it has flashing lights all over it?

We didn’t have a backup MX because:

• It usually can’t check whether recipients are valid or not, and therefore must accept mail that it can’t deliver;
• It usually doesn’t have as good antispam checks as the primary, because it’s a hassle to keep it updated;
• Spammers usually abuse backup MXes to send more spam, including Joe Jobs.

I thought that this was OK because people who send us mail also have mail servers with queues, which should hold the mail until our server comes back up. It’s normal for mail servers to go down sometimes and this should not cause mail to be lost or returned.

However, we had a report that one of our users did not receive a mail addressed to them, and was told by the sender that it had bounced. I saw the bounce messsage and suspected Exchange, so I decided to check how long Exchange holds messages before bouncing them. Turns out it’s only five hours by default. Most mail servers hold mail for far longer, for example five days, sending a warning message back to the sender after one day.

Bouncing messages looks bad on us. Apart from making our main mail server more reliable we need a backup MX to accept mail when the master is down.

However I do still want to minimise the spam problem that this will cause. Therefore I configured our backup MX to only accept mail when the master is down. Otherwise it defers it, which will tell the sender to try sending it to the master (again).

How did I achieve this magic? With a little Exim configuration that took me a day and that I’m quite proud of. I set up a new virtual machine which just has Exim on it, nothing else. I configured it as an Internet host, and to relay for our most important domains. Then I created /etc/exim4/exim4.conf.localmacros with the following contents:

CHECK_RCPT_LOCAL_ACL_FILE=/etc/exim4/exim4.acl.conf
callout_positive_expire = 5m

This allows us to create a file called /etc/exim4/exim4.acl.conf which contains additional ACL (access control list) conditions. The other change, callout_positive_expire, I’ll describe in a minute.

I created /etc/exim4/exim4.acl.conf with the following contents:

# if we know that the primary MX rejects this address, we should too
deny
        ! verify = recipient/callout=30s,defer_ok
        message = Rejected by primary MX

# detect whether the callout is failing, without causing it to
# defer the message. only a warn verb can do this.
warn
        set acl_m_callout_deferred = true
        verify = recipient/callout=30s
        set acl_m_callout_deferred = false

# if the callout did not fail, and the primary mail server is not
# refusing  mail for this address, then it's accepting it, so tell
# our client to try again later
defer
        ! condition = $acl_m_callout_deferred
        message = The primary MX is working, please use it

# callout is failing, main server must be failing,
# accept everything
accept
        message = Accepting mail on behalf of primary MX

The first clause, which has a deny verb, does a callout to the recipient. A callout is an Exim feature which makes a test SMTP connection and starts the process of sending a mail, checking that the recipient would be accepted. This is designed to catch and block emails that the main server would reject. Our backup server has no idea what addresses are valid in our domains; only the primary knows that.

The callout response is cached for the default two hours if it returns a negative result (the recipient does not exist on the master) or five minutes (see callout_positive_expire above) if the address does exist. We use a defer_ok condition here so that if we fail to contact the master, we don’t defer the mail immediately, but instead assume that the address is OK and therefore continue to the next clause.

The second clause of the ACL, which has a warn verb, is what took me so long to work out. Normally, if a condition in a statement returns a result of defer, which means that it failed, the server will defer the whole message (tell the sender to come back later). In almost all cases this is the right thing to do, but it’s the exact opposite of what we want here. We want to accept mail if the callout is failing, not defer it, otherwise our backup MX is useless (it stops accepting mail if the primary goes down).

Because this is such an unusual thing to do, there is no configurable option for it in Exim. The only workaround that I found is that there is exactly one way to avoid a deferring condition causing the message to be deferred: a warn verb. The documentation for the warn verb says:

If any condition on a warn statement cannot be completed (that is, there is some sort of defer), the log line specified by log_message is not written… After a defer, no further conditions or modifiers in the warn statement are processed. The incident is logged, and the ACL continues to be processed, from the next statement onwards.

So what we do is:

1. Set the local variable
   acl_m_callout_deferred to true;
2. Try the callout. If it defers (cannot contact the primary server) then we stop processing the rest of the conditions in the warn statement, as described above;
3. If we get to this point, we know that the callout did not defer, so we set acl_m_callout_deferred to false.

The third clause  of the ACL, which has a defer verb, simply checks the variable that we set above. If we get this far then the primary server is not rejecting this address; and if it’s not deferring either, then it must be accepting mail for the address. In that case, we defer the message, telling our SMTP client to try again later, at which point it will hopefully succeed in delivering directly to the primary.

Callout result caching becomes a problem here. If the master was not reachable, but a previous callout had verified that a particular address existed, and that callout result was cached for the default 24 hours, then the backup MX would defer subsequent mail to that address for the next 24 hours, even if the master went down. This is why we changed the positive callout result caching time to 5 minutes earlier.

The fourth clause  of the ACL, which has an accept verb, is even simpler. It accepts everything that was not denied or deferred earlier. We can only get this far if the master is not accepting or rejecting mail for that address.

So far the configuration appears to work fine and has blocked 14 spam attempts (abusing the backup MX) in 14 hours.

If one forgets to increase the MySQL max_packet_size, then the import breaks somewhere in the middle (around 3 million records) but the Java process keeps producing progress information, so it’s not at all clear that the import has failed. One sign is that the import process rate of progress, as reported by the import tool in pages per second, suddenly speeds up by a factor of 5-10. You may wish to look out for this and abort the import if it happens, and to monitor the import process with mysqladmin processlist to ensure that it’s still doing things.

Installing the MediaWiki ParserFunctions extension solves most of the problems with random program code appearing in articles.

The import will tend to slow down very badly over time. For example, on one system it started at a rate of 160 pages/second and dropped to 18 over a three-day period. At this rate, it would have taken around 5-6 days to import all 7.5 million pages. Using the MySQL disable keys command did not help much, but what did was to restructure the tables to remove all the indexes. You can even do this while the import is running (I did). The SQL commands are:

• ALTER TABLE page MODIFY COLUMN page_id INT(10) UNSIGNED NOT NULL, DROP PRIMARY KEY, DROP INDEX name_title, DROP INDEX page_random, DROP INDEX page_len;
• ALTER TABLE revision MODIFY COLUMN rev_id INT(10) UNSIGNED NOT NULL, DROP PRIMARY KEY, DROP INDEX rev_id, DROP INDEX rev_timestamp, DROP INDEX page_timestamp, DROP INDEX user_timestamp, DROP INDEX usertext_timestamp;
• ALTER TABLE text MODIFY COLUMN old_id INT(10) UNSIGNED NOT NULL, DROP PRIMARY KEY;

The following SQL commands should restore the indexes after the import is complete. If you don’t do this, the MediaWiki site will be very slow in operation.

• ALTER TABLE page MODIFY COLUMN page_id INT(10) UNSIGNED NOT NULL AUTO_INCREMENT PRIMARY KEY, ADD UNIQUE KEY name_title (page_namespace,page_title), ADD KEY page_random (page_random), ADD KEY page_len (page_len);
• ALTER TABLE revision MODIFY COLUMN rev_id int(10) UNSIGNED NOT NULL AUTO_INCREMENT PRIMARY KEY, ADD UNIQUE KEY rev_id (rev_id), ADD KEY rev_timestamp (rev_timestamp),
  ADD KEY page_timestamp (rev_page,rev_timestamp), ADD KEY user_timestamp (rev_user,rev_timestamp), ADD KEY usertext_timestamp (rev_user_text,rev_timestamp);
• ALTER TABLE text MODIFY COLUMN old_id int(10) UNSIGNED NOT NULL AUTO_INCREMENT PRIMARY KEY;

With these changes I was able to achieve import speeds around fifty times faster, or 1000 pages per second, which should make it possible to import the entire Wikipedia in about 2 hours.