jsha/README

I’ve been playing around with OpenStreetMaps recently. I downloaded a subset of the world data set covering San Francisco by using this tool: http://78.46.81.38/.  I loaded the data into a local PostGIS database with osm2pgsql.  Then I spent a couple hours installing the prerequisites for Mapnik and building Mapnik itself, which was the hardest part but still very doable.  Once Mapnik was installed, I was able to use the generate_image.py script to generate a quick map of SF.  Next step is to learn how to build tiles for slippy maps, and then how to customize them.

The process was mostly easy and rewarding.  One funny quirk: osm2pgsql can’t do password authentication to the database, so it’s most convenient to use a passwordless account.  Mapnik, on the other hand, appears to choke if it is given an empty password string, so you need to add a password to the account once you get to the Mapnik stage.

Karmic Koala, the latest release of Ubuntu, made it really easy for new installs to use encryption for home directories.  However, for users who upgraded from previous releases (Jaunty Jackalope, Intrepid Ibex, etc), the process to encrypt an existing home directory is not so obvious.  Here’s one way to do it.

You’ll need enough free space for two copies of your entire home directory. If you don’t have that, first move your existing home directory onto external media, and modify these instructions as appropriate.

First, log out of your normal account, which we’ll term myrealusername. Login as root. Create a new, temporary user, using the –encrypt-home flag to adduser.  Use the same password as your current user account.

When you ran ‘adduser’, it created two important directories: /home/.ecryptfs/tmpuser/.Private and /home/.ecryptfs/tmpuser/.ecryptfs. These are both symlinked into a skeleton home directory at /home/tmpuser. I say skeleton here, because the directory so created is only ever visible when ‘tmpuser’ is logged out. Once ‘tmpuser’ is logged in, the contents of /home/.ecryptfs/tmpuser/.Private are decrypted and mounted on top of /home/tmpuser. We want the same thing to happen for myrealusername. Before we put the skeleton directory in place, however, we want to move your existing, unencrypted home aside. That way when you log in with an encrypted home it isn’t hidden and we’re able to copy files into your new, encrypted home.

Now you want to copy these skeleton files to /home/.ecryptfs/myrealusername and change the ownership:

You also need to edit a mount description file which still refers to ‘tmpuser’:

Just for good measure we’ll copy the two informational symlinks from the skeleton directory. These are handy because if they show up when you’re logged in, you know something went wrong and you’re not accessing your encrypted home dir.

At this point, you’re ready to log in as myrealusername. Go ahead. I’ll wait.

Got it? You should have a mostly empty home directory. If you have a file named Access-Your-Private-Data.desktop, something went wrong.

If all’s well, copy your original home directory into the new, encrypted home directory. It’s best to do this as root, on the off chance you have some files in your home dir which you don’t own, but want to preserve exactly. You may be tempted to do this as an ‘mv’ if you’re low on disk space. That should work just the same, but it won’t actually unlink the original until *everything* is copied, meaning you still need enough space for two copies of your home dir. In a pinch you could move one subdirectory at a time, which means you don’t need as much spare space.

Note the trailing slashes on the rsync – those are important.

Bingo! Your home directory is now encrypted. Once you’re satisfied that everything is there, don’t forget to remove your original, unencrypted home. To be tidy you may want to delete the ‘tmpuser’ account too.

Shapefiles and KML are two common geographic formats.  Shapefiles tend to be used by geographic databases like ESRI and PostGIS, while KML originated with Google Earth, back when it was called Keyhole.  KML files are more common for data that is intended primarily to display to users.  They are supported both in Google Earth and in Google Maps, through importing to a My Map.

But for all that both formats are really popular, it’s not easy to go back and forth between the two.  Further, it’s really hard to simplify a KML file that has highly detailed polygons.  This is important because, while Google Earth may be able to handle a large number of points, Google Maps has to execute in the browser and so has tight limits on how many points can be in a polygon.  This can result in a group of polygons being split by Maps across multiple pages so they can be rendered in a reasonable amount of time.

I had some shapefile data I wanted to simplify and display in a My Map. My solution was to bite the bullet and install PostgreSQL and PostGIS.  Both are free software and are available as packages on Ubuntu Intrepid Ibex. Similar directions should apply for other Linux distributions.

First, install PostgreSQL and the PostGIS geographic layer on top of it:


$ sudo apt-get install postgresql-8.3-postgis postgis gdal-bin


Become the postgres user to create the procedural language, initialize the geographic functions, and fill the spatial_ref_sys table:


$ sudo su postgres
postgres$ createlang plpgsql
postgres$ psql -d postgres -f /usr/share/postgresql-8.3-postgis/lwpostgis.sql
postgres$ psql -d postgres -f /usr/share/postgresql-8.3-postgis/spatial_ref_sys.sql
postgres$ createuser username
Shall the new role be a superuser? (y/n) y
CREATE ROLE
postgres$ exit


Import the shapefile into your database. The -d parameter drops the table before creating it, useful if you may be reimporting the data multiple times. If you have a large file you may want to add the -D parameter, which uses the more efficient dump format for faster ingestion. The -s parameter specifies the SRID of your input data, and you should be able to figure it out from the .prj file that came with the shapefile.


$ shp2pgsql -d -s 2877 election_precincts election_precincts_table postgres | psql -d postgres


And dump output into KML.  Note that you can put any criteria you want into the -sql param, so you could restrict the set of features you output.


$ ogr2ogr -f "KML" election_precincts.kml PG:"dbname=postgres" -dsco NameField=precinct -sql "select precinct, transform(simplify(the_geom, 100), 4326) from election_precincts_table"


Note: KML explicitly supports only one SRID: 4326. That refers to WGS84. So in the ogr2ogr command above, you need the transform(…, 4326) call to produce valid lat/longs for KML. Evidently Google Maps has another SRID, 900913, but I haven’t played with that one at all. If you’re using KML as your input to Google Maps, 4326 should be fine.

You can increase or decrease the parameter to simplify(). Higher numbers mean more simplification, lower numbers mean less.

The NameField param is a KML-specific parameter to ogr2ogr which specifies which field from the query should be used as the name for a given KML feature.

Here are some other pages I found useful when working this all out:

PostGIS Wiki : Postgis On Ubuntu
PerryGeo » Converting Shapefiles (and more) to KML
Projected coordinates listed by name
PostGIS Manual
PostGIS Simplify
PostGIS ver. 1.3.1 Quick Guide – Cheatsheet
#2271 (Add built in reprojection support to KML driver) – GDAL – Trac
[postgis] SRID for LAT/LONG
[postgis-users] addgeometrycolumn() does not exist
[FWTools] ogr2ogr shp –> kml conversion failing on some complicated polygons
Generalize Vectors – GEO UTILITIES

I’m looking to start compiling JavaScript for a personal project, so I did a quick roundup of freely available JS compilers / obfuscators / minifiers.

Dojo Shrinksafe – http://dojotoolkit.org/docs/shrinksafe

YUI Compressor – http://www.julienlecomte.net/yuicompressor/

JavaScript Obfuscator by Stunnix – http://www.stunnix.com/prod/jo/

JSMin – http://www.crockford.com/javascript/jsmin.html

Packer- http://dean.edwards.name/packer/

So far I’ve tried Shrinksafe and YUI Compressor.  I couldn’t get Shrinksafe to produce any output, even though it seemed to be parsing my JS since it threw errors when I had something wrong.

YUI compressor ran on my input JS and produced output, but it had a couple of problems.  Backslash-escaped strings get unescaped from the output.  For instance,

alert("two \n lines");

Becomes:

alert("two
 lines");

Also, it seemed to have trouble with "else if (...) {" constructs, turning them into "else{if(...){".

GMail’s Contacts section now has support for a full-featured set of contacts, including phone numbers, addresses, etc.  I tried importing my addressbook from Evolution, but it wasn’t easy.  Step 1: Export your addressbook in vCard format.  Step 2: Tweak the output.GMail doesn’t like plain TEL records, it wants them to always beprefixed by phone., e.g.:phone.TEL;TYPE=CELL,OTHER:(510) 334-3594

Versus

TEL;TYPE=CELL,OTHER:(510) 555-1212Also GMail doesn’t like blank lines between vCard records.Here are a pair of vim commands to fix the above two problems::%s,^TEL,phone.TEL:v/./d

Once you’ve edited the file appropriately you should be able to import it using the Contacts > Import function within GMail.

I have an SSH connection into work, and wanted to be able to access a Windows machine there via rdesktop.  I tried this:

$ ssh -L 3890:windows-machine:3890

$ rdesktop localhost:3890

But the rdesktop command hung for a long time then died.  Some poking around revealed this page: http://www.bluestream.org/Networking/SSHTunnelRDP.htm, which details an occasional problem that occurs when trying to connect to a forwarded port on localhost.  Their suggestion was to bind to a non-localhost interface.  Since their tutorial is for Putty, here’s the equivalent for OpenSSH:

$ ssh -L mymachinename:3890:windows-machine:3890

$ rdesktop mymachinename:3890

Where mymachinename is the actual hostname of your machine, which should in theory resolve to the IP of your ethernet adaptor.

Bitpim is an open source tool to sync data from a variety of CDMA phones. In particular it supports my LG VX-10, or it did until the thing got too beat up to sync.

Now I’m faced with the task of transferring my old contacts to my new phone. Here’s how I went about doing so.

1. Jailbreak your phone.

I borrowed a Mac and ran iJailbreak. I’d be interested to hear if anyone knows of a jailbreak program for Linux.

2. Export your contacts.

In Bitpim, select File > Export > vCards… and save to the default bitpim.vcf. The Dialect section of the export dialog should be “vCard 3.0.”

We have to fix up the output a little bit. Bitpim produces output like:

TEL;TYPE=CELL,PREF:(212) 555-1212

The “,PREF” here confuses Funambol. I haven’t checked whether this is valid per the vCard spec or not, but it’s easy to fix:

$ sed -i s/,PREF// ~/bitpim.vcf

3. Sign up for a free account at my.funambol.com. While you’re there, download the sync app onto your iPhone.

4. Set up Evolution and SyncEvolution.

Evolution is most likely available as part of your distribution. SyncEvolution, you will probably have to download separately from estamos.de.

Now use File > Import… to import bitpim.vcf.

Neither SyncEvolution nor Funambol does a good job syncing deletions of contacts, so you’ll find that in order to delete a contact properly you’ll have to remove it from Evolution, Funambol, and your iPhone. For that reason, I suggest that you go through your contacts now and remove any you don’t wish to keep.

Now you need to set up SynEvolution to talk to Funambol. For this you should follow the Getting Started instructions on estamos.de. It’s pretty straightforward but involves some editing of config files. When you’re done you’ll be able to two-way sync between Evolution and Funabol with:

$ syncevolution funambol

5. Set up the Funambol iPhone app and sync!