jsha/README

This function is handy if you have some third-party gem that generates writes to stdout or stderr which you wish to suppress. Simple wrap your call to the noisy function in an `stfu’ block. If an exception is thrown, it will be reopened with stdout and stderr pointing to the console again.

  def stfu
    begin
      orig_stderr = $stderr.clone
      orig_stdout = $stdout.clone
      $stderr.reopen File.new('/dev/null', 'w')
      $stdout.reopen File.new('/dev/null', 'w')
      retval = yield
    rescue Exception => e
      $stdout.reopen orig_stdout
      $stderr.reopen orig_stderr
      raise e
    ensure
      $stdout.reopen orig_stdout
      $stderr.reopen orig_stderr
    end
    retval
  end

  require 'some_noisy_gem'
  stfu do
    some_function_that_generates_a_lot_of_cruft_on_stdout
  end

Diagnosing HTTP traffic in Wireshark can be a pain because it is not always clear from the Info column where the traffic is going. All those “GET / HTTP/1.1″ blend together. Fortunately it’s easy to add a column.

Go to Edit -> Preferences -> User Interface -> Columns. Click “+ Add”, and for “Field type” select Custom. The “Field name” box will now be enabled. In it type “http.host”. Click the “New column” text above to set the display name to “HTTP Host.” Hit OK and you are done!

It’s often useful to create self-signed SSL certificates for testing or when you don’t need the authentication that CA signing provides.  I started with Akadia’s handy tutorial on self-signing here: http://www.akadia.com/services/ssh_test_certificate.html.

Then I needed to add a list of subjectAlternateName (SAN) fields.  This is an x509v3 extension that allows a single certificate to be valid for multiple DNS names.  Here’s a shortened version of how to create a self-signed cert using those fields:


echo -e >extensions.cnf "basicConstraints=CA:true\nsubjectAltName=DNS:mysite1.com, DNS:mysite2.com"

openssl genrsa -out server.key 2048 && openssl req -new -key server.key -out server.csr -subj /CN=localhost.twitter.com && openssl x509 -req -days 3650 -extfile extensions.cnf -in server.csr -signkey server.key -out server.crt


Import server.crt into your list of trusted root certs and install server.crt and server.key in your Apache configuration. Your Apache should now be able to serve trusted SSL for the domains you specified, to your browser or any other one that imports your new certificate as a root.

NOTE: Guard your server.key carefully. Because you have just imported this as a CA cert, an attacker who gained control of your server.key could use it to impersonate any server on the web to you.

The documentation on how geotagging works in Youtube is a little slim, so I’m documenting what I’ve learned about it recently.

When you pull a Youtube feed in JSON-C format, e.g. http://gdata.youtube.com/feeds/api/users/greenpeaceusa/uploads?alt=jsonc&v=2&max-results=50&category=oilspilltruth, sometimes you will get geotags like this:

{
  data: {
     ...
     items: [
       {
         id: "s-QQWRdF-5Y"
         ...
         geoCoordinates: {
           latitude: 29.149463653564453
           longitude: -90.47636413574219
         }
       }
       ....

And sometimes, instead of geoCoordinates, you will get location:

          location: "cat island, louisiana"

The difference between these two lies in how the video was geotagged.  If a user geotagged the video by simply typing a location name into the map widget and hitting enter, it will receive a location: attribute. If a user geotagged the video by typing a location name and then dragging the marker, or simply by dragging the marker, then the video will receive a geoCoordinates: attribute. In theory whenever a video has a location: attribute, you should be able to send it through Google’s geocoding service and be confident that you will receive an answer close to what the user intended.  In practice, Google’s geocoding index will occasionally change, and the answer may not be exactly the same as what the user saw when they originally geotagged a video.

How do I geotag a video?

Assuming you already have a video uploaded, go to http://www.youtube.com/my_videos.  Click ‘Edit’ for one of your videos:

In the lower right of the video Edit page, you will see a Date and Map panel:

You can fly to a location by typing its name:

Make sure to hit the “Save changes” button in the upper left.  After doing this, your video will receive a location: attribute.  However, if you want a more precise location, or you want your video to receive the easier-to-plot geoCoordinates attribute, zoom in and drag the pin to the exact location you want:

Notice that the “Map Location” box now contains exact latitude and longitude coordinates.  Hit “Save changes” in the upper left, and your video is now tagged with precise coordinates.  You should see these show up as the geoCoordinates attribute in JSON-C output.

How can I tell if a Youtube video is geotagged?

As far as I can tell, there is no way to figure this out through the user interface on Youtube.com.  I believe the only way to access Youtube geotags is through the API.

I’ve been playing around with OpenStreetMaps recently. I downloaded a subset of the world data set covering San Francisco by using this tool: http://78.46.81.38/.  I loaded the data into a local PostGIS database with osm2pgsql.  Then I spent a couple hours installing the prerequisites for Mapnik and building Mapnik itself, which was the hardest part but still very doable.  Once Mapnik was installed, I was able to use the generate_image.py script to generate a quick map of SF.  Next step is to learn how to build tiles for slippy maps, and then how to customize them.

The process was mostly easy and rewarding.  One funny quirk: osm2pgsql can’t do password authentication to the database, so it’s most convenient to use a passwordless account.  Mapnik, on the other hand, appears to choke if it is given an empty password string, so you need to add a password to the account once you get to the Mapnik stage.

Karmic Koala, the latest release of Ubuntu, made it really easy for new installs to use encryption for home directories.  However, for users who upgraded from previous releases (Jaunty Jackalope, Intrepid Ibex, etc), the process to encrypt an existing home directory is not so obvious.  Here’s one way to do it.

You’ll need enough free space for two copies of your entire home directory. If you don’t have that, first move your existing home directory onto external media, and modify these instructions as appropriate.

First, log out of your normal account, which we’ll term myrealusername. Login as root. Create a new, temporary user, using the –encrypt-home flag to adduser.  Use the same password as your current user account.

When you ran ‘adduser’, it created two important directories: /home/.ecryptfs/tmpuser/.Private and /home/.ecryptfs/tmpuser/.ecryptfs. These are both symlinked into a skeleton home directory at /home/tmpuser. I say skeleton here, because the directory so created is only ever visible when ‘tmpuser’ is logged out. Once ‘tmpuser’ is logged in, the contents of /home/.ecryptfs/tmpuser/.Private are decrypted and mounted on top of /home/tmpuser. We want the same thing to happen for myrealusername. Before we put the skeleton directory in place, however, we want to move your existing, unencrypted home aside. That way when you log in with an encrypted home it isn’t hidden and we’re able to copy files into your new, encrypted home.

Now you want to copy these skeleton files to /home/.ecryptfs/myrealusername and change the ownership:

You also need to edit a mount description file which still refers to ‘tmpuser’:

Just for good measure we’ll copy the two informational symlinks from the skeleton directory. These are handy because if they show up when you’re logged in, you know something went wrong and you’re not accessing your encrypted home dir.

At this point, you’re ready to log in as myrealusername. Go ahead. I’ll wait.

Got it? You should have a mostly empty home directory. If you have a file named Access-Your-Private-Data.desktop, something went wrong.

If all’s well, copy your original home directory into the new, encrypted home directory. It’s best to do this as root, on the off chance you have some files in your home dir which you don’t own, but want to preserve exactly. You may be tempted to do this as an ‘mv’ if you’re low on disk space. That should work just the same, but it won’t actually unlink the original until *everything* is copied, meaning you still need enough space for two copies of your home dir. In a pinch you could move one subdirectory at a time, which means you don’t need as much spare space.

Note the trailing slashes on the rsync – those are important.

Bingo! Your home directory is now encrypted. Once you’re satisfied that everything is there, don’t forget to remove your original, unencrypted home. To be tidy you may want to delete the ‘tmpuser’ account too.

Shapefiles and KML are two common geographic formats.  Shapefiles tend to be used by geographic databases like ESRI and PostGIS, while KML originated with Google Earth, back when it was called Keyhole.  KML files are more common for data that is intended primarily to display to users.  They are supported both in Google Earth and in Google Maps, through importing to a My Map.

But for all that both formats are really popular, it’s not easy to go back and forth between the two.  Further, it’s really hard to simplify a KML file that has highly detailed polygons.  This is important because, while Google Earth may be able to handle a large number of points, Google Maps has to execute in the browser and so has tight limits on how many points can be in a polygon.  This can result in a group of polygons being split by Maps across multiple pages so they can be rendered in a reasonable amount of time.

I had some shapefile data I wanted to simplify and display in a My Map. My solution was to bite the bullet and install PostgreSQL and PostGIS.  Both are free software and are available as packages on Ubuntu Intrepid Ibex. Similar directions should apply for other Linux distributions.

First, install PostgreSQL and the PostGIS geographic layer on top of it:


$ sudo apt-get install postgresql-8.3-postgis postgis gdal-bin


Become the postgres user to create the procedural language, initialize the geographic functions, and fill the spatial_ref_sys table:


$ sudo su postgres
postgres$ createlang plpgsql
postgres$ psql -d postgres -f /usr/share/postgresql-8.3-postgis/lwpostgis.sql
postgres$ psql -d postgres -f /usr/share/postgresql-8.3-postgis/spatial_ref_sys.sql
postgres$ createuser username
Shall the new role be a superuser? (y/n) y
CREATE ROLE
postgres$ exit


Import the shapefile into your database. The -d parameter drops the table before creating it, useful if you may be reimporting the data multiple times. If you have a large file you may want to add the -D parameter, which uses the more efficient dump format for faster ingestion. The -s parameter specifies the SRID of your input data, and you should be able to figure it out from the .prj file that came with the shapefile.


$ shp2pgsql -d -s 2877 election_precincts election_precincts_table postgres | psql -d postgres


And dump output into KML.  Note that you can put any criteria you want into the -sql param, so you could restrict the set of features you output.


$ ogr2ogr -f "KML" election_precincts.kml PG:"dbname=postgres" -dsco NameField=precinct -sql "select precinct, transform(simplify(the_geom, 100), 4326) from election_precincts_table"


Note: KML explicitly supports only one SRID: 4326. That refers to WGS84. So in the ogr2ogr command above, you need the transform(…, 4326) call to produce valid lat/longs for KML. Evidently Google Maps has another SRID, 900913, but I haven’t played with that one at all. If you’re using KML as your input to Google Maps, 4326 should be fine.

You can increase or decrease the parameter to simplify(). Higher numbers mean more simplification, lower numbers mean less.

The NameField param is a KML-specific parameter to ogr2ogr which specifies which field from the query should be used as the name for a given KML feature.

Here are some other pages I found useful when working this all out:

PostGIS Wiki : Postgis On Ubuntu
PerryGeo » Converting Shapefiles (and more) to KML
Projected coordinates listed by name
PostGIS Manual
PostGIS Simplify
PostGIS ver. 1.3.1 Quick Guide – Cheatsheet
#2271 (Add built in reprojection support to KML driver) – GDAL – Trac
[postgis] SRID for LAT/LONG
[postgis-users] addgeometrycolumn() does not exist
[FWTools] ogr2ogr shp –> kml conversion failing on some complicated polygons
Generalize Vectors – GEO UTILITIES

I’m looking to start compiling JavaScript for a personal project, so I did a quick roundup of freely available JS compilers / obfuscators / minifiers.

Dojo Shrinksafe – http://dojotoolkit.org/docs/shrinksafe

YUI Compressor – http://www.julienlecomte.net/yuicompressor/

JavaScript Obfuscator by Stunnix – http://www.stunnix.com/prod/jo/

JSMin – http://www.crockford.com/javascript/jsmin.html

Packer- http://dean.edwards.name/packer/

So far I’ve tried Shrinksafe and YUI Compressor.  I couldn’t get Shrinksafe to produce any output, even though it seemed to be parsing my JS since it threw errors when I had something wrong.

YUI compressor ran on my input JS and produced output, but it had a couple of problems.  Backslash-escaped strings get unescaped from the output.  For instance,

alert("two \n lines");

Becomes:

alert("two
 lines");

Also, it seemed to have trouble with "else if (...) {" constructs, turning them into "else{if(...){".

GMail’s Contacts section now has support for a full-featured set of contacts, including phone numbers, addresses, etc.  I tried importing my addressbook from Evolution, but it wasn’t easy.  Step 1: Export your addressbook in vCard format.  Step 2: Tweak the output.GMail doesn’t like plain TEL records, it wants them to always beprefixed by phone., e.g.:phone.TEL;TYPE=CELL,OTHER:(510) 334-3594

Versus

TEL;TYPE=CELL,OTHER:(510) 555-1212Also GMail doesn’t like blank lines between vCard records.Here are a pair of vim commands to fix the above two problems::%s,^TEL,phone.TEL:v/./d

Once you’ve edited the file appropriately you should be able to import it using the Contacts > Import function within GMail.

I have an SSH connection into work, and wanted to be able to access a Windows machine there via rdesktop.  I tried this:

$ ssh -L 3890:windows-machine:3890

$ rdesktop localhost:3890

But the rdesktop command hung for a long time then died.  Some poking around revealed this page: http://www.bluestream.org/Networking/SSHTunnelRDP.htm, which details an occasional problem that occurs when trying to connect to a forwarded port on localhost.  Their suggestion was to bind to a non-localhost interface.  Since their tutorial is for Putty, here’s the equivalent for OpenSSH:

$ ssh -L mymachinename:3890:windows-machine:3890

$ rdesktop mymachinename:3890

Where mymachinename is the actual hostname of your machine, which should in theory resolve to the IP of your ethernet adaptor.