Scuttle to del.icio.us

06th of July 2006

For some reason, I wanted to convert my scuttle based bookmark collection, for which I run a local installation of scuttle 0.5.*, to del.icio.us. Fortunately del.icio.us provides an HTTP based API which is verry easy to use, and thus it was verry easy to transfer all of my collected links to del.icio.us.

The following text, explains the steps, that I used to perform my data.

For some reason, I wanted to convert my scuttle based bookmark collection, for which I run a local installation of scuttle 0.5.*, to del.icio.us. Fortunately del.icio.us provides an HTTP based API which is verry easy to use, and thus it was verry easy to transfer all of my collected links to del.icio.us.

The following text, explains the steps, that I used to perform my data.

The first step was to export scuttle’s data into XML. Since I used a verry old version of scuttle (0.5.*) I used the following link and saved the result to my local machine.

The second step was (unfortunately) a bit tedious. Since the encoding in the output file was br0ken for some reason, I decided to manually fix the file's encoding plus its contents. Of course, it is important to save the result using the proper encoding.

The result was a file employing the following structure:

<?xml version=”1.0” encoding=”iso-8859-1”?>
<posts tag="" user="bwolf">
  <post href=”http://ianhenderson.org/megazoomer.html” 
    description=”ianhenderson.org – megazoomer” 
    extended=”Megazoomer makes windows full-screen.
    Just press Command-Enter, and the front-most window 
    grows to fill your entire monitor.”
    hash=”7f872862e6675407f730b8acdbe7dd54” tag=”mac apple”
    time=”2006-07-04T04:07:34Z”/>
  ...
</posts>

Since the del.icio.us API uses different arguments, as the scuttle export file contains, I decided to pre-process the exported file to match the del.icio.us APIs arguments, using a simple XSLT file (which should be processed using a conforming processor like Saxon):

<?xml version=”1.0” encoding=”UTF-8”?>
<xsl:stylesheet version=”1.0” 
    xmlns:xsl=”http://www.w3.org/1999/XSL/Transform”>
  <xsl:output indent="yes"/>
  <xsl:template match="posts">
    <posts><xsl:apply-templates/></posts>
  </xsl:template>
  <xsl:template match="post">
    <post url=”{@href}” description=”{@description}” 
      extended=”{@extended}” tags=”{@tag}” 
      dt=”{@time}”/>
  </xsl:template>
</xsl:stylesheet>

The resulting file, is now ready to be processed using the following python program, which is written in functional style (or at least in a mix of functional and OO programming style, which was forced by the "batteries included in python):

#!/usr/bin/env python
import urllib, sys, time
from urllib import urlencode, urlopen
from xml.dom.minidom import parse, parseString

u = 'https://api.del.icio.us/v1/posts/add'
login, passw = 'del.icio.us-login', 'del.icio.us-passwd'

class Delicious(urllib.FancyURLopener):
  version = '%s del.icio.us/0.1' % login
  def prompt_user_passwd(self, host, realm):
    return (login, passw)

urllib._urlopener = Delicious()

ps = map(lambda p: dict([(a, p.getAttribute(a).encode('utf-8'))
  for a in ['url','description','extended','tags','dt']]),
    parse(sys.argv[1]).getElementsByTagName("post"))

def upload(p):
  print '.'
  xml = parseString(urlopen(u, urlencode(p)).read(8192))
  assert xml.getElementsByTagName('result')[0].getAttribute('code') == 'done'
  time.sleep(1.5)

for p in ps: upload(p);

Finally, the transfer could be performed using the command python-program xml-file. Enjoy ;P

Posted in (software, internet)
1. Comment from abaum [url] 07th of July 2006 at 00:24

Hi.  This is exactly what I need to do but I’m a bit confused.  I can export my Scuttle bookmarks to xml with the appropriate formatting.  How do I then use the xslt style sheet to make the xml file conform to Del.icio.us API format?  Where do I place the code you wrote for XSLT above?  Once that’s figured out, where then do I place the python file on my server?  In the /api dir as well? 

This is a great idea and I’m just confused a bit on how to get it to work.

2. Comment from bwolf [url] 07th of July 2006 at 00:24

Hi abaum, 1st you use for example saxon to transform your export using the stylesheet. The result is a xml document, which in turn can be feed into the python program in my post (this program uses the xml data to post your entries to de.licio.us via the de.licio.us-API). I hope this helps. Feel free to contact me.

Commenting is not available in this weblog entry.