In A Row

I had this awesome idea, evolved from my primary use for twitter -- finding interesting links that others tweet. I wrote a script to watch the public timeline and grab URLs, a script to resolve any tinyurl or is.gd or other links to their final destination, and a script to generate reports.

Then I did some googling. And found twitt(url)y where they aleady do this, with more features. ;)

Note to self: Search first, then build.

Below are some results from a limited running of my aborted project. I did learn about twitter users that simply spam their URL all day long, and thus the results are number of users that tweeted the link in the time period, not just number of times the link appeared.

Past 3 Hours
2       http://www.nicovideo.jp/watch/sm3643812
2       http://tinyurl.com/6rbvlk
2       http://www.expansion.com/edicion/exp/mercados/wall_street/es/desarrollo/1134736.html
2       http://www.floort.com/show.php/2759
2       http://www.lanacion.cl//prontus_noticias_v2/site/artic/20080613/pags/20080613083112.html
1       http://weather.livedoor.com/area/13/63.html
1       http://www.digitalspy.co.uk/spywire/
1       http://radiopopbitch.com
----------------
Past 6 Hours
2       http://www.thetimesherald.com/apps/pbcs.dll/article?AID=/20080613/NEWS01/806130302/1002
2       http://best-forex-broker.wetpaint.com/
2       http://blog.excite.co.jp/shokotan/2859817/
2       http://gribao.com/s/2325/
2       http://tinyurl.com/5jcyhu]
2       http://tinyurl.com/6rbvlk
2       http://twittercounter.com/
2       http://www.ecodazoo.com/
2       http://www.expansion.com/edicion/exp/mercados/wall_street/es/desarrollo/1134736.html
2       http://www.floort.com/show.php/2759
----------------
Past 12 Hours
12      http://traileraddict.com/trailer/transporter-3/french-teaser
8       http://pownce.com/jooanfossi/notes/2456076/
6       http://pownce.com/andi808/notes/2456075/
6       http://www.flickr.com/photos/anonymonk/2564362689/
4       http://pownce.com/accounts/login/?next=/sbflorido/notes/2449170/
4       http://www.flickr.com/photos/anonymonk/2564362689/:
2       http://www.dormiu.com.br/publieditorial/ultima-chamada-para-pequim-1555/
2       http://www.ecodazoo.com/
2       http://www.expansion.com/edicion/exp/mercados/wall_street/es/desarrollo/1134736.html
2       http://www.facebook.com/common/browser.php

Version 0.9.2 of AutoBlogger is now available.

This is the tiniest of bug fix releases. Now, whenever new photos are processed, all pages are rebuilt. This keeps the archive pages from becoming stale.

AutoBlogger Site: http://www.inarow.net/entries/projects/one_evening/autoblogger/
Grab it: Download Tarball or Anonymous SVN (svn co svn://inarow.net/trunk/autoblogger)

Version 0.9.1 of AutoBlogger is now available.

This includes two changes, one bug fix and one new feature:

• --from=aregexhere allows for simplistic "security" or running multiple blogs from one email account
• The contents of the body are now handled more intelligently. Formerly, they were doing something screwy with newlines in messages (using stringify() on the mime body, as opposed to open()).
  It now handles line endings in a way that is intuitive to me: one new line is treated as white space, whereas two newlines in a row are replaced with <br /><br />

AutoBlogger Site: http://www.inarow.net/entries/projects/one_evening/autoblogger/
Grab it: Download Tarball or Anonymous SVN (svn co svn://inarow.net/trunk/autoblogger)

Version 0.9 of AutoBlogger is now available.

This is mainly some cleanup and addition of licensing info ("oh boy!").

It does include one new feature which I've found quite useful. the ability to pass a date in the body of an email, and use that as the publish date. (I plan to use this for importing my old blogspot blog.)

AutoBlogger Intro: http://www.inarow.net/entries/projects/one_evening/autoblogger/autoblogger.html
Grab it: Download Tarball or Anonymous SVN (svn co svn://inarow.net/trunk/autoblogger)

Goal: Live up to my photoblog's name, Proudestfather, by updating it with adorable pictures on a regular basis.
Plan: Whip up a quick perl script which will take emails from an email account, parse, create thumbnails, and generate static HTML.
Results: Download Tarball -- Anonymous SVN (svn co svn://inarow.net/trunk/autoblogger) -- Proudest Father

With this in mind, I decided to implement this project in perl, for several reasons:
* Familiarity with several CPAN modules for handling email
* A desire to dig further into Template-Toolkit, after playing around with catalyst.
* An expectation that most of the project would be manipulating strings.

With this in mind, I set out to complete this project in an evening...

 5:30	Start
 5:35	README and templates/index.tpl created
 6:00	AutoBlogger::Post.pm created
		This involved learning some basics about perl objects.
		Experimented with Class::Std and h2xs to make a Module
		Went with a very simple class

 6:10	Made autoblogger.pl and it takes arguments
		Found GetOpt::Long and applied it
		Verifies that pop credentials are passed
		Verifies that you give it a publish and storage directory that exist
		
 6:15	Mail::POP3Client chosen and installed		
		Net::Pop3 doesn't seem to do SSL

Coffee

 7:10	Pop Harness in place, need SSL
		Done putzing with IO::Socket::SSL, took a while to make CPAN install it properly
		Throw in a debug_tell() sub that takes a message and optional importance

Dinner

 8:15	Start framework for parsing mails
		Add --rebuild and --nopop to support testing
		Man, getopts makes that easy

 8:50	Ok, now I have lots of command line args, write a usage()
		Currently, the script:
			logs in to pop
			downloads emails	
			parses them out to find date, text and image

		Time to refactor.
			Goals:
				Raw MIME message archived
				Ability to just rebuild index, a single month, or all
				Don't recreate images when rebuilding, unless size changes	
			Plan:
				Break storage/ into months (storage/200712/)
				Put messages in their own folder
						storage/YYYYmm/dd_hhmmss/
							raw.txt
							MIME::Parser files
		
 9:50	Ok, that's rewritten and tested.  
		Onto writing out an index.

10:40	Rewrote it so that the AutoBlogger::Post class takes the storage directory for 
			the email in the constructor, and builds the object around this.
		Now to have it ensure that the thumbnails are there.

Snack time.  Hmm, pretzels.	 Housework, exciting!

12:15	Start and finish thumbnailing, make the size optional.  This goes quickly with GD.
		File::Copy::Recursive is needed to move the MIME::Parser files into their target 
			storage directory (mainly because I happen to have /home and my docroot on 
			different partitions)

12:40	Tweaks, debuging... w00t!  index.html is there.		

Ok, time for some Kingdom of Loathing turns.  (http://www.kingdomofloathing.com/, give it a try)

 1:30	Back
 
 1:40	Refactor out the index into a do_file
 2:00	Apply that to the archives
 2:50	Test, tweak template, site up with 2 downloaded posts, index and archive!

At this point, the project is working and ready for a beta release! Quite an evening, but good results. I'm sure I'll push out updates. I want to support passing in a date in the body of the message, for importing older posts. I did take the time to create some usage details.

autoblogger.pl [args] path-to-storage path-to-published-site
-v [--verbose]      Verbose mode
-h [--host=]*       Pop Host
-u [--user=]*       Pop User
-p [--pass=]*       Pop Password
-s [--ssl]          Use SSL
-n [--nopop]        Don't get newmessages
-r [--rebuild=]     Rebuild output index,200708 or index or 200708 or all
-i [--indexsize=]   How many posts on the index
-e [--extension=]   Extension for output html (default html)
   [--tmp=]         tmp directory (defaults to /tmp
   [--thumbsize=]   thumbnail size (defaults to 150x150)
   [--fullsize=]    full image size (defaults to 800x800)
   * required

Often, I'll sit down to solve a fairly simple problem, and whip out a software solution. These projects are varied and sundry:

• Cross Implementation MySQL Library for Scheme
• Brute Force Chess Analyzer in Ruby
• Random Tools for Dungeons and Dragons
• Plug-ins and Modifications for assorted existing software

Normally, these projects get completed, put into service and forgotten. Occasionally, I get ambitious and throw them into version control, to save myself later pain.

One Evening Projects will be a regular feature here, where I take some extra time to document, describe and package these projects. Who knows, maybe somebody else will actually benefit. Most of the time these projects are also learning experiences, and I'll share some of that education as well.

You can browse these projects here