a spider for Paul Smith’s list

One
of the best collections of links to homepages of people working in
non-commutative algebra and/or geometry is maintained by Paul Smith. At regular intervals I use it to check
up on some people, usually in vain as nobody seems to update their
homepage… So, today I wrote a simple spider to check for updates in
this list. The idea is simple : it tries to get the link (and when this
fails it reports that the link seems to be broken), it saves a text-copy
of the page (using lynx) on disc which it will check on a future
check-up for changes with diff. Btw. for OS X-people I got
lynx from the Fink Project. It then collects all data (broken
links, time of last visit and time of last change and recent updates) in
RSS-feeds for which an HTML-version is maintained at the geoMetry-site, again
using server side includes. If you see a 1970-date this means that I
have never detected a change since I let this spider loose (today).
Also, the list of pages is not alphabetic, even to me it is a surprise
how the next list will look. As I check for changes with diff the
claimed number of changed lines is by far accurate (the total of lines
from the first change made to the end of the file might be a better
approximation of reality… I will change this soon).
Clearly,
all of this is still experimental so please give me feedback if you
notice something wrong with these lists. Also I plan to extend this list
substantially over the next weeks (for example, Paul Smith himself is
not present in his own list…). So, if you want your pages to be
included, let me know at lieven.lebruyn@ua.ac.be.
For those on Paul\’s list, if you looked at your log-files today
you may have noticed a lot of traffic from www.matrix.ua.ac.be as
I was testing the script. I\’ll keep my further visits down to once a
day, at most…