Comments changes again

Two changes, that only coincidentally happened at the same time.

First change is I added stickiness to the name and URL fields for comments. Those who use their own computers and don't obsessively clear cookies generally won't have to retype them. There's also a preference related to ads available if they show up, but they don't for recent posts anyways...

Second is I removed the xbl check for comment submission. It has blocked several thousand junk comments over the past few months, but about a third of the junk gets past it anyways. The next check isn't really any more resource intensive, and it will probably catch them too. Because it's probably not needed, and it hit a false positive today (unless people are putting odd names, the first in almost 8 months since I started logging failures). It was a nice check to have since it's hard to avoid (a smarter script won't help), but hopefully not necessary. If I see spam, I may put it back temporarily and devise another workaround.

On an unrelated note, sorry for the lack of posts lately. I've been busy with other things, and trying to keep my sanity by eliminating extra things. Hope things are going well for everyone.

And the spambots adapt...

So I was enjoying my relaxing Thanksgiving (well, not really, but that's another story), and I get emails about spam comments. That's the first in a year or more, so I glance at the logs to see how it evaded the setup. It appears it followed a link from googleblog, and was acting as a fully compliant browser (executing javascript, etc.). So either it was a manual attempt, or a really advanced script. To make the rest of my day easier, I implemented an XBL check (after making sure it would have helped in this case), which is something I'd seen as an interesting idea to prevent spam. As I didn't want to bother with Net::DNS for such a simple check that doesn't run often, this should be pretty portable in case anyone else uses perl and wants to try it.

if ($ENV{'REMOTE_ADDR'} =~ m/^(\d+)\.(\d+)\.(\d+)\.(\d+)$/) {
my $xblresult = qx!/usr/bin/host $4.$3.$2.$1.xbl.spamhaus.org!;
if ($xblresult =~ m/\b127\.0\.0\.\d+\b/) {
print "Content-type: text/plain\n\n";
print "Rejected due to xbl\n";
exit;
}
}

New comments feature

I've added a new system for nested comments. I've been meaning to do it for a while, but just finally got around to it. So poke at it a bit, and let me know if you find something that breaks.

Site updates

So if you ever visit my site (as opposed to just reading RSS), you've probably been noticing things changing lately. Ever since the RSS updates I've been chipping away at other things. The titles are one, which I've made show up everywhere now. Due to the way the site is built it's not available when it prints the normal header, so there's a script to put it up there. On navigation through main and category history it will also show position in the title bar. All URLs to entries changed. I have a bit of code in the new RSS feeds to prevent them all from flagging as new entries there, but if you visit it'll convert. The new format should be cleaner for linking, indexing, and stats (query strings confuse things).

Comments have a few changes. Assuming one's browser is capable of it, you can add them from the same page as viewing. This fixes both the issue of not seeing other comments and prevents loading another page. In case you're wondering, the reason the form isn't there directly is so the page itself can be indexed, but the comment form isn't indexed. That's important to keep comment spam away. Also, you can now include a URL with your comment. I'm planning on making name and URL saveable, but it's late and comcast sucks (my connection to the net keeps cutting out) so that's for another day. I plan to implement that with another interesting feature.

Categories are also there. They exist for new entries, and are still filling in for old ones. There's not a preset list, which means it's mostly for finding related entries. On a category page there's also links to my del.icio.us links (if any) for that category and a Technorati search. I'll probably have more metadata in some of them eventually too. Note up to the 5 most popular categories for an entry are listed, listed in order of declining popularity (for this defined by how many entries have it, not views). So the list may change/reorder over time as entries are added to the site. Note that categories not listed both due to not being popular enough and being flagged as hidden will still show up in a category view, so that's why you may see stuff that you don't see as being in a category. Each category also has entry and comment RSS feeds (/entries.rss and /comments.rss) if you're interested, but I haven't figured out how to best get them in the headers for them (same issue as titles).

I've started posting links on del.icio.us more than here, unless I have some extra comments on them. Currently there's just a link to it on the left ("My del.icio.us"), and that has RSS too if you want. I'm curious if that's OK, or if I should aggregate them and post some or all of them back hear (maybe daily)? Leave a comment on that one.

You can see links to blogs I read pretty regularly under "My Reads" over on the left as well. As noted at the top, let me know if you don't want to appear there. I'll probably convert it from a javascript include to plain HTML at some point just to make it cleaner.

The most popular entries and categories are now also linked under Most Popular on the left as well. Probably not useful to regular readers, but it's there. It pulls the views and rebuilds the listing at the same time it checks whether sitemap updates and external pings are needed (about hourly). I think sometimes odd old stuff is gonna pop up in there...

I think the other things are the email updates (another probably gonna be unused thing) and the geo-map (at the bottom, somewhat interesting). They don't really need any explanation though. Enjoy poking around with or ignoring any of the other stuff as you see fit. Let me know if something is broken or doesn't seem to act how you expect.

RSS and site updates

Inspired by JL! a while back, I finally got around to updating my RSS feeds. I split into two feeds, one for the main entries here, and the second for the comments. Both are also not just top x, but incorporate a time-based criteria as well. So if I stop updating (or people stop commenting), my feed won't become like some certain ones where old entries pop back up in the RSS readers since they're so old. Yea, that's working with bad client design (expire based on fetch date rather than last seen), but it seems to be a common bad client design.

Anyways, the feeds are in the source code of the main index. For those too lazy to look, they're Entries and Comments.

As I already had code to handle scheduled but action-based static files updates in place for sitemaps, I used it for this too. The feeds are also now static files, making readers that pay attention to last update times at the HTTP level more efficient. This also means that there may be a few minutes delay before entries or comments appear in the feeds. If you're pulling them at any sane interval (>= 15 minutes), you won't notice. You may have also already noticed the other new addition. If you haven't you apparently don't use RSS.