Friday was fun. A machine was spoofing and attacking another machine, and it took out a couple routers and the internet link. After that was done there were still problems. We ended up finally tracking down the problem with high latency lately. Suffice it to say the HP Procurve 4000s suck for certain tasks. Like mirroring ports under any load. It was losing about 40% of the traffic on a mirrored port. It wasn't noticed, because it was clamped on both sides to about 9Mbps (physical link on one side, packetshaper on other). When sending through floods that the packetshaper was instructed to ignore, it'd be 35-40% loss. Other traffic 0% loss, but a couple hundred ms longer rtt than there should be. Apparently the issue was being handled at layer 2 so nothing on either side ever saw it. Even VoIP traffic worked great - it's amazing how much delay people have become accustomed to on the phone. Anyways, after turning off the packetshaper (last device before the packet loss), physically bypassing it, replacing the ethernet cable, and getting ready to swap out that vlan's ports on the switch with a cheap one to see if it was acting wrong (last step since the traffic already went through that switch in another vlan with no problems), we tried turning off the mirroring. After further analysis of the port counters, the port it was mirrored to had over 10mil tx errors in the time the switch had been up (recently rebooted due to the other issues). Fun, huh? At least it's fixed now.

On Saturday my brother came home with an iPod Mini. I played with it for a few hours, attempting to see if I could get it working with his Windows 98 machine (pending replacement but on hold for now as different schools have different requirements). I could mount it fine, but only with formatting. I tried to load the Windows formatting and software. Unfortunately while I have a few virtual 2000 and XP machines on my Mac, the USB hooks bind to the Mac and the iPod triggers way too many automount hooks in the OS. It simultaneously starts iTunes and starts mounting the drive. Given it attempts a hard mount it then deciding after a delay to give the device to the virtual machine with predictable results. That's all usable if the iPod didn't like to reboot on dismount causing the virtual machine to lose the connection and the process to repeat. And if the OS doesn't grab it within a certain time from connection of the USB connection the iPod will no longer accept an attempt until the connection is removed and added again. For all the plug and play they claim... You also can't just reformat and restore the directory structure - apparently the iPod system is either embedded in the filesystem or differs based on the filesystem. So no HFS+ to FAT32 conversion works either. I finally ended up just setting it up on my mac and telling him to copy his music over and I'd add it. In the process with a couple CDs I got to learn the annoyances of iTunes CD importing (such as renaming tracks while importing won't affect ones that have already started ripping - that one's really annoying). I like just dropping the music player on an ethernet port, looking at its IP, and saying "add these files" and it works.

After I finally just went for the mac exclusive approach on the thing, I decided to test it out and copied a few hundred songs over to test the interface and quality. Sound quality seemed rather good, although the output seemed a little weak. I didn't do any real comparisons and it's been a while but nothing really sticks out in my mind as problematic there. The one thing that really annoyed me was the scroll wheel. It doesn't like me. I could drag my finger around in circles sometimes and it wouldn't detect me. Get away from it and come back and it'd usually work. I think it may be a capacitive style touchpad, which could possibly explain it given the combination of humidity and temperature of the room and my fingers. The multifunction aspect of it annoyed me as well. It has to require a bit of movement so it doesn't sense movement when you're pushing a button. Yet it has to be sensitive enough that large motions don't require many times around to get it. They actually seem to have done a very good job of balancing that, but it still bugs me. I like things that respond instantly and consistently, not with a slight variation. Speaking of which, how do people deal with the slow UI? I pretty much have the same rant with it as with Windows and OS X. I guess people are just used to it and think that's how computers are supposed to act or something. I did have a little fun with the Music Quiz game, although it seemed that game is mainly hard because it's hard to select the right song with that crazy scroll wheel. It'd take two or three attempts to get the cursor over the right song, and hope it didn't move when I took my finger off to hit the select. I got a score of 69060 after 160 songs, but that's probably not that good.

First movie of the weekend was Tangled. It's a rather messed up but interesting story. They explain everything until it makes sense, and then start ripping it apart again. Second movie was Simone. It had good parts and bad parts. It seemed to describe our culture rather well. In other news, Shatner quote of the week - "Let 'em move to Canada. Freeze his balls off." Also an awesome quote from Slashdot. "Wetware too is vulnerable to buffer overflow exploits. Annoy a person for long enough and they'll do what you say just to get you to stop talking." As far as TV goes, this is probably better for them in the long run anyways.

In the work world it's been busy. I sent out notices to a bit over 3500 people on Friday that I am going to delete their account in a couple weeks. I've gotten a decent number of replies, and had the registrar's office fix some dates on courses or sent people to get thesis extensions properly recorded in the system. Most of them are people wanting exceptions though - what else is new. I've also been tracking down load on one of my systems. Some stuff late night was seriously spiking the load. Turns out it was the building collection of requests some of our computers do when they boot, some of them up to a dozen or so http calls to report different things, request data, etc. With them all being database driven, and a few hundred machines rebooting simultaneously... I ended up optimizing a bunch and think I may change how some of the stuff that doesn't need to be as real-time is done. Other than that I've been adding more features elsewhere, which may or may not have adverse effects on system load. It should be interesting to see. Huh. I wrote a lot again.

This week started with a bunch of smaller stuff, and finished with a lot of coding. I think I'm close to done on an effectively complete rework of our knowledegebase system, which will hopefully allow CMS integration and make people happy. Once that's done I just need to figure out how to do automatic correlation of articles with Incidents in our tracking system for likely causes. Should be an interesting next week.

Outside work haven't done much that's exciting. Today I finished up the 3rd disc of Alias Season Three. I think it's better the second time around, but it still seems not nearly as well done as the previous two. For fun stuff, my dad just sent me forward with this content. I thought it was funny.

It was a long week. I did a lot of coding, but I'm making a lot of progress on the IncidentBase. I've converted almost all if not all the lists of Incidents to a new format which allows dynamic resorting and per-user context for stuff like visibility. I think the two places in which it will be found most useful are the creation when pulling all the Incidents for the user (which also now only shows 6 months unless selected) and query (which needs a separate overhaul, but I won't get started on that yet). I finished off the week by dropping in a new My Incidents system which replaces the old switchboard viewing other users' preferences with individual customizable views. The column sorts here are also sticky, and they (with the sticky column sort) can be exported as static links which can be sent to people. I'm really liking the consistent look and functions across everything, although I think I'm temporarily confusing people in how to use all the CSS tags properly across them and differentiate between them (hint, I classed everything extensively, and either classes or IDs should do it in most cases).

Toward the end of the week I felt like I was in a time warp thing. I would go and do a bunch of reworking across half the system, and look back at the clock and it'd only be 20 minutes later. I've also been a lot less tired. I'm thinking it may a combination of it's a lot more efficient to do some of this stuff now, and the fact I've apparently been getting enough sleep (can't fall asleep at night, and wake up earlier than I want to in the morning). Oh well, I guess I won't complain too much.

I decided for fun to pull the stats for IncidentBase. So far this month we've pulled some around 2.8GB from it (compared to like 800MB for www.bethel.edu if I'm reading it right). I'm curious how much that'll go up as now with mod_gzip it's snappier and I can afford to send more detailed data in some cases. Plus working towards dynamic views and interactive stuff... The fun part of the stats was browsers. IE on XP was the number one IE browser. It came in at a significant 4th with like 1/3 of the number 2 browser. #1 was Moz on Windows, and #2 was Safari on OS X. #3 was another Safari version, and numbers 5-7 I think were other Mozillas. Guess that shows what the IT group prefers. :)

Today I decided to be ambitious and go through email. After making a handful of Incidents and looking at some other stuff (hey, I was getting under 100ms latency amazingly) I decided that was enough work stuff and started on a fun project. I guess my definition of fun is probably different than that of most people, as I made a virtual magnetboard. You know those little magnets that you cut up and stick to a refridgerator or whiteboard or any other flat surface with sufficient metalic backing and move them to make sentences? Yea, I made one with DHTML and javascript. Then I decided to copy it to a server rather than just being an HTML file on my hard drive, and since I then had a database I decided to make it sticky so I could leave the page and come back and they'd be in the same position. So it makes a call back to the server whenever the mouse is released with the new position. Routinely each user's board goes and checks for updates, so multiple people share the same board too. I also stored which magnets exist in the same place, and reclassed them which combined with a couple mod_rewrite rules means I can easily create an effectively unlimited number of these magnet boards just by saying what words should be on them. Maybe I'll give people the ability to submit a list of words and it'll make a board for them. This could be almost as much fun as the user css. :)

Finished off the day watching Wicker Park. I didn't like it too much during most of the movie, but I did like how they wrapped it up and the ending. They did it in a much better way than most stories with similar styles. I'm sorta curious about the original now.

Today was nice and relaxing - Holidays are good. I woke up to my phone ringing because the server room was calling to complain about the power being out. Apparently there was a power outage, and the generators kicked in. I never heard about another one, so it must have transitioned off OK (or still running on them). Anyways, after bugging people about them having to work and me sitting around doing nothing for a while, I caught up on news and then went to watch movies.

First movie of the day was Underworld. That movie has one of the most awesome non-DTS soundtracks I've heard. Usually it seems people just don't pay attention to the regular mix, and the DTS is so much better. I sometimes think it's so the DD mix sounds better on cheap speakers, so in this case it could be that this isn't a super mainstream movie. I don't remember if the normal cut had quite the same quality either. I like the extra footage in some areas, but at least one could have been left out. More explanations for stuff, longer fight sequence at the end... Very good movie. Second movie was The Butterfly Effect. I watched the Director's cut, as that's the one I like better. Some of you may remember I watched this last fall and all the guys liked the Director's cut and the girls liked the Theatrical cut. The reasoning was mainly because of the last scene and the possibility of them getting back together. So I'm watching the deleted scenes and the extra two endings that has which are similar to the theatrical release. One of those actually has them getting back together. So if you really liked that way and wanted to see that happen, you should watch the alternates. And I have some dust in my projector apparently. It's not noticable sometimes, but other times is really noticable (especially after you notice it). I think I'll have to figure out how to get that apart and take a look at it.

Got some big stuff out of the way on Friday, which was nice. Network stayed up too, which I'm happy about. I got a new switch in my office too, which will hopefully be nice. It means I can test stuff easier at least. So you know that email I mentioned a couple days ago? Today I got another one, which was much better. The link was even plaintext. I'm amazed - maybe they do have someone who knows what they're doing there. :) So today apparently Internap's Fisher Plaza facility in Seattle lost power. Apparently it's not the first time either. Word is they may have been doing construction on one floor. Oops. In other outage news, Verizon just had a second major cable cut occurance in a week. Apparently each time has been multiple cables in diverse locations too. Sounds like someone's upset with them for some reason. And speaking of people being upset with other people, sometimes there are good reasons. You'd think they'd do some basic sanity checking... Tonight went well. Nick and I went to go look at TVs, but he didn't buy anything. He'll probably update his site though... After that went home, and my brother came home and had a friend over and we watched Windtalkers. Good movie. Now I'm gonna go try to get on a decent sleep schedule for the weekend.

Beep. Beep. Beep. Beep.

So went my phone as I was trying to enjoy some Sesame Beef at dinner with Nick and JoeBuck. Nick and I fixed the network today, as we had some trouble with bottlenecks due to backups. Of course about 45 minutes after everything was up and working, everything fell apart. After dinner and Dan trying to talk us out of going back to work because of the cold, we started poking around to figure out the problem. Around 10:30 everything was stablized. The cause appears to be one of two things. Either the move confused the switches (possible, they had inconsistent data on a couple things), or scanning of large blocks of unused IP space. Both were fixed, with network services telling the switches to forget everything about those things they were confused on and relearn, and with me adding 5,119 IP addresses (yep, eth0 was the machine, I got up to eth0:5119) to one of my machines and watching the traffic and adding firewall rules to the resnet router until the traffic slowed down to reasonable. I feel bad for anyone who sticks a Windows machine in there - it looks like a virus pit right now. One machine was doing some other type of scanning, and it's totally off the network now too. Hopefully one of those things fixed it, we'll see.

I'm tired. I think part of that has to do with getting done working a 14 hour day, with very little of it sitting. In fact it was mostly standing, with large portions of bending over, squatting, kneeling, and lifting stuff. Tuesday wasn't too exciting. I slept in quite a bit, and read stuff online, followed by watching movies. I watched .com for Murder, which could have been better, but could have been worse. Sorta interesting in how other people probably see online stuff... Then Bethel's Internet provider apparently had issues, which also seemed to take out other schools and the Star Trib. It got resolved pretty quickly, I wonder which one they cared most about. :) I was impressed that my phone resynced within seconds of it coming back up. Pretty good... Then I watched The Bounce Supremacy, which is a good movie. I stayed up later than I should have watching extras.

Wednesday I overslept and woke up at 7. Nick did too though, so we didn't care too much. We both needed breakfast and caffeine to get us going though. We started work a bit before 8. The first server room was sorta annoying, since we had to work around boxes that needed to stay up. Of course this is in the process of repatching all the network stuff and putting in new power strips. The power strips are fancy, and measure volts, amps, va, watts, power factor, kilowatt hours, and power on time, and can alert when there is too much draw. Yea, they're geeky, but they will hopefully help power management so we can keep circuits evenly balanced. We managed to do it all though, and I only had all core services down for maybe 10-15 minutes. It took me around an hour to get blackboard back up and running though (after I discovered it didn't come back cleanly). That place is a ton cleaner though, and now we just need to label some stuff and clean up some serial console lines. We probably pulled out over half the length of patch cables in the room though. We ended up finishing that room at around 2, and since schdav arrived we dragged him along to lunch at Culvers.

We got back from lunch around 3, and moved to the other server room. This one was easier since there was one server in our racks that we needed to keep up as much as possible. So it got its cables moved to the side, and moved directly to the outlets on the wall. We then proceeded to rip out every patch cable and power cable in the room. They all went on the floor, later arranged into piles. It felt like walking on some really weird carpet... Then we pulled out switches, put them in new places, repatched serial lines, moved servers to make better use of space, and mounted new fancy power strips everywhere. That room looks like a new place. Then Nick and I spent a couple hours hooking up power cables, repatching network stuff, and routing lines to be as clean and manageable as possible. We went from a room full of mostly 16ft patch cables to mostly 5 and 7ft cables. Yea, the big ones really weren't needed for most things. Even though I'm sore and tired, I'm glad it's done. It has been needed for a long time. Now to go get some sleep...

Monday was bad. First, it was a Monday. Lunch was good though, Ross and I went to C1. For some reason the rest of the day just seemed long. Then I went home. At 10:20 or so, the server room called me complaining the power was out. I also got calls from other people asking about it. A few minutes later the server room called again, this time complaining about both the power and the temperature. About 10:50 I called and it had hit 89 degrees in the room, and the power had been out for 30 minutes, so it was time to go in and start shutting stuff down. I met my boss and a network services guy there, and the fun started. 3 of us jammed in a room that has so much stuff it's a pain for one person to move around in, it's dark, there's only the server and switch LEDs and a trouble lamp for light, and it's over 90 degrees, and we're all moving around shutting down and turning off different machines while we get even hotter air blowing at us out the backs of them. And then just as we get everything down, the power kicks back in. So we spent the next hour and a half getting everything back up and fixing some glitches and figuring out what can wait until morning, and then home, which was listed in the previous entry.

Tuesday was better. It was Futurama day, since the Simpsons didn't show up. Still good though. Some goofy issues from the night before, some other things. Little random things pop up... Oh well. After work was good, got dinner with the family at Olive Garden, which is hard to argue with. I'm still debating on the Ogo thing. It's a handy little gadget. The whole doesn't work at work is rather annoying though. They need to put GSM on 850 around here... We'll see, still a few weeks to go...

Wow, parking was really bad at Bethel on Monday. I think it was Campus Visit weekend again, since everyone seemed to be talking about perspective students. I'm still not sure why studying perspective is so popular, what department it actually falls under (Art? Philosophy? BITS?), or why the heck they're visiting Bethel if they're looking for a good program that deals with perspective. The day was also refered to as a "computer snow day" since the ERP system was down. I don't know about that, since I seemed to have plenty to do telling people it wasn't my system that was broken and fixing the stuff that it broke. That was the excitement for the day though. I did some testing for other stuff though, to basically send people nastygrams.

Tuesday I observed and commented on a large discussion, and sorta disproved a possible solution. Made IB changes and clarified things, and corrected other mistakes. Also found out when I get to go fight a half-dozen people in a meeting. I do have allies and likely allies, but unfortunately they won't really be at the meeting. Wednesday I discussed more backend architecture changes, fixed a sync script that the network caused to mess up AD, cleaned up old account junk I noticed in fixing what the script did, and other miscellaneous stuff. I've been seeing a bunch of problems that have easy fixes it's just people haven't been even making a halfway decent attempt at troubleshooting ("user can't log in to x" doesn't help much). They're relatively simple ones and I know the solution immediately after looking at their account, but they could have also been fixed on the first call with the user by 2 minutes of going through a troubleshooting guide. Oh well, such is life.