I was going to set up a Google news scraper for my personal start page, so that I can get just the headlines, and just the topics that interest me (the sports section is a waste of screen space for me). I ran into a strange 403 Forbidden accessing it from python, though; it turns out that Google blocks access by User-Agent, to discourage exactly what I was trying.

It took about 30 seconds to find code to set the User-Agent in python's urllib class... Ironically, I found it using Google itself. However, after reading about how Google bans IPs that are caught doing any kind of automated queries, I'm afraid to use my workaround. Assuming they are using some sort of semi-decent AI to look for automation, regenerating my start page every hour of every day is likely to be noticed, and I would be lynched if Google banned CWRU. So I guess that will have to wait until I have a server where I can build the pages dynamically on request, instead of at set intervals. It's irritating, since this is definitely 'fair-use', since it's entirely for my own use, but I guess I understand the need for something like this.

The scary thing is, it seems like there is tremendous capacity for abuse in this draconian enforcement. What's to stop me from picking a victim IP address or domain, and sending tons of automated queries with a spoofed IP address? Bam, instant Google ban, quite possibly for the whole domain. In addition to random acts of evil, this could have really nasty corporate sabotage implications:

1. Step 1: Open a local ISP
2. Step 2: Pick a competing ISP, and ensure that Google bans their entire subnet
3. Step 3: Profit! Watch the customers leave the ISP for one where they can actually perform Google searches

I'm beginning to understand why some people fear Google's power.

Category: Geek

Comments (0)

I've recently been getting the feeling of drifting away from all my old friends. It's hard to know how much is the distance, and how much the changes in our lives. It's depressing to realize how little I know about the last few years of many people's lives. Which makes it harder to talk, since there isn't that same common base of experiences/knowledge that I always took for granted when I still lived in the same town as all of my friends.

The real problem, I think, is that even with all of the advances in ease of communication that the internet has brought to us, I still haven't found a good virtual analog to hanging out.

As one of my now-departed college friends so insightfully pointed out: even if we could play Apples to Apples online, we'd still need virtual M&Ms.

Category: Life

Comments (0)

So one of our phone jacks stopped working recently, and I messed with it for a while looking for the problem. Once I had narrowed it down to the box where it stopped working, and tried the old engineer's trick of taking it apart and putting it back together (to no avail), I was resigned to re-running the line to it. But since that would have been a big pain, I wanted to make sure it was really the wire, since it still looked fine.

And so, drawing on the resources and skills gained from a childhood of tinkering with simple circuits, I built a simple circuit tester from a battery and an LED scavenged from a dead cordless phone that I didn't have any use for but couldn't quite bring myself to get rid of (because you never know when you'll need some parts).

So I was quickly able to ascertain that the line from the last working box did work, and instead focus on the box itself (which I had mistakenly believed was not the problem once I'd unscrewed all the connectors and reattached them). A few minutes later, using only my pocket knife, I had repaired it, and all was well again. All that without ever taking circuits!

Maybe I should talk to the university about getting an honorary EE degree at the next commencement ;)

Category: Geek

Comments (1)

Graduation came off with only minor hitches, and I'm now the proud owner of a Bachelor's of Science. It's funny, because for years that's all I wanted: get my diploma, get out, get a job. But in the last year or so my views have changed so much, so that now I'm in the midst of a Master's thesis, glad not to be leaving campus yet, and considering for the first time the idea of going right into a PhD program instead of the work force.

It took me all of high school, and most of college, but I've finally learned something truly useful and important. I've learned to stop thinking about education in terms of the light at the end of the tunnel. I finally shone my flashlight around a little, and found that the tunnel has quite a lot to offer. I'm not so sure anymore that I want to leave it at all.

Category: School

Comments (0)

I'm now a better geek than I was yesterday. I've started reading Secrets & Lies, and it finally made me feel guilty enough to overcome my innate laziness enough to mess with ipfw. Granted, I used Google and don't fully understand the ipfw config script I'm now using, but I do have a better firewall than the one built in to OS 10.2. And more importantly, I understand a lot of it, and how it's actually invoked and built. As a fun bonus, now I log denied requests, so I can see anyone crazy enough to try to attack a machine with a dial-in IP that lasts at most an hour.

But most important of all, I won't feel like such a weenie when people talk about their firewalls. And I'll be safe from... uh... all those OS X exploits you hear abo-

Well that's cool. As I write this, another CWRU dial-up user scanned me for Windows file shares (port 137). Not exactly earth-shattering, but I feel that much more aware of what the heck is going on around me.

Category: Geek

Comments (0)

It's neat how two cool natural phenomena can occasionally combine to become even cooler. Today, while walking home from work, a thunder storm had rolled in enough to cover all of the sky overhead, but not enough to block the western horizon. So all the buildings around me were lit up a bright orange-red by the setting sun, against a backdrop of steel clouds lit by continuous flashes of lightning. Quite the view.

On a related note, I'm intrigued by the fact that I can see lightning that looks like it's above and all around me, but not hear thunder until several seconds later, as if it's far off.

What can I say... I'm a sucker for thunder storms.

Category: Random

Comments (0)

Whoever is responsible for making ad-ware that installs itself by pretending to be something else, has no automatic uninstall, and thwarts attempts at manual uninstall by hiding in invisible folders or by giving itself a random name and squeezing in with similarly named files that are necessary for your system to run, and which continuously bombards you with crap while at the same time downloading and installing more ad-ware without asking, should be be hung up by their thumbs until they repent their sins. I have a degree in computer science, access to good online resources for dealing with scumware, and it still took me three hours to (maybe) clean one computer.

Grad students who install this stuff on semi-public computers are right up there on my thumb-hanging list.

Category: Geek

Comments (3)

O is a good example of "dark" done right. It stands to reason that Daredevil wouldn't have sucked if:

• The producers/directors had been different
• The script had been based on a work of Shakespeare
• Julia Stiles had been cast as Electra

Ok, so that last one might not have helped the film much... but on the other hand, more movies with Julia Stiles can only be an improvement.

Category: A & E

Comments (0)

At this moment, I can think of no crueler ordeal for good people than to have the whole future of another person given into their hands, with the imperative to Decide.

Who are any of us to set the whole course of another person's life in a single moment? What is an hour's deliberation, even a day's deliberation, in the attempt to understand any person?

Category: Society

Comments (0)

I am totally, 100% done with classes at CWRU (except thesis, but that's not really a class). So now I have a year to hang around campus while working, being one of those irritating people who has no homework, no finals, and no evil professors.

I plan to enjoy it immensely.

Category: School

Comments (0)

Thunder is a totally different beast out here than it was at home. In Oregon, thunder is a low rumbling. A big thunder storm means the rumbling is louder. Here, thunder sounds like angry gods warming up by snapping huge trees in half, then moving on to breaking handfuls of trees at a time, then doing more of the same, but with a microphone and an amp set to 11.

The idea of thunder as the sounds made by gods makes a whole lot more sense to me now than it did in Oregon.

Category: Random

Comments (1)

I realized today while I was driving down Cedar, with the towers directly ahead off in the distance, that even after 4 years here I still haven't experienced the big-city part of Cleveland except for occasionally driving through/past. I don't really think anything of it when I see the towers, even though I grew up in a city with no skyline.

The question is, am I blasé because of TV/movies, which shove skylines at us whenever possible, or am a still a country bumpkin who doesn't really believe in the big buildings except as painted scenery?

I think it's the second; big land features impress me, but I don't think anything when I see a big building. I think the scale of really tall buildings is one of those things that hasn't actually sunk in yet.

Category: Random

Comments (2)

I was caught unprepared at campus by a stealth thunderstorm this evening, and was completely drenched at the end of my 15 minute walk home. It was warm out, though, and it wasn't exactly pouring, so it was actually quite nice.

I guess there's something about walking through the rain and arriving home soaked that makes me think fondly of Oregon.

Category: Life

Comments (0)