Amazon Web Services, perl and building applicatio​ns

Today I had nothing going on, so I went to the office and tried to build an application in perl to interact with Amazon.

Let's say that again, but try to be interesting:

In my copious spare time, I've been teaching myself to program by writing a script that will automatically buy me random things from Amazon.

I'm not being very original here: I'm taking as inspiration this XKCD cartoon, and this post by Dustin Curtis, which itself was (unknowingly?) standing on the shoulders of this guy, who'd already implemented a bot to buy random junk from an New Zealand ebay-a-like.

This presented an opportunity for me to learn a bit more perl. I'm dealing with enormous data files in my new job, and could often do with a clean way to select only the relevant 100,000 rows of data from all 4.8 million rows, and because I've not got a linux machine handy, and because the Windows equivalents of sed/grep et al are hard to use and not very well documented, I figured perl might be the easy option. Of course, if perl is the easy option, that might be a sign of a fundamental problem, but we'll overlook that for now.
Perl doesn't come as standard with Windows (boo!). I installed Activeperl. I spent an hour trying to make perl run, or to install modules (there's a Perl Package Manager, but it doesn't allow you to download every single perl module in the world, and occasionally this becomes frustrating), or to do anything, and eventually gave up. The default setting for Activeperl doesn't let you go to the command line and run "perl" to get it going, which isn't helpful. Yes, I'm sure if I spent all day reading docs I might figure out how to do this, but I didn't want to spend all day working out how to run the software I'd just installed. So I uninstalled it, and installed Strawberry Perl instead.

With Strawberry Perl, you do get a link on your start menu to run it, and typing perl on the command line also works, which is jolly satisfying. So use it.

Anyone on Linux can stop laughing and jeering now; we've got perl installed.

Now we want to do something. Not just run a 'hello world' program like that's the end goal of my day. I've got an Amazon Web Service to connect to. So I go to Amazon, download the pdf for getting started with AWS, copy the code, put it into a file called amazon.pl and try to run it.

And it rolls over and farts at me because the right modules aren't installed. No problem now: there's a helpful list of what to install; four commands later, my environment is set up. Then it's a quick hop and a skip to associates.amazon.com to find my user keys, stuff them into the script in the right places, and off we go.

Well, sort of. Now we can connect to Amazon, and then get another vaguely opaque message back that tells me I need to include a timestamp, and after half an hour of battering myself trying to get perl to insert a timestamp into my request, I find there's another example perl script that does this for you. It does this as an example of how to work with AWS after they changed some protocols, which makes me wonder why they didn't update the other example document to make it clear there as well.

Never mind. After all this frustration, I've now got a script that will send a request to Amazon to get a list of stuff, and a bit more reading gives up the details of ItemSearch, which clues me in on how to search for different products, and start to do clever things that you can't do easily on the site itself, like restrict yourself to certain price ranges, although it's also strange that by default you'll get a list of products and not the prices. Oh well, whatever.

So once I find the heavy-duty documentation, I spend a happy hour playing with this. Now I can start getting lists of things to buy, and tinkering with that list. There's lot of external logic to build: you don't want the automated random buying tool to keep buying the same thing for you, and you need to control how much it can spend, but we'll get to that in the future.

I look through the documentation, and there are some nice parts where it tells you exactly how to make a shopping cart and put items into it, and send that off to the checkout process, and - oh, hang on. There's no way in the API to actually tell Amazon to process a purchase. Even if you've already given them your credit card details and signed up for 1-click, it's just not possible. You can't do it from a program, you have to go and click on the buy it button and then walk through the buying process.

That's quite a good thing, in general. If it wasn't like that, I could build a link and get people to click on it, and as long as they were already logged on to their Amazon account, then blammo! they'd order whatever I wanted them to buy. But there's ways to fix for that, which Amazon don't want (like the OAuth that the bieh.net implementation used), so I'm out of luck for now at completing my script.

Well, sort of. It could still be done. I could use a Windows tool like Macro Scheduler to run my perl script, then open a Internet Explorer window and click on the right buttons, but having to vomit data from the maw of one scripting tool to another lacks the pleasing aesthetic simplicity of doing it all in one script.

It also means I can't do what I wanted - boo hoo. That's a bit frustrating after a day spent in the office at the weekend, but given all I'd be doing otherwise is picking at my teeth and staring at things on the internet, I'm not any the worse off.

I've also discovered a few things:

Some people get really angry on the internet when somebody asks a stupid question. I suppose I already knew that, but it bears repeating. After I'd read the angry answer to the question two or three times, I figured out why it was stupid and what the right answer was, but it was a bit of a shame that there wasn't a simple explanation alongside it. Does learning have to involve pain?

There's a nice script provided to allow you to query Alexa for website rankings. Probably of use to people doing research on competitors on Google, and something I wish I'd known about a year ago - maybe I'll build something with it later on.

There's not much documentation on using perl with AWS. I think (well, I hope) that two documents and three examples will be enough. But it's annoying that neither document tells you *everything* you need to know; one tells you the right libraries, one signs your requests correctly, but neither do both.

Anyway, I'm going to monkey around with this some more, and hopefully have something to show off before long. Even if 'automatic buying from Amazon' still requires me to find somebody to click on a button once a day...