Entries for label: aws

Found 4 entries.

Project AwsSum Update: Added 'Copy' to S3

Amazon added the ability to 'copy' a key within or between S3 buckets.

This functionality can also be used to change the metadata of an object too.

For example, using the thin-layer commands in Project AwsSum, you can do things like:

 $ s3-rad -b mybucket -k NewKey.jpg -K FromKey.jpg

As I said, you can also copy between buckets (so long as they are in the same location, either the States or Europe currently):

 $ s3-rad -b mybucket -B frombucket -k key.jpg

As you can see, you have to specify a destination bucket and key but you may leave off the source bucket and/or key. While the Amazon API requires everything, my S3 library fills in the blanks.

If you try to copy a key onto itself hwoever, you'll get an error back from Amazon:

 $ s3-rad copy -b mybucket -k image.jpg
 InvalidRequest: The Source and Destination may not be the same when the MetadataDirective is Copy.

but you are allowed to do this if all you want to do is change the metadata, in this case, the Content-Type:

 $ s3-rad copy -b mybucket -k image.jpg -m REPLACE -c 'image/jpg

Whilst the CopyObject operation in S3 is still beta, there has been much discussion about it on the AWS forums and I suspect the API will be final now.

Note: for those using s3bak you can now add a DefaultBucket to your ~/.awssum config file, thanks to a suggestion from Francois.

Labels: project-awssum, planet-geek, planet-catalyst, s3, aws

Inserted: 2008-06-15 12:30 (2 years, 1 month ago)

Adventures with Erlang

I started learning Erlang today and already I'm liking it.

For a couple of months I have said that I will learn Erlang and finally I've started.

Firstly, I'll tell you why I started today (apart from the bad weather) and why I want to learn Erlang, and secondly I'll tell you my thoughts on the language so far.

Why today and why Erlang?

The reason I started today is because of a culmination of a number of things. Recently I have been playing with all the AWS components S3, EC2, SimpleDB and SQS. Playing with them in Perl has been a good experience and now I know what's required to make it work - by this, I'm not talking about just querying them and seeing the results, I'm talking about a big project with the ability to scale out once demand gets high.

So my plan is to get a project up and running on EC2, using all of S3, SimpleDB and SQS and to make sure everything is massively parrallel.

Erlang fits into all this because it's a good idea to try and do it correctly from the start. It almost seems weird to think of using Perl to run something like this. Also, I really want to learn something new and out of all the languages that have tried to get my attention in the past number of years, none have gripped me and certainly none as much or as quickly as Erlang has.

I notice that Erlang already has a few web servers which can be used as well as lots of other freely available libraries. Amazon also uses Erlang behind the scenes for SimpleDB itself - no coincidence I'm sure. So overall, a culmination of wanting to do a project, use a massively scaleable architecture and learn a new language have all come together at exactly the same time.

What I think of Erlang so far

So far, I'm really enjoying it. I have learnt a shedload already and I can feel a lot of information sinking into the old grey matter. At first, I looked at some of the syntax and just shook my head but as always, once you start, it just begins to make sense.

In reality, I've done a little bit of Lisp and Scheme in the dim and distant past and so far I haven't actually seen that much which is new to me (though I'm sure that's to come). Obviously Erlang borrowed heavily from Lisp though which helps me understand some of the weirder parts of the language. All the power that recursion, passing functions or returning functions from functions isn't phasing me since it's made sense to me for a while. Also, being able to build the language up is another helpful thing you get if you really need it.

(I've always said that Lisp is the next big language but not in it's current form. All the other languages are just re-implementing certain parts of Lisp as they see fit - 40 years late. Erlang has already done that AND put it inside a concurrent environment too.)

The fact that variables don't change kinda struck me as weird but again once you start making functions and calling them, you almost forget that you can't re-assign to variables any more - it's just not something you really want to do anyway (much like you don't use (let ...) much in Lisp). It's all about the functions!

One thing that is new to me is the whole message passing thing. I also cottoned onto the fact that objects and variables are replaced with processes and that's where all the magic is.

Already I have created libraries, functions with different arity, tail-recursive functions, played with tuples, retrieved a web-page (learning about receive) and even had my own chat room and simulated a number of users all talking to each other.

There seems to be a lot of really clever people doing Erlang at the moment and it's definitely a great time for it to flourish. Having huge resources like at Amazon AWS and multi-core processors getting constantly bigger, now is the time for a language to step up to the concurrency plate.

So all in all a very productive day. I think I have the basics of the language sorted - in reality there's not much there - but now I'll need to sit down and fully work through more of the other concepts it introduces.

It certainly needs a different thinking cap to anything else I've used and that's going to be both an interesting challenge and a bit of an adventure.

Labels: planet-geek, planet-catalyst, cop, aws, erlang

Inserted: 2008-03-03 00:12 (2 years, 4 months ago)

Amazon Web Services with Mike Culver

Tonight, we got a good opportunity to hear and see what Mike Culver (Web Services Evangelist) had to say about Amazon Web Services.

It was a good talk. A bit fast to go through everything but that's a reflection of how much Amazon have to offer. Lucky for me, I knew most of it but I did learn a few things along the way. Especially from the demo of using EC2. I wish he could have spoken more about SimpleDB though.

As it turns out, my plan to ask if I can be added to the Beta program for SimpleDB worked and I have just sent an email off asking if I can join it. So yeah, that makes me very happy.

The good thing is, as you know, I've been playing with S3 and s3bak a lot recently, but I've also been making a program to play with EC2 as well. It's actually not that big but already there is some good functionality. My plan to make a small command line interface to both SimpleDB and SQS is also in the pipeline.

It certainly is exciting to be playing with these technologies and while there are some criticisms against them, I'd say to see them as they are and watch them advance over time. The whole point of the whole setup is to be easy to use and it's up to us application developers to use them in new and enlightened ways, adding value along the way.

However, I do see a different problem though - oh so many ideas and oh so little time.

2008-02-20 10:31 - Update: I've just been added to SimpleDB Beta. Wow, they work fast :-)

2008-02-20 19:32 - Update: Don Christie (president of NZOSS) has linked to this article from his Pass the Source blog - When Amazon Comes to Town. Out of the 4 quotes of feedback, mine was the top one and one which I will try and expand into a bigger article later.

Labels: ec2, simpledb, aws, mike-culver, sqs, planet-geek, planet-catalyst, s3, s3bak, amazon

Inserted: 2008-02-19 22:07 (2 years, 5 months ago)

Some Cool Stuff Coming Up

There are a few things in February I'm really looking forward to.

Firstly, it looks like Grant has snared Damian Conway to come and speak to us at Wellington.pm . That is going to one awesome evenings.

And secondly something which made me smile. I read the Amazon Web Services blog and saw this article about New Zealand and Australia the other day. So I followed their wiki entry to this page about Mike Culver's Schedule only to realise that he's already coming to WellyLug/Catalyst :-) It seems that Brenda was to blame for this - so thanks Brenda.

Also, Brenda, Reed and co. have been busy figuring out which language to use next (and it even got posted on PerlBuzz). I'd have also said Ada is Anal but then, I had to do Ada 83 (Ravenscar Profile) for 9 months once.

Labels: programming, planet-geek, aws, perlmongers, amazon

Inserted: 2008-01-10 00:28 (2 years, 6 months ago)