chilts.org
Andy Chilton
Blog - chilts.org
My personal blog. Get free updates by subscribing to the 
RSS Feed.
Things to Remember about Amazon EC2
I've been knocking around with Amazon EC2 again over the past few months and am learning a few tricks which I need to write down so I don't forget them. Using my AwsSum library, here's a few bits and pieces:
Starting a new server (Ubuntu Lucid AMD64 in region 'us-east', availability zone 'us-east-1a'):
KEYFILE=$HOME/.ssh/id_rsa.pub
REGION=us-east-1
AZ=us-east-1a
AMI=ami-3202f25b # Ubuntu Lucid LTS, 64-bit, EBS Boot (from http://alestic.com/)
TYPE=t1.micro
./bin/awssum Amazon:EC2 RunInstances \
--Region $REGION \
--ImageId $AMI \
--MinCount 1 \
--MaxCount 1 \
--KeyName $KEYNAME \
--InstanceType $TYPE \
--Placement.AvailabilityZone $AZ \
--DisableApiTermination true \
--InstanceInitiatedShutdownBehavior stop \
--BlockDeviceMapping.0.DeviceName /dev/sda1 \
--BlockDeviceMapping.0.Ebs.VolumeSize 20 \
--BlockDeviceMapping.0.Ebs.DeleteOnTermination false
What we're saying here is to create an Elastic Block Store backed micro instance. We've actually asked for 20GB worth of storage on the root filesystem.
However, when we log on to that new instance, you can see that there only seems to be about 10G on /dev/sda1. Hmm, what's going on here?
$ df -kh
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 7.9G 1.1G 6.5G 15% /
none 285M 108K 284M 1% /dev
none 318M 0 318M 0% /dev/shm
none 318M 56K 318M 1% /var/run
none 318M 0 318M 0% /var/lock
none 318M 0 318M 0% /lib/init/rw
What's happening is that the Amazon Machine Image (AMI) has been set up to use 10G worth of storage, so that's all that we can see here. However, if you look at your list of EBS volumes, you can see that' you're being charged for 20G (which is what you asked for in the first place)!
$ ./bin/awssum Amazon:EC2 DescribeVolumes
{
"xmlns" : "http://ec2.amazonaws.com/doc/2010-08-31/",
"_awssum" : {
"ok" : 1
},
"requestId" : "c569ff82-9ad5-4e39-9313-22f7079fe5da",
"volumeSet" : [
{
"volumeId" : "vol-xxxxxxxx",
"status" : "in-use",
"attachmentSet" : [
{
"volumeId" : "vol-xxxxxxxx",
"instanceId" : "i-xxxxxxxx",
"status" : "attached",
"deleteOnTermination" : "false",
"attachTime" : "2011-05-21T08:13:39.000Z",
"device" : "/dev/sda1"
}
],
"availabilityZone" : "us-east-1a",
"createTime" : "2011-05-21T08:13:21.000Z",
"snapshotId" : "snap-xxxxxxxx",
"size" : "20"
}
]
}
Ok, let's make sure you're making full use of that 20G you're paying for. Log back onto your instance and perform the following command:
$ sudo resize2fs /dev/sda1
resize2fs 1.41.11 (14-Mar-2010)
Filesystem at /dev/sda1 is mounted on /; on-line resizing required
old desc_blocks = 1, new_desc_blocks = 2
Performing an on-line resize of /dev/sda1 to 5242880 (4k) blocks.
The filesystem on /dev/sda1 is now 5242880 blocks long.
Cool, it seems to have worked. Let's just check:
$ df -kh
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 20G 1.1G 18G 6% /
none 285M 108K 284M 1% /dev
none 318M 0 318M 0% /dev/shm
none 318M 56K 318M 1% /var/run
none 318M 0 318M 0% /var/lock
none 318M 0 318M 0% /lib/init/rw
Excellent, we've got our other 10G that we're paying for. Until next time.
Inserted: 2011-05-27 05:14 (8 months, 1 week ago)
Enjoying Kiva
Just thought I’d post a short entry saying how much I am enjoying microloaning on Kiva.
Chris told me about Kiva a couple of years ago and I have seen Brenda and Callum and Bruce (all in Wgtn) on Kiva too. (My profile.)
By the way, if you want to join up then give me a shout and I’ll invite you (I’ll need your email address). You can just go and sign up yourself, but at the moment if I get 5 invitees and they make a successful loan each, then I’ll get an extra $25 to be able to loan out :)
Am up to 15 loans now (Kiva Avg = 6.4) and it seems that every month I get some money back, put a little more in and then make another loan or two.
- Update1: forgot to mention that I’m a part of the KivaKiwis and Koha teams
- Update2: sent out one invite to a friend.
- Update3: to get the $25 I need the 5 invitees to make at least one loan each before the end of October
Labels: kiva
Inserted: 2010-10-18 02:28 (1 year, 3 months ago)
What Programs Are
I’m on a lot of mailing lists, some of which I read religiously, others I dip into when the need arises and others still I just read the topics that look interesting. The NodeJS is one such list in that I just read the thread whose subjects stand out to me.
And luckily for me I clicked a fairly mundane thread this morning but then read what I thought was a fantastic description of programs and therefore the kind of emotions you get when programming. Written by Isaac Schlueter and reproduced here:
- Some programs are community centers, and you want to entice people to come inside.
- Some programs are factories, and you want to make sure the workers don’t hurt themselves.
- Some programs are shops, and you want people to visit, but only a few go in the back.
- And then some programs are art, and while you might hope that some people find them beautiful or perhaps even useful, and contributions may be accepted and feedback always welcome, the real purpose is just to hang it in your house and let it bring you joy.
I like it. Thanks Isaac.
Labels: programming, code, nodejs
Inserted: 2010-10-14 20:58 (1 year, 3 months ago)
Dear Mr Key
I just decided to fill in one of those e-cards on the Greens website. It’s about keeping KiwiBank 100% Kiwi-owned.
Here’s a copy and I’m happy for any comments or recommendations.
—-
Dear Mr Key,
Keeping KiwiBank and other national assets Kiwi owned is a must. As a recent citizen of NZ (4 years ago) I choose to live here because I think it is the best place in the world. By keeping our money local and our assets locally owned means for a better NZ. If that were to change, then I’d have to reconsider where I want to live. Since being here I recognise how important it is to buy kiwi-made and shop local which was never in my mind when I was in Britain. At first I thought it was almost insular but now I see how important it is so we don’t disappear within the global economy.
Also, having recently started a business – a web-based IT services company – most of my sales will be coming from other countries, bringing money into our economy and helping grow from exports. It’s very keenly the kind of company you want here. I could have quite easily set up my business in Australia or the UK so if I’m doing my bit for New Zealand, why can’t you do yours.
Keep KiwiBank and other nationally treasured entities locally owned by the NZ public!
Many thanks for your understanding.
Cheers,
Andy
Labels: kiwibank
Inserted: 2010-09-30 21:17 (1 year, 4 months ago)
32 Commits Today
Yes, I know “commits” are about as informative as “Lines of Code” but it does sometimes help you see where you are in comparison to other days.
Thanks to the terrible Wellington weather I stayed home all day and hacked. It was awesome. It turns out I did 32 commits today to my current project. Correction, I have two projects on the go (one at home and one at the office). This is for the one I’m doing at home. After all that I am (1) tired, (2) chuffed I was so productive and (3) happy!
The most interesting of these of course is (2). I have found that since I quit my job and am doing my own thing, I am almost infinitely more productive. Firstly, I know the ins and outs of the system, secondly I have no interruptions (irc, email, phone, in-person, clients) and finally, all my projects are just so much more interesting than previous things I’ve worked on. And yep, that includes a satellite control system! :)
Labels: hacking
Inserted: 2010-09-02 12:03 (1 year, 4 months ago)
That Friday Feeling
Today was the first Friday in a very long time that I haven’t had that Friday Feeling. It only hit me at about 4pm when I realised that my old work (Catalyst) would be starting their beer o’clock.
For years I’ve had that Friday Feeling and I have usually found Friday afternoons to be less productive than other parts of the week. However, whilst still beavering away at 6pm I realised that I’m so mad busy doing things that I don’t really think of what day it is, or even what time it is. It’s also because the things I’m busy with actually interest me and whilst I’d say I’ve always had an interesting job, doing the things you want to do and how you want to do them make it infinitely more enjoyable.
Going back to weekends. For me now, the weekends are just extensions to the week for two reasons.
Firstly, I don’t have to do housekeeping chores like my washing at the weekend since I’m able to do it when I like through the week. This even means I choose a beautiful day so that my clothes actually have a chance to dry. This is definitely an advantage of the time being your own.
And secondly, I’ll be heading into the office both days of a weekend anyway so it doesn’t mean much to me anymore. Currently I work at home in the early morning, head in to the office for a long afternoon (which broadly covers lunch and tea), then back home for some more late night hacking.
Weekends now just means that the probability of seeing certain friends is higher than on other days of the week. Obviously these are friends who have day jobs and have less ability to mould their own time. However, I also have a number of friends who are self-employed and the probability of me seeing them is higher on every single day :) This is definitely a perk of being self-employed and one I intend to take full advantage of.
Finally, this entry is a kind of “Yep, I’ve quit and the business is up and going” introduction and I promise to keep you more up to date as things progress.
Labels: business, self-employed, catalyst
Inserted: 2010-08-13 11:02 (1 year, 5 months ago)
Why Google Wave was a Failure
Dear Mr Schmidt,
Your quote on the BBC News article about Google drops Wave because of lack of users was interesting. You said “It’s a very clever product. You never know why it didn’t work”. Obviously you’re not as clever as you think your product is.
When Google Wave launched, the crowd stood up, applauded and the YouTube videos that were released made Wave look awesome. Instant view of what other people were typing, the ability to change your previous comments (and indeed, other peoples), the slider to see the history, being able to upload images and the ability to make lots of bots to run inside the thing.
The problem is, no-one wanted any of these features. Clever – yes. Useful – no.
The slider is only there since the complexity of the conversation was far too great. The complexity was far too great since you could reply to anyone anywhere as well as changing previous messages. This in itself is fairly horrible since the conversation then isn’t linear. Ever heard of why people don’t like top-posting on email? Uploading images are ho-hum, email already does previews for image attachments (and why do we need any more than that). All 'Chat’ protocols out there don’t do the 'show as you type’ thing – mainly because no-one wants it (how often do you type something and then delete it … lots). Finally, bots … more annoying than the ability to change anyone’s message … you get my drift.
Add to the fact that all the features you guys trumpeted as amazing, ambitious and new, was the slowness of actually making it work. And yes, even in Chrome it was slow. Painfully slow. Your whole company is based on speed, decreasing page load times, minimising JavaScript and CSS, having a minimal search page, updating old (HTTP) protocols for new ones (SPDY), making JavaScript go faster and deferring loads of content until after page ready … so why oh why oh why would you think a painfully slow application – such as Wave – should be any different to EVERYTHING your company stands for and yet be sucessful.
As for the things it was hyped up to do: Productivity – nope, it was too slow. Wiki(ish) – nope, it was too complicated. Conversation – nope, it was too surreal (after all, when have you been able to edit other people’s past conversation?).
Finally, it was immediately obvious to myself and the majority of my friends (that is, only two thought Wave was any good) that Wave wasn’t what Google trumpeted it as. It was in fact, a gimmick. A fun game that kept us occupied for exactly one afternoon, after which people said “Now I’m done, don’t think I’ll do that again”. Kind of like playing a new game only to realise that it has no longevity. No usefulness. No future.
So if it was painfully obvious to the rest of the world (ok, 98% of Wave users) that it was annoying, why wasn’t it obvious to you and your team. After all, you do champion measuring what your users are doing. If 98% aren’t logging in much after the first day, surely that’s a sign it’s not going well. I’m surprised it took you a year to figure that out.
All the best,
Andy
Inserted: 2010-08-07 11:53 (1 year, 5 months ago)
Opening Up Cloud Services
It’s been a while since I posted but this particular news item today has definitely made my ears prick up. Of course, I’m presuming what this also means for the future and I hope I’m right.
RackspaceCloud and NASA (of all people) have started a joint initiative called OpenStack. Basically it means that there will be an open source solution to cloud servers and other web services. Yes, I know about Eucalyptus but I think this means much more than that.
From what I understand, they will be open sourcing the code that powers their Cloud Servers API and in the not too distant future, the code that powers their Cloud Files service too.
If I extrapolate with hope, then I can only imagine that they will also be adding more services in the future, such as a queuing system, a notification service and possibly a distributed DB service! Yes, these are all examples of the wonderful (but proprietary) services that Amazon Web Services already provide.
Anyway, today is looking like it could be a good day for the future of cloud computing. Being able to run a free and open source operating system inside a virtual machine which is running a FOSS OS, using FOSS services for your own FOSS software can only be a good thing. If I didn’t say FOSS enough in this paragraph, let me say it one more time – FOSS!
Oh, and the other great thing about this is that it opens up competition and freedom to move providers. If you don’t like the provider you’re using, then due to the nature of the services being open source, you’ll be able to switch to other providers you do like! This is very much in contrast with the Amazon model where you get locked in using their services.
Finally, it seems that today also marks the first day of OSCON, and seeing as how RackspaceCloud’s API is much more RESTful than Amazon’s, and how they seem to say they’re an open source company, I can only assume that they doing the right thing and doing it well. Let’s hope they keep to their promise and deliver.
Of course, with Rackspace being a far 2nd in terms of numbers of servers to Amazon’s own EC2 offering I can only say that even if they didn’t make this decision because of those stats, it can only help it along :)
Well done Rackspace!
Labels: rackspace, foss, cloud
Inserted: 2010-07-19 09:50 (1 year, 6 months ago)
Reasons to Not Make up your Own Open Source License
If Google can’t get their WebM license right, then who can:
I did wonder why they didn’t just go with GPLv3 but I figured that Google usually like the BSD-style licenses. They have answers on their Licensing FAQ but still, it makes you wonder.
For example, I wonder if this is essentially the BSD license with the patent clauses from the GPLv3. That would be both interesting and cool, though I’d still prefer GPL over BSD.
Inserted: 2010-06-04 22:58 (1 year, 7 months ago)
Two Reasons Why I'll Never Install Flash Again
The first is pretty obvious. Now that I can watch YouTube using HTML5 features that means Flash is one step away from being chucked out forever.
And after reading this blog entry about the Google Maps API v3, Flash – as far as I’m concerned – is now out of the window. I didn’t realise but hidden in amongst a lot of other text are the words The most significant change is that Street View is entirely implemented in HTML....
I thought “No, surely not”, so I kept double clicking the map until I was sufficiently zoomed in to see 'Pegman’ and I moved him onto the road. Sure enough, in my non-Flash versions of both Chrome and Firefox I could see Street View. After twisting the view around, clicking up and down the street I was wondering to myself “I have no freakin’ idea how they did this, but I’m glad they did”. If you haven’t noticed already, I think the Flash version of Street View had two levels of zoom, well this one seems to have 5 or 6.
So yes, I’m a happy bunny tonight. Add this to the fact that GitHub also reimplemented their network graph a few months ago using the Canvas element, the future is looking decidedly bright (and very HTML5-CSS3-like).
Labels: html5, css3, gmaps, flash, github, google, street-view
Inserted: 2010-05-21 11:46 (1 year, 8 months ago)