Sunday, January 15, 2006

We need space to develop

I've talked previously about how important I think it is that individual developers (or pairs) get to work in their own sandbox (not least here). This means that developers can experiment; can try avenues that would otherwise break the workspace for other developers; can refactor the database itself.

Obviously, in order to do this, you need to actually have development databases for those developers to own. This tends to lead to the DBA team announcing that you'd need far too many databases, that you simply can't be serious, that we've no idea how overworked they already are and that we can't expect them to take on another x databases, where x equals the number of developers. Oh, and by the way... who's going to build them? I can fit the first 3 in some time the middle of the year after next, god knows when the other 7 will get built.

I can understand the reaction. I'd be exactly the same if I thought that someone was suggesting that I doubled my workload. Luckily for DBAs, that's most definitely not what I'm suggesting.

The misunderstanding comes from the thought that a development database is the same species of beast as the live database. It isn't, not by a long way.

The major difference between a live and development database is how their role defines the use of data. In a live database the single most important thing in that database is the data it holds. It is the sole purpose for the existence of that database. Fine, we reshape that database so that we can access the database in new and interesting ways, but all said and done, if that data wasn't there then the database would be useless. In the development environment this is not the case.

The most important thing in a development database is its structure. Its purpose is to work as a validation tool for the live database, to ensure that the application we're producing works against the live database and to allow us to develop the structure of that database in order to allow to us to provide our application. It's important that we hold data that has a texture similar to that which exists in live, but it doesn't matter which data we hold. The data itself has no intrinsic value. We can generate our own.
In fact, when we run unit tests on our PL/SQL code, this is exactly what we do; we generate our test data so we know exactly what form it is in. That way we can make clear statements about the form we expect it to be in when our PL/SQL has completed.

Also, we don't need to hold the same volume of data. In live, every single piece of data is of importance and so you can't just remove it*. In development we can limit ourselves to create only the data that is of importance to us at this point in time. If our development databases contain a very small amount of data, and the means to generate the data it needs, then you remove a large amount of the DBA's work.

That is, most the work a DBA will do in maintaining a live database is purely down to the fact that it is a system that contains a large volume of critical data. It must always be available, it must serve data within certain performance requirements, and if the machine it's running on suddenly blows up then we must be able to recover the data. Quickly.
Take away everything other than the 'it must always be available', and you've taken away most of the DBA's work.

When we started development, we didn't tell our DBA that we needed 'x' new development databases. We told our DBA that we needed 1. We said we didn't need it backed up, just that we needed it to be up during working hours. We said, make it about the same size as live and it'll always be big enough.
We then ran our build scripts multiple times on that database, creating a schema for each of our pairs of developers. This is the same build script we would use to upgrade live, it just has a development mode that allows us to say which schema name we want it to install into.
The DBA has needed to do next to nothing with the database since it was created.

Now I admit, we have only around 10 development databases at the moment. Maybe the problem gets a lot harder to manage when you have 100 development databases. But then again, if you have 100 developers then I'm damn sure you've got more than 1 DBA. You've got a DBA team. And if you've got a team then surely someone in that team's got the time to build you enough databases to support your development team. If not, then you've probably got more serious problems to deal with...

Of course, none of this takes into account the fact that your customer needs to test the application, that you'll need to performance tune and test along side all the other systems running before you release and you need a test environment for duplicating issues once the system goes live. But NONE of these things should be happening in your development environment... a topic for another post. Later.

* Actually, you often find that most the data in a live database is useless and only a small proportion is ever used. But try telling that to your customer and getting them to let you archive it...


Technorati Tags: , , , , , , , , ,, , ,

Tuesday, January 10, 2006

Database Upgrade Links

Around August last year I started to put together some notes on writing database upgrade scripts. It was kicked off after a discussion with Wilfred van der Deijl, based on experiences on my current project where I honestly we've got it about as right as it ever needs to be.

For a while now I've been fiddling with FeedDigest to try to get it to categorise my entries so that I can produce a summary page, and I've finally given up. So instead, I've put this entry together. It's just a link page to each of my database upgrade entries... I hope it's of use!

Sunday, January 08, 2006

Google Pack

It's been a while since I've been 'in the blogging scene', and I suppose the only way to get back is to start posting again! So, whilst this post may be small, I'm hoping it's beautiful.

Google have decided to address that age old problem... getting your new PC from the point it's at when you first buy it to actually being useful and on the way to being safe to use on the web.

They've put together the 'Google Pack'.

OK, so I like Google Earth, but I'm not sure it's quite essential, but the rest:

Google Desktop
Picasa
Google Toolbar for IE
Norton Antivirus
Ad-Aware
Adobe Reader

And THE most important one:
Mozilla Firefox

It's a stunning idea, though I'd probably add to that:
Open Office
AVG (Ok, so you've already got Norton, but AVG is ACTUALLY free)
Skype
Hitman Pro (Ad-aware is only one part of the anti spyware suite)

Wednesday, December 07, 2005

Valued estimation

In "User Stories Applied", Mike Coen puts forward a method for estimating the cost of developing stories (or any other piece of work).

In summary:
* The development team get together with the customer.
* The customer describes a story.
* The developers quiz the customer until they are happy they understand the story.
* Each developer that wants to participate writes an estimate down.
* All developers show their estimates.
* The developer with the highest estimate states why they think it's so big.
* The developer with the lowest estimate states why they thing it's so small.
* The developers discuss the problem further until they're all happy they understand what's involved, and they estimate again.

The loop keeps going round until the team all have the same estimate, or they're happy to accept a compromise on one.

Once the stories are all estimated, they're put in a line of ascending cost and examined to make sure there are no discrepancies. If there are, then that story is re-stimated.

It works a treat, and the customers like it.

In fact they like it so much that they've come up with a variation...

Excluding the project manager, our customer team consists of two full time members and a number of people that work on our project only part-time. One slight issue is that over the last 18 months the full time members are becoming more a part of the Information Systems team and less in touch with 'The Shop Floor'. The part-time members help to ground them in the reality of the system.

On the other hand, the part time members aren't available to instantly evaluate new stories, and prioritise them accordingly.

So, once a week the customer team now get together in a 'Value and Take-up estimation'. It works in the same way as the developer estimates except the customer team are evaluating the value that a given story will add to the system, and then the take-up level expected for that part of the system in a scale of one to ten,

Obviously, the priority of a story is based on the combination of all three variables, Value / Take-up and Cost. Still, as a general rule, a story that scores high on Value and high on Take-up is going to be a story that will be prioritised very highly unless the cost is extortionate!

In fact the particular values may not be that important; there is no point discussing if a Value 8, Take-up 7 story is more important that a Value 7, Take-up 8 story on the basis of the raw values alone.

Rather it's the forum for the discussion of the values that provides the real insight, and allows everyone to have their input.

It's working to ensure that the team is working on what the whole customer team decide is the most important task, and not just the one with the loudest voice...

Technorati Tags: , , , , , , , , ,

Friday, November 11, 2005

Firefox past 10 percent

According to this week's New Scientist, Firefox has managed to get more than 10% of the web browser market.

When I read that I was surprised.

It seems that readers of this blog are much more discerning than your average internet user. For some time now Firefox has been the browser used in more than HALF the requests to this site!







 New Scientist reportBobablog readers
MSIE85.45%44.42%
Firefox / Mozilla11.51%51.44%
Safari1.75%1.56%
Netscape0.77%0.32%
Opera0.26%1.56%


I'm sure that says more about bloggers and their readers than it does about this site in-particular...