Category: UpgradingOracleDatabases
Build your database from a single command
When you're building a new application or module with Oracle in the back end, you'll be building a lot of elements that will reside in the database. You'll probably need to create a new database user, tables, views, packages, roles, grants, etc, etc. There is no reason why you shouldn't be able to do this from a single central script.
You may need a few parameters on that script, or a config file to go with it, but you should be able to point that script at an empty database, and end up with a completely working back-end for your system. The only requirements of that script should be that there is a working Oracle server instance to install into, and that you have Oracle client tools installed that will connect to that server. There, theoretically, isn't any reason why you can't make our script setup the datafiles, tablespaces and suchlike for our database, but we don't want to take all the setup work from our DBA's in one go!
By writing this script you allow yourself to be able to roll out a new database workspace with the minimum of fuss. You'll need a database workspace to go with each development workspace you use. When you need to roll back your development workspace to a previous version in order to fix a bug that exists in live, you'll need a database at the same version for that system to connect to. In fact, the database workspace and the development workspace are the same thing. Unless you have a requirement that your database must not change in any way through your development, then you are developing in the database, and so it will change with the rest of the system. Therefore, a particular version of the front end of the system will require a particular version of the back end of the system. The two things are intrinsically linked.
This fact is missed by so many people it scares me.
Rebuild your databases often
Very often in software roll-out we find that the database build / upgrade is the section of the build that causes the most headaches and unexpected failures. There is a simple reason for this. Database upgrade code is very rarely ran. Many places I've worked would release upgrades that have never once been ran from start to finish without a failure. Obviously this is a bad idea, but that didn't stop it from happening: scripts would be written that would be ran once in a developers workspace and then not ran again until a 'system test' phase where they're ran once, the scripts being massaged by the developers on had. They'd then be deemed as fit for release even though they'd not actually ran through start to finish.
I've never seen a situation where other code in the system suffers quite like this (except maybe interfaces). Even when testing has been very much an after thought, if something was so broken that a developer was needed to 'just put that semi-colon in', then the system didn't get shipped.
So, we address that by getting into the habit of running our build and upgrade scripts often. Whenever a workspace is updated with the latest version from cvs, the database build script is ran. Whenever integration starts, the database build script is ran. Before a full set of tests is ran, the database build script is ran. By running the build often we ensure it runs first time, every time, with no failures.
We may 'just add that line' and not test it if once person we runs the upgrade once or twice. We always fix the problem if 8 people need to run it many times a day!
If you're supporting a live system, make an incremental build
If you're following XP you've probably 'gone live' by now. If you haven't, why not... isn't your system useful yet? ;-)
If you're supporting a live system then you've probably got a database that you can't trash and rebuild when you want to move it to the latest release. You've got data that you need to keep. You're going to have to upgrade.
That is, as soon as you go live you're probably going to need to change the way you build the database. Up until that point it's normally easiest to make a greenfield build: a script that destroys what ever exists and builds a new database schema from scratch. This is almost certainly no longer possible.
So leave your greenfield build scripts alone, at the version at which you went live. Then add patches into the build that will upgrade that greenfield. Essentially it's like having a code freeze on a small section of the build. I know, I know, I said code freeze... it is a good type of code freeze, I promise!
We do this so that when we build our databases in the development environment we run the same code as the live upgrade will use. This is, afterall, our main way of testing the upgrade scripts. Since we can't destroy the database in live, then we need to make sure that our build doesn't rely on this being possible in the development environment. Another way of looking at it is: If it doesn't run against live then we don't want to run it in our development environment; since a changed version of the greenfield installer will never get ran against live, then there's no need to change it.
Admitted, at some later 'go live' point it may be worthwhile consolidating those patches into a new greenfield installer. This will make the upgrade code a little easier to follow. However, there is always the chance that the resulting database created by the greenfield installer will differ from that created by the original patch based installer. We can, of course, test for this.
There's a lot more that can be said about the design of an incremental build script: unit testing upgrades and associated migrations, logging patch activity. That may come in a later blog entry...
Tuesday, March 01, 2005
Monday, February 28, 2005
Fundamentally, there are no No-Brainers
I've just started reading Mike Mason's new book on implementing version control using Subversion.
It's a nice easy read that starts from the fundamentals of why we use version control; it's got a really nice little illustration of how and why version control works. As any book on version control will, it mentions software configuration management, and on that topic it mentions the great book 'Software Configuration Management Patterns', which I remember Mike suggesting I read when I first described to him what Oracle development workspaces are normally like.
That little reminder made me think back to when a lot of what I do now wasn't anywhere near on my radar. I knew that I did things badly, but I wasn't sure how it could be fixed. A lot of things that I do now, that I think of as 'no-brainers', are things that I hadn't even considered.
It got me thinking that maybe it's time I took a step back and looked at how we've set up our workspace. I figure it's time for an honest appraisal. Hopefully I'll even spot something that can be improved upon.
It's reminded me that it's worth reading books on the things we regard as 'fundamental' every now and again for just this reason. It reminds us why we do what we do, and why we do it the way we do it, and that we need to think about both those things even if we regard those things as 'no-brainers'.
Also, it made me think that maybe I should describe our workspace setup in a future blog entry, to sanity check it and see if it's of use to anyone else. Mike may have pointed out that it's not as standard as I might like to think...
It's a nice easy read that starts from the fundamentals of why we use version control; it's got a really nice little illustration of how and why version control works. As any book on version control will, it mentions software configuration management, and on that topic it mentions the great book 'Software Configuration Management Patterns', which I remember Mike suggesting I read when I first described to him what Oracle development workspaces are normally like.
That little reminder made me think back to when a lot of what I do now wasn't anywhere near on my radar. I knew that I did things badly, but I wasn't sure how it could be fixed. A lot of things that I do now, that I think of as 'no-brainers', are things that I hadn't even considered.
It got me thinking that maybe it's time I took a step back and looked at how we've set up our workspace. I figure it's time for an honest appraisal. Hopefully I'll even spot something that can be improved upon.
It's reminded me that it's worth reading books on the things we regard as 'fundamental' every now and again for just this reason. It reminds us why we do what we do, and why we do it the way we do it, and that we need to think about both those things even if we regard those things as 'no-brainers'.
Also, it made me think that maybe I should describe our workspace setup in a future blog entry, to sanity check it and see if it's of use to anyone else. Mike may have pointed out that it's not as standard as I might like to think...
Tuesday, February 22, 2005
Death of the salesman (or power to the customer)
One of the main reasons why XP has been such a success for us is that the customer is so much happier when they get something they actually want. It's probably the only reason that the customers still want to work with us, despite our little idiosyncrasies. And so we have to make sure that they continue to get what they want.
Unfortunately, following the XP rules doesn't strike me as enough to ensure this is true. Following XP only guarantees that we deliver software that is in line with the stories. We also need to make sure that the stories accurately reflect what the customer wants.
We regularly get together to discuss future stories, constantly putting new stories in, taking old ones out, and tweaking the ones we have. We tend to introduce stories into the mix in two stages. First the customer (which is a team) discusses their new functionality requirements and gets the outline of the stories together, writing up some draft cards. The development team then sends a delegation to meet and discuss those stories (at length normally) and the true stories are written.
We do this in two stages so that the customers can have time to get their ideas straight in the their own minds before they approach us, whilst ensuring that every story is seen by the development team so we can work out any gaps in the knowledge embodied in those stories. In the second meeting we may put together estimates, or we may take the stories away and estimate later. The important fact is that a story isn't a story until its been through the second meeting.
On the face of it, this should ensure that the stories contain both the customer's requirements of the system and the developer's requirements of the description.
It generally does, until the salesman gets involved:
As developers we 'know' that we've been involved in system development for a lot longer than most, if not all, of our customers. This can make us feel as though we know more about system design than our customers. This may be true, but we don't know as much about this system as our customers. If we did, then we probably wouldn't be doing XP ;-)
The salesman is the developer in the story review meeting that forgets this. They 'understand' the requirements of the story as written and know of the 'best' solution. They will explain this solution loudly and then deflect any criticism fired at it with lengthy explanations or snappy comebacks. Whilst they're doing this they'll become more and more certain that their solution is the only solution to the problem. And the whole time they're doing this they'll forget that:
1 - Whilst the customer may not get it written down first time, they're more than qualified to specify what they need the system to do, that's why they're still on the team.
2 - The developers are not here to provide the solution, they're here to ensure that the problem is clearly stated.
3 - It's good to have ideas, but not all ideas are good.
4 - When the customer speaks, the developers should be listening.
It's easy to drop into this persona as a flash of brilliance pierces the fog, but it's important to remember:
When you have an amazing idea, just state it, don't sell it, and accept it's probably not as amazing as you first thought.
Unfortunately, following the XP rules doesn't strike me as enough to ensure this is true. Following XP only guarantees that we deliver software that is in line with the stories. We also need to make sure that the stories accurately reflect what the customer wants.
We regularly get together to discuss future stories, constantly putting new stories in, taking old ones out, and tweaking the ones we have. We tend to introduce stories into the mix in two stages. First the customer (which is a team) discusses their new functionality requirements and gets the outline of the stories together, writing up some draft cards. The development team then sends a delegation to meet and discuss those stories (at length normally) and the true stories are written.
We do this in two stages so that the customers can have time to get their ideas straight in the their own minds before they approach us, whilst ensuring that every story is seen by the development team so we can work out any gaps in the knowledge embodied in those stories. In the second meeting we may put together estimates, or we may take the stories away and estimate later. The important fact is that a story isn't a story until its been through the second meeting.
On the face of it, this should ensure that the stories contain both the customer's requirements of the system and the developer's requirements of the description.
It generally does, until the salesman gets involved:
As developers we 'know' that we've been involved in system development for a lot longer than most, if not all, of our customers. This can make us feel as though we know more about system design than our customers. This may be true, but we don't know as much about this system as our customers. If we did, then we probably wouldn't be doing XP ;-)
The salesman is the developer in the story review meeting that forgets this. They 'understand' the requirements of the story as written and know of the 'best' solution. They will explain this solution loudly and then deflect any criticism fired at it with lengthy explanations or snappy comebacks. Whilst they're doing this they'll become more and more certain that their solution is the only solution to the problem. And the whole time they're doing this they'll forget that:
1 - Whilst the customer may not get it written down first time, they're more than qualified to specify what they need the system to do, that's why they're still on the team.
2 - The developers are not here to provide the solution, they're here to ensure that the problem is clearly stated.
3 - It's good to have ideas, but not all ideas are good.
4 - When the customer speaks, the developers should be listening.
It's easy to drop into this persona as a flash of brilliance pierces the fog, but it's important to remember:
When you have an amazing idea, just state it, don't sell it, and accept it's probably not as amazing as you first thought.
Tools are products too
With any project there will be a plethora of tools that need developing. The unit tester may need extending, some code generators would come in handy, you could do with some patch mechanism to upgrade the database.
The code that implements these is exactly the same as any other code. It may only be a 3 line batch file, or it may be a million lines of Java code. It doesn't matter, it's still output from the project and it still needs to be created in line with XP.
Pair program it, refactor mercilessly on it and if you're not using it then throw it away.
The code that implements these is exactly the same as any other code. It may only be a 3 line batch file, or it may be a million lines of Java code. It doesn't matter, it's still output from the project and it still needs to be created in line with XP.
Pair program it, refactor mercilessly on it and if you're not using it then throw it away.
Monday, February 21, 2005
No-brainer #674
If you have an on-access virus scanner running on your DB server, tell it to NOT scan your database files.
Subscribe to:
Posts (Atom)