by Jim Karabatsos - GUI Computing
It seems that every few years, the paradigm of software development changes. Often, these changes are refinements to our previous understanding of how software development works. In these cases, we all pause to absorb the implications of the new techniques and then we get back to work, slowly modifying our work practices and refining our tools to support us in our work.
Every so often, however, a fundamentally new paradigm is put forward for our consideration. These fundamental changes typically occur when the technology undergoes a major transformation, or when the economics of our craft change such that the most cost-effective option is different to what it was before.
The previous couple of major changes were structured programming and object-oriented programming and they typify these types of changes. Structured programming only became practical when the cost of programmers became a larger issue than the cost of the hardware.
Everyone who truly understands OOP would agree that it offers tremendous scope for improving productivity. Equally, they would point out that it will take a couple of years to transform a competent non-OOP programmer into a competent OOP programmer, and that OOP programs typically require more machine resources to achieve similar performance to non-OOP implementations.
A similar change in the fundamental nature of just how we approach developing applications is happening right now. Rapid Application Development, or RAD, made possible by new tools like Visual Basic and Delphi, is turning traditional development methodologies on their ear.
I have recently been involved in a number of large projects that were primarily mainframe oriented but that had a GUI and client-server component. It was quite an eye-opener for me. As someone who has been out of the front line of mainframe software development for many years, I had all but forgotten the enormous amount of paperwork and manual procedures that accompany such a large project. It seemed that you couldn't decide to scratch yourself without filling out three forms and submitting to a walk-through of some type.
Now let me hasten to add that I am not criticising these procedures. In your typical mainframe development environment, a methodology that lets us keep track of at least some of the myriad of details is absolutely essential if we are going to have any chance of success.
The problem arises when you try to apply such a methodology in a situation that it is not designed for. In one situation, the proverbial friend of a friend was called in to write a small Visual Basic program to act as a front end to some mainframe data. She was told that she was going to be part of the overall project and that her work would be subject to the same methodology that governed the rest of the project. When she asked exactly what this meant, she was pointed at a (rather large) set of manuals that detailed the umpteen forms and procedures she would have to deal with before she could actually write any code. When she factored this work into her estimates and duly presented her expected completion date, it was met with stunned silence. The general feeling was summed up with the comment "But we thought VB was a RAD tool".
What they failed to understand was that simply using a "RAD" tool does not mean that you are doing rapid application development. If you are going to leave the R in RAD, you need to adopt a methodology that does not get in the way.
Let us step back for a moment and think about why we fill in so many forms and create so many documents in a typical formal development methodology. After the business decides that it has a particular requirement, it will typically document what those requirements are and then go looking for a solution. This may involve handing it over to the IT department or perhaps calling in a consultant. In either case, the business is typically looking for a quote. How much is it going to cost to provide a solution, and how long will it take?
Now before we, as IT professionals, can answer these questions, we must further pin down what is being requested. What functions are going to be automated? What is best left alone? What existing systems are we going to interface to, both as suppliers and consumers of information? In short, what is the scope of the system? Only when we have answers to these questions, typically in the form of a Scope Definition, can we indeed take an educated guess as to what is involved in building such a system and therefore what it will cost and how long it will take.
The various methodologies begin to diverge from this phase onwards. You will document what data is going to be maintained by the system, and how system inputs will interact with that data to create the required outputs. Somewhere in all this, you will design the screens and reports for the user interface, and write the specifications for the processes involved, usually in English but sometimes in pseudo-code to document a particular algorithm. It is at this point that I again raise the question: Why do we do this?
Why is it that I draw screens using a drawing program only to then do it all over again in the final development environment? Why do I document processes in a word processor, only to then code them in my implementation language?
The answer is obvious if my development environment is COBOL. Drawing screens is done in code statements, and writing code in COBOL requires so much overhead that even the simplest code takes a while to write.
Let's take the screen designs first, because it is easier to see the reason for my question. It is obviously easier to draw a screen using any of the common drawing packages than to create a screen with a series of COBOL statements. Furthermore, users can see the drawing and give us feedback, whereas they can't really visualise what a series of screen definition statements actually does.
Now think about VB or Delphi. I would point out that it is actually less work to create the screens directly within these development environments than to draw them using a drawing package, even without considering the fact that you will ultimately need to do it in that environment anyway. This is because the basic components that you place on your "drawing" are the actual controls that make up the user interface. It is just as easy to modify the screens in response to user feedback, too. Even more importantly, with only a very small number of lines of code, I can actually link these forms together so that button presses and menu selections correctly bring up the appropriate screen. At the end of it all, I have a working user interface, ready to flesh out with application logic and a database access layer to complete the application.
We can readily see that it makes no sense to "document" (or "draw") the screens in any environment other than the implementation product's native screen-generation tool. Indeed, and to their credit, most development teams that are involved in any VB (or now Delphi) development do in fact use the form designers in those tools to design the screens.
When we turn our attention to the detailed functional specification documentation, the exact same arguments apply. For some reason, however, development teams that are accustomed to using a formal development methodology baulk at the conclusion they arrive at and therefore reject it.
The reason that I would make sure I had a word-processing document that accurately described, in excruciating detail, every single step in the processing of the application is that it was fairly simple to change and evolve that document as my understanding of the problem domain improved. It was only when I had a very good understanding of the problem domain that I would dare to commit that understanding to code, for the simple reason that it is at least one order of magnitude more difficult to change the code than it is to change the document.
Well, at least, that was the case. You see, it turns out that I can more accurately and succinctly describe the processing requirements and logic in VB or Object Pascal code than I can in English or pseudocode. This is because English is not as precise and unambiguous as is Basic or Pascal ó after all, that is why we have computer languages in the first place. The problem is that the user cannot understand the bulk of the code and so is unable to comment on the correctness of our understanding of the underlying algorithms.
There is, however, a major difference between VB or Delphi code and any of the other languages ó you can run it right away. Given that you have built the screens already, you can hook in your code and run it for the user right away, or let them take it away and test it at their own leisure. You can sit down with the user and ask "like this?" without cringing if the answer is "No". It is no more work to change the code than to change the documentation and you have the further benefit that you don't need to ensure that the code and documentation remain synchronised.
So my question stands. Why do we persist in demanding a technical specification document for a program that is to be developed with a RAD toolset? This practice reminds me of the times when it was necessary to hand in a flowchart with a working program, at a time when compile-link-run turn-around was so fast that it became quite feasible to sit at a terminal and get your logic straight using sample runs in a much shorter time than drawing a detailed flowchart would take. It was much easier to rework your code on a terminal using an editor than it was to rework (some would say recast) a flowchart using a pencil and template. Is there anyone who was programming in the seventies who can honestly say that they never drew a flowchart after the program was completed? I didn't think so.
But don't you youngsters get too smug. Right now, my friend of a friend, the one I mentioned above, is busily doing her job using RAD tools and a RAD approach, while some-one else is sitting with her writing up a technical specification after each module is complete. Honest Injun! Again, what is the point?
So, do I recommend we throw away all our documentation after the Scope Definition if we are going to use a RAD approach? Well, no, not quite. You will definitely need to spend some time developing a data model. In fact, I believe that the importance of the data model is highlighted in a RAD environment. It may well be that the one enduring design technique will be data modelling; certainly there is no development methodology that I have seen that is not enhanced by paying sufficient attention to the data model.
Additionally, you will need to have some sort of system to document design decisions and the reasons for them, as well as the agreements (and the arguments) between the development team and the users. Why did we choose this approach? Who authorised the adoption of this option? Without a functional or a technical specification, we no longer have something to point to as the final arbiter of what is right and what is wrong. This may make many developers and clients uncomfortable but it does mean that we should finally see the end of those systems that are built to accurately implement a flawed specification.
As a related issue, we will need to have some system of monitoring scope creep. We will need to keep both usersí and developersí feet on the ground and ensure that we are going to finish the project and come up with the deliverables within budgetary and time constraints. Project Managers will need to be far more pro-active and technically savvy than they might have been in the past, just as the best ones currently are. Just looking at Gantt charts will not cut it in the brave new world. You will need to talk to the users and to the code cutters, or you will find that the project will leave you behind.
Are there any problems with RAD tools and techniques? Of course there are. First-generation RAD tools like Visual Basic made it quite difficult to share and re-use code. (Let me point out that I like VB a lot. I am not, however, blind to its limitations.) There are many things that you simply cannot do using Visual Basic unless you use a DLL or VBX, neither of which can be developed with VB. This means that you need to use C or C++ (a decidedly non-RAD environment if ever Iíve seen one) or else depend on the code quality of a third-party supplier. Some VBXs are terrific; others are a software defect waiting to happen. And has anyone come up with a workable version control system for VB? The only one that I have come up with is purely manual. Most of the automated tools I have seen are not worth the cost of the packaging they come in when used with VB.
Second generation RAD tools like Delphi address many of these limitations. While retaining the speedy development of user interfaces first exemplified by VB, it removes most of the limitations. (I was going to say all of the limitations ó I certainly am not aware of any ó but I thought that this might be premature for such a new product.) You can get at any API function or intercept any Windows message. You can create DLLs (and VBXs). You can create your own components and share them across projects. You can share and reuse code through compiled units and, more importantly, through the OOP techniques of inheritance, encapsulation and polymorphism. You can hook it into third party version control systems because it has a documented tools API and comes with built-in hooks for PVCS. And, being compiled to Intel machine code, it is significantly faster. No, Delphi is not perfect. But I have not seen a better tool for RAD development in Windows. Between VB and Delphi, I have all the tools I need for RAD development.
But whatever tool you use, RAD can offer you many benefits if you are willing to look at your entire development methodology and tune it to support rather than hinder the RAD approach. If you arenít willing to do this, then why bother? One line of VB or Delphi code takes about as long to write as one line of COBOL.