This chapter has been excerpted from the book "Flow-Based Programming: A New Approach to Application Development" (van Nostrand Reinhold, 1994), by J.Paul Morrison. A second edition (2010) is now available from CreateSpace eStore and Amazon.com. The 2nd edition is also available in e-book format from Kindle (Kindle format) and Lulu (epub format). To find out more about FBP, click on FBP.
For definitions of FBP terms, see Glossary.
|
Material from book starts here:
I would now like to describe the way generalized FBP components are made reusable, using one additional, very powerful FBP mechanism, the "Initial Information Packet" or "IIP". IIPs were developed by E. Lawton of my old department at IBM in response to some of the problems we ran into using DFDM's parameter facility.
Let's say you have built a component like one of the ones described in the previous chapters. Let's call it "Select". It should be clear from the foregoing that this component can be used in a variety of contexts, as long as it is sent data in the format it expects. Because a component communicates with the outside world only through data being sent to or received from its ports, it can be held in object form, and never modified. Such reuse is often called "black box" reuse, to suggest the idea that the user cannot see the insides of the component. In addition, since the "black box" never needs to be modified, once its developer gets it working, it can be relied on to work correctly in any context. This is the converse of "white box" or "clear box" (source level) reuse, which is what most so-called reuse tools provide today. This type of reuse is easy to provide, but in my view doesn't buy its users much. It may reduce the cost of developing new code, but the amount of net new code which has to be maintained still increases. Furthermore, if a bug is found in the reused code, there is no easy way to tell whether it is safe to fix all instances of it - if you can even find them (with some reuse tools you can't even do that).
We will of course have to tell our black box Select component which fields to select on, as, otherwise, it will only be able to select on whichever fields were hard-wired into it. Suppose we want to tell it that it is to do its selecting on the contents of a 6-byte field starting at offset 23 in each incoming IP, and also want to give it a list of acceptable field values. In classical programming, we do this kind of thing by having the calling program specify parameters. In FBP we do something similar, but there is no user-written calling program in which to specify the parameters. Instead there is a way for the application designer to specify this information, right in the application structure definition. This mechanism is called an Initial Information Packet (IIP). We also need an additional port on the component (let's call it OPTIONS - it can have any name we want). An IIP can be generated as part of the structure and associated with the chosen port.
Once the process is started, an IIP is turned into an ordinary IP by the component issuing a "receive" service call against the port the IIP is connected to. This has the added advantage that the OPTIONS port can also be fed by an upstream process instead of an IIP, so component options can either be decided at structure-building time or deferred until execution time, without any modification of the component being required. What the component sees when it does its receive from the OPTIONS port is an ordinary IP. In what follows, we will sometimes refer to this as an "options IP" - options IPs may start out life as IIPs or may be generated by upstream processes, but their function is primarily to control execution, rather than to carry data (obviously this is not a hard and fast distinction, and you may find a need for IPs which combine both functions).
One last point about options IPs: a major decision for the component designer is whether to make them free-form or give them a fixed layout - the former will generally be easier to specify, but is going to cost more processing time to scan for delimiters, convert numeric values, etc. However, since this processing is usually done just once, at the beginning of the run, it may not be significant in the context of total processing time. THREADS opted for free-form IIPs for its "off the shelf" components because of the ease of use factor. One other thing you should consider if you choose to use free-form IIPs is that you will need to decide on delimiter conventions, e.g. you might opt for commas or blanks, and you might decide to use brackets to group together sets of option values. This in turn means some convention will be needed for specifying character strings which might contain delimiter characters as valid values (the old "quotes within quoted strings" problem).
Fixed-form options, on the other hand, will take less machine processing time and avoid the above-mentioned problems of delimiter conventions. Conversely, they are more likely to result in alignment errors, and are harder to make open-ended. There are other options, however, within the fixed-form category:
- use some front-end tool to generate the fixed-form IIP, or
- use a descriptor (see Chapter 11 on Descriptors) to access the fields in the options IP.
Let's say we want to write a generalized selector where the column number, field length and permissible values are received from an OPTIONS port at execution time. Using a long, shallow rectangle to represent an IIP (by the way, you can't attach IIPs and regular processes to the same connection), we might represent our selector with its options IIP as shown below. The field being selected on starts at offset 23, for a length of 1 byte. IPs with a value of A at that position will be sent to the zero'th element of port OUT, IPs containing a B go to the number 1 element, and so on.
Figure 5.1
In a structure diagram, it is often useful to be able to show parameter values right in the picture. Note that in this example we have shown a free-form IIP.
To convert this diagram to use a connection instead of an IIP, change the block feeding OPTIONS to a component, and attach it to the OPTIONS port, as follows:
Figure 5.2
By way of comparison, in DFDM processes were parametrized by means of a variable-length character string (2 byte binary length followed by a character string) passed to the component at activation time (analogously to the way a parameter string is passed to a job step in IBM's MVS). This meant that parameters specified in the network and parameters coming from an upstream process could not be handled in a uniform manner.
There is a general problem of passing parameters from the "outside world" into networks. In DFDM's approach to parametrization, external parameters were only passed to the outermost structure, and there was a system of "inheritance" which allowed lower level structures or component access to parameters at the next higher level. Although we haven't tackled this problem in THREADS yet, the IIP technique should make this whole area much simpler as special components can be written to obtain external parameters and pass them into the network like any other IPs.
At a more general level, parametrization of components may be thought of as a spectrum, running from low to high. Low parametrization (few or no parameters) occurs when a component has no variability - either because it is custom-coded for a particular application, or because it is so simple that it always does the same work in the same way. For instance, there is a very useful component in all existing FBP systems which simply accepts and outputs all the IPs from its first input port element, followed by all the IPs from its second input port element, and so on until all the input port elements have been exhausted. In DFDM it was called the "Sequencizer" (some of my friends like to play fast and loose with the English language). This component is often used to force a sequence on data which is being generated randomly from a variety of sources. One example might be control totals being generated by different processes which you then want to print out in a fixed order on a report. You know the sequence you want them displayed in, but you do not know the time at which they are going to be created. Schematically:
Figure 5.3
Its function is so simple that it doesn't need an options port. Now you may be asking, "Why not simply take all the incoming data streams and merge them into a single input port?". The answer is that you can do this, but the effect is somewhat different. What happens in that case is that the incoming IPs are merged in a "first come, first served" sequence, which is not what you wanted. However, there is a down side to this function: because the data from input port element 1 is held up until Concatenate knows that port element 0 has been closed, and so on for the remaining ports, there is the potential for deadlock. Deadlocks are not as bad as they sound, and there are well-established ways of detecting places where they might occur, and of preventing them before they can occur. In FBP, deadlocks are viewed as a design-time problem. We will be talking about the cause and prevention of Deadlocks in a later chapter (Chapter 16).
The Select component has a median level of parametrization, and most of the components we will be talking about in this book are similar, but there are occasional components which have so many parameters that they can almost be thought of as mini-languages. If parameters are replaced by, say, rules held on a data set, or even a large IIP, then you really do have a mini-language. A later chapter (Chapter 17) goes into more detail about this whole area. We will also be describing in Chapter 18 an approach to a generalized component of this type which should be able to handle a considerable part of the logic of a business application.