A Story

Cognitive Support for Sy= stems Engineering

Jim Brander

Interactive Engineering = Pty Ltd

Sydney Australia

info@inteng.com= .au=

<= o:p>

Abstra= ct

A system for reading system specifications and converting them into = an active structure is described, as well as the extensions to Constraint Reasoning to allow its use on a complex dynamic construct. The resulting structure is intended to retain the entire meaning of the specification. So= me of the problems that needed to be overcome are mentioned, as well as the benefits of using the same structure and methodology throughout the process. The possible effects on the Systems Engineering process are noted.

Introduction<= o:p>

Specifications can be complex, and people are limited in the complexity they can handle – t= hey can handle no more than five to nine pieces of simple information in play at once. When a person encounters a new specification, this limit is overwhelm= ed and many mistakes are made. There would appear to be a need for a system to= support requirements analysis by automatically extracting semantics from specifications. We have used specifications, a small one for an airport veh= icle and a large one for a communications and processing system as test examples, with the knowledge representation being active in reading the document and extending itself.

A Story

Simplicio: (rushing into HR = store) This new specification is a real problem. It has= ten thousand things to think about. What have you got for me?=

Septimus: Our best mo= del does nine.

Simplicio: Nine thous= and – that might do at a pinch.

Septimus: No, nine. I= t can only handle nine things at once. It will just assume nothing else changes w= hile it works.

Simplicio: But that&#= 8217;s no good. We know lots of things will be changing.

Septimus: It also poi= nts out “I am only human” at regular intervals. You can claim ceteris paribus – that excuse has worked for economists for decades.

Simplicio: Yes, I can= see how well that works. Everything works until it doesn’t, and then it crash= es and burns. We need to do better.

Septimus: Well, there= is our Sixpack model.

Simplicio: If nine is= near useless, wouldn’t six be worse.

Septimus: Unlike our = Nine model, Sixpack is a combination of a person and a machine. The person handles the more subtle aspects, the machine monitors everything that can change and controls the interaction with the person to prevent overload, so six pieces of information in play from the person is enough.

Simplicio: What’= ;s the catch? Is it more expensive?

Septimus: No, quite a= lot cheaper, and without the tantrums. It does take more time to get started, because Sixpack needs to know all about the specification, whereas a Nine w= ill already be making decisions after learning about 0.1% of your problem. At t= hat point, it has already gone into overload, with ten things to worry about.

Simplicio: How does t= he machine know what can change?

Septimus: It reads the specification.

Simplicio: But I thou= ght machines couldn’t read text.

Septimus: In general = they can’t, but specification text is unusual. The language is relatively formal and “dumbed down”, it doesn’t break off mid sentence and start talking about what was on TV last night, the domain is well-bounded, there should have been an effort on= the writer’s part to minimise ambiguity, and both the person and the mach= ine learn together, with the person helping the machine read the specification,= and vice versa. Is there sufficient value in the specification to make that worthwhile?

Simplicio: The specif= ication covers a multi-million dollar job, and there were fearful blunders the last time we attempted it, so there won’t be any worries about a bit of a delay – what are you talking – days, months, years? =

Septimus: It might ta= ke a few weeks to put in enough knowledge for a big specification – what typic= ally happens then is the specification goes back to the drawing board to fix all= the mistakes Sixpack has identified – mistakes that would otherwise have gotten out into the project. Then everyone can see much more clearly what is involved. And when I say “see”, I mean see, because the machine= can show a graphical representation of everything in the specification – = all the relations and their interactions. We were talking about a maximum of ni= ne things before. Most engineers are weak when it comes to words, whereas if y= ou can use their visual channel to show them the interdependencies, they are comparatively strong.

Simplicio: What happe= ns with changes? If there is a visual representation, won’t it become out of date, like all those CPM diagrams I see pinned up, where nobody can be both= ered going back and changing the model?

Septimus: The machine= reads the new text and changes its internal structure – the visual representation is just a display of that internal structure- maybe an hour, depending on the size. That’s the point of reading text – it is= far faster than any other approach to changing complex structure.

Simplicio: Why didn&#= 8217;t you mention Sixpack first.

Septimus: Our Nines h= ave their pride – they would rather do things using their native ability - they still haven’t come to terms with cognitive assistance from a machine, probably never will. We look down on Sixpack,<= /span> call the person Joe behind their back. We only use Sixpack where there is no other choice.

Simplicio: We have to= certify that the system is safe to operate. Having ten thousand things able to change and a control unit limited, at best, to nine things changing at once sounds decidedly unsafe. How did we get away with it for so long?

Septimus: There seeme= d no alternative, or at least up until we got this new model.

Simplicio: Wait until= I tell people we can do the job properly at last.

Septimus: Can I menti= on that Sixpack comes from our project management offerings – people don̵= 7;t get upset about a machine doing that sort of boring work.=

Simplicio: Oh. So I shouldn’t say anything?=

An Analogy

In attempting to impr= ove the outcomes of projects, it became clear in the 1960s that people could not ma= nage more than about fifty activities without some tool to assist them – t= hey would become confused at the interplay of the different activities as the durations changed. An automatable methodology – Critical Path Method – was developed to assist in planning of projects. It provides a time model of the activity interactions, and projects with thousands of activiti= es are regularly planned with it. It is fairly crude, as it does not allow for= the free interplay of time and cost and risk in its algorithm, but it was a considerable improvement on unassisted human planning. The analogy with building a model for specifications is not perfect – activities can be treated as largely independent (they may share resources, but if they are t= oo strongly interdependent, the model breaks down), while the objects of a specification will usually be strongly interdependent. Concentration on just one aspect of the relations among the objects – say one of propositio= nal, existential or temporal logic, or the hierarchy of objects, would seem to b= e a mistake, as it is the holistic nature of the specification that needs to be modelled, there being no equivalent in a specification of the simple and powerful metric of the early finish date of a project. The analogy of a mod= el used as a tool to understand the real situation, where the model is used in= more than one direction, would seem pertinent.

Another example of a = problem overwhelming human ability is flying a helicopter. A helicopter pilot can be very busy, to the point now where the most hazardous manoeuvres are underta= ken on auto pilot. We are not suggesting an algorithm of this kind – it t= akes too long to write, it is narrowly focused, relies on some very strong assumptions (which need only remain valid for a few minutes) and is moving = the upper limit to no more than nine (a person can still make the manoeuvre, ju= st not with high reliability). But the same problem arises with specifications – if a person is overloaded in preparing or examining a specification, because there are too many things in play at once, the result can be far mo= re costly than a helicopter crash, while causation is much harder to track dow= n.

Systems Engin= eering

Before some readers s= cream “Artificial Intelligence!” and start rushing for the exits, we = will try to convince them that this is the domain of Systems Engineering. We will require a massively complex and intricate system, with thousands of pieces = of language machinery all working smoothly in unison, encountering situations = that cannot be predicted, and requiring to modify and extend itself as it deals = with those situations. As the dialog above outlined, the system should not just emulate the abilities of people, but have a quite different set of abilitie= s. This sounds like Systems Engineering, or at least the engineering of a comp= lex system. We understand that practitioners of SE may not wish to deal = with a system that is vague, unpredictable, with very narrow focus and severely limited in its capabilities – people, in other words. We promise the system we describe is deterministic (that is, predictable in its behaviour)= , although other aspects, such as self-extensibility, may be unfamiliar.

We will briefly descr= ibe the evolution of the system, to assure readers that this is engineering modelli= ng, not its dissolute cousin, artificial intelligence. The system started as CP= M. CPM uses essentially MAX and MIN operators – find the maximum time for the predecessors of the activity’s early start date, add the duration= of the activity, and broadcast the result as the activity’s early finish date. Repeat in the reverse direction for late start date. If we conceptual= ise an activity as an object with attributes, one of which is duration, we get = the arrangement shown in Figure 1.

Figure 1 - CPM Structure

It would be useful to= support numeric ranges on the durations and dates, which means we don’t need = two passes to find early and late dates, we can hold information about forbidden periods between those dates, and small changes not on the critical path stay local, so we have changed all the operators to work with ranges.=

We might have alterna= tive ways of doing things, and we can’t choose between them until we get i= nto the project. If we add a logical connection to the alternative activity, and make it an operator like plus or multiply, we can have the activity sit in a quiescent state. If it finds there is insufficient time to fit its duration= , it signals false and another activity is forced true through the XOR, shown in= Figure 2.

Figure 2 - Alternative Activities

If both alternatives = remain viable, we can choose whichever is the lowest cost combination, where the choice of one activity may force another anywhere else in the structure and= not already started, not to start. The logical conne= ction on the activity is controlling its existence, and we see the XOR should beh= ave the same whether it is given a logical or an existential state. If we have dedicated connections for both logic and existence on relations, while logi= cal connectives will accept either, and we allow inheritance of properties for = the object, and a few other things, we have a controllable relation as in “the system shall allow…”, and we are still in engineering modelling. This does become important in developing such a system – it may have built structure to represent thousands of words, but if there has = been no change to its knowledge structure before it started reading, when it arr= ives at a particular word in the text, it will use exactly the same structure element in the same way (the very same one – element no. 351867 - the system is “perfectly” deterministic), which may disappoint peop= le hoping for something that behaves like us, but is useful in creating a reli= able representation. The structure is identifiable – the word in the text, where possible, becomes the object in the structure. The system is also self-extensible, in that its knowledge structure is growing as it reads, ma= king it sound as though it should be nondeterministic. Reading specifications requires both massively intricate and highly reliable machinery – the heirs of the engineers who built clockwork or the Jacquard loom would seem = the most likely creators of such a system, and work on it would seem firmly grounded in Systems Engineering, not other disciplines without a systems approach.

OMG SysML also offers modelling for systems, and declares the same benefits of modelling that the current paper describes. Its genesis was different – it came from a sequential source, being a description of a computing process moving only forward in time. The connections between objects or activities are directed (they have static arrows on them), which is inappropriate for describing requirements. If we take a simple statement of propositional logic (the obj= ects are irrelevant)

If it is raining, the road is wet.

If the road is not we= t, it is not raining. This is Modus Tollens &= #8211; it is obvious to any human (without logical training – the mother tel= ls the small child “If you= are good, I will give you a lolly” and the child replies “I donR= 17;t want a lolly”, fully aware of the effect of negating the consequent),= and yet something so basic is beyond the grasp of SysML (it may now be more obv= ious why a base of CPM can lead to an undirected model structure – CPM used its model structure in more than one direction). With even propositional lo= gic closed to it, SysML cannot support more complex logics – the person m= ust do that. While initially alluring, in that it suggests that all the logic o= f a specification can be laid out in a diagrammatic form, the result is that the human, after spending days in creating a static directed structure, must st= ill support in their head all the concepts the directed structure cannot descri= be. We gave an example of alternative activities above, where the direction of information through connections is not known a priori – either alternative activity turning off can turn the other on through an XOR. The difference between SysML and active structure is exemplified in the XOR ope= rator – it does not know which are its inputs and which are its outputs until inputs arrive. The same behaviour in an implica= tion operator results in Modus Tollens (it may seem a simple extension to SysML,= but it would destroy the dataflow assumptions on which SysML and UML are based). That is, SysML is a modelling language for a simple sequential program, not useful for systems which do not map to a program, but instead contain autonomously active elements (“But I can easily model the two phases = of a CPM algorithm in UML” – yes, but the knowledge that it is the s= ame information used another way resides in you, not in the model). A similar situation, of connections which are uncommitted to being input or output, occurs in any constraint reasoning problem [Cras] – constraint reasoning is = a good metaphor for design, with possibilities flowing backwards and forwards in a= ll directions at many levels, connections switching between being inputs or outputs, objects and relations switching between existing and not existing (structure switched to not existing is not the same as negation of existence – the relation in “He can’t swim” has its existence negated, but switching the statement to not existing makes the statement it= self “disappear”, as in “section 3.2 is null and void if…”). The definition of requirements precedes the design phase= , so making the requirements structure static and directed (where it need not be) would seem a severe conceptual error. Some requirements are constraints on = what can be constructed, taking us far beyond the static structure of SysML. In = the active structure we are describing, states move through the structure, but = the directions are transient and are calculated by the operators in the structu= re, not imposed on it from outside. As an extension, operators in the structure= can build new structure, providing a dynamicity of approach to match dynamic problems of command and control. Unnecessary directionality is a blight on = many other areas – we mentioned the ceteris paribus of economic models, wh= ere only a few things are allowed to change at once and the larger picture stays hidden, because putting more things in flux would destroy the assumed directionality (and force the modeller to juggle too many things at once, j= ust like the helicopter pilot under stress).

A Good Applic= ation

If one were to look f= or a first application of Natural Language Processing (NLP), one would presumably look for:

Text that has some identifiable and unifying purpose
A reasonably bounded domain of knowledge
A small to medium size document with a high= value
Formal, clean and relatively unambiguous te= xt
A simple basic form
An application where slow reading time (abo= ut that of an attentive human) would not be a problem
A long life for the structure that is produ= ced by reading
Some human assistance to the machine while = it is reading the text would be acceptable

These are the earmark= s of a specification. A specification is about the aspects of some one grounded th= ing, not a stream of consciousness or snippets of news from anywhere round the globe.

It is essential that = the readers of a specification have sufficient knowledge to understand it. Putt= ing this knowledge in a machine is a good way of exploring what knowledge is necessary as a prerequisite. Of course, the specification provides new knowledge, and it will be assumed that the machine can acquire this knowled= ge from the specification just as well as a human reader.

Function and Performa= nce Specifications (FPS) range from a few pages to a few hundred pages – within the range of machine memory, a few gigabytes. The language is typica= lly clipped and clean. The FPS may be deliberately vague in how the specification is to be implemented, but should be unambiguous in its description of what a success= ful implementation must provide.

The basic statement i= n an FPS follows the simple form:

The cat shall sit on the mat

The reading system we= are describing makes no assumptions about the form of statements – it does not look for “shall” and ignore everything else – the int= ention is that the reading system is general purpose, and the representation of the semantics, whether they be actuality or intent, = should be accurate and complete. The simple form of the basic statement does mean = that very complex forms will be rarely encountered.

Reading time may rang= e from a few minutes for a small specification to several hours for a large one. The project for which the specification serves as a foundation might run anywhe= re from months to decades, so hours spent in machine reading the specification= and validating its structure should not be an embarrassment. The reading system= can be used through the requirements elicitation phase, so the prerequisite knowledge will already be available for the specification’s first dra= ft.

While not wishing to = belittle the difficulties in reading a specification, it is not the most difficult t= ext that needs to be read, and yet there is great value in ensuring that everyo= ne reads it the same way, making it a good application with which to start to introduce cognitive support into Systems Engineering.

Somewhere to = Go

Natural Language Proc= essing sounds worthwhile, until one asks what is the result of= the processing. A specification describes relations – those relati= ons may be between logical states, existential states, time, or objects or relations. It seems that all those relations should be represented in some = way. Herein lies a problem. Most people are familiar = with propositional logic – A implies B. Some readers may have heard of predicate logic, where logic links objects. This sounds as though it would = be suitable, until one is told that predicates may not be parameters of other predicates. That is, a statement such as

“The equipment shall allow trainees to perform diagnostic tests.”

c= annot be represented in predicate logic, as “perform” is itself a predicate (as are “diagnostic” and “test”). Higher order logic, which does allow predicates as parameters of other predicates, is both “not well-behaved” [Wikipedia-higher order logic] (a polite mathematical term for unusable) and opaque. A paper by Cant describes an approach to program verification using= one form of higher order logic. We see little merit in a person who may be unfamiliar with the subject matter of a specification turning it into somet= hing opaque to almost all users, while crushing the logic in the specification to satisfy the demands of a limited logical language. Instead, we should be seeking to turn it into an internal representation – a model - which = is as powerful as natural language. There has been considerable work on representation of knowledge – much of this has been an attempt to turn text into diagrams for humans to peruse, such as the work of Sowa - rather = than for a machine to use for its own purposes. A machine reading text needs a structure in which states and objects can propagate – an active struc= ture – rather than a diagram. A diagram is necessarily a simplification – there may be thousands of connections on one object, and millions of connections overall, which would overwhelm us. We want the machine to assis= t us in dealing with the full complexity of the problem, not with small aspects = of it, which we can manage quite well on our own.

How Hard Can = It Be

We know the upper bou= nd on the complexity of reading specifications – the same upper bound that = we are trying to get around with the provision of NLP on an FPS - no more than five to nine pieces of simple information in play at once. This suggests it= may not be as hard as we would like to think. Some of the problem areas:

Packed Noun P= hrases

A specification will typically include noun phrases up to eight words in length. The noun phrase needs to be picked apart and built into a structure which honours the relat= ions involved. An adjective may relate to the word immediately to its right, or = to the word at the end of the phrase. Some transformative relations cause a new object to be created, and this new object becomes the target of relations on the left. An example is “virtual vehicle”. A vehicle is a physi= cal object with mass and extension – a “virtual vehicle” exis= ts in a computer – it may be an image or a database cipher, with no mass. The combination of the words creates a new object, to which other words in = the same noun phrase refer. In the worst case, choosing the appropriate subject= and relation for each word becomes a dynamically constructed constraint reasoni= ng problem.

Prepositional= Chains

Prepositional chains = (we include participials and other “stuff” caught up in the chain) carry a large amount of the semantic content of a specification. A well wri= tten specification may have a chain limit of four or six, a heavily amended contract may have twenty. While the chain is short – = one or two prepositions – it is relatively easy to unravel. As the chain lengthens, it becomes increasingly difficult to be certain as to which obje= ct to its left the preposition refers, until it becomes a hypothesising “best fit” problem using relation modelling and chain decomposi= tion rules.

Self Referenc= e

The specification may= refer to itself, or to other documents. A document structure needs to be built at= the same time as the document is being read, so document references can be resolved. The internal reference may be vague – “below” &= #8211; or seemingly precise (in clause 1.2.4, a reference to “the restrictio= ns on performance in 3.2.7.a”) – the actual meaning can’t be known when the reference is encountered, so fix-up may be required, and the restrictions may have moved to 3.2.9.c, so the system must validate every reference. There are layers of fix-up jobs – some being scheduled at = the end of the sentence, such as anaphora in the sentence, some at the end of t= he section, or at the end of reading. This is no simple scheduling task –= ; a subtle modelling resolution may mean bringing forward anaphora resolution, = and that may require bringing forward a modelling resolution elsewhere. Having something as a base that started out as a scheduling tool is no bad thing.<= o:p>

Defined Terms=

Even a small specific= ation will define terms. There are various methods – an acronym in brackets= , a quoted and initial capital form in brackets, use as a heading, a fully capitalised form, sometimes a formal textual definition, where the creation= of the relation (“is defined as”, “shall mean”) also creates the dictionary entry. Sometimes the defined term is one word and le= ft uncapitalised, meaning it should overlay the word in the global dictionary.= The defined term may be used before it is defined, and it can be difficult to t= ell whether the defined term is meant. Sometimes the term is defined and then u= sed a few words later in the same sentence. The use of the defined term may be bounded to a section, to prevent collision with the same term used with a different meaning in other parts of the document (another reason for buildi= ng the document structure). The likelihood of encountering internally defined terms forces the use of dynamic methods – a local dictionary, rescann= ing, fix-up.

Ambiguity

A specification is no= tionally unambiguous, but that is only to someone who has the knowledge to understand what it is saying. Some of the causes of ambiguity:

A word can be a noun or a verb – costs.

A word can be a subordinate conjunction or a preposition – as.

A word can be a participial, an adjectival participle, a verb or part of a ve= rb with a distant verb auxiliary – delivered.

A coordinate conjunction can group adjectives or objects (Jack and Jill) or relations (… shall repair or replace the instrument) or join clauses.= The conjunction may assert simultaneity or sequence.

Anaphora provide a rich source of ambiguity – it, that, some, which.

These things taken to= gether, many sentences can be ambiguous, or at least need hypothesising about sever= al meanings to resolve them to the most likely one. Sometimes a singular meani= ng cannot be resolved, and multiple meanings need to be carried forward. A bou= nded domain of discourse helps – the “flight” of flight contro= ls of an aircraft is unlikely to have the same meaning as flight risk for a criminal. The bound reduces the number of meanings that need to be modelled= .

The Woods for= the Trees

In long sentences it = is easy to get bogged down in the detail and not see connections at long range. An outline of the sentence is constructed in parallel with the parse chain, wi= th only the components that determine the “shape” of the sentence – verbs, clause introducers such as subordinate conjunctions, relative pronouns, etc. This outline is used to determine the context of various obj= ects – participials, noun/verbs, conjuncted infinitives. The outline of the sentence allows more complex forms to be handled analytically, still with a backstop of hypothesising.

Hypothesising=

In constraint reasoni= ng problems that occur in design or planning or scheduling, a system will often need to try things out and observe consequences at long range, analysis by itself being insufficient to supply a solution – a simple example is placing eight queens on a chessboard [Wikipedia – eight queens puzzle= ] so they do not attack each other (this example, as with most constraint reason= ing problems, is static – everything is known about its structure beforeh= and, to the point where the problem can be compiled). Reading sentences is no di= fferent to other constraint problems where analysis cannot be used – “it could have that meaning, which would have the consequence of… - no, t= hat can’t be, it must have a different meaning”. The system needs to stop building analytically, and start building hypothetically, undoing connections if it doesn’t work. This is constructive hypothesising in= a dynamic structure – something gets built or destroyed – rather = than choosing objects from a set and propagating them in a static structure.

A reading system need= s two different approaches to hypothesising – one approach is where it tries different possibilities to forward the building of structure, only one of w= hich will lead to a full build. The other approach is where several paths will l= ead to a successful build, and it needs to choose between them based on “= best fit” – which alternative does less grammatical damage, or more closely follows sentence order (an unreliable guide, but sometimes the only one) or disrupts the knowledge modelling less (sentences like this should be encountered rarely in specifications, but also indicate the range of abilit= ies needed in a reading system). Where hypothesising is not successful, it can = be attempted again after more is read – another fix-up job to be automatically scheduled.

Various Forms= of Logic

A specification will = usually have very little propositional logic – it may only be that the senten= ces are notionally “ANDed” together, with “if…then̶= 1; used to describe a temporal relation. There will be a smattering of existen= tial logic – “It shall be possible…”, “the labels = can be easily read”, “the system shall be capable of….”. Some temporal logic – “when…”, “…prior to…” or “… an intermediate stage in…”.<= /span> Operations on sets – “at least one of each…”, open sets “including, but not limited to”. Some number handling (surprisingly little, given the educational emphasis on arithmetic and algebra). And the dominant form – relations on relations and objects. What is interesting is that there is this jumble of logics, with almost no attempt to teach the users of specifications about them, perhaps because teaching them separately would lead to confusion rather than understanding.= If we are to model specifications, then all of these logics need to be capture= d in a synthesis, not viewed as separate and unconnected frameworks. The synthesising element we are using to represent a relation is shown in Fig= ure 3.

It has an object (ToS= ell) attached to an operator (RELATION3), the object providing a connection for inheritance and as a parameter of other relations, and the operator providi= ng connection for logical and existential states, and the parameters of the relation. The operator manages the relation between logic and existence = 211; the logical state cannot be truer than the existential state. In this examp= le, the discourse provides a False to the existential connection, which results= in a False emanating from the logical connection (s= o the question “Did he sell the house” would be answered in the negative). ToSell is an action, with an inverse ToBuy, and the ac= tion triggers a change of state – a ToOwn rela= tion (a state, not an action) will be terminated by ToSell<= /span>.

There seems no need to differentiate between objects and relations – “He processed the film. The process took hours.” – except that relations have parameters, which can be other relations. Conceptualisi= ng relations as objects allows their properties to also come through inheritan= ce, simplifying and unifying the knowledge structure.

Figure 3 - Single relation

The operators of clau= sal relations like “think”, “tell”, grow an extra logic= al connection, allowing them to pass logical states to relations they control,= in the same way that the sentence relation is passed a logical state by the discourse. Each sentence ends up with a logical spine, connecting all the relations formed by the sentence, including those in noun phrases and prepositional chains (if the sentence as a whole isn’t true, then we can’t assume the relations within it exist). Temporal logic is handled through attribute objects which define time windows for possibility and actuality, the time objects of different relations being linked together to provide sequencing or simultaneity. The result is a highly detailed logical structure, allowing its representation of the semantics to be accurate and complete. An example of a specification statement is shown in Fig= ure 4. The “allow” has a logical true state coming to it from the discourse, and is asserting existential true states to the “load”, “edit” relations through an ANDed group (inheritance and time control links not shown). The structure is not just a human viewable representation, but is also working machinery, which the mac= hine uses in its reading, and it also uses to answer questions about the specification.

Figure 4 - Multiple relations

How Useful Is Grammar?=

Most of the work in NLP has been devoted to an approach that is driven by grammar, such as the work of Pollard. A multi-st= ep pipeline process is typically used, in that words are turned into parts of speech, the string of parts of speech is parsed, and the parse structures a= re used to show the form of sentences. Many words can be multiple parts of spe= ech. Tagging of words for parts of speech can present statistics for the likelih= ood of a word having different parts of speech, statistics which are usually ba= sed on adjacent words. Sentences of complex text are full of little asides and prepositional chains, so adjacent words may not be semantic neighbors. Some examples:=

“… to bol= dly go….”

“It shall in the interests of fairness exclude…”

“the event times (departure and arrival) shall…”

In these examples, the adverb or prepositional c= hain internal to the verb or parenthetical comment is cut out and the cut healed= , so semantic neighbours become actual neighbours. For the second example, the reader may consider that commas should be used to aid in understanding text. They are used this way, but their use is often undisciplined, making them unreliable or confusing without additional information obtained from semant= ics to either confirm them or override them. A specification may be describing = something that has not been described before, so a statistical approach based on simi= lar text may not be useful in capturing the meaning. It sometimes occurs that t= he same word is used as different parts of speech in the same sentence. One co= uld argue for an approach that uses the most likely combination of parts of spe= ech until an error occurs, but the clarifying error may occur in the real world, not just in grammar (it is called intellectual laziness when people do it). = Pruning away the alternatives using inferencing is more time-consuming, and more rigorous.

Parsing a complex multi-clause sentence is much = like developing a mathematical proof – once developed, the steps can seem obvious and inevitable, but the development process involved a great deal of jumping about and surmise. Of course, the system being described here uses pruning of alternative parts of speech based on neighbouring words – = the difference is that pruning is undone and redone if the chain of words is rearranged, say by cutting out an adverb, or a prepositional phrase between= a verb auxiliary and a verb, as in the examples above, or by the insertion of= an implied object such as a relative pronoun. In some cases, such as the prolo= gue or epilogue for an embedded list, the semantic neighbour of a word may be hundreds of words away, and requires special machinery to make it appear as= a neighbour.

Further complicating the problem of using gramma= r by itself is that the writer expects the reader to understand the semantics of what is written, so filler words can be left out. As example:

He left the engine running in the rain.

He removed the maintenance training from the pro= ject.

He upbraided the child playing in the drain.

He wanted the engine running by the weekend.

These sentences all appear to have the same part= s of speech, but engines “run” and “left” controls a relation (in the manner of a verb auxiliary), while maintenance doesn’= ;t “train”. “Want” supports an object and an infinitive (he wanted something to happen), but this is so obvious that the “to be” before “running” can be left out. The specification is not being written for a grammarian, but for a user who does not wish to wade through superfluity. Another example:

During A, B of C and D of E, F and G shall require….

Is the subject B, or B and D, or F and G, or B, = F and G? We can’t tell without semantics, and with semantics, we don’t stop to question how little grammar provides.

If we accept that a combination of grammar and semantics is required to understand the structure of a sentence, before its transformation into an internal structure, then the problem arises of representing the necessary knowledge and, crucially, ensuring the representation allows merging of new knowledge brought in by reading the specification. An ontology [Gruber] is a currently popular way of represent= ing knowledge, but ontologies as they are usually implemented are limited to inheritance and attributes (with links of IS_A a= nd HAS_A), and their structure is static, with no concept of self-extension through the reading process. A specification describes the relations among objects (“shall allow, perform, conduct, depict, = navigate”). If we allow the generality of relations instead of the limitation of typed links, there are thousands of relations used in specifications, many with multiple meanings, and many which are synonymous through rotation or reflec= tion of their parameters (“buy” and “sell” are mapped synonyms for each other). As a further example, one would need to provide mapping for various meanings of a relation such as “omit” (he omitted the data from the report, the report omitted the data) and include mapping to relations such as exclude, not include, leave out, not put in, e= dit out, redact (the mapping of meaning among relations is one reason the knowl= edge structure is so dense, daunting unless the domain is bounded). People are taught a little about synonyms, and left to intu= it the rest. We have to understand all that we were required to intuit about langu= age, to teach a machine. It is possible to enable a machine to use induction on examples, but it would make many errors before its modelling came up to a h= igh enough level of reliability. Reliability is a subject little touched on in = the reading of specifications, but if it were to be specified, across the transformation from text to internal representation, and then in the thousa= nds of relations in a specification, each with several up to several dozen meanings, and with potential error sources also in propositional, existenti= al, temporal, and locational logic, what would we s= ay? Our current goal is to achieve 99% overall, and the only way that can be achieved is to use every possible piece of information, letting nothing sli= p by or be thrown away (which means it is slow - we would prefer to say careful)= .

The upshot is that grammar is used as a helper o= f the system, not as exclusive arbiter. It is a far more detailed grammar than pe= ople are taught, with approximately six thousand patterns. Operation of the gram= mar is interwoven with semantics. It can be used to eliminate structures which = are ungrammatical or to make explicit some objects which are implicit in the te= xt. Some statements in specifications will be ungrammatical, and can’t be changed, so some tolerance for error is necessary – for example, the = system fixes up apostrophe errors without complaint.

TestBed Specifications

We have bee= n using three FPS to develop and test the system - two small (7-10 page) specificat= ions in the aerospace arena, and one larger spec (74 pages), available here (a stripped down version - the origin= al had diagrams, but the text is definitive, the diagrams act as aids only). This specification describes a system of subsystems, with system-level connectio= ns made through document references, as

5.7.3.4. The On-Board Audio Announcer shall provide …as described in clause 7.10.3.19.3.c

The connections can be seen in Figure 5.

3D"wpe1.jpg

Figure 5 - System-level Connections

Diagnostic Tools

Once the te= xt is converted into structure, it becomes easy to provide diagnostic tools to se= arch for errors, inconsistencies, ambiguities, weak statements, general statemen= ts which clash with specific statements, actions without reciprocal actions ( = one can log on, but no means is provided for logging off). The analysis can be compared with the usual approach to complex machinery - develop tools to activate and measure its response to determine its health. It is much easie= r to do on a realised working structure than it is on text.

3D"wpe2.jpg

Figure 6 - Text as Machinery

How Necessary=

When one observes the misinterpretation of a specification given the slightest ambiguity in the t= ext, or the narrow focus which excludes germane information on a different page,= the ignoring of any statement that does not use “shall”, or the des= ire to cut a specification into ever smaller pieces in the hope that this will pro= vide a holistic view, a system which provides support for reading specifications seems essential.

Some engineers traine= d in Systems Engineering have been taught that conjunctions are bad, without understanding their role in building groups, or specifying simultaneous or sequential constraints. They will be familiar with the Boolean logic of programming, they may have had a brush with propositional logic, but otherw= ise their formal logical reasoning will usually be undeveloped, and we rely on their intuitive language skills for an understanding of the interdependenci= es described in the specification.

Conclusion

The human limitation = of five to nine pieces of information in play, the limited logical training of many= of those involved in requirements management and analysis, and the large improvement in understanding that can come from an accurate visual representation of the semantics, would suggest that a tool to provide assistance in managing a complex specification would be useful.<= /span>

We are suggesting tha= t the posing of requirements requires undirectedness, which natural language does very well. The reading system we have described= may be simple minded, but it is capable of reading requirements logically, with= out asserting direction. It displays the meaning it is using, or displays to the writer of the specification that it lacks the knowledge to discriminate amo= ng several meanings, leading perhaps to clearer English that does not bump against the= six pieces limit, and leaving no room for later argument about ambiguity in the specification – the structure is more detailed and precise than words allow, while not requiring crudity of expression as the cost of that precis= ion. It provides a holistic view of complex specifications, assists in knowledge dissemination, and plugs a significant weakness in the armour of Systems Engineering.

References

J. Brander and A. Lupu, Is mining of knowledge possible? Proc.= SPIE, Vol. 6973, 697307 (2008); DOI:10.1117/12.774079, = Data Mining, Intrusion Detection, Information Assurance, and Data Networks Security, Orlando,= March 2008

A. Cant, Program Verification Using Higher Order Logic, DSTO Research Report ERL-0600-RR-1992

J-Y. Cras, = A Review of Industrial Constraint Solvi= ng Tools, AI Intelligence, Oxford, 1993.

<= /span>

Eight queens puzzle – Wikipedia

Tom Gru= ber (1993). "A translation approach to portable ontolo= gy specifications". In: Knowledge Acquisition. 5: 199-199.<= /span>

Higher order logic &#= 8211; Wikipedia

Pollard, Carl; Ivan A. Sag. Head-driven phrase structure grammar. Chicago: University of Chicago Press, 1994

John F. Sowa, Know= ledge Representation: Logical, Philosophical, and Computational Foundations, Brooks Cole Publishing Co., Pacific Grove, CA, 2000

SysML – www.omgsysml.= org

Appendix – “dumbed down” language

The suggested language characteristics shown in Table 1 are commendable, but will fail almost immediately= in the description of complex requirements.

Open sets or simultaneity or some other reason will force a need for the flexibi= lity of natural language, even if only in flashes, meaning that whatever is representing the specification has to have the same flexibility, or the goa= l of complete and accurate semantic representation of the specification is not reached.

Characteristic	Ex= planation	Co= mment
Coherent=	The requirement add= resses one and only one thing.	The “one thing” may r= equire a complex of things to be analysed – fuel use or survivability or availability.
Complete=	The requirement is = fully stated in one place with no missing information.	This does not addre= ss the “completeness” of the specification.
Consistent	The requirement doe= s not contradict any other requirement and is fully consistent with all authoritative external documentation.	There is direct contradiction, and there is contradiction through structure – two requirements are each possible, just not together, or in meeting one requirement, another requirement will not be met, but the link may not be obvious
Correct<= /span>	The requirement mee= ts all or part of a business need as authoritatively stated by stakeholders.	The “authoritatively” seems like a vague adverb. If it meets part= of a need, does some other requirement also meet that part, so they are both correct, but one is superfluous. Does one requirement subsume another?
Current<= /span>	The requirement has= not been made obsolete by the passage of time.	Did it contain time= in its statement, or did it link to something which no longer applies – was that link noted in the requirements – if so, it needs temporal logi= c to express it
Externally Observab= le	The requirement spe= cifies a characteristic of the product that is externally observable or experience= d by the user. "Requirements" that are constraints should be clearly articulated in the Constraints section of the document.=
Feasible=	The requirement can= be implemented within the constraints of the project.	How to know this un= til the project is developed? Does making this requirement feasible make other requirements infeasible? Many projects are about testing the limits of wh= at is feasible
Unambiguous	The requirement is concisely stated without recourse to technical jargon, acronyms (unless defined elsewhere in the Requirements document), or other esoteric verbia= ge. It is subject to one and only one interpretation. Negative statements and compound statements are prohibited.	“one and only one interpretation” – some things cannot be stated concisely – with a new concept, it may require the absorption of pa= ges of dense text to understand the meaning. This particularly applies to cross-boundary requirements. “negative statements” – given the depth of English, an antonym can be u= sed, but the statement is still negative in purpose, as in “negative statements are prohibited”. “including but not limited to” – an open set is described with a negative statement. Compound statements can properly express a requirement for simultaneity or sequence.
Mandatory	The requirement rep= resents a stakeholder-defined characteristic the absence of which will result in a deficiency that cannot be ameliorated.	Frequently a situat= ion is encountered where there are several combinations of alternatives, any one= of which is mandatory in the absence of the others.
Verifiable	The implementation = of the requirement can be determined through one of four possible methods: inspection, analysis, demonstration, or test.	These methods are not independent – a particular requirement may require inspection of the system to observe how the requirement is be= ing met, analysis to determine what to test, and then test, and then analysis= of the result - the alternativ= es listed are for simple cases