SECTION III
Structure of the GCOS Operating System
The description
of the operating systems written in the 1970s did use a terminology that would be not easy
to catch by 2000s readers. The concept not known as thread, multi-threading was then
named, not without rationale, process. Since then, UNIX fans have assimilated a process to
an address space more than to a chain of control (while frequently the two concepts are
mixed).
We used to call "distributed procedures" procedures executed within the user
thread, while since the late 1980s, distribution was assimilated to the client/server
model, where services are run in separate processes.
The I/O architecture of GCOS is also quite different of most open systems. It followed the
concepts of "channel" introduced by IBM in the late 1950s and pushed the
architecture of its "channel programs" to a limit closed to real process
distribution.
Repartition of threads within a shared memory multiprocessor system was standard in the
architecture, even within the nucleus of the OS, thanks to the implementation of basic
functions in "firmware", what we can claim to be a micro-kernel.
Micro-kernel
GCOS includes a
micro-kernel implemented in "firmware", a set of shared procedures that
implement "synchronous services", a system process group that gathers services
implemented by threads and events handling and in addition to those services a variable
number of servers (process groups) that may be divided in standard system servers and
in special subsystems monitors or servers.
We are here
calling micro-kernel those basic functions that implement the architecture objects
that are threads, semaphores (including I/O) and procedure calls. They are loaded at
initialization time and constitute really a foolproof kernel layer. The term of
micro-kernel may disturb purists, because the services of that level are not all
implemented by micro-threads communicating only by messages with the software. Some
services are likely synchronous operating as procedures in the calling thread (P and
V-operations), others are implemented through micro processes polling I/O channels or
dispatching visible threads candidate for execution.
The way that micro-kernel is implemented is not part of the architecture and vary slightly
according to the processor models. That micro code is implemented as a subset of the
software visible instruction set (so it is interpreted by the execution micro programs),
bypassing error checks and using directly hardware features not part of the software
visible architecture. It operates faster than the same algorithms in the faster.
The system
process group operates within its own address space (J=0). It gathers system threads
implementing asynchronous services or, more exactly the asynchronous portion of those
services. Among those the I/O termination thread (s), the interrupt handling, attention
handler.
In addition to
that system process groups that has to stay permanently alive, several process groups are
needed for a general-purpose operation: the Job scheduler, the Input Reader, the Output
Writer. Reader and Writer are multi-threaded process groups that can handle several reader
devices and several printer devices. Other process groups are also present in a fully
configurated configuration: FNPS assuring the network interface, and a variety of
transaction and data base servers.
Some recent
systems (e.g. CHORUS) claim that a well architectured micro-kernel should pass parameters
by messages so that an operating system could be distributed by processors without shared
memory. Such configurations were not designed for GCOS, so GCOS does not pretend to
include a well-architectured micro-kernel. The word itself was not used by developers who
often consider the primitives as instructions implemented by firmware and operating in a
mysterious area of memory called under-BAR (for Barricade register) memory.
Program execution.
The external
specifications of GCOS were not a workstation operating system, such as Windows,
not a time-sharing server like Unix (just being in limbo at GCOS design time), but
essentially to serve as a batch operating system handling sequential files and
progressively direct access files. Programs were initiated from a batch input file,
initially a spooled card reader, later extended to multiple sources, some of them being
TERMINALS. Programs were scheduled according their physical devices resources
requirements (many initially were using magnetic tapes). Then they were loaded, performing
some last minute binding with resources and services. Then their thread was given control
and were dispatched the most efficiently possible, and eventually they terminate and an
execution report was printed.
Program
preparation
The overall
structure of program preparation was already available in the previous generation of
computers (for Bull: Gamma M-40, GE-140). A job is submitted to the system as a sequential
file of commands (JCL) and data (source programs, subcommands for processors, and possibly
user data to be processed). In general, the system being used most for business
application, the program preparation job ended with the update of libraries for subsequent
processing of files.
Program preparation includes an optional step of macro processing, a step of language
compilation, and a linkage step. Utilities such as the binding of several compile units or
libraries maintenance could be intermixed in program preparation jobs.
Compilers
The languages
used in the majority of programs in GCOS were "compiled" in machine format prior
to their execution. The exceptions were JCL/GCL, BASIC, and APL.
One of the goals pursued in the compilers design was to achieve a common object format for
all compiled languages, a goal that IBM and GE had to be able to achieve before 1970. So
it would be possible to link together procedures written in FORTRAN, COBOL, PL/1, HPL -a
PL/1 subset used as system implementation language, MLP, NAL Assembler, and later C and
have them calling the operating services through common sequences. Sometimes, a few
languages idiosyncrasies need the compiler to insert a stub for compatibility, but the
goal was achieved.
The interfaces between languages and services included traditionally two separate portions
that were kept separate for efficiency purpose: the declarative statements defining system
tables and the action statements. In the implementations languages (specific for the
system), those statements were handled through macros: declarations macros being
translated into data declaration, action macros being usually a procedure call (but
in-line coding was sometimes substituted without changing the macro-call) or an
asynchronous service request. That allowed an adaptation of the implementation services
while limiting the impact on the client. Macros were expanded by a relatively
sophisticated Macro-Generator with string processing derived from MIT TRAC. JCL initial
implementation was performed in two passes: the command string was parsed by the
Macro-Generator before being interpreted by a more compact module. Most JCL errors were
detected by the Macro-Generator parser. That implementation was later replaced by a more
conventional dedicated processor.
A goal was to have the compilers strictly obedient to ANSI and CODASYL standards. The
compiler was accepting the whole standard (even in a controversial stage, such as CTG
-Cobol Communication Task Group- or DBTG -Data Base Task Group- in COBOL) and only the
standard. The standard features not mapping nicely with the architecture had to be
compromised by a special handling at post-processor and linkage level (e.g. by
"forking" a service in a separate thread or even in requesting a service from a
special server) or by implementing a pre-processor in front of the compiler (e.g.
translating transaction processing verbs such as COMMIT into a CALL allowed by the
standard compiler).
The first
versions of the compilers were designed in separate teams in Waltham MA and in France. RPG
was even derived from a British compiler for Level 62. So they have their own components
including code generators. The only common point was the common compile-unit format. After
1975, the compiler maintenance was centralized in a single team in Paris and they were
redesigned to use common tools and modules. FORTRAN, HPL, C and others (at the exception
of COBOL, not maintained in the same team) did produce a system independent format (OIL
object intermediary language) and shared a common code generator.
In addition to
the programming languages offered to customers, Honeywell and Honeywell-Bull used three
"implementation languages" to develop the system. An assembler, named NAL for
NPL Assembly Language, using the same macro-facility (MacGen) as the other implementation
languages and more important the same declaration statements for data, gave system
programmers an access to all the facilities of the machine. Its usage was limited to
writing specific system procedures dependent on hardware and estimated to be critical in
terms of performances.
HPL was eventually the language by excellence for implementing GCOS was a PL/1 subset,
without floating point, multi-threading facility and with a more stringent typing of data.
HPL used the macro-facility of MacGen. It was developed initially under Multics on the
GE-645. It was eventually delivered to major customers under the name GPL as GCOS
programming language.
MLP was initially a macro-assembler developed under GE-635 to implement GE-100 emulator
and other hardware-oriented products. It never evolved in a full-general purpose compiler,
while its usage was pushed by the lack of machine time on GE-645. One of its advantages
was that it gave an in-line escape mechanism to the assembly language. Progressively, MLP
was phased down for HPL, when occurred complete reimplementations of old modules. Note
that NEC called GPL its version of MLP.
PL/1 was
seen in the early 1975 a potential universal language, backed by the power of IBM.
GCOS HPL was not nevertheless delivered openly to the customers because it did not
implement all PL/1 features (floating point, tasking...). Largely because it was not
supporting the default typing of PL/1, a new parser was needed. A PL/1
"standard" compliant compiler was implemented in the late 1970s by a CII
originated team. But, the success of PL/1 was doomed.
A bigger issue in CII-Honeywell-Bull was to implement ADA or even to switch to ADA as an
implementation language. ADA design winner was Jean Ichbiah, a researcher from CII; The
effort to make such a move was discussed and finally abandoned at the beginning of the
1980s while the Bull's management was intending to disengage from proprietary systems. So
ADA was not a "prophet in its own country".
C ascension was late to significantly influence GCOS products. As a language, it has few
advantages compared to HPL, except, of course, its contribution to the port of externally
developed products. In the early 1980s, Bull considered a port of a relational data base
system. Almost of all them were available in C. The port of Oracle was made initially
using a quick port of GNU C. A proprietary compiler of the C language, producing OIL
output was also made by Bull and used for Oracle and other application ports.
Binding names
into addresses
The output of
compilers was named "compile units" (often named object format). The unit
of compilation is usually an external procedure. All local names appearing in that
external procedure are resolved by the compiler as tentative binary addresses. According
to the scope of data such as recognized by the compiler, the data are allocated to one or
several data segments. Variables with AUTOMATIC scope, parameters of the procedure call
are allocated relative address in a stack segment and gathered in as "stack
section". Names not resolved by the compilers are kept in a "linkage
section".
The linker is
activated with the name of an external procedure as its main parameter. It searches the
"compile unit libraries" (from searches rules defined by the JCL
-command- author) and starts to resolve names contained in the linkage section. The linker
continues that quest until the only names to be resolved remain system names
(conventionally on the form H_xxxxxxx). The linkage sections are merged into a
"linkage segment" and the stack initial structure is built. The linker is also
responsible to allocate segment numbers to the procedures and data segments (according
conventions set in JCL).
The last binding
of names is made at loading time. The loader binds system names remaining in the load
module with segments numbers (and entry points) set up at system generation time.
The preceding
scheme implies that each external procedure corresponds to at least one segment. To avoid
a fast exhaustion of the segment numbers resource, another facility was introduced named
the "Binder" that merges several compile units into a single one. Those
bound compile units must be with same privileges and have a good chance to be used in
common. For, some time, the Binder was only used internally but it was eventually
delivered to large users, especially Fortran customers.
Building of the
load module
The linker
had the responsibility to bind together user procedures and library subroutines and to
prepare a "program" for execution, trying to perform as much binding as possible
before execution, without compromising device independence. Linked programs were named
'load modules" stored in load modules libraries. Load modules are completely
relocatable, thanks to segmentation and they wan be moved from system to system and
usually from release to release.
The linker
beginning was somewhat controversial, because it was building the thread structure of the
job step by walking across references. The linker was the builder of the address space of
the set of threads being the structure of the job step. Later, the static linker was
complemented by a dynamic linker binding dynamically linked load modules. The FORK
primitive was not a basic GCOS primitive. The linker provided a maximum number of
threads for the job step.
File system
Level 64
specifications called for file level compatibility with contemporary systems: Honeywell
H-200, GE-100 and IBM S/360. In those systems, file may even unlabeled and let be known by
their physical position only. There was a requirement that GCOS had to support IBM DOS
disc format. Actually, the S/360 disc pack format was robust, contrarily to those of its
competition inherited from a world of non-removable discs. GCOS adopted the physical and
logical format of S/360 in terms of labels and data formats.
The files needed
to the operating system were the backing store (location of the virtual memory, containing
all the address spaces of system and user running threads), the bootload file and spool
files for input and output, and program libraries containing object code (compile units to
be linked), load modules...
Those files were contained in a system volume and labeled with conventional names.
Compile Units and Load modules were not strictly file but subfiles stored in
containers that were visible to utilities as separate files. There has been some
controversy about that concept of subfiles, but as conventional files were allocated at
track boundary, there would have been a storage loss if they have been files. Another
consideration was to avoid implementing eventually an ACL (access control list) for each
program element. So subfiles libraries were implemented as "members" of files
formatted in QAM (queued access method) allowing the recovery of space inside tracks.
Users files, on
tapes or discs, were labeled and were known by the association of volume name and file
name. Labels were gathered into a special file per volume in an IBM compatible format. The
user files are allocated in an integer number of tracks and are addressed as CCCTTRR at
physical record level. The size of records is the file creator responsibility; it could be
constant at file level or variable, each record being preceded by a header (count)
containing its length (in binary). In addition to that formatting for sequential files,
GCOS supports the IBM index-sequential format (CKD count key data) that adds two physical
records to the actual data records and allows an off-line searching of the key field by
the disc controller. This ISAM formatting became progressively obsolete with the faster
growth in main memory size compared to the slower growth in controller processing
power.
In GCOS, it was replaced by a UFAS (Unified File Access System) organization that
presented a fixed (at file level) block formatted data zone. UFAS was used for sequential,
direct and indexed sequential access methods (the latter being modeled on IBM VSAM). IDS-2
data base files were mapped on direct access UFAS files. An important peculiarity of UFAS
files was that they became supported in the mid-1980s by a centralized software cache and
by a centralized locking facility (GAC) that allows to implement file locking at bloc
level and that could batch or transaction processing applications. In addition, that
facility was later used to implement dual-copy databases on coupled systems.
A catalog was
implemented in the following years to be able to handle more conveniently a profusion of
files on discs and not to require a knowledge of the content of all off-line volumes by
the responsible of operation writing JCL. That catalog associates a long name (44
characters) with the volume name and the combination of VTOC and catalog allow the system
to know the physical address of the beginning of files. The use of a one-bit flag in the
VTOC requested the system to look in the catalog to control the access rights. Access
rights were given to accounts (typically, a set of users). No naming restriction was
required, but eventually the creator name was usually included in the file name (for
maintaining the upward compatibility with GCOS8). The use of the catalog remained an
installation option; some customers did not want to modify their tape-inspired operation.
The catalog organization that took advantage of CKD format originally was reimplemented in
the 1980s to remove that dependence by using a version of UFAS files with the same basic
functionalities as earlier The history of data management in GCOS had been somewhat hectic
because the need of supporting externally defined files and because the new methods were
separately licensed to the customer, disallowing their use for system usage.
With the advent
of non-removable discs, cataloged operations became more popular and features like disc
quotas limiting the space allocated to each account were requested and eventually
implemented.
Discs and Files
were possibly shared between two distinct GCOS systems, usually for back-up reasons,
thanks to a locking mechanism at VTOC entry (label). A read-write channel program was able
to seize control of the volume without being interrupted. Those "semaphores"
represented to much overhead to be used efficiently for anything but file (or table)
exclusive access and were limited to two systems.
Discs basic
format was modified with the advent of fixed block format in the late 1980s. CKD
functionalities were discarded for fixed sectors and previous disks reformatted in fixed
blocks. Some concerns were raised about the integrity of logical records spanning several
blocks that could have been compromised by a failure of the system during gaps between
blocks WRITEs. Critical files were given the addition of heading/tailings timer stamps to
close that low-probability risk. Customer migration to the new formats took time and
both formats remained supported for long.
A disc mirroring
facility was implemented in the late 1980s. On specific volumes, supporting fixed disc
format, files were maintained in two copies. This allowed an immediate recovery of media
errors and slightly faster read access. The facility allowed mirror discs to be shared
between two systems and provide a dynamic reconstruction of the damaged files without
stopping the operation. During the period of reconstruction, the concerned part of the
file system was journalized to secure the database, as in the absence of the dual copy.
Data
management
As said above,
files were never accessed as a "contiguous string of bytes", but as
"logical records" through an access method. This access method itself contained
three parts:
the first one, at OPEN time, set up the control features of the file (filling the file
control block) and binding the file index or "file code" (known to the user
program) to the file control bloc;
the second one dynamically invoked by the application program performs the data transfers
to and from the program address space from and to the buffer area;
the third one handled the buffer pool and issued the physical I/O requests to the media
contained in the supporting device.
The first
function required often many activities and usually was performed in several steps.
Binding a file code with a real file was specified in JCL. JCL interpretation might cause
a request to the operator for mounting media, while the job step stayed in limbo. Mounting
caused the invocation of volume recognition that bound the file name with the device and
allowed the retrieval of the labeled or catalogued features. Finally the OPEN procedure
verified the coherence of those all information data and made the initial positioning of
the file. The OPEN and CLOSE services were implemented as system procedures in the user
thread. Other functions were packaged within system process groups. Open/close services
had to retrieve information prepared by the JCL interpretation of ASSIGN and ALLOCATE
statements binding the generic data management calls with the appropriate data management
routines, the channel programs templates and the physical channel and device logical
channel characteristics.
GET and PUT
services (called READ/WRITE/UPDATE in some programming languages) essentially performed
MOVE between the buffer area and the program working area. For sequential files
processing, the number of those statements far exceeded the number of I/Os. They were
implemented as procedures within the address space of the program. After a few releases,
it was decided to shortcut the protection mechanism that originally surround those
services and to implement them as in-line macros part of the application code. They had a
limited access to only a part of the file control block -the current address- and their
transfer in the user address space did not compromise the integrity of the file system.
The reading and
writing of buffers was working in non-privileged mode (ring 1), and did not have an access
to structures like logical channel tables. However they build "tentative channel
programs" that set the sequence of channel commands required for an I/O and they fill
it with virtual memory addresses and relative address inside the secondary storage space
of the file. They were not able to compromise the integrity of other files or that of
other programs.
The last function
was the responsibility of Physical I/O services that absolutized the contents of the
channel program and set a connect signal to the IOC. The subsequent functions were
performed by the IOC and the peripheral controller. They dequeue the requests on the
logical channel and they initiate the transfer between the peripheral and the buffer(s).
All the services
listed above were initially performed inside the application thread. Ring 1 buffer
management and Ring 0 Physical I/O management implied ring crossing but not process
dispatching that would have been more expensive.
A first
generation of GCOS data management was focusing on data organizations that use media
dependent information present in older systems. Those organizations were named
collectively BFAS Basic Files Access System. A peculiar variant of them was HFAS Honeywell
Files Access System. While pure sequential files did not include device dependent data,
BFAS assumed that physical records boundary had a meaning for the application data. Direct
Access Files and Index Sequential contained relative address within the file expressed as
CCCTTRR (cylinder, track, physical record number). Index sequential files use the Key
field of the CKD format for retrieving data inside a disc cylinder by the disc controller
firmware
A second
generation designed soon after GCOS first delivery was named UFAS Unified File Access
System [Unified stood for a compatible organization for GCOS 66 and GCOS 64]. UFAS is
characterized by the removal of device dependent data within files and the use of constant
size blocks inside a file (although, the customer was allowed to attribute different sizes
of blocks to different files). UFAS included a re-implementation of disc sequential files
with functionalities similar to BFAS but using a constant block size for each file. UFAS
main improvement was a complete re-implementation of index sequential access modeled on
IBM VSAM. Indexes were stored in a special region of the files and addresses of records
were defined by a device independent value, a block (renamed control interval after IBM)
number and a relative record address within the block. Indexed UFAS provided a
control interval splitting facility easing the speed of new data insertion in the file.
Heavy insert load might require a dynamic UFAS file reorganization that could be done in
parallel with file normal access and updates in a separate thread. Control intervals
reorganization was just another multi-threading activity on the file.
UFAS control structures were also used to build data bases accessed through IDS-II
(Integrated Data Store), the Honeywell implementation of CODASYL DBTG feature, also
available on GCOS 55 and GCOS 6 (and offered by Cullinet on IBM and by ICL on their own
systems). IDS-II supported multi-record types stored in a common data structure
privileging locality of related data [instead of storing data in separate tables and
multiplying the file control structures and buffers, as in relational data bases]. IDS-II
data records relations were implemented through explicit chaining pointers expressed in
UFAS form. IDS-II primary records were retrieved either through explicit indexing or via a
"computed" indexing where the value of a field was hashed to represent the UFAS
address. When two addresses collided, a new control interval chained to the control
interval pointed by the hashed index was created to store the new record.
[IDS designer Charlie Bachmann at that time at Honeywell had dreamt that the whole system
could be based on an IDS database, coding the IDS primitives and the related storage
management in the micro-kernel. In fact, several obstacles opposed that option: it was
almost impossible to freeze a durable schema for all the operating system functions and
the CODASYL approach required such a freezing; marketing saw IDS as an important asset
against IBM DL/1 and wished the unbundling of IDS program].
From an operating system point of view, the initial implementation of UFAS and IDS
followed the BFAS design. Data access methods were services implemented as shared
procedures within the user threads and dispatches were not needed before waiting for next
records (if the record feeding was later than processing of logical records). Several
buffers were allocated to each file, providing a primitive file caching in virtual memory.
As virtual memory used the same devices as the file system, it was not really useful to
"page" buffers in secondary storage. Multiplying the number of buffers was also
a drain on the number of segments, a critical resource in GCOS. Files private to an
application were assigned local (type 2) segments for their buffers, while files
dynamically shared by several process groups had their buffers in shared (type 0)
segments.
The constraints
of such a requirement lead to implement several new functions as servers. The back end
part of the general Data Management, the Buffer Pool Management, was the most spectacular.
In that case, the server implementation was helping to override some architectural limits
(numbers of buffers in the application address space) already suspected at the time of the
original design.
MLDS Multiple
Level Data Store was another data organization introduced in the 1980s for compatibility
with the data base capability of GCOS 62/4 {and Level 61). MLDS was a multi- indexed data
organization complemented by the idea of complementary records chained to primary records.
Relational Data
Bases started to become competitive in the early 198Os. Bull decided to port Oracle data
base system to GCOS rather to port MULTICS or GCOS8 relational facilities that were just
introduced. Oracle implementation was different from the old access method. An Oracle
server - a GCOS process group- received SQL requests from their clients (batch or IOF
programs or TDS transaction server). The source code used by the port remained Oracle Corp
property and Bull was only responsible for the compilation and the coding of the
environment of the server. Several successive Oracle releases were ported. The first one
used GNU C-compiler, the other used the genuine compiler.
Data compression,
a technique appearing in the mid-1980s and popularized in the personal computer world has
been absent of the GCOS operating system itself and limited to the Open 7 world. Apart
from the issue proprietary rights in the initial algorithms, the file organizations in
GCOS usually mixed control information and user data. It would have been too expensive in
terms of real memory at those times to expand a compressed file in the buffer manager
Security of Data
GCOS has not
included integrated encryption technology. In fact, Bull emphasized the encryption of data
communications using a low level security secret key provided by CP8 smart cards. But, the
encryption of files (by a PGP mechanism for instance) was absolutely prohibited by the
French law until 1997. The French law used to be the most restrictive in the World.
In that context, no attention was given to the implementation of an encrypt/decrypt
co-processor inside the CPU à la IBM S/390 that could have used to keep the secrecy of
data in the file system including the buffer pool.
Bull hoped to
take benefit of the smart-card technology that it pioneered to improve security on
GCOS. Smart cards could be used as a convenient way to store user-id and password limiting
the risks of exposing passwords to external view. Actually, smart-cards allowed to convey
"secret keys" invisible to an intruder over communications lines. But as the
encrypted data were hiding also control informations, the Secure Access 7 feature was of
limited use in network communications.
I/O in GCOS.
Contrarily to
many systems, the I/O interface with software was architectured in a compatible manner.
Peripheral devices were seen as logical channels. The logical channel was seen as a
special processor with a specific instruction set the channel program, made of sequential
instructions named channel control entry CCE. In addition to READ and WRITE instructions
to the peripheral, CCE included SENSE instruction, BRANCH instruction and synchronization
instructions operating on semaphore threads.
From the software point of view, each logical channel was seen as an independent
processor, while several levels of multiplexing-demultiplexing might occur inside the I/O
part of the hardware system.
Actually, the channel program execution was distributed between a main processor, its IOC
and the peripheral controller.
Channel programs
used physical (real) memory addresses. Some considerations have been given to a dynamic
translation of virtual addresses into virtual memory in the IOC channel. However, as
anyway buffers had to be fixed in memory during data transfer, the dynamic translation
would have caused a synchronization burden between the virtual memory management and the
IOC, it was decided to let the IOC working with physical addresses. A firmware-assisted
translation of channel program addresses was performed by I/O initialization services
before the channel program activation.
GCOS I/O buffers
had to be locked in real memory. However, the data chaining mechanism allowed to transfer
a single data block (in the peripheral) into/or from several main memory areas. This
"data chaining" mechanism was used to isolate control information present in the
data block from the buffer accessible by the application program. It was used intensively
by communications I/O.
Peripheral errors
recovery was assured by PIAR Peripheral Integrity Assurance Routines, called by abnormal
termination of I/O instructions. Those device dependent programs filtrated those events
into application significant ones (i.e. end-of-file on sequential files), irrecoverable
errors (that lead to application "check pointing" or termination) and
recoverable errors where PIAR either was able to auto-correct the I/O (e.g. by using ECC)
or by reinitiate the channel program.
Disc and console PIAR were resident in main memory, other PIAR were swappable to backing
store.
Resources
allocation
The goal pursued
initially was to schedule an application only when it was sure of the allocation of
resources needed for its successful termination. That avoided to have a deadly embrace of
job steps fighting for the same resources. Needed resources with the exception of real
memory and files stored in resident volumes had to be reserved by the job scheduler from
indications present in JCL. Requested resources were compared to the available resources
in the same order and relinquished in totality if a case of non-availability was
encountered. Of course, in such a case, the scheduling priority of the step was augmented
to increase its chances to be scheduled as soon as possible
Virtual Memory
Most of the
virtual memory is known and allocated at linkage time. The scheduler has just to be
sure that the sum of the virtual space does not excess the size of the swapping backing
store. However, there are segments the length of which is not known statically, notably
the user stack segment. The way the stack overflow was initially implemented was let
increase the stack segment size up to 64KB and, if that size is exhausted, to
"page" the stack in allocating a new segment out of the thread private segment
table.
Real Memory
The virtual
memory does not mask the need for real memory. Some operations (I/O transfers) require to
nail down segments in real memory. It was generally inefficient to allow the swapping in
backing store of buffers (where no transfer is currently performed). Control structures
used by dispatcher and memory management must be resident. Some instructions dealing with
a pointer chain might involve several segments and might lock-up the processor if they
cause several segment missing faults. More generally, each step required a certain amount
of real memory for its working set. JCL did specify a minimum amount of real memory and an
optimum amount of memory for its operation. Those numbers were used by the job step
scheduler to start the step loading or to delay its initiation.
Bull did not use
"extended memory" that was available on NEC systems. That extended memory was
cheaper, but had a more significant access time and was not addressed at byte boundary but
as chunks of 64 or 4K bytes. It was certainly non-uniformly addressable memory (NUMA) and
was used as a drum substitute. It could be used synchronously by MOVE instructions or
asynchronously using DMA via the IOC. Its usage was more judicious in NEC high-end
machines where "main memory" was implemented in expensive SRAM than in Bull
machines where SRAM was not used and DRAM become progressively cheaper and cheaper.
Memory management
Paging was
introduced only in the mid-1980s and until then the memory management relied only on the
use of segmentation. At loading time, tentative segments from the load module were loaded
in a large file named Backing Store. To each loaded segment was appended a header
containing the access rights and the segment type. The header in addition includes the
backing store relative address of the segment frame -the aggregate composed of the header
and of the addressable segment-.
Problems usually
encountered in memory management are those of fragmentation: the waste of memory by the
alignment of programs and data at a memory block (segment or page frame) boundary
(internal fragmentation) and the waste of memory by the variable size segments to be
allocated in large enough holes (external fragmentation). The initial implementation of
virtual memory by segment aligned to a 16-bytes (later 64) boundary and size reduced
considerably the internal fragmentation. The variable segments size was more difficult and
had sometimes to trigger a "compaction" of memory (i.e. moves of segment to
increase the size of "holes". Hardware was optimized for efficient moves
decreasing the load of such a procedure.
The loader stored
the process group control block segment in physical main memory. That segment includes the
segment tables for the threads of the process group. It loaded some segment frames in main
memory according to residency criteria defined at linkage time.
When memory
management (VMM) was called -by a missing segment fault in a segment table- VMM looked for
free main memory to load the requested segment frame. It selected the smallest memory area
to accommodate the segment, updated the page table, loaded the segment frame and
reorganized the list of memory "holes". If the "holes" were not large
enough to store the segment frame, VMM attempted to compact main memory (moving segments
and updating segment tables), a process that was faster than backing store I/O in Level 64
configurations. If no fitting holes were found, swapping out segments was needed:
non-modifiable segments were never written back to backing store, so it was cheaper to
just set up code segments as "missing" and to overlay their segment frame by the
new one. If VMM had to swap out, constant size data segments, it retrieve the backing
store address of the segment from the segment header and copied the segment in backing
store. If the segment size was increased, the backing store address had to be altered.
The size of
backing store was much larger than the physical memory, allowing many programs to be
loaded in backing store in dormant state. Thrashing, i.e. high rate of segment swapping
could occur, due to a large "working set" of segments, GCOS reacted by reducing
the priority of offending threads.
When main memory
size came expressed as several Megabytes and the backing store performances were not
improved in parallel, the Virtual Memory management based on segmentation became more
expensive specially memory compaction. With the Aquila project, that introduced a NEC
designed processor on top of DPS-7 product line, GCOS VMM was modified to rely chiefly on
paging for memory management. Segment swapping was still maintained for
"overlay" management of well-behavior applications.
Files resources
Files are
allocated to a job step; they can be passed from job step to job step. The ASSIGN command
performs the binding of a file symbolic name (file code) and the real name of file in the
installation. The real name is either a catalogued name, or a name specified by the volume
name and label or even by an external name known by operator (unlabeled tapes). For
sequential files, the program ignores the device type, until the processing of the
ALLOCATE command. The ALLOCATE command request the mounting of files stored on removable
devices (tapes, disc packs) after verifying that they are not yet mounted and available.
The system relies on automatic volume recognition (AVR) letting the operator using any of
the empty drive.
Their type of access (READ, READ EXCLUSIVE, WRITE, UPDATE, WRITE EXCLUSIVE) has to be
specified in the JCL command.
The strategy is
to allocate all resources needed for the execution of a step before starting the process
group representing its execution. This not applies to the libraries that are created
dynamically nor to system files created in behalf of the step.
A usage count is
used for files opened simultaneously by several steps. Files may be dismounted only when
unused. The operator may have to kill offending steps monopolizing a file (or device)
resource.
As some steps run
almost permanently, such as transaction processing (TDS) the access rights of the
resources have to be handled by the customer operation in terms of access rights and
concurrent access with other steps.
Device resources
Some resources
(essentially discs) are shared between several process groups, but most of devices are
used exclusively by application programs or by system utilities. So, most data from or to
the devices are spooled by system process groups (input reader, output writer).
Inter process
Communication
An important
thing in GCOS architecture is inter-process communication and inter-process group
communication. The first is architectured using micro-kernel features. Communication
between the system process group (J=0) and the other also use dedicated semaphores and
shared memory. Communication between non-necessarily resident process groups used a
mechanism via the file system (queued subfiles).
Semaphores
GCOS semaphores
were designed after Dijkstra primitives. A semaphore is an architectured entity containing
a lock and a count. The creation of semaphore is usually static (either as a system
semaphore part of the application program interface or as a local semaphore set up by the
linker). The operations available on semaphores are P-operation, V-operation and a P-Test
operation.
The lock is set by a P operation. If the lock is set, the P-Operation increments the count
and put the thread in wait state and links its id in the semaphore queue.
The V-operation decrements the count and, if it became null, reschedules the first thread
of the waiting queue (i.e. put it in the ready queue).
The P-Test is a complementary statement that allows the thread not to be stopped when the
semaphore is locked. It is used to implement a WAIT on multiple-events and similar
features.
Semaphores are
handled by the micro-kernel in relation with the thread dispatcher.
Semaphores with
Messages
Many semaphores
concern system services: server threads are waiting new requests on a system semaphore. So
semaphores may support messages (16 bytes strings) that are queued by client threads
performing V-operations. The issuing by the server thread of a P-Operation on a semaphore
with message blocks the server if no request (V-op) is pending and returns the first
message in the queue if some have been notified.
Messages are
usually pointers to more detailed data structures. The semaphores messages are stored in a
special system segment used only by the micro-kernel.
Shared memory
Type 0 data
segments are within the address space of all process groups. Most of them have restricted
access (Read only and/or ring number). Type 1 data segments are not directly addressable
because that address space is dynamically allocated. However, the system maintains a table
of shared data segments allowing the sharing of the Type 1 space.
Type 2 data segments are statically assigned to all the threads of the same process-group.
Type 1 and 2 writable segments have to be protected against concurrent access by a locking
mechanism, usually locking semaphores.
Event management
An additional
mechanism has been initially introduced in GCOS, but its role decreased with time because
its lower efficiency. Event Management was a software (not micro-kernel) supported
mechanism that stored messages (like messages semaphores) in the file system (in queued
files). The facility did not require that the server was active when the client posted its
messages and extended the limits on the size and the number of messages.
However, the processing and the elapse time for operations on events caused this mechanism
to be of limited use.
Dispatching
of threads
Threads are
either stopped, blocked or ready for execution. When a thread has been started, it passes
under the control of the micro-kernel. The dispatching of the ready threads is made by the
micro-kernel according to the thread priority. The micro-kernel works according the thread
priority, threads with the same priority are allocated slices on a round-robin
basis.
Threads may be allocated a time-slice after what they are interrupted and put at the end
of the queue of the ready threads with the same priority. That feature was made available
after a few years, when interactive applications became predominant.
Threads blocked themselves when they issue a P-operation on semaphores already set by
another thread. They became ready, when another thread issued a V-operation on that
semaphore.
So all normal dispatching was made under the efficient micro-kernel control, letting to
GCOS scheduler the initiation or the killing of threads.
Initialization
The hardware
initialization of the system evolved somewhat during the time, but software bootload did
not change deeply. The initialization of the hardware configuration (specially peripheral
controllers) was finished under SIP System Initialization Program that was loaded by
firmware from the boot device. Note that the modification of the boot device was made
through operator commands to the service processor.
SIP role was to
read the system configuration table and to update it according the real hardware
availability and to load the optional firmware components. After that phase, control was
given to GCOS ISL a real memory version of the operating system that verified or built the
resources needed by GCOS.
As data integrity
was crucial to GCOS, all system malfunctions were handled by special machine integrity
assurance routines (MIAR) that attempted to recover from possible memory, IOC and
processor errors. In fact, the reliability of hardware was much higher that previous
systems and there were no many occurrences of MIAR. MIAR operated as fault procedures
called in the user threads (when the error was detected on the execution of processor
instructions) or in asynchronous threads (when it occurred in I/O). In SMP configurations,
MIAR had the possibility to remove a processor from the configuration and to siphon the
cache of that processor. The dynamic reconnection of a processor was also possible from
the service processor. As main memory had auto-correction capability (by ECC), MIAR able
to isolate blocks of memory were infrequently called. GCOS did not implement in software a
relocation of the operating system control structures, so non-corrected memory errors
usually called a system termination. The service processor initiated memory-checking lead
to the isolation of the memory block.
Multi-processor,
Partitioning and Multi-computer configurations
The first
generation of GCOS systems was supporting only single processor configurations; the
architecture had been designed to support several (initially up to 8, later -in Artemis-
up to 24) processors in a shared multi processor (SMP) configuration. Hardware became
available in 1981. According to the architecture the functions of allocating CPU to
threads was under the responsibility of the micro-kernel. The operating system
run unchanged in single processor or multi processor. A few changes were needed just to
improve performances by decreasing the path of critical sections of the kernel protected
by a lock semaphore.
The micro-kernel itself was running in all the processors and when a thread relinquished
control the firmware dispatcher look in the threads ready queue to select the new thread
to be given control. The threads ready queue was protected by a hardware lock that
inhibits access to the related memory bank.
To avoid that such dispatches contribute to flush the L1 processor caches, some
"affinity" between processor and cache was recorded in thread status to give
preference to a thread that might have still some data in the cache. A similar procedure
was used for dispatches of Artemis that shared a L2 cache between each cluster of four
processors.
The architecture
provided a way to run specific threads on specific processors as a provision for specific
features available in some processors. Such an asymmetry was unused until the mid 1990s,
where different pricing was applied to the Artemis processors, accelerating the DB and
Unix processors and slowing down GCOS application processors.
With the
availability of SMP on P7G, Siris 8 emulation coexisted with the first test of GCOS7 by
converted customers. A facility was offered to partition the SMP hardware in two systems.
IBM offered also this feature on S/390, but several years after Bull. The same system
could be operated during some shifts in a dual-processor configuration, and in other as
two independent GCOS systems.
With Artemis that
could support up to 24 processors, the partitioning facility was extended.
Distributed
Processing & Networking
Although, the
hardware systems on which GCOS run since 1979 had Distributed Systems in their named,
GCOS is NOT in itself a Distributed Operating System. Its usage of heterogeneous
peripheral processors has been the pretext for this commercial name. But all the
architectural entities of GCOS, with the exception of files, have only a meaning inside
one system. Even when a MP hardware system was partitioned in several software systems, no
mechanism had existed to provide a direct communication between GCOSes. There has
been studies made in the 1980s to attempt to architecture more efficiently Distributed
Processing in clusters of homogeneous systems, however those studies did not materialize.
They face the issue of the implementation of distributed processing across Honeywell and
Bull heterogeneous systems under evolving standards (Bull-Honeywell DSA, ISO and finally
TCP/IP). Even a standardized function like RPC -remote procedure call- had not been
architectured in GCOS.
Telecommunications
During the life of
GCOS, many changes occurred in the field of telecommunications. In the 1960s, excluding
the time sharing applications that were not yet considered as market for general purpose
computers, two main usages were considered: remote batch operation to allow the
decentralization of input/output and direct access to terminals to allow data collection
and other real-time processing. The idea of formalizing data collection in the programming
system through a message queuing facility emerged in the early 1970s via the CTG of
Codasyl. However, that approach did hardly satisfy the operation of multiprogrammed
systems and had to be integrated in the OS.
A major transformation was the universal acceptance of transaction processing that
revolutionized the industry. TP applications became the heart of business applications in
the computer, batch-processing retiring as ancillary activities.
Another revolution occurred in the late 1970s, that is the recognition of networks of
computers. The illusion that a single system could cope with all the needs of a company or
an administration faded and networking became a key point in the architecture of a system.
Until the mid 1980s, most manufacturers dreamed to dominate the customer computing world
and proprietary systems were developed. Finally, all ended by accepting open networks that
allowed all type and makes of computers to be connected to a worldwide network.
GCOS basic
architecture was used to build several implementations of products corresponding to that
evolution of needs.
To support the
message queuing facility of CTG, the GCOS COBOL compiler generated not only the calls to
the SEND and RECEIVE procedures, but also the parameters of the process group handling the
message processing. The launching of the COBOL application triggers also that of the
message processing. Queues of messages was done through access to a set of Queued files
that were filled or emptied by the Message Control Program.
Direct access was
supported by a BTNS (Basic Terminal Process Group) process group handling terminals and
accessed by the user program through shared memory (based on semaphores). BTNS supported
its JCL controlled communications lines. All terminals located on the same communication
line were to be allocated to BTNS.
After the CII
merger, CII-HB and Honeywell embarked into the definition of a competitive solution
against IBM's SNA the HDSA Honeywell Distributed Systems Architecture. HDSA differed from
SNA by its orientation around a Network Processor instead of main frames. HDSA was
designed at the same time as the ISO OSI Open Model and adhered somewhat dogmatically to
that layered model. The presentation and application levels were to be implemented in
terminals and hosts while the interface between NCP and hosts would be the session layer.
HDSA was implementing only the "connected" mode, following the tradition of
PTT's that were already offering X-25 protocols. If many applications were taking
advantage of that "connected mode" (Telnet, File Transfer...), establishing a
session for each transaction caused much overhead over the system.
The Network
Processor was a Level 6 minicomputer and was used either as a Front-end (FEP) of a GCOS
mainframe or as a remote Network Processor. The interconnection between a DPS-7 and the
FEP was via PSI channel. It violated somewhat the HDSA architecture concepts, because it
did not offer strictly a session interface but multiplexed the data exchanged over the
opened sessions as packets transmitted between the FEP and a GCOS resident program, named
FNPS. That FNPS program controlled the FEP PSI interface on a couple of logical channels
on which it read and wrote data continually on looping channel programs. In addition to
that, FNPS was performing the bootloading and the initialization of the FEP, when the FEP
was disk-less -the general case for DPS-7 systems.
Distributed
Applications
DSA, as HDSA was
named after the nationalization of CII-HB, offered the following functions:
HDSA had always
been considered as a close-gap and a learning phase before a manufacturer independent ISO
standard be adopted. Unhappily for Bull and Honeywell, their intense lobbying to attempt
to have HDSA choices adopted by ISO was hurting not only IBM and Digital that had their
own standards (IBM improving SNA, and DEC adopting "non-connected" mode), but
also the scientific and university worlds adopting en masse the TCP/IP and UDP/IP
architectures. Some ISO standards were adopted, functionally close to DSA, but requiring
many implementations. Bull and some of its customers speculated that ISO would take over
eventually but had their UNIX systems implementing TCP/IP. So, a progressive conversion to
the world of Open Systems occurred in the late 1980s.
The DPS-6 minicomputer was discontinued and a new front-end processor for DSA was based on
the 68000 architecture. The evolution started on the low end DPS-7000 with a micro-FEP
directly connected to the Multibus II I/O bus of those systems. However, firmware emulated
a PSI interconnection. Only a part of NCP software was ported from Level-6 to the 68000
architecture. In a second phase, the DPS-6 FEP and Network processor were replaced by
68000 systems supporting DSA. In addition, similar hardware, now under UNIX (SPIX)
operating systems insure the interconnection to the TCP/IP world.
GCOS did not
initially support natively the TCP/IP protocols and applications. Instead, the TCP/IP
world was attached to a UNIX subsystem OPEN7 operating under GCOS that then communicate
with the GCOS world.
Later, in 1996,
TCP/IP stack was implemented natively under GCOS, progressively decreasing the role of DSA
to the support of legacy terminals.