Operating System GCOS-64 GCOS-7

1967-200x

 

STRUCTURE OF GCOS
 

The description of the operating systems written in the 1970s did use a terminology that would be not easy to catch by 2000s readers. The concept not known as thread, multi-threading was then named, not without rationale, process. Since then, UNIX fans have assimilated a process to an address space more than to a chain of control (while frequently the two concepts are mixed).
We used to call "distributed procedures" procedures executed within the user thread, while since the late 1980s, distribution was assimilated to the client/server model, where services are run in separate processes.
The I/O architecture of GCOS is also quite different of most open systems. It followed the concepts of "channel" introduced by IBM in the late 1950s and pushed the architecture of its "channel programs" to a limit closed to real process distribution.
Repartition of threads within a shared memory multiprocessor system was standard in the architecture, even within the nucleus of the OS, thanks to the implementation of basic functions in "firmware", what we can claim to be a micro-kernel.

Micro-kernel

GCOS includes a micro-kernel implemented in "firmware", a set of shared procedures that implement "synchronous services", a system process group that gathers services implemented by threads and events handling and in addition to those services a variable number of servers (process groups) that may be divided in standard system servers and in special subsystems monitors or servers.

We are here calling micro-kernel those basic functions that implement the architecture objects that are threads, semaphores (including I/O) and procedure calls. They are loaded at initialization time and constitute really a foolproof kernel layer. The term of micro-kernel may disturb purists, because the services of that level are not all implemented by micro-threads communicating only by messages with the software. Some services are likely synchronous operating as procedures in the calling thread (P and V-operations), others are implemented through micro processes polling I/O channels or dispatching visible threads candidate for execution.
The way that micro-kernel is implemented is not part of the architecture and vary slightly according to the processor models. That micro code is implemented as a subset of the software visible instruction set (so it is interpreted by the execution micro programs), bypassing error checks and using directly hardware features not part of the software visible architecture. It operates faster than the same algorithms in the faster.

The system process group operates within its own address space (J=0). It gathers system threads implementing asynchronous services or, more exactly the asynchronous portion of those services. Among those the I/O termination thread (s), the interrupt handling, attention handler.

In addition to that system process groups that has to stay permanently alive, several process groups are needed for a general-purpose operation: the Job scheduler, the Input Reader, the Output Writer. Reader and Writer are multi-threaded process groups that can handle several reader devices and several printer devices. Other process groups are also present in a fully configurated configuration: FNPS assuring the network interface, and a variety of transaction and data base servers. 

Some recent systems (e.g. CHORUS) claim that a well architectured micro-kernel should pass parameters by messages so that an operating system could be distributed by processors without shared memory. Such configurations were not designed for GCOS, so GCOS does not pretend to include a well-architectured micro-kernel. The word itself was not used by developers who often consider the primitives as instructions implemented by firmware and operating in a mysterious area of memory called under-BAR (for Barricade register) memory.

Program execution.

The external specifications of GCOS were not a workstation operating system, such as Windows, not a time-sharing server like Unix (just being in limbo at GCOS design time), but essentially to serve as a batch operating system handling sequential files and progressively direct access files. Programs were initiated from a batch input file, initially a spooled card reader, later extended to multiple sources, some of them being TERMINALS. Programs were scheduled according their physical devices resources requirements (many initially were using magnetic tapes). Then they were loaded, performing some last minute binding with resources and services. Then their thread was given control and were dispatched the most efficiently possible, and eventually they terminate and an execution report was printed.

Program preparation

The overall structure of program preparation was already available in the previous generation of computers (for Bull: Gamma M-40, GE-140). A job is submitted to the system as a sequential file of commands (JCL) and data (source programs, subcommands for processors, and possibly user data to be processed). In general, the system being used most for business application, the program preparation job ended with the update of libraries for subsequent processing of files.
Program preparation includes an optional step of macro processing, a step of language compilation, and a linkage step. Utilities such as the binding of several compile units or libraries maintenance could be intermixed in program preparation jobs.

Compilers

The languages used in the majority of programs in GCOS were "compiled" in machine format prior to their execution. The exceptions were JCL/GCL, BASIC, and APL. 
One of the goals pursued in the compilers design was to achieve a common object format for all compiled languages, a goal that IBM and GE had to be able to achieve before 1970. So it would be possible to link together procedures written in FORTRAN, COBOL, PL/1, HPL -a PL/1 subset used as system implementation language, MLP, NAL Assembler, and later C and have them calling the operating services through common sequences. Sometimes, a few languages idiosyncrasies need the compiler to insert a stub for compatibility, but the goal was achieved.
The interfaces between languages and services included traditionally two separate portions that were kept separate for efficiency purpose: the declarative statements defining system tables and the action statements. In the implementations languages (specific for the system), those statements were handled through macros: declarations macros being translated into data declaration, action macros being usually a procedure call (but in-line coding was sometimes substituted without changing the macro-call) or an asynchronous service request. That allowed an adaptation of the implementation services while limiting the impact on the client. Macros were expanded by a relatively sophisticated Macro-Generator with string processing derived from MIT TRAC. JCL initial implementation was performed in two passes: the command string was parsed by the Macro-Generator before being interpreted by a more compact module. Most JCL errors were detected by the Macro-Generator parser. That implementation was later replaced by a more conventional dedicated processor.
A goal was to have the compilers strictly obedient to ANSI and CODASYL standards. The compiler was accepting the whole standard (even in a controversial stage, such as CTG -Cobol Communication Task Group- or DBTG -Data Base Task Group- in COBOL) and only the standard. The standard features not mapping nicely with the architecture had to be compromised by a special handling at post-processor and linkage level (e.g. by "forking" a service in a separate thread or even in requesting a service from a special server) or by implementing a pre-processor in front of the compiler (e.g. translating transaction processing verbs such as COMMIT into a CALL allowed by the standard compiler).

The first versions of the compilers were designed in separate teams in Waltham MA and in France. RPG was even derived from a British compiler for Level 62. So they have their own components including code generators. The only common point was the common compile-unit format. After 1975, the compiler maintenance was centralized in a single team in Paris and they were redesigned to use common tools and modules. FORTRAN, HPL, C and others (at the exception of COBOL, not maintained in the same team) did produce a system independent format (OIL object intermediary language) and shared a common code generator. 

In addition to the programming languages offered to customers, Honeywell and Honeywell-Bull used three "implementation languages" to develop the system. An assembler, named NAL for NPL Assembly Language, using the same macro-facility (MacGen) as the other implementation languages and more important the same declaration statements for data, gave system programmers an access to all the facilities of the machine. Its usage was limited to writing specific system procedures dependent on hardware and estimated to be critical in terms of performances.
HPL was eventually the language by excellence for implementing GCOS was a PL/1 subset, without floating point, multi-threading facility and with a more stringent typing of data. HPL used the macro-facility of MacGen. It was developed initially under Multics on the GE-645. It was eventually delivered to major customers under the name GPL as GCOS programming language.
MLP was initially a macro-assembler developed under GE-635 to implement GE-100 emulator and other hardware-oriented products. It never evolved in a full-general purpose compiler, while its usage was pushed by the lack of machine time on GE-645. One of its advantages was that it gave an in-line escape mechanism to the assembly language. Progressively, MLP was phased down for HPL, when occurred complete reimplementations of old modules. Note that NEC called GPL its version of MLP.

PL/1 was seen in the early 1975 a potential universal language, backed by the power of IBM. GCOS HPL was not nevertheless delivered openly to the customers because it did not implement all PL/1 features (floating point, tasking...). Largely because it was not supporting the default typing of PL/1, a new parser was needed. A PL/1 "standard" compliant compiler was implemented in the late 1970s by a CII originated team. But, the success of PL/1 was doomed. 
A bigger issue in CII-Honeywell-Bull was to implement ADA or even to switch to ADA as an implementation language. ADA design winner was Jean Ichbiah, a researcher from CII; The effort to make such a move was discussed and finally abandoned at the beginning of the 1980s while the Bull's management was intending to disengage from proprietary systems. So ADA was not a "prophet in its own country".
C ascension was late to significantly influence GCOS products. As a language, it has few advantages compared to HPL, except, of course, its contribution to the port of externally developed products. In the early 1980s, Bull considered a port of a relational data base system. Almost of all them were available in C. The port of Oracle was made initially using a quick port of GNU C. A proprietary compiler of the C language, producing OIL output was also made by Bull and used for Oracle and other application ports.

Binding names into addresses

The output of compilers was named "compile units" (often named object format). The unit of compilation is usually an external procedure. All local names appearing in that external procedure are resolved by the compiler as tentative binary addresses. According to the scope of data such as recognized by the compiler, the data are allocated to one or several data segments. Variables with AUTOMATIC scope, parameters of the procedure call are allocated relative address in a stack segment and gathered in as "stack section". Names not resolved by the compilers are kept in a "linkage section".

The linker is activated with the name of an external procedure as its main parameter. It searches the "compile unit libraries" (from searches rules defined by the JCL -command- author) and starts to resolve names contained in the linkage section. The linker continues that quest until the only names to be resolved remain system names (conventionally on the form H_xxxxxxx). The linkage sections are merged into a "linkage segment" and the stack initial structure is built. The linker is also responsible to allocate segment numbers to the procedures and data segments (according conventions set in JCL).

The last binding of names is made at loading time. The loader binds system names remaining in the load module with segments numbers (and entry points) set up at system generation time.

The preceding scheme implies that each external procedure corresponds to at least one segment. To avoid a fast exhaustion of the segment numbers resource, another facility was introduced named the "Binder" that merges several compile units into a single one. Those bound compile units must be with same privileges and have a good chance to be used in common. For, some time, the Binder was only used internally but it was eventually delivered to large users, especially Fortran customers.

Building of the load module

The linker had the responsibility to bind together user procedures and library subroutines and to prepare a "program" for execution, trying to perform as much binding as possible before execution, without compromising device independence. Linked programs were named 'load modules" stored in load modules libraries. Load modules are completely relocatable, thanks to segmentation and they wan be moved from system to system and usually from release to release.

The linker beginning was somewhat controversial, because it was building the thread structure of the job step by walking across references. The linker was the builder of the address space of the set of threads being the structure of the job step. Later, the static linker was complemented by a dynamic linker binding dynamically linked load modules. The FORK primitive was not a basic GCOS primitive. The linker provided a maximum number of threads for the job step.

File system

Level 64 specifications called for file level compatibility with contemporary systems: Honeywell H-200, GE-100 and IBM S/360. In those systems, file may even unlabeled and let be known by their physical position only. There was a requirement that GCOS had to support IBM DOS disc format. Actually, the S/360 disc pack format was robust, contrarily to those of its competition inherited from a world of non-removable discs. GCOS adopted the physical and logical format of S/360 in terms of labels and data formats.

The files needed to the operating system were the backing store (location of the virtual memory, containing all the address spaces of system and user running threads), the bootload file and spool files for input and output, and program libraries containing object code (compile units to be linked), load modules...
Those files were contained in a system volume and labeled with conventional names. Compile Units and Load modules were not strictly file but subfiles stored in containers that were visible to utilities as separate files. There has been some controversy about that concept of subfiles, but as conventional files were allocated at track boundary, there would have been a storage loss if they have been files. Another consideration was to avoid implementing eventually an ACL (access control list) for each program element. So subfiles libraries were implemented as "members" of files formatted in QAM (queued access method) allowing the recovery of space inside tracks.

Users files, on tapes or discs, were labeled and were known by the association of volume name and file name. Labels were gathered into a special file per volume in an IBM compatible format. The user files are allocated in an integer number of tracks and are addressed as CCCTTRR at physical record level. The size of records is the file creator responsibility; it could be constant at file level or variable, each record being preceded by a header (count) containing its length (in binary). In addition to that formatting for sequential files, GCOS supports the IBM index-sequential format (CKD count key data) that adds two physical records to the actual data records and allows an off-line searching of the key field by the disc controller. This ISAM formatting became progressively obsolete with the faster growth in main memory size compared to the slower growth in controller processing power. 
In GCOS, it was replaced by a UFAS (Unified File Access System) organization that presented a fixed (at file level) block formatted data zone. UFAS was used for sequential, direct and indexed sequential access methods (the latter being modeled on IBM VSAM). IDS-2 data base files were mapped on direct access UFAS files. An important peculiarity of UFAS files was that they became supported in the mid-1980s by a centralized software cache and by a centralized locking facility (GAC) that allows to implement file locking at bloc level and that could batch or transaction processing applications. In addition, that facility was later used to implement dual-copy databases on coupled systems.

A catalog was implemented in the following years to be able to handle more conveniently a profusion of files on discs and not to require a knowledge of the content of all off-line volumes by the responsible of operation writing JCL. That catalog associates a long name (44 characters) with the volume name and the combination of VTOC and catalog allow the system to know the physical address of the beginning of files. The use of a one-bit flag in the VTOC requested the system to look in the catalog to control the access rights. Access rights were given to accounts (typically, a set of users). No naming restriction was required, but eventually the creator name was usually included in the file name (for maintaining the upward compatibility with GCOS8). The use of the catalog remained an installation option; some customers did not want to modify their tape-inspired operation.
The catalog organization that took advantage of CKD format originally was reimplemented in the 1980s to remove that dependence by using a version of UFAS files with the same basic functionalities as earlier The history of data management in GCOS had been somewhat hectic because the need of supporting externally defined files and because the new methods were separately licensed to the customer, disallowing their use for system usage.

With the advent of non-removable discs, cataloged operations became more popular and features like disc quotas limiting the space allocated to each account were requested and eventually implemented.

Discs and Files were possibly shared between two distinct GCOS systems, usually for back-up reasons, thanks to a locking mechanism at VTOC entry (label). A read-write channel program was able to seize control of the volume without being interrupted. Those "semaphores" represented to much overhead to be used efficiently for anything but file (or table) exclusive access and were limited to two systems.

Discs basic format was modified with the advent of fixed block format in the late 1980s. CKD functionalities were discarded for fixed sectors and previous disks reformatted in fixed blocks. Some concerns were raised about the integrity of logical records spanning several blocks that could have been compromised by a failure of the system during gaps between blocks WRITEs. Critical files were given the addition of heading/tailings timer stamps to close that low-probability risk.  Customer migration to the new formats took time and both formats remained supported for long.

A disc mirroring facility was implemented in the late 1980s. On specific volumes, supporting fixed disc format, files were maintained in two copies. This allowed an immediate recovery of media errors and slightly faster read access. The facility allowed mirror discs to be shared between two systems and provide a dynamic reconstruction of the damaged files without stopping the operation. During the period of reconstruction, the concerned part of the file system was journalized to secure the database, as in the absence of the dual copy.

Data management 

As said above, files were never accessed as a "contiguous string of bytes", but as "logical records" through an access method. This access method itself contained three parts: 
the first one, at OPEN time, set up the control features of the file (filling the file control block) and binding the file index or "file code" (known to the user program) to the file control bloc;
the second one dynamically invoked by the application program performs the data transfers to and from the program address space from and to the buffer area;
the third one handled the buffer pool and issued the physical I/O requests to the media contained in the supporting device.

The first function required often many activities and usually was performed in several steps. Binding a file code with a real file was specified in JCL. JCL interpretation might cause a request to the operator for mounting media, while the job step stayed in limbo. Mounting caused the invocation of volume recognition that bound the file name with the device and allowed the retrieval of the labeled or catalogued features. Finally the OPEN procedure verified the coherence of those all information data and made the initial positioning of the file. The OPEN and CLOSE services were implemented as system procedures in the user thread. Other functions were packaged within system process groups. Open/close services had to retrieve information prepared by the JCL interpretation of ASSIGN and ALLOCATE statements binding the generic data management calls with the appropriate data management routines, the channel programs templates and the physical channel and device logical channel characteristics. 

GET and PUT services (called READ/WRITE/UPDATE in some programming languages) essentially performed MOVE between the buffer area and the program working area. For sequential files processing, the number of those statements far exceeded the number of I/Os. They were implemented as procedures within the address space of the program. After a few releases, it was decided to shortcut the protection mechanism that originally surround those services and to implement them as in-line macros part of the application code. They had a limited access to only a part of the file control block -the current address- and their transfer in the user address space did not compromise the integrity of the file system.

The reading and writing of buffers was working in non-privileged mode (ring 1), and did not have an access to structures like logical channel tables. However they build "tentative channel programs" that set the sequence of channel commands required for an I/O and they fill it with virtual memory addresses and relative address inside the secondary storage space of the file. They were not able to compromise the integrity of other files or that of other programs.

The last function was the responsibility of Physical I/O services that absolutized the contents of the channel program and set a connect signal to the IOC. The subsequent functions were performed by the IOC and the peripheral controller. They dequeue the requests on the logical channel and they initiate the transfer between the peripheral and the buffer(s).

All the services listed above were initially performed inside the application thread. Ring 1 buffer management and Ring 0 Physical I/O management implied ring crossing but not process dispatching that would have been more expensive.

A first generation of GCOS data management was focusing on data organizations that use media dependent information present in older systems. Those organizations were named collectively BFAS Basic Files Access System. A peculiar variant of them was HFAS Honeywell Files Access System. While pure sequential files did not include device dependent data, BFAS assumed that physical records boundary had a meaning for the application data. Direct Access Files and Index Sequential contained relative address within the file expressed as CCCTTRR (cylinder, track, physical record number). Index sequential files use the Key field of the CKD format for retrieving data inside a disc cylinder by the disc controller firmware  

A second generation designed soon after GCOS first delivery was named UFAS Unified File Access System [Unified stood for a compatible organization for GCOS 66 and GCOS 64]. UFAS is characterized by the removal of device dependent data within files and the use of constant size blocks inside a file (although, the customer was allowed to attribute different sizes of blocks to different files). UFAS included a re-implementation of disc sequential files with functionalities similar to BFAS but using a constant block size for each file. UFAS main improvement was a complete re-implementation of index sequential access modeled on IBM VSAM. Indexes were stored in a special region of the files and addresses of records were defined by a device independent value, a block (renamed control interval after IBM) number and a relative record address within the block.  Indexed UFAS provided a control interval splitting facility easing the speed of new data insertion in the file. Heavy insert load might require a dynamic UFAS file reorganization that could be done in parallel with file normal access and updates in a separate thread. Control intervals reorganization was just another multi-threading activity on the file.
UFAS control structures were also used to build data bases accessed through IDS-II (Integrated Data Store), the Honeywell implementation of CODASYL DBTG feature, also available on GCOS 55 and GCOS 6 (and offered by Cullinet on IBM and by ICL on their own systems). IDS-II supported multi-record types stored in a common data structure privileging locality of related data [instead of storing data in separate tables and multiplying the file control structures and buffers, as in relational data bases]. IDS-II data records relations were implemented through explicit chaining pointers expressed in UFAS form. IDS-II primary records were retrieved either through explicit indexing or via a "computed" indexing where the value of a field was hashed to represent the UFAS address. When two addresses collided, a new control interval chained to the control interval pointed by the hashed index was created to store the new record. 
[IDS designer Charlie Bachmann at that time at Honeywell had dreamt that the whole system could be based on an IDS database, coding the IDS primitives and the related storage management in the micro-kernel. In fact, several obstacles opposed that option: it was almost impossible to freeze a durable schema for all the operating system functions and the CODASYL approach required such a freezing; marketing saw IDS as an important asset against IBM DL/1 and wished the unbundling of IDS program].
From an operating system point of view, the initial implementation of UFAS and IDS followed the BFAS design. Data access methods were services implemented as shared procedures within the user threads and dispatches were not needed before waiting for next records (if the record feeding was later than processing of logical records). Several buffers were allocated to each file, providing a primitive file caching in virtual memory. As virtual memory used the same devices as the file system, it was not really useful to "page" buffers in secondary storage. Multiplying the number of buffers was also a drain on the number of segments, a critical resource in GCOS. Files private to an application were assigned local (type 2) segments for their buffers, while files dynamically shared by several process groups had their buffers in shared (type 0) segments.

The constraints of such a requirement lead to implement several new functions as servers. The back end part of the general Data Management, the Buffer Pool Management, was the most spectacular. In that case, the server implementation was helping to override some architectural limits (numbers of buffers in the application address space) already suspected at the time of the original design.

MLDS Multiple Level Data Store was another data organization introduced in the 1980s for compatibility with the data base capability of GCOS 62/4 {and Level 61). MLDS was a multi- indexed data organization complemented by the idea of complementary records chained to primary records.

Relational Data Bases started to become competitive in the early 198Os. Bull decided to port Oracle data base system to GCOS rather to port MULTICS or GCOS8 relational facilities that were just introduced. Oracle implementation was different from the old access method. An Oracle server - a GCOS process group- received SQL requests from their clients (batch or IOF programs or TDS transaction server). The source code used by the port remained Oracle Corp property and Bull was only responsible for the compilation and the coding of the environment of the server. Several successive Oracle releases were ported. The first one used GNU C-compiler, the other used the genuine compiler.

Data compression, a technique appearing in the mid-1980s and popularized in the personal computer world has been absent of the GCOS operating system itself and limited to the Open 7 world. Apart from the issue proprietary rights in the initial algorithms, the file organizations in GCOS usually mixed control information and user data. It would have been too expensive in terms of real memory at those times to expand a compressed file in the buffer manager

Security of Data

GCOS has not included integrated encryption technology. In fact, Bull emphasized the encryption of data communications using a low level security secret key provided by CP8 smart cards. But, the encryption of files (by a PGP mechanism for instance) was absolutely prohibited by the French law until 1997. The French law used to be the most restrictive in the World. 
In that context, no attention was given to the implementation of an encrypt/decrypt co-processor inside the CPU à la IBM S/390 that could have used to keep the secrecy of data in the file system including the buffer pool.

Bull hoped to take benefit of the smart-card technology that it pioneered to improve security on GCOS. Smart cards could be used as a convenient way to store user-id and password limiting the risks of exposing passwords to external view. Actually, smart-cards allowed to convey "secret keys" invisible to an intruder over communications lines. But as the encrypted data were hiding also control informations, the Secure Access 7 feature was of limited use in network communications.

I/O in GCOS.

Contrarily to many systems, the I/O interface with software was architectured in a compatible manner. Peripheral devices were seen as logical channels. The logical channel was seen as a special processor with a specific instruction set the channel program, made of sequential instructions named channel control entry CCE. In addition to READ and WRITE instructions to the peripheral, CCE included SENSE instruction, BRANCH instruction and synchronization instructions operating on semaphore threads.
From the software point of view, each logical channel was seen as an independent processor, while several levels of multiplexing-demultiplexing might occur inside the I/O part of the hardware system.
Actually, the channel program execution was distributed between a main processor, its IOC and the peripheral controller.

Channel programs used physical (real) memory addresses. Some considerations have been given to a dynamic translation of virtual addresses into virtual memory in the IOC channel. However, as anyway buffers had to be fixed in memory during data transfer, the dynamic translation would have caused a synchronization burden between the virtual memory management and the IOC, it was decided to let the IOC working with physical addresses. A firmware-assisted translation of channel program addresses was performed by I/O initialization services before the channel program activation.

GCOS I/O buffers had to be locked in real memory. However, the data chaining mechanism allowed to transfer a single data block (in the peripheral) into/or from several main memory areas. This "data chaining" mechanism was used to isolate control information present in the data block from the buffer accessible by the application program. It was used intensively by communications I/O.

Peripheral errors recovery was assured by PIAR Peripheral Integrity Assurance Routines, called by abnormal termination of I/O instructions. Those device dependent programs filtrated those events into application significant ones (i.e. end-of-file on sequential files), irrecoverable errors (that lead to application  "check pointing" or termination) and recoverable errors where PIAR either was able to auto-correct the I/O (e.g. by using ECC) or by reinitiate the channel program.
Disc and console PIAR were resident in main memory, other PIAR were swappable to backing store.

 

Resources allocation

The goal pursued initially was to schedule an application only when it was sure of the allocation of resources needed for its successful termination. That avoided to have a deadly embrace of job steps fighting for the same resources. Needed resources with the exception of real memory and files stored in resident volumes had to be reserved by the job scheduler from indications present in JCL. Requested resources were compared to the available resources in the same order and relinquished in totality if a case of non-availability was encountered. Of course, in such a case, the scheduling priority of the step was augmented to increase its chances to be scheduled as soon as possible

Virtual Memory

Most of the virtual memory is known and allocated at linkage time. The scheduler has just to be sure that the sum of the virtual space does not excess the size of the swapping backing store. However, there are segments the length of which is not known statically, notably the user stack segment. The way the stack overflow was initially implemented was let increase the stack segment size up to 64KB and, if that size is exhausted, to "page" the stack in allocating a new segment out of the thread private segment table.

Real Memory

The virtual memory does not mask the need for real memory. Some operations (I/O transfers) require to nail down segments in real memory. It was generally inefficient to allow the swapping in backing store of buffers (where no transfer is currently performed). Control structures used by dispatcher and memory management must be resident. Some instructions dealing with a pointer chain might involve several segments and might lock-up the processor if they cause several segment missing faults. More generally, each step required a certain amount of real memory for its working set. JCL did specify a minimum amount of real memory and an optimum amount of memory for its operation. Those numbers were used by the job step scheduler to start the step loading or to delay its initiation.

Bull did not use "extended memory" that was available on NEC systems. That extended memory was cheaper, but had a more significant access time and was not addressed at byte boundary but as chunks of 64 or 4K bytes. It was certainly non-uniformly addressable memory (NUMA) and was used as a drum substitute. It could be used synchronously by MOVE instructions or asynchronously using DMA via the IOC. Its usage was more judicious in NEC high-end machines where "main memory" was implemented in expensive SRAM than in Bull machines where SRAM was not used and DRAM become progressively cheaper and cheaper.

Memory management

Paging was introduced only in the mid-1980s and until then the memory management relied only on the use of segmentation. At loading time, tentative segments from the load module were loaded in a large file named Backing Store. To each loaded segment was appended a header containing the access rights and the segment type. The header in addition includes the backing store relative address of the segment frame -the aggregate composed of the header and of the addressable segment-. 

Problems usually encountered in memory management are those of fragmentation: the waste of memory by the alignment of programs and data at a memory block (segment or page frame) boundary (internal fragmentation) and the waste of memory by the variable size segments to be allocated in large enough holes (external fragmentation). The initial implementation of virtual memory by segment aligned to a 16-bytes (later 64) boundary and size reduced considerably the internal fragmentation. The variable segments size was more difficult and had sometimes to trigger a "compaction" of memory (i.e. moves of segment to increase the size of "holes". Hardware was optimized for efficient moves decreasing the load of such a procedure.

The loader stored the process group control block segment in physical main memory. That segment includes the segment tables for the threads of the process group. It loaded some segment frames in main memory according to residency criteria defined at linkage time. 

When memory management (VMM) was called -by a missing segment fault in a segment table- VMM looked for free main memory to load the requested segment frame. It selected the smallest memory area to accommodate the segment, updated the page table, loaded the segment frame and reorganized the list of memory "holes". If the "holes" were not large enough to store the segment frame, VMM attempted to compact main memory (moving segments and updating segment tables), a process that was faster than backing store I/O in Level 64 configurations. If no fitting holes were found, swapping out segments was needed: non-modifiable segments were never written back to backing store, so it was cheaper to just set up code segments as "missing" and to overlay their segment frame by the new one. If VMM had to swap out, constant size data segments, it retrieve the backing store address of the segment from the segment header and copied the segment in backing store. If the segment size was increased, the backing store address had to be altered.

The size of backing store was much larger than the physical memory, allowing many programs to be loaded in backing store in dormant state. Thrashing, i.e. high rate of segment swapping could occur, due to a large "working set" of segments, GCOS reacted by reducing the priority of offending threads.

When main memory size came expressed as several Megabytes and the backing store performances were not improved in parallel, the Virtual Memory management based on segmentation became more expensive specially memory compaction. With the Aquila project, that introduced a NEC designed processor on top of DPS-7 product line, GCOS VMM was modified to rely chiefly on paging for memory management. Segment swapping was still maintained for "overlay" management of well-behavior applications.

Files resources

Files are allocated to a job step; they can be passed from job step to job step. The ASSIGN command performs the binding of a file symbolic name (file code) and the real name of file in the installation. The real name is either a catalogued name, or a name specified by the volume name and label or even by an external name known by operator (unlabeled tapes). For sequential files, the program ignores the device type, until the processing of the ALLOCATE command. The ALLOCATE command request the mounting of files stored on removable devices (tapes, disc packs) after verifying that they are not yet mounted and available. The system relies on automatic volume recognition (AVR) letting the operator using any of the empty drive. 
Their type of access (READ, READ EXCLUSIVE, WRITE, UPDATE, WRITE EXCLUSIVE) has to be specified in the JCL command. 

The strategy is to allocate all resources needed for the execution of a step before starting the process group representing its execution. This not applies to the libraries that are created dynamically nor to system files created in behalf of the step.

A usage count is used for files opened simultaneously by several steps. Files may be dismounted only when unused. The operator may have to kill offending steps monopolizing a file (or device) resource.

As some steps run almost permanently, such as transaction processing (TDS) the access rights of the resources have to be handled by the customer operation in terms of access rights and concurrent access with other steps.

Device resources

Some resources (essentially discs) are shared between several process groups, but most of devices are used exclusively by application programs or by system utilities. So, most data from or to the devices are spooled by system process groups (input reader, output writer). 

Inter process Communication

An important thing in GCOS architecture is inter-process communication and inter-process group communication. The first is architectured using micro-kernel features. Communication between the system process group (J=0) and the other also use dedicated semaphores and shared memory. Communication between non-necessarily resident process groups used a mechanism via the file system (queued subfiles).

Semaphores

GCOS semaphores were designed after Dijkstra primitives. A semaphore is an architectured entity containing a lock and a count. The creation of semaphore is usually static (either as a system semaphore part of the application program interface or as a local semaphore set up by the linker). The operations available on semaphores are P-operation, V-operation and a P-Test operation. 
The lock is set by a P operation. If the lock is set, the P-Operation increments the count and put the thread in wait state and links its id in the semaphore queue. 
The V-operation decrements the count and, if it became null, reschedules the first thread of the waiting queue (i.e. put it in the ready queue). 
The P-Test is a complementary statement that allows the thread not to be stopped when the semaphore is locked. It is used to implement a WAIT on multiple-events and similar features.

Semaphores are handled by the micro-kernel in relation with the thread dispatcher.

Semaphores with Messages

Many semaphores concern system services: server threads are waiting new requests on a system semaphore. So semaphores may support messages (16 bytes strings) that are queued by client threads performing V-operations. The issuing by the server thread of a P-Operation on a semaphore with message blocks the server if no request (V-op) is pending and returns the first message in the queue if some have been notified.

Messages are usually pointers to more detailed data structures. The semaphores messages are stored in a special system segment used only by the micro-kernel.

Shared memory

Type 0 data segments are within the address space of all process groups. Most of them have restricted access (Read only and/or ring number). Type 1 data segments are not directly addressable because that address space is dynamically allocated. However, the system maintains a table of shared data segments allowing the sharing of the Type 1 space.
Type 2 data segments are statically assigned to all the threads of the same process-group.
Type 1 and 2 writable segments have to be protected against concurrent access by a locking mechanism, usually locking semaphores.

Event management

An additional mechanism has been initially introduced in GCOS, but its role decreased with time because its lower efficiency. Event Management was a software (not micro-kernel) supported mechanism that stored messages (like messages semaphores) in the file system (in queued files). The facility did not require that the server was active when the client posted its messages and extended the limits on the size and the number of messages.
However, the processing and the elapse time for operations on events caused this mechanism to be of limited use.

Dispatching of threads

Threads are either stopped, blocked or ready for execution. When a thread has been started, it passes under the control of the micro-kernel. The dispatching of the ready threads is made by the micro-kernel according to the thread priority. The micro-kernel works according the thread priority, threads with the same priority are allocated slices on a round-robin basis. 
Threads may be allocated a time-slice after what they are interrupted and put at the end of the queue of the ready threads with the same priority. That feature was made available after a few years, when interactive applications became predominant.
Threads blocked themselves when they issue a P-operation on semaphores already set by another thread. They became ready, when another thread issued a V-operation on that semaphore.
So all normal dispatching was made under the efficient micro-kernel control, letting to GCOS scheduler the initiation or the killing of threads.

Initialization 

The hardware initialization of the system evolved somewhat during the time, but software bootload did not change deeply. The initialization of the hardware configuration (specially peripheral controllers) was finished under SIP System Initialization Program that was loaded by firmware from the boot device. Note that the modification of the boot device was made through operator commands to the service processor.

SIP role was to read the system configuration table and to update it according the real hardware availability and to load the optional firmware components. After that phase, control was given to GCOS ISL a real memory version of the operating system that verified or built the resources needed by GCOS.

As data integrity was crucial to GCOS, all system malfunctions were handled by special machine integrity assurance routines (MIAR) that attempted to recover from possible memory, IOC and processor errors. In fact, the reliability of hardware was much higher that previous systems and there were no many occurrences of MIAR. MIAR operated as fault procedures called in the user threads (when the error was detected on the execution of processor instructions) or in asynchronous threads (when it occurred in I/O). In SMP configurations, MIAR had the possibility to remove a processor from the configuration and to siphon the cache of that processor. The dynamic reconnection of a processor was also possible from the service processor. As main memory had auto-correction capability (by ECC), MIAR able to isolate blocks of memory were infrequently called. GCOS did not implement in software a relocation of the operating system control structures, so non-corrected memory errors usually called a system termination. The service processor initiated memory-checking lead to the isolation of the memory block.

Multi-processor, Partitioning and Multi-computer configurations

The first generation of GCOS systems was supporting only single processor configurations; the architecture had been designed to support several (initially up to 8, later -in Artemis- up to 24) processors in a shared multi processor (SMP) configuration. Hardware became available in 1981. According to the architecture the functions of allocating CPU to threads was under the responsibility of the micro-kernel.   The operating system run unchanged in single processor or multi processor. A few changes were needed just to improve performances by decreasing the path of critical sections of the kernel protected by a lock semaphore. 
The micro-kernel itself was running in all the processors and when a thread relinquished control the firmware dispatcher look in the threads ready queue to select the new thread to be given control. The threads ready queue was protected by a hardware lock that inhibits access to the related memory bank.
To avoid that such dispatches contribute to flush the L1 processor caches, some "affinity" between processor and cache was recorded in thread status to give preference to a thread that might have still some data in the cache. A similar procedure was used for dispatches of Artemis that shared a L2 cache between each cluster of four processors.

The architecture provided a way to run specific threads on specific processors as a provision for specific features available in some processors. Such an asymmetry was unused until the mid 1990s, where different pricing was applied to the Artemis processors, accelerating the DB and Unix processors and slowing down GCOS application processors.

With the availability of SMP on P7G, Siris 8 emulation coexisted with the first test of GCOS7 by converted customers. A facility was offered to partition the SMP hardware in two systems. IBM offered also this feature on S/390, but several years after Bull. The same system could be operated during some shifts in a dual-processor configuration, and in other as two independent GCOS systems.

With Artemis that could support up to 24 processors, the partitioning facility was extended.

Distributed Processing & Networking

Although, the hardware systems on which GCOS run since 1979 had Distributed Systems in their named, GCOS is NOT in itself a Distributed Operating System. Its usage of heterogeneous peripheral processors has been the pretext for this commercial name. But all the architectural entities of GCOS, with the exception of files, have only a meaning inside one system. Even when a MP hardware system was partitioned in several software systems, no mechanism had existed to provide a direct communication between GCOSes. There has been studies made in the 1980s to attempt to architecture more efficiently Distributed Processing in clusters of homogeneous systems, however those studies did not materialize. They face the issue of the implementation of distributed processing across Honeywell and Bull heterogeneous systems under evolving standards (Bull-Honeywell DSA, ISO and finally TCP/IP). Even a standardized function like RPC -remote procedure call- had not been architectured in GCOS.

Telecommunications

During the life of GCOS, many changes occurred in the field of telecommunications. In the 1960s, excluding the time sharing applications that were not yet considered as market for general purpose computers, two main usages were considered: remote batch operation to allow the decentralization of input/output and direct access to terminals to allow data collection and other real-time processing. The idea of formalizing data collection in the programming system through a message queuing facility emerged in the early 1970s via the CTG of Codasyl. However, that approach did hardly satisfy the operation of multiprogrammed systems and had to be integrated in the OS.
A major transformation was the universal acceptance of transaction processing that revolutionized the industry. TP applications became the heart of business applications in the computer, batch-processing retiring as ancillary activities.
Another revolution occurred in the late 1970s, that is the recognition of networks of computers. The illusion that a single system could cope with all the needs of a company or an administration faded and networking became a key point in the architecture of a system. Until the mid 1980s, most manufacturers dreamed to dominate the customer computing world and proprietary systems were developed. Finally, all ended by accepting open networks that allowed all type and makes of computers to be connected to a worldwide network.

GCOS basic architecture was used to build several implementations of products corresponding to that evolution of needs.

To support the message queuing facility of CTG, the GCOS COBOL compiler generated not only the calls to the SEND and RECEIVE procedures, but also the parameters of the process group handling the message processing. The launching of the COBOL application triggers also that of the message processing. Queues of messages was done through access to a set of Queued files that were filled or emptied by the Message Control Program.

Direct access was supported by a BTNS (Basic Terminal Process Group) process group handling terminals and accessed by the user program through shared memory (based on semaphores). BTNS supported its JCL controlled communications lines. All terminals located on the same communication line were to be allocated to BTNS.

After the CII merger, CII-HB and Honeywell embarked into the definition of a competitive solution against IBM's SNA the HDSA Honeywell Distributed Systems Architecture. HDSA differed from SNA by its orientation around a Network Processor instead of main frames. HDSA was designed at the same time as the ISO OSI Open Model and adhered somewhat dogmatically to that layered model. The presentation and application levels were to be implemented in terminals and hosts while the interface between NCP and hosts would be the session layer. HDSA was implementing only the "connected" mode, following the tradition of PTT's that were already offering X-25 protocols. If many applications were taking advantage of that "connected mode" (Telnet, File Transfer...), establishing a session for each transaction caused much overhead over the system.

The Network Processor was a Level 6 minicomputer and was used either as a Front-end (FEP) of a GCOS mainframe or as a remote Network Processor. The interconnection between a DPS-7 and the FEP was via PSI channel. It violated somewhat the HDSA architecture concepts, because it did not offer strictly a session interface but multiplexed the data exchanged over the opened sessions as packets transmitted between the FEP and a GCOS resident program, named FNPS. That FNPS program controlled the FEP PSI interface on a couple of logical channels on which it read and wrote data continually on looping channel programs. In addition to that, FNPS was performing the bootloading and the initialization of the FEP, when the FEP was disk-less -the general case for DPS-7 systems. 

Distributed Applications

DSA, as HDSA was named after the nationalization of CII-HB, offered the following functions:

  • Terminal control for all direct access applications
  • File transfer and remote batch processing
  • Security
  • Cooperative transaction processing (mapped on SNA LU6.2)
  • Access to IBM SNA terminals -through Janus options in FEP-

HDSA had always been considered as a close-gap and a learning phase before a manufacturer independent ISO standard be adopted. Unhappily for Bull and Honeywell, their intense lobbying to attempt to have HDSA choices adopted by ISO was hurting not only IBM and Digital that had their own standards (IBM improving SNA, and DEC adopting "non-connected" mode), but also the scientific and university worlds adopting en masse the TCP/IP and UDP/IP architectures. Some ISO standards were adopted, functionally close to DSA, but requiring many implementations. Bull and some of its customers speculated that ISO would take over eventually but had their UNIX systems implementing TCP/IP. So, a progressive conversion to the world of Open Systems occurred in the late 1980s. 
The DPS-6 minicomputer was discontinued and a new front-end processor for DSA was based on the 68000 architecture. The evolution started on the low end DPS-7000 with a micro-FEP directly connected to the Multibus II I/O bus of those systems. However, firmware emulated a PSI interconnection. Only a part of NCP software was ported from Level-6 to the 68000 architecture. In a second phase, the DPS-6 FEP and Network processor were replaced by 68000 systems supporting DSA. In addition, similar hardware, now under UNIX (SPIX) operating systems insure the interconnection to the TCP/IP world.

GCOS did not initially support natively the TCP/IP protocols and applications. Instead, the TCP/IP world was attached to a UNIX subsystem OPEN7 operating under GCOS that then communicate with the GCOS world.

Later, in 1996, TCP/IP stack was implemented natively under GCOS, progressively decreasing the role of DSA to the support of legacy terminals

© 2001-2003 Jean Bellec