Kort Explores conundrums of the human condition

Humanity has always been obsessed with information. Our need to share information with our fellow man caused us to evolve ever more complex languages, eventually developing the ability to write things down. Writing opened up the possibility of communicating with people over time, of preserving information beyond the lifetime of the individual.

Down through all of recorded history people have tried to devise ways to leave a permanent account of their time here on Earth, and to pass on their accumulated wisdom. Today we refer to this ever growing body of knowledge as recorded history, and it is key to our understanding of the past. The ancients painted on cave wall, carved symbols into rocks, scratched clay tablets, and smeared ink on a wide variety of natural and manufactured materials.

Basic human motivations haven't changed much over the centuries, only now we have a lot more people with much more powerful technologies at their disposal all contributing to the growing mass of accumulated information. This newsletter and its on-line website are a prime example.

The sheer volume of information generated by modern civilization would bury us all if it had to be recorded on clay tablets - or even printed on paper. The rate at which mankind generates information had been slowly increasing since the first written language, and really started to ramp up in the latter half of the current century. The rate is often referred to as information doubling theory.

Depending on whose calculations you use, information doubling goes something like this: Take every scrap of information generated by mankind from day one of recorded history up the current moment. The time it takes to double the amount of accumulated information is the doubling period. By the second half of this century the time required to double the information accumulated over thousands of years had shrunk to a matter of decades. The doubling period has continued to shorten until now information only takes a little over a year to double. And the rate of increase continues to accelerate. Before long we'll be in monthly then weekly doubling periods. In another decade we could well be into an hourly doubling period.

Only the high density storage offered by computers, microfiche, etc. make modern life possible. But we've made some significant tradeoffs in our rush to embrace these new high tech information systems.

There's a saying that goes something like: "To err is human, to really screw up takes a computer." The much publicized millennium computer crisis predicted to bring the world to its knees in 3 years is basically a simple housekeeping matter blown into crisis proportions by the sheer volume of information that computers have generated over the last couple decades. The problem originated back in the 70's when programmers tried to save a little then-expensive storage space by storing years as 2 digits instead of 4.

At every opportunity to fix the problem easily and relatively cheaply, managers decided to defer the cost and preserve the problem. The problem spiraled into a crisis when instead of fixing the accumulating volume of flawed data, new generations of software were written with the same flaw, carrying the problem forward for someone else to fix later. This postponement of the solution allowed the problem to be compounded over and over through the doubling of flawed information.

The obvious solution is relatively straightforward - convert all the 2 digit year databases and programs to use 4 digits years. The crisis grows out of the sheer volume of data and software that must now be fixed, and the opportunities for mischances when management again tries to cut corners and finesse the problem. The mismanagement of the millennium crisis doesn't bode well for information preservation and management in the future.

Another major tradeoff in modern information technology is the relative impermanence of the underlying media. While the actual lifetime will only truly be known when it degrades beyond usability, the projected life expectancy of most common modern media is distressingly short. Flexible magnetic media like floppy disks, VCR tapes, cassette tapes, etc. are expected to last only a couple of decades. Seemingly durable media like CD's are projected to last a similar lifetime before oxidation and chemical decomposition make them unreadable. Black and white photographs have survived for over a century with no fading, but the dyes in modern color photos are unlikely to be visible after half that long. Books published in the middle ages are still readable today, but the high acid content in the paper used for many books printed today make it unlikely they will still be readable in 50 years.

Perhaps the most risky tradeoff is accessibility. The ancients carved their stone and penned their scrolls to be directly readable down through the ages. In spite of the intentions of the creators, changes in language have made many of them difficult if not impossible to decipher. Countless modern scientists have spent their lives trying to reconstruct lost languages so that the ancient inscriptions could once again be understood.

The information deemed important enough to keep accessible has had to be translated from language to language as old languages died out and new ones took their place. For example, in order to carry forward what we know of the science and technology of ancient Greece over the thousands of intervening years, it was necessary for the information to be first translated into Arabic where it survived until after the European dark ages. It was then translated into Latin during the Renaissance, before being translated again into various modern languages.

We almost lost the ability to decipher ancient Egyptian hieroglyphics when the language died out. If not for the Rosetta Stone, we might have permanently lost the ability to access the clearly visible information. In South and Central America, the histories of whole civilizations are recorded in frustratingly indecipherable glyphs written in languages that were already lost before the days of the Mayans.

Even works written in languages that are still nominally in use have become difficult for large segments of the population to access. For example the English language has changed so much over the centuries that few products of modern American education can understand the works of Shakespeare or Chaucer in their original form.

But as if the basic language wasn't enough of a barrier, modern information storage encodes content into an abstract form that requires a transitory technology to access. A floppy disk is useless without a computer to read it. The movie on a VCR tape loses a lot in the translation without a VCR and TV to convert the abstract signal into a human perceivable form.

The problem occurs when the underlying technology becomes obsolete and disappears. The relatively short useful life expectancy of the media becomes moot when you consider the rate at which new storage technologies are developed and "old" ones become obsolete and disappear. 8-track tapes were all the rage in the 60's. But if you burn out the player in your Rambler and need a new one to listen to all those treasured cartridges in the glove compartment, you'd be hard pressed to find a working replacement. In another 30 years finding a replacement CD player will likely be as difficult as finding an 8-track player in good condition today.

The rate at which computer technology is changing is even faster than most other types of data storage. And as each new generation of computer appears, the ability to access the artifacts from the previous generation is compromised. Unless users took the time to convert their data as they upgraded their computers, the information they stored on computer disks as short a time as a decade ago is already verging on being lost.

As an example of the risks of computer based data storage, consider the following story of a story.

Back in 1984 I used a state of the art word processing system to write "The Closet" - a short story that appeared in POC a couple of years ago. I used an early laptop computer to write the rough draft. The laptop had a microcassette tape for data storage. When I returned home I transferred the story from the microcassette in the laptop through a serial cable into a desktop computer where it was written onto a floppy disk.

Both the laptop and desktop computers were CP/M based. I develop software and when CP/M lost out to Ms.Dos in the desktop computer market, I had to upgrade my computer system. My new computer couldn't read the disks from my old computer. However, since my old computer was still working, I was able to link it to the new one with a serial cable and transmit The Closet over to the new system. That got the story onto the 5 1/4 in. double sided double density floppy disks my first Ms.Dos based computer used.

Over the years newer computers have dropped the ability to access the early Ms.Dos format disk on which The Closet was stored. I mounted a drive in my current computer that can read but not write to the early Ms.Dos disks, but which at least allowed me to copy the files onto first high density 5 1/4 in. disks and then later onto 3 1/2 in. disks. Someday the current 3 1/2 in. disks will become obsolete and unsupported, requiring yet another conversion.

To further complicate matters, the software I used to originally write the story also employed its own peculiar format, making the data files unusable by other software. Fortunately the next word processing software I used had a built in converter to allow importing files written with the old software. The format had to be translated again last year into HTML in order to make The Closet accessible over the web.

Consider that the knowledge of ancient Greece started out on relatively stable media and only required translation 3 times to keep it accessible for over 3 thousand years. The Closet had to be translated 4 times onto different abstract storage media and converted into different internal formats twice in just 14 years in order to keep it accessible.

I kept converting The Closet because I believed I'd find a use for it someday. It's now expressed in a variety of media - printed copies of POC, computer disk, and on the web. Perhaps at least one of these will prove more durable than the original CP/M floppy disks.

The stories I didn't think were salvageable I left on the original disks. Now those stories (among them undoubtedly my masterwork) as well as all the other old disks full of data I didn't think were worth the effort to convert at the time, have already become effectively lost to posterity.

And I'm just one minor computer user. Consider how much information has been generated over the last 50 years by all the world's computers that is now lost on deteriorated or obsolete computer media. What critical information needed to solve the problems of tomorrow have we allowed to slip out of our grasp because somebody decided it was too much bother to preserve on new media? How much current effort is being wasted recreating information that was lost because the technology needed to access it became obsolete?

As we put more and more information on short term technology dependent media, we commit ourselves to an ever increasing overhead just maintaining the existing information. And the more effort we must allocate to maintaining old information, the less time and energy we have for creating new information.

The problem will continue to get worse as both the information doubling period and the operational lifetimes of new storage technologies get shorter. What is needed is a durable means of long term high density information storage that is good enough to fend off obsolescence for at least several decades if not centuries. Storing information in the molecules of crystals or quantum effects of atoms might push the physical limits hard enough to remain a standard long enough to at least slow down the loss of information through obsolescence.

If we continue in our current pattern of increasingly rapid obsolescence, we may be faced with the unpleasant choice of either allowing more and more of the past to slip away through obsolescence so we can concentrate on the future, or restricting the future so we can concentrate on preserving the past.