The basic concept behind the Internet is to have a distributed, decentralized communication network that can continue to function even with the loss or addition of nodes.
1957 - U.S. Advanced Research Projects Agency (ARPA)
1969 - ARPANET (1965 - 1969 planning & design)
1986 - NSFNET
1991 - WWW & Gopher
1995 - NSFNET back to research
1996 - Internet2 research only network
Both the Internet and the systems that run it are built up in independent layers.
Open Systems Interconnect is a model which allow communication largely independent of software and hardware. Each layer of communication only needs to interact with adjacent layers making it easy to replace, modify, or upgrade any given layer without having to redo the whole stack. Or that one layer can be produced by one company, organization or consortium and another layer by a different one.
| Layer | Description | ||
|---|---|---|---|
| 7. | Application | authentication, privacy, data syntax requirements e.g. telnet, ftp, http, stmp, ntp | |
| 6. | Presentation | transforms from application to network format, and vice
versa e.g. encryption | |
| 5. | Session | sessions for stateful protocols | |
| 4. | Transport | hides lower layers, ensures complete data transfer e.g. TCP, UDP | |
| 3. | Network | routing, flow control, error control e.g. IP | |
| 2. | Data link | actual physical signal e.g. Ethernet | |
| 1. | Physical | electrical, optical, and physical specifications e.g. voltages, cable and connector types (RS-232, RJ-45, 10BASE-T, T3) | |
In theory there are 7 layers, but in application it is common that several are merged.
It is also common to joke about two or three additional, unofficial layers. The 8th, 9th and 10th layers being the financial, political and religious layers.
Unix and similar operating systems play an integral role in the development of the Internet.
First and foremost, these are modular and designed from the beginning to operate in a networked, multi-user, multi-tasking environment. The philosophy is to create small, often simple tools, that can be combined in many ways to perform often complex tasks. A tool should be as simple as needed, no more no less.
Below is a very abridged list of some common operating systems found around the Internet.
AIX
BSD/OS
Darwin
Debian
GNU/Hurd
Debian
GNU/Linux
Debian
GNU/NetBSD
FreeBSD
GNU
GNU-Darwin
HP-UX
IRIX
Linux
Macintosh OS X
Macintosh OS X
Server
MicroBSD
Minix
NetBSD
NeXTSTEP
Open
UNIX
OpenBSD
OpenServer
OPENSTEP
OS/390
Unix
Plan 9
QNX
QNX/Neutrino
Solaris
SPIX
SunOS
Triance OS
Tru64
Unix
Unicos
Because these systems are modular, it is possible to borrow good ideas or take out unneeded parts. See the Unix Timeline for more details about how some of the below have evolved. Over time, bad ideas have been dropped, and good ideas spread. In other cases, some systems have become specialized, and others have become generalized.
Or parts can be replaced. For example when most people refer to a "Linux" distribution, they usually mean all of the tools, most of which come from the GNU project. In that case, the jjj
Some systems, like QNX, specialize in reliability so with the proper hardware they can run with no restarts, reboots, or crashes (neither scheduled nor unscheduled) for 5, 10 or more years. This is what you want controlling your airplanes, subways, ships, telephone networks, hospital equipment and other essential services. There is no excuse to use these essential services for a testing ground for unfinished products, no matter how good the marketing.
The Netcraft Uptime Survey lists how long some of the more popular web sites have been running uniterrupted. It offers several views like ISP avg.
Like the OSI model, each of these operating systems is built up in layers, with the most basic, essential tasks at the center. Each layer only needs to communicate with adjacent layers, in that way any individual layer can be replaced, modified, or upgraded as desired. This also contributes to the overal robustsness of the system making it both more stable and more secure.
The kernel is the heart of the system. It can contain just barely the bare essentials or everything up to an including the end-user programs and interface, depending on the philosophy and goals of the designers. Here as elsewhere, there the fewer things there are, the less likely that one of them will go wrong. And, as with other things, the simpler it is, the less likely it is to go wrong in the first place. Thus, some systems use what is called a microkernel.
| Operating System Layer | ||
|---|---|---|
| User interface (optional) | ||
| User applications | ||
| Operating system services (optional) | ||
| System calls | Kernel | |
| Kernel services | (Microkernel) | |
| Device drivers | ||
| Hardware | ||
Everything is a file, which makes things easy to deal with. Programs are simple enough to accomplish their task and given just enough access to be able to accomplish the task, making for a secure system.
CPUs are designed and optimized for different specialties, e.g. speed, timing, simple instructions, low power, etc. The CPU in your microwave will not need the same capability as the one in your mobile phone, nor as in your notebook, PDA, or desktop machine.
Some common, and a few once common but now famous, CPUs include 650x, Z80, x86, PPC, Itanum, SPARC, Alpha, Opteron, and ARM. As with everything else, it is a matter of finding the right tool for the job rather than a one-size-fits-all philosophy.
For example, the Debian distribution can choose to run on 11 different architectures, and can choose from either Linux, Hurd or BSD kernels. OpenBSD can run on 14 different architectures, and so on. Either can be optimized for special tasks, have unnecessary parts removed, or extra tools added.
Being able to choose to run the operating system on more than one architecture gives a lot of power through flexibility.
" Many people don't realise that when the computing age started it was just considered natural to share your source code with other people, just like the academic community shares ideas via journals and scientific collaboration. "
- Andrew Tridgell (samba / rsync)
http://www.zdnet.com.au/newstech/os/story/0,2000048630,20280072,00.htm
As a development methodology most work related to the Internet is done similar to traditional scientific method and scholarly publication. An idea, in the form of a program, algorithm, or routine, can be spread, used, simplified, or built upon by others. A few decades ago, this mostly went without saying. Now it is stated explicitly.
Open source can also be thought of as a development model, which follows closely scholarly research and publishing. Ideas and improvements are discussed and shared freely, so that less time is spent re-inventing the wheel and more time spent making progress.
A corresponding metaphor from Finnish culture is talcoot.
Another advantage is the short turn around time for patches and bug fixes. In comparing MSIE and IIS vs Mozilla and Apache, the latter have had fewer problems and those problems have been solved faster and often are less severe. The former have had known problems for months and in some cases years without being addressed.
It is also possible to start new projects from old, or share good ideas. The *BSDs are a good example. FreeBSD, OpenBSD, and NetBSD all have a common ancestor, but different priorities. NetBSD's focuses on portability, that is having it run on every piece of hardware imaginable. FreeBSD's concentrates on performance, scalability, managability, robustness. OpenBSD specializes in security, with only two remote holes in the default install in over 7 years. When improvements and breakthroughs get established and then polished in one branch, they quickly percolate through the others.
For a while, peaking in the mid 1990's, we were starting to see many desktop applications and systems, and even a few servers, where the innerworkings were kept hidden by the creator. One of the core differences between open and closed models is the license by which people are encouraged or restricted from using, distributing, modifying, or analysing the software.
Prior to the late 1990's, the main open source licenses were the GPL, LGPL, BSD, and MIT. Since then Mozilla, Apache and dozens of others license variations.
Free Software is a specific type of idea, stemming from the GNU project, founded in 1984 by Dr. Richard Stallman while at the MIT AI lab. The idea is to have a license that encourages collaboration.
The GNU Manifesto was written by Richard Stallman and published in Dr. Dobb's Journal of Software Tools Volume 10, Number 3, March, 1985 It outlines the goals and objectives of the GNU project.
"Free software is a matter of the users' freedom to run, copy, distribute, study, change and improve the software. More precisely, it refers to four kinds of freedom, for the users of the software:
- The freedom to run the program, for any purpose (freedom 0).
- The freedom to study how the program works, and adapt it to your needs (freedom 1). Access to the source code is a precondition for this.
- The freedom to redistribute copies so you can help your neighbor (freedom 2).
- The freedom to improve the program, and release your improvements to the public, so that the whole community benefits (freedom 3). Access to the source code is a precondition for this.
A program is free software if users have all of these freedoms."
Those 4 freedoms are encoded in the GNU Public License (GPL). The main bone of contention which differentiates the GPL from other open source licenses is that if the code or modifications to the code are distributed, then the source code or modifications must be part of that distribution. Distribution is not obligatory, but if it is done, then it must include the source code.
The principle is more general than those the GNU project. When developers can use, read, copy, change the source code, the software improves at a rate much faster than capable in the close source model.
- Free Redistribution
- Source Code
- Derived Works
- Integrity of The Author's Source Code
- No Discrimination Against Persons or Groups
- No Discrimination Against Fields of Endeavor
- Distribution of License
- License Must Not Be Specific to a Product
- License Must Not Restrict Other Software
- License Must Be Technology-Neutral
From the Open Source Definition
Looking at the code is not the same as being able to change and compile it. One cannot trust code that one did not create. This was proven and illustrated by Ken Thompson in his article "Reflections on Trusting Trust" from Communication of the ACM, Vol. 27, No. 8, August 1984. For example, in order for a government, agency, or business to guarantee that a program is free of backdoors or other compromises, not just the code must be available, but also the tools and operating system needed to compile that code must be gone over by hand.
Open standards are considered even more important to a good future for the Internet and computing as a whole. Open standards are very good for, but does not require, open source.
"To attain interoperability in the context of pan-European eGovernment services, guidance needs to focus on open standards.The following are the minimal characteristics that a specification and its attendant documents must have in order to be considered an open standard:
- The standard is adopted and will be maintained by a not-for-profit organisation, and its ongoing development occurs on the basis of an open decision-making procedure available to all interested parties (consensus or majority decision etc.).
- The standard has been published and the standard specification document is available either freely or at a nominal charge. It must be permissible to all to copy, distribute and use it for no fee or at a nominal fee.
- The intellectual property ? i.e. patents possibly present ? of (parts of) the standard is made irrevocably available on a royalty-free basis.
- There are no constraints on the re-use of the standard. "
From the conference, Open Standards and Libre Software in Government, 18 Nov 2004, The Hague, The Netherlands.
Eric S. Raymond summarizes benefits on a general level.
From Halloween I, 1998
Again, it is useful to point out that there were many networks in the 1960's and 1970's, but these could not take off because their protocols were proprietary.
MS on OSS:
- OSS poses a direct, short-term revenue and platform threat to Microsoft, particularly in server space. Additionally, the intrinsic parallelism and free idea exchange in OSS has benefits that are not replicable with our current licensing model and therefore present a long term developer mindshare threat.
- Recent case studies (the Internet) provide very dramatic evidence ... that commercial quality can be achieved / exceeded by OSS projects.
- ...to understand how to compete against OSS, we must target a process rather than a company.
- OSS is long-term credible ... FUD tactics can not be used to combat it.
- Linux and other OSS advocates are making a progressively more credible argument that OSS software is at least as robust if not more than commercial alternatives. The Internet provides an ideal, high-visibility showcase for the OSS world.
- Linux has been deployed in mission critical, commercial environments with an excellent pool of public testimonials. ... Linux outperforms many other UNIXes ... Linux is on track to eventually own the x86 UNIX market ...
- Linux can win as long as services / protocols are commodities.
- OSS projects have been able to gain a foothold in many server applications because of the wide utility of highly commoditized, simple protocols. By extending these protocols and developing new protocols, we can deny OSS projects entry into the market.
- The ability of the OSS process to collect and harness the collective IQ of thousands of individuals across the Internet is simply amazing. More importantly, OSS evangelization scales with the size of the Internet much faster than our own evangelization efforts appear to scale.
From Halloween I, 1998
The word "Free" is usually used in the meaning of liberty rather than price. As one South African wrote in a discussion forum, that freedom is a primary benefit in and of itself. It would be ridiculous to talk of either an apartheid or free state as having a higher or lower cost.
Independence can be in the form of better licensing conditions, like Ernie Ball Guitars has done.
Or it can be independence from a given architecture.
In addition to Ken Thompson's article above, there is also the issue of peer review. The more people using, examining or even auditing auditing the code (OpenBSD does that) the higher likelihood that a problem will be spotted.
And there is fast turn around time for bugs with more people. e.g. Apache has sometimes issued patches within hours of being notified of a problem, and when the issue goes for some days a temporary work around is describe.
Programs that run on Linux, BSD, Macintosh, MS-Windows and a few others can be written using qt or gtk+.
Overgeneralized here for the sake of simplification and lump several things into this category:
OpenOffice.org, http://www.openoffice.org/
Just to give an idea how much work has already been done and can be added to any of your projects, here are some selected examples:
For more, see Freshmeat or recently released GNU software.
So, what was the meaning of parts I - III?
Their meaning is to show that there is the option to use and combine many pre-existing peices to solve library problems and create new services. There is also the liberty to copy, customize or improve these pieces.
Knowing about the tools is more important for librians than knowing how to build with them. These tasks would require collaboration with colleagues in computer science or similar fields.
These seemingly complex results are built of modular building blocks. It's possible to hire someone or a project to complete a task you don't have the time or skill for and that there are many other options beyond the ones below.
Combine an OS (e.g. BSD or Linux) and install a web server (e.g. Apache or other, smaller ones)
Add a Z39.50 client to your web server, like YAZ from Indexdata.
Install a desktop system (KDE, Gnome, Fluxbox, etc.) and strip out all the parts that aren't used or needed.
Modify Koha or Greenstone.
Customize RT (Request Tracker) or a similar tool. Or write a new one.
Use a proxy like Squid.
Use OpenLDAP + Kerberos
Look at Samba, AFS, or Netware (not OSS ... yet)
Combine the last example, file sharing, with the first one, a web server. i.e. the web server and the web server where they have access to eachother.
Brooks, Frederick P. The Mythical Man-Month,
Anniversary Edition : Essays on Software Engineering. 2nd
edition. Addison-Wesley Pub Co (1995). ISBN-0201835959
Note: See especially Chapter 2 "The
Mythical Man Month", Chapter 3 "The Surgical Team", Chapter 11
"Plan to Throw One Away", Chapter 14 "Hatching a
Catastrophe"
Daffara, Carlo (ed.) &
González-Barahona, Jesús M. (ed.)
Free Software / Open Source: Information
Society Opportunities for Europe? Working group on Libre
Software. April 2000. Version 1.2
URL: http://eu.conecta.it/paper/paper.html
The September that Never Ended
http://www.catb.org/~esr/jargon/html/S/September-that-never-ended.html
Patrick Maslen
Control,
Change and the Internet. Honors Thesis. Final draft, Oct
1996.
URL: http://home.vicnet.net.au/~qbird/cci-intr.htm
Himanen, Pekka. The Hacker Ethic and the Spirit of the Information Age. (2001) New York: Random House. ISBN 0-375-50566-0.
Moody, Glyn. Rebel Code: Linux and the Open Source Revolution. (2001) London, UK: Allen Lane The Penguin Press. ISBN 0-713-99520-3.
Raymond, Eric S.
The Cathedral and the Bazaar.
(1997/2000)
URL: http://www.tuxedo.org/~esr/writings/cathedral-bazaar/
Stallman, Richard M.
GNU Manifesto
http://www.gnu.org/gnu/manifesto.html
Adam Gaffin with Jörg Heitkötter.
Big Dummy's Guide to the Internet Texinfo Edition 1.04, Apple
Computer & The Electronic Frontier Foundation. Jan 1994.
URL:
http://www.math.fu-berlin.de/bdgtti-1.04-htmlroot/bdgtti-1.04_toc.html
Henry Edward Hardy.
The History of
the Net. Master's Thesis School of Communications. Grand Valley
State University. Allendale, MI 49401, USA. v 8.5 28 Sep
1993.
URL: http://www.vrx.net/usenet/history/hardy/
Christopher Koch
Your
Open Source Plan CIO Magazine. 15 Mar 2003.
http://www.cio.com/archive/031503/opensource.html
Robert H'obbes' Zakon.
Hobbes'
Internet Timeline. v7.0 1993-2004.
URL: http://www.zakon.org/robert/internet/timeline/
The
7 Layers of the OSI Model. Webopedia.
http://webopedia.internet.com/quick_ref/OSI_Layers.asp