last updated: Fri Aug 26 22:00 BST
Ideal vs Reality: The chain of Free Software Development in Hardware Products
Linux, GNU/Linux and derivative Operating Systems have effectively won,
and are in danger of being overwhelmed by their success. The GPL,
which is part of that success, is only effective if it is actually being
enforced or respected. It helps therefore to illustrate the "ideal"
scenario and then to also describe what's actually occurring.
Then, lastly, the question is asked: what can be done to offset the
stark discrepancy between the ideal scenario and reality? Within the
question lies the answer: take a risk - lead by example, and begin a
mass-volume Product (or range of Products) that is both desirable
and at the same time honours and encourages Free Software principles and
practices.
The Ideal Scenario
These scenarios are the chain of events that, if they took place, would
be the absolute ideal, as throughout every case, Free Software Licenses
and Free Software principles and practices are respected and honoured.
(Note: it has to be pointed out that many of these scenarios occur
in isolation: they just don't often line up all at once to create a
successful mass-volume product!)
A Software Company takes or begins a GPL Source Code Project, funds
it, publishes all Source Code at all times, documents it, encourages
and also actively allows Free Software Developers to contribute, and
actually completes the Project to useable and useful milestones. At no time
is an NDA signed by any parties to gain access to the Source Code of the
Software Project.
A Hardware Peripheral Chip Manufacturer creates a device, creates
a Free Software Compliant Device Driver, documents it, releases all
details on the device (Device Driver Source Code and documentation) without
requiring an NDA, completes the chip and successfully sells it in enough
volume to pay for its development cost and for future products.
A Fabless Semiconductor Design House creates a Hard Macro for a
peripheral that can be licensed to go into a CPU. They document it,
write a Device Driver and associated Software Tools and Libraries to use
it, and publish all Software and Documentation under Free Software Licenses
without requiring an NDA to be signed.
A CPU SoC Fabless Semiconductor company creates an embedded CPU,
with Hard Macros licensed from Fabless Design Houses. They port various
Operating Systems to it and various Free Software Kernels (such as Linux),
cooperating and collaborating with Free Software best Development Practices,
releasing full documentation, Board Support Packages and even full CAD/CAM
drawings for an example PCB - all without requiring an NDA and all respecting
Free Software Licenses. They then sell enough units in order to pay for
the development costs and for future products.
A Hardware Company creates a Product, and takes, amongst other Software
Projects, a number of Free Software Projects, funds the improvement and
integration of all Free Software Projects to customise it for the Product,
releasing all Software, documenting all Software changes, provides access to
the Free Software Components at all times, even encourages and actively allows
Free Software Developers to contribute, and actually completes the Product,
ships it, and sells it in enough volume to pay for its development cost and
for future products.
Now this may seem strange, but there are actually precedents for all of the
above actually occurring, including, in some cases, the actual development
of the Hardware itself - including in some cases the full CPU and the
peripheral's VLSI / VHDL designs! These include:
OpenCores.org: fully-working
CPU designs as well as peripherals are being developed as VLSI / VHDL
Projects, the most advanced and successful of which is the
OR1200 ORSoc.
Its predecessor, the OR1000, actually made it into a production chip.
The Beagleboard was the first
of a line of CPUs directly from CPU Manufacturers who released not only
the full documentation without an NDA, but also complete CAD/CAM Drawings
for the example engineering board. Freescale, with the
IMX53QSB
also followed suit, as have Samsung with their
Origen Exynos 4210 board.
By contrast, however, the actual number of successful commercial devices which
honour the GPL, follow and respect the spirit as well as the letter of Free
Software Development, can be counted on the fingers of one or two hands -
one if you are
Polydactylic.
(The OpenMoko does not count, as despite its successor being still in
development, it has not yet gone beyond prototypes. At least the casework,
which is a massive part of the cost of development, can and is being used,
in the development of the next revision, the
GTA04)
This is the complete list of devices (known to the author at the time of
writing). An analysis of these devices shows that:
Not one of them is an Android Device. Android's "Free" OS is developed
in secret by Google, then shipped secretly to members of Google's Club.
The highest performance CPU on the list is the 10 watt 1ghz Loongson 2F,
which uses 33mhz PCI bus Architecture (from 10 years ago) and has no 3D
Graphics.
The second-highest performance devices all exclusively use the Texas
Instruments OMAP 35xx Cortex A8 series CPU
Even the the next OpenMoko prototype, the GTA04, uses the OMAP 37xx.
Correction: AlwaysInnovating have just released an upgrade, which uses
the 1ghz Cortex A8 DM37xx Series CPU.
With the exception of the $25 Arduino, none of these devices have
reached large volume sales, and none of them have reached mass-volume sales
figures.
Amongst hundreds of thousands of computing devices designed and built
across the world, that's a pathetic track record. We can now move on
to describe the reality - the rest of the world's involvement in Free
Software.
The Rest of the World: the Reality of Commercial use of Free Software
Here is what the rest of the world actually does with Free Software:
A Commercial Company takes or creates a Project which is touted
as Free Software, but in reality it is developed as a Proprietary
Software Product, with the Free Software-Licensed releases being delayed,
and absolutely no interaction encouraged with Free Software Developers,
except by forcing such Developers to become Employees of the Commercial
Company. Examples of such practices include Google's Android,
Sun/Oracle's MySQL, Cygwin's gcc contributions (a decade ago),
and Nokia-TrollTech's QT4.
A Hardware Peripheral Chip Manufacturer creates a device, forces
NDAs to be signed for access to documentation or Device Drivers, or worse
releases binary-only pre-compiled Device Drivers which are solely and
exclusively compatible with an ultra-limited range of Linux Kernels on
very specific hardware platforms, only. This practice is extremely
common, especially in China or Taiwan.
A Fabless Semiconductor Design House creates a Hard Macro for a
peripheral that can be licensed to go into a CPU. NDAs are forced to
be signed in order to access the limited documentation and Device Drivers.
Any licensees are forced to write Device Drivers under NDAs which are
then incompatible with the GPL Linux Source Code License, or worse, to
release binary-only Kernel Modules (again, for specific versions of the
Linux Kernel).
A Fabless Semiconductor Design House creates a Hard Macro for
a complex peripheral (such as a 3D GPU). NDAs are forced to be signed;
compliance with the GPL is honoured to the letter by creating a "shim"
driver that provides a communications channel from userspace to kernelspace,
and then proprietary tools and proprietary software libraries, often
qualifying as "System Libraries" under the GPL's exemption clause, are
provided on a limited and highly restricted and controlled basis.
A CPU SoC Fabless Semiconductor company creates an embedded CPU,
with Hard Macros licensed from Fabless Design Houses.
They port various Operating Systems to it and various Free Software
Kernels (such as Linux), with no cooperation whatsover and with zero
consultation with the Free Software Community.
The various Hard Macros require Device Drivers which
are duplicated across multiple other CPU SoC vendors, often being identical
except for memory and IRQ addresses.
They port Software such as ffmpeg or other GPL and LGPL applications
to match on-board proprietary Hard Macros for Video and Audio Acceleration.
They create a mish-mash Board Support Package - usually Android -
comprising GPL Software, Apache2 Software, in-house developed proprietary
Software Libraries and 3rd Party externally-licensed Software which
came from the 3rd Party Hard Macro Design Houses (3D, Video, SATA etc.)
Licensees (ODMs, OEMs) of the CPU are forced to sign an NDA before
being allowed access to a complete abortional mix of GPL, Apache-2 and
proprietary software, some of which is only available in binary
object form, some of which comes from the 3rd party Design Houses.
The NDA is often in direct violation of the GPL, thus irrevocably
losing both the SoC manufacturer and the Licensee (ODM/OEM)
from being able to use the GPL Linux and other GPL Source Code!
At some point, a BSP which includes GPL Linux Source Code "leaks"
onto the Internet, in direct violation of the Fabless SoC Manufacturer's
NDA, but in honouring the GPL, the ODM is tracked down and punished by
the SoC Manufacturer and is never allowed access to their CPUs again.
A Hardware Company Designer (ODM) creates an example Product, signs all
the NDAs which are incompatible with the GPL, takes the BSP
(Board Support Package), and makes the absolute minimum modifications it can
to the dog's dinner mixture of Software it can get away with in order to
create a useful saleable Product. This is done in complete secrecy, usually
by word-of-mouth, and usually in China, Taiwan or Korea so there is no way
that the Copyright Holders of the GPL Software can even know of its existence.
A Factory pays an ODM for the completed product, and is not informed
of the existence of the dog's dinner Software behind it.
Sometimes the factory decides to get creative, and modifies the
Hardware (adding a webcam or a microphone) but sells it with the exact
same Software, which was derived from the SoC Vendor's
"Engineering Board", so the Web Product has a
Web Mic and a Web Cam which don't ******* work!
When faced with requests for the GPL Software, the factory
has absolutely no idea what is going on, stalls for time and asks
the ODM for an answer. The usual answer is, "It's Our Secret, It's
Our Competitive Advantage - sorry we cannot give". The next answer
which is attempted is "If you place an order for 20,000, we will
supply the Software".
Negotiations often deteriorate from that point
onwards, but if they continue, they work backwards through a complete
minefield that makes Hell look like a walk through sweet-smelling
summer meadows in comparison.
Sometimes, a very large OEM, especially one with world-wide
presence where Copyright Law is enforceable, will actually honour
the GPL Software Licenses as best they can, but usually it is done
after the fact, having had to utilise a 3rd Party Software House who
have to walk the exact same dog's dinner minefield as outlined above.
In these cases, they usually have to violate the NDAs that were signed
with 3rd party suppliers (including the CPU Manufacturer) in order to
comply. Across International Boundaries, and especially if there are
large enough volumes, this strategy actually works (nobody cares to
sue a large customer).
Sometimes, a very large OEM, especially one with world-wide
presence where Copyright Law is enforceable, will comply very very
reluctantly and only with the repeated threat of a Cease and Desist Order
across repeated Product Releases, again and again releasing GPL Software
several months or years after a product is actually shipped, after it
has become defunct or obsolete.
Sometimes, a company will spend an absolute fortune on taking the
highest Legal Advice it can pay for, to get information on how to best
circumvent the GPL. The advice they receive includes:
Tivoisation. The practice of "locking" the software so that
only DRM-approved software can be loaded onto the device. That
Software includes GPLv2 Software, which, despite being available
in full conformance with the GPL, is still utterly useless because
the means to create Firmware (with Key Signing) is not available.
The GPLv3 was written to help terminate this practice, but unfortunately
the core critical piece of Software, the Linux Kernel, is GPLv2
only and requires permission from thousands of contributors
to be upgraded to GPLv3.
Total Control. These companies consider it to be a complete
total failure of their Strategy if you even notice that their
appliance has GPL Software in it. These companies ask you for your
advice on what product you want: they design it, they write the software
and they ship it to you or your customers. Such products are usually
TVs, PVRs or Satellite Receivers (including Sky Boxes and LG Televisions).
There is no OEM in the chain; there is no ODM in the chain; there is no
supply or licensing of the product to multiple Factories. Some companies
that fall into this category also even design and manufacture their own
CPUs.
Amongst these real-world cases, there is some cross-over between "ideal" and
"reality". For example, some Hardware OEMs use CPUs from SoC vendors where
the BSP is publicly available, because starting from a published CAD/CAM
design is easier. This is, however, the less common case: it is, sadly,
much easier for a Factory to just sign the NDA, hand over some money and
receive a ready-to-go design and just begin shipping it out the door.
What is particularly bad is that in both the ideal and the real-world cases,
the SoC Fabless Companies are all under pressure to deliver as fast as
possible. When their BSP eventually becomes public, it shows evidence
of rushed development, complete lack of cooperation and collaboration,
duplicated drivers, shortcuts and shortcomings that are making life absolute
hell for Free Software Developers (especially Linux Kernel Developers).
An analysis of this particular situation is given
here, along with a
proposed approach to fix the problem, the core of which is to broadly
reward collaboration and cooperation (regardless of how that collaboration and
cooperation actually occurs).
So it is perhaps not surprising that Free Software Developers are forced
to carry out extensive reverse-engineering of hardware, right across the
board. and the effect of patents haven't even been mentioned, at all!
Android - The World's Darling
One special case in particular stands out: the Operating System known as
"Android". There is a great deal of ignorance surrounding Android.
Google develops Android in secret as a proprietary product, yet releases it as
a one-way "push" (delayed), as an "Open" Software Project. However, the
software itself is absolutely useless for installation on an arbitrary device,
not least because that software - released by Google - is only a
"Reference Platform". This situation is further confused by Google's
direct or indirect involvement or sponsorship of actual hardware devices
which it refuses to confirm or deny, which get dubbed "Google Phones".
To make matters worse, it has to be appreciated that, contrary to popular
belief, Android is not Apache-2 Licensed: only the userspace portion
of the Android OS is Apache-2 Licensed. The kernel portion is Linux,
and includes some heavy modifications - by Google - to add for example the
Android Security Model. Yet, again, any releases made by Google
of even the Android-enhanced Linux Kernel, are reference platform
releases not suitable for arbitrary use on arbitrary devices.
In order to get hold of a particular device's Android-aware Linux Kernel,
in compliance with the GPL, it is necessary, usually, to go through the
exact hell-on-earth chain as described above, because both the Android-aware
GPL'd Linux Kernel as well as the often heavily-modified Apache2-Licensed
userspace suite of applications which make up the Android OS,
go through several hands before actually hitting the streets.
The worst part of this is that whilst the Android-aware GPL'd Linux Kernel
has to be released in compliance with the GPL, the userspace Android OS
applications do not.
There is, however, hope, in the form of two startling announcements by two
of the major players in the Android Market. The first is the announcement
that HTC has employed the developer behind "CyanogenMod". The second is
that Google have purchased Motorola (in order to gain Motorola's strategic
patents).
But this is just two players, and it does not take into account the fact
that once Google does release the software, many many 3rd party
manufacturers pick it up, create BSPs, create hardware platforms around it,
helping to create the dog's dinner situation that's outlined above.
Conclusion
The Advocates of Free Software could never have predicted the success that
has occurred, and so there are severe shortcomings in the Software Licenses
that are in use. Commercial reality and shortcuts are overwhelming the
Linux Kernel Developers. NDAs, Secrecy and sheer ignorance are taking
precedence over Copyright Law, and Copyright Holders of Free Software,
even if they could find out where their Copyright is being violated,
often do not enforce their rights and are thus at risk of losing them
even if they tried, through an "Estoppel" Legal Defense.
With very few exceptions, the Free Software Community, on whom hardware
manufacturers are now critically relying in order to make money, has stayed
clear of taking the initiative on hardware, and is rewarded by fait-accomplit
devices which further insult us by forcing us to reverse-engineer them,
jailbreak them or initiate a lawsuit for Copyright violation!
So the reality of the situation is therefore far from ideal, and there is
a great deal of work still to be done. There is however the occasional
sign of hope and the very occasional real-world Product that miraculously
springs into existence.
So the question has to be asked: wouldn't it be even better if a real-world
Product was initiated that could actually tick all the boxes, honouring the
GPL and other Free Software Licenses, involved the Free Software Community
right from the start, and was sold in mass-volume? Surely such a Product
(or product range)
could be a model example to all Hardware Manufacturers and SoC Vendors
right across the world (instead of being considered implicitly to be
non-commercial
by virtue of being even remotely associated with Free Software Developers).
Crucially, therefore, what, fundamentally, would make such a Product (or
Product family) successful? What Products would actually sell? And why
and how would Free Software Developers be attracted to be involved in
such Products?
Comments welcome to: lkcl@lkcl.net