Root/docs/adding.tex

1Linux is now one of the most widespread operating system for embedded devices due
2to its openess as well as the wide variety of platforms it can run on. Many
3manufacturer actually use it in firmware you can find on many devices: DVB-T
4decoders, routers, print servers, DVD players ... Most of the time the stock
5firmware is not really open to the consumer, even if it uses open source software.
6
7You might be interested in running a Linux based firmware for your router for
8various reasons: extending the use of a network protocol (such as IPv6), having
9new features, new piece of software inside, or for security reasons. A fully
10open-source firmware is de-facto needed for such applications, since you want to
11be free to use this or that version of a particular reason, be able to correct a
12particular bug. Few manufacturers do ship their routers with a Sample Development Kit,
13that would allow you to create your own and custom firmware and most of the time,
14when they do, you will most likely not be able to complete the firmware creation process.
15
16This is one of the reasons why OpenWrt and other firmware exists: providing a
17version independent, and tools independent firmware, that can be run on various
18platforms, known to be running Linux originally.
19
20\subsection{Which Operating System does this device run?}
21
22There is a lot of methods to ensure your device is running Linux. Some of them do
23need your router to be unscrewed and open, some can be done by probing the device
24using its external network interfaces.
25
26\subsubsection{Operating System fingerprinting and port scanning}
27
28A large bunch of tools over the Internet exists in order to let you do OS
29fingerprinting, we will show here an example using \textbf{nmap}:
30
31\begin{Verbatim}
32nmap -P0 -O <IP address>
33Starting Nmap 4.20 ( http://insecure.org ) at 2007-01-08 11:05 CET
34Interesting ports on 192.168.2.1:
35Not shown: 1693 closed ports
36PORT STATE SERVICE
3722/tcp open ssh
3823/tcp open telnet
3953/tcp open domain
4080/tcp open http
41MAC Address: 00:13:xx:xx:xx:xx (Cisco-Linksys)
42Device type: broadband router
43Running: Linksys embedded
44OS details: Linksys WRT54GS v4 running OpenWrt w/Linux kernel 2.4.30
45Network Distance: 1 hop
46\end{Verbatim}
47
48nmap is able to report whether your device uses a Linux TCP/IP stack, and if so,
49will show you which Linux kernel version is probably runs. This report is quite
50reliable and it can make the distinction between BSD and Linux TCP/IP stacks and others.
51
52Using the same tool, you can also do port scanning and service version discovery.
53For instance, the following command will report which IP-based services are running
54on the device, and which version of the service is being used:
55
56\begin{verbatim}
57nmap -P0 -sV <IP address>
58Starting Nmap 4.20 ( http://insecure.org ) at 2007-01-08 11:06 CET
59Interesting ports on 192.168.2.1:
60Not shown: 1693 closed ports
61PORT STATE SERVICE VERSION
6222/tcp open ssh Dropbear sshd 0.48 (protocol 2.0)
6323/tcp open telnet Busybox telnetd
6453/tcp open domain ISC Bind dnsmasq-2.35
6580/tcp open http OpenWrt BusyBox httpd
66MAC Address: 00:13:xx:xx:xx:xx (Cisco-Linksys)
67Service Info: Device: WAP
68\end{verbatim}
69
70The web server version, if identified, can be determining in knowing the Operating
71System. For instance, the \textbf{BOA} web server is typical from devices running
72an open-source Unix or Unix-like.
73
74\subsubsection{Wireless Communications Fingerprinting}
75
76Although this method is not really known and widespread, using a wireless scanner
77to discover which OS your router or Access Point run can be used. We do not have
78a clear example of how this could be achieved, but you will have to monitor raw
79802.11 frames and compare them to a very similar device running a Linux based firmware.
80
81\subsubsection{Web server security exploits}
82
83The Linksys WRT54G was originally hacked by using a "ping bug" discovered in the
84web interface. This tip has not been fixed for months by Linksys, allowing people
85to enable the "boot\_wait" helper process via the web interface. Many web servers
86used in firmwares are open source web server, thus allowing the code to be audited
87to find an exploit. Once you know the web server version that runs on your device,
88by using \textbf{nmap -sV} or so, you might be interested in using exploits to reach
89shell access on your device.
90
91\subsubsection{Native Telnet/SSH access}
92
93Some firmwares might have restricted or unrestricted Telnet/SSH access, if so,
94try to log in with the web interface login/password and see if you can type in
95some commands. This is actually the case for some Broadcom BCM963xx based firmwares
96such as the one in Neuf/Cegetel ISP routers, Club-Internet ISP CI-Box and many
97others. Some commands, like \textbf{cat} might be left here and be used to
98determine the Linux kernel version.
99
100\subsubsection{Analysing a binary firmware image}
101
102You are very likely to find a firmware binary image on the manufacturer website,
103even if your device runs a proprietary operating system. If so, you can download
104it and use an hexadecimal editor to find printable words such as \textbf{vmlinux},
105\textbf{linux}, \textbf{ramdisk}, \textbf{mtd} and others.
106
107Some Unix tools like \textbf{hexdump} or \textbf{strings} can be used to analyse
108the firmware. Below there is an example with a binary firmware found other the Internet:
109
110\begin{verbatim}
111hexdump -C <binary image.extension> | less (more)
11200000000 46 49 52 45 32 2e 35 2e 30 00 00 00 00 00 00 00 |FIRE2.5.0.......|
11300000010 00 00 00 00 31 2e 30 2e 30 00 00 00 00 00 00 00 |....1.0.0.......|
11400000020 00 00 00 00 00 00 00 38 00 43 36 29 00 0a e6 dc |.......8.C6)..??|
11500000030 54 49 44 45 92 89 54 66 1f 8b 08 08 f8 10 68 42 |TIDE..Tf....?.hB|
11600000040 02 03 72 61 6d 64 69 73 6b 00 ec 7d 09 bc d5 d3 |..ramdisk.?}.???|
11700000050 da ff f3 9b f7 39 7b ef 73 f6 19 3b 53 67 ea 44 |???.?9{?s?.;Sg?D|
118\end{verbatim}
119
120Scroll over the firmware to find printable words that can be significant.
121
122\subsubsection{Amount of flash memory}
123
124Linux can hardly fit in a 2MB flash device, once you have opened the device and
125located the flash chip, try to find its characteristics on the Internet. If
126your flash chip is a 2MB or less device, your device is most likely to run a
127proprietary OS such as WindRiver VxWorks, or a custom manufacturer OS like Zyxel ZynOS.
128
129OpenWrt does not currently run on devices which have 2MB or less of flash memory.
130This limitation will probably not be worked around since those devices are most
131of the time micro-routers, or Wireless Access Points, which are not the main
132OpenWrt target.
133
134\subsubsection{Pluging a serial port}
135
136By using a serial port and a level shifter, you may reach the console that is being shown by the device
137for debugging or flashing purposes. By analysing the output of this device, you can
138easily notice if the device uses a Linux kernel or something different.
139
140\subsection{Finding and using the manufacturer SDK}
141
142Once you are sure your device run a Linux based firmware, you will be able to start
143hacking on it. If the manufacturer respected the GPL, it will have released a Sample
144Development Kit with the device.
145
146\subsubsection{GPL violations}
147
148Some manufacturers do release a Linux based binary firmware, with no sources at all.
149The first step before doing anything is to read the license coming with your device,
150then write them about this lack of Open Source code. If the manufacturer answers
151you they do not have to release a SDK containing Open Source software, then we
152recommend you get in touch with the gpl-violations.org community.
153
154You will find below a sample letter that can be sent to the manufacturer:
155
156\begin{verse}
157Miss, Mister,
158
159I am using a <device name>, and I cannot find neither on your website nor on the
160CD-ROM the open source software used to build or modify the firmware.
161
162In conformance to the GPL license, you have to release the following sources:
163
164\begin{itemize}
165\item complete toolchain that made the kernel and applications be compiled (gcc, binutils, libc)
166\item tools to build a custom firmware (mksquashfs, mkcramfs ...)
167\item kernel sources with patches to make it run on this specific hardware, this does not include binary drivers
168\end{itemize}
169
170Thank you very much in advance for your answer.
171
172Best regards, <your name>
173\end{verse}
174
175\subsubsection{Using the SDK}
176
177Once the SDK is available, you are most likely not to be able to build a complete
178or functional firmware using it, but parts of it, like only the kernel, or only
179the root filesystem. Most manufacturers do not really care releasing a tool that
180do work every time you uncompress and use it.
181
182You should anyway be able to use the following components:
183
184\begin{itemize}
185\item kernel sources with more or less functional patches for your hardware
186\item binary drivers linked or to be linked with the shipped kernel version
187\item packages of the toolchain used to compile the whole firmware: gcc, binutils, libc or uClibc
188\item binary tools to create a valid firmware image
189\end{itemize}
190
191Your work can be divided into the following tasks:
192
193\begin{itemize}
194\item create a clean patch of the hardware specific part of the linux kernel
195\item spot potential kernel GPL violations especially on netfilter and USB stack stuff
196\item make the binary drivers work, until there are open source drivers
197\item use standard a GNU toolchain to make working executables
198\item understand and write open source tools to generate a valid firmware image
199\end{itemize}
200
201\subsubsection{Creating a hardware specific kernel patch}
202
203Most of the time, the kernel source that comes along with the SDK is not really
204clean, and is not a standard Linux version, it also has architecture specific
205fixes backported from the \textbf{CVS} or the \textbf{git} repository of the
206kernel development trees. Anyway, some parts can be easily isolated and used as
207a good start to make a vanilla kernel work your hardware.
208
209Some directories are very likely to have local modifications needed to make your
210hardware be recognized and used under Linux. First of all, you need to find out
211the linux kernel version that is used by your hardware, this can be found by
212editing the \textbf{linux/Makefile} file.
213
214\begin{verbatim}
215head -5 linux-2.x.x/Makefile
216VERSION = 2
217PATCHLEVEL = x
218SUBLEVEL = y
219EXTRAVERSION = z
220NAME=A fancy name
221\end{verbatim}
222
223So now, you know that you have to download a standard kernel tarball at
224\textbf{kernel.org} that matches the version being used by your hardware.
225
226Then you can create a \textbf{diff} file between the two trees, especially for the
227following directories:
228
229\begin{verbatim}
230diff -urN linux-2.x.x/arch/<sub architecture> linux-2.x.x-modified/arch/<sub architecture> > 01-architecture.patch
231diff -urN linux-2.x.x/include/ linux-2.x.x-modified/include > 02-includes.patch
232diff -urN linux-2.x.x/drivers/ linux-2.x.x-modified/drivers > 03-drivers.patch
233\end{verbatim}
234
235This will constitute a basic set of three patches that are very likely to contain
236any needed modifications that has been made to the stock Linux kernel to run on
237your specific device. Of course, the content produced by the \textbf{diff -urN}
238may not always be relevant, so that you have to clean up those patches to only
239let the "must have" code into them.
240
241The first patch will contain all the code that is needed by the board to be
242initialized at startup, as well as processor detection and other boot time
243specific fixes.
244
245The second patch will contain all useful definitions for that board: addresses,
246kernel granularity, redefinitions, processor family and features ...
247
248The third patch may contain drivers for: serial console, ethernet NIC, wireless
249NIC, USB NIC ... Most of the time this patch contains nothing else than "glue"
250code that has been added to make the binary driver work with the Linux kernel.
251This code might not be useful if you plan on writing drivers from scratch for
252this hardware.
253
254\subsubsection{Using the device bootloader}
255
256The bootloader is the first program that is started right after your device has
257been powered on. This program, can be more or less sophisticated, some do let you
258do network booting, USB mass storage booting ... The bootloader is device and
259architecture specific, some bootloaders were designed to be universal such as
260RedBoot or U-Boot so that you can meet those loaders on totally different
261platforms and expect them to behave the same way.
262
263If your device runs a proprietary operating system, you are very likely to deal
264with a proprietary boot loader as well. This may not always be a limitation,
265some proprietary bootloaders can even have source code available (i.e : Broadcom CFE).
266
267According to the bootloader features, hacking on the device will be more or less
268easier. It is very probable that the bootloader, even exotic and rare, has a
269documentation somewhere over the Internet. In order to know what will be possible
270with your bootloader and the way you are going to hack the device, look over the
271following features :
272
273\begin{itemize}
274\item does the bootloader allow net booting via bootp/DHCP/NFS or tftp
275\item does the bootloader accept loading ELF binaries ?
276\item does the bootloader have a kernel/firmware size limitation ?
277\item does the bootloader expect a firmware format to be loaded with ?
278\item are the loaded files executed from RAM or flash ?
279\end{itemize}
280
281Net booting is something very convenient, because you will only have to set up network
282booting servers on your development station, and keep the original firmware on the device
283till you are sure you can replace it. This also prevents your device from being flashed,
284and potentially bricked every time you want to test a modification on the kernel/filesystem.
285
286If your device needs to be flashed every time you load a firmware, the bootlader might
287only accept a specific firmware format to be loaded, so that you will have to
288understand the firmware format as well.
289
290\subsubsection{Making binary drivers work}
291
292As we have explained before, manufacturers do release binary drivers in their GPL
293tarball. When those drivers are statically linked into the kernel, they become GPL
294as well, fortunately or unfortunately, most of the drivers are not statically linked.
295This anyway lets you a chance to dynamically link the driver with the current kernel
296version, and try to make them work together.
297
298This is one of the most tricky and grey part of the fully open source projects.
299Some drivers require few modifications to be working with your custom kernel,
300because they worked with an earlier kernel, and few modifications have been made
301to the kernel in-between those versions. This is for instance the case with the
302binary driver of the Broadcom BCM43xx Wireless Chipsets, where only few differences
303were made to the network interface structures.
304
305Some general principles can be applied no matter which kernel version is used in
306order to make binary drivers work with your custom kernel:
307
308\begin{itemize}
309\item turn on kernel debugging features such as:
310\begin{itemize}
311\item CONFIG\_DEBUG\_KERNEL
312\item CONFIG\_DETECT\_SOFTLOCKUP
313\item CONFIG\_DEBUG\_KOBJECT
314\item CONFIG\_KALLSYMS
315\item CONFIG\_KALLSYMS\_ALL
316\end{itemize}
317\item link binary drivers when possible to the current kernel version
318\item try to load those binary drivers
319\item catch the lockups and understand them
320\end{itemize}
321
322Most of the time, loading binary drivers will fail, and generate a kernel oops.
323You can know the last symbol the binary drivers attempted to use, and see in the
324kernel headers file, if you do not have to move some structures field before or
325after that symbol in order to keep compatibily with both the binary driver and
326the stock kernel drivers.
327
328\subsubsection{Understanding the firmware format}
329
330You might want to understand the firmware format, even if you are not yet capable
331of running a custom firmware on your device, because this is sometimes a blocking
332part of the flashing process.
333
334A firmware format is most of the time composed of the following fields:
335
336\begin{itemize}
337\item header, containing a firmware version and additional fields: Vendor, Hardware version ...
338\item CRC32 checksum on either the whole file or just part of it
339\item Binary and/or compressed kernel image
340\item Binary and/or compressed root filesystem image
341\item potential garbage
342\end{itemize}
343
344Once you have figured out how the firmware format is partitioned, you will have
345to write your own tool that produces valid firmware binaries. One thing to be very
346careful here is the endianness of either the machine that produces the binary
347firmware and the device that will be flashed using this binary firmware.
348
349\subsubsection{Writing a flash map driver}
350
351The flash map driver has an important role in making your custom firmware work
352because it is responsible of mapping the correct flash regions and associated
353rights to specific parts of the system such as: bootloader, kernel, user filesystem.
354
355Writing your own flash map driver is not really a hard task once you know how your
356firmware image and flash is structured. You will find below a commented example
357that covers the case of the device where the bootloader can pass to the kernel its partition plan.
358
359First of all, you need to make your flash map driver be visible in the kernel
360configuration options, this can be done by editing the file \
361\textbf{linux/drivers/mtd/maps/Kconfig}:
362
363\begin{verbatim}
364config MTD_DEVICE_FLASH
365        tristate "Device Flash device"
366        depends on ARCHITECTURE && DEVICE
367        help
368         Flash memory access on DEVICE boards. Currently only works with
369         Bootloader Foo and Bootloader Bar.
370\end{verbatim}
371
372Then add your source file to the \textbf{linux/drivers/mtd/maps/Makefile}, so
373that it will be compiled along with the kernel.
374
375\begin{verbatim}
376obj-\$(CONFIG_MTD_DEVICE_FLASH) += device-flash.o
377\end{verbatim}
378
379You can then write the kernel driver itself, by creating a
380\textbf{linux/drivers/mtd/maps/device-flash.c} C source file.
381
382\begin{verbatim}
383// Includes that are required for the flash map driver to know of the prototypes:
384#include <asm/io.h>
385#include <linux/init.h>
386#include <linux/kernel.h>
387#include <linux/mtd/map.h>
388#include <linux/mtd/mtd.h>
389#include <linux/mtd/partitions.h>
390#include <linux/vmalloc.h>
391
392// Put some flash map definitions here:
393#define WINDOW_ADDR 0x1FC00000 /* Real address of the flash */
394#define WINDOW_SIZE 0x400000 /* Size of flash */
395#define BUSWIDTH 2 /* Buswidth */
396
397static void __exit device_mtd_cleanup(void);
398
399static struct mtd_info *device_mtd_info;
400
401static struct map_info devicd_map = {
402       .name = "device",
403       .size = WINDOW_SIZE,
404       .bankwidth = BUSWIDTH,
405       .phys = WINDOW_ADDR,
406};
407
408static int __init device_mtd_init(void)
409{
410      // Display that we found a flash map device
411       printk("device: 0x\%08x at 0x\%08x\n", WINDOW_SIZE, WINDOW_ADDR);
412       // Remap the device address to a kernel address
413       device_map.virt = ioremap(WINDOW_ADDR, WINDOW_SIZE);
414
415       // If impossible to remap, exit with the EIO error
416       if (!device_map.virt) {
417               printk("device: Failed to ioremap\n");
418               return -EIO;
419       }
420
421       // Initialize the device map
422       simple_map_init(&device_map);
423
424       /* MTD informations are closely linked to the flash map device
425           you might also use "jedec_probe" "amd_probe" or "intel_probe" */
426       device_mtd_info = do_map_probe("cfi_probe", &device_map);
427
428         if (device_mtd_info) {
429               device_mtd_info->owner = THIS_MODULE;
430
431                int parsed_nr_parts = 0;
432
433                        // We try here to use the partition schema provided by the bootloader specific code
434                       if (parsed_nr_parts == 0) {
435                               int ret = parse_bootloader_partitions(device_mtd_info, &parsed_parts, 0);
436                               if (ret > 0) {
437                                       part_type = "BootLoader";
438                                       parsed_nr_parts = ret;
439                               }
440                       }
441
442                       add_mtd_partitions(devicd_mtd_info, parsed_parts, parsed_nr_parts);
443
444                       return 0;
445               }
446       iounmap(device_map.virt);
447
448       return -ENXIO;
449}
450
451// This function will make the driver clean up the MTD device mapping
452static void __exit device_mtd_cleanup(void)
453{
454      // If we found a MTD device before
455       if (device_mtd_info) {
456               // Delete every partitions
457               del_mtd_partitions(device_mtd_info);
458               // Delete the associated map
459               map_destroy(device_mtd_info);
460       }
461    
462        // If the virtual address is already in use
463       if (device_map.virt) {
464                    // Unmap the physical address to a kernel space address
465               iounmap(device_map.virt);
466                // Reset the structure field
467              device_map.virt = 0;
468       }
469}
470
471
472// Macros that indicate which function is called on loading/unloading the module
473module_init(device_mtd_init);
474module_exit(device_mtd_cleanup);
475
476
477// Macros defining license and author, parameters can be defined here too.
478MODULE_LICENSE("GPL");
479MODULE_AUTHOR("Me, myself and I <memyselfandi@domain.tld");
480\end{verbatim}
481
482\subsection{Adding your target in OpenWrt}
483
484Once you spotted the key changes that were made to the Linux kernel
485to support your target, you will want to create a target in OpenWrt
486for your hardware. This can be useful to benefit from the toolchain
487that OpenWrt builds as well as the resulting user-space and kernel
488configuration options.
489
490Provided that your target is already known to OpenWrt, it will be
491as simple as creating a \texttt{target/linux/board} directory
492where you will be creating the following directories and files.
493
494Here for example, is a \texttt{target/linux/board/Makefile}:
495
496\begin{Verbatim}[frame=single,numbers=left]
497#
498# Copyright (C) 2009 OpenWrt.org
499#
500# This is free software, licensed under the GNU General Public License v2.
501# See /LICENSE for more information.
502#
503include $(TOPDIR)/rules.mk
504
505ARCH:=mips
506BOARD:=board
507BOARDNAME:=Eval board
508FEATURES:=squashfs jffs2 pci usb
509
510LINUX_VERSION:=2.6.27.10
511
512include $(INCLUDE_DIR)/target.mk
513
514DEFAULT_PACKAGES += hostapd-mini
515
516define Target/Description
517        Build firmware images for Evaluation board
518endef
519
520$(eval $(call BuildTarget))
521\end{Verbatim}
522
523\begin{itemize}
524    \item \texttt{ARCH} \\
525        The name of the architecture known by Linux and uClibc
526    \item \texttt{BOARD} \\
527        The name of your board that will be used as a package and build directory identifier
528    \item \texttt{BOARDNAME} \\
529        Expanded name that will appear in menuconfig
530    \item \texttt{FEATURES} \\
531        Set of features to build filesystem images, USB, PCI, VIDEO kernel support
532    \item \texttt{LINUX\_VERSION} \\
533        Linux kernel version to use for this target
534    \item \texttt{DEFAULT\_PACKAGES} \\
535        Set of packages to be built by default
536\end{itemize}
537
538A partial kernel configuration which is either named \texttt{config-default} or which matches the kernel version \texttt{config-2.6.x} should be present in \texttt{target/linux/board/}.
539This kernel configuration will only contain the relevant symbols to support your target and can be changed using \texttt{make kernel\_menuconfig}.
540
541To patch the kernel sources with the patches required to support your hardware, you will have to drop them in \texttt{patches} or in \texttt{patches-2.6.x} if there are specific
542changes between kernel versions. Additionnaly, if you want to avoid creating a patch that will create files, you can put those files into \texttt{files} or \texttt{files-2.6.x}
543with the same directory structure that the kernel uses (e.g: drivers/mtd/maps, arch/mips ..).
544
545The build system will require you to create a \texttt{target/linux/board/image/Makefile}:
546
547\begin{Verbatim}[frame=single,numbers=left]
548#
549# Copyright (C) 2009 OpenWrt.org
550#
551# This is free software, licensed under the GNU General Public License v2.
552# See /LICENSE for more information.
553#
554include $(TOPDIR)/rules.mk
555include $(INCLUDE_DIR)/image.mk
556
557define Image/BuildKernel
558        cp $(KDIR)/vmlinux.elf $(BIN_DIR)/openwrt-$(BOARD)-vmlinux.elf
559        gzip -9 -c $(KDIR)/vmlinux > $(KDIR)/vmlinux.bin.gz
560        $(STAGING_DIR_HOST)/bin/lzma e $(KDIR)/vmlinux $(KDIR)/vmlinux.bin.l7
561        dd if=$(KDIR)/vmlinux.bin.l7 of=$(BIN_DIR)/openwrt-$(BOARD)-vmlinux.lzma bs=65536 conv=sync
562        dd if=$(KDIR)/vmlinux.bin.gz of=$(BIN_DIR)/openwrt-$(BOARD)-vmlinux.gz bs=65536 conv=sync
563endef
564
565define Image/Build/squashfs
566    $(call prepare_generic_squashfs,$(KDIR)/root.squashfs)
567endef
568
569define Image/Build
570        $(call Image/Build/$(1))
571        dd if=$(KDIR)/root.$(1) of=$(BIN_DIR)/openwrt-$(BOARD)-root.$(1) bs=128k conv=sync
572
573        -$(STAGING_DIR_HOST)/bin/mkfwimage \
574                -B XS2 -v XS2.ar2316.OpenWrt \
575                -k $(BIN_DIR)/openwrt-$(BOARD)-vmlinux.lzma \
576                -r $(BIN_DIR)/openwrt-$(BOARD)-root.$(1) \
577                -o $(BIN_DIR)/openwrt-$(BOARD)-ubnt2-$(1).bin
578endef
579
580$(eval $(call BuildImage))
581
582\end{Verbatim}
583
584\begin{itemize}
585    \item \texttt{Image/BuildKernel} \\
586        This template defines changes to be made to the ELF kernel file
587    \item \texttt{Image/Build} \\
588    This template defines the final changes to apply to the rootfs and kernel, either combined or separated
589    firmware creation tools can be called here as well.
590\end{itemize}
591

Archive Download this file



interactive