Root/Documentation/networking/eql.txt

1  EQL Driver: Serial IP Load Balancing HOWTO
2  Simon "Guru Aleph-Null" Janes, simon@ncm.com
3  v1.1, February 27, 1995
4
5  This is the manual for the EQL device driver. EQL is a software device
6  that lets you load-balance IP serial links (SLIP or uncompressed PPP)
7  to increase your bandwidth. It will not reduce your latency (i.e. ping
8  times) except in the case where you already have lots of traffic on
9  your link, in which it will help them out. This driver has been tested
10  with the 1.1.75 kernel, and is known to have patched cleanly with
11  1.1.86. Some testing with 1.1.92 has been done with the v1.1 patch
12  which was only created to patch cleanly in the very latest kernel
13  source trees. (Yes, it worked fine.)
14
15  1. Introduction
16
17  Which is worse? A huge fee for a 56K leased line or two phone lines?
18  It's probably the former. If you find yourself craving more bandwidth,
19  and have a ISP that is flexible, it is now possible to bind modems
20  together to work as one point-to-point link to increase your
21  bandwidth. All without having to have a special black box on either
22  side.
23
24
25  The eql driver has only been tested with the Livingston PortMaster-2e
26  terminal server. I do not know if other terminal servers support load-
27  balancing, but I do know that the PortMaster does it, and does it
28  almost as well as the eql driver seems to do it (-- Unfortunately, in
29  my testing so far, the Livingston PortMaster 2e's load-balancing is a
30  good 1 to 2 KB/s slower than the test machine working with a 28.8 Kbps
31  and 14.4 Kbps connection. However, I am not sure that it really is
32  the PortMaster, or if it's Linux's TCP drivers. I'm told that Linux's
33  TCP implementation is pretty fast though.--)
34
35
36  I suggest to ISPs out there that it would probably be fair to charge
37  a load-balancing client 75% of the cost of the second line and 50% of
38  the cost of the third line etc...
39
40
41  Hey, we can all dream you know...
42
43
44  2. Kernel Configuration
45
46  Here I describe the general steps of getting a kernel up and working
47  with the eql driver. From patching, building, to installing.
48
49
50  2.1. Patching The Kernel
51
52  If you do not have or cannot get a copy of the kernel with the eql
53  driver folded into it, get your copy of the driver from
54  ftp://slaughter.ncm.com/pub/Linux/LOAD_BALANCING/eql-1.1.tar.gz.
55  Unpack this archive someplace obvious like /usr/local/src/. It will
56  create the following files:
57
58
59
60       ______________________________________________________________________
61       -rw-r--r-- guru/ncm 198 Jan 19 18:53 1995 eql-1.1/NO-WARRANTY
62       -rw-r--r-- guru/ncm 30620 Feb 27 21:40 1995 eql-1.1/eql-1.1.patch
63       -rwxr-xr-x guru/ncm 16111 Jan 12 22:29 1995 eql-1.1/eql_enslave
64       -rw-r--r-- guru/ncm 2195 Jan 10 21:48 1995 eql-1.1/eql_enslave.c
65       ______________________________________________________________________
66
67  Unpack a recent kernel (something after 1.1.92) someplace convenient
68  like say /usr/src/linux-1.1.92.eql. Use symbolic links to point
69  /usr/src/linux to this development directory.
70
71
72  Apply the patch by running the commands:
73
74
75       ______________________________________________________________________
76       cd /usr/src
77       patch </usr/local/src/eql-1.1/eql-1.1.patch
78       ______________________________________________________________________
79
80
81
82
83
84  2.2. Building The Kernel
85
86  After patching the kernel, run make config and configure the kernel
87  for your hardware.
88
89
90  After configuration, make and install according to your habit.
91
92
93  3. Network Configuration
94
95  So far, I have only used the eql device with the DSLIP SLIP connection
96  manager by Matt Dillon (-- "The man who sold his soul to code so much
97  so quickly."--) . How you configure it for other "connection"
98  managers is up to you. Most other connection managers that I've seen
99  don't do a very good job when it comes to handling more than one
100  connection.
101
102
103  3.1. /etc/rc.d/rc.inet1
104
105  In rc.inet1, ifconfig the eql device to the IP address you usually use
106  for your machine, and the MTU you prefer for your SLIP lines. One
107  could argue that MTU should be roughly half the usual size for two
108  modems, one-third for three, one-fourth for four, etc... But going
109  too far below 296 is probably overkill. Here is an example ifconfig
110  command that sets up the eql device:
111
112
113
114       ______________________________________________________________________
115       ifconfig eql 198.67.33.239 mtu 1006
116       ______________________________________________________________________
117
118
119
120
121
122  Once the eql device is up and running, add a static default route to
123  it in the routing table using the cool new route syntax that makes
124  life so much easier:
125
126
127
128       ______________________________________________________________________
129       route add default eql
130       ______________________________________________________________________
131
132
133  3.2. Enslaving Devices By Hand
134
135  Enslaving devices by hand requires two utility programs: eql_enslave
136  and eql_emancipate (-- eql_emancipate hasn't been written because when
137  an enslaved device "dies", it is automatically taken out of the queue.
138  I haven't found a good reason to write it yet... other than for
139  completeness, but that isn't a good motivator is it?--)
140
141
142  The syntax for enslaving a device is "eql_enslave <master-name>
143  <slave-name> <estimated-bps>". Here are some example enslavings:
144
145
146
147       ______________________________________________________________________
148       eql_enslave eql sl0 28800
149       eql_enslave eql ppp0 14400
150       eql_enslave eql sl1 57600
151       ______________________________________________________________________
152
153
154
155
156
157  When you want to free a device from its life of slavery, you can
158  either down the device with ifconfig (eql will automatically bury the
159  dead slave and remove it from its queue) or use eql_emancipate to free
160  it. (-- Or just ifconfig it down, and the eql driver will take it out
161  for you.--)
162
163
164
165       ______________________________________________________________________
166       eql_emancipate eql sl0
167       eql_emancipate eql ppp0
168       eql_emancipate eql sl1
169       ______________________________________________________________________
170
171
172
173
174
175  3.3. DSLIP Configuration for the eql Device
176
177  The general idea is to bring up and keep up as many SLIP connections
178  as you need, automatically.
179
180
181  3.3.1. /etc/slip/runslip.conf
182
183  Here is an example runslip.conf:
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199  ______________________________________________________________________
200  name sl-line-1
201  enabled
202  baud 38400
203  mtu 576
204  ducmd -e /etc/slip/dialout/cua2-288.xp -t 9
205  command eql_enslave eql $interface 28800
206  address 198.67.33.239
207  line /dev/cua2
208
209  name sl-line-2
210  enabled
211  baud 38400
212  mtu 576
213  ducmd -e /etc/slip/dialout/cua3-288.xp -t 9
214  command eql_enslave eql $interface 28800
215  address 198.67.33.239
216  line /dev/cua3
217  ______________________________________________________________________
218
219
220
221
222
223  3.4. Using PPP and the eql Device
224
225  I have not yet done any load-balancing testing for PPP devices, mainly
226  because I don't have a PPP-connection manager like SLIP has with
227  DSLIP. I did find a good tip from LinuxNET:Billy for PPP performance:
228  make sure you have asyncmap set to something so that control
229  characters are not escaped.
230
231
232  I tried to fix up a PPP script/system for redialing lost PPP
233  connections for use with the eql driver the weekend of Feb 25-26 '95
234  (Hereafter known as the 8-hour PPP Hate Festival). Perhaps later this
235  year.
236
237
238  4. About the Slave Scheduler Algorithm
239
240  The slave scheduler probably could be replaced with a dozen other
241  things and push traffic much faster. The formula in the current set
242  up of the driver was tuned to handle slaves with wildly different
243  bits-per-second "priorities".
244
245
246  All testing I have done was with two 28.8 V.FC modems, one connecting
247  at 28800 bps or slower, and the other connecting at 14400 bps all the
248  time.
249
250
251  One version of the scheduler was able to push 5.3 K/s through the
252  28800 and 14400 connections, but when the priorities on the links were
253  very wide apart (57600 vs. 14400) the "faster" modem received all
254  traffic and the "slower" modem starved.
255
256
257  5. Testers' Reports
258
259  Some people have experimented with the eql device with newer
260  kernels (than 1.1.75). I have since updated the driver to patch
261  cleanly in newer kernels because of the removal of the old "slave-
262  balancing" driver config option.
263
264
265  o icee from LinuxNET patched 1.1.86 without any rejects and was able
266     to boot the kernel and enslave a couple of ISDN PPP links.
267
268  5.1. Randolph Bentson's Test Report
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331  From bentson@grieg.seaslug.org Wed Feb 8 19:08:09 1995
332  Date: Tue, 7 Feb 95 22:57 PST
333  From: Randolph Bentson <bentson@grieg.seaslug.org>
334  To: guru@ncm.com
335  Subject: EQL driver tests
336
337
338  I have been checking out your eql driver. (Nice work, that!)
339  Although you may already done this performance testing, here
340  are some data I've discovered.
341
342  Randolph Bentson
343  bentson@grieg.seaslug.org
344
345  ---------------------------------------------------------
346
347
348  A pseudo-device driver, EQL, written by Simon Janes, can be used
349  to bundle multiple SLIP connections into what appears to be a
350  single connection. This allows one to improve dial-up network
351  connectivity gradually, without having to buy expensive DSU/CSU
352  hardware and services.
353
354  I have done some testing of this software, with two goals in
355  mind: first, to ensure it actually works as described and
356  second, as a method of exercising my device driver.
357
358  The following performance measurements were derived from a set
359  of SLIP connections run between two Linux systems (1.1.84) using
360  a 486DX2/66 with a Cyclom-8Ys and a 486SLC/40 with a Cyclom-16Y.
361  (Ports 0,1,2,3 were used. A later configuration will distribute
362  port selection across the different Cirrus chips on the boards.)
363  Once a link was established, I timed a binary ftp transfer of
364  289284 bytes of data. If there were no overhead (packet headers,
365  inter-character and inter-packet delays, etc.) the transfers
366  would take the following times:
367
368      bits/sec seconds
369      345600 8.3
370      234600 12.3
371      172800 16.7
372      153600 18.8
373      76800 37.6
374      57600 50.2
375      38400 75.3
376      28800 100.4
377      19200 150.6
378      9600 301.3
379
380  A single line running at the lower speeds and with large packets
381  comes to within 2% of this. Performance is limited for the higher
382  speeds (as predicted by the Cirrus databook) to an aggregate of
383  about 160 kbits/sec. The next round of testing will distribute
384  the load across two or more Cirrus chips.
385
386  The good news is that one gets nearly the full advantage of the
387  second, third, and fourth line's bandwidth. (The bad news is
388  that the connection establishment seemed fragile for the higher
389  speeds. Once established, the connection seemed robust enough.)
390
391  #lines speed mtu seconds theory actual %of
392     kbit/sec duration speed speed max
393  3 115200 900 _ 345600
394  3 115200 400 18.1 345600 159825 46
395  2 115200 900 _ 230400
396  2 115200 600 18.1 230400 159825 69
397  2 115200 400 19.3 230400 149888 65
398  4 57600 900 _ 234600
399  4 57600 600 _ 234600
400  4 57600 400 _ 234600
401  3 57600 600 20.9 172800 138413 80
402  3 57600 900 21.2 172800 136455 78
403  3 115200 600 21.7 345600 133311 38
404  3 57600 400 22.5 172800 128571 74
405  4 38400 900 25.2 153600 114795 74
406  4 38400 600 26.4 153600 109577 71
407  4 38400 400 27.3 153600 105965 68
408  2 57600 900 29.1 115200 99410.3 86
409  1 115200 900 30.7 115200 94229.3 81
410  2 57600 600 30.2 115200 95789.4 83
411  3 38400 900 30.3 115200 95473.3 82
412  3 38400 600 31.2 115200 92719.2 80
413  1 115200 600 31.3 115200 92423 80
414  2 57600 400 32.3 115200 89561.6 77
415  1 115200 400 32.8 115200 88196.3 76
416  3 38400 400 33.5 115200 86353.4 74
417  2 38400 900 43.7 76800 66197.7 86
418  2 38400 600 44 76800 65746.4 85
419  2 38400 400 47.2 76800 61289 79
420  4 19200 900 50.8 76800 56945.7 74
421  4 19200 400 53.2 76800 54376.7 70
422  4 19200 600 53.7 76800 53870.4 70
423  1 57600 900 54.6 57600 52982.4 91
424  1 57600 600 56.2 57600 51474 89
425  3 19200 900 60.5 57600 47815.5 83
426  1 57600 400 60.2 57600 48053.8 83
427  3 19200 600 62 57600 46658.7 81
428  3 19200 400 64.7 57600 44711.6 77
429  1 38400 900 79.4 38400 36433.8 94
430  1 38400 600 82.4 38400 35107.3 91
431  2 19200 900 84.4 38400 34275.4 89
432  1 38400 400 86.8 38400 33327.6 86
433  2 19200 600 87.6 38400 33023.3 85
434  2 19200 400 91.2 38400 31719.7 82
435  4 9600 900 94.7 38400 30547.4 79
436  4 9600 400 106 38400 27290.9 71
437  4 9600 600 110 38400 26298.5 68
438  3 9600 900 118 28800 24515.6 85
439  3 9600 600 120 28800 24107 83
440  3 9600 400 131 28800 22082.7 76
441  1 19200 900 155 19200 18663.5 97
442  1 19200 600 161 19200 17968 93
443  1 19200 400 170 19200 17016.7 88
444  2 9600 600 176 19200 16436.6 85
445  2 9600 900 180 19200 16071.3 83
446  2 9600 400 181 19200 15982.5 83
447  1 9600 900 305 9600 9484.72 98
448  1 9600 600 314 9600 9212.87 95
449  1 9600 400 332 9600 8713.37 90
450
451
452
453
454
455  5.2. Anthony Healy's Report
456
457
458
459
460
461
462
463  Date: Mon, 13 Feb 1995 16:17:29 +1100 (EST)
464  From: Antony Healey <ahealey@st.nepean.uws.edu.au>
465  To: Simon Janes <guru@ncm.com>
466  Subject: Re: Load Balancing
467
468  Hi Simon,
469      I've installed your patch and it works great. I have trialed
470      it over twin SL/IP lines, just over null modems, but I was
471      able to data at over 48Kb/s [ISDN link -Simon]. I managed a
472      transfer of up to 7.5 Kbyte/s on one go, but averaged around
473      6.4 Kbyte/s, which I think is pretty cool. :)
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529

Archive Download this file



interactive