Root/
1 | EQL Driver: Serial IP Load Balancing HOWTO |
2 | Simon "Guru Aleph-Null" Janes, simon@ncm.com |
3 | v1.1, February 27, 1995 |
4 | |
5 | This is the manual for the EQL device driver. EQL is a software device |
6 | that lets you load-balance IP serial links (SLIP or uncompressed PPP) |
7 | to increase your bandwidth. It will not reduce your latency (i.e. ping |
8 | times) except in the case where you already have lots of traffic on |
9 | your link, in which it will help them out. This driver has been tested |
10 | with the 1.1.75 kernel, and is known to have patched cleanly with |
11 | 1.1.86. Some testing with 1.1.92 has been done with the v1.1 patch |
12 | which was only created to patch cleanly in the very latest kernel |
13 | source trees. (Yes, it worked fine.) |
14 | |
15 | 1. Introduction |
16 | |
17 | Which is worse? A huge fee for a 56K leased line or two phone lines? |
18 | It's probably the former. If you find yourself craving more bandwidth, |
19 | and have a ISP that is flexible, it is now possible to bind modems |
20 | together to work as one point-to-point link to increase your |
21 | bandwidth. All without having to have a special black box on either |
22 | side. |
23 | |
24 | |
25 | The eql driver has only been tested with the Livingston PortMaster-2e |
26 | terminal server. I do not know if other terminal servers support load- |
27 | balancing, but I do know that the PortMaster does it, and does it |
28 | almost as well as the eql driver seems to do it (-- Unfortunately, in |
29 | my testing so far, the Livingston PortMaster 2e's load-balancing is a |
30 | good 1 to 2 KB/s slower than the test machine working with a 28.8 Kbps |
31 | and 14.4 Kbps connection. However, I am not sure that it really is |
32 | the PortMaster, or if it's Linux's TCP drivers. I'm told that Linux's |
33 | TCP implementation is pretty fast though.--) |
34 | |
35 | |
36 | I suggest to ISPs out there that it would probably be fair to charge |
37 | a load-balancing client 75% of the cost of the second line and 50% of |
38 | the cost of the third line etc... |
39 | |
40 | |
41 | Hey, we can all dream you know... |
42 | |
43 | |
44 | 2. Kernel Configuration |
45 | |
46 | Here I describe the general steps of getting a kernel up and working |
47 | with the eql driver. From patching, building, to installing. |
48 | |
49 | |
50 | 2.1. Patching The Kernel |
51 | |
52 | If you do not have or cannot get a copy of the kernel with the eql |
53 | driver folded into it, get your copy of the driver from |
54 | ftp://slaughter.ncm.com/pub/Linux/LOAD_BALANCING/eql-1.1.tar.gz. |
55 | Unpack this archive someplace obvious like /usr/local/src/. It will |
56 | create the following files: |
57 | |
58 | |
59 | |
60 | ______________________________________________________________________ |
61 | -rw-r--r-- guru/ncm 198 Jan 19 18:53 1995 eql-1.1/NO-WARRANTY |
62 | -rw-r--r-- guru/ncm 30620 Feb 27 21:40 1995 eql-1.1/eql-1.1.patch |
63 | -rwxr-xr-x guru/ncm 16111 Jan 12 22:29 1995 eql-1.1/eql_enslave |
64 | -rw-r--r-- guru/ncm 2195 Jan 10 21:48 1995 eql-1.1/eql_enslave.c |
65 | ______________________________________________________________________ |
66 | |
67 | Unpack a recent kernel (something after 1.1.92) someplace convenient |
68 | like say /usr/src/linux-1.1.92.eql. Use symbolic links to point |
69 | /usr/src/linux to this development directory. |
70 | |
71 | |
72 | Apply the patch by running the commands: |
73 | |
74 | |
75 | ______________________________________________________________________ |
76 | cd /usr/src |
77 | patch </usr/local/src/eql-1.1/eql-1.1.patch |
78 | ______________________________________________________________________ |
79 | |
80 | |
81 | |
82 | |
83 | |
84 | 2.2. Building The Kernel |
85 | |
86 | After patching the kernel, run make config and configure the kernel |
87 | for your hardware. |
88 | |
89 | |
90 | After configuration, make and install according to your habit. |
91 | |
92 | |
93 | 3. Network Configuration |
94 | |
95 | So far, I have only used the eql device with the DSLIP SLIP connection |
96 | manager by Matt Dillon (-- "The man who sold his soul to code so much |
97 | so quickly."--) . How you configure it for other "connection" |
98 | managers is up to you. Most other connection managers that I've seen |
99 | don't do a very good job when it comes to handling more than one |
100 | connection. |
101 | |
102 | |
103 | 3.1. /etc/rc.d/rc.inet1 |
104 | |
105 | In rc.inet1, ifconfig the eql device to the IP address you usually use |
106 | for your machine, and the MTU you prefer for your SLIP lines. One |
107 | could argue that MTU should be roughly half the usual size for two |
108 | modems, one-third for three, one-fourth for four, etc... But going |
109 | too far below 296 is probably overkill. Here is an example ifconfig |
110 | command that sets up the eql device: |
111 | |
112 | |
113 | |
114 | ______________________________________________________________________ |
115 | ifconfig eql 198.67.33.239 mtu 1006 |
116 | ______________________________________________________________________ |
117 | |
118 | |
119 | |
120 | |
121 | |
122 | Once the eql device is up and running, add a static default route to |
123 | it in the routing table using the cool new route syntax that makes |
124 | life so much easier: |
125 | |
126 | |
127 | |
128 | ______________________________________________________________________ |
129 | route add default eql |
130 | ______________________________________________________________________ |
131 | |
132 | |
133 | 3.2. Enslaving Devices By Hand |
134 | |
135 | Enslaving devices by hand requires two utility programs: eql_enslave |
136 | and eql_emancipate (-- eql_emancipate hasn't been written because when |
137 | an enslaved device "dies", it is automatically taken out of the queue. |
138 | I haven't found a good reason to write it yet... other than for |
139 | completeness, but that isn't a good motivator is it?--) |
140 | |
141 | |
142 | The syntax for enslaving a device is "eql_enslave <master-name> |
143 | <slave-name> <estimated-bps>". Here are some example enslavings: |
144 | |
145 | |
146 | |
147 | ______________________________________________________________________ |
148 | eql_enslave eql sl0 28800 |
149 | eql_enslave eql ppp0 14400 |
150 | eql_enslave eql sl1 57600 |
151 | ______________________________________________________________________ |
152 | |
153 | |
154 | |
155 | |
156 | |
157 | When you want to free a device from its life of slavery, you can |
158 | either down the device with ifconfig (eql will automatically bury the |
159 | dead slave and remove it from its queue) or use eql_emancipate to free |
160 | it. (-- Or just ifconfig it down, and the eql driver will take it out |
161 | for you.--) |
162 | |
163 | |
164 | |
165 | ______________________________________________________________________ |
166 | eql_emancipate eql sl0 |
167 | eql_emancipate eql ppp0 |
168 | eql_emancipate eql sl1 |
169 | ______________________________________________________________________ |
170 | |
171 | |
172 | |
173 | |
174 | |
175 | 3.3. DSLIP Configuration for the eql Device |
176 | |
177 | The general idea is to bring up and keep up as many SLIP connections |
178 | as you need, automatically. |
179 | |
180 | |
181 | 3.3.1. /etc/slip/runslip.conf |
182 | |
183 | Here is an example runslip.conf: |
184 | |
185 | |
186 | |
187 | |
188 | |
189 | |
190 | |
191 | |
192 | |
193 | |
194 | |
195 | |
196 | |
197 | |
198 | |
199 | ______________________________________________________________________ |
200 | name sl-line-1 |
201 | enabled |
202 | baud 38400 |
203 | mtu 576 |
204 | ducmd -e /etc/slip/dialout/cua2-288.xp -t 9 |
205 | command eql_enslave eql $interface 28800 |
206 | address 198.67.33.239 |
207 | line /dev/cua2 |
208 | |
209 | name sl-line-2 |
210 | enabled |
211 | baud 38400 |
212 | mtu 576 |
213 | ducmd -e /etc/slip/dialout/cua3-288.xp -t 9 |
214 | command eql_enslave eql $interface 28800 |
215 | address 198.67.33.239 |
216 | line /dev/cua3 |
217 | ______________________________________________________________________ |
218 | |
219 | |
220 | |
221 | |
222 | |
223 | 3.4. Using PPP and the eql Device |
224 | |
225 | I have not yet done any load-balancing testing for PPP devices, mainly |
226 | because I don't have a PPP-connection manager like SLIP has with |
227 | DSLIP. I did find a good tip from LinuxNET:Billy for PPP performance: |
228 | make sure you have asyncmap set to something so that control |
229 | characters are not escaped. |
230 | |
231 | |
232 | I tried to fix up a PPP script/system for redialing lost PPP |
233 | connections for use with the eql driver the weekend of Feb 25-26 '95 |
234 | (Hereafter known as the 8-hour PPP Hate Festival). Perhaps later this |
235 | year. |
236 | |
237 | |
238 | 4. About the Slave Scheduler Algorithm |
239 | |
240 | The slave scheduler probably could be replaced with a dozen other |
241 | things and push traffic much faster. The formula in the current set |
242 | up of the driver was tuned to handle slaves with wildly different |
243 | bits-per-second "priorities". |
244 | |
245 | |
246 | All testing I have done was with two 28.8 V.FC modems, one connecting |
247 | at 28800 bps or slower, and the other connecting at 14400 bps all the |
248 | time. |
249 | |
250 | |
251 | One version of the scheduler was able to push 5.3 K/s through the |
252 | 28800 and 14400 connections, but when the priorities on the links were |
253 | very wide apart (57600 vs. 14400) the "faster" modem received all |
254 | traffic and the "slower" modem starved. |
255 | |
256 | |
257 | 5. Testers' Reports |
258 | |
259 | Some people have experimented with the eql device with newer |
260 | kernels (than 1.1.75). I have since updated the driver to patch |
261 | cleanly in newer kernels because of the removal of the old "slave- |
262 | balancing" driver config option. |
263 | |
264 | |
265 | o icee from LinuxNET patched 1.1.86 without any rejects and was able |
266 | to boot the kernel and enslave a couple of ISDN PPP links. |
267 | |
268 | 5.1. Randolph Bentson's Test Report |
269 | |
270 | |
271 | |
272 | |
273 | |
274 | |
275 | |
276 | |
277 | |
278 | |
279 | |
280 | |
281 | |
282 | |
283 | |
284 | |
285 | |
286 | |
287 | |
288 | |
289 | |
290 | |
291 | |
292 | |
293 | |
294 | |
295 | |
296 | |
297 | |
298 | |
299 | |
300 | |
301 | |
302 | |
303 | |
304 | |
305 | |
306 | |
307 | |
308 | |
309 | |
310 | |
311 | |
312 | |
313 | |
314 | |
315 | |
316 | |
317 | |
318 | |
319 | |
320 | |
321 | |
322 | |
323 | |
324 | |
325 | |
326 | |
327 | |
328 | |
329 | |
330 | |
331 | From bentson@grieg.seaslug.org Wed Feb 8 19:08:09 1995 |
332 | Date: Tue, 7 Feb 95 22:57 PST |
333 | From: Randolph Bentson <bentson@grieg.seaslug.org> |
334 | To: guru@ncm.com |
335 | Subject: EQL driver tests |
336 | |
337 | |
338 | I have been checking out your eql driver. (Nice work, that!) |
339 | Although you may already done this performance testing, here |
340 | are some data I've discovered. |
341 | |
342 | Randolph Bentson |
343 | bentson@grieg.seaslug.org |
344 | |
345 | --------------------------------------------------------- |
346 | |
347 | |
348 | A pseudo-device driver, EQL, written by Simon Janes, can be used |
349 | to bundle multiple SLIP connections into what appears to be a |
350 | single connection. This allows one to improve dial-up network |
351 | connectivity gradually, without having to buy expensive DSU/CSU |
352 | hardware and services. |
353 | |
354 | I have done some testing of this software, with two goals in |
355 | mind: first, to ensure it actually works as described and |
356 | second, as a method of exercising my device driver. |
357 | |
358 | The following performance measurements were derived from a set |
359 | of SLIP connections run between two Linux systems (1.1.84) using |
360 | a 486DX2/66 with a Cyclom-8Ys and a 486SLC/40 with a Cyclom-16Y. |
361 | (Ports 0,1,2,3 were used. A later configuration will distribute |
362 | port selection across the different Cirrus chips on the boards.) |
363 | Once a link was established, I timed a binary ftp transfer of |
364 | 289284 bytes of data. If there were no overhead (packet headers, |
365 | inter-character and inter-packet delays, etc.) the transfers |
366 | would take the following times: |
367 | |
368 | bits/sec seconds |
369 | 345600 8.3 |
370 | 234600 12.3 |
371 | 172800 16.7 |
372 | 153600 18.8 |
373 | 76800 37.6 |
374 | 57600 50.2 |
375 | 38400 75.3 |
376 | 28800 100.4 |
377 | 19200 150.6 |
378 | 9600 301.3 |
379 | |
380 | A single line running at the lower speeds and with large packets |
381 | comes to within 2% of this. Performance is limited for the higher |
382 | speeds (as predicted by the Cirrus databook) to an aggregate of |
383 | about 160 kbits/sec. The next round of testing will distribute |
384 | the load across two or more Cirrus chips. |
385 | |
386 | The good news is that one gets nearly the full advantage of the |
387 | second, third, and fourth line's bandwidth. (The bad news is |
388 | that the connection establishment seemed fragile for the higher |
389 | speeds. Once established, the connection seemed robust enough.) |
390 | |
391 | #lines speed mtu seconds theory actual %of |
392 | kbit/sec duration speed speed max |
393 | 3 115200 900 _ 345600 |
394 | 3 115200 400 18.1 345600 159825 46 |
395 | 2 115200 900 _ 230400 |
396 | 2 115200 600 18.1 230400 159825 69 |
397 | 2 115200 400 19.3 230400 149888 65 |
398 | 4 57600 900 _ 234600 |
399 | 4 57600 600 _ 234600 |
400 | 4 57600 400 _ 234600 |
401 | 3 57600 600 20.9 172800 138413 80 |
402 | 3 57600 900 21.2 172800 136455 78 |
403 | 3 115200 600 21.7 345600 133311 38 |
404 | 3 57600 400 22.5 172800 128571 74 |
405 | 4 38400 900 25.2 153600 114795 74 |
406 | 4 38400 600 26.4 153600 109577 71 |
407 | 4 38400 400 27.3 153600 105965 68 |
408 | 2 57600 900 29.1 115200 99410.3 86 |
409 | 1 115200 900 30.7 115200 94229.3 81 |
410 | 2 57600 600 30.2 115200 95789.4 83 |
411 | 3 38400 900 30.3 115200 95473.3 82 |
412 | 3 38400 600 31.2 115200 92719.2 80 |
413 | 1 115200 600 31.3 115200 92423 80 |
414 | 2 57600 400 32.3 115200 89561.6 77 |
415 | 1 115200 400 32.8 115200 88196.3 76 |
416 | 3 38400 400 33.5 115200 86353.4 74 |
417 | 2 38400 900 43.7 76800 66197.7 86 |
418 | 2 38400 600 44 76800 65746.4 85 |
419 | 2 38400 400 47.2 76800 61289 79 |
420 | 4 19200 900 50.8 76800 56945.7 74 |
421 | 4 19200 400 53.2 76800 54376.7 70 |
422 | 4 19200 600 53.7 76800 53870.4 70 |
423 | 1 57600 900 54.6 57600 52982.4 91 |
424 | 1 57600 600 56.2 57600 51474 89 |
425 | 3 19200 900 60.5 57600 47815.5 83 |
426 | 1 57600 400 60.2 57600 48053.8 83 |
427 | 3 19200 600 62 57600 46658.7 81 |
428 | 3 19200 400 64.7 57600 44711.6 77 |
429 | 1 38400 900 79.4 38400 36433.8 94 |
430 | 1 38400 600 82.4 38400 35107.3 91 |
431 | 2 19200 900 84.4 38400 34275.4 89 |
432 | 1 38400 400 86.8 38400 33327.6 86 |
433 | 2 19200 600 87.6 38400 33023.3 85 |
434 | 2 19200 400 91.2 38400 31719.7 82 |
435 | 4 9600 900 94.7 38400 30547.4 79 |
436 | 4 9600 400 106 38400 27290.9 71 |
437 | 4 9600 600 110 38400 26298.5 68 |
438 | 3 9600 900 118 28800 24515.6 85 |
439 | 3 9600 600 120 28800 24107 83 |
440 | 3 9600 400 131 28800 22082.7 76 |
441 | 1 19200 900 155 19200 18663.5 97 |
442 | 1 19200 600 161 19200 17968 93 |
443 | 1 19200 400 170 19200 17016.7 88 |
444 | 2 9600 600 176 19200 16436.6 85 |
445 | 2 9600 900 180 19200 16071.3 83 |
446 | 2 9600 400 181 19200 15982.5 83 |
447 | 1 9600 900 305 9600 9484.72 98 |
448 | 1 9600 600 314 9600 9212.87 95 |
449 | 1 9600 400 332 9600 8713.37 90 |
450 | |
451 | |
452 | |
453 | |
454 | |
455 | 5.2. Anthony Healy's Report |
456 | |
457 | |
458 | |
459 | |
460 | |
461 | |
462 | |
463 | Date: Mon, 13 Feb 1995 16:17:29 +1100 (EST) |
464 | From: Antony Healey <ahealey@st.nepean.uws.edu.au> |
465 | To: Simon Janes <guru@ncm.com> |
466 | Subject: Re: Load Balancing |
467 | |
468 | Hi Simon, |
469 | I've installed your patch and it works great. I have trialed |
470 | it over twin SL/IP lines, just over null modems, but I was |
471 | able to data at over 48Kb/s [ISDN link -Simon]. I managed a |
472 | transfer of up to 7.5 Kbyte/s on one go, but averaged around |
473 | 6.4 Kbyte/s, which I think is pretty cool. :) |
474 | |
475 | |
476 | |
477 | |
478 | |
479 | |
480 | |
481 | |
482 | |
483 | |
484 | |
485 | |
486 | |
487 | |
488 | |
489 | |
490 | |
491 | |
492 | |
493 | |
494 | |
495 | |
496 | |
497 | |
498 | |
499 | |
500 | |
501 | |
502 | |
503 | |
504 | |
505 | |
506 | |
507 | |
508 | |
509 | |
510 | |
511 | |
512 | |
513 | |
514 | |
515 | |
516 | |
517 | |
518 | |
519 | |
520 | |
521 | |
522 | |
523 | |
524 | |
525 | |
526 | |
527 | |
528 | |
529 |
Branches:
ben-wpan
ben-wpan-stefan
javiroman/ks7010
jz-2.6.34
jz-2.6.34-rc5
jz-2.6.34-rc6
jz-2.6.34-rc7
jz-2.6.35
jz-2.6.36
jz-2.6.37
jz-2.6.38
jz-2.6.39
jz-3.0
jz-3.1
jz-3.11
jz-3.12
jz-3.13
jz-3.15
jz-3.16
jz-3.18-dt
jz-3.2
jz-3.3
jz-3.4
jz-3.5
jz-3.6
jz-3.6-rc2-pwm
jz-3.9
jz-3.9-clk
jz-3.9-rc8
jz47xx
jz47xx-2.6.38
master
Tags:
od-2011-09-04
od-2011-09-18
v2.6.34-rc5
v2.6.34-rc6
v2.6.34-rc7
v3.9