|
1 <html><head> |
|
2 <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"> |
|
3 <title>DTN External Router Interface</title><meta name="generator" content="DocBook XSL Stylesheets V1.69.1"></head><body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"><div class="article" lang="en"><div class="titlepage"><div><div><h2 class="title"><a name="d0e1"></a>DTN External Router Interface</h2></div><div><h3 class="subtitle"><i>DRAFT</i></h3></div><div><p class="releaseinfo">Approved for Public Release; Distribution Unlimited. Case |
|
4 Number: 06-1215</p></div><div><p class="copyright">Copyright © 2006 The MITRE Corporation. All rights reserved.</p></div><div><div class="revhistory"><table border="1" width="100%" summary="Revision history"><tr><th align="left" valign="top" colspan="2"><b>Revision History</b></th></tr><tr><td align="left">Revision 0.90</td><td align="left">11.13.06</td></tr></table></div></div></div><hr></div><div class="toc"><p><b>Table of Contents</b></p><dl><dt><span class="section"><a href="#d0e24">Overview</a></span></dt><dt><span class="section"><a href="#Section2">Interface Operation</a></span></dt><dt><span class="section"><a href="#Section3">Protocol Elements</a></span></dt><dd><dl><dt><span class="section"><a href="#d0e111">Rendezvous</a></span></dt><dt><span class="section"><a href="#d0e142">Links and Contacts</a></span></dt><dt><span class="section"><a href="#d0e189">Routes</a></span></dt><dt><span class="section"><a href="#d0e204">Bundles</a></span></dt><dt><span class="section"><a href="#d0e246">Peer Router Communication</a></span></dt><dt><span class="section"><a href="#d0e265">Failure Recovery</a></span></dt></dl></dd><dt><span class="appendix"><a href="#AppendixA">A. New Configuration Directives</a></span></dt><dt><span class="appendix"><a href="#AppendixB">B. Interface Design FAQ</a></span></dt></dl></div><div class="section" lang="en"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="d0e24"></a>Overview</h2></div></div></div><p>A new implementation architecture is under development that promises |
|
5 to ease the task of adding new capabilities to DTN. Using generic external |
|
6 interfaces, third-party developers will be able to implement plug-in |
|
7 modules without having to rewrite or understand the internal workings of |
|
8 the DTN2 reference implementation (RI). Plug-in modules are stand-alone |
|
9 processes designed to work in collaboration with DTN. In the longer term, |
|
10 this effort provides the basis for extending commercial off-the-shelf |
|
11 (COTS) DTN implementations with domain specific capabilities.</p><p>The External Router Interface, documented here, is the first DTN |
|
12 external interface designed to move bundle forwarding decisions to an |
|
13 external process or processes. Benefits of this interface include:</p><div class="itemizedlist"><ul type="disc"><li><p>Classified and proprietary bundle routing algorithms are |
|
14 protected.</p></li><li><p>External routers may be written in any programming language (XML |
|
15 based interface).</p></li><li><p>Prototyping of new bundle routing algorithms is |
|
16 streamlined.</p></li><li><p>Routing algorithms may be added and removed at runtime.</p></li></ul></div><p>This guide is directed at external router developers and to those |
|
17 interested in an early product of the new DTN implementation architecture. |
|
18 <a href="#Section2" title="Interface Operation">"Interface Operation"</a> describes interface |
|
19 fundamentals such as underlying protocols and message structure. <a href="#Section3" title="Protocol Elements">"Protocol Elements"</a> introduces the set of router |
|
20 interface messages used for inter-process communication. <a href="#AppendixA" title="A. New Configuration Directives">Appendix A</a> summarizes new configuration options |
|
21 introduced with the DTN2 RI external router interface implementation. |
|
22 <a href="#AppendixB" title="B. Interface Design FAQ">Appendix B</a> is an interface design FAQ of |
|
23 sorts that attempts to address questions that often arise when first |
|
24 learning about the interface.</p><div class="important" style="margin-left: 0.5in; margin-right: 0.5in;"><h3 class="title">Important</h3><p>Please note that the external router interface is still evolving |
|
25 and will likely be modified as experience is gained externalizing DTN |
|
26 bundle routers. Future work on other interfaces, including |
|
27 convergence-layer and storage interfaces, may also impact the interface. |
|
28 The latest "bleeding edge" version of the DTN2 RI includes support for |
|
29 the external router interface documented here and is available at <a href="http://www.dtnrg.org" target="_top">www.dtnrg.org</a>.</p></div></div><div class="section" lang="en"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="Section2"></a>Interface Operation</h2></div></div></div><p>Before diving into the details, this section discusses the overall |
|
30 design and operation of the router interface. When compiled, the DTN2 RI |
|
31 2.2.0 release (the latest formal DTN2 release as of this writing) produces |
|
32 one multi-threaded executable, a DTN daemon. This daemon is a complete DTN |
|
33 node; it accepts bundles from a number of built-in convergence layers, |
|
34 provides persistent bundle storage, delivers bundles to local DTN |
|
35 applications, selects routes using built-in routing logic, and forwards |
|
36 bundles to peer DTN nodes. For the purposes of this document, a <span class="">forwarder</span> is a DTN daemon as described above, but one |
|
37 that depends on external routing processes to make bundle routing |
|
38 decisions on its behalf.</p><p>The forwarder communicates with external routing processes:</p><div class="itemizedlist"><ul type="disc"><li><p>with an XML-based messaging protocol</p></li><li><p>using a well-known IPv4 multicast address and UDP port</p></li><li><p>on local or remote hosts</p></li></ul></div><p>Before forwarders and external routers can communicate, they each |
|
39 must join the all-routers multicast group (224.0.0.2) and bind to a |
|
40 well-known UDP port. Forwarders are not aware if zero, one, or more |
|
41 external routers have joined the multicast group. It is the responsibility |
|
42 of the system administrator to ensure router availability.</p><p>Nothing in the interface design precludes running forwarders and |
|
43 external routers on different hosts, however this approach is not |
|
44 recommended especially in wireless and/or bandwidth-constrained |
|
45 environments. The interface is fairly chatty, UDP is unreliable, and there |
|
46 is a high risk of packet loss leading to a breakdown in synchronized |
|
47 state. (The DTN2 RI external router interface is hard coded to use the |
|
48 loopback interface and therefore requires forwarders and external routers |
|
49 to reside on the same host.)</p><p>Inter-process messages are XML-based and must be valid against the |
|
50 <a href="router-xsddoc/index.html" target="_top">external router XML schema</a>. |
|
51 Interface messages are broadly divided into four categories. Event |
|
52 messages are issued by forwarders to indicate state changes that may be of |
|
53 interest to external routers. Request messages are sent by external |
|
54 routers to direct forwarders to perform an action (e.g. "forward the |
|
55 bundle with ID 56 out link tcp0"). Query and report messages are used by |
|
56 external routers to synchronize their state with forwarders after bootup |
|
57 or during failure recovery. The proper usage and interpretation of each |
|
58 interface message is covered in the next section.</p><p>How does the forwarder authenticate external processes? It doesn't. |
|
59 The forwarder makes the assumption that (local or remote) external routers |
|
60 are within the same security domain. In addition, system administrators |
|
61 must ensure there exists one authoritative router or policy module per DTN |
|
62 "node”, or that multiple external routers are configured in a cooperative |
|
63 manner to correctly handle all events.</p><p>In addition to external routers, other "read-only" types of |
|
64 processes may join the all-routers multicast group. An external logging |
|
65 application, for example, could be written to display the current state of |
|
66 the forwarder simply by listening to external router interface messages. |
|
67 However, this concept is not pursued further in this document. The |
|
68 following discussion is limited to inter-process communication between |
|
69 forwarders and external routers.</p></div><div class="section" lang="en"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="Section3"></a>Protocol Elements</h2></div></div></div><p>This section describes each interface message according to purpose. |
|
70 Hyper-links to generated schema documentation are located throughout the |
|
71 text. To generate the documentation, <span class="type">cd</span> to the top level |
|
72 directory of DTN2 source code and type <span class="type">make xsddoc</span>. You will |
|
73 need <a href="http://xframe.sourceforge.net/xsddoc/" target="_top">xsddoc-1.0</a> |
|
74 and java installed for this to work.</p><div class="section" lang="en"><div class="titlepage"><div><div><h3 class="title"><a name="d0e111"></a>Rendezvous</h3></div></div></div><p>When a forwarder is configured for external routing, it |
|
75 periodically multicasts hello messages to announce its presence to |
|
76 external routers. Each hello message includes the EID of the sender and |
|
77 the interval in seconds between each message. For example, the |
|
78 message</p><p><code class="literal"><dtn eid="dtn://nodeX" |
|
79 hello_interval="30"/></code></p><p>notifies external routers that the DTN forwarder with EID |
|
80 dtn://nodeX is running and will send another hello message in 30 |
|
81 seconds. Router implementations may filter incoming messages by EID if |
|
82 it becomes necessary to run more than one forwarder on a single |
|
83 platform.</p><p>After routers receive one or more hello messages from any (or a |
|
84 specific) DTN node, they must synchronize their state with the |
|
85 forwarder. Several routing interface messages are designed to facilitate |
|
86 state synchronization and have a common format. If x represents |
|
87 meta-information of interest, the message x_query prompts the forwarder |
|
88 to send an x_report message. For example, a <a href="router-xsddoc/noNamespace/element/bundle_query.html" target="_top">bundle_query</a> |
|
89 message asks the forwarder to generate and send a bundle_report |
|
90 message.</p><p>Routers may query the forwarder for four types of |
|
91 meta-information. <a href="router-xsddoc/noNamespace/element/contact_report.html" target="_top">contact_report</a> |
|
92 includes meta-information on all open connections between the local |
|
93 forwarder and remote forwarders. Low-level convergence layer |
|
94 meta-information is also included for each contact. <a href="router-xsddoc/noNamespace/element/link_report.html" target="_top">link_report</a> |
|
95 contains meta information on all forwarder links regardless of the |
|
96 current state. <a href="router-xsddoc/noNamespace/element/route_report.html" target="_top">route_report</a> |
|
97 is composed of meta-information on all static routes configured on the |
|
98 DTN console (if any). <a href="router-xsddoc/noNamespace/element/bundle_report.html" target="_top">bundle_report</a> |
|
99 lists meta-information on each bundle stored by the local |
|
100 forwarder.</p><p>After synchronizing their state with a forwarder, external routers |
|
101 process incoming events as they arrive and send requests when |
|
102 necessary.</p></div><div class="section" lang="en"><div class="titlepage"><div><div><h3 class="title"><a name="d0e142"></a>Links and Contacts</h3></div></div></div><p>DTN links represent a one way communication channel to next hop |
|
103 DTN nodes. Links are either explicitly created in the forwarder |
|
104 configuration file (<code class="filename">dtn.conf</code>) or spring into |
|
105 existence as a result of the neighbor discovery mechanism. Four types of |
|
106 links exist: <span class="emphasis"><em>always on</em></span>, <span class="emphasis"><em>on |
|
107 demand</em></span>, <span class="emphasis"><em>scheduled</em></span>, and |
|
108 <span class="emphasis"><em>opportunistic</em></span>. <a href="router-xsddoc/noNamespace/element/link_created_event.html" target="_top">link_created_event</a> |
|
109 messages announce the existence of new links and include such attributes |
|
110 as name, type, and destination EID along with other meta-information. |
|
111 <a href="router-xsddoc/noNamespace/element/link_deleted_event.html" target="_top">link_deleted_event</a> |
|
112 notifies external routers that a named link has been destroyed. <a href="router-xsddoc/noNamespace/element/link_available_event.html" target="_top">link_available_event</a> |
|
113 and <a href="router-xsddoc/noNamespace/element/link_unavaiable.html" target="_top">link_unavailable_event</a> |
|
114 messages are automatically generated by the forwarder when links are |
|
115 open or closed respectively.</p><p>The forwarder attempts to keep always on and opportunistic links |
|
116 open always. Scheduled links are automatically opened and closed |
|
117 according to configured start and duration settings. On demand links, |
|
118 however, must be explicitly opened by the router using an <a href="router-xsddoc/noNamespace/element/open_link_request.html" target="_top">open_link_request</a>. |
|
119 After on demand links are idle for a set period of time (currently 30 |
|
120 seconds), they are closed by the forwarder.</p><p>When the state of any link is either open or busy, a contact |
|
121 element is also available which tracks connection and convergence layer |
|
122 statistics that may be useful to routing algorithms. <a href="router-xsddoc/noNamespace/element/contact_up_event.html" target="_top">contact_up_event</a> |
|
123 and <a href="router-xsddoc/noNamespace/element/contact_down_event.html" target="_top">contact_down_event</a> |
|
124 messages notify routers when contacts are available or |
|
125 unavailable.</p><p>Most link and contact messages carry a coarse "reason" attribute |
|
126 to explain what prompted each link state change.</p></div><div class="section" lang="en"><div class="titlepage"><div><div><h3 class="title"><a name="d0e189"></a>Routes</h3></div></div></div><p>The primary function of DTN external routers is to maintain a |
|
127 forwarding information base of optimal next hop DTN nodes for remote EID |
|
128 prefixes. Routes to remote EID prefixes are either statically |
|
129 configured, learned from the DTN neighbor discovery mechanism, or |
|
130 learned from peer router communications. Specific DTN routing algorithms |
|
131 and implementations are beyond the scope of this document.</p><p>Static routes may be configured in the forwarder configuration |
|
132 file or directly on the DTN console. When static routes are specified, a |
|
133 <a href="router-xsddoc/noNamespace/element/route_add_event.html" target="_top">route_add_event</a> |
|
134 is multicast to external routers. Router implementations may choose to |
|
135 add static routes to internal routing tables (possibly with an |
|
136 identifying marker indicating the source or administrative distance) or |
|
137 simply ignore their existence. The <a href="router-xsddoc/noNamespace/element/route_delete_event.html" target="_top">route_delete_event</a> |
|
138 message is issued when a static route has been deleted at the DTN |
|
139 console.</p><p>The external router interface does not include messages to support |
|
140 the display of external routing tables on the DTN console. All but the |
|
141 most simplistic external routers should have their own "console" for |
|
142 configuration and routing table displays.</p></div><div class="section" lang="en"><div class="titlepage"><div><div><h3 class="title"><a name="d0e204"></a>Bundles</h3></div></div></div><p>Bundles arriving at a forwarder, either from a remote DTN node or |
|
143 from a local application, are announced with a <a href="router-xsddoc/noNamespace/element/bundle_received_event.html" target="_top">bundle_received_event</a> |
|
144 message. Bundle payload is not forwarded over the interface, only |
|
145 meta-information.</p><p>When an opportunity exists to forward a bundle, the external |
|
146 router issues a <a href="router-xsddoc/noNamespace/element/send_bundle_request.html" target="_top">send_bundle_request</a> |
|
147 which includes the local bundle id, name of the outgoing link, and the |
|
148 desired forwarding behavior. Meta-information about the bundle should |
|
149 not be erased until a positive acknowledgement is received from the |
|
150 forwarder.</p><p><a href="router-xsddoc/noNamespace/element/bundle_transmitted_eventt.html" target="_top">bundle_transmitted_event</a> |
|
151 states that a bundle has been successfully transmitted using the |
|
152 identified contact. On reliable links, this means acknowledgements were |
|
153 received, on unreliable links, a best effort attempt was made to send |
|
154 the bundle. A failed transmission is indicated by the <a href="router-xsddoc/noNamespace/element/bundle_transmit_failed_eventt.html" target="_top">bundle_transmit_failed_event</a> |
|
155 message and may warrant a retransmission by the router if |
|
156 possible.</p><p>For successfully transmitted bundles, routers can release all |
|
157 resources dedicated to tracking the bundle unless the sender had |
|
158 requested custody transfer. If custody was requested, the router must |
|
159 continue to track the bundle until it receives a corresponding custody |
|
160 message. If custody was accepted by a downstream DTN node, a <a href="router-xsddoc/noNamespace/element/custody_signal_event.html" target="_top">custody_signal_event</a> |
|
161 is generated. Bundle meta-information can now be released by the router. |
|
162 If custody failed, indicated by the <span class="emphasis"><em>succeeded</em></span> |
|
163 attribute in the <a href="router-xsddoc/noNamespace/element/custody_signal_event.html" target="_top">custody_signal_event</a>, |
|
164 or the router receives a <a href="router-xsddoc/noNamespace/element/custody_timeout_event.html" target="_top">custody_timeout_event</a> |
|
165 (i.e. no response from a downstream DTN node), the router may attempt to |
|
166 retransmit the bundle if possible.</p><p><a href="router-xsddoc/noNamespace/element/bundle_expired_event.html" target="_top">bundle_expired_event</a> |
|
167 notifies the router that a bundle has expired and is being deleted from |
|
168 the store. External routers should remove all records of the |
|
169 bundle.</p><p><a href="router-xsddoc/noNamespace/element/cancel_bundle_event.html" target="_top">cancel_bundle_request</a> |
|
170 asks the forwarder to abort a previous send_bundle request if possible. |
|
171 (As of this writing, cancel_bundle_request is not implemented in the |
|
172 DTN2 RI.)</p></div><div class="section" lang="en"><div class="titlepage"><div><div><h3 class="title"><a name="d0e246"></a>Peer Router Communication</h3></div></div></div><p>Many routing algorithms require information exchange between peer |
|
173 routers. Communication between peer DTN external routers boils down to |
|
174 identifying methods for sending and receiving router message bundles. |
|
175 Two methods are currently available; the first utilizes the DTN2 |
|
176 application interface and the second is offered by the external router |
|
177 interface itself.</p><p>External router developers may elect to build their external |
|
178 routers with two personalities, to behave as an external router and as a |
|
179 DTN application. The external router interface handles all bundle |
|
180 forwarding tasks while the application interface is used to send and |
|
181 receive peer router messages. In order for this method to work, external |
|
182 routers must register with the forwarder as a DTN application and supply |
|
183 a well-known service tag for bundle deliveries. It seems intuitive that |
|
184 the service tag would signify the routing algorithm. For example, an |
|
185 external router implementing a flooding algorithm may register with the |
|
186 forwarder using the service tag "flood_rtr".</p><p>This approach offers several benefits: the DTN application |
|
187 interface is reliable, there is no limit on bundle size, and |
|
188 router-originated bundles follow the same path as "normal" application |
|
189 bundles and are subject to the same policies. Cons include increased |
|
190 development complexity and the fact that external routers can't specify |
|
191 an outgoing link using the application API. The forwarder defers to the |
|
192 external router the decision of where to forward router-originated |
|
193 bundles.</p><p>The second alternative is to send locally generated bundle |
|
194 payloads to the forwarder on <a href="router-xsddoc/noNamespace/element/inject_bundle_request.html" target="_top">inject_bundle_request</a> |
|
195 messages. Destination EID, outgoing link, and other attributes required |
|
196 for bundle forwarding are supplied as well as the payload to inject. |
|
197 Router-originated bundle payloads are assumed to be binary blobs and are |
|
198 Base64 encoded. No attempt is made to decode or process |
|
199 router-originated bundle payloads except by the destination remote |
|
200 router process. Forwarders notify external routers of successful or |
|
201 failed transmissions in the same manner as "normal" bundles (I need to |
|
202 check this...). Unlike DTN application bundles, router injected bundles |
|
203 are not cached. If transmission fails, bundles are dropped.</p><p>At boot-up, the external router interface registers with the |
|
204 forwarder to receive all bundles destined to “ext.rtr/*”. As in the |
|
205 previous example, a service tag is chosen by the external router |
|
206 developer to uniquely identify the router. Arriving bundles matching the |
|
207 registration "ext_rtr/*" are announced with the <a href="router-xsddoc/noNamespace/element/bundle_delivery_event.html" target="_top">bundle_delivery_event</a> |
|
208 message. Upon receipt of this message, routers must examine the bundle's |
|
209 destination EID to determine the intended recipient of the bundle. |
|
210 Bundle payloads are read in or otherwise accessed using the filename |
|
211 attribute associated with bundle payload meta-information.</p></div><div class="section" lang="en"><div class="titlepage"><div><div><h3 class="title"><a name="d0e265"></a>Failure Recovery</h3></div></div></div><p>Since IPC between forwarders and external routers is |
|
212 connectionless, one process can fail without the other's immediate |
|
213 knowledge. Consider the case where there is one forwarder process and |
|
214 one external router process. Five states are possible 1) both process |
|
215 are up, 2) forwarder is up, external router is down, 3) forwarder is |
|
216 down, external router is up, 4) forwarder crashes and comes back up |
|
217 within a hello interval, and 5) both processes are down. Obviously, Case |
|
218 1 does not belong in a Failure Recovery Section.</p><p>A discussion of Case 2 yields an interesting point. The forwarder |
|
219 does not monitor whether zero, one, or more external routers are running |
|
220 and/or are members of the all-routers multicast group. Event messages |
|
221 are multicast to the all-routers group, by the forwarder, regardless of |
|
222 external router state. When an external router is restarted, it should |
|
223 immediately synchronize its state with the forwarder in much the same |
|
224 manner as the initial bootup sequence discussed in the previous |
|
225 section.</p><p>There are two instances in the life of a forwarder where the hello |
|
226 message includes an additional attribute. The "alert" attribute provides |
|
227 additional information about the overall state of the forwarder and |
|
228 facilitates failure recovery in the remaining cases. Two values are |
|
229 currently valid: "justBooted" and "shuttingDown". At bootup, the |
|
230 forwarder always sends the "justBooted" alert with the first hello |
|
231 message. When the forwarder is shut down in an orderly fashion (by |
|
232 issuing "quit" at the console for example), a hello message is |
|
233 immediately sent that includes the "shuttingDown" alert. However, if the |
|
234 forwarder should crash or is otherwise shutdown in an unorderly manner, |
|
235 there is no guarantee that a "shuttingDown" alert will be sent.</p><p>For Case 3, the external router determines the forwarder is down |
|
236 either by receiving a hello message with a "shuttingDown" alert or after |
|
237 hello interval seconds have passed without receiving a message. At this |
|
238 point, the router must stop sending messages, wait until another hello |
|
239 message is received, synchronize its state with the forwarder, and |
|
240 resume normal operations.</p><p>The external router knows it is dealing with Case 4 if it receives |
|
241 a "justBooted" alert in less than or equal to one hello interval. The |
|
242 router must stop sending messages, synchronize its state with the |
|
243 forwarder, and resume normal operations.</p><p>Towards the goal of making DTN a turnkey solution, additional |
|
244 "helper" processes are being developed whose sole purpose is to watch |
|
245 and restart forwarders and external routers if and when they fail. |
|
246 Helper processes will play a critical role in addressing Cases 2 - |
|
247 5.</p></div></div><div class="appendix" lang="en"><h2 class="title" style="clear: both"><a name="AppendixA"></a>A. New Configuration Directives</h2><p>This section describes four new configuration directives that |
|
248 accompany the DTNRG RI external router interface.</p><p>To enable the external router feature, set the forwarder router type |
|
249 to "external".</p><p><code class="literal">route set type external</code></p><p>Next, provide the full pathname of the XML schema file "router.xsd" |
|
250 (originally located in the DTN2 source distribution daemon |
|
251 directory).</p><p><code class="literal">route set schema <pathname></code></p><p>The remaining directives are optional (i.e. they have default |
|
252 values).</p><p>The default UDP port for external router communications is 8001. To |
|
253 change it, include the following directive in |
|
254 <code class="filename">dtn.conf</code>.</p><p><code class="literal">route set server_port <udp port></code></p><p>The number of seconds between forwarder hello messages can also be |
|
255 changed from the default of 30 seconds.</p><p><code class="literal">route set hello_interval <seconds></code></p></div><div class="appendix" lang="en"><h2 class="title" style="clear: both"><a name="AppendixB"></a>B. Interface Design FAQ</h2><p>Why multicast?</p><p></p><p>Why UDP?</p><p></p><p>Why XML?</p><p></p><p>...</p></div></div></body></html> |