TAO/docs/releasenotes/rtc10_sched.html

   1 <!doctype html public "-//w3c//dtd html 4.0 transitional//en">
   2 <!--  -->
   3 <html>
   4 <head>
   5    <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
   6    <meta name="Author" content="Venkita Subramonian">
   7    <meta name="GENERATOR" content="Mozilla/4.79 [en] (Windows NT 5.0; U) [Netscape]">
   8    <title>RTCORBA 1.0 Scheduling Service</title>
   9 </head>
  10 <body>
  11
  12 <center>
  13 <h2>
  14 RTCORBA 1.0 Scheduling Service</h2></center>
  15
  16 <p><br>Matt Murphy &lt;murphym@cs.uri.edu>
  17 <br>University of Rhode Island
  18 <p>This is an implementation of the RTCORBA 1.0 Scheduling Service. Per
  19 section 3 of the RTCORBA 1.0 specification (OMG), the scheduling service
  20 is comprised of two local interfaces, a ClientScheduler and a ServerScheduler.
  21 <br>&nbsp;
  22 <h3>
  23 Build Issues:</h3>
  24 Run tao_idl -I $TAO_ROOT/ RTCosScheduling.pidl.
  25 Run make -f Makefile.RTCosScheduling from ../
  26 <br>&nbsp;
  27 <h3>
  28 Synopsis:</h3>
  29 The RTCosScheduler allows clients to schedule tasks according to scheduling
  30 information determined a priori.&nbsp; This scheduling information is stored
  31 in a config file so that both the client and the server has access to it.&nbsp;
  32 (If the client and server exists on different nodes then place a copy of
  33 the config file on each node.)
  34 <p>Per the RTCORBA 1.0 spec, clients use a ClientScheduler object and servers
  35 use a ServerScheduler object to schedule activities on the system.&nbsp;
  36 Since each may or may not use its scheduler, there are four possible scenarios
  37 in which the system may run.&nbsp; These are:
  38 <p>1. Client uses ClientScheduler, Server uses ServerScheduler.&nbsp; In
  39 this case the system follows the rules set forth in the "Scheduling Service"
  40 section of this document below.
  41 <p>2. Client uses ClientScheduler, Server does not use ServerScheduler.
  42 In this case activities are scheduled on the client and run at the mapped
  43 Real Time priority set forth in the config&nbsp; file while executing on
  44 the client.&nbsp; However, any activity on the server does not run at a
  45 real time priority.&nbsp; This means that Multiprocessor Priority Ceiling
  46 Protocol does not manage activities on the server.&nbsp; Currently, the
  47 client has no way of knowing that activity on the server did not follow
  48 the MPCP protocol.&nbsp; Future enhancements to the RTCORBA 1.0 scheduling
  49 service should notify the client (perhaps through a flag to a client interceptor)
  50 that&nbsp;&nbsp; the server did not use MPCP.&nbsp; Please note that this
  51 scenario is generally&nbsp; not recommended as there is a strong possibility
  52 for&nbsp; priority inversion or unexpected blocking in this situation since
  53 any and all server activity that uses the ServerScheduler&nbsp; will run
  54 at a higher priority that server activity that does not.&nbsp;&nbsp; Use
  55 scenario 1 above.&nbsp; Here, the server's priority lowers from&nbsp; RTCORBA::maxPriority
  56 to RTCORBA::minPriority and things will&nbsp; execute on a best effort
  57 basis.
  58 <p>3. Client does not use ClientScheduler, Server uses ServerScheduler.&nbsp;
  59 In this case the client does not use priorities set forth in the config
  60 file.&nbsp; The ServerScheduler, on the other hand, does use MPCP to schedule
  61 execution on the server.&nbsp; It uses the priority sent to the server
  62 by the client, which is the default priority that the
  63 <br>&nbsp;&nbsp; client ran at (since the client priority was not changed
  64 by schedule_activity().&nbsp; This follows the scenario of the ServerScheduler
  65 set forth below.&nbsp; Please note that it is recommended
  66 that you use
  67 scenario 1, above, instead so that the client sends appropriate priorities
  68 to the server.
  69 <p>4. Client does not use ClientScheduler, server does not use ServerScheduler.&nbsp;
  70 In this case neither the client nor the server take advantage of&nbsp;
  71 the RTCORBA 1.0 Scheduler.
  72 <br>&nbsp;
  73 <h3>
  74 Scheduling Service:</h3>
  75 ClientScheduler:
  76 <br>Clients wishing to use the ClientScheduler to schedule activities
  77 <br>must first create a local ClientScheduler object reference.&nbsp; The
  78 <br>ClientScheduler is declared as:
  79 <p>RTCosScheduling_ClientScheduler_i (
  80 <br>&nbsp;&nbsp; CORBA::ORB_var orb, /// Orb reference
  81 <br>&nbsp;&nbsp; char* node,&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
  82 /// Node the client resides on
  83 <br>&nbsp;&nbsp; char* file);&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
  84 /// Config file holding scheduling information
  85 <br>&nbsp;
  86 <p>The ClientScheduler constructor parses the config file and populates
  87 an ACE_MAP with the activity/priority associations for the node on which
  88 the client resides.&nbsp; It also constructs a ClientScheduler_Interceptor
  89 that adds a service context the send_request interceptor that contains
  90 the priority the client is running at when the call is made.
  91 <p>Once initialized, calls to the ClientScheduler schedule_activity(const
  92 char * activity_name) method will match the activity_name parameter to
  93 the CORBA priority value in the ACE_Map.&nbsp; It linearly maps CORBA priority
  94 to a local OS priority and sets the local OS priority using RT Current.&nbsp;
  95 If the activity name provided is not valid (i.e. not found in the config
  96 file), a RTCosScheduling::UnknownName exception is thrown.
  97 <p>The ClientScheduler also registers an client side interceptor with the
  98 orb.&nbsp; This ClientScheduler_Interceptor finds the CORBA priority that
  99 the client is running at when the remote method call is made and adds this
 100 priority to a service context for the ServerScheduler_Interceptor to use.&nbsp;
 101 Initial tests find that this interceptor adds 0.00015 seconds of execution
 102 on an Intel 3.0 GHz processor.
 103 <br>&nbsp;
 104 <h3>
 105 ServerScheduler:</h3>
 106 Servers that contain local objects that will accept CORBA calls must create
 107 a local ServerScheduler object.&nbsp; The ServerScheduler uses TAO's PortableInterceptors
 108 to intercept incoming client requests and schedule execution on the server.&nbsp;
 109 These interceptors are registered by the ORB_Core as explained in the create_POA
 110 method below. The ServerScheduler is defined as:
 111 <p>&nbsp; RTCosScheduling_ServerScheduler_i (
 112 <br>&nbsp;&nbsp;&nbsp; char *node,&nbsp;&nbsp;&nbsp; /// Node the ServerScheduler
 113 resides on
 114 <br>&nbsp;&nbsp;&nbsp; char *file,&nbsp;&nbsp;&nbsp; /// Config file holding
 115 scheduling information
 116 <br>&nbsp;&nbsp;&nbsp; char *shared_file,&nbsp; /// File used for shared
 117 memory
 118 <br>&nbsp;&nbsp;&nbsp; int numthreads);&nbsp;&nbsp; /// Number of threads
 119 to create in the threadpool
 120 <p>During initialization, the ServerScheduler finds the appropriate node
 121 information in the config file and stores resources (key) on the node and
 122 the appropriate priority ceiling (value) in a map.&nbsp; It also reads
 123 in the base priority for the resource.
 124 <p>The ServerScheduler constructor then registers the PortableInterceptors
 125 necessary to scheduler execution on the server.&nbsp; It also set up the
 126 linear mapping policy and a reference to the RT Current object, both of
 127 which are used for adjusting the server's local OS priority when using
 128 the priority ceiling control protocol.
 129 <p>Once the ServerScheduler object is constructed, users may create an
 130 orb and establish any non real time POA policies they wish to install by
 131 calling the ServerScheduler's create_POA method.
 132 <p>ServerScheduler's create_POA method creates a real time POA that will
 133 set and enforce all non-real time policies.&nbsp; This method also sets
 134 the real time POA to enforce the Server Declared Priority Model Policy
 135 and creates a threadpool responsible for executing calls to the server.
 136 Server Declared Priority Model is used so that the server threads may run
 137 at a high enough priority to intercept requests as soon as they come in.
 138 If Client Propagated Priority Ceilings were used, incoming requests would
 139 not be intercepted until all existing servant execution is completed.&nbsp;
 140 This is because MPCP elevates the priority of servant execution to be higher
 141 than the client priorities.
 142 <p>Recall that the number of threads in the threadpool was supplied by
 143 the ServerScheduler constructor.&nbsp; The create_POA method is defined
 144 as:
 145 <p>&nbsp; virtual ::PortableServer::POA_ptr create_POA (
 146 <br>&nbsp;&nbsp;&nbsp; PortableServer::POA_ptr parent,&nbsp;&nbsp;&nbsp;
 147 /// Non RT POA parent
 148 <br>&nbsp;&nbsp;&nbsp; const char * adapter_name,&nbsp;&nbsp;&nbsp; ///
 149 Name for the POA
 150 <br>&nbsp;&nbsp;&nbsp; PortableServer::POAManager_ptr a_POAManager,&nbsp;
 151 /// Manager for the POA
 152 <br>&nbsp;&nbsp;&nbsp; const CORBA::PolicyList &amp; policies&nbsp; ///
 153 List of non RT policies
 154 <br>&nbsp;&nbsp;&nbsp; ACE_ENV_ARG_DECL)
 155 <br>&nbsp; ACE_THROW_SPEC ((
 156 <br>&nbsp;&nbsp;&nbsp; CORBA::SystemException
 157 <br>&nbsp;&nbsp;&nbsp; , PortableServer::POA::AdapterAlreadyExists
 158 <br>&nbsp;&nbsp;&nbsp; , PortableServer::POA::InvalidPolicy
 159 <br>&nbsp; ));
 160 <br>&nbsp;
 161 <p>Once a RT POA has been created, schedule_object is called to store CORBA
 162 Object references (key) with a name (value) in an ACE_MAP.&nbsp; An
 163 <br>RTCosScheduling::UnknownName exception is thrown if the schedule_object
 164 name parameter is not found in the resource map (i.e. it was not in the
 165 <br>config file.) The schedule_object method is declared as:
 166 <p>&nbsp; virtual void schedule_object (
 167 <br>&nbsp;&nbsp;&nbsp; CORBA::Object_ptr obj,&nbsp; /// A CORBA object
 168 reference
 169 <br>&nbsp;&nbsp;&nbsp; const char * name&nbsp;&nbsp;&nbsp; /// Name to
 170 associate with obj
 171 <br>&nbsp;&nbsp;&nbsp; ACE_ENV_ARG_DECL)
 172 <br>&nbsp; ACE_THROW_SPEC ((
 173 <br>&nbsp;&nbsp;&nbsp; CORBA::SystemException
 174 <br>&nbsp;&nbsp;&nbsp; , RTCosScheduling::UnknownName
 175 <br>&nbsp; ));
 176 <br>&nbsp;
 177 <p>Once all objects that will receive client requests have been scheduled
 178 using schedule_object, clients are free to make calls on those objects.
 179 The scheduling service interceptors catch these calls and perform the necessary
 180 priority ceiling control measures to ensure that the calls are executed
 181 in the appropriate order.&nbsp; The ServerScheduler_Interceptor receive_request
 182 method intercepts all incoming request immediately since it is set to run
 183 at RTCORBA::maxPriority (the highest priority on the server OS).&nbsp;
 184 It then gets the client priority sent in the service context as well as
 185 the resource ceiling for the object and the base priority for the server.
 186 <br>Initial tests indicate that the receive_request interceptor takes around
 187 0.002 seconds to complete on an Intel 3.0 GHz processor.
 188 <p>Given these values it is able to use the Multiprocessor Priority Ceiling
 189 Protocol to schedule execution on the server to handle the request.&nbsp;
 190 MPCP schedules all global critical sections at a higher priority than tasks
 191 on the local processor by adding the client priority to the base priority
 192 of the servant, then adding the resource ceiling of the resource to the
 193 base priority to find the appropriate priority ceiling. For more information
 194 about MPCP, please refer to the book "Real Time Systems", By Jane Liu (2000).
 195 <p>Please not that the locking mechanisms are stored in shared memory on
 196 the server.&nbsp; This means that the locks cannot be stored in linked
 197 lists and are therefore manipulated using memory offsets.&nbsp; The total
 198 number of locks that may be stored in shared memory is currently set at
 199 1024.
 200 <p>When remote execution is complete the send_reply interceptor resets
 201 the thread to listen at RTCORBA::maxPriority and removes the task form
 202 the Invocation
 203 <br>list.&nbsp; Initial test indicate that the send_reply interceptor takes
 204 0.000075 seconds to complete on an Intel 3.0 GHz processor.
 205 <br>&nbsp;
 206 <h3>
 207 Scheduling Service Config File:</h3>
 208 The scheduling service config file holds the information necessary to schedule
 209 the system.&nbsp; Task and resource ceiling information is stored for each
 210 of the nodes as follows:
 211 <p>Node 1&nbsp;&nbsp;&nbsp; /// The node name is 1
 212 <p>Resources:
 213 <br>BP&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 6000&nbsp; /// The base priority for
 214 the resource
 215 <br>Server1 1000&nbsp;&nbsp; /// A list of resources and their priority
 216 ceiling
 217 <br>Server2 2000
 218 <br>END&nbsp;&nbsp;&nbsp; /// The end of the resource list
 219 <p>Tasks:&nbsp;&nbsp;&nbsp; /// A list of tasks that will execute on the
 220 node
 221 <br>Client1 1000
 222 <br>Client2 3000
 223 <br>Client3 5000
 224 <br>END&nbsp;&nbsp;&nbsp; /// The end of the task list.
 225 <p>Please note that these associations are tab delimited.&nbsp; Please
 226 do not include comments in the scheduling service config file.&nbsp; The
 227 priorities associated
 228 <br>with each task and resource are considered to be CORBA priorities,
 229 and will be mapped to local OS level priorities using the Linear Mapping
 230 <br>model.&nbsp; Per the OMG RT CORBA spec, CORBA priorities have a valid
 231 range up to 32767, where a larger value indicates a higher priority.&nbsp;
 232 The current
 233 <br>config file assumes that the Multiprocessor Priority Ceiling Protocol
 234 is used.
 235 <br>&nbsp;
 236 <h3>
 237 Known Issues:</h3>
 238 TAO does not currently support request buffering, and there are no immediate
 239 plans to do so.&nbsp; Consequently, the RT CORBA 1.0 Scheduling Service
 240 is
 241 <br>limited in that it will only function properly in systems that do not
 242 require request buffering on the servant side.
 243 <p>There is a bug in TAO in which mapped priorities are mapped a second
 244 time when using Client Propagated Priority Ceiling Protocol.&nbsp; This
 245 in effect
 246 <br>lowers the priority that the servant receives.&nbsp; This happens to
 247 each priority, so there should be no effect on the system.
 248 <p>The config file assumes CORBA priorities in the range of 0 to 32767.
 249 The Linear Priority Mapping Manager will map these to valid local OS priorities.&nbsp;
 250 Take care though, in determining the priority range in the config file,
 251 as low numbers or numbers very close in value may produce priority inversion
 252 and other issues.&nbsp; For example, if the CORBA priorities used for three
 253 tasks are 100 200 300, these will all map to OS priority 1 in on some real
 254 time Linux systems. Please take this into
 255 <br>account when determining the CORBA priority range to use.
 256 <p>The 1.0 Scheduling service currently works with one orb and one POA.
 257 If someone tries to install more than one scheduling service (client or
 258 server side) on a single POA, then it should not add a second interceptor.&nbsp;
 259 Please use a single scheduling service per POA.&nbsp; Furthermore, there
 260 is a bug when more than one orb is created, an invalid policy exception
 261 is thrown during the second call to create_POA.&nbsp; This bug is actively
 262 being investigated.&nbsp; In the meantime please use the scheduling service
 263 with one ORB.
 264 <br>&nbsp;
 265 <h3>
 266 Future Enhancements:</h3>
 267 ACE_XML
 268 <br>The current RT CORBA 1.0 Scheduling Service uses a private method to
 269 read the config file.&nbsp; This will soon be replaced with a new XML based
 270 config
 271 <br>file using ACE_XML to parse the config file.
 272 <p>Priority Lanes
 273 <br>Although not currently implemented, Priority Lanes and Thread Borrowing
 274 may increase performance as they would help to prevent lower priority tasks
 275 from exhausting all threads.&nbsp; This is considered a possible future
 276 enhancement.
 277 <p>Client Interceptor
 278 <br>A client interceptor that sends a flag to notify the server interceptor
 279 if schedule_activity() was used to set the client priority.&nbsp; If schedule_activity()
 280 was not used, then the server should probably not try and schedule server
 281 execution using MPCP.&nbsp; Doing so adds competition to other method calls
 282 by other client requests that were scheduled with schedule_activity().
 283 <br>&nbsp;
 284 <h3>
 285 References</h3>
 286 The Object Management Group, Real Time CORBA 1.0 Specification, www.omg.org
 287 <br>Liu, Jane, Real Time Systems, Prentice Hall, 2000
 288 <br>&nbsp;
 289 </body>
 290 </html>