gears/sdk/architecture.html

   1 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "DTD/xhtml1-transitional.dtd">
   2
   3 <!--
   4 Copyright 2007, Google Inc.
   5
   6 Redistribution and use in source and binary forms, with or without
   7 modification, are permitted provided that the following conditions are met:
   8
   9  1. Redistributions of source code must retain the above copyright notice,
  10     this list of conditions and the following disclaimer.
  11  2. Redistributions in binary form must reproduce the above copyright notice,
  12     this list of conditions and the following disclaimer in the documentation
  13     and/or other materials provided with the distribution.
  14  3. Neither the name of Google Inc. nor the names of its contributors may be
  15     used to endorse or promote products derived from this software without
  16     specific prior written permission.
  17
  18 THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR IMPLIED
  19 WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
  20 MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO
  21 EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
  22 SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
  23 PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS;
  24 OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
  25 WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
  26 OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
  27 ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
  28 -->
  29
  30 <html>
  31   <head>
  32     <title>Architecture</title>
  33     <link rel="stylesheet" type="text/css" href="gears.css" />
  34   </head>
  35
  36 <body>
  37
  38 <h1>Architecture</h1>
  39
  40 <div id="pagecontent">
  41
  42 <p> During development of Gears, we experimented with many different architectures for offline-enabled web applications. In this document we  briefly look at some of them and explore their advantages and disadvantages.</p>
  43
  44 <p>As we experimented, we found some common themes. All the offline-enabled applications we created had the following design issues that needed to be addressed: </p>
  45 <ul>
  46         <li>isolating the data layer</li>
  47         <li>deciding which features to implement offline  (connection strategy)</li>
  48         <li>deciding on the app's modality</li>
  49         <li>implementing data synchronization</li>
  50 </ul>
  51
  52 <h1>Isolating the Data Layer</h1>
  53
  54 <p> In most web applications today there is no real data layer. </p>
  55 <p>
  56         <center>
  57         <img src="resources/figure_1.jpg" alt="No data layer" style="border:none;"><br>
  58         <b>Figure: No data layer</b>
  59         </center>
  60 </p>
  61
  62 <p>AJAX calls originate throughout the code, without any single clean communication layer. This is usually fine for AJAX applications because there is only one data source: the server. In effect, the API that the server exposes to AJAX acts as a data layer. </p>
  63 <h2>Architecture with a data layer</h2>
  64 <p><b><i>In general, isolating the data layer is a good first step.</i></b></p>
  65
  66 <p>When you add a local datastore to your application,  you will have a single place through which all data storage and retrieval requests  pass.</p>
  67 <p>
  68         <center>
  69         <img src="resources/figure_2.jpg" alt="Data Layer" style="border:none;"><br>
  70         <b>Figure: Data Layer</b>
  71         </center>
  72 </p>
  73
  74 <p>For example, if your AJAX application  issues a JSON request directly to the server to get all the accounts for a user, you might change this to instead ask an intermediate object for all the accounts for the user. This object could then decide whether to retrieve the data from the server, the local store, or some combination of both. Similarly, when the application wants to update the user's accounts, the app does so by calling the intermediate object. The intermediate object can then decide whether to write the data locally, whether to send the data to the server, and it can schedule synchronization.</p>
  75
  76 <p>You can think of this intermediate object as a data switch layer that implements the same interface as the data layer. As a first step, you can make the data switch forward all your calls to the data layer that interacts with the server. This step is useful since it is  the code path that is followed when Google Gears is not installed or if the user doesn't want to enable the application to work offline. Note that a data switch is not strictly necessary (for example GearPad does not have a data switch layer).</p>
  77
  78 <p>
  79         <center>
  80         <img src="resources/figure_3.jpg" alt="Data Switch Layer" style="border:none;"><br>
  81         <b>Figure: Data Switch Layer</b>
  82         </center>
  83 </p>
  84
  85
  86 <p>The next step, as shown in the figure below, is to create a new local data layer that uses a Google Gears database instead of going to the web server for data. It's simpler if this data layer has the same interface as the existing data layer used to communicate with the server. If the interface is different then some translation needs to be done and you might as well do that inside this data layer.</p>
  87
  88 <p>To test this step, you can set the data switch layer to talk to this new (local) data layer. You might want to pre-populate the database to make things easier to test.</p>
  89
  90 <p>
  91         <center>
  92         <img src="resources/figure_4.jpg" alt="Local Data Layer" style="border:none;"><br>
  93         <b>Figure: Local Data Layer</b>
  94         </center>
  95 </p>
  96
  97 <h2>Architecture without a data layer</h2>
  98 <p>If the application is not structured with a data layer and adding a data layer is not an option, it is still possible to isolate the data layer by intercepting all the calls to the web server just before they are sent. For example, you could intercept a form submit (listen to the submit event) and decide if the application should use the local data store or the data on the server. </p>
  99 <p>Implementing this approach   involves finding all functions and methods that send requests to the server, and rerouting them. The challenge is that this method requires a lot of extra work, like parsing URLs, iterating over forms as well as generating the same result as the server would. In practice you end up reimplementing large parts of the web server on the client side. Regardless, this can be a viable option for existing AJAX applications that can't be otherwise rearchitected.</p>
 100 <h1>Feature Availability Offline</h1>
 101 <p>For practical reasons, every feature of an application may not become a feature that's available offline. You need to  choose which features you want to support locally and implement the logic that decides when to use the local store and when to connect to the server. We call this the &quot;connection strategy.&quot; </p>
 102 <p>You might think that you would always want to use the local store since it is faster. However,  there are many practical reasons why you may  want or need to  access the data on server instead. For instance: </p>
 103 <ul><li>Data may be so transient in nature that it makes no sense to cache it. <br />
 104     For example, an application providing real-time stock quotes would not benefit from serving old stock quotes.</li>
 105   <li>Some application's data  makes sense only while online. <br />
 106     As an extreme example, an Instant Messaging application makes sense only while connected. </li>
 107   <li>The app may choose to  store  only the most frequently accessed data.<br />
 108 For example, if the user can set a preference that changes the language of the application, this preference change might not be supported offline because the cost of implementing this  offline feature is not worth the benefit for  a preference that is seldom changed.</li>
 109   <li>Computational and/or disk space requirements make it unfeasible to recreate the feature offline.<br />
 110     For example, if the feature requires huge amounts of data to be useful, beyond a reasonable amount for a personal computer. </li>
 111 </ul>
 112 <p>Typically, the optimal solution is to use the local store as much as possible, since it's usually  faster than a remote connection. However, the more work an application does locally, the more  code you need to write to implement the feature locally and  to synchronize the corresponding data. There is a cost/benefit tradeoff to consider, and some features may not be worthwhile to support locally.</p>
 113
 114 <h1>Modality</h1>
 115 <p>One fundamental question that all offline-enabled applications must answer early is that of &quot;modality&quot;.</p>
 116 <ul>
 117   <li> Modal  applications have distinct offline versus online modes, usually indicated through some change in the user interface. The user is made aware of the state and  participates in switching states in some manner. </li>
 118   <li>Modeless applications  attempt to  transition seamlessly between online and offline states, without significant UI changes. The user does not need to participate in switching states, which the application does automatically. </li>
 119   </ul>
 120 <h2>Modal</h2>
 121 <p>In a modal application, when the application is online it communicates with the server. When it's offline, it uses the local store. Data must be synchronized  when the app switches between modes. </p>
 122 <p>The advantage of making a modal application is that it's relatively simple to implement and therefore a reasonable way to bootstrap the application to function offline. </p>
 123 <p>The disadvantages are:</p>
 124 <ul>
 125   <li>The user must remember to switch modes. If they forget, they will either not have the data they need when offline, or will work online in an unintentionally disconnected mode.</li>
 126   <li>If the network connection is intermittent, the user will either need to choose one of the settings or will have to constantly switch between modes as the connection comes and goes. </li>
 127   <li>Since the local store is not always up-to-date, it can't be used to improve the application's responsiveness when connected to the server.</li>
 128   </ul>
 129 <h2>Modeless</h2>
 130 <p>In a modeless application, the application  works with an assumption that it is  offline, or that it can lose the network connection at any time. The app  uses the local store as much as possible, and does continuous, small data syncs in the background when the server is available. Data synchronization is also done when coming back online.</p>
 131 <p>The advantages of  modeless applications are:</p>
 132 <ul>
 133   <li>A better user experience. The user does not have to be aware of network connectivity or switching states.</li>
 134   <li>The application works smoothly even with intermittent network connections.</li>
 135   <li>Since the local store is kept up-to-date, it can be used to optimize the server connection.</li>
 136   </ul>
 137 <p>The disadvantages of modeless applications are:</p>
 138 <ul>
 139   <li>It is  more difficult to implement. </li>
 140   <li>Care must be taken to avoid letting the synchronization background process   consume too many resources and  make the overall application feel sluggish. </li>
 141   <li>Testing the app can be more challenging, since synchronization logic occurs in the background and not in reaction to specific user actions. </li>
 142 </ul>
 143 <p>&nbsp;</p>
 144 <p>The sample application <a href="sample.html">Gearpad</a> is an example of a modeless offline application. It always writes to the local database, and then independently synchronizes the changes with the server. </p>
 145 <p>Google Reader is currently an example of a modal application. It has a distinct offline mode that a user must explicitly enable. In Reader's case, implementing a modal state was a pragmatic choice, as it was faster to implement and made it possible to release an early version of Google Reader with Gears. </p>
 146 <h1>Data Synchronization</h1>
 147 <p>No matter which connection and modality strategy you use, the data in the local database will get out of sync with the server data. For example, local data and server data get out of sync when:</p>
 148 <ul>
 149   <li>The user makes changes while offline</li>
 150   <li>Data  is shared and can be changed by external parties</li>
 151   <li>Data comes from an external source, such as a feed </li>
 152 </ul>
 153 <p>Resolving these differences so that the two stores are the same is called &quot;synchronization&quot;. There are many approaches to synchronization and none are perfect for all situations. The solution you ultimately  choose will likely be highly customized to your particular application.</p>
 154
 155 <p>Below are some general synchronization strategies.</p>
 156
 157 <h1>Manual Sync</h1>
 158 <p>The simplest solution to synchronization is what we call &quot;manual sync&quot;. It's manual because the user decides when to synchronize. It can be implemented simply by uploading all the old local data to the server, and then downloading a fresh copy from the server before going offline. </p>
 159 <p>Manual sync  requires that: </p>
 160 <ul>
 161   <li>The amount of data is small enough to download in a reasonable amount of time. </li>
 162   <li>The user explicitly indicates when he or she is going  offline, typically via a button in the user interface.</li>
 163   </ul>
 164 <p>The problems with this method and with the offline mode it creates, are:</p>
 165 <ul>
 166   <li>Users don't always know the state of their network connections. Internet connections may die unexpectedly or  be intermittent, for example, on a bus. </li>
 167   <li>Users may forget to synchronize before going offline.</li>
 168   </ul>
 169 <p>Manual sync can be a good way to get started as it is relatively easy to implement. However, it requires the user to have awareness and involvement in the synching process. </p>
 170 <h1>Background Sync</h1>
 171 <p>In a  &quot;background sync&quot;, the application continuously synchronizes the data between the local data store and the server. This can be implemented by pinging the server every once in a while or better yet, letting the server push or stream data to the client (this is called Comet in the Ajax lingo). </p>
 172 <p>The benefits of background synching are:</p>
 173 <ul>
 174   <li>Data is ready at all times, whenever the user chooses to go offline, or is accidentally disconnected.</li>
 175   <li>The performance is enhanced when using a  slow Internet connection. </li>
 176 </ul>
 177 <p>The downside is that the sync engine might consume resources or slow down the online experience with its background processing (if  it's not using the WorkerPool). Using WorkerPool the cost of synching is minimized and no longer affects the user's experience. </p>
 178 <p>
 179         <center>
 180         <img src="resources/figure_5.jpg" style="border:none;" alt="Background sync architecture"><br>
 181         <b>Figure: Background sync architecture</b>
 182         </center>
 183 </p>
 184
 185 <h1>Conclusion</h1>
 186 <p>There are many design choices to be made along the way to enabling an application to work offline. This document reviewed a few of the possible choices for the common design issues that arise. The decisions made for an application will need to reflect the user experience you want to achieve as well as the application's limitations and goals. </p>
 187 </div>
 188
 189
 190 </div>
 191 </body>
 192 </html>