Building Custom Clients

This page describes the components that make up what may become the next method of building clients for Weave.

The current Weave 2.x desktop client is developed using the Sencha Ext.js framework, 2.x in Weave 2.4 and 3.x in Weave 2.5, and as part of the migration to newer versions of the Sencha libraries, including Ext.js 4.x and later and Sencha Touch 2.x and later, a new method of providing the client content to browsers is required.

The new method of providing client content to the browser should address some or all of the following:

• Provisioning of third party content
  • Currently there is no standard method of including third party JavaScript libraries in Weave
    • This implementation should provide a way to package and publish third party JavaScript libraries
  • Currently some of the third party libraries that Weave uses must be changed before that can be published, e.g. paths to images must be changed
    • This implementation should minimise the customisation required to third party libraries before they can be used
  • Currently there is no standard method of versioning third party libraries
    • This implementation should provide a way to publish multiple versions of third party libraries
  • Currently there is no standard method of publishing third party components for use with the Sencha frameworks
    • This implementation should provide a way to package and publish components in a standard manner
• Provisioning of Weave client content
  • Currently the JavaScript code that makes up the Weave client has minimal dependency analysis, i.e. it uses a pre-defined list of folder names to order content, but there is no way to specify dependencies within those folders
    • Code in newer versions of the Sencha frameworks are more structured in the way the specify dependencies, this can be leveraged by Weave to ensure full dependency analysis is performed on the code supplied to the browser
• Separation of JavaScript code
  • Currently code from any bundle installed in a Weave instance will be pushed to the client when it starts, regardless of whether the code is used or not, e.g. if the Pathway bundle is installed every client that's started includes the Pathway JavaScript code, even if the client is not configured to expose the Pathway components.
    • This implementation should provide a way to separate code into different packages that can be included only if required
• Provisioning of debug versus production code
  • There is currently no standard method of supplying different development and production versions of code to the client, e.g. minified code versus commented code
    • This implementation should provided a means of publishing both development versions and production versions of JavaScript code
• Provisioning different types of clients
  • Currently there is only one client type that can be easily published, i.e. there is only one Weave client, all be it a very customisable one
    • This implementation should allow for the creation of other client types, like the Admin UI for example
• Embedding within other web pages
  • Currently it's not easy to embed the Weave client directly within another web page
    • This implementation should help make it possible to embed Weave resources within another page, e.g. within a div element on a page
    • Note that the primary issue to be addressed here is the inclusion of Weave JavaScript code and CSS styles into another web page, it does not necessarily imply the ability of the actual Weave components to be embedded in the page, that may require changing the code of the components, that is, this update is primarily intended to address the provisioning of the code to the client, not the code itself.
    • This should be done by providing "standard" url's for retrieving Weave code and resources.

Provisioning of Third Party Content

Currently there are three ways that third party JavaScript libraries are pushed to the browser:

1. Third party libraries that Weave depends upon just to run are published by the com.cohga.client.weave bundle (note that some of the other resources for those libraries are published by the com.cohga.client.weave.main bundle), and these are directly included by the Weave client startup code when the browser loads.
2. Libraries embedded in client bundles, this code is currently extracted from the bundle and appended to a giant script sent to the browser after the third party libraries listed above.
3. Dynamically loaded JavaScript code, this code is downloaded on the fly when it's first required.

There are a number of issues with this:

1. The com.cohga.client.weave and com.cohga.client.weave.main bundles are too dependant upon each other and have become large an unwieldy
   1. JavaScript is in one bundle CSS resources in another.
   2. The com.cohga.client.weave bundle performs too many tasks. Currently this is the one bundle that's responsible for the entire loading of the client (with the com.cohga.client.weave.main providing primarily core Weave code and resources), which isn't necessarily a bad thing, but it also performs other function it does not need to, e.g. being the repository for the core libraries, providing the default "index" page.
2. Updates to the libraries are unnecessary complex
   1. A minor bug fix to one of the core libraries could mean that one or both of the com.cohga.client.weave and com.cohga.client.weave.main bundles needs to be updated.
   2. Some of the third party libraries must be changed before they can be published
   3. Change to Weave code that uses a third party library embedded in the same bundle, or vis-versa, an update to the third party library in the bundle, requires that both things be updated when they don't need to be, i.e. the Weave code that uses a third party library should not require the third party library to be updated if it hasn't changed.
3. The release of an update to a third party library may break an existing clients
   1. In certain circumstances it may be advantageous to tie a client to a well tested and consistent version of a library, rather than automatically updating it to the latest version when a semi-major update is applied, for example an update of a library from 4.2 to 4.3 may introduce new functionality, but it also may introduce new bugs, so it would be useful to specify that a client only uses the latest 4.2.x version of a library. Initially this may primarily be of use in customised clients, or when Weave is embedded in other applications.

The proposed solution to this is to package third party libraries into their own bundles, which must be individually included into the client as (and when) required. Note that at the moment we're just addressing how the code and resources will be published, not how the client will retrieve them.

The new implementation allows packaging of the libraries into their own bundles so they can be updated independently of any code that uses them and the bundle can include multiple versions of the third party library. Additionally it minimises the changes required to the libraries, with currently one exception being third party Ext.js components that do not use a distinct package name, e.g. two bundles, one providing Ext.ux.Grid and another providing Ext.ux.Table can cause issues because both libraries are trying to expose content at Ext.ux, these components will need to be changed before they can be published, for example by renaming them to Ext.ux.grid.Grid and Ext.ux.table.Table (which is actually more standard for Sencha components anyway).

Building a third party library bundle

Packaging a third party library into its own bundle has two requirements, firstly the content for the library is copied into the bundle at a specific location, and secondly a custom header is added to the MANIFEST.MF file.

The content for the library code and resources should be stored within a directory called 'content' within the bundle, additionally a sub-directory for each version of the library should be created under the content directory, this is where the code and resources are placed, generally exactly as it comes from the original download.

Third party library versions

The name of the "version" directory will specify the version number that the code and resources will be published as, and should follow the pattern X.Y.Z.Q, where X and Y are required and should be numbers representing the major and minor version of the library, Z is an optional (unless a qualifier is specified) release number for the library, and Q is an optional qualifier.

Some examples of valid library version numbers are:

• 2.2
• 2.2.1
• 2.2.1.rc5

These version numbers are not valid

• 2 – minor version number is always required
• 2.rc5 – minor version and release number required if a qualifier is specified, or alternatively "rc5" isn't a valid minor version number, it should be numeric
• 2.2.latest – release number required if a qualifier is specified, or alternatively "latest" isn't a valid release version number, it should be numeric

As an example assume that we wish to package up the jQuery library, and assume that the current version is 1.11.2, then the location of the jQuery library should be extracted to the directory /content/1.11.2/ within our new jQuery library bundle. Then down the track a new version of the jQuery library is published and by the time we notice it's up to version 2.0.3, so the content for the new version should be extracted to the /content/2.0.3 directory, and the existing 1.11.2 version left alone. Further if additionally versions 1.11.3 and 2.1.4 of jQuery are release they would be extracted to /content/1.11.3 and /content/2.1.4 respectively.

In addition to the directory name there's another location in a bundle that's related to the version, and that's the actual OSGi bundle version, as set in the MANIFEST.MF. The bundle version should always reflect the version number of the latest release of the third party library contained within the bundle. So in our jQuery example above the Bundle-Version header in the MANIFEST.MF file would start out as 1.11.2, then be updated to 2.0.3, then finally to 2.1.4.

Finally, the version numbers that are used in the directory names are the full version numbers that the content is published as, so it's possible for a client to retrieve the jquery.js file using the path .../2.1.4/jquery.js (as well as .../2.0.3/jquery.js, .../1.11.3/jquery.js and .../1.11.2/jquery.js). But, Weave also published the "latest" versions of the libraries using abbreviated version numbers, for example the 2.1.4 version of the jquery.js file is also available using the path .../2.1/jquery.js or .../2/jquery.js, and the 2.0.3 version available at .../path/2.0/jquery.js, and finally the 1.11.3 release at .../1.11/jquery.js or .../1/jquery.js.

This allows a client to reference a preferred version of the third party library but still receive bug fix updates without having to change the client itself, this is assuming that the ignored portion of the version number signifies a bug fix and not a whole new version, but this is generally dependant upon the library itself. So for example having our client include .../2.1/jquery.js mean that it will automatically receive bug fixes to the jQuery library if it's updated to 2.1.5. This is assuming the change from 2.1.4 to 2.1.5 signifies a bug fix release, it could be that a library used just X.Y version number and a change from 2.1 to 2.2 signifies a bug fix, and a change from 2.1 to 3.0 signifies new functionality, in which case you'd use .../2/filename.js to include the library.

Finally, every library is also published using "latest" as the version number, so regardless of the version number the newest version of the library will be available using "latest" rather than the version number, so for our jquery example version 2.1.4 of jquery would also be available at .../latest/jquery.js.

Third party library paths

I mentioned earlier that there are two requirements to publish a third party library, extracting the content to the correct location, and adding a custom header to the MANIFEST.MF file. The purpose of the custom header in the MANIFEST.MF file it to provide the base portion of the path that the library content will be published at. In the examples in the last section I presented the locations of the files as .../2.1/jquery.js for example, this header specifies what will be used for the ... portion of that path.

The name of the header to include in MANIFEST.MF is X-Weave-Library and should specify a unique base path for the content, for example with the jQuery example it could be to to /jquery

X-Weave-Library: /jquery

Weave will then publish the /content directory in our jQuery example bundle at /jquery, so the jquery.js file for version 2.1.4 would be available at /jquery/2.1.4/jquery.js (plus /jquery/2.1/jquery.js, /jquery/2/jquery.js and /jquery/latest/jquery.js)

Third party library content

So far we've just made a passing reference to the actual content of the third party library that gets extracted into the bundle, but it's important to understand what has to go into this directory, and this largely depends upon the library itself.

For the jQuery example it's pretty straight forward as there's really only two files that would be extracted into the bundle, the compressed and the uncompressed versions of the jQuery JavaScript files, almost exactly as they're downloaded from the jQuery web site. I say almost because jQuery, like a number of libraries, include the version number in the file name, so for example if I were to download the latest version of uncompressed jQuery library right now I'd get a file called jquery-2.1.4.js, and if I were to just include that file as it was into the bundle, in the /content/2.1.4/ directory, the client would need t reference it as /jquery/2.1.4/jquery-2.1.4.js, which means that if the client just wants the latest version, or even the latest 2.1 version, then they'd have to ask for /jquery/latest/jquery-2.1.4.js or /jquery/2.1/jquery-2.1.4.js, and what happens when 2.1.5 is release? Now the client needs to be updated to ask for /jquery/2.1/jquery-2.1.5.js, defeating our whole mechanism of exposing multiple versions of the same library.

What this means is just that the jquery-2.1.4.js file, and jquery-2.1.5.js file after it, should be renamed to jquery.js once it's been extracted into the bundle. The same for the compressed version, jquery-2.1.4.min.js, which should be renamed to jquery-min.js, or something similar.

Note that there is currently no standard for how these files should be named, either from the original third parties themselves, or if they're renamed to confirm to the requirements of Weave. This may have to be addressed eventually so that client's can reference a library by name, and not have to search for the correct file name to include the library, especially when we're going to build the client web page automatically rather than hand crafting the HTML code, which at this point is time is how these clients are created.