Tech spec

Introduction

This document represents the requirements and proposed architecture and implementation to transform the Scenario Editor from its current state to a database driven and user role based web application.

Requirements

1. Browser Support

• Chrome (latest)
• Safari (latest)
• Firefox (latest)
• IE 9 or earlier?

2. Full Functionality of Network Editor

3. Ability to Create, Retrieve, Update and Delete Project and Scenario Elements to/from DB.

• Project and dependencies (with the dependencies being retrieve only)
• Scenario and dependencies (with the dependencies being retrieve only)
• Network and dependencies (nodes and links)
• Node List and dependencies & Individual node
• Link List and dependencies & Individual link
• Controller Set and dependencies & Individual controller
• Event Set and dependencies & Individual event
• Sensor Set and dependencies & Individual sensor
• Split Ratio Set and dependencies & Individual split ratio profile
• Demand Set and dependencies & Individual demand profile
• Fundamental Diagram Set and dependencies & individual FD profile
• Initial Density Set and dependencies
• Downstream Boundary Capacity Profile Set and dependencies & Individual DBC profile

4. Ensure Scenario State is preserved once Estimation of Simulation has been run on it.

5. Ability to read in, visualized and save both Google map or Navteq map geometries

• All links geometries should be stored in the database SRID 8307
• Links must be converted to Google Map API shapes for visualization.

6. User Authentication and read and write permission based on user roles defined by:

• Project and Scenario Elements
• Within the context of a particular project

7. Ensure Scenario and Project element changes are not overwritten inadvertently

• User must have latest version of scenario elements loaded before they can save their changes (keep track of modified timestamp)

8. Aid or Prevent Users from editing the same scenario or projects elements at once

• Either provide interface to track loads and changes on scenario elements changed by another user so user knows instantly who is working with what scenario elements so they can reload OR lock scenario/project elements when they are loaded for edit

9. Validation

• Validate Scenario and Project Elements on retrieval
• Validate Scenario and Projects Elements on save

10. All Units must be converted to SI for DB storage and then to display preference

11. Automated client and server side testing and manual test plan

Use Cases

Use Case #1 : Log In

Use Case #2 : Load Scenario

Use Case #3 : Create New Scenario

Use Case #4 : Create New Network in Scenario and Save

Use Case #5 : Edit Network Structure in Scenario ( Geometry Changes ) and Save

Use Case #6 : Edit Network Elements ( Change in Link/Node Browsers ) and Save

Use Case #7 : Create New Project wide Set Element ( Controller, Event, or Sensor )

Use Case #8: Edit Project Wide Set Elements ( Controller, Event, or Sensor )

Use Case #9 : Edit Network Structure and Elements and then Save

Use Case #10 : Edit Network Structure, Elements, Project Wide Set Elements and then Save

Use Case #11 : Edit Project wide Sets linked to current Scenario ( FD, Demand, Split Ratio )

System Architecture Overview

figure 1: Overview of Scenario Editor data flow.

The scenario client is based on backbone.js (see figure 1). Backbone is a javascript library designed for presenting the information contained in the models (the Scenario Elements) in the browser via view templates. Collections are essentially just lists of models. When a model or collections is changed events then propagate to the views, which re-render the appropriate templates. Likewise when a view corresponding to a particular model or collection is changed, events handle storing new information within the model or collection. One big benefit of backbone is that models and collections can be easily used to fire AJAX server requests to seamlessly pass the current representation of the scenario and project models to the server.

The scenario editor is designed to pass JSON representations of scenario and project models to and from the client via a RESTful API. A RESTful API was chosen since backbone is designed to work with this type of interface and REST simplifies the requests by breaking them all into essentially being an action on a resource which is mapped to a particular URL. The actions are essentially the HTTP request verbs POST, GET, PUT and DELETE which map to create, retrieve, update, delete functionality. Resources are a particular scenario or project element, which have an associated URL pattern. So a basic request to retrieve the scenario object with id 1 within project id 1 would look like this:

GET http://scenario-editor-url/project/1/scenario/1

REST API’s are also stateless which means the server should store no user session information and that hypermedia on the client should drive the application state. As a result each request must also pass user credentials information. Since we are going to be deploying the Scenario Editor through SSL socket (HTTPS), which encrypts requests and responses, we will for now not encrypt usernames and passwords within the code. Instead they will be encoded in Base64 and decoded on the server.

A resource is passed from a server to the client and vice-versa via a JSON representation of the resource. On scenario retrieval from the database to the client, a JSON representation of it is obtained by querying the scenario DB and constructing a model element representation, which is serialized to JSON (figure 1). On a scenario saved event within the client scenario editor a JSON representation of the scenario is sent to the server as proper scenario element resources which construct the appropriate model elements and call the scenario database access layer to save these changes within the DB.

Database Access

The biggest challenge in implementing the scenario editor is to ensure changes to scenario element resources on the client get mapped to the appropriate database queries/stored procedures, so that the client state of a scenario is accurately represented in the database after a save event. Because scenarios are represented in a relational database, edits which span across high level scenario and project elements are difficult to map to appropriate insert, update and delete queries.

Design Approach #1 – Map Scenario Changes to appropriate SQL

One approach is to keep track of scenario changes within the client and pass back a representation of the scenario with the appropriate changes noted. For instance looking at use case #10 where a user edits some network elements like adding a new link, then editing it’s name and then creating a new Project wide profile for it, say adding a new fundamental diagram profile and capturing this all in one save event is a challenge. To transfer these edits in the database we would need a way to do the following:

1. Keep track of all changes to the scenario and map them accordingly to insert, update and deletes.

2. For new elements (ie. links) their id’s must be passed to other edits which reference them.

3. Wrap all of these changes within one transaction to ensure no partial saves occur. All or none of the edits should be saved.

4. When an element is deleted ensure all of its dependencies are removed, or can be removed. For instance if a link is removed all of it’s associated fundamental diagram profiles should be removed to prevent orphaned records.

The result of this that it becomes very challenging to save elements within some generic scenario database API that allows scenario and project wide save events.

So to simplify this we could abandon the notion of a scenario wide save event. Instead saves could be broken into sub-scenario elements. This will result in a re-design of the current scenario and will pose some UI design challenges. To make this work save events should be broken into network geometry changes (adding, deleting or moving a new link or node), Updating node or link attributes and Project wide sets. There should be no editing allowed of more than 1 of the above categories between save events. This would allow for a more simplified database API, simplifying the need to keep track of large scale changes and referencing new scenario element id’s which are needed for later changes.

Design Approach #2 – Completely Re-write Scenario Elements on Save Event

Another approach is to completely rewrite the scenario and changed project elements after a save event. This is currently how the scenario database access layer is designed at least on the network level. This effectively is like using a key value store but within a relational database since the entire scenario gets deleted and recreated just as if the entire scenario was getting re-written as a serialized object file on each save. The upside to this approach is that implementing code to track differences in the scenario editor on a save event is much simpler since we are doing a complete re-write and save. This however is more computationally costly at the database level.

To make this work we would need a way to do the following

1. Ensure speedy saves on large scenario and networks occur, otherwise AJAX save request will time out.

2. Since each save completely wipes out the old scenario, extra validation would be needed to ensure the scenario object representation is accurately expressed on both client and server sides.

3. Dependencies outside of the scenario (ie. project wide sets) have to be maintained. For instance if a link is removed all of it’s associated fundamental diagram profiles should be removed to prevent orphaned records.

Preserve run Scenario in DB

Syncing Oracle and Google Map Geometries

User Roles

Concurrency

Data Validation

SI Unit Conversions

Testing

Conclusion