| Note copyright and disclaimer restrictions. | © Wm James 1999-2002  |   Questions?  |  Updated 02/01/04 |
| Cite: "James, Wm. (1999). 05-661,05-662 Web site. U. of Guelph, Sch. of Eng'rg. www.eos.uoguelph.ca/ webfiles/james"  | 

05-662 Water pollution control planning (for UCT students, CIV530Z  is a programme of individual study on a specialized topic - examination by report/s and possibly an oral) is a graduate engineering course, comprising the six even-numbered modules below: 2. philosophy underlying public water pollution; 4. methods of developing area-wide pollution control plans and sustainable use plans in Ontario and elsewhere; 6. introduction to BMPs and the SLAMM model; 8.  introduction to the WASP model; 10. Urban litter in drainage systems;  12. examination of quantitative and non-quantitative information in the context of planning. No field trips are planned for Jan-Apr 2000. More info is provided in module 0.   

Current modules in this website are for January to April 2002.   

module 3

GIS, data management, model complexity, catchment discretization and process disaggregration (PCSWMMGIS)

contents

Introduction
Engineering design and the development of deterministic models
Newtonian mechanics, pde's, fde's and computer programs
Spatial discretization
Temporal resolution
Averaging high-frequency processes
Model complexity
Data collection
GIS
Some urban water systems GIS concepts and terminology
Frequently asked questions
Assignment A3
Reading
Reminders

Introduction

Pedagogic note: This module is a merging of some key concepts in (a) modelling, (b) GIS, and (c) urban drainage infrastructure. You are required to read into these 3 disciplines, and summarise the main points of your interest in a web page. Learning objectives for this module include consideration of model complexity in terms of spatial and temporal discretization, and process dissaggregation. These concerns are closely tied to the development of input datafiles, and spatio-graphical data is the essence of urban water systems modelling. You will be expected to be au fait with the terminology and concepts. 

Design and development of deterministic models

• A model is a concept (or object) that is used to represent something else. It is reality simplified to a form we can comprehend. This is very likely the best way to design solutions to complicated problems with arbitrary 3-D geometry (such as natural topography).  A model helps us deal with complexity.
• If you know everything you would not need to model anything - you would be able to simply state the solution out aloud and it would be correct...
• If you had all the data that one could imagine, both for the AS-IS and all possible WHAT-IF or TO-BE scenarios, you could also look up a solution without recourse to a model...
• neither option is remotely plausible, and so models remain an essential method of approximation for engineering planning and design.
• Remember:
  • All models are wrong - but some may be useful [G. E. Box]
  • it's not enough to know simply when or how a model may be said to be useful - it's more important to know how reliable it is.. [WJ!]
• A math model has components that are equations, inequalities, functions, variables, parameters and constants, and can be programmed and the program executed (called a "run") and the resulting model tested for "sensitivity".
• It's useful to recall that a "problem" is "a difficult and doubtful matter requiring solution; it is something that is hard to understand, or accomplish, or deal with." (Oxford Dictionary of Current English).
• Steps involved in building a model are similar to those involved in design (or indeed, in scientific enquiry and critical thinking).
• They are:
  1. state the problem
  2. identify the significant factors
  3. estimate the accuracy required
  4. derive or obtain a suitable math description
  5. write the code
  6. evaluate the program
  7. collect data
  8. develop the AS-IS input dataset
  9. evaluate and optimize the model performance
  10. use it for inferring behaviour of various TO-BE scenarios/solutions
  11. select an optimum solution
  12. report and present the best solution and its uncertainty
• Design and model building are both iterative:
  E.g.
  1. state the problem
  2. formulate the model
  3. select model performance evaluation criteria
  4. select an objective function
  5. manipulate the model
  6. satisfied? If no - go back to 1; If yes - continue
  7. model several theoretical situations
  8. select best alternatives
  9. report

Newtonian mechanics, pde's, fde's and computer programs

• By Newtonian mechanics, I mean the technique of developing a mathematical relationship that is derived from one or both of the great Principles of Conservation (mass, and energy or momentum).
• Usually we consider an elementary parallelepiped such as a rectangular block, and consider for example fluxes in and out and change of the quantity within.
• Scales may vary widely, e.g. Lake Superior at one end to a rain barrel at the other.
• Time scale is related to the spatial scale, and number of processes involved may be large.
• Some say that key trick is to consider the simplest combination of processes.
• Let us apply the Principle of Conservation of Matter (continuity):

  In words: input - output = rate of change of mass

  In simplest math form (assumes homogenous substance): I - O = dS/dt

  Or (for arbitrary storage area): I - O = A(z).dz/dt

  where: dS/dz = A = f(z)

  In finite differences: 0.5(I₁+I₂) - 0.5(O₁+O₂) = 0.5(A₁+A₂)* (z₂-z₁)/dT

  The resulting simple algebraic expressions can usually be re-arranged to explicitly solve for an unknown of interest, and programmed, for instance in a spreadsheet. This is how our models are built.

• We have not touched on the huge subject of numerical analysis, which fits between the fde's and the executable program.
• The state of the model is given e.g. by the instantaneous lake levels - called the state variables.
• The initial conditions of the model are specified e.g. the starting depth of the water levels in each lake at the onset of the model run (simulation).
• If rain stopped forever, or fell at a constant rate forever, the models would reach a steady state.
• The boundary condition is given by e.g. the size of the outlets from each lake, like the capacity of the St. Lawrence River at Kingston.

Spatial discretization

• Consider a lake which has rain as input and streamflow as output, and whose water level changes from time to time.
• One could consider all of the Great Lakes acting as one tank (call this model GL1; here dx, dy are about 1000 km or more), or
• As a networked system of six lakes, linked sequentially:

  Lakes Superior and Michigan both flow into Lake Huron, which then flows into Lake StClair, which then flows into Lake Erie, which then flows into Lake Ontario, which finally outflows at Kingston. - call this model GL6; now dx, dy are now about 100 km.

• We say that GL6 is six times more complex than GL1.
• Now consider modelling several thousands of the pipes in your city (for me it's Guelph) alone (call this G1000; dx will now be about 100 m).
• Such a model would be 10³ times or more complex than GL1.
• BTW, such models also form a network which may be dendritic (tree-structured) like GL6 or looped.

Temporal resolution

• The higher the spatial resolution, the faster the frequency of the processes involved, and the finer the time resolution (dt) should be.
• In the Great Lakes system, response time of outflow is only slightly variable on a monthly basis. They respond to cycles of years, e.g. of wet weather. Individual thunderstorms have no measurable impact. We could set dt = 1 month.
• Small gutters and pipes in Guelph respond strongly to short thunderstorms, even if they last only 10 minutes. Here dt is about 1 minute.
• Flow in the pipe is a higher frequency process that must be modelled in G1000,and dt = 10 s.

Averaging and high-frequency processes

• Choosing a coarse spatial resolution controls to some extent the time step, through the speed of the processes involved.
• Choosing the timestep controls the processes that will be involved.
• E.g. little ripples on the surface of the water have a natural frequency of about 10 Hz. They would be averaged to zero if dt = 1 minute or more.
• A short sharp thunderstorm would have a negligible rain intensity if averaged out over a year.
• Coarse models consider only longer frequency processes such as evaporation.
• Models that require fewer input parameters are inherently less complex (the number of input variables is related to the complexity).

Model complexity

• A system is a representation of a situation, and comprises an assembly of elements arranged in an organised whole.
• Structure is the term used to describe how the component processes can be related to each other.
• An element is is the representation of a process described by a simple noun phrase and whose existence, informed observers agree, is worth assuming in order to gain insight. Behaviour of the process results in a change in a significant attribute.
• A relationship is said to exist between A and B if the behaviour of either is influenced by the other. Relationships may involve flow of materials, information or energy.
• All elements and processes are representations built on assumptions.
• Some attributes of the processes are called state variables, and their relationships may be plotted in state space.
• If the state variable map on a one-to-one or simple basis the system is said to be deterministic. If the system is complex and the system is fuzzy, it is said to be poorly structured. A system that comprises interacting processes that are difficult to appreciate as a whole has been defined as  a "mess".
• A "black box" lumps all the systems processes into a single transfer function, not based on underlying physics.
• If the system attains a steady-state or dynamic equilibrium the state is termed "homeostasis".
• The concept of the tendency for a system to move toward greater disorder is termed entropy.
• Component processes require specific input parameters, the more detailed the more the parameters required.
• Complexity arises when there is a large number of processes and interactions, when the relationships are non-linear, and where there are nonholonomic constraints (e.g. if a pump station in a drainage system switches on or off without regard to the system as a whole).
• We may think of model complexity as the product of the number of spatial sub-spaces modelled, and the number of processes modelled in each, and the number of input parameters required for each process.
• This is approximately and simply the total number of input parameters required in the input dataset.
• It does not include the input time series (e.g. rainfall or inflows) that drives the system model.
• A rain time series for our lake model may be 100 yr. long at 1 month timesteps, about 1.2x10³ timesteps.
• While the system model may have six component lakes each requiring just (say) ten parameters to describe their geometry.
• Such a system model may be said to have a complexity of 60. It is independent of the number of time steps.
• One of the difficulties that arises is that of attaining a parsimonious model.
• Recall however that the whole system is usually greater than the sum of the parts.
• If there are several system models that interact, the system that sits above it in a hierarchy of control, is called a "metasystem" and its control words constitute a "metalanguage". PCSWMM is an example.
• Physical systems are the simplest. Chemical systems are more complex since the physics must be correct. Biological (and human-medical) systems are yet more complex. Even more complex are animal behaviour (human: psychology) problems, and most complex in this hierarchy are ecology and sociological systems.
• Aesthetics, which may be defined as the many auditory and visual perceptions that accompany a certain feeling of value, is an example of a complex system.
• A "technique" is a precise, specific program of action that produces a standard result.
• A "methodology" lacks the precision of a technique but is a firm guide to action.
• A "philosophy" is a broad non-specific guideline for action.
• We will discuss all three types.

Data collection

• Data is required to prove, validate and calibrate your model.
• Data gathering is closely allied to model building.
• Both follow a procedure of discovery, and they should proceed in harmony.
• Data collection can be managed by conducting sensitivity analysis on your model.
• Data collection is often more expensive than modelling.
• Data if sufficient can be used to detect trends, calculate basic statistics, and estimate model performance.
• Faulty data often lead to wrong model interpretation, and wrong designs.
• It's essential to know the accuracy of your data.
• Strict quality controls are essential.
• Data should be analysed as it is gathered for:
  1. mean
  2. median
  3. variance
  4. correlations
• all can be easily computed in your spreadsheet.

GIS

Ideally this part of the course should be demonstrated by using a GIS package on-line. I use PCSWMM GIS in my class, as it provides a graphical plan view editor for quickly creating, editing and/or querying the physical entities of a SWMM model and their attributes, using any coordinate system. PCSWMM GIS acts as an independent Geographical Information System (GIS). However, PCSWMM GIS also supports linkages to existing GIS and/or Facilities/Information Management Systems (FMS/IMS), databases, spreadsheets and text files, with full SQL query support. Users can import from and export to the Runoff, Transport and Extran modules of SWMM, SWMM model results can be dynamically displayed as a layer and links are provided to other PCSWMM tools. The fllowing are four of the main attributes:

1. SWMM models
SWMM entities can include conveyances (conduits), nodes and subcatchments. Entities can be graphically created with simple mouse clicks and/or can be imported from existing SWMM files, many database formats, spreadsheets and other file types. Entity attributes are edited through linked tables and tools for the reduction of model complexity are provided with the Aggregation wizard. PCSWMM GIS features intelligent connectivity checking and reporting. Runoff, Transport, or Extran input data files can be updated with a single click.

2. Display background images
Most common map and image formats can be displayed as backgrounds to the SWMM entity layers. For example, you may wish to display multiple tiled TIFF images of a USGS topographic map, an AutoCAD DXF vector drawing file of city streets, and a layer containing objects representing the results of a point database query, all overlaid by your Extran and Runoff SWMM model layers. Supported formats include ArcView shape files, Autocad DXF and DWG files, MapInfo, and Microstation, as well as raster image support for TIFF, JPG, BMP and more (click here for a complete list).

3. Animate model results
PCSWMM GIS also features dynamic thermometer-style playback of computed Extran model results. In addition, PCSWMM GIS provides path and element selection for the display/analysis of computed results in other PCSWMM tools (e.g. the Dynamic Hydraulic Gradeline tool).

4. Using on-line help
Use the help file in conjunction with the on-line Getting Started video tutorials to get up and running quickly with PCSWMM GIS. The Contents dialog box should be used for navigating the topics of the help file (click on the Contents button in the upper-right corner of this help window). The Definitions section provide some help with terminology, the How-To topics step you through the various PCSWMM GIS procedures, and a Frequently Asked Questions (FAQ) topic is provided to answer some questions on the capabilities and approach of the program. For more information and updates, refer to CHI’s extensive website support for SWMM and PCSWMM GIS at http://www.chi.on.ca.

Urban water systems GIS concepts alphabetically

Active layer
The active, or selected, layer is indicated by a beveled edge around the layer entry in the Table of Contents. Only entities in the active layer can be selected, edited, or have their attributes displayed. Only the active layer properties can be edited. Active layers can be selected in the Table of Contents.

Aggregation
Aggregation is the process of simplifying a model by reducing the number of entities it contains. Through this process pathways are replaced by single, hydraulically-equivalent conduits. At the conclusion of this process, only the hydraulically significant entities remain. The aggregation wizard should be used for this process.

Aggregation Wizard
This dialog box allows a selected pathway to be aggregated into single, hydraulically-equivalent conduit. Attributes of the resulting conduit entity are computed from customizable equations specific to each attribute and these various aggregation configurations can be saved/recalled instantly. Manual overriding of the defined operations is also facilitated.

Attribute
A piece of information describing an entity. Attributes of a conduit might include length, cross-sectional shape, roughness, etc.

Attributes Sheet
Attributes of a selected entity can be viewed/edited in a pop-up list called the Attributes Sheet. This sheet can be expanded to show technical drawings illustrating each attribute, or collapsed to just display the attribute list. To view the Attributes Sheet, simply double-click on an entity.

Auto-numbering
New nodes and conduits created with the Add Conduit tool in the View window are assigned IDs based on the next available number. Starting numerical ID's are specified by the user in the Properties dialog box.

Background Layers
Background maps or images can be used to provide visual reference information when displaying SWMM layers. Each background image or map is connected and displayed through its own layer. Multiple background layers can be displayed at the same time, either tiled or overlaid. For example, you may wish to connect layers represented by many TIFF image tiles of a USGS topographic map, an AutoCAD DXF vector drawing file of city streets, and a layer containing objects representing the results of a point database query. Supported background layers include most of the common map and raster formats.

Default subcatchment shape
If no Easting and Northing coordinate pairs are assigned to a subcatchment, the subcatchment entity appears as a user-selected default shape located at the inlet node. If no inlet node entity exists, the subcatchment entity appears at coordinate position 0,0. The default shapes can be edited/created and the scale of the shape can be quickly adjusted. Default shape definitions are stored in subcatchment polygon files residing in the PCSWMM GIS application folder and can be easily selected. The number of coordinate pairs in the selected default subcatchment shape can affect program performance for large models. Subcatchment entities can also be represented by individual, real world, polygon shapes.

Dynamic Hydraulic Gradeline Tool (DHGL)
The DHGL tool can be used to display the profile of an Extran layer pathway and can playback the dynamic hydraulic gradeline results of the Extran simulation. The DHGL tool is part of the complete PCSWMM software package. For more information, please contact CHI.

Entity
A map representation of a geographical object. Entities can exist on either SWMM layers or background layers. Only entities on SWMM layers can be selected, edited, queried and have attributes There are three types of SWMM entities: nodes, conduits, and subcatchments.

External databases
Entities and attributes can be imported from external databases. This is done with the Import Wizard. Import configurations can be saved and quickly recalled for various external database formats.

Importing to PCSWMM GIS
The Import Wizard can import node/conduit data into an existing or newly created PCSWMM GIS database. Sources of data include:
-    External databases (including, but not limited to, the underlying database of a GIS/IMS),
-    Existing SWMM input files (Runoff, Transport or Extran), and
-    Map files (for importing nodal coordinates).
-    XYS files (for importing nodal coordinates).

The attributes imported from these sources are given in each section below. However, some importing rules apply to all sources:
-    If existing node/conduit entities have the same ID as the imported data, existing attributes (some or all, depending on the source) will be updated with the imported data.
-    If there is no matching existing ID for the imported data, a new entity (node/conduit) is created and given the imported attributes.
-    If there is no ID given (for an external database source only) in the imported data, a new unique ID is created by PCSWMM GIS.

Importing from an external database
Importing data from an external database involves matching node and/or conduit attributes to fields in an external database table or query. There are three basic steps:
1    Specify an external database file.
2    Specify a table or query from that database for importing node and/or conduit attributes. An optional SQL statement can also be used.
3    Match any or all of the entity (node/conduit) attributes to fields in the specified table/query.

Any or all attributes can be assigned fields, those attributes that are not assigned are handled as follows:
-    If a new entity is created (i.e. there is no existing entity with the same ID), default values are given to the unassigned attributes.
-    If an existing entity is updated (i.e. an existing entity has the same ID as the imported record), unassigned attributes are left with their original values (only assigned attributes are updated for existing entities).

An optional, user-defined SQL query statement may be utilized for importing nodes/conduits. This gives the user more control over the importing process. For more information on SQL, please see your database information resources.

Importing from a SWMM input file:
All Node and Conduit attributes can be imported from SWMM input files . The type of SWMM input file must match the selected SWMM layer (e.g. a Runoff input file can only be imported to a Runoff layer).

There is an option to import Easting and Northing (X,Y) attributes from SWMM input files. Although this data is not part of a standard SWMM input file, the SWMM engine ignores it if appended to the end of a data line. PCSWMM GIS imports and exports geospatial data to the following data lines:
-    Runoff input data file: Easting and Northing data appended on the conduit data lines (G1)
-    Transport input data file: Easting and Northing data appended on the type 19 (MH) non-conduit element data lines (E1)
-    Extran input data file: Easting and Northing data appended on the junction data lines (D1)

Importing from a Map file:
Only the following 3 node attributes can be imported from a Map file: ID, Easting, and Northing. The required Map file format is identical to the USEPA EPANET ver 1.x map file format.

Linkages to GIS
A powerful application of PCSWMM GIS is the linkage of a Geographic Information System (GIS) or Facilities Management System (FMS) to the USEPA StormWater Management Model (SWMM). This is accomplished through a three-step process:
1    Importing data from the GIS/FMS database,
2    Reducing the complexity and massaging the data into a useful model, and
3    Creating data file(s) for input to SWMM.

Importing data
Data can be imported from the underlying databases of many GIS systems. Internal PCSWMM GIS fields are matched to fields in tables or queries of the external database. PCSWMM GIS supports the full SQL query language for additional power and flexibility when importing.

Reducing the model complexity
Once the model data is in PCSWMM GIS, it can be edited graphically, through the local tables, and/or with help from the Aggregation Wizard.

Creating SWMM input
Conduit, node and subcatchment information in Runoff, Transport and/or Extran input data files can be updated quickly and easily from PCSWMM GIS

Information on using the Attributes Sheet
The Attributes Sheet provides a convenient way of viewing/editing the attributes of a single entity. To view the Attributes Sheet, simply double-click on an entity. The following information should help you maximize the use of this sheet:
-    Attribute information is updated whenever the following events occur: the Enter key is pressed, a different attribute is selected, and/or the Attribute sheet is closed.
-    In order to immediately see the view window results of editing some attributes (e.g. the Easting, Northing, Inlet, Node1, and Node2 attributes), the Attributes Sheet can be moved off to the side of the screen.
-    With the Attribute Sheet open, you can quickly jump to another entity’s attributes by pressing the F3 key.
-    For subcatchment entities, the Eastings and Northings Attribute Sheet text boxes contain a button, which opens the Coordinates Editor.
-    The Attributes Sheet can display technical drawings for each attribute. Use the horizontal arrow button in the Attribute Sheet to display/hide these drawings.
-    Entity Comments can be displayed/edited in the Attributes Sheet. Click on the vertical arrow to display/hide the comments area. The Comments area will automatically be displayed for entities containing comments.
-    You cannot undo changes to the PCSWMM GIS database.

Layers
The information displayed graphically in the View window is divided into layers. There are three categories of layers:

SWMM layers
These 3 layers are always loaded and provide access to the Runoff, Transport and Extran data sets. They are the only layers that may be edited, queried, imported to and/or exported from.

Background layers
Background maps or images can be used to provide visual reference information when displaying SWMM layers and/or results data. Each background image or map is connected and displayed through its own layer. Multiple background layers can be displayed at the same time, either tiled or overlaid. For example, you may wish to connect layers represented by many TIFF image tiles of a USGS topographic map, an AutoCAD DXF vector drawing file of city streets, and a layer containing objects representing the results of a point database query. Supported background layer formats include both vector and raster images. Any number of background layers can be connected, however the number and complexity of visible background layers impacts scrolling and zooming performance.

Results layers
These layers display dynamic views of SWMM model results. Currently, the computed time-varying node depths can be loaded from the intermediate print cycle of Extran Output files. In future versions, support will be provided for a variety of dynamic and static views of model results.

All layers have their title and legend displayed in the Table of Contents, to the left of the View window. The active layer is indicated in the Table of Contents by a beveled edge. Layer visibility can be toggled on/off and layer properties can be edited.

Map files
Contain Northing and Easting data (X,Y) which can be imported to a SWMM layer in PCSWMM GIS. Map files follow the same format as that used by EPANET ver 1.x. Use the Import Wizard to import data from Map files.

Node/Conduit/Subcatchment tables
These tables display the attributes of all entities contained in the opened PCSWMM GIS database. Using the tables, the selected layer’s entities can be selected, created, or deleted, and the entity attributes can be edited. These tables can be displayed or hidden.

Pathway
A pathway is a selection of consecutive, joined nodes and conduits (e.g. a linear branch of a sewer system). Pathways are useful for checking connectivity and aggregating conduits. They can also be used in conjunction with the Dynamic Hydraulic Gradeline tool (DHGL) to display the profile and/or animate the hydraulic gradeline model results for Extran layers.

PCSWMM GIS database
An Microsoft Access compatible (.mdb) database that stores all of the data used in a PCSWMM GIS project. It contains tables of SWMM model node and conduit data, as well as paths of connected layer files and SWMM input files. Information in this database can be viewed/edited through the PCSWMM GIS interface or by another program (such as Microsoft Access). This database is created by PCSWMM GIS from a template database.

Query
A logical statement used to select entities or records. A simple query contains a field name, an operator, and a value. PCSWMM GIS supports the full SQL query language.

Example:
Select * from [ExtranConduits] where [Length] > 300

Special Replace operations

The following special Replace operations are available from the Replace dialog box:

Switch conduit flow direction
Conduit flow direction can be quickly switched. Choose either the Node1 or Node2 attribute and select the “Switch Node1 & Node2” operator.

Determine conduit length from the map coordinates
Conduit Length attributes can be determined from the Map coordinate system. Choose the Length attribute and select the “Map Length: multiply by” operator. Specify a Value other than 1 if you wish to convert/adjust the coordinate system units.

Determine subcatchment area from map coordinates
Subcatchment Area attributes can be determined from the Map coordinate system. Choose the Area attribute and select the “Map Area: multiply by” operator. Specify a Value other than 1 if you wish to convert/adjust the coordinate system units.

Convert the conduit offset between actual elevations and relative elevations
Conduit Invert1 and Invert2 attributes can be converted between actual and relative (to node invert) elevations. Select the Invert1 attribute and choose either the “Subtract Junction Invert” or “Add Junction Invert” operation. Repeat for the Invert2 attribute.

Working with the Results Layer
The Results layer can currently be used to animate the results of Extran model runs. Upon loading a results layer, the time-varying junction (node) depth information is loaded from the intermediate print cycle of the selected Extran output file. The node depths are graphically represented by dynamic vertical bars located at the corresponding node locations. Labels indicate the depth through numeric values. If surcharging occurs, the portion of the node depth above the conduit obvert is represented in a different user-specified color (default surcharge color is red).

To playback the simulation results:
1    Select the desired Results layer
2    In the Table of Contents, click on the Play/Pause button (to the right of the slider bar).
If you wish to pause the playback, click on the Play/Pause button. You can manually control the playback by moving the sliderbar with the mouse. The current time step is displayed in the Results layer item in the Table of Contents.

Notes
-    The Results layer has color and label properties, which can be set through the Properties dialog box.
-    The time step of the intermediate print cycle controls the number of animation frames available for playback in the Results Layer. This time step is set with the B1 line’s INTER parameter in the Extran input data file.
-    You may need to adjust the width of the Table of Contents to view the entire simulation time step readout.

Import Data File Formats
Attribute data for SWMM entities can be extracted from the following file formats:

Database/spreadsheet formats:

Access
Btreive
Dbase III
Dbase IV
Dbase 5.x
Excel 3.0
Excel 4.0
Excel 5.0
FoxPro 2.0
FoxPro 2.5
FoxPro 2.6
Lotus WKS
Lotus WK1
Lotus WK3
Paradox 3.x
Paradox 4.x
Paradox 5.x
Delimited text

SWMM input file formats:

Runoff
Transport
Extran

Other formats:

EPANET map file (X,Y node coordinates only, click here for details on format)

Background Layer File Formats
PCSWMM GIS supports a large number of background layer formats.

Vector map formats:

ESRI Shape File    .SHP,.SHX
AutoCAD DWG    .DWG (Release 14 and earlier)
AutoCAD DXF    .DXF (Release 14 and earlier)
Blue Marble Layer    .BML
MapInfo Vector    .MIF
MicroStation DGN    .DGN
AutoCAD DWG    .DWG
Vector Product Format    .VPF

Raster image formats:

TIFF    .TIF (Specification 5.0. Supported compression types: no compression, packbits, Modified Huffman encoding, CCITT Group 3 1D, CCITT Group 3 2D, and CCITT Group 4. Supported color formats: monochrome, 256 color, grayscale, and 24 bit)
Windows Bitmap    .BMP,.DIB (Supports 1,4, 8, 24 bit images in Windows or OS/2 format files. Run Length encoded files are not currently supported.)
Windows PCX    .PCX (Supported color formats: monochrome, 16 color, 256 color, and 24 bit)

JPEG    .JPG (Supports JFIF compliant files)
Targa    .TGA

Supported spatial reference file formats for raster images:

ESRI World File (TIFF)    .TFW
ESRI World File (Bitmap)    .WLD
ESRI World File (JPEG)    .JGW
Blue Marble RSF    .RSF
MapInfo TAB    .TAB
ESRI World File Format

The ESRI World File format stores the parameters for a six-parameter affine transformation to effect its image to world transformation:
x' = Ax + By + C
y' = Dx + Ey + F
where    x' is the world coordinate of the pixel located at row x and column y;
    y' is the world coordinate of the pixel located at row x and column y;
    A is the x-scale or dimension of a pixel in world units in x direction;
    B,D are rotation terms;
    E is the negative of the y-scale or dimension of a pixel in world units in y direction assuming an upper left image origin with row values increasing downward; and
    C,F are translation terms or the x and y world coordinates of the center of the upper left pixel.

The transformation parameters are in ASCII format (one per line) in the following order: A, D, B, E, C, F. The following is an example:

A    20.0
D    0.0
B    0.0
E    -19.6
C    100000.0
F    200000.0

The letters are included to remind you of the order, as only numbers are placed in the world file.
ESRI World files have a .TFW (for TIFF images), .WLD (for Bitmap images), or a .JGW (for JPEG images) extension.

Map file format
Map files follow the same format as EPANET ver 1.x map files, however only the COORDINATES section is read by PCSWMM GIS (i.e. only node coordinates are imported). They are space delimited ASCII text files. The node coordinate section should be headed with “[Coordinates]” and must contain no blank lines until the end of the section. Comment lines are allowed at any point – all common comment symbols are recognized.

Example:
[COORDINATES]
;Nodal coordinates for Extran Example 1
10208 535800 4809960
10309 535740 4809885
15009 535550 4809910
16009 535560 4809860
16109 535460 4809795
80408 535750 4809520
80608 535665 4809690
81009 535310 4810235
81309 535440 4810015
82309 535510 4809745

Subcatchment Fill Style
The subcatchment fill style defines the fill for all subcatchments, whether they are represented by the default subcatchment shape or by individually specified subcatchment polygons. The fill style can include the following:

No fill
Solid fill
Horizontal hatch
Vertical hatch
45-degree downward hatch, left to right
45-degree upward hatch, left to right       
Horizontal and vertical cross hatch
45-degree cross hatch

Subcatchment polygons
Runoff subcatchment entities can be represented in the view area as polygons. These polygons can have a variety of fill styles , including solid and transparent fill, and can contain up to 250 coordinate pairs. The number of coordinate pairs per subcatchment affects program performance for large models. Subcatchment entities can be selected by clicking in the polygon area. If Easting and Northing coordinate pairs are not assigned for a subcatchment entity, it is represented in the view area by the default subcatchment shape.

Subcatchment Polygon File
Subcatchment polygon files (*.sub) store the coordinate pairs for a subcatchment polygon. Subcatchment coordinate pairs can be imported from or exported to a subcatchment polygon file (for backup or storage). Default subcatchment shapes are also stored in these files (in the PCSWMM GIS application folder).

SWMM input files
Input data files contain all the information SWMM requires for running a simulation. This includes descriptions of all the physical entities of a model and their connectivity, any hyetograph or hydrograph time series required for driving the model and runtime information such as timestep, duration and locations for outputing results.
PCSWMM GIS can export data to or import data from Runoff, Transport and Extran SWMM input files. For data exporting, a SWMM input file must be specified in the Input File dialog box for the SWMM layer of interest. Use the Import Wizard to import data from SWMM input files.Information on input data file requirements is available in the SWMM help file (choose SWMM Input Files from the Help menu of PCSWMM GIS). More information on the various SWMM file types and their structures is available in the set of SWMM manuals.

SWMM layers
These layers (3 of them) are always loaded and provide access to the Runoff, Transport and Extran data sets. They are the only layers that may be edited, imported to and/or exported from (and only when selected as the active layer).

Table of Contents
The list of layers on the left side of the PCSWMM GIS main window (next to the View window). The Table of Contents (TOC) lists the title and legend for each layer and provides a means of selecting the active layer and displaying or hiding each layer. The width of the TOC can be adjusted with the mouse by pressing the left mouse button down on the separator bar between the TOC and the view window and dragging the bar to the left or right.

Template database
The template database, PCSWMM.SRC, resides in the PCSWMM GIS program folder and serves as a template for all new PCSWMM GIS databases. It contains all the tables and fields required by PCSWMM GIS – if the template database is corrupted for any reason, the original can be restored from the PCSWMM GIS CD-R.

Toolbar buttons
All toolbar button functions are used in conjunction with the View window. Hover the mouse pointer over a toolbar button to display its pop-up name.

Object Selector: Select this tool to select one or more nodes and/or conduits, or to move nodes.
Add Conduit: Select this tool to create conduits and/or nodes.
Zoom In: Select this tool to zoom in to a portion of the displayed view.
Zoom Out: Select this tool to zoom out or increase the displayed view area.

Pan Map: Select this tool to drag the map in the direction you wish.
Adjust Conduit Display Size: Cycles the conduit line width through 3 increasing thicknesses.
Adjust Label Display Size: Cycles the font size of node and conduit labels through 3 increasing sizes.

Lock or Unlock SWMM Layers: Depressing this button locks the node positions so that they cannot be accidentally nudged when selecting.

View Profile: Displays the dynamic hydraulic gradeline for the selected pathway. Works only in conjunction with the Extran layer. There must be a SWMM input file associated with the Extran layer for which an output file exists. This toolbar item works with the Dynamic HGL tool of PCSWMM – you must have PCSWMM ’98 or later installed on your computer to use this function.

View
A View is the state of the View Window . It includes the properties of all layers , any loaded background or results layers, and the node/subcatchment coordinates. Views can be saved and loaded at any time by means of View files.

View files
A View file stores a View of the model entities for instant recall. All layer properties are stored (colors, label orientation and size, etc), any references to background layers and/or loaded results layers are stored. Node/subcatchment coordinates are also stored. View files can be created (saved) and/or recalled (loaded) at any point. During the loading of a View file the user can choose whether or not to update the node/subcatchment coordinates. Storing and recalling node coordinates is an easy way to switch between schematic and scaled (“real world”) views.

View Window
The graphical information of various layers is displayed in the View window of the main PCSWMM GIS window (to the right of the Table of Contents). Layers can be toggled on and off and/or selected using the Table of Contents. The Properties dialog box controls layer display properties.

Supported SWMM entity attributes
Runoff, Transport and Extran entity attributes currently supported by PCSWMM GIS are:

Node Attributes:

ID,
easting
northing
ground elev (Extran only)
invert elev (Extran only)
inflow
initial depth (Extran only)
up to 4 pollutant concentrations in inflow (Transport only)

Conduit Attributes:

ID
upstream node
downstream node
type,
length,
roughness,
up to three measurements for determining cross-sectional shape (depending on type),
invert elev. at each end (Extran only),
side slopes (if applicable),
initial flow (Extran only),
initial depth (Runoff only)

Subcatchment Attributes:

ID
inlet node
width
area
impervious area
slope
roughness of impervious area
roughness of pervious area
storage in impervious area
storage in pervious area
3 infiltration parameters (for Horton's or Green-Ampt equation)

Frequently Asked Questions

In this topic I address the questions most frequently asked by beginners.

What is the primary focus of PCSWMM GIS?
PCSWMM GIS has two main focuses. The first is as a standalone GIS for graphically creating, editing and/or querying SWMM model entities and attributes, displaying these SWMM layers with background layers and dynamic model results, and exporting data to SWMM input files. The second focus is as an interface between a GIS/IMS and SWMM. It is assumed (though not necessary) that most model development will take place within the PCSWMM GIS and PCSWMM '98 environment.

Is this a standalone tool or are you driving the GIS application with it using either ESRI or MapInfo's VB/VC++ components?
Standalone. The product interfaces directly with the underlying database(s). ESRI, MapInfo and AutoCAD layers/themes can all be displayed. Look and feel is most similar to ArcView (although MapObjects were not employed).

Is the interface to other tools completely through ASCII text files or something more direct? Can it work directly from an Access database?
Data is extracted directly from the GIS's underlying database (MS Access or ODBC compliant) using SQL queries and setup in PCSWMM GIS's internal database (MS Access) for 'data tweaking' into a suitable model (i.e. element aggregation, etc.) External access to the PCSWMM GIS internal database is also possible.

What type of evaluation and display tools does it have?
The beta release contains no output interpretation tools. These will come fairly quickly over the next few months (i.e. mostly dynamic displays such as simulation playback thermometer plots). It is, however, tightly linked with PCSWMM '98's array of tools. For example, profiles can be selected graphically within the PCSWMM GIS view and displayed in the Dynamic HGL tool (with simulation playback).

Can I have a *.bmp or meta-file as a background (albeit, hard to scale it right...)?
Yes and with completely accurate scaling (as long as you have a georeference file). In fact you may have as many tiled or layered images loaded as you like. Supported raster file formats include: .TIF, .BMP, .JPG, .PCX, .DIB, and .TGA. Other (vector) layer formats supported are: MapInfo MIF, MicroStation DGN, AutoCAD DXF, AutoCAD DWG, and ESRI Shape File.

Does GIS stand for Geographical Info Sys and therefore maintains and operates in real world coordinates and projections, or is it primarily a Graphical Info Sys and therefore is a flat-world, simple schematic view?
It stands for Geographical Information System. PCSWMM GIS '98 does not render in 3D (no fly-thoughs or 3D points of view). The view is restricted to a plan view. However, vertical detail is stored in the Extran layer of course, and PCSWMM GIS supports real world coordinates (and any other coordinate system).

Can the interface be used to automate data input such as watershed areas, land use, and soils related parameters such as runoff curve numbers or infiltration rates (I have GIS data with these attributes)?
Yes - node, conduit, and subcatchment data is pulled from the underlying GIS database (ODBC) into an intermediate database for processing into a useful model. This data is then exported to a SWMM input file (Runoff, Transport or Extran). However, SWMM does not use runoff curve numbers. See the answer to the next question for a list of supported parameters. Watershed area and conduit lengths can be calculated by PCSWMM GIS.

What other data can be prepared using GIS?
Attributes supported by PCSWMM GIS are:

Node Attributes:

ID,
easting
northing
ground elev (Extran only)
invert elev (Extran only)
inflow
initial depth (Extran only)
up to 4 pollutant concentrations in inflow (Transport only)

Conduit Attributes:

ID
upstream node
downstream node
type,
length,
roughness,
up to three measurements for determining cross-sectional shape (depending on type),
invert elev. at each end (Extran only),
side slopes (if applicable),
initial flow (Extran only),
initial depth (Runoff only)

Subcatchment Attributes:

ID
inlet node
width
area
impervious area
slope
roughness of impervious area
roughness of pervious area
storage in impervious area
storage in pervious area
3 infiltration parameters (for Horton's or Green-Ampt equation)

Does PCSWMM GIS calculate/estimate SWMM input parameters and then insert them into an input file for SWMM execution, or does the GIS simply calculate input parameters that you then must insert yourself into the SWMM input file?
PCSWMM GIS can calculate conduit lengths and subcatchment areas from the map display. Also, the Aggregation wizard calculates equivalent parameters when aggregating two or more conduits to a single conduit. As for updating SWMM input data files, you simply associate an input file with a SWMM layer (Runoff, Transport or Extran) and click on the 'Export to Input File' menu item. The selected elements are written to the input file.

I have Arcview shape files for several watershed characteristics (land use, catchments, some sewer network, ...) and am planning on doing a large-scale simulation making the GIS a needed tool. Will PCSWMM GIS expedite the determination of input parameters and the preparation of input files?
PCSWMM GIS cannot determine input parameters directly from shape files. However, as stated above, it can calculate conduit lengths and subcatchment areas from the SWMM layers. Other attributes must be filled in by the user, imported from the underlying GIS attribute database, or imported from spreadsheets or other databases. We feel true GIS (ArcView and/or ArcInfo for example) products are best suited for other types of attribute determination (e.g. subcatchment slope, percent imperviousness, etc.). Once these attributes have been determined and exist in the GIS, PCSWMM GIS facilitates the data extraction and model development.

What else do I need in addition to PCSWMM GIS to obtain a functional SWMM modeling environment?
It is recommended that you purchase PCSWMM ‘98, our decision support system (DSS) for SWMM, and use PCSWMM GIS as a plug in tool. The PCSWMM GIS '98 installation CD-R includes the SWMM4.4, SWMM4.31 and the official SWMM4.3 engines, however they are not installed by default. In any case, you will need a set of SWMM manuals, which are also available from us. We have just released an update of the manuals (99.01), however any set of manuals from 1988 or more recent will do in a pinch.

How effective is PCSWMM GIS without PCSWMM?
Basically PCSWMM GIS allows you to create SWMM entities (subcatchment, node, conduit) from scratch or pull in entities and attributes from a GIS database, edit the data to build a model and update an existing SWMM input file with this data. In addition it can dynamically display the depth results of Extran models and provide network connectivity checking. All other aspects of modeling (file management, input data file development, running the model, output visualization and interpretation, sensitivity, calibration and error analysis, storm dynamics analysis, and much more) are handled by PCSWMM ‘98.

Assignment A3:

We don't know how long this assignment will take you, and caution you to keep a note of the effort and to control it. This means perhaps that you will have to read selectively. It's OK if you are not able to complete all the suggested reading, e.g. the GIS details.

In this assignment you will evaluate a hypothetical area of approximately 50 ha.  You are required to build and run a SWMM/Runoff files for pre-development conditions. You will also be required to perform model parameter calibration against observed flow. In addition for graduate course credit you will discuss how the concepts of model complexity, uncertainty and data available for modeling apply to your modeling approach.

For this assignment you will need a copy of SWMM, AutoCAD or GIS application for SWMM, map files and data describing the area. If necessary a copy of SWMM will be provided by  Dr. James. The data and maps supplied by Eduardo Siqueira are at the ftp site reached by the links in Table 2. The dataset that you will use is given in Table 3 so please check that the data files you get are as shown in the table.  Each individual will receive a different set of data and we expect that the results of the modeling will be different for the various modelers. We urge you to not exchange or share derived data with your colleagues, but just help one another with information related to modeling how-to procedure. Students who have never worked with SWMM before will be linked with students who have, in a way that we can all get the modeling accomplished. 

Watershed and model parameters

You are asked to develop and calibrate your model for peak flows and flow volumes at location A shown on the AutoCad map layer “Gauges”.

For your model development our recommendations and preferences are:

Use metric units

No snowmelt

No quality

No groundwater routines

Assume a nominal evaporation rate of 3mm/day

For the open channel cross-section, use the following shape

Figure 1 – Watson Creek cross-section

 For the general land area:

Assume that the area is a fallow field (bare soil with sparse corn stalks). 

Soil is sandy clay loam – contains fractions of inorganic fine silts and clays with slight plasticity. 

For observed infiltration parameters, check the attached table and the AutoCAD map for their locations

Note that not all the locations in the map have been sampled

 E-mail Ed at if you have any question about the area that has not been covered.

 Data to be provided by Eduardo includes:

Table 1: Data provided for modeling and file formats

The .TSF files can be cut and pasted into your spreadsheet if needed.

The goal of the modeling is to calculate peak flow and total volume in the Hypothetical Creek, at location A, identified on the map.

Table 2: Modelers, datasets and respective data for modeling  Click on the marked cell to get your data (if the link does not work, try ftp to stargate.nw.uoguelph.ca and login as anonymous using your email address as the password, then look for the folder wjexpt).

Table 3: Data available in each dataset

Submission:

Graduate students are required to post a web page with the results of your modeling and e-mail a copy of the generated PCSWMM/ Runoff  input files to the instructors. Be sure that you have included all the items below in you final submission.

Files:

a) email to the instructors nine files: the input (.dat), interface (.int) and output (.out) files, before and after model calibration (for both flow peak and total flow), even if your initial data files generate no runoff.

In your web-page, post:

b)

c) Let us know the total time spent in the modeling.

d) Provide the following information:

 e) All graduate students should discuss how the concepts of model complexity, uncertainty and data available for modeling apply to your modeling approach. Discuss also how these three concepts relate to each other and give some general examples. What levels of complexity are relevant to specific design problems? How does varying model complexity require conceptualization of the real problem at various degrees of spatial resolution? How does the activation of different physical processes relate to data acquisition and input data files of varying size? Your web page should prove that you understand terminology and concepts of complexity, uncertainty and data availability and how they relate to urban storm water systems modeling. Reference material is available on this subject on the web and in technical libraries in books, monographs, conference proceedings, and journals, but you do not need to spend much time searching it because reading this module will cover most of your needs. Do not be afraid to express your own point of view on the subject.

 f) Please email the answers to the following user-evaluation and participation questions to us. The answers will be used to classify model users according to their experience in modeling. All the information provided in the form will remain confidential.

1. What is your name?
2. What is your e-mail address?
3. What is your current degree? E.g. BSc, MSc or PhD; and whether in course or complete.
4. Have you taken any general courses on hydrology or hydraulics or urban water systems design? (Yes or No)
5. Have you taken any specific course that involved  hydrological modeling? (Yes or No)
6. About how many related courses in total?
7. If you have experience in runoff modeling, approximately for how many years and for how many designs?
8. Have you ever worked with SWMM before? (Yes or  No)
9. If you have experience with SWMM modeling, approximately for how many years and for how many designs?
10. What SWMM engine are you using for this experiment?
11. What shell are you using?
12. For those who are working with PCSWMM, have you ever worked with it before? (Yes or No)
13. For the following hydrological processes, identify those with which you are reasonably familiar (ie. with the physical and mathematical fundamentals): rainfall, infiltration, evaporation, depression storage, routing, groundwater and runoff.
14. Do you agree that the results of your model (i.e. the input datafile that you will generate) can be used anonymously in Eduardo Siqueira’s thesis?