Wastewater Planning & Design System (WPDS) architecture

Project purpose

The Wastewater Planning & Design System (WPDS) is used to design a project's wastewater collection network on a map. The outcome is not a standalone drawing: it is a network model stored in an enterprise geodatabase, tied to a single project, that engineering staff can calculate, review, adjust, and export.

Work is split into two phases.

Request. The person opening the project specifies where the area is, draws the project boundary, and places the collection point. When they submit the request, the platform registers the project and, in the background, prepares starting data: it downloads cadastral parcels for that area (Lightbox), clips streets (OpenStreetMap) to the boundary, and generates a preview image. The project is then ready for engineering to open.

Design. The engineering team builds the network in five steps, always within the same project:

1. Connections. Computes service laterals from each parcel to the street. They can add auxiliary streets manually or restart the calculation.
2. Parcels. Reviews and corrects parcels, boundary, and laterals on the map.
3. Main collector. Proposes routes or adjusts a hand-drawn route and inserts it into the network.
4. Costs. Reviews construction parameters and cost per pipe segment.
5. EDU. Calculates Equivalent Dwelling Units, used to size the network, and allows exporting the project.

Everything that belongs to a project (parcels, streets, laterals, main collector, costs, EDU) shares the same identifier, workflowid. Copying, cloning, resetting, or deleting a project therefore means working with that identifier, not hunting for loose geometries.

The network is stored as an ArcGIS Utility Network: mains, laterals, connections, and the collection point are network elements with connectivity rules. The network is branch-versioned, so the application and calculations do not write directly to the database. They read and change data through web services published on ArcGIS Server. Heavy work (street graph, laterals, routing, EDU) runs on the server as Python geoprocessing (ArcPy); the web app only submits jobs, shows progress, and displays results on the map.

This documentation explains how that platform is built and how its parts connect, so anyone on the team can understand the system and assess the impact of a change before making it.

The full source code (web application, geoprocessing scripts, and ArcGIS Pro project) is in the repository Wastewater-System--WPDS-.

How it is organized

The generated/ annex holds the detailed inventory: widgets, application, web–geoprocessing links, toolboxes, scripts, ArcGIS Pro, geodatabase, and services. Interactive diagrams live in diagrams/; index.html combines everything in a single offline page.

Glossary

Scope

Out of scope:

• .sde connections and validation of broken layers, which require network access to the databases.

Change log

Diagrams

1. Overview

In brief: the platform has four layers. The user works in a web application; each calculation action is sent to a geoprocess on ArcGIS Server; the geoprocess reads and updates project data in the geodatabase through web services; and ArcGIS Pro is where those geoprocesses and services are prepared and published.

Diagram: System architecture

The four layers

Path of a request

The example below follows what happens when the user clicks Send to Engineers to submit a project request. Other calculation actions follow the same pattern.

1. Preparation. The create-workflow widget collects the drawn boundary, discharge point, and form data.
2. Authentication. The shared module wws-server-fetch obtains an ArcGIS Server token from the user's portal session.
3. Execution. The application submits geoprocess CreateWorkflow. The script registers the boundary, the project (status Pending), and the collection point, creates the commercial contact in HubSpot (production only), and returns the project identifier.
4. Initial calculation. Without waiting for a response, the application submits CalculateInitialNetwork. That job downloads parcels from Lightbox, clips streets from the roads database, generates a preview image, and sets the project to Ready.
5. Confirmation. The user sees a confirmation message. Engineering opens the project later from List projects, which shows the updated status.

Diagram: Geoprocess sequence

Technology

What is required to implement it

A new deployment needs the source repository, the ArcGIS licenses below, and a set of reference datasets. The application and geoprocessing services cannot run without them.

Source code

The full codebase is in the repository Wastewater-System--WPDS-.

Licenses

Reference data

Design principles

• Everything is organized by project. Each project element (mains, connections, parcels, EDU, costs) stores the project's workflowid. Copy, reset, clone, or delete a project by filtering on that identifier.
• Data is edited through services. The Utility Network is branch-versioned, so it can only be changed through its web services. Scripts do not write directly to the database; they use service URLs defined in UtilsUN/vars.py.
• Calculation runs locally (on the GP server). Each geoprocess copies project data to a temporary geodatabase, computes with ArcPy, and pushes results back to the service.
• Configuration lives in the application. Geoprocess URLs are set per widget in Experience Builder, not hard-coded.
• One calculation at a time. The application blocks starting a geoprocess while another is running.

2. Web application

In brief: the application is a single ArcGIS Experience Builder experience with four pages. It is built from custom widgets that share a common library (shared-code). Widgets coordinate through configuration, messages, and shared state, and every geoprocessing call goes through the same authentication module.

Annexes: application · widgets · web–geoprocessing links

Pages

Application ExB/server/public/apps/1 ("Staging | WPDS") has these pages:

How each page looks, in project order, is in 4. Workflow.

Web maps

Maps come from the WPDS portal (https://<wpds-server>/portal). Phase 2 is exported in the repository (shared-code/Map Phase 2.json):

Widgets reach layers through map data sources, not raw URLs. If the web map or a republished service changes, you may need to re-select layers in widget configuration.

Communication between widgets

The application uses four mechanisms:

1. Configuration references. A widget points to another widget it controls. For example, each stepper targets the widget whose steps it displays, and advanced-editing points at workflow-controller.
2. Messages. When the user edits features in advanced-editing, it publishes a data-change message. workflow-project receives it and reacts by layer—for example, a boundary edit triggers geoprocess UpdateNetwork.
3. Shared state. A global "geoprocess running" flag (gp-running-lock) prevents two calculations at once. Other modules keep the editor aligned with the active step.
4. URL parameters. The active project moves between pages as ?workflow=<id>, and the chosen location as ?lat=&lng=.

Authentication

Module shared-code/wws-server-fetch.ts centralizes ArcGIS Server access:

• Token. Requests a server token from the user's portal session. If there is no session (e.g. local development), it uses the API key configured on the widget.
• Job submission. Geoprocesses are submitted with POST so large geometries are not sent in the URL.
• Polling. Associates the token with job status queries so polling stays authenticated.

Geoprocess configuration

Each widget stores its tools in application configuration with two fields: service URL and a utility reference (the app's service registry). Practical effects:

• GP services. URLs and utilities configured in the documented application point at folder AWWSP_UN.

3. Widgets

In brief: there are 11 custom widgets, all used in the application. They rely on the shared-code library. Paths in this document are relative to ExB/client/your-extensions/widgets/.

Annexes: widget inventory · web–geoprocessing links

Project request

init-workflow — Start

• Role: welcome screen where the user picks the project location. Looks up State Plane zones for the point to suggest the right coordinate system.
• Page: Init workflow.
• Geoprocesses: none.

create-workflow — Project request

• Role: guides the four request steps (location, boundary, discharge point, and form with client data and contracted package). On submit, registers the project and starts initial calculation in the background.
• Page: Create workflow.
• Geoprocesses: CreateWorkflow, then CalculateInitialNetwork.
• Main code: steps/step-4.tsx (submit) and shared-code/build-create-workflow-gp-params.ts (form to GP parameters).

create-workflow-floating — Request on small screens

• Role: same request logic as a floating bar for small viewports.
• Page: Create workflow.

stepper — Step indicator

• Role: shows flow progress and drives the step of the linked widget.
• Pages: all flow pages.

Project design (Workflow manager)

workflow-controller — Coordinator

• Role: switches between map editor and step panel depending on the active step.

workflow-project — Design steps

• Role: core of design. Presents steps 5–9, checks required data before advancing, persists the current step on the project, and reacts to map edits.

advanced-editing — Map editor

• Role: edit features on the map: single, batch, and attachments. Applies Utility Network connectivity rules in the browser. Each edit notifies workflow-project to recalculate as needed.

data-statistics — Statistics

• Role: loads project metrics used for time and cost estimates.
• Geoprocess: CalculateEstimateInputs (read-only).

clone-workflow — Clone project

• Role: creates a copy with a new name and description.
• Geoprocess: CloneWorkflow.

title-project — Header

• Role: shows project and client names and assigned users.

Listing

list-projects — Projects

• Role: shows projects as cards with preview, status, and assignee. While status is Pending, polls every 5 seconds and blocks opening. Supports bulk delete.
• Geoprocess: DeleteMultipleProject.

Shared library (shared-code)

The library has 32 modules. The most important:

The full list, with widgets per module, is in the widget inventory.

4. Project workflow

In brief: a project goes through two phases. In the request, the user defines the area and discharge point, and the system automatically prepares parcels and streets. In design, engineering completes the network in five steps inside Workflow manager. Screenshots in this chapter show that journey in the application. Names, emails, and phone numbers are redacted.

Diagrams: Phase 1 · Project request · Phase 2 · Design · Geoprocess sequence · Project states

Phase 1 · Project request

CreateWorkflow (Create_workflow.py):

• Registers boundary, project (status Pending, step 5), and collection point.
• In production, creates the HubSpot contact.
• Returns the project identifier.

CalculateInitialNetwork (Calculate_initial_network.py):

1. Generates a preview image and stores it as an attachment.
2. Clips streets from the roads database (OpenStreetMap) to the boundary, keeps only those connected to the collection point, and if needed adds a segment linking the collection point to the nearest street. These become auxiliary lines.
3. Downloads parcels inside the boundary from Lightbox.
4. Sets the project to Ready, or Ready with errors if something fails.

After submit, the app shows confirmation. The project appears in List projects and can be opened once it leaves Pending.

Phase 2 · Design

The project opens from List projects. The current step is stored on the project record; the app blocks advancing if the next step lacks required data.

Two more actions are available throughout design:

• Statistics (CalculateEstimateInputs): project metrics.
• Clone (CloneWorkflow): project copy.

From List projects, projects can be deleted (DeleteMultipleProject).

Project states

Besides status, the project stores the current step (5–9) and whether connections have been calculated (calculated_connections).

General rules

• One geoprocess at a time. The app does not start a calculation while another is running.
• Shared authentication. All geoprocesses use the same mechanism (see 2. Web application).
• Data version. Calculations use geodatabase version sde.DEFAULT.

5. Geoprocessing

In brief: all calculations are Python scripts defined as tools in ArcGIS Pro toolbox Steps UN.atbx and published as asynchronous services in ArcGIS Server folder AWWSP_UN. Each service maps to one tool, and each task to one script.

https://<wpds-server>/server/rest/services/AWWSP_UN/CreateWorkflow/GPServer/Create_Workflow
                                              folder   service = tool   task = script (Create_workflow.py)

Some names contain typos that are part of the contract with the web app (for example, CalulateEdus). Do not fix them without republishing the service and updating application configuration.

Annexes: web–geoprocessing links · toolboxes · scripts

Catalog

The Contract column shows whether the published contract could be compared to the local toolbox definition: ✔ match, ✖ differs, 🔒 requires authentication.

Project request

Network design

Project management

Tools not used by the application

• CopyToLocal: copies a project to a local geodatabase for debugging.
• Scripts Second_1_on_edus.py and Second_1_priori.py: experimental route variants, not published.

Script structure

All entry scripts follow the same pattern:

1. Parameters. Read by index (arcpy.GetParameter(0), GetParameter(1), …). Order must match the published tool.
2. Data access. Project data elements from UtilsUN.vars (e.g. workflows_fc, sewer_line_fc). Each wraps the corresponding service URL.
3. Local calculation. Copy project data to a temporary geodatabase (Manager), compute with ArcPy or NetworkX, push results to the service.
4. Cleanup. Update project status when needed and close the edit session.

Environment

Environment is not inferred from where a service is published: it is fixed in UtilsUN/vars.py (environment = "STAGING").

Adding a geoprocess

1. Script. Create Scripts Utiliy Network/<Task_name>.py following the structure above. Access data only through UtilsUN.vars.
2. Tool. Add it to AWWSP Dev/Steps UN.atbx with parameters in the same order the script reads them.
3. Publish to staging. Publish as an asynchronous service in folder AWWSP_UN. Confirm vars.py environment first.
4. Application. Register the service as an application utility and select it in widget configuration. In code, submit through the shared auth module and respect the running-job lock.
5. Documentation. Update this documentation.

6. ArcGIS Pro

In brief: project Geoproccess/AWWSP Dev/AWWSP.aprx is the preparation and publishing environment. From it you maintain calculation tools, publish services, and inspect data. The web application does not use this project directly—it uses published services.

Annex: ArcGIS Pro inventory

Toolboxes

Many tools point at scripts via paths from another machine (C:\Repo\DIG_Python\…). On a new machine, relink them to Geoproccess/Scripts Utiliy Network.

Maps

Database connections

Connection files live under Local connections/ and Remote connections/. Scripts connect directly only to the roads database (Remote connections/Awwsp Roads.sde). Everything else goes through services.

Relationship to services

• Geoprocessing. Tools in Steps UN.atbx publish as GP services under AWWSP_UN.
• Data. Maps Utility Network and Utility Network - Auxiliar are the source of data services. Scripts reach them by layer index (/FeatureServer/2, /5, /12, …), so when republishing, preserve layer order. The full mapping is in 7. Data.

7. Data

In brief: data lives in enterprise geodatabase awwsp_un (PostgreSQL, owner dig). There are two groups: the Utility Network (sanitary sewer network) and project data (boundary, parcels, EDU…), plus several management tables. All data for one project links through workflowid.

Source: schema export AWWSP Dev/Awwsp UN_ExportXMLWorkspaceD.xml.

Diagrams: Data model · Data flow · Annex with fields, domains, subtypes: geodatabase

Utility Network (UtilityNetwork)

The Utility Network was built with ArcGIS Pro 3.4.3 and uses branch versioning, so it can only be edited through its services. Classes the platform uses:

The model includes other classes the platform does not use directly: SewerAssembly, SewerSubnetLine, Structure classes, and internal Utility Network tables.

Project data (ProjectData) and management tables

Published services

Both services require authentication.

Who uses each table

Exact table-by-table and script-by-script links are in the geodatabase annex.

External sources

8. Successful implementations

In brief: the same wastewater planning workflow has been delivered as two products. ARXInfrastructure is the branded application shown below, from location through EDU export. DIG (Digital Infrastructure) is an earlier delivery of the Waste Water Planning Tool; the frames come from a recorded walkthrough of one project, from map extent through the hydraulic profile.

ARXInfrastructure

The platform is the Wastewater Planning & Design System (WPDS). ARXInfrastructure is one branded deployment of that workflow. A project starts with a location, a drawn boundary, and a collection point, then goes to engineering. Design continues with connections, parcels, the main collector, costs, and EDU.

These frames are the same sequence used in 4. Workflow. Names, emails, and phone numbers are redacted.

DIG

The product on screen is Digital Infrastructure — Automated Waste Water System Planning (Waste Water Planning Tool). The walkthrough covers one project: set the map extent, draw the boundary, place the collection point, send the request, generate laterals, choose a main route, then review segments, the hydraulic grade line, and design-flow parameters.

Experience Builder applications

Pages, data sources, and widget coordination for each app under ExB/server/public/apps.

Updated 2026-09-29.

App 1 — Staging | Wastewater Planning & Design System (WPDS)

Experience Builder 1.17.0 · ExB/server/public/apps/1

Pages

Data sources

Widget messages

Widget references in configuration

ArcGIS Pro projects

Maps, layers, toolboxes, and connections for .aprx projects (read with arcpy 3.5).

Updated 2026-09-29.

Geoproccess/AWWSP Dev/AWWSP.aprx

Toolboxes: Geoproccess/AWWSP Dev/Steps UN.atbx, ../DIG/AWWSP Geoproccess/AWWSP Dev/Steps classical.atbx

Databases: D:\WWS\Geoproccess\AWWSP Dev\AWWSP.gdb, .\Roads\Roads full USA - TIGER.gdb, D:\WWS\Geoproccess\AWWSP Dev\Remote connections\Awwsp Roads.sde, D:\WWS\Geoproccess\AWWSP Dev\Remote connections\Awwsp UN - Admin.sde, D:\WWS\Geoproccess\AWWSP Dev\Remote connections\Awwsp UN.sde, ..\..\..\Roads_USA_cleaned.gdb

Enterprise connections used by layers

Connection files

Local connections/Awwsp EGDB Remote.sde, Local connections/Awwsp Roads.sde, Local connections/Awwsp UN - Admin.sde, Local connections/Awwsp UN.sde, Remote connections/Awwsp Roads.sde, Remote connections/Awwsp UN - Admin.sde, Remote connections/Awwsp UN.sde

Maps

Geodatabase schema

Geodatabase structure from the XML schema export, and how each table links to services, code, and maps.

Updated 2026-09-29.

Geoproccess/AWWSP Dev/Awwsp UN_ExportXMLWorkspaceD.xml

Version sde.DEFAULT · database awwsp_un · owner dig

Feature datasets

Tables and feature classes

Relationships

Utility Network UtilityNetwork

Created with ArcGIS Pro 3.4.3 · schema generation 7 · branch versioned: yes · service territory ServiceTerritory

Network attributes: Source ID, Terminal ID, Asset group, Asset type, Is subnetwork controller, Tier name, Shape length, Position from, Position to, Flow direction

Terminal configurations: Single terminal

Domains

Web application to geoprocessing links

Each row is the full chain: widget and config key → GP service → published toolbox tool → Python script.

Updated 2026-09-29.

App 1

Tool copies in other toolboxes

No items.

Geoprocessing scripts

Scripts under Geoproccess/Scripts Utiliy Network: parameters read and returned, and dependencies.

Updated 2026-09-29.

Geoprocessing entry scripts

Helper modules

Services and web maps

Feature services and web maps used by the application, scripts, and ArcGIS Pro projects.

Updated 2026-09-29.

Data services

Web maps

Toolboxes and tools

Tools in each ArcGIS Pro toolbox, their parameters, and the script each runs.

Updated 2026-09-29.

Geoproccess/AWWSP Dev/Steps UN.atbx (Steps 3.0)

Widget inventory

Custom widgets under ExB/client/your-extensions/widgets and shared library shared-code.

Updated 2026-09-29.

Custom widgets

shared-code modules

Per-widget dependencies