In addition to the high-level overview presented in the figure below “Architectural representation”, this chapter brings detailed view into each of the logical components:
- SQL Server Databases
- PetroVisor Server Host
- Data Integration Tool
- PetroVisor Client Applications
- Web Apps/Dashboards
- R server
- Web API REST web service
These major components are described by the following chapters.
SQL Server Databases
PetroVisor server host requires access to PetroVisor SQL Server databases. PetroVisor server can use existing empty database and automatically create or update required database schema – for that PetroVisor need user with ‘db_owner’ role for PetroVisor databases.
If additional administrative privileges are provided, PetroVisor can create new PetroVisor databases.
Database access credentials can be configured for every database separately. Both ‘Windows’ and ‘SQL Server’ authentication types are supported.
PetroVisor physical data model is segmented into 3 types of databases.
- Workspace – used as store for imported data and all objects (entities, hierarchies, P# scripts, etc.) which are specific for one asset.
- Registry – used as store for PetroVisor host settings, configuration and the data which is common to every workspace (asset).
- Users – standard user database used by Microsoft ASP.NET Identity as store for the users, roles, etc.
PetroVisor workspace is a main part of PetroVisor physical data model. Conceptually every PetroVisor workspace represents one data store (database). It contains all required data for one organizational unit which could be modeled independently like asset, field, etc.
Working with one workspace, it is generally not possible to access the data in other workspaces (without explicit import/export data operations).
Workspaces
The requirement is to have one PetroVisor user account for every Windows/AD user account which can be used to access PetroVisor.
Centralized storage and user interface allow Administrator to set up which workspaces each user can access. See the following screenshot – Administrator can also set to which user roles and groups user belongs.
User Management
Administrator can also create and modify user roles and groups, which can have different access rights for PetroVisor operations.
User roles
It is possible to use another Windows/AD user (not current one) by using built-in Windows ‘Run as’ command/option (user will have to enter valid another Windows/AD user credentials).
Administrator can run Active Directory user import for every workspace – so corresponding PetroVisor user accounts for selected workspace will be automatically created from selected AD users (including details like phone numbers, addresses, photo, titles, etc.).
Active Directory User Import
PetroVisor Server Host
PetroVisor server host is a set of the following installed Windows services:
- PetroVisor Identity and Discovery Service handles OAuth authentication and provides discovery document which contains configuration of other services. Uses open source IdentityServer 4.1.2 components.
- PetroVisor REST Web API Host service provides access to PetroVisor objects and functionality using https protocol and Json. It is also responsible for communication between PetroVisor components (using SignalR technology/components). This service is required to be running for any PetroVisor functionality.
- PetroVisor Activity service is responsible for any time scheduled PetroVisor activities (for example scheduled workflow execution). If this service is stopped, time scheduled activities can't be triggered.
- PetroVisor Workflow service executes PetroVisor workflows. If this service is stopped, workflows can't be executed.
PetroVisor Server Configuration tool is a GUI application which helps to setup, configure and maintain PetroVisor server host. It includes Windows services installation/removal, databases creation/verification, workspace data configuration, users/roles management, PetroVisor logs viewing and workflow execution monitoring tools. There is web version of this application available.
PetroVisor services require valid license file to start.
Services
Data Integration Tool
PetroVisor Data Integration tool is a client GUI application which allows to create and modify the mapping and other settings required to import the data from external data source to PetroVisor workspace database. Created mappings and settings are saved as named PetroVisor Data Integration Sets. Each of the Data Integration Set can be used to import the data right from the tool or as part of scheduled automatic workflow.
The source data is filtered from empty or invalid values, cleansed using P# cleansing rules and time-aggregated during data import to PetroVisor workspace database.
The following external data sources are supported:
- Relational databases
- Microsoft SQL server
- Oracle
- ODBC data sources
- Text files
- With custom delimiter (.csv, etc.)
- Fixed width data columns
- LAS
- Special
- Microsoft Excel
- Teradata Data Warehouse
- Pi server
PetroVisor Data Integration tool
PetroVisor Client Applications
PetroVisor client applications is a client GUI tools which consists of the several modules to create, configure and maintain automated workflows and view and modify PetroVisor domain objects for one workspace. It consists of Workflow Automation and Analysis applications. Workflow Automation application contains of the following modules:
- Hierarchy Editor – allows to view, create, modify and delete PetroVisor entities and hierarchies
- Context Editor – allows to view, create, modify and delete PetroVisor contexts
- P# Editor - allows to view, create, modify, delete and run P# scripts
- R Editor - allows to view, create, modify, delete and run R scripts/functions
- Workflow Designer – allows to view, create, modify and delete automated PetroVisor workflows
- Scenarios – allows to view and maintain data scenarios and runs
- Workflow Scheduler – allows to schedule automated PetroVisor workflow execution
- Value Editor – allows to view and edit workspace data
- Communication - allows to view, create, modify and maintain PetroVisor tickets and user messages
- Pivot Tables – allow to view, create, modify and maintain PetroVisor Pivot tables
PetroVisor Workflow Automation Application
Analysis application allows to create, view and modify special model objects to use them on their own and to integrate them to PetroVisor workflows:
- Decline Curve Analysis – allows to view, create, calculate, modify and delete PetroVisor DCA fits
- VoronoiGrids– allows to view, create, calculate, modify and delete PetroVisor Voronoi grids
- Tags - allows to view, create, modify and delete PetroVisor tags and tag entries
- Logs – allow to view the workspace logs
PetroVisor Analysis Application
Web Dashboards
PetroVisor web dashboards is a set of custom ASP.NET web sites hosted in Windows Internet Information Services. They let the user quickly drill down to a workspace (asset) of interest and review its performance using an internet browser.
The user can create his own web pages and sub-pages and populate them with customized plots and tables.
PetroVisor now includes client web apps, which are providing most of the desktop client apps functionalities and intended to replace desktop app in the future.
PetroVisor - ChartBuilder
R server
Optionally PetroVisor workflows can contain the calls of R script functions. To enable this functionality, PetroVisor can be configured to connect to an existing R server using one of the available providers:
- NET: Open source provider to connect to an MRO instance on the local machine (this provider is used by the client automatically)
- Rserve: provider to connect to any R instance (on the local and remote machines) running Rserve on top of it.
R scripts may be created using PetroVisor’s R editor or imported using the server configuration tool. Please note that the R scripts module is like a sandbox, i.e. running an R script will not save the results to the database. Once the activity has been implemented and tested, it can be used as a workflow activity. Running an activity in the workflow will save results to the database.
No data is stored on the R server. PetroVisor prepares the data, calls R script functions, retrieves the results from the R server and stores them in PetroVisor’s workspace database.
PetroVisor's R scripts module
Import of existing R scripts using PetroVisor's server configuration tool
Web API REST service
PetroVisor data and functionality can be accessible through web API REST service. It supports Open API 3.0 specification and OAuth 2.0 authentication using IdentityServer. The service can be used directly or through public available PetroVisor client library.
Significant sub-components
There are 2 significant sub-components of PetroVisor server, which are described in more details.
P# language
P# language uses a data-driven domain objects to define:
- Which data has to be retrieved
- Which calculations have to be done on retrieved data
- Which calculated data should be saved for later use
- Data context like time range, entities, hierarchy, etc.
PetroVisor uses open source ANLTR (http://www.antlr.org) lexer/parser generator to translate P# language text to corresponding PetroVisor API objects and calls expressed in C# code. That generated C# code is executed by PetroVisor to retrieve the data in context, make calculations with it and optionally save calculated data for later use.
It is possible to save defined data objects and then use named references to them instead of whole data objects definitions. Therefore, most used data objects could be defined once and used several times.
A P# script is one complete piece of P# language code which could be executed. Usually one P# script calculates and saves the data which is used then by another P# script as input data. Thus the sequence of P# scripts represents a PetroVisor workflow and contain all required data definitions and calculations to solve domain specific problem.
This means that the business logic for specific problem (defined in P# scripts using P# language) is not a part of PetroVisor codebase and this logic could be updated without doing any changes in PetroVisor code or deployment. In other words, certain workflows could be updated (or completely new workflow could be created) using the same already deployed version of PetroVisor.
P# Script Editor
PetroVisor Calculation Engine
P# language allows to define the calculations on the data using standard mathematical and logical operations. Additionally, predefined P# functions (like MovingSum, GapFilling, Cumulative, etc.) can be used. The combination of them allows to define complex nested expressions. The final result of the expression is dynamically calculated using PetroVisor calculation engine.
PetroVisor calculation engine converts P# language expressions to .NET Expression Trees (https://msdn.microsoft.com/en-us/library/bb397951(v=vs.120).aspx ), which are executed later on the retrieved data. This allows to define data calculations as dynamic queries which has the following advantages:
- Run-time evaluation and execution, results available on the fly
- No need of separate compilation
- New and updated P# functions can be added without engine code changes
Currently PetroVisor calculation engine and P# language supports the following types of the functions:
- Mathematical functions (arithmetic and all from .NET Math namespace)
- Time-series smoothing (including gap filling, moving average/sum/min/max)
- Neighborhood functions (based on entities location coordinates and distances between entities)
- Aggregation (from simple min/max/average to based on conditions/distances)
- Statistical (Standard deviation, variance, percentile, etc.)
- Based on entities and their relationships in hierarchies
- Date/time related calculations