Explore, Process and Visualize your data right in your browser.
Device Attributes and On Page Load are included in Config settings
As mentioned, when a custom function runs it will drop and recreate the result table which is of a predefined structure. This structure is;
Custom Functions operate against a single input table (and not joined tables). If you need to apply a custom function using input data from 2 or more tables, the source tables will need to be joined prior to passing to the joined table to the Custom Function. Please refer to an example in General Help guide. When joining input tables, typically a cross join or Cartesian Product is to be generated to be passed to the Custom Function. In order to minimise the number of input rows in the join table, it is suggested to apply any additional relational filtering predicates as possible prior to joining tables. Relating the results of Custom Function output back to the source input tables can then still be achieved via the SRC_ROWID values from the Result table - but in the case of joined table input may involve a select statement of multiple tables to achieve this. Initially, using the ROWID's in this way may seem awkward until you get understand how to easily apply this. Because ROWID's are used in this way, the inputs to Custom Functions must be a table and not a database view.
Some Custom Functions require an array of input values. For such functions, arrays can be passed as a delimited list of values. Such functions will typically accept a list parameter and a list separator and delimiter parameter. This is how arrays can be passed to Custom Functions. Generating delimited lists from SQL, for input to Custom Functions, might seem difficult at first but thankfully WebSQL (sqlite) provides the fantastic group_concat function that can make the generation of such lists very simple. Please refer to the General Help section for some examples of using group_concat.The fields on the Define Custom Function pane are;
Self Drilldown is a mechanism that will enable you to create a hyperlink from your presented data that will link back to the Visual Field page (ie, vf.html). This is provided to enable the creation of mini data driven applications.
create index node_idx on my_nodes(node_id);
create index edge_idx on my_edges(from_node_id, to_node_id);
Graphs saved during the execution of a Network Visualization are referred to as "Saved Graphs". These can be browsed under this section. Clicking on the Label or the "Run" button will re-run the Network Visualization and restore the saved graph. Saved Graphs assume the Network Definition and source Node and Edge Tables/Views are all available and current. A saved graph just consists of a set of Network Nodes and Network Visualization properties.
This pane facilitates the import of media files into your Visual Field database. Media Files may be of type; Image, Audio or Video. Multiple files may be imported at a time. Simply drag and drop your media files onto the Import area. Individual file sizes are limited to a maximum of 40Mb. Files larger than 40Mb or non Image, Audio or Video files will be rejected. Files will be inserted into the VF_MEDIA table using the file name as the primary key. If an existing entry in VF_MEDIA exists for a file of the same name, then that will be replaced the new media - otherwise a new entry will be made. If Exif data is present in your media files, then this will be extracted and inserted into a table called VF_IMG_EXIF. VF_IMG_EXIF is linked to VF_MEDIA by using the FILE_NAME column as the key. There may be multiple rows in VF_IMG_EXIF for a given file name. If the embedded Exif contains extended parameters such as XMP tags, etc (eg, from Drone Imagery), then these extended Exif values will also attempted to be extracted and placed into the VF_IMG_EXIF. Such extended Exif values can be distinguished by the CATEGORY field in VF_IMG_EXIF.
It is assumed that your browser will be capable of displaying your particular media. It may be necessary to convert your media into a browser enabled format prior to Import. Because of the 40Mb limit and otherwise, you may need to consider reducing the file size prior to Import. Whilst raw files upto 40Mb may be imported, you may achieve better browser response time by converting imported media prior to Import. Eg, converting Video to a format optimised for Browser viewing. Or resizing your Images for Browser viewing. This is optional however.
From within Visual Field, Media fields can be viewed just like regular fields from Browse functions and form SQL statements, etc.
The METADATA that can be generated and imported back into Visual Field is essentially a simplistic JSON object with 4 key elements. This may be optionally URI encoded but underlying it is still a simple JSON object. This API page is included for those who may wish to automatically generate Visual Field metadata such as an automated report for example. The 4 elements in the metadata are;
The TABLES element is made up of an array of elements that are the Visual Field internal tables that are represented in the metadata. TABLE element themselves are then made up of an array of rows. The list of TABLES that can be included in metadata definitions is a subset of all of the Visual Field internal tables. Select a TABLE below to detail the row components to be included in a TABLES definition entry;
Custom Functions are identified by a numeric ID within Visual Field. This is an extensible array should you wish to add your own function definitions (refer to the General Help section). To assist in identifying a Custom Function definition and its ID details, please choose from the following selection. Custom functions that are deterministic in that their output is solely dependent upon their input parameters, should be defined as "type=immutable".
select "Upper Grade" from SCHOOLS;Or,
select [Upper Grade] from SCHOOLS;
create view if not exists SCHOOLS_GRADE as select cast("Upper Grade" as INT) "Upper Grade", "Campus Name" from SCHOOLS;
create table if not exists SCHOOLS_LOWER_GRADES as select group_concat("Lower Grade",',') as lower_grade_list from SCHOOLS;
There are also several Spatial Custom Functions in Visual Field (courtesy of the fantastic Turf js library) that accept an array of geometries as input. Again, the group_concat function may be useful in producing a list (array) of WKT geometries for input. In this case however, WKT geometries may contain commas, so it is safer to enclose the grouped geometries in double quotes (delimiter). Eg, suppose our SCHOOLS table has a WKT geometry field called "Location 1", then to get a list (array) of geometries, you could use the following;
create table if not exists SCHOOLS_GEOMS_LIST as select group_concat('"'||"Location 1"||'"',',') as geom_list from SCHOOLS;
But what about the case where your input may contain double quotes or commas? The approach here is to take group_concat further with the replace function and replace any occurrence of a double quote with a double double quote. In this case you will essentially be producing a standards based CSV input list (array) from SQL! Eg, suppose we wish to make a listing of the "Campus Name" field in the SCHOOLS table and that field may contain either commas or double quotes, then the following approach is suggested;
create table if not exists SCHOOLS_NAME_LIST as select group_concat('"'||replace("Campus Name",'"','""')||'"',',') as name_list from SCHOOLS;
If for some reason you need the inverse function to group_concat, please see the Custom Function Misc:ListSplit - which takes a list (delimited array) and splits the results out to individual rows in the result table.
To obtain Query String expressions within SQL, it may be necessary to query the VF_QUERY_STRING table directly. No shortcut technique is provided - it is up to you to manually query this table. Some points worth noting also are;
select "Campus Name" from SCHOOLS where "Lower Grade" = (select cast(value as INT) from VF_QUERY_STRING where name = 'A');And for a multi valued parameter;
select "Campus Name" from SCHOOLS where "Lower Grade" in (select cast(value as INT) as "Value" from VF_QUERY_STRING where name = 'B');
Similar to VF_QUERY_STRING the VF_DEVICE_ATTRIBUTES will be populated with a Collect Device Attributes task invoked and acceptance by the user. Device Attributes are similar to VF_QUERY_STRING in that they may need to be explicitly queried and CAST to appropriate datatypes.
The VF_TILE_STORE houses stored offline Map Tiles. This table is initially empty and gets populated if a user clicks the button in a Map Visualization and downloads Map Tiles for possible Offline use. Map Tile images must be CORS enabled and are simply stored in the VF_TILE_STORE table as an encoded array of Uint8 values. This is referred internally as ENCODING_ALGORITHM 1. However, more advanced users should recognise that the VF_TILE_STORE can be Browsed in Visual Field just like any other table and extracted as a CSV download. So, if you are technically impelled, you could create a tile store, extract it, and make the CSV available as a web service. You could then define a Table (Import URL) as part of a Page load, Run Menu, or Sequence that will fetch the web accessible CSV Tile Store and reload it into the VF_TILE_STORE table. You will then be able to create a mini application complete with an Offline Map Tile store. Please Note, as mentioned in the Reference Guide, downloading Map tiles for Offline use may be subject to your Tile providers usage policies.
create table if not exists SCHOOLS_HAZARD as select s.rowid as SCHOOLS_ROWID, s."Location 1" as the_point, h.rowid as HAZARDS_ROWID, h."the_geom" as the_polygon from SCHOOLS s, HAZARDS h;
The following are some fixed internal limits within Visual Field;
The following are some observed limits;
Visual Field Metadata Exports primarily define the key aspects of your Visual Field configuration. That is in respect to what you define in various Visual Field screens in the way of CSV file ingestion, defined sequences and visualizations for example. Visual Field Metadata is primarily intended to be useful if you are using Visual Field in an online (web) capacity, or a reporting, or mini data-driven application. Metadata is a JSON representation of your Visual Field configuration and can be imported into Visual Field by use of the METADATA parameter in the Query String when you invoke vf.html as a standalone file or from a web page. Visual Field Metadata can be used in both an online and offline capacity to define a particular Visual Field configuration.
A Visual Field Export Dump is a variable width csv file of encoded DDL and DML instructions. Visual Field Export Dumps are specific to Visual Field and should be able to be imported into Visual Field using another device or HTML5 WebSQL browser running Visual Field. As such Visual Field Dumps are intended to be a robust offline way of transferring your database objects between devices, browsers, or just making a backup.
SQL Dumps just consist of a series of DDL and DML SQL statements. SQL Dumps are intended to be a portable mechanism to facilitate the offline transfer of your Visual Field objects to another database such as sqlite or PostgreSQL. If you are using Visual Field for offline asset management, this SQL Dump option may be useful to send you offline edits to a central database.
However, SQL Dumps are perhaps not as robust as Export Dumps for a number of reasons.
If you experience this error during the Import phase (there was not enough remaining storage space, or the storage quota was reached and the user declined to allow more space), it may indicate that a request for the initial WebSQL database storage was initially successful however during actual allocation, your browser is enforcing quota restrictions on the amount of usable storage/disk space on your device for TEMPORARY objects. Common quota restrictions for Chrome and Opera on Windows and Android platforms for TEMPORARY objects (ie, WebSQL database) may mean that it is necessary to have up to 10 times the required database storage space available. Such quota restrictions commonly apply the following rules; 1) only 50% of remaining storage/disk space on your device will be considered in the usable "pool", 2) of the usable "pool" a single web page may only be permitted to use a maximum of 20% of the pool. In other words, you may need up to 10 times free space on your device that what your Offline Visual Field WebSQL database will occupy.
Whilst Visual Field is intended to be able to function as a stand alone file (via the file:// protocol context), it is also capable of operating as a true offline web application. To do this, it will be necessary to place the optional vf_service_worker.js file into the same directory that you have placed vf.html on your web server. Please see instructions at visualfield.org on how to download and use the vf_service_worker.js file. The optional vf_service_worker.js file needs to reside in same directory that you are hosting vf.html to make it into an offline accessible web application.
Copyright (c) 2019-2020 Harris Hudson All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Omit Metadata Objects
Export Object Failures:
Import Processing File Line:
Import Failed on File Lines:
Omit Metadata Objects
Export Object Failures: