Explore, Process and Visualize your data right in your browser.
November 2023 Update
As of Chromium version v119 (November 2023), WebSQL appears to now have been deprecated.
In order to keep your copy of Visual Field still functioning, you might consider to add
a Chrome "Deprecation Trial Token" API string to your http headers (eg, in your .htaccess file)
when serving Visual Field from a web server, or to modify your copy of vf.html to add an
appropriate meta tag to include a token for
your domain. You will need to do this in order to keep your copy of Visual Field still
functioning in Chromium based browsers. Please search; "Google Deprecation Trial Token"
and register under "WebSQL" for an API token for yourself. Then setup your API token accordingly.
You will need a Google
account in order to do this. Chromium version v123 (due March 2024) appears to be the final
browser release date that WebSQL will be supported in this way. Going forward - What does this
mean for Visual Field?
The immediate impact is, as from now, and in to the future, will be that running Visual Field
as a stand alone html file (eg, loading in a file:// protocol context) will probably no longer
work. Harris Hudson, the author of Visual Field, is currently assessing the proposed
alternative solution of using SQLite directly as a wasm/javascript port along with local
offline file storage. This will be a significant exercise to analyse, assess the feasibility,
refactor and add additional code to the existing 23,500 line code base that makes up Visual
Field today. If a SQLite wasm/javascript alternative port can be leveraged, then a new release of
Visual Field will be targeted hopefully for some time in 2024. Realistically however, this will be
unlikely before March 2024. Having said that, if the alternate SQLite wasm/javascript
engine does indeed work in a true offline sense - then future possibilities may be encouraging;
including broader browser support and access to more SQLite functions within Visual Field.
Stay Tuned.
Run Now or Download
Current Version (3.4.0) Run Now; vf.html,
Download; (1013 kB, 164 kB gzipped)
For these examples, your Visual Field metadata is not reset and will just be added to cumulatively. Reseting your metadata between each example, may make it easier for you to understand individual example configuration. Go; Dictionary Reset Metadata Reset.
Press for a Help Page about each example.
What is Visual Field
Visual Field is essentially an Open Source single html file - vf.html, with some other associated files to support different usage modes. It enables you to consume and process delimited data files and web services (eg; CSV).
It is driven by declarative Metadata via the web interface - No HTML or Javascript
programming is required (although SQL may be useful).
You can apply SQL statements, numeric, statistical, text and spatial functions against
your datasets.
Tasks can be arranged to run as a predefined sequence based upon page load or
other user interactions. You can make ETL driven workflows.
Table, Chart, Map, HeatMap, Media and Network Visualizations
can easily be made to present your data.
Visual Field is easy to get up and running. It is intended to function either as a
stand alone file (just download it and you are ready to go - no IT skills and no zip file
extraction is required). Or it is intended to be able to be served from your webserver -
either as a stand alone web page or as an embed (eg, within an iframe). Provided the additional associated files are also included on your web server, in this way, end users can use Visual Field as; an Online Web Application, an Offline Web Application, or an installed Progressive Web Application.
With Visual Field you
can process your datasets, or other remote CORS enabled datasets, directly in the browser -
no back-end or cloud database is required. It's truly a server-less database.
Visual Field
has features to enable the extraction, re-import, and runtime use, of your Metadata, Export Dumps and SQL Dumps.
So you can share your data driven workflows and visualizations with your colleagues or
put them on your website.
You may find Visual Field useful if you are; a Data Analyst,
GIS Analyst, Data Scientist, Report Writer, Web Developer, Teacher, Student or anyone who may wish to
create data driven presentations or rapid prototypes.
Change Log
Version 3.4.0 (Current Version) - Released September 2021
Gracefully cater for database objects that have no ROWID field
Not all current browsers support HTML5 and WebSQL. And of those browsers that do, not all
underlying devices support a WebSQL database when invoked as a standalone html file.
The browsers currently supporting a fully WebSQL database is still somewhat
fairly dynamic and changing (even as at 2019). See "Get Involved" section below.
Application Usage
Visual Field is intended to be able to be used in any of the following ways;
For an online web application, just install the vf.html file in your intended web server directory.
This will allow Visual Field on your web server to be used in an online way.
Offline Web Application
For an offline web application, you will need to install both vf.html and the vf_service_worker.js
files in your intended web server directory. This will allow Visual Field to be loaded in a browser even if your
users do not have an internet connection (after initial load). Both the vf.hml and vf_service_worker.js will
need to reside in the same directory.
As a stand alone html file
Visual Field can function as a stand alone html file just by loading the vf.html file in your
browser using a file:// protocol context. That is, no web server is necessary to use the full functions
of Visual Field. This usage option may be useful if you are unable to install Visual Field on a web server
or unable to install it as a Progressive Web Application.
As an Installed Progressive Web Application
Progressive Web Applications are intended to be able to be installed on your device by using HTML
component "Add to Home Screen" (A2HS). As of version 3.1.0, Visual Field is typically able to be installed
on your device by a simple click on the "Install" option under the "Help" tab. This enables a quick,
simple, non-technical install of Visual Field on to your users device. In order to support Visual Field
as a Progressive Web Application, you will need to place vf.html, vf_service_worker.js,
vf.webmanifest and the favicon.svg all 4 files on to your web server in the same directory.
Should all these files be installed on your web server, and your web server can serve these over https,
then end users should be able to successfully invoke the "Install" option, under the "Help" tab, to add Visual Field to their
device as an installed Progressive Web Application. Ie, a simple, non-technical, install on their device.
Optional Offline Web Configuration - Example.
The vf.html file is intended to be able
to be loaded as a stand alone file (from a file:// protocol context) or
served via a web server. If you do serve it via a web server, you may wish to
download and add the optional vf_service_worker.js
file and place it in the same directory as vf.html.
This will turn your served version of Visual Field into a true Offline Web Application
in that it can be reloaded and used in a browser (from a https:// protocol context)
without needing internet access. Something to consider if you make presentations,
or you may be an emergency worker out in the field, with unreliable internet access.
To demonstrate the Offline Web capability, follow this scenario;
Harris, the author of Visual Field, will be giving a short talk
about Visual Field in this years FOSS4G 2021 online meetup - with this years focus on Buenos Aires Argentina.
Starting with a brief introduction, then a short demonstration, and
finishing with what you can do - how to get involved. Whether you are
technical or non-technical, if you work with Open Data, data manipulation
and visualizations, you may find this talk of interest to you.
A link to the video of the talk will be placed here after September 30 2021.
Hope to see you there - September 30 2021. To watch the video - see Tutorial "FOSS4G 2021" above.
Get Involved - What Can You Do - Raise Your Voice
WebSQL (browser based SQLite database variant) has had an uncertainty cloud overhead for it's whole lifespan since it's introduction in 2009. There was a W3 impasse, in relation to standards support of WebSQL, in 2010 that persists today. Some browsers support WebSQL, some do not, some browsers have it as an experimental feature, some browsers removed the full text search (FTS) capability from WebSQL over security concerns, some browsers don't support outer joins, some browsers removed support in third party contexts, some are intending to remove other features of WebSQL, and some have future plans to deprecate and remove it entirely. WebSQL is becoming mangled. The Web Ecosystem has continued to evolve since 2010 - things like CORS HTTP headers, the ubiquity of javascript visualization frameworks, and the ubiquity of Open Data CSV datasets - now makes browser based processing of relational style data more important than ever. WebSQL isn't perfect. But to remove WebSQL now from browsers without replacing it with a standards based alternative SQL engine would be a real travesty. Half of the worlds population now have a smartphone (as at 2021) and more than 80% do in 2023. In many use cases now (for processing Open Data) - WebSQL means there is no longer a need for back-end, or cloud, databases for processing datasets in many cases. For those browsers that support it, WebSQL brings these benefits over other browser based storage mechanisms;
It's embedded with Zero installation
It can handle big data (Web Storage cannot for example)
It's incredibly fast to load, index and query
It can interpret end-user input as SQL (ie user input to interact
with the storage layer). IndexedDB cannot for example - that's
javascript based - and not everyone is a javascript programmer
It can perform exceptionally well for an arbitrary relational join
query (multiple tables). IndexedDB cannot for example which requires
data extraction to be implemented as code (javascript)
It performs incredibly fast overall. An SQL framework built upon
another storage model will NOT. Eg, SQL on top of IndexedDB will not match the
performance of WebSQL
WebSQL supports the construction of views (not just tables) Eg;
non-persistent stored queries
If you can see the benefit of WebSQL, or an alternate standards based SQL engine built right into the browser - to process your relational style datasets (eg; CSV) - then you need to get vocal with the developers of your favourite browser NOW!
It has taken a significant effort to develop this application.
If you find it useful in your project - consider making a donation.
Donations will go directly towards continued development efforts.
Whilst there remains incomplete browser support, and incomplete
standards support for WebSQL, it's not really possible to provide any sort of
product support but its hoped that you still find this application useful
none-the-less. Thanks for your support. Donation and Sponsorship opportunities are available by contacting the author directly.