This post continues my Mental Health Awareness Week project. Refer to the original post for background information about the goal of this project.
After deciding to build a box for each facility, I realized I never actually had a unique ID column in my source spreadsheet. That means I can't maintain a routine import/update from that spreadsheet. So I need to find an alternative source of meta-data.
Pressing F12 in Chrome lets me inspect what's happening when I use the SAMHSA site. Reloading the page and running the search for Florida again lets me see all of the end-points (web URLs) this User Interface touched. Hovering over the entries gives me insight into the nature of the network traffic..
I can see with this one entry that Chrome received back a listing of facilities in Json form. It was made over a secure connection and took 534 ms seconds to retrieve. The response tab provides me with the full content of the request's response, which I copied to my clipboard.
I then opened the Online JSON Viewer website in a new tab, pasted the response, and clicked the viewer tab. It gave me a good, quick visualization of this data.
I've found my unique ID in the "frid" property of each "row" of the results. I also can see that there are 946 results spanning across 32 pages of results. I can also see that there are properties on the rows which are misleading, such as: _irow, _rclass, rnum. I should really take note of that or ensure my solution allows me to exclude the visibility of those properties.
I copied the URL for the original network event and removed the paging feature. That results in a single web request for all the Florida facilities being returned in 4.3 seconds. This will become my source of facility boxes (for the foreseeable future).
After flipping over to DataPort I realize I don't have any way to routinely import from Json. No worries! I took my last DataPort ImportDataFormatter and whipped up a new Json importer.
I've now got some much needed functionality:
- I can select Json as a potential import source
- I can specify a webservice end-point
- I can specify a path to the records in the Json data
- I can build up a transformation map between the Json data and fields in Content Manager
- I can prevent the exposure of properties from the Json data
- I can format data to match CM requirements
When I run it via DataPort it behaves as expected....
The boxes created during the initial import will now be updated or added to, without me having to do anything. Furthermore, I can now import data from any Spreadsheet or Webservice I come across. And all this cool DataPort work has me thinking bigger.
I'm now ready to start finding records to place within these boxes. Before I start visiting websites of regulators, facility operators, and governmental bodies, I worry how I'm going to capture all the things I come across. I might need some new tools to help me import in bold new ways......