Visualization of 40M Cell Towers


OpenCelliD is the realm’s largest
commence database of cell towers with a license
CC BY-SA 4.0

Recordsdata has paunchy world protection and freely accessible for
bag.

This tabular recordsdata[1] has ~40 million rows and 6 columns in it but supreme 3
columns (latitude, longitude, and variety) are extinct in this visualization.



[1] Building of enter recordsdata.

Above desk recordsdata is read line by line and grouped by their say recordsdata.
Grouping is accomplished as within the event that they had been XYZ Tiles.

First, empty 2nd integer arrays for every tile are created and its facets are
initialized with cost=0.
These values defend a counter that reveals the different of rows that
geographically contained in that pixel.
Every row increases the associated counter by one[2].
After iterating all rows with this form, we stop up with a bunch of
integer values (in varied 2nd arrays) that signify row counts per space.




[2] Every row increases associated counter.

In tell to enact metropolis-level decision for visualization, tiles as much as
zoom level 9 could perchance quiet be generated.

For XYZ tiles, the different of tiles per zoom level is 4^zoom.

Therefore, for zoom ranges between 0 to 9, the different of tiles that can quiet be generated
turns into 4^0 + 4^1 + 4^2 + .. + 4^9 which equals to 349525.
Every tile has 256×256 integer values.
Even with the utilization of 32 bit-sized integer values,
one tile occupies 256 256 32 bit=256 kilobytes of storage,
and 349525 tiles occupies 87 gigabytes of storage which is
positively no longer acceptable.

To resolve this dimension enviornment every tile blob is compressed independently
with a latest compression algorithm
brotli.
Despite being slack to encode, brotli has a rapidly decoding tempo and high compression ratio
when put next to older compression techniques relish gzip. Additionally,
brotli-compressed-recordsdata is
natively decodable with all smartly-liked browsers
.
Since there are no cell towers in oceans or uninhabitable areas, tiles for those areas
are paunchy of zero values or very sparse. Due to the this high redundancy,
compression performed extraordinarily smartly for those areas (256kb compressed to 20~100bytes).

In tell to simplify the algorithm, it is described as if there was supreme one
counter cost in every pixel, but there are 4 counter values per pixel every
preserving different of row counts for numerous cell tower varieties[3]




[3] Every pixel shops better than one channels (counter)

Even though these tiles are saved relish XYZ tiles, they are no longer
genuine XYZ tiles.Since these blobs are correct brotli-compressed-integer-arrays.
Visualization requires a blob to image conversion. This conversion is
accomplished by client facet javascript and HTML5 Canvas API.

Underlying device displaying library in this visualization is
leaflet.
Kudos to leaflet author(s) for writing the form of smartly documented and extendable
device library, I was ready to write a custom tile layer for leaflet that
intercepts the image tile requests made by leaflet and redirects those
image tile requests to an image generating (client-facet) web employee that
works relish a tile server. Internet employee follows a straightforward algorithm for image technology.

It reads the row count values pixel by pi

Read More

Recent Content