We are happy to see the wide range of OpenStreetMap in Government outlined on the OpenStreetMap.US blog.

There are lots of opportunities to improve the data set and make your mark on open data, while improving available map data for everyone.

For those with little background with the organization, it didn't start in the US, but in Europe, where most Geospatial data was (at the time) locked up behind huge licensing restrictions. To combat this, founders and an army of interested people fanned out over the continent and later the globe to create map data which was free of those restrictions and open to the public.

This release ended up taking a bit longer than we expected as we have been working hard to fix the bugs reported in 1.4 and 1.4.1.    But, we haven't just been working on fixing bugs, we've also been enhancing the capabilities and performance of the existing capabilities of Cartographica, with improvements in everything from WFS compatibility to execution speed and progress information for many of our analysis tools.

I won't take up the space here to enumerate all of the changes, as they are covered almost exhaustively in the release notes, however I will point out a few of the areas we have been working on.

Analysis Tools

We introduced a lot of new tools in version 1.4, and we have been making improvements to the performance and interface for these.   In particular, almost all of the analysis tools provide more progress information now and may be cancelled.   For those who work in more than one window at a time, Cartographica no longer makes you wait in all windows while analysis is performed for one window.   This means you can continue to work on another map while performing more complex analysis on a map in the background.   All search and overlay operations are substantially improved and we are continuing to follow additional optimization paths (while maintaining accuracy).

Geocoding

Because of the Bing geocoder service that we also support, many folks don't use the built-in geocoder.  However, for tasks involving historical data, or large amounts of data being proccessed in a small area, it's often a faster mechanism (and it doesn't require a network connection).  In this release, we've significantly improved the handling of whitespace, negative address ranges in the streets files, additional abbreviations, and support for streets whose names begin with "Ste".

File Import/Export

Most of the changes to file import involved upgrading to the most recent libraries for external file formats, such as MrSID®.  These should improve compatibility and performance.

For export, we've fixed a bug involving line style export to Illustrator and enhanced the Save window to include file extensions that appropriately default for the selecte file type.

ESRI File Geodatabases can now receive the entire map instead of being broken into a set of GDB files per layer.

Live Maps

The big change for live maps is that Custom OSM layers can now point at basically any tile source that uses a similar format to OpenStreetMap but isn't actually an OpenStreetMap server.   For those of you with access to tiled services from an ESRI ArcServer map, this means that you can access those tiles from Cartographica as well. For an OpenStreetMap server, you can just use the URL of the service in the box.   For other services, you'll need to use a more complete URL with {zoom}, {x}, and {y}, representing the zoom level, x tile id, and y tile id.   Here are some examples:

For ESRI ArcServer maps, end the URL with {zoom}/{y}/{x}, so if the base URL of the map is "http://myserver/ArcGIS/rest/services/MyMapLayerName/MapServer", the URL you would use is: "http://myserver/ArcGIS/rest/services/MyMapLayerName/MapServer/tile/{zoom}/{y}/{x}".

For Google-style maps, the URL is more complicated, so if the base URL of the map is "https://mymapserver.com/foo/x=9&y=23&z=6&scale=2&s=Ga", the URL you would use is: ""https://mymapserver.com/foo/x={x}&y={y}&z={zoom}&scale=2&s=Ga".

At the moment, all of the key services are using the same CRS, which is the "Web Pseudo-Mercator" and thus Cartographica will automatically determine the right information and display it correctly.

WFS Layers

We continue our improvement of support for WFS layers.    This release includes better handling of WFS 2.0.0 servers (such as most recent versions of GeoServer) as well as backwards compatibility with older 1.1 and 1.0 services.   We also fixed a bug in handling single polygon layers in GMLv3 layers (multipolygon layers were fine).   

The Web Feature Layer Attributes window (available from Layer > Web Feature Layer Attributes... and when you add a new web feature layer using File > Add Web Feature Layer...) has been enhanced to allow limiting of requests to the server (necessary for some servers that don't know their own limitations) and the introduction of manual filters.    We hope to make filters easier in future versions, but for now if you know the XML filter language (or can have somebody work up a filter for you), then you can paste that directly into the Filter box and it will be applied to layer requests.

The Future

We have many plans going forward, including improvements to our existing features and additional features for analysis, presentation and exploration.   We sincerely appreciate your support.

According to iLoveMountain.org, "Mountaintop removal is a relatively new type of coal mining that began in Appalachia in the 1970s as an extension of conventional strip mining techniques. Primarily, mountaintop removal is occurring in West Virginia, Kentucky, Virginia and Tennessee. Coal companies in Appalachia are increasingly using this method because it allows for almost complete recovery of coal seams while reducing the number of workers required to a fraction of what conventional methods require."

Part of the problem with mountain top removal is that the processes is destructive to the environment and the local habitats in the area where the mining is occurring. "Before mining can begin, all topsoil and vegetation must be removed. Because coal companies frequently are responding to short-term fluctuations in the price of coal, these trees are often not even used comercially in the rush to get the coal, but instead are burned or sometimes illegally dumped into valley fills" (iLoveMountains.org). Addittionally, "While reclamation efforts such as stabilization and revegetation are required for mountaintop removal sites, in practice, state agencies that regulate mining are generous with granting waivers to coal companies. Most sites receive little more than a spraying of exotic grass seed, but even the best reclamation provides no comfort to nearby families and communities whose drinking water supplies have been polluted and whose homes will be threatened by floods for the hundred or thousands of years it will require to re-grow a forest on the mined site" (iLoveMountains.org).

The problem that iLoveMountains.org is attempting to combat is inherently a spatial problem, and due to this fact they disseminate a lot of interesting spatial data to help promote additional research and understanding by the public. To help highlight the problem that they are pointing out this blog post uses a number of the dataset provided to create maps and help visualize the problems. To download the iLoveMountain.org data visit their data download webpage. 

Before importing any of the data, first add a Live Map by choosing File > Add Live Map. This will provide a basemap and will allow you to see what the mountain top removal sites look like based on satellite imagery. Additionally, a shapefile of U.S. States was imported by choosing File > Import Vector Data to more clearly show the state boundaries in the area. Download the states shapefile at the U.S. Census website. Once the basemaps are imported you can import each of the shapefiles provided by iLoveMountains.org by choosing File > Import Vector Data.

The first map below shows all of the mountains that are being or have been mined in the Appalachian region. This is the NRDC_500_Mountains layer. The mountains have been color coded to indicate the states that they are within. To color code the mountains double-click on the mountains layer in the layer stack, change the based on option to state, click on the gear box and select Distribute Unique Values (4), and then assign colors to each of the categories. 

The next image shows all of the active mines in the region. This is the Active_Mine_Sites layer. The large red points are the active mine locations.

The next image is a Kernel Density Map of the Active Mine sites. To create a KDM select the Active_Mine_Sites layer in the Layer Stack and then while holding down the option key choose Tools > Create Kernel Density Map. Select the Visible Area option and then click Analyze. 

The next image is a look at the satellite imagery of the largest hot spot located in Southeastern Kentucky. Notice the large gaps in the tree canopy. 

And a closer look at a large mine near Lamont, KY.

The final maps show the mines as digitized polygons highlighting the amount of area taken up by the mines. This is the Skytruth_Mines_Merge layer. 

After downloading the DEM import the file into Cartographica by choosing File > Import Raster Data. Once the DEM is imported into Cartographica you can create a contour map that will provide the needed elevation data. To create a contour map choose Tool > Create Contours. Set the increment value to 5. In the case we want highly accurate elevation data so we will use a small increment. Set the base value to 0. See below for an image of the create contour windows. 

Next, we need to create a layer showing the location of the greens on the golf course. To do this we can use satellite imagery and the Add Feature tool. Choose File > Add Live Map. Select the Bing Map layer in the Layer Stack and the choose Layer > Include in Map Extent.  Uncheck the K42 layer in the Layer Stack. Zoom in to the Lower Left Corner of the Contours layer and look for High Point Golf Club. Once you find the golf course uncheck the contours layer. Choose Layer > New Layer and then Edit > Add Feature. Select to Add a point layer and then add points on top of the greens. See below for an example of the add points process. 

See below for an image of the full set of points. 

We can highlight the general elevations of in the area by showing color differences for the various elevations. Double-click on the contours layer in the Layer Stack to bring up the Layer Styles Window. Change the based on selection to Elevation. Add four categories by clicking on the + button four times. Choose Window > Show Uber Browser and then click on the palettes tab. Select a palette and while holding down the option key, click and drag it to the table in the Layer Styles Window. See below for images of the Layer Styles Window and the Contours. 

And the Contours

Finally, we can perform a spatial join to join the elevation data from the contours layer with the greens points layer. This join will add a new column to the greens points that contains the elevation data so that the analyst can determine the elevations for each of the greens. To perform the spatial join choose Tools > Spatial Join. Select the Closest option and type in 50 meters in the distance box. The distance value is used to specify a search radius for the Closest operation, which will select the nearest feature in the Contours layer and use it as the feature to be joined. The Closest operation is preferred over the Within Distance operation in this context because the Within Distance operation will find all matches and then randomly select a value where as the Closest operation will only find the nearest feature. Uncheck the discard unmatched features box. See Below for an image of the spatial join window. 

Finally, using the Layer Styles Window you can show the elevations of each of the points using the Item Labels. See below for the final image. 

The first map below shows the shale basins within the U.S. These are areas where there is enough shale gas fracking to occur. To download this data click on the EIA link above and then scroll down about half way down the page until you see the "Geospatial Data in Shapefile (.shp) Format". Click on the the Shapefiles for Basin Boundaries under the Data for Shale Plays Map heading. To import the data choose File > Import Vector Data. To add the live map as a basemap choose File > Add Live Map. 

 The second map below show the areas known as Shale Plays, which are areas that have shale gas currently being harvested using fracking. The areas being harvested are dark red.

Recently, fracking made the news when NASA released satellite imagery highlighting the growth in fracking in undeveloped areas around the country. Specifically, NASA highlighted the growth in nighttime lights in the North Dakota region, which is a relatively undeveloped part of the county. Despite the low levels of development there are significant nighttime lights visible in areas where shale gas fracking operations are underway. Download a Geo.tiff image created by the Visible Infrared Imaging Radiometer Suite (VIRUS). Once you download the image you can import it into Cartographica by choosing File > Import Raster Data. Below is an image showing the geo.tiff with the Shale Plays layer. 

And a closer look at the increased nighttime lights due to fracking operations in North Dakota. 

If you have followed this blog at all you might have notice that I have a light obsession with all things military. Part of the reason is that military objects (especially naval) make interesting maps. Over time I have determined the locations of several interesting military ships and before Bing Maps gets updated I wanted to provide a look at these locations while also conducting some analysis and highlight a few of the functions of Cartographica. Below are a few descriptions of some of the most recent additions to the United States and Chinese Navies. 

The U.S.S. Gerald R. Ford (CVN 78): 

The U.S.S. Gerald R. Ford is the newest addition of the U.S. Navy Aircraft Carrier and is the first ship in the new Gerald R. Ford Class of supercarrier. The new carrier comes at a cost of $13.5 billion and it includes numerous improvement over past classes of carriers. One of the biggest improvements is the aircraft launch system which moves from steam power to electromagnets. Like other nuclear powered carriers the GRF will have an unlimited service range for a period of 25-30 years and will only need to come to port for supplies and regular maintenance. The ship is being built by Huntington Ingalls Industries in Newport News, VA. Below is an image of the port where the ship is being constructed. Like a game of ISpy, do you see the ship?

To highlight the location of the of the GRF add a new feature by choosing Layer > New Layer and then Edit > Add Feature. Select to add a polygon feature and then trace the outline of the ship. 

A closer look at the U.S.S. Gerald R. Ford under construction.

Chinese Aircraft Carrier: Liaoning

Recently the Chinese military acquired an 67,500 ton Soviet era aircraft carrier and has spent the past several years refurbishing and upgrading the ship to make it battle ready. Last November, China landed its first plane on the surface of the Liaoning. See this CNN video of the plane landing. Below is an image of Dailan, China where the Liaoning has been under construction for several years (its has completed tests in the Yellow Sea). Again, like a game of ISpy, do you see the ship?

We can again create a new feature to show the location of the ship choose Layer > New Layer and then Edit > Add Feature. Select to add a new polygon feature and then draw the outline of the ship. 

A closer look:

Based on the polygons that we have create we can use Cartographica's table tools to help enhance what we know about the ships. First, add a new area column to each of the new polygons so that we can see the size of each ship's deck space. To add the area column choose Tools > Add Area Column. Based on this analysis the Gerald R. Ford has a deck space of 32,356 square meters and the Lioaning has a deck space of 27,494 square meters. Also, we can add polygon coordinate columns which will give us the coordinates for each of the ships. To add coordinate columns choose Tools > Add Centroid Coordinate Columns. The final map below shows the general location of the ships. 

In this post we emphasize the use of the new contour mapping functions in version 1.4 by using data from the Kentucky Division of Geographic Information. The data on the DGI website are free for download and include DEMs for the entire state of Kentucky.  To illustrate the contour mapping we will use data from the Middlesboro, KY area.  An interesting geological feature about Middlesboro is that experts believe that its location between Pine Mountain and the Cumberland Mountains is actually an ancient crater from an asteroid impact. This fact makes Middelsboro among the few cities in the world that is seated within an impact crater. 

In order to view the entire area for Middlesboro we actually need to download 4 separate DEMs. We need the cells U47, U48, V47, and v48 from the DGI link listed above. To download the DEMs individually control-click within each cell and then select Download Linked File as. Save the Files to your Desktop.   

To import the DEMs choose File > Import Raster Data. The following image shows what your map should look like. 

Notice the blue colored circular area near the center of the map. That is the location of Middlesboro and the fairly clear outline of an impact crater. Next, we are going to create contour lines for each of the DEMs that will show the differences in elevations throughout the map. To create contour lines select the DEM layer in the layer stack and then choose Tools > Create Contour Lines. This step has to be done individually for each of the DEM layers. A window will appear that allows you to choose the increments and the base. Select 50 as the increment and leave the base at 0. See below for the contour window. 

When the contour lines are created they will all be black in color. In order to enhance the visibility of the contours we need to adjust the color schemes. Double-click on the U47 DEM in the Layer Stack. Change the Based on selection to Elevation. Click the + button below the table 6 times and then click the Gear box and select Distribute with Natural Breaks (Jenks). Next, choose Window > Show Uber Browser, click on the Palettes tab, and while holding down the option key click and drag a color palette of your choice into the table within the Layer Styles Window. In the map presented below a blue-red scheme was used where blue = lower elevations and red = higher elevations. Repeat this process for each of the DEMs. See the images below for an example of the Layer Styles window and the map.

Layer Styles Window:

Contour Line Map:

The next image is a closer look at the Impact Basin. Also the following image has the DEM layers turned off and a Live Map added. To add a live map choose File > Add Live Map.

In the spirit of using street segments as units of analysis we thought highlighting how to use Cartographica to aggregate data points to street segments would be a fun and interesting exercise. Recent upgrades to Cartographica 1.4 include the ability to spatially join data based on their spatial association. This includes joining points to lines. 

The data being used in this example are from Washington D.C.'s GIS repository DCGIS. The data are 2011 crime incident locations and a street file containing all of the street segments in the city. To import the data into Cartographica choose File > Import Vector Data. Below is an image of the data described above. At first glance it appears as though the points are "on top of" the line segments. Observe. 

However, upon closer inspection you can see that the points are not directly on top of the lines. As a result, we will need to use a distance based spatial join to join the crime points to the line segments. 

Because we want to join the crime points to the street segments we first need to select the Streets layer in the layer stack. Once selected, choose Tools > Spatial Join. The following image shows the set up for the Spatial Join. Uncheck the Discard Unmatched Features so that the new join layer will have all of the street segments including those that have zero crimes. Note that the "Within Distance" operation is selected and a distance of 50 meters is the designated distance. Also note in the table that the 2011 crime data fields are set to ignore. The reason for this is that at this point we do not need all of the excess crime information. All we want is the counts of crimes on the various street segments. Finally, note that the last field in the table "Join Count" is selected to be copied. This will be the new field on the street segments layer that will contain the crime counts. 

After the points are counted and aggregated to the street segments you can reclassify the counts into groups and then vary the street segment colors by the groups. Double-click on the streets join layer in the Layer Stack to brig up the Layer Styles Window. Add three categories to the table by click on the + button. Click on the Gear Box and select Distribute with Natural Breaks (Jenks). Change the colors of each of the categories by clicking on the fill box and choosing a series of colors. See the map below for the final result. 

We decided to check out a few of the Open Data sources and wanted to create a few maps using Cartographica. The Philadelphia Open Data website was listed as the best Open Data website of 2012. We were able to download a .csv file for all of the crime incidents in Philly since 2006 from the website. Download the Philly Crime data.  

To start, add a Live Map to use as a baseamp by choosing File > Add Live Map. 

To geocode the data using Cartographica choose File > Import Table Data. The Import File window will appear. Select the coordinates tab in the top right. Change the Map to selection for the Point X and Point Y fields to X (or longitude) and Y (or latitude) and then click Import. See below for an example of the set up. 

*When we geocoded the Philly Crime data 164 incidents were geocoded to the Cincinnati area. Upon further inspection the points were geocoded to the appropriate location based on the data entries. The X and Y coordinates for all of the 164 incorrect points were, -84.69369121 longitude and 39.1207947 latitude, which places the points at the corner of Salyer Ave and Goodrich Lane just West of Cincinnati. These points were considered erroneous and were discarded. To delete the points select them using the identify tool and then choose Edit > Delete. 

The map below shows the geocoded crime incident points in Philadelphia since 2006. The Philly_Crime layer contains 592,064 crime points. 

You can filter the data by using the Filter bar. Change the Filter bar selection to Text_General_Code and then type in Homicide. The data will be filtered to only show homicide incident locations. Since 2006, there were 3051 homicides in Philadelphia. Use the identify tool to select all of the points and then hold down the option button and choose Tools > Create Kernel Density Map for Selection. The result of the KDM analysis is shown below. The red areas indicate places where homicide was highly concentrated.

A closer look at central Philadelphia.

Another way to filter the data is by date. The Philly_Crime layer contains a field called Dispatch_2 that contains the date of the offense. With the Philly Crime layer selected in the layer stack type in 12/31/2010. A small box will appear at the top of the Map Window. You can filter based on multiple criteria to limit the dataset to any year that you want. To select only the 2011 data use the settings shown below. Notice that we are selecting crime incidents that fall between 12/31/2010 and 1/1/2012. 

Use the identify tool to select all of the 2011 crime points and then choose Layer > Create Layer from Selection. The new layer should have 82197 crime points. You can repeat these steps so that you can create a new layer for each of the years in the data set. Create another Kernel Density map by Choosing Tools > Create Kernel Density Map. The result of the KDM analysis for the year 2011 data are shown below. 

In order to add more context to situation we have created a few maps to highlight the areas where the Patriot Missile systems are being deployed. The Patriot Missile systems are being deployed to Kahramanmaras, Turkey, which is about 70 miles North of the Syrian border. We can confirm the distance between Kahramanmaras and Syria by using Cartographica's Live Maps and measurement tools. See the image below for an example. To add a Live Map, Choose Tools > Add Live Map, and to use the measurement tools click on the measure tool button and then drag a line to determine the distance. 

We can also look at the radar and missile ranges of the Patriot Missile systems by using buffers. The radar used by the Patriot Missile system has a detection range of about 180 miles in all directions around the system. The map below shows how far the systems deployed in Kahramanmaras will be able to detect incoming SCUD missiles. To create the map you first need to add a new layer by choosing Layer > Add Layer. Add a new feature to the layer by choosing Edit > Add Feature and then select to add a new Point feature. Create a buffer for the point by Choosing Tools > Create Buffer for Layer's Features. Make the buffer for 180 miles. 

You can add to the map above by also creating a buffer for the effective range of the Patriot Missiles. The range is about 70 miles. Using the same steps as above create a second buffer for 70 miles. The map below shows the results.