How to use¶
- User Manual
- Mapping User Manual
- Geocoding Data
- KML File Upload
- Data Classification Methodology
- GeoIQ Analysis
- GeoIQ Geocoder
- What is Geocoding?
- Required Geographic Data for GeoCoding
- What can the GeoCoder not GeoCode?
- Preparing your CSV for GeoCoding: Single Field and Separated Fields
- Saving my Excel Spreadsheet as a CSV
- Uploading your CSV and GeoCoding
- Formatting Your Data During the GeoCoding Process
- Address Attributes
- Geographic Attributes
- Standard attributes
- Review your data
- Identifying GeoCoder Errors
GeoIQ built the geocoder powering GeoCommons and GeoIQ appliances in order to provide users with an easy way to georeference their data from spreadsheets. The Geocoder works in the United States to the street level using the US Census road data and internationally to the city level using the Geonames gazetteer.
The GeoIQ Geocoder is open-source and available from our repository at http://github.com/geocommons/geocoder.
What is Geocoding?¶
Geocoding is the process of finding geographic coordinates from other geographic data, such as street addresses or ZIP codes.
The built in geocoder within GeoCommons makes it simple to retrieve longitude and latitude coordinates from your locations based CSV files. The geocoding process has been fully integrated into the upload process, and will retrieve coordinates based upon the location based values found in your CSV file.
Required Geographic Data for GeoCoding¶
There are many types of geographic data out there but the GeoCommons GeoCoder will only accept certain types of geographic data for GeoCoding purposes. By having these data types in your CSV, the GeoCoder will be able to take your uploaded data and assign latitude and longitude coordinates to the data for mapping purposes. Below are the accepted data types that will work in the GeoCoder.
- Postal Code/ZIP Code
- City, State
- City, Country
- City, State, Postal Code/ZIP Code
- City, Postal Code/ZIP Code
- State, Postal Code/ZIP Code
- Street Name, City, State
- Street Name, Postal Code/ZIP Code
- Street Name, City, Postal Code/ZIP Code
- Street Name, City, State, Postal Code/ZIP Code
- Street Name, State, Postal Code/ZIP Code
- Street Address, Postal Code/ZIP Code
- Street Address, City, State
- Street Address, City, Postal Code/ZIP Code
- Street Address, State, Postal Code/ZIP Code
- Street Address, City, State, Postal Code/ZIP Code
- Full or Complete Address
What can the GeoCoder not GeoCode?¶
If your CSV does not have one of the above combinations you will receive an error during the upload process into Finder! Below are some unaccepted combinations that will cause an error:
- State (by itself)
- City (by itself)
- Street Name (by itself)
- Street Address (by itself)
- Street Name and State
- Street Name and City
- Street Address and City
- Street Address and State
Be sure your street addresses do not contain PO Box numbers. The GeoCoder is not able to GeoCode such locations and will give latitude and longitude coordinate results based on the postal code/ZIP code given in the CSV.
Preparing your CSV for GeoCoding: Single Field and Separated Fields¶
So how do you make a CSV with geographic data ready for GeoCoding? These tips can help you learn the correct methods when preparing a CSV for GeoCoding.
When formatting your CSV for upload, you can either breakdown your file into basic fields or you can place them together in a single field. Below is an example of a CSV that has been broken down into parts:
Please note that each row of your spreadsheet will represent one point location to the geocoder and eventually any maps created from your data.
As you can see above the address/city/state/country/ZIP code fields are separated into different columns. When the GeoCoder receives this file it will take the separated fields, recognize their geographic types, combine them appropriately, and then GeoCode for latitude and longitude results.
Your geographic data can also be combined into a single field. See the example below:
When combining the data into one address form, there are a few things to keep in mind to allow for the best possible GeoCoded results.
- Use commas to separate the different geographic fields
- Keep the fields consistent
- Use same separators (commas) in all attribute entries
- Use same geographic types in all attribute entries
By following these formatting guidelines, you will be able to successfully GeoCode your data in the GeoCommons GeoCoder.
Saving my Excel Spreadsheet as a CSV¶Once you have properly formatted your spreadsheet you will want to save it as a CSV file. This can be done following these steps in Microsoft Excel:
- Select the Save As option from the Excel options dropdown.
- Choose the location you wish to save your file to.
- Title your dataset in the File Name: field.
- Select the Save as type: CSV (Comma Delimited) option from the drop down menu.
- Finally click the Save button.
Please note that when saving a CSV file multiple sheets cannot be saved. After clicking the save button you will be prompted with this information. Please make sure that you do not have any important data on any sheets other than the one you wish to save.
Uploading your CSV and GeoCoding¶
Formatting Your Data During the GeoCoding Process¶
The first step in the GeoCoding process is to choose to GeoCode based on an Address or Place Name.
From there you will guided through a step by step process where either GeoCommons will easily recognize your address in which case you can choose to Continue or you can direct the GeoCoder as to what data should be used in the GeoCoding process by chooseing select different columns.
Later in the process you will have the opportunity to edit any data that was not GeoCoded properly.
If you choose to select your own columns for GeoCoding, you have the opportunity to format each data column in order to specify which type of address data you have.
For each column you can specify the type of data by choosing from format options in Addresses, Geographic and Standard. Your data will be classified by default but you can change it if a different class is more appropriate.
The Address option should be used on all of your data that should be used to GeoCode. By selecting any of these options, you tell the geocoder to include this information in the geocoding process.
- Full or Partial Address: You can use this option for any of your address fields. This is more of a general catch all option.
- Street Name: Use for your attributes containing just a street name.
- Street Number: Use for the attribute containing the street number.
- State or Province: Use for the attribute containing State or Province information.
- ZIP or Postal Code: Use for the attribute containing the Zip code or equivalent.
- City: Use for a city attribute
- Country: Use for a country attribute
The geographic attributes can be used if you already have geographic information in your dataset such as latitude and longitude.
- Latitude: If you already have Latitude coordinates for your CSV file, select this option for your Latitude Attribute.
- Longitude: If you already have Longitude coordinates for your CSV file, select this option for your Longitude Attribute.
- PostGIS EWKT
- PostGIS EWKB
Standard attributes¶Please use the standard options for all of your non geographic data. Your standard options are as follows:
- Plain Text: Use this option when for of your text based entries. For example, a store name is a plain text field.
- Whole Number: This option should be used for all of your numerical entries that do not require a decimal degree (rankings for example).
- Decimal Number: Please use this option for all your numerical entries that are in decimal format. By selecting this type of data, the decimal places will be preserved. This can be very important when dealing with statistics that require a fine degree of accuracy.
- Time or Date: Select this option for all of you Time or Date entries. This will allow you to animate your map later.
Review your data¶
As mentioned above, once your file has been GeoCoded, you will have the opportunity to review the results and edit the data of any features that were not correctly GeoCoded.
- Select the Save button.
- You will be taken to the Review screen.
- Please review your data to make sure that everything is correct. You have the option to expand the window to see the entire table by selecting the Expand View option.
- Two Columns of data will be added to your geocoded layer: Precision and Score
bq. Precision – this label shows the level at which the data was geo coded to. Street level has the finest granularity. If the precision is at Zip, City, or State, the location was geo-coded to the centroid of that geography.
bq. Score (Geo-confidence score) - An estimate of the probability that the latitude and longitude assigned by the Geocommons / GeoIQ geocoder to a place name correspond to the place the user intended. 0.0-1.0 scale, 0 zero being the least confidence, 1 being very confident.
- Select Continue when you are satisfied.
Identifying GeoCoder Errors¶
GeoCoding large CSV files may be challenging for the GeoCommons Geocoder. Our GeoCoder has a limit of 5,000 rows.
The GeoCoder is able to report to you the number of attribute records that were GeoCoded and those that were not. Your overall match and unmatched totals will be recorded at the top of the page.
After identifying the mistakes, you can double click on the cell with the error and correct it. Click Update for each cell. When you have made all of you corrections you can click Save & Resubmit. The GeoCoder will go back and resubmit the corrections and hopefully correctly GeoCode those features.
If you have received errors or a large number of you entries have not been GeoCoded, we strongly recommend looking back over your CSV that you have uploaded. Check to make sure that the data is formatted cleanly. Also check to see that the data is consistent.
Once your data is correctly GeoCoded, click Continue. Finally, Describe you data providing the appropriate metadata. Make sure you site the GeoIQ GeoCoder as the geocode service used. Save, and now you can map your points. Please see the Describing Your Data portion of this manual for more information regarding this process.