geocommons / api watch
public
Forks2Right
Watchers14Right
Description: GeoCommons API
Home | Edit | New

Finder

Finder is a data repository where users can upload, find, and download data. The API makes it easy to programmatically tie in existing data stores, or updating feeds, into a Finder repository. In addition, the API provides for easy ways to search and download data.

Overall Capabilities

  1. Upload data
  2. Update metadata
  3. Search for data
  4. Download data

Resources

Finder has a number of RESTful resource endpoints you can use for searching, listing, editing, or creating new overlays.

  • /overlays.#{format}
  • /users/#{login}/overlays.#{format}
  • /searches.#{format}?query=#{searchQuery}

All endpoints support Atom publishing protocol (APP), as well as output via JSON.

Overlay

Here are some example representations of Overlays, or datasets, in Finder:

Atom

 <entry>
    <title>World Bank, Trends in Average Applied Tariff Rates in Developing and Industrial Countries, Global, 1981-2005</title>
    <link type="text/html" href="http://finder.geocommons.com/overlays/167" rel="alternate"/>
    <link type="application/atom+xml" href="http://finder.geocommons.com/overlays/167.atom" rel="alternate"/>
    <link type="application/vnd.google-earth.kml+xml" href="hhttp://finder.geocommons.com/overlays/167.kml" rel="alternate"/>
    <link type="application/atom+xml;type=entry" href="http://finder.geocommons.com/overlays/167.atom" rel="edit"/>
    <id>http://finder.geocommons.com/overlays/167</id>
    <updated>2008-04-29T03:20:52Z</updated>
    <contributor>
      <name>data</name>
      <uri>http://finder.geocommons.com/users/data</uri>
    </contributor>
    <category term="business" scheme="http://localhost:4001/tags"/>
    <category term="economics" scheme="http://localhost:4001/tags"/>
    <category term="finance" scheme="http://localhost:4001/tags"/>
    <category term="money" scheme="http://localhost:4001/tags"/>
    <category term="tariff" scheme="http://localhost:4001/tags"/>
    <category term="tax" scheme="http://localhost:4001/tags"/>
    <category term="trade" scheme="http://localhost:4001/tags"/>
    <content type="html">This dataset tracks the average applied tariff rates in both industrial and developing countries. Data is averaged for the years 1981-2005. Figures for 2005 have been estimated. Notes: All tariff rates are based on unweighted averages for all goods in ad valorem rates, or applied rates, or MFN rates whichever data is available in a longer period. Tariff data is primarily based on UNCTAD TRAINS database and then used WTO IDB data for gap filling if possible. </content>
  </entry>

Upload data

You can currently upload data using two methods: via files from your hard drive, or via a URL from the Internet.

Upload files

GeoCommons Finder accepts files via multipart form data uploads by posting to the /overlays endpoint. You can explicitly specify the file format you are uploading using any of: kml; shp,shx,dbf,(prj optional); csv; and rss/atom/xml. Just specify the individual files in an overlay[FORMAT]= parameter in the multipart upload.

An example of uploading a CSV file:

curl -i -X POST -u "yourusername:yourpassword" -F "overlay[csv]=@spreadsheet.csv;name=spreadsheet.csv" http://finder.geocommons.com/overlays.xml

An example of uploading a Shapefiles parts:

curl -i -X POST -u "username:password"  -F "overlay[shp]=@elect_precincts.shp;name=elect_precincts.shp" -F "overlay[shx]=@elect_precincts.shx;name=elect_precincts.shx" -F "overlay[dbf]=@elect_precincts.dbf;name=elect_precints.dbf" http://finder.geocommons.com/overlays.xml

Link to URL

Finder can register URL’s from the internet. To do so, provide the URL to a file type that is supported in GeoCommons and POST using the overlay[wild_url] parameter to /overlays.xml.

curl -i -X POST -u "yourusername:yourpassword"  -d "overlay[wild_url]=http://earthquake.usgs.gov/eqcenter/catalogs/1day-M2.5.xml" http://finder.geocommons.com/overlays.xml

URL’s can only be registered once in GeoCommons. So if the URL has already been registered by a user, you will receive a redirect to this existing Overlay. You can make a copy of that overlay if you want to modify the metadata.

Update metadata

Finder uses the AtomPub protocol for accepting modifications to Overlays. To modify the metadata, you must do an HTTP PUT to the overlay resource you are modifying.

cat updates.atom | curl -i -u "yourusername:yourpassword"  -d @- -X PUT http://finder.geocommons.com/overlays/9313.atom

updates.atom

<entry>
    <title>Proposed Stimulus Spending, US, 2009</title>
    <source>http://www.stimuluswatch.org/project/by_state</source>
    <id>http://www.stimuluswatch.org/project/by_state</id>
    <updated>2009-02-03T12:00:00</updated>
    <author>
        <name>SwimulusWatch.org</name>
        <uri>http://stimuluswatch.org</uri>
    </author>
    <content type="html">
        These are "shovel-ready" projects for which the mayors of this state have requested federal stimulus funding. Data was aggregated to each city, so if there were multiple projects for one city the number of jobs and cost would be for all projects in the city.
    </content>
</entry></code

The response will be an “OK” from the server:

 HTTP/1.1 200 OK
Connection: close
Date: Wed, 21 Jan 2009 22:19:59 GMT

Geocoding

For datasets without geographic coordinate information or if your latitude and longitude columns have non-standard names you must define the column types in order to be able to map. Currently GeoCommons supports U.S. geocoding only, but will be adding additional countries in the future.

Acceptable data types:

Standard:

string: Plain Text
integer: Whole Numbers
decimal: Decimal Number
datetime: Time or Date

Geographic:

address: Full or Partial Address
street: Street Address:
region: State of Province
postal_code: ZIP or Postal Code
city: City
country: Country
latitude: Latitude
longitude: Longitude

This is an example file with the various data-types defined.

 <?xml version="1.0"?>
  <entry xmlns="http://www.w3.org/2005/Atom" xmlns:gm="http://www.google.com/schemas/gm/1.1">
      <title>Overlay Title</title>
      <source>http://www.source.com</source>
      <id>http://www.source.com</id>
      <updated>2009-02-03T12:00:00</updated>
      <contributor>
          <name>Authorname</name>
          <uri>http://sample.com</uri>
     </contributor>
     <content type="html">
         Description of the dataset would go here.
     </content>
	 <gm:attributes>
	   <gm:attribute name='id' type='integer' />
	   <gm:attribute name='name' type='string' />
	   <gm:attribute name='percentage' type='decimal' />
	   <gm:attribute name='Date' type='datetime' />
	   <gm:attribute name='address' type='address' />
	   <gm:attribute name='street' type='street' />
	   <gm:attribute name='state' type='region' />
	   <gm:attribute name='ZIP' type='postal_code' />
	   <gm:attribute name='city' type='city' />
	   <gm:attribute name='country' type='country' />
	   <gm:attribute name='address' type='address' />
	   <gm:attribute name='lat' type='latitude' />
	   <gm:attribute name='lon' type='longitude' />
	 </gm:attributes>
 </entry>

A minimum amount of data is required in order to successfully geocode, below are column combinations that are acceptable:

Full/Partial Address
Zipcode/Postal Code
Street, City, State
Street, Zipcode/Postal Code
City, State
City, State, Zipcode/Postal Code
Latitude, Longitude

Download data

Finder datasets are available in the following formats:

Format Content-Type Example URL
KML vnd.google-earth.kml+xml http://finder.geocommons.com/overlays/167.kml
CSV text/csv http://finder.geocommons.com/overlays/167.csv
Shapefile (zipped) application/zip http://finder.geocommons.com/overlays/167.zip
Atom* application/atom+xml http://finder.geocommons.com/overlays/167.atom
JSON* application/json http://finder.geocommons.com/overlays/167.json
Spatialite* application/x-sqlite3 http://finder.geocommons.com/overlays/167.sqlite

*BETA

Atom Output

The Atom output includes attribute data using the Google Data elements, <g:data/>

 <feed xmlns="http://www.w3.org/2005/Atom" xmlns:georss="http://www.georss.org/georss" 
    xmlns:g='http://base.google.com/ns/1.0'>
  <title>Election Precincts</title>
  <id>gc:overlay:1</id>
  <rights>Creative Commons Attribution-Share Alike 2.0</rights>
  <link rel="self" href="http://finder.geocommons.com/overlays/1.atom" type="application/atom+xml"/>
  <link rel="edit" href="http://finder.geocommons.com/overlays/1.atom" type="application/atom+xml;type=entry"/>
  <link rel="alternate" href="http://finder.geocommons.com/overlays/1.html" type="text/html"/>
  <link rel="alternate" href="http://finder.geocommons.com/overlays/1.kml" type="application/vnd.google-earth.kml+xml"/>
  <updated>2009-05-18T11:49:28-04:00</updated>
    <entry>
      <id>gc:overlay:1:feature:75dfecc9</id>
      <g:precname name="Precname">2415</g:precname>
      <g:objectid name="Objectid">1.0</g:objectid>
      <georss:polygon>...</georss:polygon>
    </entry>
</feed>

Pages feed

Last edited by wonderchook, Sun Sep 20 17:29:14 -0700 2009
Home | Edit | New
Versions: