24 ms·
U.S. National Park Service API
- francisofascii 3y agoI don't think I see visitor stats? Oh well. Historical visitor data is here: https://irma.nps.gov/Stats/SSRSReports/Park%20Specific%20Reports/Visitation%20by%20Month https://irma.nps.gov/Stats/SSRSReports/Park%20Specific%20Rep...
- lukew3 3y agoLove that this exists. I found that they also have a Github(1) with a public data archive. Any existing projects using this API in an interesting way? [1] https://github.com/nationalparkservice https://github.com/nationalparkservice
- jimmyrocks 3y agoThe GitHub is mostly outdated repos, although Symbol Library is still active. The developer API is great though.
- tonymet 3y agoI migrated this data to BigQuery. Test it out and if it works well https://console.cloud.google.com/bigquery?hl=en&project=nps-public-data&ws=!1m4!1m3!3m2!1snps-public-data!2snps_public_data https://console.cloud.google.com/bigquery?hl=en&project=nps-... If you like it I’ll update the batch job to keep it fresh
- deleted 3y ago[deleted]
- deleted 3y ago[deleted]
- tonymet 3y agoBigQuery public datasets would be a better hosting platform for this kind of data. I worry they are not anticipating the security & budgeting issues of hosting a real-time API. With PUblic Datasets, the account making queries pays for the queries. NPS only pays for the storage (which is minimal). With this API, NPS has to pay for every call to the API. That’s not cheap.
- tonymet 3y agoWith their API they have to write a bunch of boilerplate code to transform from their SQL db to REST. Authentication, throttling, threat prevention, encoding, etc etc. With BigQuery they just copy the data in via CSV and Big Query handles the indexing & query engine.
- mlhpdx 3y agoWho says they have a SQL DB? This looks to be almost entirely static data, occasionally updated.
- tonymet 3y agoWhatever storage format they have, they are writing boilerplate to transform it into REST . Regardless, it will be cheaper to just ingest into BigQuery
- jimmyrocks 3y agoIt’s Apache Solr. Most of the data is static, but alerts and events get frequent updates.
- tonymet 3y agoSame concern about unnecessary code and compute stands.
- dragonwriter 3y ago> With their API they have to write a bunch of boilerplate code to transform from their SQL db to REST. Open source tools that will present a simple, read-only REST API over an SQL db with little to no custom code exist (so do proprietary tools, often from DB vendors and sometimes as part of SQL db products.) Same with NoSQL or sorta-SQL storage solutions. The idea that they have to write a bunch of boilerplate code to do this is false. They might choose to do that, but its defintely not necessary. > Authentication, throttling, threat prevention, encoding, etc etc. Again, open source canned solutions that take a little bit of configuration exist for many of those, and some of them are likely shared services that they just point at whatever service needs them.
- AnthonyMouse 3y agoWhy is the government cargo-culting the scourge that is API keys? The goal of this should be for everyone to have access and lower barriers to entry, not put bureaucracy in the way of access and de facto suppress use by open source projects because each user would need their own API key unless someone publishes one.
- gausswho 3y agoWhat makes API keys a scourge?
- scq 3y agoAPI keys are important for effective rate limiting/abuse prevention.
- beachy 3y agoIf the provider is bearing the costs (like here) then they always need some kind of authorization, or they have no way to shut off abusers or people with misbehaving clients. An API key is about the simplest possible way to achieve that, and appears to be perfectly adequate in this case. What do you suggest? SAML?
- AnthonyMouse 3y ago> If the provider is bearing the costs (like here) then they always need some kind of authorization, or they have no way to shut off abusers or people with misbehaving clients. HTTP is an "API" that has no API keys and all the public web servers in the world seem to manage this without any trouble. > What do you suggest? SAML? No authentication required by default -- it's public data. Just impose a reasonable rate limit by IP address and require registration only if someone has a legitimate reason to exceed that.
- mlhpdx 3y ago> HTTP is an "API" that has no API keys and all the public web servers in the world seem to manage this without any trouble. Um, no. That’s just not true.
- bamnet 3y agoIt's a super cool API, but last I chatted with the team who maintains this (~2020?) it was already understaffed and they only had capacity for bug fixes. The roadmap hasn't been updated since 2017 so YMMV.
- jeffennell 3y agoGood to know! Thanks for sharing.
- tonymet 3y agodo you still have the contact? I have a proof of concept i'd like to share with them.
- jackstraw14 3y agoThat sounds about right tbh... funding is not great for these kinds of things :(
- marginallen 3y ago[dead]
- gigatexal 3y agothis isn't out of 18F -- the really cool tech team at the whitehouse(?)? what happened to all that digital govt funding?
- cowsandmilk 3y agoWell, the president who launched all of that left office in 2017; and we had a new president with a bias against national parks… meaning the funding NPS did receive had to be spent on critical work in the parks themselves.
- redcobra762 3y agoThe President does not decide funding for the NPS, Congress does.
- hasty_pudding 3y agoDoes anyone know any APIs to get BLM land?
- stevehawk 3y agoI think it's the ACH API, you need a routing number, account number, and a price tag for your local republican.
- smellyshaft1 3y agoI remember when I used to think only one party destroyed public lands for a paycheck.... ^^ Change republican to politician Biden granted more oil and gas drilling permits than Trump in his first 2 years in office https://news.yahoo.com/biden-granted-more-oil-and-gas-drilling-permits-than-trump-in-his-first-2-years-in-office-190528616.html https://news.yahoo.com/biden-granted-more-oil-and-gas-drilli...
- newman8r 3y agoI don't know, but you might want to reach out to http://www.mylandmatters.org/ http://www.mylandmatters.org/ - that's where I go to get a lot of land use data, so they might be able to point you in the right direction.
- tonymet 3y agoI copied this data into Bigquery -- it's much more accessible this way for Jupiter notebooks and doing SQL queries nps-public-data.nps_public_data
- tonymet 3y agohttps://console.cloud.google.com/bigquery?hl=en&project=nps-public-data&ws=!1m4!1m3!3m2!1snps-public-data!2snps_public_data https://console.cloud.google.com/bigquery?hl=en&project=nps-...
- crancher 3y agoThank you for creating this. How do you access/find it from within BigQuery Studio?
- tonymet 3y agoIs it not showing up in the search bar ? You can search by name . Let me see how to generate a link
- tonymet 3y agoDoes this link work ? https://console.cloud.google.com/bigquery?hl=en&project=nps-public-data&ws=!1m4!1m3!3m2!1snps-public-data!2snps_public_data https://console.cloud.google.com/bigquery?hl=en&project=nps-...
- crancher 3y agoCouldn't get it to show up in search but the link works, thanks again.
- tonymet 3y agoI wrote up some docs please add your positive & negative feedback by filing github issues here https://github.com/tonymet/nps-public-data/blob/main/README.md https://github.com/tonymet/nps-public-data/blob/main/README....
- Sparkyte 3y agoThis sounds incredibly fun. I wish morr government programs like this received more funding.
- CaptWillard 3y agoTheir Twitter has been consistently funny. Whoever is making these decisions over there, I'd like to thank them.
- deleted 3y ago[deleted]
- deleted 3y ago[deleted]
- DerCommodore 3y agoInteresting API
- deleted 3y ago[deleted]
- greenie_beans 3y agofyi, there are some undocumented but public apis if anybody needs more data. you can see them if you visit certain nps websites.
- tylervigen 3y agoI think this is a funny joke about how HTTP is technically an API. However, with more and more website fighting back against scraping (in part due to LLMs), I suspect explicit APIs are going to become increasingly important and useful.
- mckn1ght 3y agoThis made me wonder why websites would be against scraping, at least as long as the content has DoS protection behind an edge cache. Aside from the copyright/etc issues. Then it made me wonder, since AWS/cloudflare already have ML products and have data stored in edge caches already, if they don’t/couldn’t train directly from the caches?
- HanClinto 3y agoMight be a good use-case to use something like this extension to auto-build an API spec whilst browsing a website: https://github.com/AndrewWalsh/openapi-devtools https://github.com/AndrewWalsh/openapi-devtools
- greenie_beans 3y agowhoa that's sick, thanks for sharing
- tonymet 3y agocan you share them?
- jeffheard 3y agoOh good. I was literally wishing for this on Saturday when trying to parse the table of which mileposts were closed on the blue ridge parkway.
- jordanvincent 3y agoA lot of great data here too: https://irma.nps.gov/Stats/ https://irma.nps.gov/Stats/ I used it to build my National Park visitation visualization: https://jordan-vincent.com/night-under-the-stars https://jordan-vincent.com/night-under-the-stars
- jeffennell 3y agoYour project is awesome! Nice work
- mceoin 3y agoThe NPS API is still missing a bookings API, which is the carrot that was needed for developers to create new and useful experiences.
- jimmyrocks 3y agoThe bookings are all done through rec.gov
- darkwizard42 3y agoI believe this is intentional... many reasons come up for why.
- d0gsg0w00f 3y agoDoes one of them rhyme with Ticketmaster?
- culi 3y agoThe list of national parks makes it seem like there are only 61 when there are actually 63. After comparing the list to Wikipedia, the reasons for this discrepancy are: - National Park of American Samoa (est. 1988) is missing any designation - Sequoia and Kings Canyon are listed as the same park (parkCode: seki)