12 ms·
Pirate Weather: A free, open, and documented forecast API
- nimbius 4y agowhat ive always wanted is a competent API for weather.gov, but theres just no impetus to get NOAA to drop their convoluted pseudo-XML gobbledygook.
- Namidairo 4y agoIf that makes you feel bad, take a look at Australia's Bureau of Meteorology. A disorganised mix of access methods, and went around breaking people's requests randomly with 403's and accusing them of screen-scraping even when accessing json. (Some getting hit with at just 1 request/day) Emails to them resulted in replies asking to instead use their FTP. Additionally they're one of the few Australian Gov agencies that still don't have proper https deployment yet.
- hectormalot 4y agoI’ve used https://open-meteo.com/ https://open-meteo.com/ before and I think it’s the same type of open data being exposed. These types of projects are great for stuff like home automation. I’m using to to improve my predictions for power generation (PV) and consumption (heat pump). Planning to is ergst to optimize home battery charging in the future. (Disclaimer; open sourced a small go library for open meteo, but otherwise not affiliated)
- tjhowse 4y agoI've done exactly that! https://www.youtube.com/watch?v=oFSQYK20YUU&list=PLT7ckgz8vcoaKxfTzRYrETgsowRkP41y3&index=4 https://www.youtube.com/watch?v=oFSQYK20YUU&list=PLT7ckgz8vc... I've got a bunch of moving parts in my system, including realtime (5-minutely) energy pricing. If it looks like it's going to be cloudy tomorrow I put my thumb on the scales to make it more likely that my system will buy power from the grid to top off the battery so I can ride through any price spikes. I don't have the stats chops to determine whether I'm actually saving any money with this approach, but it sure is a lot of fun.
- meteo-jeff 4y agoHi, creator of open-meteo.com here! I am using a more wide range of weather models to better cover Europe, Northern Africa and Asia. North America is covered as well with GFS+HRRR and even weather models from the Canadian weather service. In contrast to pirate weather, I am using compressed local files to more easily run API nodes, without getting a huge AWS bill. Compression is especially important for large historical weather datasets like ERA5 or the 10 km version ERA5-Land. Let me know if you have any questions!
- ehaughee 4y agoopen-meteo.com looks awesome. I've been messing around writing a snow forecast app for skiing/snowboarding for a while now and the main thing I'm missing is historical snowfall data. Do these data sources exist in a machine readable format and I've just not been able to find them? If so, would you ever consider adding precip + kind of precip to your historical API?
- meteo-jeff 4y agoSnowfall is already available in the historical weather API. Because the resolution is fairly limited for long term weather reanalysis data, snow analysis for single mountains slopes/peaks may not be that accurate. If you only want to analyse the weeks to get the date of last snowfall and how much power might be there, use the forecast API and the "past_days" parameter to get a continuous time-series of past high-resolution weather forecasts.
- NoboruWataya 4y ago> Why "PirateWeather"? I've always thought that the HRRR model was pronounced the same way as the classic pirate "ARRR". Also, there is one company out there that thinks APIs can be copyrighted, which might apply here. This answers the first question I had upon seeing the name, which was, is it free, open, documented and legal? Based on the above the answer seems to be "probably". Very commendable effort, and I hope the project can last. It seems to be very difficult to maintain a free and reliable weather API so hopefully the dev is not biting off more than he can chew.
- paxys 4y ago> Based on the above the answer seems to be "probably". Why do you draw that conclusion? It seemed to be making a joke about the fact that APIs are NOT copyrightable and considered fair use. The site is pretty clearly fully legal.
- kxrm 4y ago> is it free, open, documented and legal? I think it says something that we live in a world where NOAA data could be seen as an underground or less than legal means of getting the weather. Perhaps AccuWeather has been successful in their campaign to keep free weather data as obscure as possible.
- pbmango 4y agoThis is great. Can endorse LuckGrib for IoS folks as well. Great UI and can pull NOAA and EU models. https://luckgrib.com/ https://luckgrib.com/. Not FOSS but worth the $20 I paid once several years ago. Popular among wind sports enthusiasts many of who are hobbyist meteorologists.
- jron 4y agoI was really hoping https://merrysky.net/ https://merrysky.net/ had a graphical forecast similar to weather underground's 10 day. I've yet to find anything else that quickly shows everything you'd want to know in a single image. Even the mouseover timeline is perfection. It is so good and seemingly exclusive to WU that I sometimes wonder if they hold a patent for it.
- PraetorianGourd 4y agoThe _presentation_ is impressive, but the forecast is often the raw output from various models, and thus not entirely reliable. For example, I live in a valley where most forecast models smooth over giving more snow/cold than we ever get.
- bleomycin 4y agoI have a similar issue living on a hill with the windy.com forecasts. Have you had success finding any forecast source/app that’s accurate for your microclimate? Everyone always points to forecastadvisor.com but that’s always pulling from some airport 20 miles away for me totally irrelevant.
- chrisweekly 4y agoRelated tangent: Windy (iOS app) does a remarkably good job providing multiple data sources and terrific dataviz.
- DotaFan 4y agoI really like this free Forecase API too https://developer.yr.no/ https://developer.yr.no/
- shagie 4y agoOne of the things that I've found on yr.no that I haven't seen on any other weather app is the classification of cloud level. For example: https://www.yr.no/en/details/table/2-4407066/United%20States/Missouri/St.%20Louis/St%20Louis https://www.yr.no/en/details/table/2-4407066/United%20States... You will see not only the cloud cover as an overall percentage, but also the different levels. A 50 in the middle is very different than 50 in low in terms of what you can expect that day (for photographs).
- pca2 4y agoCan confirm pirateweather.net's forecast API working as a drop-in replacement for DarkSky's. I was able to fix a soon to be broken DarkSky bitbar script by just replacing the URLs/API keys to pirateweather's in under 5 minutes.
- nerdponx 4y agoI assume there's no recourse for iOS users?
- aendruk 4y agoDo you run BitBar on iOS?
- jagged-chisel 4y agoAny particular reason you couldn’t do the same? Just replace URLs.
- justusthane 4y agoI think they're asking about continuing to use the DarkSky app. In which case, no. The new Weather app in iOS 16 does add some DarkSky features, but I don't think it's as good.
- jagged-chisel 4y agoTotally read “users” as “developers.” Thanks for straightening out my brain :)
- shagie 4y agoIf you are using the current version of iOS, the new weather app rocks in terms of functionality (IMHO better than the previous version of Dark Sky). If you are on an old version of iOS and mourn the loss of DarkSky app, I personally like MyRadar - it has some very nice features and I used in tandem with DarkSky in the past. If you are after writing an iOS app, you already have an Apple developer account allowing for 500k calls/month. If you are after writing an iOS app and want to use DarkSky rather than WeatherKit, Pirate Weather should also be a drop in replacement.
- earthscienceman 4y agoIt's going to be a tangential comment but I work in science research that's adjacent to weather forecasting and I find the political/technical jockeying that is happening with forecasting to be fascinating. It's a nexus of capitalism, federal government spending, politics, and technology that has very real implications for individual Americans. In summary: horrible oversight by the federal govt (read, congress) of our technical/scientific forecasting resources means that our forecasting ability is extremely fragmented and poorly organized. This has lead to a lot of companies being essentially resellers of public data. These companies claim to create a lot of value added products ('cleaner APIs', 'minutecasts', etc etc) that are either scientifically dubious or technically simple and then these companies walk away with huge profits based on being a portal to government data. It's so American it is almost laughable, all while the European ECMWF eats our lunch in terms of accuracy even for the CONUS. I've discussed this on technical internet forums often enough that I can practically already write the replies to my own comment. "What's the problem with that?" etc et al. But the reality of it is that it's emblematic of how politically broken the US is, in particular with regards to the agencies in charge of scientific products and funding. Not to mention the concrete problems with the forecast products themselves. Anyway. Good luck pirate weather and godspeed. Information was meant to be free and open, especially the forecast. It's such a laughably simple problem that could/should be so easily solved but, alas, there is money to be made!
- jaybo_nomad 4y agoA thousand times this!
- billiam 4y agoHave any nerdy Congresspeople wanted to solve this ridiculous inefficiency in recent years. Seems like a natural "the Europeans kick our ass at forecasting" message would go over well and potentially negate the lobbying from Weather Channel and others with a vested interest in perpetuating this stupid situation.
- counters 4y agoYes, Congress passed the Weather Research and Forecasting Innovation Act back in 2017 which not only gave massive funding injection to model development activities, data procurement, and more, but also formalized the new next-generation community modeling initiatives which are focused on tackling the dominance of the EC models in the 2-5 day forecast space. The diagnosis of the problems of the American forecast modeling community here is based on flawed premises. There are three major factors which led to the ECMWF leap-frogging the US in day-ahead forecasting capability. The first is the consolidated investment in supercomputing resources; the WRFIA tackles this by earmarking a much larger allocation of funding for NOAA's next gen supercomputer, but this still pales in comparison to ECMWF investments. The second factor is the fragmentation of the research and operational weather modeling communities due to the divergent investment from NOAA and USAF in the 90's and 2000's; USAF in conjunction with NCAR sponsored the development of the WRF model which was widely adopted by the research community. NOAA continued investing the GFS lineage of models. The bifurcation of these communities slowed down the ability to matriculate advances in model developments to operations, and this was exacerbated by an old, closed-off approach by NOAA which made it extraordinarily difficult to run and develop the GFS on anything other than NOAA hardware. Finally, the ECMWF went all-in on 4DVAR data assimilation in the late 90's, whereas the American community pursued a diversity of other approaches ranging from 4DVAR to ensemble Kalman filters. 4DVAR necessitates advances to core weather model software (e.g. you need to write a model's adjoint or its tangent linear in order to actually use 4DVAR) adn the US' failure to adopt it led, imo to a "double edged sword" effect of (a) failing to provide impetus to greatly improve the US modeling software suite and supporting tools, and (b) being a worse assimilation technique unless advanced EnsKF techniques are employed using very large ensembles of models (expensive). The other problem as others have pointed out is that there is no accountability in the US private sector weather market. Virtually every player is re-transmitting raw model guidance or statistical post-processed forecasts using DiCast, _maybe_ with a some manual tweaking of forecast fields by humans. But this is not transparent, and many companies - if we're being charitable, here - are not honest about what they're actually doing to produce their forecasts. Put another way - there's a lot of BS claims out there, and it seems that investors have been more than happy to fund it over the past few years.
- ushakov 4y agoThe tech diagram on AWS is insane. This is what I call vendor lock-in porn I'm wondering how much your monthly bill is?
- whalesalad 4y agoEasy enough to move to other clouds if it was defined in something like terraform or even cloudformation. Could even move out of the cloud to your own bare metal services pretty easily, since the various components are so well defined. AWS makes the most sense though as stated in the docs because a lot of the data being processed is in S3, and reading data from S3 inside of most AWS services is free.
- jacurtis 4y agoI am an Infrastructure Architect (aka "Cloud Architect") so I design cloud systems like this on the daily. The "vendor lock-in" argument always makes me laugh. Its the #1 thing I hear all day long. This diagram is actually pretty simple. It looks worse than it is. All it uses are Lambdas (serverless functions), S3 buckets (object storage), and SNS (broadcast/push queues). There appears to be one traditional server in there with EFS, which is just an elastic file system. All of these systems have equivalents in all the major cloud providers. So if the builder of this wanted to move to GCP or Azure, they are not really locked to AWS. This can all be built in another cloud. Now, could you do it in a day? No. Assuming they are building it with Infrastructure as Code (such as Terraform) then they would need to convert the provider and change resource blocks. But this akin to refactoring in a codebase. Its work, but its not terribly difficult. Then they point it to their new cloud and run `terraform apply`. There is almost no way to entirely remove vendor lock-in. The closest you could come is by designing everything yourself on bare metal servers and renting those from a cloud provider. So instead of using a managed queue system, you run some sort of messaging queue on the server. Then you host files on the server's filesystem, and you run the "lambdas" as applications on the server. But that almost causes more headaches than you save or solve for. I look at Cloud Providers as similar to cell phone providers. I know people who live in fear of being locked into a contract with Verizon or something. But really, what are you going to do? You will always need a cell phone. The only other real choice is AT&T or maybe Sprint/TMobile. How often are you really going to switch and what are you really gaining by doing so? Energy spent worrying about being "locked in" to a cloud vendor is energy wasted. Yeah you can move from AWS to Azure or GCP. But that's about it. What do you gain by switching? Probably almost nothing. They are all pretty comparable at this point in reliability, features, and price (GCP is the slight laggard here, but not by much). If Google calls your company and offers you a huge discount to switch, you could still do it. Aside from that, there's minimal incentive to do so. There are a few weird services that AWS has for example that might be considered "lock-in" services. This would be things like AWS Snowball or AWS Groundstation. These don't have comparable systems on other platforms. In the case of Snowball you probably have so much data on AWS that just transferring data would take months (or even years) which could be considered a form of lock-in. tl;dr - This is a very tame arch diagram. A few lambdas, s3 buckets, and messaging queues, all of which have comparable services on all major clouds. There isn't significant vendor lock-in, this could be rebuilt fairly easily (assuming they used IaC) on any major cloud provider.
- davchana 4y agoI usually use forecase.weather.gov They provide a nice textual weather page, with English, for next 7 or 8 days [1]. But it is tiring to open 3 or 4 bookmarks, so I thought to consume that as json & make a single page html app with ajax requests. Its still in progress, I spent few minutes yesterday. The surprising thing was (I was ready to fetch the html text & parse it, difficult), I simply changed one parameter to JSON & I got json [2]. I didn't find any easy to find documentation about json endpoint. My plan is to have city abbreviations on top in horizontal menu, then each click fetches json, puts it into div below. 1. https://forecast.weather.gov/MapClick.php?lat=37.78&lon=-122.42&unit=0&lg=english&FcstType=text&TextType=2 https://forecast.weather.gov/MapClick.php?lat=37.78&lon=-122... 2. https://forecast.weather.gov/MapClick.php?lat=37.98&lon=-120.38&unit=0&lg=english&FcstType=json https://forecast.weather.gov/MapClick.php?lat=37.98&lon=-120...
- vel0city 4y agoThat is cool the web front end exposes itself as JSON as well, good tip. FWIW, they also have a pretty decent API. Its based around zones though, which you'd need to look up. So from that lat and lon, you'd get the zone from: https://api.weather.gov/points/37.7827,-120.38 https://api.weather.gov/points/37.7827,-120.38 Using the zone information, you can get to a forecast: https://api.weather.gov/gridpoints/STO/72,24/forecast https://api.weather.gov/gridpoints/STO/72,24/forecast If you're wanting the current observations, you'd pick a station for that grid such as MOUC1 and go to its observations/latest endpoint: https://api.weather.gov/stations/MOUC1/observations/latest https://api.weather.gov/stations/MOUC1/observations/latest The API is in a JSON-LD format so its got a lot of links to related topics in the actual JSON payload. Looking at the JSON can make it somewhat easy to feel out what you need. The documentation is here: https://www.weather.gov/documentation/services-web-api https://www.weather.gov/documentation/services-web-api
- davchana 4y agoThanks, on that day I already was playing with the api. Initially I had few bookmarks saved on my android home screen, for each city I was interested in weather forecast of. But soon there were total 4 cities; & their text size is too small, need to zoom. I slurped their json api & wrote this: https://spa.bydav.in/weather/ https://spa.bydav.in/weather/ You need to add URL for each city you want to see weather, in Settings. All that data is saved in your device local storage. No funny trackers or home phone. You can also paste the following json in Settings > Text Area > Click Import JSON: [ { "abb": "STK", "full": "Stockton", "url": "https://forecast.weather.gov/MapClick.php?lat=37.95&lon=-121.29&unit=0&lg=english&FcstType=json&TextType=1" }, { "abb": "SFO", "full": "San Francisco", "url": "https://forecast.weather.gov/MapClick.php?lat=37.7771&lon=-122.4197&unit=0&lg=english&FcstType=json&TextType=1" } ]
- ChrisArchitect 4y agoRelated discussion about Merry Sky, a Dark Sky replacement built on this https://news.ycombinator.com/item?id=34155191 https://news.ycombinator.com/item?id=34155191
- wendyshu 4y agohttps://pirateweather.net/apis https://pirateweather.net/apis isn't loading for me
- ehaughee 4y agoSame here. Tried a few hours apart. If you're looking for the API docs, as I was, they are here: https://docs.pirateweather.net/en/latest/ https://docs.pirateweather.net/en/latest/
- alexander0042 4y agoThis is a limitation of how the Developer Portal is set up, you have to be registered to view the APIs. Like the other poster said, if you're looking for general info, go to the docs section, since all the information is there
- ourmandave 4y agoHow does this differ from NOAA weather.gov api? https://www.weather.gov/documentation/services-web-api https://www.weather.gov/documentation/services-web-api Or is this a friendlier overlay of their interface?
- cantaloupe 4y agoUnder “Weather Sources” > All weather data comes from the AWS open data program https://registry.opendata.aws/collab/noaa/ https://registry.opendata.aws/collab/noaa/. This is a fantastic program, since it is more reliable than the NOAA distribution, and means there are no data transfer changes!
- alexander0042 4y agoGood question! It's the same underlying models, but three key differences: 1. Pirate Weather returns data following the Dark Sky syntax, as opposed to the custom NWS format. 2. Pirate Weather has global coverage, the NWS API is only for the US. 3. The NWS one uses human forecasters to come up with their forecasts, compared to Pirate Weather's use of raw model results.
- earthscienceman 4y agoOnly on HN would the NWS format be described as 'custom'.
- hnarn 4y agoI know literally nothing about weather data standardization, but if the format was — for the sake of argument — unique to the US for example and another more popular format was used either internationally or de-facto by third parties, I’d say it qualifies as a “custom” format.
- earthscienceman 4y agoQuite the opposite. It's literally the global format that nearly everyone in the world provides data in.
- byhemechi 4y agoappears to pretty much US only, unfortunately. it's -5 where i am right now, but this claims that it is actually -17 (a very big difference)
- waynecochran 4y agoI like having an open API. Every weather site eventually becomes trash as it is loaded with obnoxious ads. Maybe the answer it just to have numerous weather front ends.
- alexander0042 4y agoIt's a real issue, that's why I made my own! The beauty of an open source one is that if things start to go off the rails, someone else could also pick up and recreate the data source
- none_to_remain 4y agoIs there a linguistic shift happening here where this is no longer "a service implementing the same API as Dark Sky", rather it is "an API"
- DrBenCarson 4y ago@OP you've exceeded your daily cognito limit
- hk1337 4y agoI cannot load the apis page right now. Is this just current weather data or do you offer historical weather data too?
- vorpalhex 4y agoOne of Dark Sky's features was integrating hyperlocal weather stations. Right now there are thousands of weather stations posting regular reports via APRS. Any chance to see those integrated?
- alexander0042 4y agoThey (sort of) are! It gets into the nuance of the weather model side of things, which I don't know as much about, but the HRRR/ GFS models ingest data from APRS and incorporate that into their forecasts. This means that these observations are included (especially in the US), but are at about a 2 hour lag compared to direct readings
- stephen123 4y agoIs it a common startup idea to open source a successful product and change the payment to be donation based?
- 10g1k 4y agoWhy are pirate systems the best? They just AARRRR!
- datalus 4y agoI found this old blog article at a high level on how Dark Sky works... about 12 years old now. I always loved this app and was kind of sad when Apple acquired them... but to have Apple come in and just buy your app out is probably a great feeling for the founders :) Anyhow, here's the url: https://jackadam.github.io/2011/how-dark-sky-works https://jackadam.github.io/2011/how-dark-sky-works
- deleted 4y ago[deleted]
- alexander0042 4y agoThis is fascinating, thank you for sharing! What is even more amazing about Dark Sky is that they did this in 2011, when a lot of the tools I have access to now didn't exist. For Pirate Weather, I avoid working with radar data at all, instead using the 15-minute forecast that the HRRR model provides
- azinman2 4y agoWhy avoid radar data?
- WoodenChair 4y agoAwesome, but it's not yet got the text summary feature, which curiously is also missing from Apple's official Dark Sky replacement, WeatherKit: https://developer.apple.com/documentation/weatherkitrestapi https://developer.apple.com/documentation/weatherkitrestapi I created the start of a Python wrapper for WeatherKit, if anyone is interested in helping with that effort: https://github.com/davecom/PyWeatherKit https://github.com/davecom/PyWeatherKit
- heystefan 4y agoThis is amazing. You're amazing. Just bought you a coffee, everyone who upvoted this should chip in!
- alexander0042 4y agoThank you so much! I really want there to be a free level available, and it's donations like this that make it possible
- FatActor 4y agoIf you asked me 25 years ago, "Wanna bet hackers will be excited about a weather API in the future?" ... Sink me ... I would have lost that bet.
- coucoualpha 4y agoSame here, what are the use cases that fueled this demand?
- Dentrax 4y agoThis is really cool! Can't wait to try. But I'm getting: "Exceeded daily email limit." during sign-up.
- Humphrey 4y agoAs an Australian, my main issue with almost all Weather API's and services is that the data is almost always sub-par to the weather data provided by the Bureau of Meteorology (BOM). It would be great if there was a backend provider concept, so that I could tell my devices (such as an iPhone) to only use BOM data for it's inbuilt widgets.
- ace2358 4y agoIt’s truly maddening as an Aussie. How many apps just ignore the BoM? I expected the Apple one not to, but it does. It’s frustrating when friends visit me and check the weather on their phone and the data for the Google and Apple apps is so bad. I use Willy weather that uses bom. I don’t love it though. Any recommendations? I personally can’t stand the actual BoM app as an iPhone user but I appreciate that they tried… R.I.P Pocket Weather :(
- sandos 4y agoI wonder if this isn't the same all over? In Sweden we also have a local metrology bureu which produces much better forecasts than most others. As I understand it they basically tweak the global models with .. sort of local "knowledge". And there are many such models, a few EU-specific and, I assume, one local for most countries.
- 83 4y agoIs metrology a typo, or are there countries that call it metrology instead of meteorology? In the US metrology is a specific field dealing with measurement, often the department dealing with high precision calibrating/measuring tools in the metal machining industry.
- kuhewa 4y agoNate Silver's The Signal and the Noise's chapter about weather forecasting described forecasters during a visit doing that, a manual tweak essentially based on experienced intuition at the end that the models couldn't yet replicate.
- Semaphor 4y ago> An error was encountered with the requested page. > Exceeded daily email limit. Happens when I try to signup
- gkoberger 4y agoThis is awesome! If you want to use ReadMe for free for your docs, just email me (my email's in my profile) and I can hook ya up!
- tucosan 4y agoI applaud the effort. I just checked the forecast data for Berlin, Germany and it is quite inaccurate compared to the quality models offered for my region. It seems that they only use NOAA data even though there are vastly superior models for the EU, e.g. ICON-EU and ECMWF.
- kish15 4y agoI cannot seem to sign up, it says "Exceeded daily email limit."
- wohfab 4y agoConfirmed. Me, too.
- huhtenberg 4y agoTried it indirectly via MerrySky after it was mentioned here the other day. I am in Europe it was completely off in both its 24-hour forecast and the actual real-time weather. It indicated a continuous heavy snowfall whereby in reality the sky was just lightly clouded with no precipitation. Just 2c.
- metadat 4y agoIs there a decent free weather app for Android or iOS which doesn't have ads and / or unnecessary tracking? Would be cool if Pirate Weather could serve as the foundation for such a thing.
- Tempest1981 4y agoWeaWow is good. Donation-supported.
- metadat 4y agoWow, thanks for the quick reply and info, Tempest1981! Cheers.
- oindividuo 4y agoThere is also yr.no, it has many details and looks great