About My Climate Data

 

Mission: To provide easy to use access to global climate data.

Not so long ago after listening to a news segment on climate change, a simple question popped into my mind….how is the climate changing in my hometown of Houston Texas? I know it feels hotter, but maybe the truth is that I’m just older,  fatter, and spoiled by air conditioning. An exhaustive search turned up nothing, but I soon stumbled upon the NOAA web site and mountains of raw data. After fumbling around for a while I managed to get some data downloaded into excel and from there it was off to the races.

I am not a scientist, or an engineer….in other words I have absolutely no expertise in this field. What I am is an accountant, a collector and organizer of data, an and analyst. My day job is to take mountains of raw financial data, and turn it into useful information. My goal with My-Climate-Data.com is to take the mountains of raw climate data available to the public and present it in an easy to use format for everybody.

Data Source: I started with the NOAA monthly Station data available here. That’s about 27 Million records covering about 92000 global sites….some sites have 100+ years of data, some just a few. Some have just temperature or precipitation data, while some have more than a dozen other data points, including: heating and cooling degree days, snowfall, days with maximum temperature over 90F, and days with a maximum temperature under 32F…just to name a few.

Temperature Methodology: Starting with the above monthly data set, we first throw out all years with missing data. If a given year only has 11 months of data, I throw the whole year out. From there, we calculate a straight annual average. No adjustment for days of month, leap years etc… It’s not perfect, but my understanding is that this is what the pros do….If that’s not accurate, please let me know. Making those adjustments would add quite a bit of complexity but make very little difference in the final numbers, so if nothing else we are consistent. After this processing is done, we have an average temperature for all station/year combo’s with sufficient data. From there, the data is sliced up into an excel file for each country, and each US state. This is primarily to keep the file size well under 5MB, which is the maximum that can be displayed using the excel web app…one of the keys to making this data widely available. Each excel file has a handful of pivot tables and pivot charts that can be controlled using excel’s slicer and timeline features. These files are best viewed using excel 2013 or 365. They will be mostly compatible with Excel 2010, and moderately functional in 2007…the data and pivot tables/charts will still be there, but the Slicers/timeline will not function. Excel calculates the charted trailing averages…do note that Excel is not perfect…For example if a year is missing due to missing data, Excel will take the last 15 data points in calculating a 15 year trailing average…even if they aren’t actually the last 15 calendar years. Again…correcting for this adds a huge amount of complexity for very little gain, so I decided not to.