James Linden

~# data ninja / linux guru / web dev geek / robotics nerd / idea machine / N6NRD

Datasets / Current Weather Observation Importer

Current Weather Observations Importer

Overview

Weather.gov Current Observations Importer is a script to download the current observations archive of the XML documents provided by weather.gov and insert the records into a MySQL database.

Dataset Source

URL: http://www.weather.gov/xml/current_obs/all_xml.zip

Updates: hourly

Environment

  • GNU/Linux
  • PHP 5.3 +
  • MySQL 5.x

Notes

  • This script is designed to run on the command line.

Howto

Create the database (insert your database name):

CREATE DATABASE IF NOT EXISTS database_name DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_bin;

Create the table:

CREATE TABLE IF NOT EXISTS t_report (
c_id SERIAL PRIMARY KEY,
c_station VARCHAR(8) DEFAULT NULL,
c_datetime DATETIME DEFAULT NULL,
c_tempf DEC(5,2) SIGNED DEFAULT NULL,
c_tempc DEC(5,2) SIGNED DEFAULT NULL,
c_humidity TINYINT(2) UNSIGNED DEFAULT NULL,
c_winddir VARCHAR(32) DEFAULT NULL,
c_winddeg INT(3) UNSIGNED DEFAULT NULL,
c_windmph DEC(6,2) UNSIGNED DEFAULT NULL,
c_windgustmph DEC(6,2) UNSIGNED DEFAULT NULL,
c_windkt DEC(6,2) UNSIGNED DEFAULT NULL,
c_windgustkt DEC(6,2) UNSIGNED DEFAULT NULL,
c_pressuremb DEC(6,2) DEFAULT NULL,
c_pressurein DEC(6,2) DEFAULT NULL,
c_dewf DEC(5,2) SIGNED DEFAULT NULL,
c_dewc DEC(5,2) SIGNED DEFAULT NULL,
c_chillf DEC(5,2) SIGNED DEFAULT NULL,
c_chillc DEC(5,2) SIGNED DEFAULT NULL,
c_visibility DEC(5,2) UNSIGNED DEFAULT NULL
) ENGINE=MyISAM;

Download weather.gov-currentobservations-import.php and edit it, setting your database configuration:

$CFG['db']['host'] = 'localhost';
$CFG['db']['port'] = 3306;
$CFG['db']['user'] = null;
$CFG['db']['pass'] = null;
$CFG['db']['name'] = null;

Run the script -- it should require less than a minute on average hardware.

License

This project is BSD (2 clause) licensed.

photo of James Linden
Founder / Head Geek
Digital Dock, LLC
aka kodekrash & N6NRD
Alexandria, LA USA

What I Do

Linux administration & virtualization
Data mining, storage & analysis
Web development

What I've Done

Rescued a skunk
Built Prime GNU/Linux
Contributed to Spidering Hacks