Slides from “Tapping the Data Deluge with R” lightning talk #rstats #PAWCon

October 2, 2012 — Jeffrey Breen

Here is my presentation from last night’s Boston Predictive Analytics Meetup graciously hosted by Predictive Analytics World Boston.

The talk is meant to provide an overview of (some) of the different ways to get data into R, especially supplementary data sets to assist with your analysis.

All code and data files are available at github: http://bit.ly/pawdata (https://github.com/jeffreybreen/talk-201210-data-deluge)

The slides themselves are on slideshare: http://bit.ly/pawdatadeck (http://www.slideshare.net/jeffreybreen/tapping-the-data-deluge-with-r)

Posted in Tutorials. Tags: , , , , , , , , , , , . 1 Comment »

Use geom_rect() to add recession bars to your time series plots #rstats #ggplot

August 15, 2011 — Jeffrey Breen

Zach Mayer’s work reproducing John Hussman’s Recession Warning Composite prompted me to dig this trick out of my (Evernote) notebook.

First, let’s grab some data to plot using the very handy getSymbols() function from Jeffrey Ryan’s quantmod package. We’ll load the U.S. unemployment rate (UNRATE) from the St. Loius Fed’s Federal Reserve Economic Data (src="FRED") and load the time series into a data.frame:

unrate = getSymbols('UNRATE',src='FRED', auto.assign=F) 
unrate.df = data.frame(date=time(unrate), coredata(unrate) )

Now FRED provides a USREC time series which we could use to draw the recessions. It’s a bit awkward, though, as it contains a boolean to flag recession months since January 1921. All we really want are the start and end dates of each recession. Fortunately, the St. Louis Fed publishes just such a table on their web site. (See the answer to “What dates are used for the US recession bars in FRED graphs?” on http://research.stlouisfed.org/fred2/help-faq/.) Sometimes it’s still easier to cut-and-paste (and the static table covers another 64 years, go figure):

recessions.df = read.table(textConnection(
"Peak, Trough
1857-06-01, 1858-12-01
1860-10-01, 1861-06-01
1865-04-01, 1867-12-01
1869-06-01, 1870-12-01
1873-10-01, 1879-03-01
1882-03-01, 1885-05-01
1887-03-01, 1888-04-01
1890-07-01, 1891-05-01
1893-01-01, 1894-06-01
1895-12-01, 1897-06-01
1899-06-01, 1900-12-01
1902-09-01, 1904-08-01
1907-05-01, 1908-06-01
1910-01-01, 1912-01-01
1913-01-01, 1914-12-01
1918-08-01, 1919-03-01
1920-01-01, 1921-07-01
1923-05-01, 1924-07-01
1926-10-01, 1927-11-01
1929-08-01, 1933-03-01
1937-05-01, 1938-06-01
1945-02-01, 1945-10-01
1948-11-01, 1949-10-01
1953-07-01, 1954-05-01
1957-08-01, 1958-04-01
1960-04-01, 1961-02-01
1969-12-01, 1970-11-01
1973-11-01, 1975-03-01
1980-01-01, 1980-07-01
1981-07-01, 1982-11-01
1990-07-01, 1991-03-01
2001-03-01, 2001-11-01
2007-12-01, 2009-06-01"), sep=',',
colClasses=c('Date', 'Date'), header=TRUE)

Now the only “gotcha” is that our recession data start long before our unemployment data, so let’s trim it to match:

recessions.trim = subset(recessions.df, Peak >= min(unrate.df$date) )

Finally, we use ggplot2’s geom_line() layer to draw the unemployment data and transparent (alpha=0.2) pink rectangles to overlay the recessions:

g = ggplot(unrate.df) + geom_line(aes(x=date, y=UNRATE)) + theme_bw()
g = g + geom_rect(data=recessions.trim, aes(xmin=Peak, xmax=Trough, ymin=-Inf, ymax=+Inf), fill='pink', alpha=0.2)

Posted in Tips. Tags: , , , , , . 6 Comments »

quantmod makes it easy to watch silver prices crash in R #rstats

May 7, 2011 — Jeffrey Breen

As if there hasn’t been enough going on this week, silver prices have fallen nearly $10 per ounce. That’s a reduction of over 20%. Jeffrey Ryan’s quantmod package makes it easy to download the latest prices from OANDA’s web site and plot the excitement.

The getSymbols() function is at the heart of quantmod’s data retrieval prowess, currently handling Yahoo! Finance, Google Finance, the St. Louis Fed’s FRED, and OANDA sites, in addition to MySQL databases and RData and CSV files.

First a word of warning: if you have a computer science background, you may cringe at the way getSymbols() returns data. Rather than returning the fetched data as the result of a function call, it populates your R session’s .GlobalEnv environment (or another one of your choosing via the env parameter) with xts and zoo objects containing your data. For example, if you ask for IBM’s stock prices via getSymbols("IBM"), you will find the data in a new “IBM” object in your .GlobalEnv. This behavior can be changed by setting auto.assign=F, but then you can only request one symbol at a time. But this is a minor nit about an incredibly useful package.

There’s even a wrapper function to help retrieve precious metal prices, and we will use this getMetals() function to retrieve the last year’s worth of prices for gold (XAU) and silver (XAG):

library(quantmod)
getMetals(c('XAU', 'XAG'), from=Sys.Date()-365)

Yup — that’s it. getMetals() lets us know it has created two new objects:

[1] "XAUUSD" "XAGUSD"

There were also few warning messages complaining about the last line in the downloaded file. I haven’t bothered to dig into it as the data seem fine, including today’s price:

> ls()
[1] "XAGUSD" "XAUUSD"

> head(XAGUSD)
           XAG.USD
2010-05-07 17.6600
2010-05-08 18.4600
2010-05-09 18.4320
2010-05-10 18.4336
2010-05-11 18.5400
2010-05-12 19.3300

> tail(XAGUSD)
           XAG.USD
2011-05-02 47.9850
2011-05-03 45.2373
2011-05-04 44.0238
2011-05-05 40.9171
2011-05-06 37.9939
2011-05-07 35.0598

And here’s how easy it is to use the package’s built-in graphing facilities:

chartSeries(XAUUSD, theme="white")


chartSeries(XAGUSD, theme="white")

Yup — that’s quite a shellacking for silver.

Now I tend to be a ggplot2 guy myself, and I have never actually worked with xts or zoo objects before, but it’s pretty easy to get them into a suitable data.frame:

silver = data.frame(XAGUSD)
silver$date = as.Date(rownames(silver))
colnames(silver)[1] = 'price'

library(ggplot2)
ggplot(data=silver, aes(x=date, y=price)) + geom_line() + theme_bw()

Posted in Tips. Tags: , , , , , . 17 Comments »