The TSP decided to change the way that they make the data available randomly since I originally made the script. The old version of the page looked like this:

It only allowed you to access 30 days at a time, and required more of a human-touch to gather all of the data. In order to navigate this, I had to request the data (and resulting page) 30 prices at a time. To load the whole database, this took a considerable amount of time.

Luckily (although unluckily because I had figured out all of this already), they changed it to make it a little easier:

This new version of the site did a few new things that makes getting the data easier. First, gathering a specific date range in history is easier, and has no limit on the number of lines (the default page displays 30, though). All one has to do is put in a date range for the start and end dates, click “Retrieve Share Prices” and voila, out comes all of the data. On top of that, it’s possible to download a CSV of this data by clicking the “Download Share Prices” button.

As an aside, now that you can request a date range, when you do so, it will give you the prices only on days that it changed. For instance, if your start date is on a Saturday, the data returned will start on the Monday after (the next valid day the TSP and the market operated). So, if you were to request a single day’s price, what would it return? Well, I tested that:

Since the date is invalid, it just returns a “None ,” string in the Date column and “N/A” in the fund price columns. This is interesting, because you can tell the coders didn’t do a great job catching that exception. Either way, it means it’s easy to see if you accidentally tried to retrieve a weekend.

One last thing: they’ve added checkboxes so that you can request only a specific fund’s data. This may be useful in requesting quotes more like stocks from Google or Yahoo!. However, in terms of a general maintenance of a database, this isn’t useful, as you’d want all of the data.

Bottom line: the TSP historical data is becoming more akin to a real data service. Ideally, you could request a JSON update string just like you could from Google, but that will leave room for their improvement in the future.