Skip to main content
Posts by:

Tim Booher

OFX for USAA via Ruby

My wife and I have been through roughly 10-15 different budget/financial tracking systems. We started with every penny in MS Money, used several different spreadsheets, spent several years in Mint and have pretty much dropped all of that for a top-down strategy that has us budgeting savings, non-discretionary spending, and a rainy day buffer and arriving at a fixed weekly budget for groceries, clothes, snacks, eating out and random household supplies. We use a debit card for this, and transfer the allotted amount every Thursday into the daily spending account. The problem is that we started pushing money into the account whenever it runs low, and we end up losing our focus and even the ability to track how much we spend in a given week. In an audit of last year’s spending, it was surprising to see that we were routinely 100{aaa01f1184b23bc5204459599a780c2efd1a71f819cd2b338cab4b7a2f8e97d4} over our budget when we looked at other spending sources.

Since I code web applications, I decided to play with bringing in some of the data we create, both household and financial to ultimately create a personal dashboard for our family. In doing so, we aren’t locked into any one system and we can create something custom that works for us. This way, we can track our fitness, finances, journal and home systems all in one place and own the data and experience. One lesson learned is that our tracking systems need to be on autopilot as our different interests surge. A fragile system doesn’t work. Our needs will vary, but we want any tracking system to be able to produce a report on request.

While fun and useful, this takes familiarity with some new protocols (OXF for finance and LUUP for home automation). On a plane flight to Las Vegas, I was able to get OFX to successfully connect to USAA. First I had to set a module with USAA’s specifics:

With this in place, I can generate a valid OFX request:

This request passes all the assertions designed to test for a valid signon response:

def verify_usaa_signon_response(response_document)
        signon_message = response_document.message_sets[0]
        assert signon_message.kind_of?(OFX::SignonMessageSet)
        assert_equal(1, signon_message.responses.length)

        signon_response = signon_message.responses[0]
        assert signon_response.kind_of?(OFX::SignonResponse)
        assert_not_equal(nil, signon_response.status)
        assert signon_response.status.kind_of?(OFX::Information)
        assert signon_response.status.kind_of?(OFX::Success)
        assert_equal(0, signon_response.status.code)
        assert_equal(:information, signon_response.status.severity)
        assert_not_equal(nil, signon_response.status.message)
        assert_not_equal(nil, signon_response.date)
        assert_equal(nil, signon_response.user_key)
        assert_equal('ENG', signon_response.language)
        #assert_not_equal(nil, signon_response.date_of_last_profile_update)
        #assert_not_equal(nil, signon_response.date_of_last_account_update)
        assert_not_equal(nil, signon_response.financial_institution_identification)
        assert_equal('USAA', signon_response.financial_institution_identification.organization)
        assert_equal('24591', signon_response.financial_institution_identification.financial_institution_identifier)
        assert_equal(nil, signon_response.session_cookie)
    end

One of the difficult parts was to determine the required length of my account number in the absence of documentation. It took some experimentation to find out that USAA wants exactly nine digits for the username (member number) and ten digits for an account number. Instead of making code that robustly input padded zeros (through sprintf or similiar), I just changed the input values.

I also noticed that USAA did have

<LEDGERBAL><BALAMT>290.51<DTASOF>20140211120000</LEDGERBAL></STMTRS>

, but did not have the available balance fields that the gem expected. In any case, I can now get transactions and full access to my bank programmatically, which is pretty cool.

Links:

By 0 Comments

Setting up the Aeon Labs Aeotec Z-Wave Smart Energy Monitor

I struggled for awhile trying to set up the Aeon Labs Aeotec Z-Wave Smart Energy Monitor to monitor my electricity. The manual or any instructions were difficult to find online.

The first article that was absolutely necessary explained how to pair the device. After reading this article, pairing was pretty trivial.

Great details in the developer’s manual

part number: DSB09104-ZWUS

the manufacturer is also marginally useful.

the ‘manual’

the amazon page

Key advise is to wait after installation. I can’t get anything from Watts, but I can read each clamp regularly. While I look into this later, you can still see what is going on:

Screenshot 2014-02-10 07.09.37

Household temperature and set point for the last week

Household temperature and set point for the last week

By 0 Comments

MiCasa Verde (MiOS) DataMine Logging

For my home automation goals, I chose MiCasaVerde VeraLite due to my friend’s recommendation. The VeraLite is a small linux controller that runs the MIDIbox Operating System MiOS and gives a homeowner the ability to easily control lights, security cameras, door locks, alarm systems, and even the thermostat, among many other home systems. For example, you could set your temperature from your mobile device or web browser. I chose this device because of my friend’s experience and the Vera’s compatibly with INSTEON, Z-Wave, and X10 devices and I had previously made a big investment in X10.

Setup was easy and everything worked well with my 2gig CT100 Z-Wave Programmable Thermostat after I completed my initial pairing.

I was happy to see the dataMine Graphing and Logging plugin for my micasaverde veralite, but after mounting my usb, I still couldn’t see any channels.

MiCasaVerde Screenshot

MiCasaVerde Screenshot

First thing was to get SSH access to my account so I could do some troubleshooting. To do that, I had to go to account, tech support and enable remote access. If you do that, you get a message like this:

Tech support full control enabled, access code 34014212-123456 (SSH: SSH_22=27032 TS_SRV=ts2)

And your password is 123456 for the user “remote”. In order to get the root password you need to run:

nvram show | grep pass

You’ll see something like this:

vera_wifipass=shade83forest

So in this case, shade83forest, is the actual root password.1

In my ssh shell, I was able to see that my mount was correct and available:

/dev/sda1               506.2M     16.5M    464.0M   3{aaa01f1184b23bc5204459599a780c2efd1a71f819cd2b338cab4b7a2f8e97d4} /dataMine

Through using the file command I could tell this was an ext3 partition:

/dev/sda1: Linux rev 1.0 ext3 filesystem data (needs journal recovery) (large files)

I also could get any details on my MiOS version:

Linux MiOS_35017272 2.6.37.1 #2 Fri Feb 22 04:07:32 PST 2013 mips GNU/Linux

After installing, your /dataMine/ directory should be mounted and have the following files:

/dataMine# ls -l
-rw-r--r--    1 root     root           252 Jan 26 07:38 InternetOk.log
-rw-r--r--    1 root     root         27646 Jan 26 07:48 LuaUPnP.log
-rw-r--r--    1 root     root             0 Jan 26 07:38 NetworkMonitor.log
-rw-r--r--    1 root     root          2112 Jan 29 06:37 Notifications [R2299].txt
-rw-r--r--    1 root     root           468 Feb  1 06:20 Notifications [R2300].txt
-rw-r--r--    1 root     root           150 Feb  1 06:20 dataMineConfig.json
drwxr-xr-x    2 root     root          4096 Jan 26 07:55 database
drwx------    2 root     root         16384 Jan 26 07:37 lost+found
-rw-r--r--    1 root     root            26 Jan 26 07:38 mount_tests
-rw-r--r--    1 root     root          2110 Jan 26 07:38 serproxy.log
-rw-r--r--    1 root     root             0 Jan 26 07:48 signal.flag.log
-rw-r--r--    1 root     root            50 Jan 26 07:48 signal.log
-rw-r--r--    1 root     root           414 Feb  1 07:15 sunriseSunset.txt

If I request debugging information (via /port_3480/data_request?id=lr_dmCtrl&control=debug), I get:

First I found the variable I wanted (current temperature) and enabled logging:

Current Settings for HVAC

Current Settings for HVAC

Initially, you don’t get good feedback. I thought I was getting an error and couldn’t get to any logs (see the red exclamation mark where the results should be)

Error

Error

But that should change once events occur and the logging starts.

My debug file has provided the following information:

{"Version":"0.980","dbVersion":2,"Events":{"count":17,"last":1391253610},"guiConfig":[],"Variables":[{"Service":"urn:upnp-org:serviceId:TemperatureSensor1","LastRec":0,"FilterMaximum":0,"Type":0,"Logging":1,"FilterEnable":0,"Device":4,"FilterMinimum":0,"Id":1,"DrowsyWarning":0,"DrowsyError":0,"DataOffset":0,"Name":"CurrentTemp","LastVal":0,"FirstRec":0,"Variable":"CurrentTemperature","DataType":1}],"Graphs":[],"LastWrite":1391261432,"nextId":2}

One problem was that I was getting lots of errors like:

50  02/01/14 12:08:54.833   luup_log:11: dataMine: 1:Unable to open file for read - /dataMine/database/4/raw/2300.txt <0x2d6fa680>

I fixed this this through a little shell script:

cd /dataMine
mkdir database
cd database
for x in 1 2 3 4 5 6 7 8 9
do
mkdir $x
mkdir $x/raw
for y in 0 1 2 3 4 5 6 7 8 9
do
mkdir $x$y
mkdir $x$y/raw
done
done

Here are some initial questions I had:

  • How do I change update frequency? (Answer: You don’t, you record on state changes.)
  • How can I export? (Answer: You ssh to the data directories, and pull out the raw data.)
  • How can I match the scales of two different variables? (This one I don’t know.)

It seems all the data are stored in this structure:

root@MiOS_35017272:/dataMine/database# find . -iname "*.txt"
./4/raw/2300.txt
./1/raw/2300.txt
./3/raw/2300.txt

Pretty basic, even though the data are very straightforward (unix time stamp and value):

root@MiOS_35017272:/dataMine/database# more ./3/raw/2300.txt
1391275155,64

Now, I can make these plots:

Display from dataMine app

Display from dataMine app

This is direct from the dataMine app, which has really great browsing capability, but I can’t get the axes to match.

Custom Plot (using MATLAB)

Custom Plot (using MATLAB)

Here is a much better plot with a common axis and appropriate scaling. It is interesting how slowly my house cooled. I need to compare this with local temperatures.

Any thoughts appreciated in the comments.

Resources

By 3 Comments

Review: Lone Survivor Movie

Today, I watched the Lone Survivor with two friends and was once again reminded of the brevity of life, the importance of principle and the value of friendship. I found the movie to be a masterful combination of plot, emphasis and character development. It is tough to capture the breadth emotions present in Marcus Luttrell’s detailed book in two hours, but the movie does capture the heartbreak, the passion, the sacrifice of modern warfare — all juxtaposed with the rapid transition from the safety, comfort and contentedness of modern forward bases. Further contrast comes from American optimism (“Good things happen to good people, right?”) with middle-eastern determinism, but nothing hits you harder than the fierce determination of the seals contrasted with our comfortable armchair warrior lives.

Continue reading →

By 3 Comments

WordPress Automatic Updates

If you want to get WordPress to accept automatic updates and are running your own server, you want the flexibility to not go through these steps every time an update arrives. You also want good security. I had to do this recently on several sites and thought I would share my notes.

Continue reading →

By One Comment

TelexFree — Higher fidelity model

My previous post on TelexFree was the result of a quick spreadsheet model that assumed a geometric growth rate that was uniformly assumed for all members. On a recent flight, I built an object oriented model in MATLAB to more realistically show how telex works. This is a really fascinating system.

Continue reading →

By 6 Comments

TelexFree — A quick business case assessment

Ever wonder why fund managers can’t beat the S&P 500? ‘Cause they’re sheep — and the sheep get slaughtered. I been in the business since ’69. Most of these high paid MBAs from Harvard never make it. You need a system, discipline, good people, no deal junkies, no toreadores, the deal flow burns most people out by 35. Give me PSHs — poor, smart and hungry. And no feelings. You don’t win ’em all, you don’t love ’em all, you keep on fighting . . . and if you need a friend, get a dog . . . it’s trench warfare out there sport and in here too. — Gordon Gecko

I built a much better model with more realistic constraints available here

Outside of generating new information or product, risk, diversity and time horizon are only variables I’m convinced an investor can control. This means invest broadly over long time horizons and keep taxes and expenses low. If you want outsized returns, you must take on more risk or make something people want to buy. Most wealth is created by businesses making real products, but wealth can still accumulate from appreciating assets (real estate, land, gold, internet domain names, etc). However, both these methods take a lot of time and effort. Can you make a lot more quickly through selling VOIP services, posting internet ads and joining a Brazilian-focused multi-level marketing (MLM) club called TelexFree?

Continue reading →

By 39 Comments

“I’m Busy”

As we enter the Christmas/holiday season, I’m asking lots of friends how they are doing and how their year has been. Almost everyone says “it’s busy”. This takes many forms, from “crazy busy” to “absolutely busy” or “scary busy”. Since words often lose their meaning, I’ve thought about a scale we could use to better convey the level of busyness in our lives. I think the best way to delineate levels is by the loss of freedom that occurs at each inflection point.

Continue reading →

By 4 Comments

New Blog Title

I’ve always been intrigued by the story of the Harvard applicant who penned a three word answer to an essay prompt of “what is courage?” The myth has that he was accepted. I like the story because it highlights the risk and reward trade that we calculate in all our decisions.

Continue reading →

By 0 Comments

WordPress Development Workflow

When I work with WordPress, I am still a ruby coder in a PHP environment. I spent several years doing PHP only and have watched the language develop nicely, but still like to focus on Concept/Design, CSS3, Ruby and JavaScript when I’m doing web development work. In the hope that someone might benefit from my workflow, I’m sharing my basic setup.

Continue reading →

By 0 Comments