Akr.asia is a project I have been working on for the past week or so. Inspired by beemind.me and other clever Beeminder hacks, I decided to make additional integrations for Beeminder as well, but with a focus on creating more of a “framework” for integrations. Akrasia generalizes and abstracts away the boilerplate code (e.g. OAuth, data persistence, etc. code) that is shared among all the integrations, so new integrations can be written very quickly.

Sections

  1. Design Overview
  2. Example Integrations
  3. Defining Integrations
  4. Fetching Data and Returning Results
  5. Screenshots

Design Overview

Previously, if you wanted to create an autofetch-ing integration using the Beeminder API, you would have to do the following:

  1. Register a client app to access the API.
  2. Create an endpoint for user authentication via OAuth.
  3. Set up a database or some way to persist user info/access tokens after they authenticate through OAuth.
  4. Get the user’s goals so you can ask which to connect the integration to.
  5. After the user selects a goal, ask the user for details needed for the integration (e.g. usernames, secrets, other options and credentials).
  6. Persist all this data related to the goal and the integration somewhere.
  7. Write the logic to actually fetch the value for the datapoint for the integration (to be performed when autofetch is called) (which requires reading from the database for the user’s options for the integration).
  8. Write the code to POST the new datapoint to the correct Beeminder goal (which requires reading from the database to get the access token for the user).

Akrasia eliminates the need to do all of these steps except step 7, the actual integration-specific logic. Instead, to create a new integration, you just specify which fields/options are needed by the integration (e.g. username, secrets, etc.) and an HTTP endpoint (which should accept the user’s values for the fields, run the logic to calculate the datapoint value, and call a callback to Akrasia). Akrasia handles all the other steps. So now, autofetch for a new datapoint looks like this:

  1. Beeminder calls Akrasia’s autofetch endpoint.
  2. Akrasia pulls the user’s options for that integration from its database.
  3. Akrasia makes an “autofetch” request to the integration endpoint, passing along the user’s options/configs for the integration.
  4. The integration endpoint receives the user’s options, runs its logic, and POSTs its datapoint to Akrasia.
  5. Akrasia receives that datapoint and POSTs the datapoint to Beeminder (this step is necessary because only Akrasia has the access token for the user, so the integration can’t directly POST the datapoint to Beeminder).

Example Integrations

Below are some example integrations written for Akrasia. The links go to the source code on Glitch, where they can be “remixed” (i.e. forked) as well. Note that they all only implement a single /fetch endpoint handler and don’t utilize any datastore.

  • Last.fm: Scrobble/Play Count
    This integration makes an HTTP request and fetches the datapoint value from the JSON response (and uses a secret key stored in the .env file).
  • Beeminder Forum: Posts Count
    This integration makes an HTTP request and fetches the datapoint value from the JSON response.
  • Project Euler: Problems Solved
    This integration makes an HTTP request and fetches the datapoint value from a CSV-formatted string.
  • Memrise: Points
    This integration fetches a webppage and scrapes the datapoint value from the HTML.
  • Leetcode: Problems Solved
    This integration fetches a webppage and scrapes the datapoint value from the HTML.

Defining Integrations

The fields and endpoints for integrations are defined in JSON. Here is an example of the format, with comments, that Akrasia uses to include an integration:

// This is an example of the JSON that Akrasia will
//   use to configure the integration
{
  // The key is a unique slug that identifies each integration:
  "beeminder_forum_posts": {
    // The title of the integration (to be displayed to users):
    "title": "Beeminder Forum Posts Count",
    
    // The description of the integration
    //   (to be displayed to users) (optional):
    "description": "Use an odometer goal for this integration.",
    
    // The endpoint that Akrasia will call (via POST request)
    //   when it wants a new datapoint fetched. See server.js
    //   for an example implementation to handle this request.
    "fetch_url": "https://forum-minder.glitch.me/fetch",
    
    // The configuration/options for this integration.
    //   Akrasia will ask users to provide these fields
    //   when setting up the integration. These values
    //   will be passed to the fetch_url when Akrasia
    //   makes a request. All values will be passed as
    //   strings. Parsing and validation should be done
    //   by the integration's fetch endpoint.
    "fields": {
      // A unique key for each field. (Fields will
      //   be in request.body.user_options.field_key
      //   when passed to the integration fetch endpoint)
      "forum_username": {
        // The name of the field (to be displayed to users)
        "name": "Beeminder Forum Username",
        
        // Additional description for the field
        //   (to be displayed to users) (optional)
        "description": "Your Beeminder Forum username."
      }
    }
  }
}

If you make a new integration following these patterns, let me know and I can add the definition for the integration to Akrasia.

Fetching Data and Returning Results

When Akrasia receives an autofetch request from Beeminder, it makes a POST request to the integration’s fetch_url, with the user’s configs for the integration in the request body (as JSON) in the following format:

POST <your_integration_fetch_url>
{
  "session": "some-random-uuid",
  "user_options": {
    "my_field_name": "user's input value for my_field_name"
  }
}

The fetch endpoint handler can then pull these details from the request body (this example is in Node.js and Express):

app.post("/fetch", function(request, response) {
  // Get the session and user options from Akrasia's request
  var callbackSession = request.body.session;
  var userOptions = request.body.user_options;
  
  // Get the user's forum name (set and stored through Akrasia)
  //   from the passed options
  var forumUsername = userOptions.forum_username;

The session value (also passed by Akrasia in the request body) is used in the POST request back to Akrasia (after the integration has done its logic and gotten the datapoint) and is used to map the datapoint back to the corresponding Beeminder user and goal. The integration should POST to the following endpoint with a JSON body in the following format when it is ready to add the datapoint:

POST https://akr.asia/integrations/callback
{
  "session": "the-session-received-earlier",
  "result": {
    "value": 3.0, // the datapoint value
    // other permitted keys here are:
    //   timestamp, daystamp, comment, requestid
    // (see Beeminder API documentation for what they do)
  }
}

Screenshots

Finally, here are some screenshots of what this ends up looking like to the user.

After logging in, the user sees their goals, along with which Akrasia integrations are connected, if any:


After clicking “Add” on a goal, the user is presented with a list of all the integrations Akrasia knows of:


After selecting an integration to add, the user is presented with a form to input the options/fields needed for the integration (the form fields come from the JSON definition for the integration):

And that’s it! If you have any questions, feel free to contact me.