Marlin Configurator - Part 1

by Thinkyhead - April 10, 2015

All 3D printers require on-board code to translate textual commands (called GCODE) into movement and molten plastic. For this job, most RepRaps (and many commercial 3D printers) rely on Marlin, a compact, adaptable, and streamlined firmware. Marlin's flexibility comes from having a huge number of configurable options. In this article (and accompanying video) I'll be discussing the challenges of building a configurator to help wrangle Marlin's many options, and I'll be showing off some cool Javascript tricks in the process.

Configuring with Compiler Directives

Let's start by taking a look at how Marlin is currently configured. As an embedded Arduino application, Marlin is written entirely in C and C++. It contains two main configuration files, Configuration.h and Configuration_adv.h. I won't paste the entire contents here, but here are some lines from the main "PID" settings:

// Comment the following line to disable PID and enable bang-bang.
#define PIDTEMP
#define BANG_MAX 255 // limits current to nozzle while in bang-bang mode; 255=full current
#define PID_MAX BANG_MAX // limits current to nozzle while PID is active (see PID_FUNCTIONAL_RANGE below); 255=full current
#ifdef PIDTEMP
  //#define PID_DEBUG // Sends debug data to the serial port.
  //#define PID_OPENLOOP 1 // Puts PID in open loop. M104/M140 sets the output power from 0 to PID_MAX
  //#define SLOW_PWM_HEATERS // PWM with very low frequency (roughly 0.125Hz=8s) and minimum state time of approximately 1s useful for heaters driven by a relay
  //#define PID_PARAMS_PER_EXTRUDER // Uses separate PID parameters for each extruder (useful for mismatched extruders)
                                    // Set/get with gcode: M301 E[extruder number, 0-2]
  #define PID_FUNCTIONAL_RANGE 10 // If the temperature difference between the target temperature and the actual temperature
                                  // is more then PID_FUNCTIONAL_RANGE then the PID will be shut off and the heater will be set to min/max.
  #define PID_INTEGRAL_DRIVE_MAX PID_MAX  //limit for the integral term
  #define K1 0.95 //smoothing factor within the PID
  #define PID_dT ((OVERSAMPLENR * 10.0)/(F_CPU / 64.0 / 256.0)) //sampling period of the temperature routine

// If you are using a pre-configured hotend then you can use one of the value sets by uncommenting it
// Ultimaker
    #define  DEFAULT_Kp 22.2
    #define  DEFAULT_Ki 1.08
    #define  DEFAULT_Kd 114
#endif // PIDTEMP

Not very pretty. Should these values be edited? Even seasoned tinkerers and hackers get queasy around such unfamiliar names and numbers. There has to be an easier way!

The Marlin configuration files are pretty well documented, and in fact you can ignore most settings. But even the minimal settings are more than enough to make your head spin. And the most important options aren't necessarily the ones users find first. Most of the time users will start with a configuration tailored for a machine like theirs, then — usually as they tune the extruder — they will have to cautiously deal with settings like DEFAULT_STEPS_PER_MM and a few others. Builders of custom 3D printers have to tune even more values, but presumably they know what they are doing down even to the pins level.

So one of the more common requests is to make Marlin easier to configure. But where do you even begin? We can't throw out the entire system, which actually works pretty well, and we don't want to add more steps to an already complicated process. But this area needs to be explored, and so an experimental configuration utility has been added to Marlin.

Enter the Marlin Configurator

The Marlin Configurator is written in Javascript so that it can run in any web browser and doesn't require any specific operating system. It connects to Github to get the latest configuration files. (This method also allows it to run in file:// mode in the browser, which disallows reading local files.) It parses and writes standard Marlin configuration files, which have been annotated so configuration utilities can group related settings together.

You can run the latest version of the Marlin Configurator directly from my Github repository any time. [As of this writing (Oct. 3, 2015) it's mostly working with current Marlin configuration files, but not completely]

Configurator Development

The configurator is near completion of Phase I. It can read the configurations, present a form for directly editing the settings, and even disable (or hide) settings that don't currently apply. It only lacks the ability to catch and process "overrides" (and it does not yet account for the newest additions to the configuration system, Conditionals.h and SanityCheck.h, which both provide additional information about how configuration options work together, and could provide warnings).

In Phase II of development the configurator will account for overridden values: cases where one setting forces other settings to be enabled, disabled, or changed. It will use SanityCheck.h to apply extra rules about which options are allowed to be set together, and which values are legal. And it will get overrides from Conditionals.h. There will be better organization of values.

As soon as possible, in Phase III I want to make a simplified interface, then the current UI can become the "All Settings" view. The simplified interface will go step-by-step through the settings, asking for simpler information like the type of printer, number of motor steps, the number of teeth in the extruder gears, and try to figure our steps-per-mm so you don't have to. But calculators for these values can also be included.

Have to start somewhere…

There are a lot of ways a configurator could be written, and one popular idea is to use Python to implement a GUI, invoke the build tools, and even flash the printer. (Something a Javascript app can't do without a companion native app or host.) I support any approach that makes Marlin easier to configure, but I needed to start with the essentials of being able to easily read and write these things.

The Code

What Code, Where

I could paste a lot of code here, but I will leave the majority for you to peruse at Github. The Javascript application code that I will be discussing most is in the file Marlin/configurator/js/configurator.js — implemented using a common "Singleton" pattern. I've used jQuery and a couple of other add-on libraries to keep the code light.

Parsing Configurations

The code makes liberal use of regular expressions to find the relevant configuration lines, and probably ends up parsing them a lot like the gcc compiler preprocessor might, with similar results. The configurator produces a structured representation of the settings which could be used in many ways. For example, the configurator uses this data to create intelligent form fields.

I'm sure it will require a multi-part article to discuss all the pieces of this tool, but let's start with a small teaser — the code to download the configuration files from Github. You'll notice, as mentioned, that configurator.js is implemented as a "singleton." This singleton uses the closure feature of Javascript to create a private scope, and returns an object that represents the singleton, still bound to its enclosed scope. We have to call "init" on the object so it can –at minimum– assign a private "self" variable for those instances where "this" is reassigned and it needs to call its other functions.

window.singletonApp = (function(){
  var self, myPrivateVar = 0;
  return {
    init: function() {
      self = this;
    },
    myFunction: function(a, b, c) {
      return this.myOtherFunction(Math.sqrt(a*a+b*b+c*c));
    },
    myOtherFunction: function(d) {
      return d * 10;
    },
    EOF: null
  };  
})();
window.singletonApp.init();

First we need some way to configure different kinds of connections. So we'll just construct an ad-hoc object with the info we need and build code around it:

 // GitHub
 // Warning! Limited to 60 requests per hour!
var config = {
  type:  'github',
  host:  'https://api.github.com',
  owner: 'MarlinFirmware',
  repo:  'Marlin',
  ref:   'Development',
  path:  'Marlin/configurator/config'
};
function github_command(conf, command, path) {
  var req = conf.host+'/repos/'+conf.owner+'/'+conf.repo+'/'+command;
  if (path) req += '/' + path;
  return req;
}
function config_path(item) {
  var path = ', ref = ';
  switch(config.type) {
    case 'github':
      path = github_command(config, 'contents', config.path);
      if (config.ref !== undefined) ref = '?ref=' + config.ref;
      break;
    case 'remote':
      path = config.host + '/' + config.path + '/';
      break;
    case 'local':
      path = config.path + '/';
      break;
  }
  return path + '/' + item + ref;
}

Now we'll need some local variables to implement the loader:

var self,
    has_boards = false, has_config = false, has_config_adv = false,
    boards_file = 'boards.h',
    config_file = 'Configuration.h',
    config_adv_file = 'Configuration_adv.h',
    config_file_list = [ boards_file, config_file, config_adv_file ];

Now, jumping down to the end of the "init" function, here's the actual loader. This will handle all the ad-hoc protocols we defined above, but it's most especially handy with Github. No doubt you will find this handy if you ever need to grab up-to-date data from Github and use it in a web app, as we have.

// Read boards.h, Configuration.h, Configuration_adv.h
var ajax_count = 0, success_count = 0;
var loaded_items = {};
var isGithub = config.type == 'github';
var rateLimit = 0;
$.each(config_file_list, function(i,fname){ // (1)
  var url = config_path(fname); // (2)
  $.ajax({
    url: url,
    type: 'GET',
    dataType: isGithub ? 'jsonp' : undefined, // (3)
    async: true,
    cache: false,
    error: function(req, stat, err) { // (4)
      if (req.status == 200) { // (5)
        if (typeof req.responseText === 'string') { // (6)
          var txt = req.responseText;
          loaded_items[fname] = function(){ self.fileLoaded(fname, txt, true); }; // (7)
          success_count++;
          // self.setMessage('The request for "'+fname+'" may be malformed.', 'error');
        }
      }
      else {
        self.setRequestError(req.status ? req.status : '(Access-Control-Allow-Origin?)', url); // (8)
      }
    },
    success: function(txt) {
      if (isGithub && typeof txt.meta.status !== undefined && txt.meta.status != 200) {
        self.setRequestError(txt.meta.status, url); // (9)
      }
      else {
        // self.log(txt, 1);
        if (isGithub) { // (10)
          rateLimit = {
            quota: 1 * txt.meta['X-RateLimit-Remaining'],
            timeLeft: Math.floor(txt.meta['X-RateLimit-Reset'] - Date.now()/1000),
          };
        }
        if (isGithub) txt = decodeURIComponent(escape(atob(txt.data.content.replace(/\s/g,'')))); // (11)
        loaded_items[fname] = function(){ self.fileLoaded(fname, txt, true); }; // (12)
        success_count++;
      }
    },
    complete: function() {
      ajax_count++;
      if (ajax_count >= config_file_list.length) { // (13)
        // If not all files loaded set an error
        if (success_count < ajax_count)
          self.setMessage('Unable to load configurations. Try the upload field.', 'error');

        // Is the request near the rate limit? Set an error.
        var r;
        if (r = rateLimit) {
          if (r.quota < 20) { // (14)
            self.setMessage(
              'Approaching request limit (' +
              r.quota + ' remaining.' +
              ' Reset in ' + Math.floor(r.timeLeft/60) + ':' + (r.timeLeft%60+'').zeroPad(2) + ')',
              'warning'
            );
          }
        }
        // Post-process all the loaded files
        $.each(config_file_list, function(){ if (loaded_items[this]) loaded_items[this](); }); // (15)
      }
    }
  });
});

So, let's make some sense of this.

  1. $.each() is the jQuery way to process each array item.
  2. Construct an AJAX path to the configuration.
  3. For Github we need to get "jsonp" data. Otherwise, try raw data.
  4. Might as well put the error handler first.
  5. Even an "error" reply might be a good (200) response...
  6. The responseText must be in the form of a string (not undefined).
  7. Save a function that will be used to process the file. (Javascript!)
  8. For errors show the request status (or the server may need Access-Control-Allow-Origin).
  9. Even on "success" there may still be a "meta" error with Github.
  10. After a successful request, make a note of the request quota.
  11. Github sends a packet, so this code is needed to extract the text.
  12. Again, just add a function that will process the file later.
  13. The complete handler is always fired. Wait until all the files are loaded.
  14. Put up an alert message if the quota is getting close.
  15. Finally, run all the handler functions, but in the required order.

All in all, this is a surprisingly small amount of code to download 3 files directly from Github and call processing functions on them, but that's the modern web!

So that's the end of Part 1. In the next installment I will get into parsing the configuration text using Regular Expressions, turning ugly Marlin compiler directives into slightly less ugly dynamic HTML form fields. There's bound to be a lot of nitty-gritty code wizardry in that article, so I might have to do it as a screen-cast…. We shall see!

∴