Safe expression evaluation in Ruby for user submitted formulas

TL;DR; checkout Keisan for evaluation of user-submitted expressions and formulas. The gem has plenty of built-in functions and can be easily extended.

Our reporting API powers all the data driven results on Keypup, such as the Priority Inbox, Team Board and Data Explorer. Recently we decided to try out a small MVP allowing us to pass dynamic expressions as part our report API calls to perform post-processing (~ data mapping/formatting).

Our objective with that MVP was to allow report queries to become self-contained (e.g. chart queries) and reduce the amount of data massage we have to do in frontend.

The main problem with any kind of user input evaluation is security of course. You do not want to use Ruby eval or give expression evaluators access to the context of your app (i.e. give access to Rails.application.credentials for instance).

So we needed a purely interpreted expression processor with no direct connection to Ruby when evaluating expressions. A few Google searches later we ended up finding...

Keisan, expression parser and evaluator

This gem provides an expression evaluator which is close enough to Ruby but without running expressions in the Ruby context.

Instead expressions provided to Keisan are parsed as an Abstract Syntax Tree (AST) then each node in the tree gets evaluated. Each evaluation is done on primitive values without access to the Ruby context. Invoking a function which is not defined in the evaluator (e.g. Rails.application.credentials) will raise an error.

Because expressions are interpreted from scratch using a string parser and a whitelist of functions, this is considerably safer than any form of eval.

Thanks to Keisan we can safely allow our reporting API to accept dynamic processors:

Prototype allowing clients to post-process API data

Getting started with Keisan

First install the gem

Then open an irb console and start playing with the evaluator. Keisan provides a whole lot of base functions to play with, so don't be shy.

Convenient isn't it? Now let's move onto Keisan's most powerful feature.

Defining your own functions

Keisan allows you to extend its functionalities with custom functions.

Custom functions can be defined within the evaluator or in Ruby directly.

Because Keisan can be extended with custom functions, you can create very powerful and flexible calculators.

Let's say you wish to give your app administrators a web UI allowing them to derive stats from your data. You could go as far as providing Keisan functions retrieving data from your database and let them play with them. Let's see what this would look like in Rails.

Creating your own calculator

Alright, you know almost everything there is to know about Keisan. So what about creating your own calculator?

Below is an example of custom calculator with predefined functions.

You may note two things about the calculator above which are related to security. I said earlier Keisan was safe...it is, mostly...except for Denial of Service (DoS) attacks using infinite loops. Fortunately there is way to disable them.

First we disable recursive functions. This prevents users from defining functions looping indefinitely.

Second we disable the while loop. This also prevents users from entering never ending expressions.

That's it. Now feel free to get creative!

I think Keisan can be particularly appealing for any app / admin UI where formulas must be maintained and saved in database (think tax calculations, premium calculation etc..). Keisan provides a safe and extensible way of handling these use cases.

About us

Keypup is on a mission to help developers and tech leads work better with each others on development projects. Our platform automatically centralizes, prioritizes and assigns people and actions on issues and pull requests to optimize your development flow.

Don't get lost because you have to juggle with twenty pull requests across five development projects. We'll clean and organize that for you to ensure a smooth landing.

---

Code snippets hosted with ❤ by GitHub
Thumbnail designed by Vexels.com
Banner designed by pngtree.com

Don't miss these stories: