Faster Deployment and Testing using Feature Toggling techniques — Split IO

Overview:

Leading organizations always aim to deliver their best to the users. Some of the important aspects these organizations strive towards are providing all the features their users desire and providing a high level of availability of their systems to their users. Deploying and testing the functionalities are important aspects of high quality systems and organizations are constantly trying to improve the experience of their users. Here, in this blog, we explore how a tool Split, which uses a technique called feature flags or feature toggles, enables organizations to deploy, test and provide multiple functionalities in a more convenient way in production. We provide a demonstration of how this tool, Split, can be used for a Machine Learning System which provides movie recommendations to its users.

Introduction:

Feature flags or feature toggles form a part of a technique in software engineering which allow organizations/teams to modify the behavior of their system without having to change the underlying code. They can toggle between two different functionalities or turn on and off a functionality during runtime without the need to deploy the whole code again. In more simpler terms, they can be considered to be similar to turning a fan on or off with a button, without having to explicitly remove the plug from the socket.

About Split.io and problems addressed:

Split is a feature delivery platform which utilises feature flags and makes it not only possible to release new features or target features to particular users, it also uses metrics related to customer experience and allows organizations to measure the impact of these features on their users. It is built in a way that allows new ideas to be easily tested and organizations can start measuring their impact instantaneously. In addition, it also allows organizations to modify or add additional configurations without going into the code.

Setting up Split.io

Step 1: Create an account

Split.io is primarily a paid software. However, for the purpose of exploring the tool, we signed up for the 14-day free trial version. The best part about creating an account for free trial is that they don’t ask for a credit card to sign-up.

After creating an account, you can start to set up a simple split (‘feature-flag’) that you can import as a configuration in your code.

Creating a split is extremely simple. You need to provide a name for the split — something meaningful based on the type of feature split you want to create. The traffic type is “user” because we want to enable/disable flows for different types of users.

We can also add tags for easy search and creating groups of feature-flags. For example, if there are 2–3 different feature flows for a particular large scale release, you can tag the feature splits under a single tag.

Finally, provide a description of the feature split and you are good to go!

Once you complete creating a split you will be asked to choose the environment you want to enable the feature in and also the rules you want to add.

We added a split called feature_enabled and chose 2 rules — “on” and “off”. Once you are done, hit the save changes button.

Note: By default the feature is “off”.

Now let’s look at using it in our code.

Step 1: Using split.io in you code

The documentation for setting up split.io split management is extremely good. The documentation can be found at the following link — https://help.split.io/hc/en-us

In this example we wanted to show how it can be set up for a python project.

Here are the following steps:

1. Getting started with using the python SDK (software development kit) of split.io

To install the python SDK client run the following command:

pip install splitio_client==8.4.0

This should successfully install the python client.

2. Next up, we need to import the library into our code in order to be able to use it.

from splitio import get_factory
from splitio.exceptions import TimeoutException

Here, we first import the get_factory method which is the factory to create the splitio client. The second line is used to handle TimeOutExceptions in case the splitio get_factory is not able to return a factory object within the desired time period.

3. Next, let’s try to load the client. For this, you need to use your API key. You can find your API key under the Admin Settings, API keys section.

factory = get_factory(‘YOUR_API_KEY’)
try:
 factory.block_until_ready(5) # wait up to 5 seconds
except TimeoutException:
 # Now the user can choose whether to abort the whole execution, or just keep going
 # without a ready client, which if configured properly, should become ready at some point.
 pass

In the above code, the get_factory method takes in your API key and tries to download the factory object. If the factory object is not ready in 5 seconds, the code will throw a TimeoutException. This means you can retry the previous operation or abort.

When the SDK is instantiated in in-memory mode, it kicks off background tasks to update an in-memory cache with small amounts of data fetched from Split servers. This process can take up to a few hundred milliseconds depending on the size of data. If the SDK is asked to evaluate which treatment to show to a customer for a specific split while it’s in this intermediate state, it may not have the data necessary to run the evaluation. In this case, the SDK doesn’t fail, rather, it returns the control treatment. To make sure the SDK is properly loaded before asking it for a treatment, block until the SDK is ready.

4. Let’s assume that the factory object was ready in 5 seconds.

We can then go on to obtain our split client and try out some flows.

split = factory.client()
treatment = split.get_treatment(user_id, # unique identifier for your user
 ‘feature_enabled’)
if treatment == ‘on’:
 print(“feature is on”)
 # insert on code here
elif treatment == ‘off’:
 print(“feature is off”)
 # insert off code here
else:
 print(“default flow”)
 # insert control code here

Trying out the code gives us — Feature is off as we have not yet enabled the feature.

$ python split_io_demo.py
Feature is off

Now, let’s use the split.io UI to set the flag value to On.

In your splits portal, select the feature_enabled split and then scroll down to the set default rule

Change the rule to On. Once you click Save Changes, you will have to confirm in the following screen again.

Now you will still see that we get the value as off. This is because we have not directed any traffic to this rule. Let’s go ahead and do that.

Under the allocate traffic section, move the bar to whatever percentage of the user base you want to target with this rule.

For the purpose of this demo, let us make it 100%. You can choose an arbitrary percentage based on your use case.

Now, if we run the code

$ python split_io_demo.py
feature is on

Finally, we get the feature on.

Use Case: Movie Recommendation System

Now I want to demonstrate the use of this feature for a recommendation system similar to Netflix wherein we want to choose different models based on the feature enablement.

Suppose we have 2 different ML models and we want to know which model to use based on the user or based on some form of percentage roll-out. We can set the treatment name to a particular model name and accordingly load that model.As part of our group project for the Machine Learning in Production course, we worked on developing ML models for movie recommendations.

We have 2 types of models, user based, where the clustering of movies is done for each user and item based model where the clustering is done for each movie.

Initially I will use the user based model and later move on to enable the item based model.

Below is the demo for this scenario:

Demo: Movie Recommendation system

In the earlier implementation, we had to manually go into the code and fix the version of the model by commenting out the one version and enabling the other version.

Here is the code snippet for our recommendation api

model = UserBasedModel()
# model = ItemBasedModel()
@model_server.route(‘/recommend/<user_id>’)
def serve_recommendations(user_id):
 print(request)
 rankings = model.ranking(int(user_id), 20)
 return rankings

As you can see above, model = UserBasedModel() is used and ItemBasedModel() is commented out. To avoid such a scenario, we can use split.io to easily manage our configurations.

Here is how that can be done.

We created a new feature flag called model_version and added two treatment settings -

1. user-based
2. Item-based

This can be then used in our code as:

model1 = UserBasedModel()
model2 = ItemBasedModel()
from splitio import get_factory
from splitio.exceptions import TimeoutException
config = {'ready': 5000}
factory = get_factory('8oe93jbsb8hj7ar7o36dcad0c8p00gskkna6', config=config)
try:
   factory.block_until_ready(5)
except TimeoutException:
   # The SDK failed to initialize in a second. Abort!
   sys.exit()
split = factory.client()
treatment = split.get_treatment('CUSTOMER_ID', 'model_version')
@model_server.route('/recommend/<user_id>')
def serve_recommendations(user_id):
   print(request)
   if treatment == 'user-based':
      rankings = model1.ranking(int(user_id), 20)
   elif treatment == 'item-based':
      rankings = model2.ranking(int(user_id), 20)
   return rankings

Now based on what we enable in split.io, the model changes and returns the different results.

Other Interesting Features

Monitoring Traffic

Choosing Environment

We can choose the environment version to be stage or prod.

Monitoring impact of model

Monitoring the traffic based on which feature gets how much traffic as well as per user monitoring is possible. For the dashboards to be able to show any data a minimum of 14 days of data needs to be present.

Adding additional attributes to the feature

This is a really good feature where you can add additional attributes and configurations to your treatments. Suppose you are using something like this to train a model, choosing the model hyperparameters can be an easy configuration to push to all servers running the training. For example, the following screenshot shows how different parameters can be set as configs.

We can include this in the code as follows -

treatment, raw_config = client.get_treatment_with_config(‘key’,’SPLIT_NAME’, attributes)
configs = json.loads(raw_config)

Advantages and Limitations

Advantages

1. The tool provides a simple and easy to use user interface.
2. The documentation provided for this tool is clear to understand for anyone getting started with this tool.
3. Extremely flexible in terms of data type (can be string, boolean or json object).
4. Adding additional configurations to the feature flag is a really useful feature especially if we want to add extra information along with basic feature enablement flow control.
5. Split does not access or store any user-identification information on its servers and it leverages industry standard security practices.
6. Split works with multiple languages like Java, JavaScript, Python, Go etc. and it also provides a generic SDK wrapped with a REST API endpoint called the Split Evaluator to support any code, in any language, front-end or back-end.
7. Based on releases, multiple feature flags can be grouped together under a single tag.
8. There is a capability to do specified percentage rollouts as well as target user based feature toggling.

Limitations:

1. It is a paid software — based on the usage and number of features added, the charge gets calculated.
2. The monitoring capabilities work only if the product has been used for 14 days which does not give a clear idea about the feature during the trial period.
3. The factory method of importing the splitioclient takes at least 5 seconds to download.

By Ganesh Ramadurai and Vasudev Luthra

Carnegie Mellon University