Quick Start

This guide assumes you have deployed your own empty instance of Rafiki and you want to try a full train-inference flow as the Super Admin:

  1. Authenticating on Rafiki

  2. Submitting models

  3. Uploading datasets

  4. Creating a model training job

  5. Creating a model serving job after the model training job completes

  6. Making predictions

Follow the sequence of examples below to submit the Fashion MNIST dataset for training and inference. Alternatively, refer and run the scripted version of this quickstart ./examples/scripts/quickstart.py.

To learn more about what else you can do on Rafiki, explore the methods of rafiki.client.Client.

Note

If you haven’t set up Rafiki on your local machine, refer to Quick Setup before continuing.

Note

Installing the client

  1. Install Python 3.6 such that the python and pip point to the correct installation of Python (see Installing Python)

  2. Clone the project at https://github.com/nginyc/rafiki (e.g. with Git)

  3. Within the project’s root folder, install Rafiki’s client-side Python dependencies by running:

    pip install -r ./rafiki/requirements.txt

Initializing the client

Example:

from rafiki.client import Client
client = Client(admin_host='localhost', admin_port=3000)
client.login(email='superadmin@rafiki', password='rafiki')

See also

rafiki.client.Client.login()

Creating models

To create a model, you’ll need to submit a model class that conforms to the specification by rafiki.model.BaseModel, written in a single Python file. The model’s implementation should conform to a specific task (see Supported Tasks).

Refer to the parameters of rafiki.client.Client.create_model() for configuring how your model runs on Rafiki, and refer to Model Development Guide to understand more about how to write & test models for Rafiki.

Example:

client.create_model(
    name='TfFeedForward',
    task='IMAGE_CLASSIFICATION',
    model_file_path='examples/models/image_classification/TfFeedForward.py',
    model_class='TfFeedForward',
    dependencies={ 'tensorflow': '1.12.0' }
)

client.create_model(
    name='SkDt',
    task='IMAGE_CLASSIFICATION',
    model_file_path='examples/models/image_classification/SkDt.py',
    model_class='SkDt',
    dependencies={ 'scikit-learn': '0.20.0' }
)

See also

rafiki.client.Client.create_model()

Listing available models by task

Example:

client.get_available_models(task='IMAGE_CLASSIFICATION')

Output:

[{'access_right': 'PRIVATE',
 'datetime_created': 'Mon, 17 Dec 2018 07:06:03 GMT',
 'dependencies': {'tensorflow': '1.12.0'},
 'id': '45df3f34-53d7-4fb8-a7c2-55391ea10030',
 'name': 'TfFeedForward',
 'task': 'IMAGE_CLASSIFICATION',
 'user_id': 'fb5671f1-c673-40e7-b53a-9208eb1ccc50'},
 {'access_right': 'PRIVATE',
 'datetime_created': 'Mon, 17 Dec 2018 07:06:03 GMT',
 'dependencies': {'scikit-learn': '0.20.0'},
 'id': 'd0ea96ce-478b-4167-8a84-eb36ae631235',
 'name': 'SkDt',
 'task': 'IMAGE_CLASSIFICATION',
 'user_id': 'fb5671f1-c673-40e7-b53a-9208eb1ccc50'}]

See also

rafiki.client.Client.get_available_models()

Creating datasets

You’ll first need to convert your dataset into a format specified by one of the tasks (see Supported Tasks), and split them into two files: one for training & one for validation. After doing so, you’ll create 2 corresponding datasets on Rafiki by uploading them from your filesystem.

Example (pre-processing step):

# Run this in shell
python examples/datasets/image_files/load_mnist_format.py

Example:

client.create_dataset(
    name='fashion_mnist_train',
    task='IMAGE_CLASSIFICATION',
    dataset_path='data/fashion_mnist_train.zip'
)

client.create_dataset(
    name='fashion_mnist_val',
    task='IMAGE_CLASSIFICATION',
    dataset_path='data/fashion_mnist_val.zip'
)

Output:

{'id': 'ecf87d2f-6893-4e4b-8ed9-1d9454af9763',
'name': 'fashion_mnist_train',
'size_bytes': 36702897,
'task': 'IMAGE_CLASSIFICATION'}

{'id': '7e9a2f8a-c61d-4365-ae4a-601e90892b88',
'name': 'fashion_mnist_val',
'size_bytes': 6116386,
'task': 'IMAGE_CLASSIFICATION'}

See also

rafiki.client.Client.create_dataset()

Note

The code that preprocesses the original Fashion MNIST dataset is available at ./examples/datasets/image_files/load_mnist_format.py.

Creating a train job

To create a model training job, you’ll specify the train & validation datasets by their IDs, together with your application’s name and its associated task.

After creating a train job, you can monitor it on Rafiki Web Admin (see Using Rafiki’s Web Admin).

Refer to the parameters of rafiki.client.Client.create_train_job() for configuring how your train job runs on Rafiki, such as enabling GPU usage & specifying which models to use.

Example:

client.create_train_job(
    app='fashion_mnist_app',
    task='IMAGE_CLASSIFICATION',
    train_dataset_id='ecf87d2f-6893-4e4b-8ed9-1d9454af9763',
    val_dataset_id='7e9a2f8a-c61d-4365-ae4a-601e90892b88',
    budget={ 'MODEL_TRIAL_COUNT': 5 }
)

Output:

{'app': 'fashion_mnist_app',
'app_version': 1,
'id': 'ec4db479-b9b2-4289-8086-52794ffc71c8'}

See also

rafiki.client.Client.create_train_job()

Listing train jobs

Example:

client.get_train_jobs_of_app(app='fashion_mnist_app')

Output:

[{'app': 'fashion_mnist_app',
'app_version': 1,
'budget': {'MODEL_TRIAL_COUNT': 5},
'datetime_started': 'Mon, 17 Dec 2018 07:08:05 GMT',
'datetime_stopped': None,
'id': 'ec4db479-b9b2-4289-8086-52794ffc71c8',
'status': 'RUNNING',
'task': 'IMAGE_CLASSIFICATION',
'val_dataset_id': '7e9a2f8a-c61d-4365-ae4a-601e90892b88',
'train_dataset_id': 'ecf87d2f-6893-4e4b-8ed9-1d9454af9763'}]

See also

rafiki.client.Client.get_train_jobs_of_app()

Creating an inference job with the latest train job

To create an model serving job, you’ll have to wait for your train job to stop. Then, you’ll submit the app name associated with the train job (with a status of STOPPED). The inference job would be created from the best trials from that train job.

Example:

client.create_inference_job(app='fashion_mnist_app')

Output:

{'app': 'fashion_mnist_app',
'app_version': 1,
'id': '0477d03c-d312-48c5-8612-f9b37b368949',
'predictor_host': '127.0.0.1:30001',
'train_job_id': 'ec4db479-b9b2-4289-8086-52794ffc71c8'}

See also

rafiki.client.Client.create_inference_job()

Listing inference jobs

Example:

client.get_inference_jobs_of_app(app='fashion_mnist_app')

Output:

{'app': 'fashion_mnist_app',
  'app_version': 1,
  'datetime_started': 'Mon, 17 Dec 2018 07:15:12 GMT',
  'datetime_stopped': None,
  'id': '0477d03c-d312-48c5-8612-f9b37b368949',
  'predictor_host': '127.0.0.1:30000',
  'status': 'RUNNING',
  'train_job_id': 'ec4db479-b9b2-4289-8086-52794ffc71c8'}

See also

rafiki.client.Client.get_inference_jobs_of_app()

Making predictions

Send a POST /predict to predictor_host with a body of the following format in JSON:

{
    "query": <query>
}

…where the format of <query> depends on the associated task (see Supported Tasks).

The body of the response will be of the following format in JSON:

{
    "prediction": <prediction>
}

…where the format of <prediction> depends on the associated task.

Example:

If predictor_host is 127.0.0.1:30000, run the following in Python:

predictor_host = '127.0.0.1:30000'
query_path = 'examples/data/image_classification/fashion_mnist_test_1.png'

# Load query image as 3D list of pixels
from rafiki.model import utils
[query] = utils.dataset.load_images([query_path]).tolist()

# Make request to predictor
import requests
res = requests.post('http://{}/predict'.format(predictor_host), json={ 'query': query })
print(res.json())

Output:

{'prediction': [0.9364003576825639, 1.016065009906697e-08, 0.0027604885399341583, 0.00014587241457775235, 6.018594376655528e-06, 1.042887332047826e-09, 0.060679372351310566, 2.024707311532037e-11, 7.901770004536957e-06, 1.5299328026685544e-08],
'predictions': []}

Stopping a running inference job

Example:

client.stop_inference_job(app='fashion_mnist_app')

See also

rafiki.client.Client.stop_inference_job()