Offline Execution

Table of Contents

Introduction

You have ported your existing machine learning experiment to use Neptune or you have written down the whole new one taking advantage of all Neptune features. Your scripts use Neptune objects like Channels, Charts and Actions. You track progress of your experiments on our amazing WebUI and everything is awesome.

But one day, you:

What is common for all above cases is that it would be really convenient to temporarily disable Neptune and use the script as it is. But you have already included all these Context calls and you send values using Channel objects in your script. Does it mean that every time you want to run your job, you need to engage whole Neptune machinery? It seems limiting. What now? Do you need to comment out all Neptune related calls?

Thankfully not, because Neptune provides you Offline Context.

Offline Context

Offline Context is a feature of Neptune enabling you to run your experiments totally offline. What is more, you do not need any modifications in your source code to achieve that. The only thing you need to do to work offline, is to execute your script in a different way. You can add --offline flag to Neptune command call, or simply run your script without using neptune entry point.

You might wonder what happens when you use all these Neptune objects in your script and you are executing your job offline. Well, not much really. Most of the calls do absolutely nothing and simply return. However, there are some exceptions. We assume that you might be interested with on-going partial output of your scripts. Especially, you might be interested in values you would send to channels, if you have been working in online mode. These values get logged in an eye pleasant format in a special file called offline_job.log, created in the directory when you execute your script.

Executing jobs offline with neptune

The easiest way to execute a Neptune job offline is to add --offline argument you your run command. For example:

neptune run --offline -- --use_ai True --animal hedgehog

Neptune parses your config file and parameters provided via command line and makes them accessible via Context as in regular execution.

Executing jobs offline without neptune

Other way to execute a job offline, is to run it using python interpreter. Assuming that your script with job is named main.py and your job does not require any parameters, you can simple type:

python main.py

Other way to execute a job offline, is to run it using Rscript command. Assuming that your script with job is named main.r and your job does not require any parameters, you can simple type:

Rscript main.R

Other way to execute a job offline, is to run it using java -jar command. Assuming that jar file with your code is named main.jar and your job does not require any parameters, you can simple type:

java -jar main.jar

It is all you need to do. Neptune detects how your job script was started and initializes to appropriate running mode.

However, in most cases, your job requires parameters. For example, let’s assume that your job uses four parameters: DOF, temperature, use_ai and animal. Normally you would declare them in neptune.yaml, but this file gets parsed only if the job is executed using neptune. So if you run your script without neptune, parameters from config file cannot be accessed from your job. What is more, we cannot really guess what parameters are used in your script, because they are dynamically accessed via Neptune Context.

So, when omitting neptune entry point, all parameters need to be provided via command line or via offline_parameters arguments of neptune.Context function.

neptune: Trying to access the 'DOF' parameter which is not defined.
         In order to run a Neptune job offline, you need to provide all parameters
         via command line or offline_parameters argument of neptune.Context function.

So, when omitting neptune entry point, all parameters need to be provided via command line or via offlineParams function.

Error in params("DOF") : 
  neptune: Trying to access the 'DOF' parameter which is not defined.
           In order to run a Neptune job offline, you need to provide all parameters
           via command line or offlineParams function.

So, when omitting neptune entry point, all parameters need to be provided via command line or via jobParameters argument of NeptuneContextFactory.createContext function.

neptune: Trying to access the 'DOF' parameter which is not defined.
         In order to run a Neptune job offline, you need to provide all parameters
         via command line or JobParameters argument of NeptuneContextFactory.createContext.

Note: Your script is not parsed in any way and we do not check if all parameters are provided before job execution. Whenever your job tries to access an undeclared parameter, you will be warned with information similar to following and your job will exit.

Providing job with parameters

There are a few ways or providing your job with parameters in offline mode.

You can provide job parameters via command line. Types of parameters passed via command line are automatically recognized under the hood, and you can be sure that you will get appropriate types of parameters in your script. Rules of parameter type guessing are simple:

  • bool if it is literally True or False,
  • int if possible,
  • float if possible,
  • str otherwise.

For example:

python main.py --DOF 3 --temperature 176.1 --use_ai True --animal hedgehog

In this example, types of parameters would be, consecutively: int, float, bool and str.

  • logical if it is literally TRUE or FALSE,
  • integer if possible,
  • double if possible,
  • character otherwise.

For example:

Rscript main.R --DOF 3 --temperature 176.1 --use_ai TRUE --animal hedgehog

In this example, types of parameters would be, consecutively: integer, double, logical and character.

  • Boolean if it is literally true or false,
  • Integer if possible,
  • Double if possible,
  • String otherwise.

For example:

java -jar main.jar --DOF 3 --temperature 176.1 --use_ai true --animal hedgehog

In this example, types of parameters would be, consecutively: Integer, Double, Boolean and String.

If you are tired of providing arguments via command line, you can also provide them via neptune.Context() call. This might be useful if you work in offline mode work for a while, and you need to turn off your machine, but you want preserve these super promising values of parameters you were working with.

Simply add parameters to neptune.Context() call:

from deepsense import neptune

neptune.Context(
    offline_parameters={
        'DOF': 3,
        'temperature': 176.1,
        'use-ai': True,
        'animal': 'hedgehog'
        }
    )

You can also declare properties and tags using neptune.Context call.

This is enough to make Neptune work in offline mode with all parameters provided. Now you can simply run your parametrized job with:

python main.py

Simply call offlineParams() function after library load:

library(neptune)

offlineParams(DOF=3, temperature=176.1, 'use-ai'=TRUE, animal="hedgehog")

You can also set properties and tags calling appropriate functions.

This is enough to make Neptune work in offline mode with all parameters provided. Now you can simply run your parametrized job with:

Rscript main.R

Simply add parameters to NeptuneContextFactory.createContext() call:

import io.deepsense.neptune.clientlibrary.NeptuneContextFactory;
import io.deepsense.neptune.clientlibrary.models.impl.parameters.OfflineJobParameters;
import io.deepsense.neptune.clientlibrary.models.NeptuneContext;

public class Main {

public static void main(String[] args) throws InterruptedException {
    NeptuneContext context = NeptuneContextFactory.createContext(args,
            new OfflineJobParameters()
                    .put("DOF", 3)
                    .put("temperature", 176.1)
                    .put("use-ai", true)
                    .put("animal", "hedgehog"));
    }
}

You can also declare properties and tags using NeptuneContext call.

This is enough to make Neptune work in offline mode with all parameters provided. Now you can simply run your parametrized job with:

java -jar main.jar