Run Notebook Tests

During development or research, you may need to validate your notebook tests. This topic describes how to author, run, and debug notebook tests to ensure that they run as expected.

Copy link to this section

Limitations

Before running notebook tests, note the following limitations for running notebook tests:

1. Notebook tests can be executed only from k8s based clusters.
2. The notebook's kernel must correspond to a seeded ImplLanguage.Runtime.

Copy link to this section

Seeding notebook tests for CI

To test notebooks in CI, the following setup is necessary:

1. The notebook file should be stored in either package resources or in external storage for the package (for example, s3). Usually, .../<pkg>/resources/**/*.ipynb is a good place.

   1. <pkg> denotes your test's package name
   2. /**/ implies you can have any sub-path directory structure

If the notebook is used only for testing you can put it under test resources: .../<pkg>/test/resources/**/*.ipynb.

1. Seed a JupyterNotebook json for this notebook. The seeded file can go in either .../<pkg>/seed/JupyterNotebook/*.json or to .../<pkg>/test/seed/JupyterNotebook/*.json.

2. If you have a dependent file that is needed for a notebook test to run (for example, a csv or txt file), seed a corresponding JupyterFile json for the dependent file. The seeded file can go in either .../<pkg>/seed/JupyterFile/*.json or to .../<pkg>/test/seed/JupyterFile/*.json. After seeding the JupyterFile json, you need to add the JupyterFile to the JupyterNotebook json as a dependency. To do this, add the type, id, name, path of seeded JupyterFile to the notebookDependencies array in the seeded JupyterNotebook's json.

3. Create test file in .../<pkg>/test/notebook/**/test_*.json. This file corresponds to JupyterNotebook.TestSpec, and contains the path to the notebook as defined in seeded JupyterNotebook (not externalFile but path).

   Example of file:

   JSON

   Copy code to clipboard

   {
      "path": "home/notebooks/simple-notebook.ipynb"
   }

   Where home/notebooks/simple-notebook.ipynb is a path to the simple-notebook.ipynb in the JupyterLab file browser. This path can be copied from JupyterLab UI by right-clicking at the notebook name in Jupyter file browser and selecting Copy Path.

   JupyterNotebook.TestSpec has other options like timeoutSeconds or connectionTimeoutSeconds for controlling notebook testing and can be set like in the following example:

   JSON

   Copy code to clipboard

   {
      "path": "home/notebooks/simple-notebook.ipynb",
      "timeoutSeconds": 180,
      "connectionTimeoutSeconds": "180"
   }

   This example reduces notebook execution timeout and connection timeout to 3 minutes. See defaults values and other parameters in documentation for Type JupyterNotebook.TestSpec.

Copy link to this section

See also

• Configure and Use C3 AI Managed JupyterLab Notebooks
• Troubleshoot Jupyter using OpenSearch