Which command installs a Python package scoped at the notebook level in Databricks?

Using %pip install in a notebook cell is the appropriate command for installing a Python package at the notebook level in Databricks. This command allows you to manage dependencies locally within the notebook, as opposed to installing packages at the cluster level. The %pip magic command seamlessly integrates with the Databricks environment and ensures that the installed package is available for the duration of the notebook's execution.

Additionally, this method provides a straightforward way to install or update packages as needed without affecting other notebooks or the shared environment within the cluster. It’s particularly useful for isolating the dependencies specific to a single notebook or for testing new packages before committing to a more global installation. This feature is particularly advantageous in collaborative environments where different notebooks might require varied library versions or packages.

Using commands like %sh pip install would attempt to execute the pip install using a shell context, which might not properly register the installed packages with the notebook environment and thus could lead to unavailability of the package for use in subsequent cells.