How to implement your own plotters and plugins

New plotters and plugins to the psyplot framework are highly welcomed. In this guide, we present how to create new plotters and explain to you how you can include them as a plugin in psyplot.

Creating plotters

Implementing new plotters can be very easy or quite an effort depending on how sophisticated you want to do it. In principle, you only have to implement the Formatoption.update() method and a default value. I.e., one simple formatoption would be

In [1]: from psyplot.plotter import Formatoption, Plotter

In [2]: class MyFormatoption(Formatoption):
   ...:     default = 'my text'
   ...:     def update(self, value):
   ...:         self.ax.text(0.5, 0.5, value, fontsize='xx-large')
   ...: 

together with a plotter

In [3]: class MyPlotter(Plotter):
   ...:     my_fmt = MyFormatoption('my_fmt')
   ...: 

and your done. Now you can make a simple plot

In [4]: from psyplot import open_dataset

In [5]: ds = open_dataset('demo.nc')

In [6]: plotter = MyPlotter(ds.t2m)

../_images/docs_demo_MyPlotter_simple.png

However, if you’re using the psyplot framework, you probably will be a bit more advanced so let’s talk about attributes and methods of the Formatoption class.

If you look into the documentation of the Formatoption class, you find quite a lot of attributes and methods which probably is a bit depressing and confusing. But in principle, we can group them into 4 categories, the interface to the data, to the plotter and to other formatoptions. Plus an additional category for some Formatoption internals you definitely have to care about.

Interface for the plotter

The first interface is the one, that interfaces to the plotter. The most important attributes in this group are the key, priority, plot_fmt, initialize_plot() and most important the update() method.

The key is the unique key for the formatoption inside the plotter. In our example above, we assign the 'my_fmt' key to the MyFormatoption class in MyPlotter. Hence, this key is defined when the plotter class is defined and will be automatically assigned to the formatoption.

The next important attribute is the priority attribute. There are three stages in the update of a plotter:

The stage with data manipulation. If formatoptions manipulate the data that shall be visualized (the data attribute), those formatoptions are updated first. They have the psyplot.plotter.START priority
The stage of the plot. Formatoptions that influence how the data is visualized are updated here (e.g. the colormap or formatoptions that do the plotting). They have the psyplot.plotter.BEFOREPLOTTING priority.
The stage of the plot where additional informations are inserted. Here all the labels are updated, e.g. the title, xlabel, etc.. This is the default priority of the Formatoption.priority attribute, the psyplot.plotter.END priority.

If there is any formatoption updated within the first two groups, the plot of the plotter is updated. This brings us to the third important attribute, the plot_fmt. This boolean tells the plotter, whether the corresponding formatoption is assumed to make a plot at the end of the second stage (the BEFOREPLOTTING stage). If this attribute is True, then the plotter will call the Formatoption.make_plot() method of the formatoption instance.

Finally, the initialize_plot() and update() methods, this is were your contribution really is required. The initialize_plot() method is called when the plot is created for the first time, the update() method when it is updated (the default implementation of the initialize_plot() simply calls the update() method). Implement these methods in your formatoption and thereby make use of the interface to the data and other formatoptions.

Interface to the data

The next set of attributes help you to interface to the data. There are two important parts in this section the interface to the data and the interpretation of the data.

The first part is mainly represented to the Formatoption.data and Formatoption.raw_data attributes. The plotter that contains the formatoption often creates a copy of the data because the data for the visualization might be modified (see for example the psy_reg.plotter.LinRegPlotter). This modified data can be accessed through the Formatoption.data and should be the standard approach to access the data within a formatoption. Nevertheless, the original data can be accessed through the Formatoption.raw_data attribute. However, it only makes sense to access this data for formatoption with START priority.

The result of these two attributes depend on the Formatoption.index_in_list attribute. The data objects in the psyplot framework are either a xarray.DataArray or a list of those in a psyplot.data.InteractiveList. If the index_in_list attribute is not None, and the data object is an InteractiveList, then only the array at the specified position is returned. To completely avoid this issue, you might also use the iter_data or iter_raw_data attributes.

The second part in this section is the interpretation of the data and here, the formatoption can use the Formatoption.decoder attribute. This subclass of the psyplot.data.CFDecoder helps you to identify the x- and y-variables in the data.

Interfacing to other formatoptions

A formatoption is the lowest level in the psyplot framework. It is represented at multiple levels:

at the lowest level through the subclass of the Formatoption class
at the Plotter class level which includes the formatoption class as a descriptor (in our example above it’s MyPlotter.my_fmt)
at the Plotter instance level through
1. a personalized instance of the corresponding Formatoption class (i.e. plotter = MyPlotter(); plotter.my_fmt is not MyPlotter.my_fmt)
2. an item in the plotter (i.e. plotter = MyPlotter(); plotter['my_fmt'])
In the update methods of the Plotter, psyplot.data.InteractiveBase and psyplot.data.ArrayList as a keyword (i.e. plotter = MyPlotter(); plotter.update(my_fmt='new value'))

Hence, there is one big to the entire framework, that is: the functionality of a new formatoption has to be completely defined through exactly one argument, i.e. it must be possible to assign a value to the formatoption in the plotter.

For complex formatoption, this might indeed be quite a challenge for the developer and there are two solutions to it:

The simple solution for the developer: Allow a dictionary as a formatoption, here we also have the psyplot.plotter.DictFormatoption to help you.
Interface to other formatoptions

First solution: Use a `dict`

That said, to implement a formatoption that inserts a custom text and let the user define the size of the text, you either create a formatoption that accepts a text via

class CustomText(DictFormatoption):

    default = {'text': ''}

    text = None

    def validate(self, value):
        if not isinstance(value, dict):
            return {'text': value}
        return value

    def initialize_plot(self, value):
        self.text = self.ax.text(0.2, 0.2, value['text'],
                                 fontsize=value.get('size', 'large'))

    def update(self, value):
        self.text.set_text(value['text'])
        self.text.set_fontsize(value.get('size', 'large'))


class MyPlotter(Plotter):

    my_fmt = CustomText('my_fmt')

and then you could create and update a plotter via

p = MyPlotter(xarray.DataArray([]))
p.update(my_fmt='my text')  # updates the text
p.update(my_fmt={'size': 14})  # updates the size
p.update(my_fmt={'size': 14, 'text': 'Something'})  # updates text and size

This solution has the several advantages:

The user does not get confused through too many formatoptions
It is easy to allow more keywords for this formatoption

Indeed, the psy_simple.plotter.Legend formatoption uses this framework since the matplotlib.pyplot.legend() function accepts that many keywords that it would be not informative to create a formatoption for each of them.

Otherwise you could of course avoid the DictFormatoption and just force the user to always provide a new dictionary.

Second solution: Interact with other formatoptions

Another possibility is to implement a second formatoption for the size of the text. And here, the psyplot framework helps you with several attributes of the Formatoption class:

the children attribute: Forces the listed formatoptions in this list to be updated before the current formatoption is updated
the dependencies attributes: Same as children but also forces an update if one of the named formatoptions are updated
the parents attribute: Skip the update if one of the parents is updated
the connections attribute: just provides connections to the listed formatoptions

Each of those attributes accept a list of strings that represent the formatoption keys of other formatoptions. Those formatoptions are then accessible within the formatoption via the usual getattr(). I.e. if you list a formatoption in the children attribute, you can access it inside the formatoption (self) via self.other_formatoption.

In our example of the CustomText, this could be implemented via

class CustomTextSize(Formatoption):
    """
    Set the fontsize of the custom text

    Possible types
    --------------
    int
        The fontsize of the text
    """

    default = int

    def validate(self, value):
        return int(value)

    # this text has not to be updated if the custom text is updated
    children = ['text']

    def update(self, value):
        self.text.text.set_fontsize(value)


class CustomText(Formatoption):
    """
    Place a text

    Possible types
    --------------
    str
        The text to display""""

    def initialize_plot(self, value):
        self.text = self.ax.text(0.2, 0.2, value['text'])

    def update(self, value):
        self.text.set_text(value)


class MyPlotter(Plotter):

    my_fmt = CustomText('my_fmt')
    my_fmtsize = CustomTextSize('my_fmtsize', text='my_fmt')

the update in that sense would be like

and then you could create and update a plotter via

p = MyPlotter(xarray.DataArray([]))
p.update(my_fmt='my text')  # updates the text
p.update(my_fmtsize=14)  # updates the size
p.update(my_fmt='Something', my_fmtsize=14)  # updates text and size

The advantages of this methodology are basically:

The user straight away sees two formatoptions that can be interpreted easiliy
The formatoption that controls the font size could easily be subclassed and replaced in a subclass of MyPlotter. In the first framework using the DictFormatoption, this would mean that the entire process has to be rewritten.

As you see in the above definition my_fmtsize = CustomTextSize('my_fmtsize', text='my_fmt'), we provide an additional text keyword. That is because we explicitly named the text key in the children attribute of the CustomTextSize formatoption. In that way we can tell the my_fmtsize formatoption how to find the necessary formatoption. That works for all keys listed in the children, dependencies, parents and connections attributes.

Creating new plugins

Now that you have created your plotter, you may want to include it in the plot methods of the Project class such that you can do something like

import psyplot.project as psy
psy.plot.my_plotter('netcdf-file.nc', name='varname')

There are three possibilities how you can do this:

The easy and fast solution for one session: register the plotter using the psyplot.project.register_plotter() function
The easy and steady solution: Save the calls you used in step 1 in the 'project.plotter.user' key of the rcParams
The steady and shareable solution: Create a new plugin

The third solution has been used for the psy-maps and psy-simple plugins and will be described in the following section.

Creating a package with the psyplot-plugin-template

The psyplot-plugin-template provides a template to create a python package that integrates with the psyplot environment. We recommend using this template as it already contains a setup for automated formatters and linters, and a setup for continuous integration.

Note

When creating a real package, we strongly recommend to use cruft instead of cookiecutter!

For our demonstration, let’s create a plugin named my-plugin. We will save this name in to a YAML-file and use this to create our new plugin.

In [7]: !echo "default_context: {project_slug: my-plugin}" > "config.yaml"

In [8]: cookiecutter --no-input --config-file config.yaml https://codebase.helmholtz.cloud/psyplot/psyplot-plugin-template.git

In [9]: import glob

In [10]: glob.glob('my-plugin/**', recursive=True)
Out[10]: 
['my-plugin/',
 'my-plugin/my_plugin',
 'my-plugin/my_plugin/py.typed',
 'my-plugin/my_plugin/_version.py',
 'my-plugin/my_plugin/plugin.py',
 'my-plugin/my_plugin/__init__.py',
 'my-plugin/my_plugin/plotters.py',
 'my-plugin/tests',
 'my-plugin/tests/test_imports.py',
 'my-plugin/setup.py',
 'my-plugin/pyproject.toml',
 'my-plugin/conftest.py',
 'my-plugin/docs',
 'my-plugin/docs/installation.md',
 'my-plugin/docs/index.md',
 'my-plugin/docs/requirements.txt',
 'my-plugin/docs/_templates',
 'my-plugin/docs/_templates/footer.html',
 'my-plugin/docs/make.bat',
 'my-plugin/docs/contributing.md',
 'my-plugin/docs/_static',
 'my-plugin/docs/_static/license_logo.png.license',
 'my-plugin/docs/_static/license_logo.png',
 'my-plugin/docs/api.md',
 'my-plugin/docs/conf.py',
 'my-plugin/docs/Makefile',
 'my-plugin/README.md',
 'my-plugin/CITATION.cff',
 'my-plugin/CHANGELOG.md',
 'my-plugin/MANIFEST.in',
 'my-plugin/tox.ini',
 'my-plugin/Makefile']

The following files are created in a directory named 'my-plugin':

pyproject.toml: The python package configuration
'my_python_package/plugin.py': The file that sets up the configuration of our plugin. This file should define the rcParams for the plugin (see also rcParams handling in plugins)
'my_python_package/plotters.py': The file in which we define the plotters. This file should contain the plotters and formatoptions from our previous section.

If you want to see more, look into the comments in the created files.

`rcParams` handling in plugins

Every formatoption does have default values. In our example above, we simply set it via the default attribute. This is a hard-coded, but easy, stable and quick solution.

However, your formatoption could also be used in different plotters, each requiring a different default value. Or you want to give the user the possibility to set his own default value. For this, we implemented the

psyplot.plotter.Plotter._rcparams_string

List of base strings in the psyplot.rcParams dictionary

attribute. Here you can specify a string for this plotter which is used to get the default value of the formatoptions in this plotter from the rcParams. The expected default_key for one formatoption would then be the_chosen_string + fmt_key.

The following example illustrates this:

In [11]: from psyplot.config.rcsetup import rcParams
   ....: from psyplot.plotter import Plotter, Formatoption
   ....: 

First we define our defaultParams, a mapping from default key to the default value, a validation function, and a description (see the psyplot.config.rcsetup.defaultParams dictionary).

In [12]: defaultParams = {
   ....:     'plotter.example_plotter.fmt1': [
   ....:         1, lambda val: int(val), 'Example formatoption']
   ....:     }
   ....: 

Then we update the defaultParams of the psyplot.rcParams and set the value

In [13]: rcParams.defaultParams.update(defaultParams)

In [14]: rcParams.update_from_defaultParams(defaultParams)
   ....: print(rcParams['plotter.example_plotter.fmt1'])
   ....: 
1

Now we define a formatoption for our new plotter class and implement it in a new plotter object.

In [15]: class ExampleFmt(Formatoption):
   ....:     def update(self, value):
   ....:         pass
   ....: 

In [16]: class ExamplePlotter(Plotter):
   ....:     # we use our base string, 'plotter.example_plotter.'
   ....:     _rcparams_string = ['plotter.example_plotter.']
   ....:     # and register a formatoption for the plotter
   ....:     fmt1 = ExampleFmt('fmt1')
   ....: 

If we now create a new instance of this ExamplePlotter, the fmt1 formatoption will have a value of 1, as we defined it in the above defaultParams:

In [17]: plotter = ExamplePlotter()

In [18]: print(plotter['fmt1'])
1

# and the default_key is our string in the defaultParams, a combination
# of the _rcparams_string and the formatoption key
In [19]: print(plotter.fmt1.default_key)
plotter.example_plotter.fmt1

In [20]: print(plotter.fmt1.default)
1

Changing the value in the rcParams, also changes the default value for the plotter

In [21]: rcParams['plotter.example_plotter.fmt1'] = 2

In [22]: print(plotter.fmt1.default)
2

Also, if we subclass this plotter, the default_key will not change

In [23]: class SecondPlotter(ExamplePlotter):
   ....:     # we set a new _rcparams_string
   ....:     _rcparams_string = ['plotter.another_plotter.']
   ....: 

In [24]: plotter = SecondPlotter()

# still the same key, although we defined a different _rcparams_string
In [25]: print(plotter.fmt1.default_key)
plotter.example_plotter.fmt1

If you’re developing a new plugin you would then have to define the rcParams and defaultParams in the plugin.py script (see Creating new plugins) and they will then be automatically implemented in psyplot.rcParams.