From c625da52495a82762e6c32149a90ec53dc528b94 Mon Sep 17 00:00:00 2001 From: Marc Garcia Date: Wed, 14 Feb 2018 22:05:43 +0000 Subject: [PATCH 1/8] DOC: Adding guide for the pandas documentation sprint --- doc/source/contributing_docstring.rst | 634 ++++++++++++++++++++++++++ 1 file changed, 634 insertions(+) create mode 100644 doc/source/contributing_docstring.rst diff --git a/doc/source/contributing_docstring.rst b/doc/source/contributing_docstring.rst new file mode 100644 index 0000000000000..aeec7a21a514a --- /dev/null +++ b/doc/source/contributing_docstring.rst @@ -0,0 +1,634 @@ +====================== +pandas docstring guide +====================== + +About docstrings and standards +------------------------------ + +A Python docstring is a string used to document a Python function or method, +so programmers can understand what it does without having to read the details +of the implementation. + +Also, it is a commonn practice to generate online (html) documentation +automatically from docstrings. `Sphinx `_ serves +this purpose. + +Next example gives an idea on how a docstring looks like: + +.. code-block:: python + + def add(num1, num2): + """Add up two integer numbers. + + This function simply wraps the `+` operator, and does not + do anything interesting, except for illustrating what is + the docstring of a very simple function. + + Parameters + ---------- + num1 : int + First number to add + num2 : int + Second number to add + + Returns + ------- + int + The sum of `num1` and `num2` + + Examples + -------- + >>> add(2, 2) + 4 + >>> add(25, 0) + 25 + >>> add(10, -10) + 0 + """ + return num1 + num2 + +Some standards exist about docstrings, so they are easier to read, and they can +be exported to other formats such as html or pdf. + +The first conventions every Python docstring should follow are defined in +`PEP-257 `_. + +As PEP-257 is quite open, and some other standards exist on top of it. In the +case of pandas, the numpy docstring convention is followed. There are two main +documents that explain this convention: + +- `Guide to NumPy/SciPy documentation `_ +- `numpydoc docstring guide `_ + +numpydoc is a Sphinx extension to support the numpy docstring convention. + +The standard uses reStructuredText (reST). reStructuredText is a markup +language that allows encoding styles in plain text files. Documentation +about reStructuredText can be found in: + +- `Sphinx reStructuredText primer `_ +- `Quick reStructuredText reference `_ +- `Full reStructuredText specification `_ + +The rest of this document will summarize all the above guides, and will +provide additional convention specific to the pandas project. + +Writing a docstring +------------------- + +General rules +~~~~~~~~~~~~~ + +Docstrings must be defined with three double-quotes. No blank lines should be +left before or after the docstring. The text starts immediately after the +opening quotes (not in the next line). The closing quotes have their own line +(meaning that they are not at the end of the last sentence). + +**Good:** + +.. code-block:: python + + def func(): + """Some function. + + With a good docstring. + """ + foo = 1 + bar = 2 + return foo + bar + +**Bad:** + +.. code-block:: python + + def func(): + + """ + Some function. + + With several mistakes in the docstring. + + It has a blank like after the signature `def func():`. + + The text 'Some function' should go in the same line as the + opening quotes of the docstring, not in the next line. + + There is a blank line between the docstring and the first line + of code `foo = 1`. + + The closing quotes should be in the next line, not in this one.""" + + foo = 1 + bar = 2 + return foo + bar + +Section 1: Short summary +~~~~~~~~~~~~~~~~~~~~~~~~ + +The short summary is a single sentence that express what the function does in a +concise way. + +The short summary must start with a verb infinitive, end with a dot, and fit in +a single line. It needs to express what the function does without providing +details. + +**Good:** + +.. code-block:: python + + def astype(dtype): + """Cast Series type. + + This section will provide further details. + """ + pass + +**Bad:** + +.. code-block:: python + + def astype(dtype): + """Casts Series type. + + Verb in third-person of the present simple, should be infinitive. + """ + pass + + def astype(dtype): + """Method to cast Series type. + + Does not start with verb. + """ + pass + + def astype(dtype): + """Cast Series type + + Missing dot at the end. + """ + pass + + def astype(dtype): + """Cast Series type from its current type to the new type defined in + the parameter dtype. + + Summary is too verbose and doesn't fit in a single line. + """ + pass + +Section 2: Extended summary +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The extended summary provides details on what the function does. It should not +go into the details of the parameters, or discuss implementation notes, which +go in other sections. + +A blank line is left between the short summary and the extended summary. And +every paragraph in the extended summary is finished by a dot. + +.. code-block:: python + + def unstack(): + """Pivot a row index to columns. + + When using a multi-index, a level can be pivoted so each value in + the index becomes a column. This is especially useful when a subindex + is repeated for the main index, and data is easier to visualize as a + pivot table. + + The index level will be automatically removed from the index when added + as columns. + """ + pass + +Section 3: Parameters +~~~~~~~~~~~~~~~~~~~~~ + +The details of the parameters will be added in this section. This section has +the title "Parameters", followed by a line with a hyphen under each letter of +the word "Parameters". A blank line is left before the section title, but not +after, and not between the line with the word "Parameters" and the one with +the hyphens. + +After the title, each parameter in the signature must be documented, including +`*args` and `**kwargs`, but not `self`. + +The parameters are defined by their name, followed by a space, a colon, another +space, and the type (or types). Note that the space between the name and the +colon is important. Types are not defined for `*args` and `**kwargs`, but must +be defined for all other parameters. After the parameter definition, it is +required to have a line with the parameter description, which is indented, and +can have multiple lines. The description must start with a capital letter, and +finish with a dot. + +**Good:** + +.. code-block:: python + + class Series: + def plot(self, kind, **kwargs): + """Generate a plot. + + Render the data in the Series as a matplotlib plot of the + specified kind. + + Parameters + ---------- + kind : str + Kind of matplotlib plot. + **kwargs + These parameters will be passed to the matplotlib plotting + function. + """ + pass + +**Bad:** + +.. code-block:: python + + class Series: + def plot(self, kind, **kwargs): + """Generate a plot. + + Render the data in the Series as a matplotlib plot of the + specified kind. + + Note the blank line between the parameters title and the first + parameter. Also, not that after the name of the parameter `kind` + and before the colo, a space is missing. + + Also, note that the parameter descriptions do not start with a + capital letter, and do not finish with a dot. + + Finally, the `**kwargs` parameter is missing. + + Parameters + ---------- + + kind: str + kind of matplotlib plot + """ + pass + +Parameter types +^^^^^^^^^^^^^^^ + +When specifying the parameter types, Python built-in data types can be used +directly: + +- int +- float +- str + +For complex types, define the subtypes: + +- list of [int] +- dict of {str : int} +- tuple of (str, int, int) +- set of {str} + +In case there are just a set of values allowed, list them in curly brackets +and separated by commas (followed by a space): + +- {0, 10, 25} +- {'simple', 'advanced'} + +If the type is defined in a Python module, the module must be specified: + +- datetime.date +- datetime.datetime +- decimal.Decimal + +If the type is in a package, the module must be also specified: + +- numpy.ndarray +- scipy.sparse.coo_matrix + +If the type is a pandas type, also specify pandas: + +- pandas.Series +- pandas.DataFrame + +If more than one type is accepted, separate them by commas, except the +last two types, that need to be separated by the word 'or': + +- int or float +- float, decimal.Decimal or None +- str or list of str + +If None is one of the accepted values, it always needs to be the last in +the list. + +Section 4: Returns or Yields +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If the method returns a value, it will be documented in this section. Also +if the method yields its output. + +The title of the section will be defined in the same way as the "Parameters". +With the names "Returns" or "Yields" followed by a line with as many hyphens +as the letters in the preceding word. + +The documentation of the return is also similar to the parameters. But in this +case, no name will be provided, unless the method returns or yields more than +one value (a tuple of values). + +The types for "Returns" and "Yields" are the same as the ones for the +"Parameters". Also, the description must finish with a dot. + +For example, with a single value: + +.. code-block:: python + + def sample(): + """Generate and return a random number. + + The value is sampled from a continuos uniform distribution between + 0 and 1. + + Returns + ------- + float + Random number generated. + """ + return random.random() + +With more than one value: + +.. code-block:: python + + def random_letters(): + """Generate and return a sequence of random letters. + + The length of the returned string is also random, and is also + returned. + + Returns + ------- + length : int + Length of the returned string. + letters : str + String of random letters. + """ + length = random.randint(1, 10) + letters = ''.join(random.choice(string.ascii_lowercase) + for i in range(length)) + return length, letters + +If the method yields its value: + +.. code-block:: python + + def sample_values(): + """Generate an infinite sequence of random numbers. + + The values are sampled from a continuos uniform distribution between + 0 and 1. + + Yields + ------ + float + Random number generated. + """ + while True: + yield random.random() + + +Section 5: See also +~~~~~~~~~~~~~~~~~~~ + +This is an optional section, used to let users know about pandas functionality +related to the one being documented. + +An obvious example would be the `head()` and `tail()` methods. As `tail()` does +the equivalent as `head()` but at the end of the `Series` or `DataFrame` +instead of at the beginning, it is good to let the users know about it. + +To give an intuition on what can be considered related, here there are some +examples: + +* `loc` and `iloc`, as they do the same, but in one case providing indices and + in the other positions +* `max` and `min`, as they do the opposite +* `iterrows`, `itertuples` and `iteritems`, as it is easy that a user looking + for the method to iterate over columns ends up in the method to iterate + over rows, and vice-versa +* `fillna` and `dropna`, as both methods are used to handle missing values +* `read_csv` and `to_csv`, as they are complementary +* `merge` and `join`, as one is a generalization of the other +* `astype` and `pandas.to_datetime`, as users may be reading the documentation + of `astype` to know how to cast as a date, and the way to do it is with + `pandas.to_datetime` + +When deciding what is related, you should mainly use your common sense and +think about what can be useful for the users reading the documentation, +especially the less experienced ones. + +This section, as the previous, also has a header, "See Also" (note the capital +S and A). Also followed by the line with hyphens, and preceded by a blank line. + +After the header, we will add a line for each related method or function, +followed by a space, a colon, another space, and a short description that +illustrated what this method or function does, and why is it relevant in +this context. The description must also finish with a dot. + +Note that in "Returns" and "Yields", the description is located in the +following line than the type. But in this section it is located in the same +line, with a colon in between. + +For example: + +.. code-block:: python + + class Series: + def head(self): + """Return the first 5 elements of the Series. + + This function is mainly useful to preview the values of the + Series without displaying the whole of it. + + Return + ------ + pandas.Series + Subset of the original series with the 5 first values. + + See Also + -------- + tail : Return the last 5 elements of the Series. + """ + return self.iloc[:5] + +Section 6: Notes +~~~~~~~~~~~~~~~~ + +This is an optional section used for notes about the implementation of the +algorithm. Or to document technical aspects of the function behavior. + +Feel free to skip it, unless you are familiar with the implementation of the +algorithm, or you discover some counter-intuitive behavior while writing the +examples for the function. + +This section follows the same format as the extended summary section. + +Section 7: Examples +~~~~~~~~~~~~~~~~~~~ + +This is one of the most important sections of a docstring, even if it is +placed in the last position. As often, people understand concepts better +with examples, than with accurate explanations. + +Examples in docstrings are also unit tests, and besides illustrating the +usage of the function or method, they need to be valid Python code, that in a +deterministic way returns the presented output. + +They are presented as a session in the Python terminal. `>>>` is used to +present code. `...` is used for code continuing from the previous line. +Output is presented immediately after the last line of code generating the +output (no blank lines in between). Comments describing the examples can +be added with blank lines before and after them. + +The way to present examples is as follows: + +1. Import required libraries + +2. Create the data required for the example + +3. Show a very basic example that gives an idea of the most common use case + +4. Add commented examples that illustrate how the parameters can be used for + extended functionality + +A simple example could be: + +.. code-block:: python + + class Series: + def head(self, n=5): + """Return the first elements of the Series. + + This function is mainly useful to preview the values of the + Series without displaying the whole of it. + + Parameters + ---------- + n : int + Number of values to return. + + Return + ------ + pandas.Series + Subset of the original series with the n first values. + + See Also + -------- + tail : Return the last n elements of the Series. + + Examples + -------- + >>> import pandas + >>> s = pandas.Series(['Ant', 'Bear', 'Cow', 'Dog', 'Falcon', + ... 'Lion', 'Monkey', 'Rabbit', 'Zebra']) + >>> s.head() + 0 Ant + 1 Bear + 2 Cow + 3 Dog + 4 Falcon + dtype: object + + With the `n` parameter, we can change the number of returned rows: + + >>> s.head(n=3) + 0 Ant + 1 Bear + 2 Cow + dtype: object + """ + return self.iloc[:n] + +Conventions for the examples +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. note:: + numpydoc recommends avoiding "obvious" imports and importing them with + aliases, so for example `import numpy as np`. While this is now an standard + in the data ecosystem of Python, it doesn't seem a good practise, for the + next reasons: + + * The code is not executable anymore (as doctests for example) + + * New users not familiar with the convention can't simply copy and run it + + * Users may use aliases (even if it is a bad Python practise except + in rare cases), but if maintainers want to use `pd` instead of `pandas`, + why do not name the module `pd` directly? + + * As this is becoming more standard, there are an increasing number of + aliases in scientific Python code, including `np`, `pd`, `plt`, `sp`, + `pm`... which makes reading code harder + +All examples must start with the required imports, one per line (as +recommended in `PEP-8 `_) +and avoiding aliases. Avoid excessive imports, but if needed, imports from +the standard library go first, followed by third-party libraries (like +numpy) and importing pandas in the last place. + +When illustrating examples with a single `Series` use the name `s`, and if +illustrating with a single `DataFrame` use the name `df`. If a set of +homogeneous `Series` or `DataFrame` is used, name them `s1`, `s2`, `s3`... +or `df1`, `df2`, `df3`... If the data is not homogeneous, and more than +one structure is needed, name them with something meaningful, for example +`df_main` and `df_to_join`. + +Data used in the example should be as compact as possible. The number of rows +is recommended to be 4, unless the example requires a larger number. As for +example in the head method, where it requires to be higher than 5, to show +the example with the default values. + +Avoid using data without interpretation, like a matrix of random numbers +with columns A, B, C, D... And instead use a meaningful example, which makes +it easier to understand the concept. Unless required by the example, use +names of animals, to keep examples consistent. And numerical properties of +them. + +When calling the method, keywords arguments `head(n=3)` are preferred to +positional arguments `head(3)`. + +**Good:** + +.. code-block:: python + + def method(): + """A sample DataFrame method. + + Examples + -------- + >>> import numpy + >>> import pandas + >>> df = pandas.DataFrame([389., 24., 80.5, numpy.nan] + ... columns=('max_speed'), + ... index=['falcon', 'parrot', 'lion', 'monkey']) + """ + pass + +**Bad:** + +.. code-block:: python + + def method(): + """A sample DataFrame method. + + Examples + -------- + >>> import numpy + >>> import pandas + >>> df = pandas.DataFrame(numpy.random.randn(3, 3), + ... columns=('a', 'b', 'c')) + """ + pass + +Once you finished the docstring +------------------------------- + +When you finished the changes to the docstring, go to the +:ref:`instructions to submit your changes ` to continue. From 97dd08d900184b78cecd48b087592853a2d79cf6 Mon Sep 17 00:00:00 2001 From: Marc Garcia Date: Sun, 25 Feb 2018 23:00:53 +0000 Subject: [PATCH 2/8] Made changes addressing comments in the different reviews --- doc/source/contributing_docstring.rst | 98 ++++++++++++++++----------- 1 file changed, 58 insertions(+), 40 deletions(-) diff --git a/doc/source/contributing_docstring.rst b/doc/source/contributing_docstring.rst index aeec7a21a514a..0605e380da29b 100644 --- a/doc/source/contributing_docstring.rst +++ b/doc/source/contributing_docstring.rst @@ -221,12 +221,19 @@ required to have a line with the parameter description, which is indented, and can have multiple lines. The description must start with a capital letter, and finish with a dot. +Keyword arguments with a default value, the default will be listed in brackets +at the end of the description (before the dot). The exact form of the +description in this case would be "Description of the arg (default is X).". In +some cases it may be useful to explain what the default argument means, which +can be added after a comma "Description of the arg (default is -1, which means +all cpus).". + **Good:** .. code-block:: python class Series: - def plot(self, kind, **kwargs): + def plot(self, kind, color='blue', **kwargs): """Generate a plot. Render the data in the Series as a matplotlib plot of the @@ -236,6 +243,8 @@ finish with a dot. ---------- kind : str Kind of matplotlib plot. + color : str + Color name or rgb code (default is 'blue'). **kwargs These parameters will be passed to the matplotlib plotting function. @@ -288,7 +297,8 @@ For complex types, define the subtypes: - set of {str} In case there are just a set of values allowed, list them in curly brackets -and separated by commas (followed by a space): +and separated by commas (followed by a space). If one of them is the default +value of a keyword argument, it should be listed first.: - {0, 10, 25} - {'simple', 'advanced'} @@ -304,10 +314,21 @@ If the type is in a package, the module must be also specified: - numpy.ndarray - scipy.sparse.coo_matrix -If the type is a pandas type, also specify pandas: +If the type is a pandas type, also specify pandas except for Series and +DataFrame: + +- Series +- DataFrame +- pandas.Index +- pandas.Categorical +- pandas.SparseArray + +If the exact type is not relevant, but must be compatible with a numpy +array, array-like can be specified. If Any type that can be iterated is +accepted, iterable can be used: -- pandas.Series -- pandas.DataFrame +- array-like +- iterable If more than one type is accepted, separate them by commas, except the last two types, that need to be separated by the word 'or': @@ -398,7 +419,8 @@ Section 5: See also ~~~~~~~~~~~~~~~~~~~ This is an optional section, used to let users know about pandas functionality -related to the one being documented. +related to the one being documented. While optional, this section should exist +in most cases, unless no related methods or functions can be found at all. An obvious example would be the `head()` and `tail()` methods. As `tail()` does the equivalent as `head()` but at the end of the `Series` or `DataFrame` @@ -419,22 +441,30 @@ examples: * `astype` and `pandas.to_datetime`, as users may be reading the documentation of `astype` to know how to cast as a date, and the way to do it is with `pandas.to_datetime` +* `where` is related to `numpy.where`, as its functionality is based on it When deciding what is related, you should mainly use your common sense and think about what can be useful for the users reading the documentation, especially the less experienced ones. +When relating to other methods (mainly `numpy`), use the name of the module +first (not an alias like `np`). If the function is in a module which is not +the main one, like `scipy.sparse`, list the full module (e.g. +`scipy.sparse.coo_matrix`). + This section, as the previous, also has a header, "See Also" (note the capital S and A). Also followed by the line with hyphens, and preceded by a blank line. After the header, we will add a line for each related method or function, followed by a space, a colon, another space, and a short description that -illustrated what this method or function does, and why is it relevant in -this context. The description must also finish with a dot. +illustrated what this method or function does, why is it relevant in this +context, and what are the key differences between the documented function and +the one referencing. The description must also finish with a dot. Note that in "Returns" and "Yields", the description is located in the following line than the type. But in this section it is located in the same -line, with a colon in between. +line, with a colon in between. If the description does not fit in the same +line, it can continue in the next ones, but it has to be indenteted in them. For example: @@ -489,14 +519,14 @@ be added with blank lines before and after them. The way to present examples is as follows: -1. Import required libraries +1. Import required libraries (except `numpy` and `pandas`) 2. Create the data required for the example 3. Show a very basic example that gives an idea of the most common use case -4. Add commented examples that illustrate how the parameters can be used for - extended functionality +4. Add examples with explanations that illustrate how the parameters can be + used for extended functionality A simple example could be: @@ -525,9 +555,8 @@ A simple example could be: Examples -------- - >>> import pandas - >>> s = pandas.Series(['Ant', 'Bear', 'Cow', 'Dog', 'Falcon', - ... 'Lion', 'Monkey', 'Rabbit', 'Zebra']) + >>> s = pd.Series(['Ant', 'Bear', 'Cow', 'Dog', 'Falcon', + ... 'Lion', 'Monkey', 'Rabbit', 'Zebra']) >>> s.head() 0 Ant 1 Bear @@ -549,29 +578,20 @@ A simple example could be: Conventions for the examples ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -.. note:: - numpydoc recommends avoiding "obvious" imports and importing them with - aliases, so for example `import numpy as np`. While this is now an standard - in the data ecosystem of Python, it doesn't seem a good practise, for the - next reasons: - - * The code is not executable anymore (as doctests for example) +Code in examples is assumed to always start with these two lines which are not +shown: - * New users not familiar with the convention can't simply copy and run it +.. code-block:: python - * Users may use aliases (even if it is a bad Python practise except - in rare cases), but if maintainers want to use `pd` instead of `pandas`, - why do not name the module `pd` directly? + import numpy as np + import pandas as pd - * As this is becoming more standard, there are an increasing number of - aliases in scientific Python code, including `np`, `pd`, `plt`, `sp`, - `pm`... which makes reading code harder -All examples must start with the required imports, one per line (as +Any other module used in the examples must be explicitly imported, one per line (as recommended in `PEP-8 `_) and avoiding aliases. Avoid excessive imports, but if needed, imports from the standard library go first, followed by third-party libraries (like -numpy) and importing pandas in the last place. +matplotlib). When illustrating examples with a single `Series` use the name `s`, and if illustrating with a single `DataFrame` use the name `df`. If a set of @@ -603,11 +623,9 @@ positional arguments `head(3)`. Examples -------- - >>> import numpy - >>> import pandas - >>> df = pandas.DataFrame([389., 24., 80.5, numpy.nan] - ... columns=('max_speed'), - ... index=['falcon', 'parrot', 'lion', 'monkey']) + >>> df = pd.DataFrame([389., 24., 80.5, numpy.nan] + ... columns=('max_speed'), + ... index=['falcon', 'parrot', 'lion', 'monkey']) """ pass @@ -620,10 +638,10 @@ positional arguments `head(3)`. Examples -------- - >>> import numpy - >>> import pandas - >>> df = pandas.DataFrame(numpy.random.randn(3, 3), - ... columns=('a', 'b', 'c')) + >>> import numpy as np + >>> import pandas as pd + >>> df = pd.DataFrame(numpy.random.randn(3, 3), + ... columns=('a', 'b', 'c')) """ pass From 9b275ac54c319a1590585385ec7fcae1937c9598 Mon Sep 17 00:00:00 2001 From: Marc Garcia Date: Sun, 25 Feb 2018 23:09:04 +0000 Subject: [PATCH 3/8] Adding comment on why the original numpy docstring convention is referenced --- doc/source/contributing_docstring.rst | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/doc/source/contributing_docstring.rst b/doc/source/contributing_docstring.rst index 0605e380da29b..0796a35e80c47 100644 --- a/doc/source/contributing_docstring.rst +++ b/doc/source/contributing_docstring.rst @@ -54,11 +54,12 @@ The first conventions every Python docstring should follow are defined in `PEP-257 `_. As PEP-257 is quite open, and some other standards exist on top of it. In the -case of pandas, the numpy docstring convention is followed. There are two main -documents that explain this convention: +case of pandas, the numpy docstring convention is followed. The conventions is +explained in this document: -- `Guide to NumPy/SciPy documentation `_ - `numpydoc docstring guide `_ + (which is based in the original `Guide to NumPy/SciPy documentation + `_) numpydoc is a Sphinx extension to support the numpy docstring convention. From 8e558b08c76878748a4f57f8b6b8c9d21bf0701e Mon Sep 17 00:00:00 2001 From: Marc Garcia Date: Sun, 25 Feb 2018 23:12:00 +0000 Subject: [PATCH 4/8] removing note saying that examples are unit tests --- doc/source/contributing_docstring.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/doc/source/contributing_docstring.rst b/doc/source/contributing_docstring.rst index 0796a35e80c47..7d9ab91e34259 100644 --- a/doc/source/contributing_docstring.rst +++ b/doc/source/contributing_docstring.rst @@ -508,9 +508,9 @@ This is one of the most important sections of a docstring, even if it is placed in the last position. As often, people understand concepts better with examples, than with accurate explanations. -Examples in docstrings are also unit tests, and besides illustrating the -usage of the function or method, they need to be valid Python code, that in a -deterministic way returns the presented output. +Examples in docstrings, besides illustrating the usage of the function or +method, they must be valid Python code, that in a deterministic way returns +the presented output, and that can be copied and run by users. They are presented as a session in the Python terminal. `>>>` is used to present code. `...` is used for code continuing from the previous line. From 0598a79d19e0bc732ac972ea63a1ada6387b6be9 Mon Sep 17 00:00:00 2001 From: Marc Garcia Date: Wed, 28 Feb 2018 22:05:48 +0000 Subject: [PATCH 5/8] Addressing comments from reviews, and linking the docstring guide from the contributing page --- doc/source/contributing.rst | 9 ++---- doc/source/contributing_docstring.rst | 44 ++++++++++++++++++++------- 2 files changed, 36 insertions(+), 17 deletions(-) diff --git a/doc/source/contributing.rst b/doc/source/contributing.rst index 258ab874cafcf..d0c5032079288 100644 --- a/doc/source/contributing.rst +++ b/doc/source/contributing.rst @@ -292,12 +292,9 @@ Some other important things to know about the docs: overviews per topic together with some other information (what's new, installation, etc). -- The docstrings follow the **Numpy Docstring Standard**, which is used widely - in the Scientific Python community. This standard specifies the format of - the different sections of the docstring. See `this document - `_ - for a detailed explanation, or look at some of the existing functions to - extend it in a similar manner. +- The docstrings follow a pandas convention, based on the **Numpy Docstring + Standard**. Follow the :ref:`pandas docstring guide ` for detailed + instructions on how to write a correct docstring. - The tutorials make heavy use of the `ipython directive `_ sphinx extension. diff --git a/doc/source/contributing_docstring.rst b/doc/source/contributing_docstring.rst index 7d9ab91e34259..f5701b6df281f 100644 --- a/doc/source/contributing_docstring.rst +++ b/doc/source/contributing_docstring.rst @@ -1,3 +1,5 @@ +.. _docstring: + ====================== pandas docstring guide ====================== @@ -36,6 +38,10 @@ Next example gives an idea on how a docstring looks like: int The sum of `num1` and `num2` + See Also + -------- + subtract : Subtract one integer from another + Examples -------- >>> add(2, 2) @@ -74,9 +80,13 @@ about reStructuredText can be found in: The rest of this document will summarize all the above guides, and will provide additional convention specific to the pandas project. +.. _docstring.tutorial: + Writing a docstring ------------------- +.. _docstring.general: + General rules ~~~~~~~~~~~~~ @@ -123,6 +133,8 @@ opening quotes (not in the next line). The closing quotes have their own line bar = 2 return foo + bar +.. _docstring.short_summary: + Section 1: Short summary ~~~~~~~~~~~~~~~~~~~~~~~~ @@ -177,6 +189,8 @@ details. """ pass +.. _docstring.extended_summary: + Section 2: Extended summary ~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -202,6 +216,8 @@ every paragraph in the extended summary is finished by a dot. """ pass +.. _docstring.parameters: + Section 3: Parameters ~~~~~~~~~~~~~~~~~~~~~ @@ -280,6 +296,8 @@ all cpus).". """ pass +.. _docstring.parameter_types: + Parameter types ^^^^^^^^^^^^^^^ @@ -289,6 +307,7 @@ directly: - int - float - str +- bool For complex types, define the subtypes: @@ -341,6 +360,8 @@ last two types, that need to be separated by the word 'or': If None is one of the accepted values, it always needs to be the last in the list. +.. _docstring.returns: + Section 4: Returns or Yields ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -415,8 +436,9 @@ If the method yields its value: while True: yield random.random() +.. _docstring.see_also: -Section 5: See also +Section 5: See Also ~~~~~~~~~~~~~~~~~~~ This is an optional section, used to let users know about pandas functionality @@ -448,7 +470,7 @@ When deciding what is related, you should mainly use your common sense and think about what can be useful for the users reading the documentation, especially the less experienced ones. -When relating to other methods (mainly `numpy`), use the name of the module +When relating to other libraries (mainly `numpy`), use the name of the module first (not an alias like `np`). If the function is in a module which is not the main one, like `scipy.sparse`, list the full module (e.g. `scipy.sparse.coo_matrix`). @@ -478,9 +500,9 @@ For example: This function is mainly useful to preview the values of the Series without displaying the whole of it. - Return - ------ - pandas.Series + Returns + ------- + Series Subset of the original series with the 5 first values. See Also @@ -489,6 +511,8 @@ For example: """ return self.iloc[:5] +.. _docstring.notes: + Section 6: Notes ~~~~~~~~~~~~~~~~ @@ -501,6 +525,8 @@ examples for the function. This section follows the same format as the extended summary section. +.. _docstring.examples: + Section 7: Examples ~~~~~~~~~~~~~~~~~~~ @@ -576,6 +602,8 @@ A simple example could be: """ return self.iloc[:n] +.. _docstring.example_conventions: + Conventions for the examples ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -645,9 +673,3 @@ positional arguments `head(3)`. ... columns=('a', 'b', 'c')) """ pass - -Once you finished the docstring -------------------------------- - -When you finished the changes to the docstring, go to the -:ref:`instructions to submit your changes ` to continue. From b170a7321223cb029a4a7d5e5e17c439fc2654ec Mon Sep 17 00:00:00 2001 From: Joris Van den Bossche Date: Wed, 7 Mar 2018 23:52:04 +0100 Subject: [PATCH 6/8] Latest updates from sprint website --- doc/source/contributing_docstring.rst | 203 ++++++++++++++++++-------- 1 file changed, 139 insertions(+), 64 deletions(-) diff --git a/doc/source/contributing_docstring.rst b/doc/source/contributing_docstring.rst index f5701b6df281f..f113a37c2f6d5 100644 --- a/doc/source/contributing_docstring.rst +++ b/doc/source/contributing_docstring.rst @@ -11,7 +11,7 @@ A Python docstring is a string used to document a Python function or method, so programmers can understand what it does without having to read the details of the implementation. -Also, it is a commonn practice to generate online (html) documentation +Also, it is a common practice to generate online (html) documentation automatically from docstrings. `Sphinx `_ serves this purpose. @@ -91,8 +91,8 @@ General rules ~~~~~~~~~~~~~ Docstrings must be defined with three double-quotes. No blank lines should be -left before or after the docstring. The text starts immediately after the -opening quotes (not in the next line). The closing quotes have their own line +left before or after the docstring. The text starts in the next line after the +opening quotes. The closing quotes have their own line (meaning that they are not at the end of the last sentence). **Good:** @@ -100,7 +100,8 @@ opening quotes (not in the next line). The closing quotes have their own line .. code-block:: python def func(): - """Some function. + """ + Some function. With a good docstring. """ @@ -114,19 +115,18 @@ opening quotes (not in the next line). The closing quotes have their own line def func(): - """ - Some function. + """Some function. With several mistakes in the docstring. - + It has a blank like after the signature `def func():`. - - The text 'Some function' should go in the same line as the - opening quotes of the docstring, not in the next line. - + + The text 'Some function' should go in the next line then the + opening quotes of the docstring, not in the same line. + There is a blank line between the docstring and the first line of code `foo = 1`. - + The closing quotes should be in the next line, not in this one.""" foo = 1 @@ -150,7 +150,8 @@ details. .. code-block:: python def astype(dtype): - """Cast Series type. + """ + Cast Series type. This section will provide further details. """ @@ -161,28 +162,32 @@ details. .. code-block:: python def astype(dtype): - """Casts Series type. + """ + Casts Series type. Verb in third-person of the present simple, should be infinitive. """ pass def astype(dtype): - """Method to cast Series type. + """ + Method to cast Series type. Does not start with verb. """ pass def astype(dtype): - """Cast Series type + """ + Cast Series type Missing dot at the end. """ pass def astype(dtype): - """Cast Series type from its current type to the new type defined in + """ + Cast Series type from its current type to the new type defined in the parameter dtype. Summary is too verbose and doesn't fit in a single line. @@ -201,10 +206,14 @@ go in other sections. A blank line is left between the short summary and the extended summary. And every paragraph in the extended summary is finished by a dot. +The extended summary should provide details on why the function is useful and +their use cases, if it is not too generic. + .. code-block:: python def unstack(): - """Pivot a row index to columns. + """ + Pivot a row index to columns. When using a multi-index, a level can be pivoted so each value in the index becomes a column. This is especially useful when a subindex @@ -239,11 +248,19 @@ can have multiple lines. The description must start with a capital letter, and finish with a dot. Keyword arguments with a default value, the default will be listed in brackets -at the end of the description (before the dot). The exact form of the -description in this case would be "Description of the arg (default is X).". In -some cases it may be useful to explain what the default argument means, which -can be added after a comma "Description of the arg (default is -1, which means -all cpus).". +at the end of the type. The exact form of the type in this case would be for +example "int (default is 0)". In some cases it may be useful to explain what +the default argument means, which can be added after a comma "int (default is +-1, which means all cpus)". + +In cases where the default value is `None`, meaning that the value will not be +used, instead of "str (default is None)" it is preferred to use "str, optional". +When `None` is a value being used, we will keep the form "str (default None). +For example consider `.fillna(value=None)`, in which `None` is the value being +used to replace missing values. This is different from +`.to_csv(compression=None)`, where `None` is not a value being used, but means +that compression is optional, and will not be used, unless a compression type +is provided. In this case we will use `str, optional`. **Good:** @@ -251,7 +268,8 @@ all cpus).". class Series: def plot(self, kind, color='blue', **kwargs): - """Generate a plot. + """ + Generate a plot. Render the data in the Series as a matplotlib plot of the specified kind. @@ -260,8 +278,8 @@ all cpus).". ---------- kind : str Kind of matplotlib plot. - color : str - Color name or rgb code (default is 'blue'). + color : str, default 'blue' + Color name or rgb code. **kwargs These parameters will be passed to the matplotlib plotting function. @@ -274,14 +292,15 @@ all cpus).". class Series: def plot(self, kind, **kwargs): - """Generate a plot. + """ + Generate a plot. Render the data in the Series as a matplotlib plot of the specified kind. Note the blank line between the parameters title and the first - parameter. Also, not that after the name of the parameter `kind` - and before the colo, a space is missing. + parameter. Also, note that after the name of the parameter `kind` + and before the colon, a space is missing. Also, note that the parameter descriptions do not start with a capital letter, and do not finish with a dot. @@ -302,26 +321,33 @@ Parameter types ^^^^^^^^^^^^^^^ When specifying the parameter types, Python built-in data types can be used -directly: +directly (the Python type is preferred to the more verbose string, integer, +boolean, etc): - int - float - str - bool -For complex types, define the subtypes: +For complex types, define the subtypes. For `dict` and `tuple`, as more than +one type is present, we use the brackets to help read the type (curly brackets +for `dict` and normal brackets for `tuple`): -- list of [int] +- list of int - dict of {str : int} - tuple of (str, int, int) -- set of {str} +- tuple of (str,) +- set of str -In case there are just a set of values allowed, list them in curly brackets -and separated by commas (followed by a space). If one of them is the default -value of a keyword argument, it should be listed first.: +In case where there are just a set of values allowed, list them in curly +brackets and separated by commas (followed by a space). If the values are +ordinal and they have an order, list them in this order. Otherwuse, list +the default value first, if there is one: - {0, 10, 25} - {'simple', 'advanced'} +- {'low', 'medium', 'high'} +- {'cat', 'dog', 'bird'} If the type is defined in a Python module, the module must be specified: @@ -357,7 +383,7 @@ last two types, that need to be separated by the word 'or': - float, decimal.Decimal or None - str or list of str -If None is one of the accepted values, it always needs to be the last in +If `None` is one of the accepted values, it always needs to be the last in the list. .. _docstring.returns: @@ -384,9 +410,10 @@ For example, with a single value: .. code-block:: python def sample(): - """Generate and return a random number. + """ + Generate and return a random number. - The value is sampled from a continuos uniform distribution between + The value is sampled from a continuous uniform distribution between 0 and 1. Returns @@ -401,7 +428,8 @@ With more than one value: .. code-block:: python def random_letters(): - """Generate and return a sequence of random letters. + """ + Generate and return a sequence of random letters. The length of the returned string is also random, and is also returned. @@ -423,9 +451,10 @@ If the method yields its value: .. code-block:: python def sample_values(): - """Generate an infinite sequence of random numbers. + """ + Generate an infinite sequence of random numbers. - The values are sampled from a continuos uniform distribution between + The values are sampled from a continuous uniform distribution between 0 and 1. Yields @@ -487,7 +516,7 @@ the one referencing. The description must also finish with a dot. Note that in "Returns" and "Yields", the description is located in the following line than the type. But in this section it is located in the same line, with a colon in between. If the description does not fit in the same -line, it can continue in the next ones, but it has to be indenteted in them. +line, it can continue in the next ones, but it has to be indented in them. For example: @@ -507,7 +536,9 @@ For example: See Also -------- - tail : Return the last 5 elements of the Series. + Series.tail : Return the last 5 elements of the Series. + Series.iloc : Return a slice of the elements in the Series, + which can also be used to return the first or last n. """ return self.iloc[:5] @@ -555,6 +586,13 @@ The way to present examples is as follows: 4. Add examples with explanations that illustrate how the parameters can be used for extended functionality +.. note:: + Which data should be used in examples is a topic still under discussion. + We'll likely be importing a standard dataset from `pandas.io.samples`, but + this still needs confirmation. You can work with the data from this pull + request: https://github.com/pandas-dev/pandas/pull/19933/files but + consider this could still change. + A simple example could be: .. code-block:: python @@ -630,15 +668,17 @@ one structure is needed, name them with something meaningful, for example `df_main` and `df_to_join`. Data used in the example should be as compact as possible. The number of rows -is recommended to be 4, unless the example requires a larger number. As for -example in the head method, where it requires to be higher than 5, to show -the example with the default values. - -Avoid using data without interpretation, like a matrix of random numbers -with columns A, B, C, D... And instead use a meaningful example, which makes -it easier to understand the concept. Unless required by the example, use -names of animals, to keep examples consistent. And numerical properties of -them. +is recommended to be around 4, but make it a number that makes sense for the +specific example. For example in the `head` method, it requires to be higher +than 5, to show the example with the default values. If doing the `mean`, +we could use something like `[1, 2, 3]`, so it is easy to see that the +value returned is the mean. + +For more complex examples (groupping for example), avoid using data without +interpretation, like a matrix of random numbers with columns A, B, C, D... +And instead use a meaningful example, which makes it easier to understand the +concept. Unless required by the example, use names of animals, to keep examples +consistent. And numerical properties of them. When calling the method, keywords arguments `head(n=3)` are preferred to positional arguments `head(3)`. @@ -647,23 +687,58 @@ positional arguments `head(3)`. .. code-block:: python - def method(): - """A sample DataFrame method. + class Series: + def mean(self): + """ + Compute the mean of the input. - Examples - -------- - >>> df = pd.DataFrame([389., 24., 80.5, numpy.nan] - ... columns=('max_speed'), - ... index=['falcon', 'parrot', 'lion', 'monkey']) - """ - pass + Examples + -------- + >>> s = pd.Series([1, 2, 3]) + >>> s.mean() + 2 + """ + pass + + + def fillna(self, value): + """ + Replace missing values by `value`. + + Examples + -------- + >>> s = pd.Series([1, np.nan, 3]) + >>> s.fillna(9) + [1, 9, 3] + """ + pass + + def groupby_mean(df): + """ + Group by index and return mean. + + Examples + -------- + >>> s = pd.Series([380., 370., 24., 26], + ... name='max_speed', + ... index=['falcon', 'falcon', 'parrot', 'parrot']) + >>> s.groupby_mean() + falcon 375. + parrot 25. + """ + pass **Bad:** .. code-block:: python def method(): - """A sample DataFrame method. + """ + A sample DataFrame method. + + Do not import numpy and pandas. + + Try to use meaningful data, when it adds value. Examples -------- @@ -672,4 +747,4 @@ positional arguments `head(3)`. >>> df = pd.DataFrame(numpy.random.randn(3, 3), ... columns=('a', 'b', 'c')) """ - pass + pass \ No newline at end of file From 96da2c855124b86a76391e71a02973219660cccc Mon Sep 17 00:00:00 2001 From: Marc Garcia Date: Thu, 8 Mar 2018 01:14:11 +0000 Subject: [PATCH 7/8] Updating contributing_docstring.rst with the last changes, and some fixes --- doc/source/contributing_docstring.rst | 338 +++++++++++++++++++------- 1 file changed, 254 insertions(+), 84 deletions(-) diff --git a/doc/source/contributing_docstring.rst b/doc/source/contributing_docstring.rst index f5701b6df281f..58de31b74b809 100644 --- a/doc/source/contributing_docstring.rst +++ b/doc/source/contributing_docstring.rst @@ -7,11 +7,11 @@ pandas docstring guide About docstrings and standards ------------------------------ -A Python docstring is a string used to document a Python function or method, -so programmers can understand what it does without having to read the details -of the implementation. +A Python docstring is a string used to document a Python module, class, +function or method, so programmers can understand what it does without having +to read the details of the implementation. -Also, it is a commonn practice to generate online (html) documentation +Also, it is a common practice to generate online (html) documentation automatically from docstrings. `Sphinx `_ serves this purpose. @@ -91,22 +91,33 @@ General rules ~~~~~~~~~~~~~ Docstrings must be defined with three double-quotes. No blank lines should be -left before or after the docstring. The text starts immediately after the -opening quotes (not in the next line). The closing quotes have their own line +left before or after the docstring. The text starts in the next line after the +opening quotes. The closing quotes have their own line (meaning that they are not at the end of the last sentence). +In rare occasions reST styles like bold text or itallics will be used in +docstrings, but is it common to have inline code, which is presented between +backticks. It is considered inline code: + +- The name of a parameter +- Python code, a module, function, built-in, type, literal... (e.g. `os`, `list`, `numpy.abs`, `datetime.date`, `True`) +- A pandas class (in the form ``:class:`~pandas.Series```) +- A pandas method (in the form ``:meth:`pandas.Series.sum```) +- A pandas function (in the form ``:func:`pandas.to_datetime```) + **Good:** .. code-block:: python - def func(): - """Some function. + def add_values(arr): + """ + Add the values in `arr`. + + This is equivalent to Python `sum` of :meth:`pandas.Series.sum`. - With a good docstring. + Some sections are omitted here for simplicity. """ - foo = 1 - bar = 2 - return foo + bar + return sum(arr) **Bad:** @@ -114,19 +125,18 @@ opening quotes (not in the next line). The closing quotes have their own line def func(): - """ - Some function. + """Some function. With several mistakes in the docstring. - + It has a blank like after the signature `def func():`. - - The text 'Some function' should go in the same line as the - opening quotes of the docstring, not in the next line. - + + The text 'Some function' should go in the line after the + opening quotes of the docstring, not in the same line. + There is a blank line between the docstring and the first line of code `foo = 1`. - + The closing quotes should be in the next line, not in this one.""" foo = 1 @@ -141,16 +151,18 @@ Section 1: Short summary The short summary is a single sentence that express what the function does in a concise way. -The short summary must start with a verb infinitive, end with a dot, and fit in -a single line. It needs to express what the function does without providing -details. +The short summary must start with a capital letter, end with a dot, and fit in +a single line. It needs to express what the object does without providing +details. For functions and methods, the short summary must start with an +infinitive verb. **Good:** .. code-block:: python def astype(dtype): - """Cast Series type. + """ + Cast Series type. This section will provide further details. """ @@ -161,28 +173,32 @@ details. .. code-block:: python def astype(dtype): - """Casts Series type. + """ + Casts Series type. Verb in third-person of the present simple, should be infinitive. """ pass def astype(dtype): - """Method to cast Series type. + """ + Method to cast Series type. Does not start with verb. """ pass def astype(dtype): - """Cast Series type + """ + Cast Series type Missing dot at the end. """ pass def astype(dtype): - """Cast Series type from its current type to the new type defined in + """ + Cast Series type from its current type to the new type defined in the parameter dtype. Summary is too verbose and doesn't fit in a single line. @@ -201,10 +217,14 @@ go in other sections. A blank line is left between the short summary and the extended summary. And every paragraph in the extended summary is finished by a dot. +The extended summary should provide details on why the function is useful and +their use cases, if it is not too generic. + .. code-block:: python def unstack(): - """Pivot a row index to columns. + """ + Pivot a row index to columns. When using a multi-index, a level can be pivoted so each value in the index becomes a column. This is especially useful when a subindex @@ -238,12 +258,20 @@ required to have a line with the parameter description, which is indented, and can have multiple lines. The description must start with a capital letter, and finish with a dot. -Keyword arguments with a default value, the default will be listed in brackets -at the end of the description (before the dot). The exact form of the -description in this case would be "Description of the arg (default is X).". In -some cases it may be useful to explain what the default argument means, which -can be added after a comma "Description of the arg (default is -1, which means -all cpus).". +For keyword arguments with a default value, the default will be listed after a +comma at the end of the type. The exact form of the type in this case will be +"int, default 0". In some cases it may be useful to explain what the default +argument means, which can be added after a comma "int, default -1, meaning all +cpus". + +In cases where the default value is `None`, meaning that the value will not be +used. Instead of "str, default None" is preferred "str, optional". +When `None` is a value being used, we will keep the form "str, default None". +For example, in `df.to_csv(compression=None)`, `None` is not a value being used, +but means that compression is optional, and no compression is being used if not +provided. In this case we will use `str, optional`. Only in cases like +`func(value=None)` and `None` is being used in the same way as `0` or `foo` +would be used, then we will specify "str, int or None, default None". **Good:** @@ -251,7 +279,8 @@ all cpus).". class Series: def plot(self, kind, color='blue', **kwargs): - """Generate a plot. + """ + Generate a plot. Render the data in the Series as a matplotlib plot of the specified kind. @@ -260,8 +289,8 @@ all cpus).". ---------- kind : str Kind of matplotlib plot. - color : str - Color name or rgb code (default is 'blue'). + color : str, default 'blue' + Color name or rgb code. **kwargs These parameters will be passed to the matplotlib plotting function. @@ -274,14 +303,15 @@ all cpus).". class Series: def plot(self, kind, **kwargs): - """Generate a plot. + """ + Generate a plot. Render the data in the Series as a matplotlib plot of the specified kind. Note the blank line between the parameters title and the first - parameter. Also, not that after the name of the parameter `kind` - and before the colo, a space is missing. + parameter. Also, note that after the name of the parameter `kind` + and before the colon, a space is missing. Also, note that the parameter descriptions do not start with a capital letter, and do not finish with a dot. @@ -302,26 +332,33 @@ Parameter types ^^^^^^^^^^^^^^^ When specifying the parameter types, Python built-in data types can be used -directly: +directly (the Python type is preferred to the more verbose string, integer, +boolean, etc): - int - float - str - bool -For complex types, define the subtypes: +For complex types, define the subtypes. For `dict` and `tuple`, as more than +one type is present, we use the brackets to help read the type (curly brackets +for `dict` and normal brackets for `tuple`): -- list of [int] +- list of int - dict of {str : int} - tuple of (str, int, int) -- set of {str} +- tuple of (str,) +- set of str -In case there are just a set of values allowed, list them in curly brackets -and separated by commas (followed by a space). If one of them is the default -value of a keyword argument, it should be listed first.: +In case where there are just a set of values allowed, list them in curly +brackets and separated by commas (followed by a space). If the values are +ordinal and they have an order, list them in this order. Otherwuse, list +the default value first, if there is one: - {0, 10, 25} - {'simple', 'advanced'} +- {'low', 'medium', 'high'} +- {'cat', 'dog', 'bird'} If the type is defined in a Python module, the module must be specified: @@ -357,7 +394,7 @@ last two types, that need to be separated by the word 'or': - float, decimal.Decimal or None - str or list of str -If None is one of the accepted values, it always needs to be the last in +If `None` is one of the accepted values, it always needs to be the last in the list. .. _docstring.returns: @@ -384,9 +421,10 @@ For example, with a single value: .. code-block:: python def sample(): - """Generate and return a random number. + """ + Generate and return a random number. - The value is sampled from a continuos uniform distribution between + The value is sampled from a continuous uniform distribution between 0 and 1. Returns @@ -401,7 +439,8 @@ With more than one value: .. code-block:: python def random_letters(): - """Generate and return a sequence of random letters. + """ + Generate and return a sequence of random letters. The length of the returned string is also random, and is also returned. @@ -423,9 +462,10 @@ If the method yields its value: .. code-block:: python def sample_values(): - """Generate an infinite sequence of random numbers. + """ + Generate an infinite sequence of random numbers. - The values are sampled from a continuos uniform distribution between + The values are sampled from a continuous uniform distribution between 0 and 1. Yields @@ -441,9 +481,9 @@ If the method yields its value: Section 5: See Also ~~~~~~~~~~~~~~~~~~~ -This is an optional section, used to let users know about pandas functionality -related to the one being documented. While optional, this section should exist -in most cases, unless no related methods or functions can be found at all. +This section is used to let users know about pandas functionality +related to the one being documented. In rare cases, if no related methods +or functions can be found at all, this section can be skipped. An obvious example would be the `head()` and `tail()` methods. As `tail()` does the equivalent as `head()` but at the end of the `Series` or `DataFrame` @@ -487,7 +527,7 @@ the one referencing. The description must also finish with a dot. Note that in "Returns" and "Yields", the description is located in the following line than the type. But in this section it is located in the same line, with a colon in between. If the description does not fit in the same -line, it can continue in the next ones, but it has to be indenteted in them. +line, it can continue in the next ones, but it has to be indented in them. For example: @@ -507,7 +547,9 @@ For example: See Also -------- - tail : Return the last 5 elements of the Series. + Series.tail : Return the last 5 elements of the Series. + Series.iloc : Return a slice of the elements in the Series, + which can also be used to return the first or last n. """ return self.iloc[:5] @@ -602,6 +644,10 @@ A simple example could be: """ return self.iloc[:n] +The examples should be as concise as possible. In cases where the complexity of +the function requires long examples, is recommended to use blocks with headers +in bold. Use double star \*\* to make a text bold, like in \*\*this example\*\*. + .. _docstring.example_conventions: Conventions for the examples @@ -623,22 +669,24 @@ the standard library go first, followed by third-party libraries (like matplotlib). When illustrating examples with a single `Series` use the name `s`, and if -illustrating with a single `DataFrame` use the name `df`. If a set of -homogeneous `Series` or `DataFrame` is used, name them `s1`, `s2`, `s3`... -or `df1`, `df2`, `df3`... If the data is not homogeneous, and more than -one structure is needed, name them with something meaningful, for example -`df_main` and `df_to_join`. +illustrating with a single `DataFrame` use the name `df`. For indices, `idx` +is the preferred name. If a set of homogeneous `Series` or `DataFrame` is used, +name them `s1`, `s2`, `s3`... or `df1`, `df2`, `df3`... If the data is not +homogeneous, and more than one structure is needed, name them with something +meaningful, for example `df_main` and `df_to_join`. Data used in the example should be as compact as possible. The number of rows -is recommended to be 4, unless the example requires a larger number. As for -example in the head method, where it requires to be higher than 5, to show -the example with the default values. - -Avoid using data without interpretation, like a matrix of random numbers -with columns A, B, C, D... And instead use a meaningful example, which makes -it easier to understand the concept. Unless required by the example, use -names of animals, to keep examples consistent. And numerical properties of -them. +is recommended to be around 4, but make it a number that makes sense for the +specific example. For example in the `head` method, it requires to be higher +than 5, to show the example with the default values. If doing the `mean`, +we could use something like `[1, 2, 3]`, so it is easy to see that the +value returned is the mean. + +For more complex examples (groupping for example), avoid using data without +interpretation, like a matrix of random numbers with columns A, B, C, D... +And instead use a meaningful example, which makes it easier to understand the +concept. Unless required by the example, use names of animals, to keep examples +consistent. And numerical properties of them. When calling the method, keywords arguments `head(n=3)` are preferred to positional arguments `head(3)`. @@ -647,23 +695,111 @@ positional arguments `head(3)`. .. code-block:: python - def method(): - """A sample DataFrame method. + class Series: + def mean(self): + """ + Compute the mean of the input. - Examples - -------- - >>> df = pd.DataFrame([389., 24., 80.5, numpy.nan] - ... columns=('max_speed'), - ... index=['falcon', 'parrot', 'lion', 'monkey']) - """ - pass + Examples + -------- + >>> s = pd.Series([1, 2, 3]) + >>> s.mean() + 2 + """ + pass + + + def fillna(self, value): + """ + Replace missing values by `value`. + + Examples + -------- + >>> s = pd.Series([1, np.nan, 3]) + >>> s.fillna(0) + [1, 0, 3] + """ + pass + + def groupby_mean(self): + """ + Group by index and return mean. + + Examples + -------- + >>> s = pd.Series([380., 370., 24., 26], + ... name='max_speed', + ... index=['falcon', 'falcon', 'parrot', 'parrot']) + >>> s.groupby_mean() + index + falcon 375.0 + parrot 25.0 + Name: max_speed, dtype: float64 + """ + pass + + def contains(self, pattern, case_sensitive=True, na=numpy.nan): + """ + Return whether each value contains `pattern`. + + In this case, we are illustrating how to use sections, even + if the example is simple enough and does not require them. + + Examples + -------- + >>> s = pd.Series('Antelope', 'Lion', 'Zebra', numpy.nan) + >>> s.contains(pattern='a') + 0 False + 1 False + 2 True + 3 NaN + dtype: bool + + **Case sensitivity** + + With `case_sensitive` set to `False` we can match `a` with both + `a` and `A`: + + >>> s.contains(pattern='a', case_sensitive=False) + 0 True + 1 False + 2 True + 3 NaN + dtype: bool + + **Missing values** + + We can fill missing values in the output using the `na` parameter: + + >>> s.contains(pattern='a', na=False) + 0 False + 1 False + 2 True + 3 False + dtype: bool + """ + pass **Bad:** .. code-block:: python - def method(): - """A sample DataFrame method. + def method(foo=None, bar=None): + """ + A sample DataFrame method. + + Do not import numpy and pandas. + + Try to use meaningful data, when it makes the example easier + to understand. + + Try to avoid positional arguments like in `df.method(1)`. They + can be all right if previously defined with a meaningful name, + like in `present_value(interest_rate)`, but avoid them otherwise. + + When presenting the behavior with different parameters, do not place + all the calls one next to the other. Instead, add a short sentence + explaining what the example shows. Examples -------- @@ -671,5 +807,39 @@ positional arguments `head(3)`. >>> import pandas as pd >>> df = pd.DataFrame(numpy.random.randn(3, 3), ... columns=('a', 'b', 'c')) + >>> df.method(1) + 21 + >>> df.method(bar=14) + 123 """ pass + + +.. _docstring.example_plots: + +Plots in examples +^^^^^^^^^^^^^^^^^ + +There are some methods in pandas returning plots. To render the plots generated +by the examples in the documentation, the `.. plot::` directive exists. + +To use it, place the next code after the "Examples" header as shown below. The +plot will be generated automatically when building the documentation. + +.. code-block:: python + + class Series: + def plot(self): + """ + Generate a plot with the `Series` data. + + Examples + -------- + + .. plot:: + :context: close-figs + + >>> s = pd.Series([1, 2, 3]) + >>> s.plot() + """ + pass From 5ebac5551c8ffd02c59ee88c252b4424abd3c708 Mon Sep 17 00:00:00 2001 From: Joris Van den Bossche Date: Mon, 12 Mar 2018 18:27:42 +0100 Subject: [PATCH 8/8] synchronize with sprint version --- doc/source/contributing_docstring.rst | 157 ++++++++++++++++++-------- 1 file changed, 111 insertions(+), 46 deletions(-) diff --git a/doc/source/contributing_docstring.rst b/doc/source/contributing_docstring.rst index 58de31b74b809..cd56b76fa891b 100644 --- a/doc/source/contributing_docstring.rst +++ b/doc/source/contributing_docstring.rst @@ -4,6 +4,10 @@ pandas docstring guide ====================== +.. note:: + `Video tutorial: Pandas docstring guide + `_ by Frank Akogun. + About docstrings and standards ------------------------------ @@ -20,7 +24,8 @@ Next example gives an idea on how a docstring looks like: .. code-block:: python def add(num1, num2): - """Add up two integer numbers. + """ + Add up two integer numbers. This function simply wraps the `+` operator, and does not do anything interesting, except for illustrating what is @@ -100,7 +105,8 @@ docstrings, but is it common to have inline code, which is presented between backticks. It is considered inline code: - The name of a parameter -- Python code, a module, function, built-in, type, literal... (e.g. `os`, `list`, `numpy.abs`, `datetime.date`, `True`) +- Python code, a module, function, built-in, type, literal... (e.g. ``os``, + ``list``, ``numpy.abs``, ``datetime.date``, ``True``) - A pandas class (in the form ``:class:`~pandas.Series```) - A pandas method (in the form ``:meth:`pandas.Series.sum```) - A pandas function (in the form ``:func:`pandas.to_datetime```) @@ -148,8 +154,8 @@ backticks. It is considered inline code: Section 1: Short summary ~~~~~~~~~~~~~~~~~~~~~~~~ -The short summary is a single sentence that express what the function does in a -concise way. +The short summary is a single sentence that expresses what the function does in +a concise way. The short summary must start with a capital letter, end with a dot, and fit in a single line. It needs to express what the object does without providing @@ -253,7 +259,7 @@ After the title, each parameter in the signature must be documented, including The parameters are defined by their name, followed by a space, a colon, another space, and the type (or types). Note that the space between the name and the colon is important. Types are not defined for `*args` and `**kwargs`, but must -be defined for all other parameters. After the parameter definition, it is +be defined for all other parameters. After the parameter definition, it is required to have a line with the parameter description, which is indented, and can have multiple lines. The description must start with a capital letter, and finish with a dot. @@ -265,7 +271,7 @@ argument means, which can be added after a comma "int, default -1, meaning all cpus". In cases where the default value is `None`, meaning that the value will not be -used. Instead of "str, default None" is preferred "str, optional". +used. Instead of "str, default None", it is preferred to write "str, optional". When `None` is a value being used, we will keep the form "str, default None". For example, in `df.to_csv(compression=None)`, `None` is not a value being used, but means that compression is optional, and no compression is being used if not @@ -352,7 +358,7 @@ for `dict` and normal brackets for `tuple`): In case where there are just a set of values allowed, list them in curly brackets and separated by commas (followed by a space). If the values are -ordinal and they have an order, list them in this order. Otherwuse, list +ordinal and they have an order, list them in this order. Otherwise, list the default value first, if there is one: - {0, 10, 25} @@ -394,9 +400,13 @@ last two types, that need to be separated by the word 'or': - float, decimal.Decimal or None - str or list of str -If `None` is one of the accepted values, it always needs to be the last in +If ``None`` is one of the accepted values, it always needs to be the last in the list. +For axis, the convention is to use something like: + +- axis : {0 or 'index', 1 or 'columns', None}, default None + .. _docstring.returns: Section 4: Returns or Yields @@ -492,28 +502,28 @@ instead of at the beginning, it is good to let the users know about it. To give an intuition on what can be considered related, here there are some examples: -* `loc` and `iloc`, as they do the same, but in one case providing indices and - in the other positions -* `max` and `min`, as they do the opposite -* `iterrows`, `itertuples` and `iteritems`, as it is easy that a user looking - for the method to iterate over columns ends up in the method to iterate - over rows, and vice-versa -* `fillna` and `dropna`, as both methods are used to handle missing values -* `read_csv` and `to_csv`, as they are complementary -* `merge` and `join`, as one is a generalization of the other -* `astype` and `pandas.to_datetime`, as users may be reading the documentation - of `astype` to know how to cast as a date, and the way to do it is with - `pandas.to_datetime` -* `where` is related to `numpy.where`, as its functionality is based on it +* ``loc`` and ``iloc``, as they do the same, but in one case providing indices + and in the other positions +* ``max`` and ``min``, as they do the opposite +* ``iterrows``, ``itertuples`` and ``iteritems``, as it is easy that a user + looking for the method to iterate over columns ends up in the method to + iterate over rows, and vice-versa +* ``fillna`` and ``dropna``, as both methods are used to handle missing values +* ``read_csv`` and ``to_csv``, as they are complementary +* ``merge`` and ``join``, as one is a generalization of the other +* ``astype`` and ``pandas.to_datetime``, as users may be reading the + documentation of ``astype`` to know how to cast as a date, and the way to do + it is with ``pandas.to_datetime`` +* ``where`` is related to ``numpy.where``, as its functionality is based on it When deciding what is related, you should mainly use your common sense and think about what can be useful for the users reading the documentation, especially the less experienced ones. -When relating to other libraries (mainly `numpy`), use the name of the module -first (not an alias like `np`). If the function is in a module which is not -the main one, like `scipy.sparse`, list the full module (e.g. -`scipy.sparse.coo_matrix`). +When relating to other libraries (mainly ``numpy``), use the name of the module +first (not an alias like ``np``). If the function is in a module which is not +the main one, like ``scipy.sparse``, list the full module (e.g. +``scipy.sparse.coo_matrix``). This section, as the previous, also has a header, "See Also" (note the capital S and A). Also followed by the line with hyphens, and preceded by a blank line. @@ -535,7 +545,8 @@ For example: class Series: def head(self): - """Return the first 5 elements of the Series. + """ + Return the first 5 elements of the Series. This function is mainly useful to preview the values of the Series without displaying the whole of it. @@ -577,8 +588,8 @@ placed in the last position. As often, people understand concepts better with examples, than with accurate explanations. Examples in docstrings, besides illustrating the usage of the function or -method, they must be valid Python code, that in a deterministic way returns -the presented output, and that can be copied and run by users. +method, must be valid Python code, that in a deterministic way returns the +presented output, and that can be copied and run by users. They are presented as a session in the Python terminal. `>>>` is used to present code. `...` is used for code continuing from the previous line. @@ -588,7 +599,7 @@ be added with blank lines before and after them. The way to present examples is as follows: -1. Import required libraries (except `numpy` and `pandas`) +1. Import required libraries (except ``numpy`` and ``pandas``) 2. Create the data required for the example @@ -603,7 +614,8 @@ A simple example could be: class Series: def head(self, n=5): - """Return the first elements of the Series. + """ + Return the first elements of the Series. This function is mainly useful to preview the values of the Series without displaying the whole of it. @@ -646,7 +658,7 @@ A simple example could be: The examples should be as concise as possible. In cases where the complexity of the function requires long examples, is recommended to use blocks with headers -in bold. Use double star \*\* to make a text bold, like in \*\*this example\*\*. +in bold. Use double star ``**`` to make a text bold, like in ``**this example**``. .. _docstring.example_conventions: @@ -668,19 +680,20 @@ and avoiding aliases. Avoid excessive imports, but if needed, imports from the standard library go first, followed by third-party libraries (like matplotlib). -When illustrating examples with a single `Series` use the name `s`, and if -illustrating with a single `DataFrame` use the name `df`. For indices, `idx` -is the preferred name. If a set of homogeneous `Series` or `DataFrame` is used, -name them `s1`, `s2`, `s3`... or `df1`, `df2`, `df3`... If the data is not -homogeneous, and more than one structure is needed, name them with something -meaningful, for example `df_main` and `df_to_join`. +When illustrating examples with a single ``Series`` use the name ``s``, and if +illustrating with a single ``DataFrame`` use the name ``df``. For indices, +``idx`` is the preferred name. If a set of homogeneous ``Series`` or +``DataFrame`` is used, name them ``s1``, ``s2``, ``s3``... or ``df1``, +``df2``, ``df3``... If the data is not homogeneous, and more than one structure +is needed, name them with something meaningful, for example ``df_main`` and +``df_to_join``. Data used in the example should be as compact as possible. The number of rows is recommended to be around 4, but make it a number that makes sense for the -specific example. For example in the `head` method, it requires to be higher -than 5, to show the example with the default values. If doing the `mean`, -we could use something like `[1, 2, 3]`, so it is easy to see that the -value returned is the mean. +specific example. For example in the ``head`` method, it requires to be higher +than 5, to show the example with the default values. If doing the ``mean``, we +could use something like ``[1, 2, 3]``, so it is easy to see that the value +returned is the mean. For more complex examples (groupping for example), avoid using data without interpretation, like a matrix of random numbers with columns A, B, C, D... @@ -688,8 +701,8 @@ And instead use a meaningful example, which makes it easier to understand the concept. Unless required by the example, use names of animals, to keep examples consistent. And numerical properties of them. -When calling the method, keywords arguments `head(n=3)` are preferred to -positional arguments `head(3)`. +When calling the method, keywords arguments ``head(n=3)`` are preferred to +positional arguments ``head(3)``. **Good:** @@ -815,13 +828,65 @@ positional arguments `head(3)`. pass +.. _docstring.doctest_tips: + +Tips for getting your examples pass the doctests +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Getting the examples pass the doctests in the validation script can sometimes +be tricky. Here are some attention points: + +* Import all needed libraries (except for pandas and numpy, those are already + imported as ``import pandas as pd`` and ``import numpy as np``) and define + all variables you use in the example. + +* Try to avoid using random data. However random data might be OK in some + cases, like if the function you are documenting deals with probability + distributions, or if the amount of data needed to make the function result + meaningful is too much, such that creating it manually is very cumbersome. + In those cases, always use a fixed random seed to make the generated examples + predictable. Example:: + + >>> np.random.seed(42) + >>> df = pd.DataFrame({'normal': np.random.normal(100, 5, 20)}) + +* If you have a code snippet that wraps multiple lines, you need to use '...' + on the continued lines: :: + + >>> df = pd.DataFrame([[1, 2, 3], [4, 5, 6]], index=['a', 'b', 'c'], + ... columns=['A', 'B']) + +* If you want to show a case where an exception is raised, you can do:: + + >>> pd.to_datetime(["712-01-01"]) + Traceback (most recent call last): + OutOfBoundsDatetime: Out of bounds nanosecond timestamp: 712-01-01 00:00:00 + + It is essential to include the "Traceback (most recent call last):", but for + the actual error only the error name is sufficient. + +* If there is a small part of the result that can vary (e.g. a hash in an object + represenation), you can use ``...`` to represent this part. + + If you want to show that ``s.plot()`` returns a matplotlib AxesSubplot object, + this will fail the doctest :: + + >>> s.plot() + + + However, you can do (notice the comment that needs to be added) :: + + >>> s.plot() # doctest: +ELLIPSIS + + + .. _docstring.example_plots: Plots in examples ^^^^^^^^^^^^^^^^^ There are some methods in pandas returning plots. To render the plots generated -by the examples in the documentation, the `.. plot::` directive exists. +by the examples in the documentation, the ``.. plot::`` directive exists. To use it, place the next code after the "Examples" header as shown below. The plot will be generated automatically when building the documentation. @@ -839,7 +904,7 @@ plot will be generated automatically when building the documentation. .. plot:: :context: close-figs - >>> s = pd.Series([1, 2, 3]) - >>> s.plot() + >>> s = pd.Series([1, 2, 3]) + >>> s.plot() """ pass