Coding doc structure

This document describes the standard procedure for adding documentation to Python modules in qlat-utils, qlat, qlat-scripts-v1, and auto-contractor.

Overview

Each Python module (.py, .pyx, or .py.in/.pyx.in template) should have:

1. A module docstring in the source file that references the documentation location.

2. A markdown documentation file in the appropriate docs/ subdirectory:

Package	Source	Docs Directory
qlat-utils	qlat-utils/qlat_utils/	docs/qlat-utils/
qlat	qlat/qlat/	docs/qlat/
qlat-scripts-v1	qlat/qlat_scripts/v1/	docs/qlat-scripts-v1/
auto-contractor	qlat/auto_contractor/	docs/auto-contractor/

Procedure

Step 1: Identify the Module

Locate the source file to document. For example:

• qlat-utils/qlat_utils/json.py

• qlat/qlat/field.pyx

• qlat/qlat/field_types.pyx.in (template that generates .pyx)

• qlat/qlat/c.py.in (template that generates .py)

• qlat/qlat_gpt.py

• qlat/qlat_scripts/v1/gen_data.py

• qlat/auto_contractor/operators.py

Note: qlat/qlat/*.py.in and qlat/qlat/*.pyx.in are template files that generate the actual .py/.pyx modules. Document the generated module, not the template itself.

Step 2: Add a Module Docstring to the Source File

Add a module docstring at the top of the source file (after any Cython directives or shebang lines). The docstring must include:

1. The module name as a heading.

2. A one-line description of the module.

3. The path to the documentation file.

4. A note to update the documentation when updating the source.

Template for .py files:

"""
Module ``package.module_name``
==============================

Short description of what this module provides.

Documentation: ``docs/package/module_name.md``

.. note:: Update the documentation when updating this source file.
"""

Template for .pyx files (after the cython directive line):

# cython: binding=True, embedsignature=True, c_string_type=unicode, c_string_encoding=utf8

"""
Module ``package.module_name``
==============================

Short description of what this module provides.

Documentation: ``docs/package/module_name.md``

.. note:: Update the documentation when updating this source file.
"""

Template for .py.in files (after the Mako {{py: ...}} block):

{{py:
# Mako template definitions ...
}}

"""
Module ``package.module_name``
==============================

Short description of what this module provides.

Documentation: ``docs/package/module_name.md``

.. note:: Update the documentation when updating this source file.
"""

Template for .pyx.in files (after the cython directive and Mako block):

# cython: binding=True, embedsignature=True, c_string_type=unicode, c_string_encoding=utf8

{{py:
# Mako template definitions ...
}}

"""
Module ``package.module_name``
==============================

Short description of what this module provides.

Documentation: ``docs/package/module_name.md``

.. note:: Update the documentation when updating this source file.
"""

If the module has existing attribution comments (e.g., license, original author), move them into the docstring as a reference link at the bottom.

Step 3: Create or Update the Documentation File

The documentation file lives in the appropriate docs/ subdirectory and is named after the module. Examples:

• docs/qlat-utils/qlat_json.md for qlat_utils/json.py

• docs/qlat/qlat_field.md for qlat/field.pyx

• docs/qlat_gpt.md for qlat/qlat_gpt.py (top-level package module)

• docs/qlat-scripts-v1/qlat_scripts_gen_data.md for qlat_scripts/v1/gen_data.py

• docs/auto-contractor/auto_contractor_operators.md for auto_contractor/operators.py

Top-level package modules (files directly under qlat/ such as qlat_gpt.py) place their documentation at the docs/ root (e.g., docs/qlat_gpt.md) rather than under a package subdirectory. Their toctree entry goes in docs/index.rst rather than docs/qlat.rst.

The document must begin with:

# `package.module_name` — Short Description

Source: `package-path/module_name.py`

> **Note:** Update this document when updating the source file.

Then include:

1. Outline — A table of contents with anchor links.

2. Overview — What the module provides, key properties, usage summary.

3. Detailed sections — One section per major class, function group, or concept.

4. Examples — Practical usage examples at the end.

Important: Code examples in qlat documentation must call q.begin_with_mpi(size_node_list) after imports to initialize the MPI environment before using any qlat functionality, and q.end_with_mpi() at the end to finalize properly. The Geometry constructor requires a Coordinate object (e.g., q.Geometry(q.Coordinate([4, 4, 4, 8]))). This is not required for qlat-utils modules that do not depend on MPI.

Code that is intended to run on a single process (e.g., file I/O, printing, or computations that do not involve MPI collectives) should be wrapped in if q.get_id_node() == 0: to avoid duplicate output or conflicting writes when running under MPI with multiple processes.

For auto-contractor modules that use GPT, code examples must call qg.begin_with_gpt() after imports and qg.end_with_gpt() at the end.

Step 4: Add a Link in the Package Index

Add a toctree entry for the new documentation file in the appropriate package index:

• docs/qlat-utils.rst for qlat-utils modules

• docs/qlat.rst for qlat modules (files under qlat/qlat/)

• docs/index.rst for top-level qlat package modules (e.g., qlat/qlat_gpt.py)

• docs/qlat-scripts-v1.rst for qlat-scripts-v1 modules

• docs/auto-contractor.rst for auto-contractor modules

Add the entry in the toctree directive near the top of the file:

.. toctree::
   :maxdepth: 1

   qlat-utils/qlat_rng_state.md

   qlat-utils/qlat_json.md

   qlat-utils/qlat_new_module.md

For qlat-scripts-v1 or auto-contractor, the pattern is the same but with the corresponding directory name:

.. toctree::
   :maxdepth: 1

   qlat-scripts-v1/qlat_scripts_gen_data.md

   auto-contractor/auto_contractor_operators.md

Step 5: Verify Examples

After adding or updating documentation that includes code examples, verify that the examples actually work against the compiled module.

Procedure:

1. Build the environment with nix:

   cd nixpkgs && name='' ./install-py-local-kernel-with-nix.sh
   

2. Source the environment:

   source result-py-local/bin/setenv-qlat.sh
   

3. Write a verification script in the appropriate examples directory.

   • For modules that do not require GPT/Grid, write to examples-py/: examples-py/docs-<module_name>.py

   • For modules that require GPT/Grid, write to examples-py-gpt/: examples-py-gpt/docs-<module_name>.py

   The script should exercise all the code examples from the documentation, using assert statements to confirm correct behavior. The script should verify:

   • Round-trip fidelity for each supported type

   • Type preservation through nested structures

   • Edge cases (compact output, plain types, etc.)

4. Run the verification script using the standard test runner:

   # For non-GPT scripts:
   ./nixpkgs/run-one-example-py.py docs-<module_name>
   
   # For GPT scripts:
   ./nixpkgs/run-one-example-py-gpt.py docs-<module_name>
   

Example — verifying qlat_utils.json:

# write examples-py/docs-qlat_utils_json.py
./nixpkgs/run-one-example-py.py docs-qlat_utils_json

The verification script should print “CHECK: finished successfully.” at the end (this is the standard test completion marker used by the test runner). If any assertion fails, fix the documentation or the source code, rebuild, and re-run until all examples pass.

5. Add the new test to the tests (or tests_gpt) list in the corresponding Makefile so it is included in the full test suite:

   • examples-py/Makefile — add docs-<module_name>.log \ to the tests list.

   • examples-py-gpt/Makefile — add docs-<module_name>.log \ to the tests_gpt list.

Step 6: Final Checks

Check that:

• The docstring renders correctly (e.g., help(module) or IDE hover).

• The markdown file is readable and all internal links work.

• The source path in the doc matches the actual file location.

Naming Conventions

Source File	Documentation File
qlat_utils/json.py	docs/qlat-utils/qlat_json.md
qlat_utils/rng_state.pyx	docs/qlat-utils/qlat_rng_state.md
qlat/field.pyx	docs/qlat/qlat_field.md
qlat/field_types.pyx.in	docs/qlat/qlat_field_types.md
qlat/selected_field_types.pyx.in	docs/qlat/qlat_selected_field_types.md
qlat/selected_points_types.pyx.in	docs/qlat/qlat_selected_points_types.md
qlat/c.py.in	docs/qlat/qlat_c.md
qlat/qlat_gpt.py	docs/qlat_gpt.md
qlat_scripts/v1/gen_data.py	docs/qlat-scripts-v1/qlat_scripts_gen_data.md
qlat_scripts/v1/rbc_ukqcd.py	docs/qlat-scripts-v1/qlat_scripts_rbc_ukqcd.md
auto_contractor/operators.py	docs/auto-contractor/auto_contractor_operators.md
auto_contractor/auto_contract_compilation.py	docs/auto-contractor/auto_contractor_auto_contract_compilation.md

Template files (.py.in, .pyx.in) use Mako templating to generate the actual .py/.pyx modules. Document the template file as the source (e.g., Source: qlat/qlat/field_types.pyx.in) since that is what developers edit. The existing docs for field_types, selected_field_types, and selected_points_types already follow this convention.

The documentation filename uses the package prefix (e.g., qlat_) to avoid collisions between packages. For qlat-scripts-v1 modules, the source lives in qlat_scripts/v1/ and documentation uses the qlat_scripts_ prefix. For auto-contractor modules, the documentation uses the auto_contractor_ prefix.