Troubleshooting

After this page, you will know how to diagnose the most common bolt-pipeliner==0.2.6 failures. You will also know the fastest fix for each one.

ImportError: cannot import name 'AnalyzeArgument' from 'pyspark.sql.udtf'

This error means your Spark stack is out of sync with what Bolt expects.

In practice, this usually happens when your environment has an older pyspark installed.

Fix:

1. Reinstall Bolt with Spark extras in a clean virtual environment.
2. Verify the active pyspark version from that environment.

python -m venv .venv
source .venv/bin/activate
pip install --upgrade pip
pip install "bolt-pipeliner[spark]==0.2.6"
python -c "import pyspark; print(pyspark.__version__)"

If you use a managed Spark platform, align local and remote Spark related package versions before running bolt run.

No jobs found for layer 'X', skipping...

You will see this warning during generation commands when a layer exists in layers: but has no valid jobs.

Common causes:

• The layer is declared under layers: but its top level block is missing.
• The layer block exists, but it is not a YAML list.
• The layer block uses a placeholder like silver: ..., which loads as an empty list.

Fix the config so each layer has a real list of job dictionaries:

layers:
  silver: etl/1_silver
 
silver:
  - module: customer_features
    input_tables:
      base: bronze_customers
    output_table_name: customer_features

Documentation pages show empty schema columns

bolt generate documentation can still render HTML when Spark is unavailable. In that fallback path, table columns are read from outputs/schema/schema.csv.

If schema.csv is missing, table pages render with empty schema sections.

Fix:

1. Run bolt generate documentation once to emit outputs/schema/schema.py.
2. Run outputs/schema/schema.py in a Spark environment.
3. Save the printed CSV as outputs/schema/schema.csv.
4. Run bolt generate documentation again.

The CSV must include: table_name, col_name, data_type, comment.

You added a new layer and cannot run only that layer

bolt run has fixed shorthand flags for built in layers (--flatfile, --bronze, --silver, --gold, --diamond). A custom layer name does not get a new flag automatically.

Use --layer for custom layer names:

bolt run --layer mylayer

If that still skips your layer, check that:

• mylayer exists under layers: in configs/etl_config.yaml
• A top level mylayer: list exists with at least one job
• Each job includes module, input_tables, and output_table_name